2024-05-27T09:03:12.316 INFO:root:teuthology version: 0.0.1.dev291+ge7ae758 2024-05-27T09:03:12.316 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --verbose --archive /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727678 --name vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi --owner scheduled_vshankar@teuthology -- /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727678/orig.config.yaml 2024-05-27T09:03:12.370 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-27T09:03:12.469 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727678 branch: wip-vshankar-testing-20240521.105740-debug description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: vshankar@redhat.com first_in_suite: false job_id: '7727678' kernel: flavor: default kdb: 1 sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/v18.2.0 ' name: vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240521.105740-debug ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: a6507388dd5057528934822c0163b0c347ef1d5d ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a6507388dd5057528934822c0163b0c347ef1d5d kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240521.105740-debug sha1: a6507388dd5057528934822c0163b0c347ef1d5d owner: scheduled_vshankar@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 2274 sha1: a6507388dd5057528934822c0163b0c347ef1d5d sleep_before_teardown: 0 subset: 442/1024 suite: fs suite_branch: wip-vshankar-testing-20240521.105740-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a6507388dd5057528934822c0163b0c347ef1d5d targets: smithi119.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO/IVuevYU4Oz/3yGM8Mxcz7AVixivuU/uF46kshc8Zqt6wx2Yoyc+g47z3fQ9pNDUMJ1TTwsmSNs1GhZAkz1m0= smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBvtcz+9+vSuL1aQtTBRfczx6qnyvpXgHl5C/mL027EOmgFQk1FaM2yEhNuTOpPL/cokABunHku65GEgf03sUx4= tasks: - install: exclude_packages: - ceph-volume tag: v18.2.0 - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.io/ceph/ceph:v18.2.0 roleless: true - print: '**** done end installing v18.2.0 cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - kclient: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: e7ae7580c417d6ab27fda98fecadca5d9f85d63e timestamp: 2024-05-27_06:46:53 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-05-27T09:03:12.469 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa; will attempt to use it 2024-05-27T09:03:12.470 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a6507388dd5057528934822c0163b0c347ef1d5d/qa/tasks 2024-05-27T09:03:12.470 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-27T09:03:12.471 INFO:teuthology.task.internal:Checking packages... 2024-05-27T09:03:12.492 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a6507388dd5057528934822c0163b0c347ef1d5d' 2024-05-27T09:03:12.492 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-27T09:03:12.492 INFO:teuthology.packaging:ref: None 2024-05-27T09:03:12.492 INFO:teuthology.packaging:tag: None 2024-05-27T09:03:12.492 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240521.105740-debug 2024-05-27T09:03:12.492 INFO:teuthology.packaging:sha1: a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:03:12.492 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240521.105740-debug 2024-05-27T09:03:12.720 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3868.ga6507388 2024-05-27T09:03:12.722 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-27T09:03:12.728 INFO:teuthology.task.internal:no buildpackages task found 2024-05-27T09:03:12.728 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-27T09:03:12.759 INFO:teuthology.task.internal:Saving configuration 2024-05-27T09:03:12.780 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-27T09:03:12.787 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-27T09:03:12.812 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727678', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 08:58:18.482338', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO/IVuevYU4Oz/3yGM8Mxcz7AVixivuU/uF46kshc8Zqt6wx2Yoyc+g47z3fQ9pNDUMJ1TTwsmSNs1GhZAkz1m0='} 2024-05-27T09:03:12.837 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727678', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 08:58:18.480328', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBvtcz+9+vSuL1aQtTBRfczx6qnyvpXgHl5C/mL027EOmgFQk1FaM2yEhNuTOpPL/cokABunHku65GEgf03sUx4='} 2024-05-27T09:03:12.837 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-27T09:03:12.844 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-05-27T09:03:12.844 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-05-27T09:03:12.844 INFO:teuthology.run_tasks:Running task console_log... 2024-05-27T09:03:12.911 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f13344a3ac0>, signals=[15]) 2024-05-27T09:03:12.911 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-27T09:03:12.917 INFO:teuthology.task.internal:Opening connections... 2024-05-27T09:03:12.918 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi119.front.sepia.ceph.com 2024-05-27T09:03:12.919 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:03:12.994 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-05-27T09:03:12.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:03:13.071 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-27T09:03:13.077 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2024-05-27T09:03:13.095 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2024-05-27T09:03:13.095 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2024-05-27T09:03:13.153 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="CentOS Stream" 2024-05-27T09:03:13.154 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="9" 2024-05-27T09:03:13.154 INFO:teuthology.orchestra.run.smithi119.stdout:ID="centos" 2024-05-27T09:03:13.154 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="rhel fedora" 2024-05-27T09:03:13.154 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="9" 2024-05-27T09:03:13.154 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el9" 2024-05-27T09:03:13.154 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T09:03:13.154 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2024-05-27T09:03:13.154 INFO:teuthology.orchestra.run.smithi119.stdout:LOGO="fedora-logo-icon" 2024-05-27T09:03:13.154 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T09:03:13.154 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://centos.org/" 2024-05-27T09:03:13.155 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T09:03:13.155 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T09:03:13.155 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T09:03:13.155 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2024-05-27T09:03:13.179 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-05-27T09:03:13.197 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-05-27T09:03:13.198 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-05-27T09:03:13.255 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-05-27T09:03:13.255 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-05-27T09:03:13.255 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-05-27T09:03:13.256 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-05-27T09:03:13.256 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-05-27T09:03:13.256 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-05-27T09:03:13.256 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T09:03:13.256 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-05-27T09:03:13.256 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-05-27T09:03:13.256 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T09:03:13.256 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-05-27T09:03:13.256 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T09:03:13.256 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T09:03:13.256 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T09:03:13.257 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-05-27T09:03:13.460 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-27T09:03:13.470 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-27T09:03:13.475 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-27T09:03:13.476 DEBUG:teuthology.orchestra.run.smithi119:> test '!' -e /home/ubuntu/cephtest 2024-05-27T09:03:13.478 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-05-27T09:03:13.496 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-27T09:03:13.502 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-27T09:03:13.502 DEBUG:teuthology.orchestra.run.smithi119:> test -z $(ls -A /var/lib/ceph) 2024-05-27T09:03:13.536 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-05-27T09:03:13.572 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-27T09:03:13.625 INFO:teuthology.run_tasks:Running task kernel... 2024-05-27T09:03:13.639 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'} 2024-05-27T09:03:13.639 INFO:teuthology.task.kernel:config {'host.a': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}, 'host.b': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}}, timeout 300 2024-05-27T09:03:13.640 DEBUG:teuthology.orchestra.run.smithi119:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T09:03:13.640 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T09:03:13.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:03:13.657 DEBUG:teuthology.task.kernel:sha1 for host.b is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:03:13.657 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-27T09:03:13.657 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-05-27T09:03:13.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:03:13.658 DEBUG:teuthology.task.kernel:sha1 for host.a is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:03:13.658 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-27T09:03:13.659 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2024-05-27T09:03:13.713 INFO:teuthology.orchestra.run.smithi138.stdout:5.14.0-437.el9.x86_64 2024-05-27T09:03:13.714 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:03:13.714 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-27T09:03:13.714 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:03:13.718 INFO:teuthology.orchestra.run.smithi119.stdout:5.14.0-437.el9.x86_64 2024-05-27T09:03:13.718 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:03:13.718 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-27T09:03:13.718 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:03:13.864 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum upgrade -y linux-firmware 2024-05-27T09:03:13.865 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum upgrade -y linux-firmware 2024-05-27T09:03:16.266 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.1 MB 00:00 2024-05-27T09:03:17.491 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.1 MB 00:02 2024-05-27T09:03:18.026 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-05-27T09:03:19.808 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-05-27T09:03:22.251 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-05-27T09:03:24.001 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-05-27T09:03:24.727 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 17 kB 00:01 2024-05-27T09:03:25.618 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 17 kB 00:00 2024-05-27T09:03:26.444 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 30 MB/s | 21 MB 00:00 2024-05-27T09:03:31.703 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-27T09:03:33.714 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:03:33.715 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-05-27T09:03:33.715 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:03:33.819 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on host.a... 2024-05-27T09:03:33.819 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:03:33.958 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-27T09:03:33.959 DEBUG:teuthology.orchestra.run.smithi119:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-27T09:03:33.961 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi119.front.sepia.ceph.com to complete... 2024-05-27T09:03:44.512 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-05-27T09:03:49.851 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 7.3 kB/s | 1.7 kB 00:00 2024-05-27T09:03:51.821 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:03:51.821 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-05-27T09:03:51.822 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:03:51.935 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on host.b... 2024-05-27T09:03:51.935 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T09:03:52.199 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-27T09:03:52.199 DEBUG:teuthology.orchestra.run.smithi138:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-27T09:03:52.201 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi138.front.sepia.ceph.com to complete... 2024-05-27T09:04:04.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27 09:04:04 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-27T09:04:04.303 INFO:teuthology.orchestra.run.smithi119.stderr:FINISHED --2024-05-27 09:04:04-- 2024-05-27T09:04:04.303 INFO:teuthology.orchestra.run.smithi119.stderr:Total wall clock time: 30s 2024-05-27T09:04:04.303 INFO:teuthology.orchestra.run.smithi119.stderr:Downloaded: 1 files, 340M in 30s (11.5 MB/s) 2024-05-27T09:04:04.306 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on host.a... 2024-05-27T09:04:04.306 DEBUG:teuthology.orchestra.run.smithi119:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-27T09:04:06.408 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying... ######################################## 2024-05-27T09:04:07.320 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing... ######################################## 2024-05-27T09:04:07.439 INFO:teuthology.orchestra.run.smithi119.stdout:Updating / installing... 2024-05-27T09:04:09.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27 09:04:09 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-27T09:04:09.460 INFO:teuthology.orchestra.run.smithi138.stderr:FINISHED --2024-05-27 09:04:09-- 2024-05-27T09:04:09.460 INFO:teuthology.orchestra.run.smithi138.stderr:Total wall clock time: 17s 2024-05-27T09:04:09.460 INFO:teuthology.orchestra.run.smithi138.stderr:Downloaded: 1 files, 340M in 17s (20.1 MB/s) 2024-05-27T09:04:09.463 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on host.b... 2024-05-27T09:04:09.463 DEBUG:teuthology.orchestra.run.smithi138:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-27T09:04:11.605 INFO:teuthology.orchestra.run.smithi138.stdout:Verifying... ######################################## 2024-05-27T09:04:11.889 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing... ######################################## 2024-05-27T09:04:12.004 INFO:teuthology.orchestra.run.smithi138.stdout:Updating / installing... 2024-05-27T09:04:15.398 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-27T09:04:20.008 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-27T09:04:52.476 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi119.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-27T09:04:52.476 DEBUG:teuthology.orchestra.run.smithi119:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-27T09:04:52.560 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-27T09:04:52.560 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-27T09:04:52.560 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-27T09:04:52.560 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-27T09:04:52.560 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-27T09:04:52.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-27T09:04:52.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-27T09:04:52.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-27T09:04:52.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-27T09:04:52.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-27T09:04:52.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-27T09:04:52.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-27T09:04:52.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-27T09:04:52.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-27T09:04:52.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-27T09:04:52.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-27T09:04:52.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-27T09:04:52.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-27T09:04:52.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-27T09:04:52.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-27T09:04:52.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-27T09:04:52.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-27T09:04:52.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-27T09:04:52.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-27T09:04:52.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-27T09:04:52.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-27T09:04:52.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-27T09:04:52.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-27T09:04:52.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-27T09:04:52.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-27T09:04:52.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-27T09:04:52.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-27T09:04:52.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-27T09:04:52.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-27T09:04:52.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-27T09:04:52.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-27T09:04:52.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-27T09:04:52.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-27T09:04:52.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-27T09:04:52.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-27T09:04:52.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-27T09:04:52.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-27T09:04:52.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-27T09:04:52.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-27T09:04:52.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-27T09:04:52.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-27T09:04:52.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-27T09:04:52.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-27T09:04:52.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-27T09:04:52.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-27T09:04:52.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-27T09:04:52.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-27T09:04:52.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-27T09:04:52.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-27T09:04:52.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-27T09:04:52.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-27T09:04:52.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-27T09:04:52.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-27T09:04:52.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-27T09:04:52.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-27T09:04:52.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-27T09:04:52.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-27T09:04:52.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-27T09:04:52.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-27T09:04:52.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-27T09:04:52.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-27T09:04:52.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-27T09:04:52.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-27T09:04:52.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-27T09:04:52.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-27T09:04:52.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-27T09:04:52.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-27T09:04:52.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-27T09:04:52.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-27T09:04:52.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-27T09:04:52.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-27T09:04:52.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-27T09:04:52.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-27T09:04:52.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-27T09:04:52.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-27T09:04:52.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-27T09:04:52.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-27T09:04:52.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-27T09:04:52.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-27T09:04:52.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-27T09:04:52.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-27T09:04:52.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-27T09:04:52.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-27T09:04:52.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-27T09:04:52.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-27T09:04:52.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-27T09:04:52.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-27T09:04:52.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-27T09:04:52.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-27T09:04:52.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-27T09:04:52.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-27T09:04:52.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-27T09:04:52.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-27T09:04:52.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-27T09:04:52.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-27T09:04:52.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-27T09:04:52.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-27T09:04:52.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-27T09:04:52.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-27T09:04:52.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-27T09:04:52.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-27T09:04:52.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-27T09:04:52.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-27T09:04:52.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-27T09:04:52.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-27T09:04:52.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-27T09:04:52.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-27T09:04:52.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-27T09:04:52.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-27T09:04:52.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-27T09:04:52.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-27T09:04:52.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-27T09:04:52.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-27T09:04:52.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-27T09:04:52.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-27T09:04:52.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-27T09:04:52.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-27T09:04:52.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-27T09:04:52.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-27T09:04:52.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-27T09:04:52.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-27T09:04:52.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-27T09:04:52.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-27T09:04:52.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-27T09:04:52.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-27T09:04:52.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-27T09:04:52.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-27T09:04:52.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-27T09:04:52.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-27T09:04:52.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-27T09:04:52.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-27T09:04:52.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-27T09:04:52.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-27T09:04:52.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-27T09:04:52.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-27T09:04:52.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-27T09:04:52.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-27T09:04:52.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-27T09:04:52.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-27T09:04:52.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-27T09:04:52.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-27T09:04:52.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-27T09:04:52.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-27T09:04:52.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-27T09:04:52.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-27T09:04:52.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-27T09:04:52.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-27T09:04:52.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-27T09:04:52.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-27T09:04:52.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-27T09:04:52.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-27T09:04:52.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-27T09:04:52.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-27T09:04:52.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-27T09:04:52.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-27T09:04:52.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-27T09:04:52.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-27T09:04:52.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-27T09:04:52.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-27T09:04:52.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-27T09:04:52.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-27T09:04:52.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-27T09:04:52.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-27T09:04:52.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-27T09:04:52.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-27T09:04:52.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-27T09:04:52.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-27T09:04:52.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-27T09:04:52.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-27T09:04:52.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-27T09:04:52.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-27T09:04:52.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-27T09:04:52.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-27T09:04:52.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-27T09:04:52.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-27T09:04:52.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-27T09:04:52.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-27T09:04:52.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-27T09:04:52.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-27T09:04:52.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-27T09:04:52.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-27T09:04:52.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-27T09:04:52.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-27T09:04:52.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-27T09:04:52.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-27T09:04:52.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-27T09:04:52.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-27T09:04:52.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-27T09:04:52.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-27T09:04:52.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-27T09:04:52.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-27T09:04:52.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-27T09:04:52.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-27T09:04:52.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-27T09:04:52.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-27T09:04:52.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-27T09:04:52.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-27T09:04:52.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-27T09:04:52.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-27T09:04:52.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-27T09:04:52.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-27T09:04:52.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-27T09:04:52.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-27T09:04:52.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-27T09:04:52.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-27T09:04:52.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-27T09:04:52.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-27T09:04:52.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-27T09:04:52.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-27T09:04:52.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-27T09:04:52.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-27T09:04:52.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-27T09:04:52.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-27T09:04:52.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-27T09:04:52.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-27T09:04:52.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-27T09:04:52.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-27T09:04:52.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-27T09:04:52.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-27T09:04:52.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-27T09:04:52.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-27T09:04:52.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-27T09:04:52.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-27T09:04:52.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-27T09:04:52.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-27T09:04:52.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-27T09:04:52.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-27T09:04:52.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-27T09:04:52.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-27T09:04:52.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-27T09:04:52.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-27T09:04:52.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-27T09:04:52.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-27T09:04:52.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-27T09:04:52.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-27T09:04:52.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-27T09:04:52.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-27T09:04:52.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-27T09:04:52.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-27T09:04:52.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-27T09:04:52.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-27T09:04:52.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-27T09:04:52.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-27T09:04:52.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-27T09:04:52.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-27T09:04:52.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-27T09:04:52.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-27T09:04:52.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-27T09:04:52.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-27T09:04:52.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-27T09:04:52.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-27T09:04:52.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-27T09:04:52.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-27T09:04:52.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-27T09:04:52.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-27T09:04:52.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-27T09:04:52.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-27T09:04:52.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-27T09:04:52.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-27T09:04:52.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-27T09:04:52.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-27T09:04:52.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-27T09:04:52.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-27T09:04:52.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-27T09:04:52.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-27T09:04:52.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-27T09:04:52.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-27T09:04:52.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-27T09:04:52.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-27T09:04:52.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-27T09:04:52.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-27T09:04:52.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-27T09:04:52.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-27T09:04:52.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-27T09:04:52.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-27T09:04:52.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-27T09:04:52.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-27T09:04:52.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-27T09:04:52.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-27T09:04:52.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-27T09:04:52.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-27T09:04:52.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-27T09:04:52.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-27T09:04:52.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-27T09:04:52.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-27T09:04:52.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-27T09:04:52.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-27T09:04:52.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-27T09:04:52.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-27T09:04:52.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-27T09:04:52.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-27T09:04:52.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-27T09:04:52.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-27T09:04:52.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-27T09:04:52.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-27T09:04:52.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-27T09:04:52.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-27T09:04:52.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-27T09:04:52.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-27T09:04:52.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-27T09:04:52.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-27T09:04:52.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-27T09:04:52.584 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-27T09:04:52.584 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-27T09:04:52.584 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-27T09:04:52.584 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-27T09:04:52.584 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-27T09:04:52.584 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-27T09:04:52.584 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-27T09:04:52.584 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-27T09:04:52.584 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-27T09:04:52.584 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-27T09:04:52.584 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-27T09:04:52.584 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-27T09:04:52.584 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-27T09:04:52.584 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-27T09:04:52.585 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-27T09:04:52.585 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-27T09:04:52.585 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-27T09:04:52.585 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-27T09:04:52.585 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-27T09:04:52.585 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-27T09:04:52.585 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-27T09:04:52.585 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-27T09:04:52.585 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-27T09:04:52.585 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-27T09:04:52.585 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-27T09:04:52.585 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-27T09:04:52.585 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-27T09:04:52.585 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-27T09:04:52.586 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-27T09:04:52.586 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-27T09:04:52.586 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-27T09:04:52.586 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-27T09:04:52.586 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-27T09:04:52.586 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-27T09:04:52.586 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-27T09:04:52.586 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-27T09:04:52.586 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-27T09:04:52.586 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-27T09:04:52.586 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-27T09:04:52.586 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-27T09:04:52.586 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-27T09:04:52.587 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-27T09:04:52.587 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-27T09:04:52.587 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-27T09:04:52.587 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-27T09:04:52.587 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-27T09:04:52.587 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-27T09:04:52.587 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-27T09:04:52.587 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-27T09:04:52.587 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-27T09:04:52.587 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-27T09:04:52.587 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-27T09:04:52.587 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-27T09:04:52.587 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-27T09:04:52.588 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-27T09:04:52.588 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-27T09:04:52.588 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-27T09:04:52.588 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-27T09:04:52.588 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-27T09:04:52.588 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-27T09:04:52.588 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-27T09:04:52.588 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-27T09:04:52.588 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-27T09:04:52.588 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-27T09:04:52.588 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-27T09:04:52.588 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-27T09:04:52.588 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-27T09:04:52.588 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-27T09:04:52.589 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-27T09:04:52.589 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-27T09:04:52.589 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-27T09:04:52.589 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-27T09:04:52.589 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-27T09:04:52.589 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-27T09:04:52.589 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-27T09:04:52.589 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-27T09:04:52.589 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-27T09:04:52.589 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-27T09:04:52.589 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-27T09:04:52.589 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-27T09:04:52.589 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-27T09:04:52.590 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-27T09:04:52.590 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-27T09:04:52.590 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-27T09:04:52.590 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-27T09:04:52.590 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-27T09:04:52.590 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-27T09:04:52.590 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-27T09:04:52.590 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-27T09:04:52.590 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-27T09:04:52.590 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-27T09:04:52.590 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-27T09:04:52.590 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-27T09:04:52.590 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-27T09:04:52.591 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-27T09:04:52.591 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-27T09:04:52.591 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-27T09:04:52.591 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-27T09:04:52.591 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-27T09:04:52.591 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-27T09:04:52.591 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-27T09:04:52.591 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-27T09:04:52.591 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-27T09:04:52.591 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-27T09:04:52.591 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-27T09:04:52.591 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-27T09:04:52.591 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-27T09:04:52.592 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-27T09:04:52.592 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-27T09:04:52.592 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-27T09:04:52.592 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-27T09:04:52.592 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-27T09:04:52.592 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-27T09:04:52.592 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-27T09:04:52.592 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-27T09:04:52.592 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-27T09:04:52.592 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-27T09:04:52.592 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-27T09:04:52.592 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-27T09:04:52.592 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-27T09:04:52.592 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-27T09:04:52.593 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-27T09:04:52.593 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-27T09:04:52.593 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-27T09:04:52.593 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-27T09:04:52.593 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-27T09:04:52.593 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-27T09:04:52.593 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-27T09:04:52.593 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-27T09:04:52.593 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-27T09:04:52.593 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-27T09:04:52.593 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-27T09:04:52.593 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-27T09:04:52.593 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-27T09:04:52.594 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-27T09:04:52.594 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-27T09:04:52.594 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-27T09:04:52.594 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-27T09:04:52.594 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-27T09:04:52.594 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-27T09:04:52.594 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-27T09:04:52.594 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-27T09:04:52.594 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-27T09:04:52.594 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-27T09:04:52.594 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-27T09:04:52.594 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-27T09:04:52.594 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-27T09:04:52.595 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-27T09:04:52.595 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-27T09:04:52.595 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-27T09:04:52.595 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-27T09:04:52.595 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-27T09:04:52.595 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-27T09:04:52.595 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-27T09:04:52.595 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-27T09:04:52.595 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-27T09:04:52.595 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-27T09:04:52.595 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-27T09:04:52.595 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-27T09:04:52.595 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-27T09:04:52.595 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-27T09:04:52.596 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-27T09:04:52.596 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-27T09:04:52.596 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-27T09:04:52.596 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-27T09:04:52.596 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-27T09:04:52.596 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-27T09:04:52.596 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-27T09:04:52.596 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-27T09:04:52.596 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-27T09:04:52.596 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-27T09:04:52.596 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-27T09:04:52.596 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-27T09:04:52.596 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-27T09:04:52.596 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-27T09:04:52.597 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-27T09:04:52.597 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-27T09:04:52.597 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-27T09:04:52.597 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-27T09:04:52.597 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-27T09:04:52.597 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-27T09:04:52.597 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-27T09:04:52.597 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-27T09:04:52.597 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-27T09:04:52.597 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-27T09:04:52.597 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-27T09:04:52.597 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-27T09:04:52.597 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-27T09:04:52.597 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-27T09:04:52.598 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-27T09:04:52.598 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-27T09:04:52.598 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-27T09:04:52.598 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-27T09:04:52.598 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-27T09:04:52.598 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-27T09:04:52.598 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-27T09:04:52.598 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-27T09:04:52.598 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-27T09:04:52.598 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-27T09:04:52.598 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-27T09:04:52.599 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-27T09:04:52.599 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-27T09:04:52.599 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-27T09:04:52.599 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-27T09:04:52.599 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-27T09:04:52.599 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-27T09:04:52.599 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-27T09:04:52.599 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-27T09:04:52.599 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-27T09:04:52.599 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-27T09:04:52.599 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-27T09:04:52.599 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-27T09:04:52.599 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-27T09:04:52.599 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-27T09:04:52.600 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-27T09:04:52.600 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-27T09:04:52.600 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-27T09:04:52.600 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-27T09:04:52.600 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-27T09:04:52.600 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-27T09:04:52.600 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-27T09:04:52.600 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-27T09:04:52.600 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-27T09:04:52.600 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-27T09:04:52.600 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-27T09:04:52.600 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-27T09:04:52.600 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-27T09:04:52.601 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-27T09:04:52.601 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-27T09:04:52.601 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-27T09:04:52.601 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-27T09:04:52.601 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-27T09:04:52.601 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-27T09:04:52.601 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-27T09:04:52.601 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-27T09:04:52.601 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-27T09:04:52.601 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-27T09:04:52.601 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-27T09:04:52.601 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-27T09:04:52.601 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-27T09:04:52.601 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-27T09:04:52.602 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-27T09:04:52.602 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-27T09:04:52.602 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-27T09:04:52.602 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-27T09:04:52.602 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-27T09:04:52.602 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-27T09:04:52.602 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-27T09:04:52.602 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-27T09:04:52.602 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-27T09:04:52.602 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-27T09:04:52.602 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-27T09:04:52.602 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-27T09:04:52.602 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-27T09:04:52.603 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-27T09:04:52.603 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-27T09:04:52.603 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-27T09:04:52.603 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-27T09:04:52.603 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-27T09:04:52.603 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-27T09:04:52.603 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-27T09:04:52.603 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-27T09:04:52.603 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-27T09:04:52.603 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-27T09:04:52.603 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-27T09:04:52.603 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-27T09:04:52.603 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-27T09:04:52.604 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-27T09:04:52.604 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-27T09:04:52.604 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-27T09:04:52.604 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-27T09:04:52.604 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-27T09:04:52.604 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-27T09:04:52.604 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-27T09:04:52.604 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-27T09:04:52.604 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-27T09:04:52.604 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-27T09:04:52.604 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-27T09:04:52.604 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-27T09:04:52.604 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-27T09:04:52.605 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-27T09:04:52.605 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-27T09:04:52.605 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-27T09:04:52.605 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-27T09:04:52.605 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-27T09:04:52.605 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-27T09:04:52.605 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-27T09:04:52.605 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-27T09:04:52.605 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-27T09:04:52.605 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-27T09:04:52.605 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-27T09:04:52.605 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-27T09:04:52.605 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-27T09:04:52.606 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-27T09:04:52.606 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-27T09:04:52.606 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-27T09:04:52.606 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-27T09:04:52.606 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-27T09:04:52.606 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-27T09:04:52.606 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-27T09:04:52.606 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-27T09:04:52.606 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-27T09:04:52.606 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-27T09:04:52.606 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-27T09:04:52.606 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-27T09:04:52.606 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-27T09:04:52.606 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-27T09:04:52.607 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-27T09:04:52.607 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-27T09:04:52.607 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-27T09:04:52.607 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-27T09:04:52.607 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-27T09:04:52.607 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-27T09:04:52.607 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-27T09:04:52.607 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-27T09:04:52.607 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-27T09:04:52.607 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-27T09:04:52.607 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-27T09:04:52.607 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-27T09:04:52.608 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-27T09:04:52.608 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-27T09:04:52.608 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-27T09:04:52.608 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-27T09:04:52.608 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-27T09:04:52.608 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-27T09:04:52.608 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-27T09:04:52.608 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-27T09:04:52.608 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-27T09:04:52.608 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-27T09:04:52.608 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-27T09:04:52.608 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-27T09:04:52.608 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-27T09:04:52.609 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-27T09:04:52.609 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-27T09:04:52.609 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-27T09:04:52.609 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-27T09:04:52.609 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-27T09:04:52.609 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-27T09:04:52.609 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-27T09:04:52.609 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-27T09:04:52.609 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-27T09:04:52.609 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-27T09:04:52.609 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-27T09:04:52.609 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-27T09:04:52.609 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-27T09:04:52.609 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-27T09:04:52.610 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-27T09:04:52.610 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-27T09:04:52.610 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-27T09:04:52.610 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-27T09:04:52.610 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-27T09:04:52.610 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-27T09:04:52.610 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-27T09:04:52.610 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-27T09:04:52.610 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-27T09:04:52.610 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-27T09:04:52.610 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-27T09:04:52.610 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-27T09:04:52.610 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-27T09:04:52.611 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-27T09:04:52.611 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-27T09:04:52.611 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-27T09:04:52.611 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-27T09:04:52.611 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-27T09:04:52.611 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-27T09:04:52.611 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-27T09:04:52.611 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-27T09:04:52.611 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-27T09:04:52.611 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-27T09:04:52.611 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-27T09:04:52.611 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-27T09:04:52.611 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-27T09:04:52.612 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-27T09:04:52.612 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-27T09:04:52.612 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-27T09:04:52.612 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-27T09:04:52.612 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-27T09:04:52.612 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-27T09:04:52.612 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-27T09:04:52.612 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-27T09:04:52.612 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-27T09:04:52.612 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-27T09:04:52.612 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-27T09:04:52.612 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-27T09:04:52.612 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-27T09:04:52.612 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-27T09:04:52.613 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-27T09:04:52.613 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-27T09:04:52.613 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-27T09:04:52.613 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-27T09:04:52.613 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-27T09:04:52.613 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-27T09:04:52.613 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-27T09:04:52.613 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-27T09:04:52.613 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-27T09:04:52.613 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-27T09:04:52.613 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-27T09:04:52.613 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-27T09:04:52.613 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-27T09:04:52.613 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-27T09:04:52.614 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-27T09:04:52.614 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-27T09:04:52.614 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-27T09:04:52.614 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-27T09:04:52.614 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-27T09:04:52.614 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-27T09:04:52.614 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-27T09:04:52.614 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-27T09:04:52.614 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-27T09:04:52.614 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-27T09:04:52.614 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-27T09:04:52.614 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-27T09:04:52.614 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-27T09:04:52.615 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-27T09:04:52.615 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-27T09:04:52.615 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-27T09:04:52.615 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-27T09:04:52.615 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-27T09:04:52.615 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-27T09:04:52.615 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-27T09:04:52.615 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-27T09:04:52.615 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-27T09:04:52.615 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-27T09:04:52.615 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-27T09:04:52.615 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-27T09:04:52.615 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-27T09:04:52.616 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-27T09:04:52.616 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-27T09:04:52.616 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-27T09:04:52.616 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-27T09:04:52.616 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-27T09:04:52.616 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-27T09:04:52.616 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-27T09:04:52.616 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-27T09:04:52.616 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-27T09:04:52.616 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-27T09:04:52.616 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-27T09:04:52.616 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-27T09:04:52.616 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-27T09:04:52.616 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-27T09:04:52.617 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-27T09:04:52.617 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-27T09:04:52.617 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-27T09:04:52.617 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-27T09:04:52.617 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-27T09:04:52.617 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-27T09:04:52.617 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-27T09:04:52.617 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-27T09:04:52.617 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-27T09:04:52.617 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-27T09:04:52.617 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-27T09:04:52.617 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-27T09:04:52.617 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-27T09:04:52.618 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-27T09:04:52.618 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-27T09:04:52.618 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-27T09:04:52.618 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-27T09:04:52.618 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-27T09:04:52.618 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-27T09:04:52.618 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-27T09:04:52.618 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-27T09:04:52.618 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-27T09:04:52.618 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-27T09:04:52.618 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-27T09:04:52.618 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-27T09:04:52.618 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-27T09:04:52.618 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-27T09:04:52.619 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-27T09:04:52.619 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-27T09:04:52.619 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-27T09:04:52.619 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-27T09:04:52.619 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-27T09:04:52.619 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-27T09:04:52.619 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-27T09:04:52.619 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-27T09:04:52.619 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-27T09:04:52.619 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-27T09:04:52.619 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-27T09:04:52.619 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-27T09:04:52.619 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-27T09:04:52.620 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-27T09:04:52.620 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-27T09:04:52.620 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-27T09:04:52.620 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-27T09:04:52.620 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-27T09:04:52.620 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-27T09:04:52.620 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-27T09:04:52.620 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-27T09:04:52.620 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-27T09:04:52.620 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-27T09:04:52.620 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-27T09:04:52.620 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-27T09:04:52.620 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-27T09:04:52.620 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-27T09:04:52.621 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-27T09:04:52.621 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-27T09:04:52.621 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-27T09:04:52.621 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-27T09:04:52.621 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-27T09:04:52.621 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-27T09:04:52.621 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-27T09:04:52.621 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-27T09:04:52.621 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-27T09:04:52.621 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-27T09:04:52.621 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-27T09:04:52.621 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-27T09:04:52.621 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-27T09:04:52.621 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-27T09:04:52.622 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-27T09:04:52.622 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-27T09:04:52.622 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-27T09:04:52.622 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-27T09:04:52.622 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-27T09:04:52.622 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-27T09:04:52.622 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-27T09:04:52.622 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-27T09:04:52.622 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-27T09:04:52.622 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-27T09:04:52.622 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-27T09:04:52.622 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-27T09:04:52.622 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-27T09:04:52.623 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-27T09:04:52.623 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-27T09:04:52.623 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-27T09:04:52.623 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-27T09:04:52.623 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-27T09:04:52.623 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-27T09:04:52.623 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-27T09:04:52.623 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-27T09:04:52.623 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-27T09:04:52.623 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-27T09:04:52.623 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-27T09:04:52.623 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-27T09:04:52.623 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-27T09:04:52.624 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-27T09:04:52.624 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-27T09:04:52.624 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-27T09:04:52.624 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-27T09:04:52.624 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-27T09:04:52.624 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-27T09:04:52.624 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-27T09:04:52.624 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-27T09:04:52.624 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-27T09:04:52.624 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-27T09:04:52.624 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-27T09:04:52.624 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-27T09:04:52.624 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-27T09:04:52.625 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-27T09:04:52.625 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-27T09:04:52.625 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-27T09:04:52.625 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-27T09:04:52.625 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-27T09:04:52.625 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-27T09:04:52.625 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-27T09:04:52.625 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-27T09:04:52.625 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-27T09:04:52.625 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-27T09:04:52.625 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-27T09:04:52.625 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-27T09:04:52.625 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-27T09:04:52.625 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-27T09:04:52.626 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-27T09:04:52.626 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-27T09:04:52.626 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-27T09:04:52.626 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-27T09:04:52.626 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-27T09:04:52.626 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-27T09:04:52.626 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-27T09:04:52.626 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-27T09:04:52.626 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-27T09:04:52.626 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-27T09:04:52.626 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-27T09:04:52.626 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-27T09:04:52.627 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-27T09:04:52.627 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-27T09:04:52.627 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-27T09:04:52.627 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-27T09:04:52.627 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-27T09:04:52.627 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-27T09:04:52.627 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-27T09:04:52.627 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-27T09:04:52.627 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-27T09:04:52.627 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-27T09:04:52.627 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-27T09:04:52.627 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-27T09:04:52.627 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-27T09:04:52.628 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-27T09:04:52.628 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-27T09:04:52.628 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-27T09:04:52.628 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-27T09:04:52.628 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-27T09:04:52.628 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-27T09:04:52.628 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-27T09:04:52.628 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-27T09:04:52.628 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-27T09:04:52.628 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-27T09:04:52.628 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-27T09:04:52.628 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-27T09:04:52.628 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-27T09:04:52.629 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-27T09:04:52.629 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-27T09:04:52.629 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-27T09:04:52.629 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-27T09:04:52.629 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-27T09:04:52.629 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-27T09:04:52.629 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-27T09:04:52.629 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-27T09:04:52.629 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-27T09:04:52.629 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-27T09:04:52.629 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-27T09:04:52.629 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-27T09:04:52.629 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-27T09:04:52.629 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-27T09:04:52.630 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-27T09:04:52.630 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-27T09:04:52.630 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-27T09:04:52.630 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-27T09:04:52.630 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-27T09:04:52.630 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-27T09:04:52.630 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-27T09:04:52.630 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-27T09:04:52.630 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-27T09:04:52.630 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-27T09:04:52.630 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-27T09:04:52.630 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-27T09:04:52.630 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-27T09:04:52.631 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-27T09:04:52.631 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-27T09:04:52.631 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-27T09:04:52.631 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-27T09:04:52.631 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-27T09:04:52.631 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-27T09:04:52.631 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-27T09:04:52.631 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-27T09:04:52.631 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-27T09:04:52.631 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-27T09:04:52.631 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-27T09:04:52.631 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-27T09:04:52.631 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-27T09:04:52.631 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-27T09:04:52.631 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-27T09:04:52.632 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-27T09:04:52.632 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-27T09:04:52.632 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-27T09:04:52.632 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-27T09:04:52.632 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-27T09:04:52.632 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-27T09:04:52.632 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-27T09:04:52.632 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-27T09:04:52.632 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-27T09:04:52.632 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-27T09:04:52.632 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-27T09:04:52.632 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-27T09:04:52.632 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-27T09:04:52.632 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-27T09:04:52.633 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-27T09:04:52.633 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-27T09:04:52.633 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-27T09:04:52.633 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-27T09:04:52.633 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-27T09:04:52.633 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-27T09:04:52.633 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-27T09:04:52.633 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-27T09:04:52.633 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-27T09:04:52.633 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-27T09:04:52.633 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-27T09:04:52.633 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-27T09:04:52.633 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-27T09:04:52.634 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-27T09:04:52.634 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-27T09:04:52.634 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-27T09:04:52.634 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-27T09:04:52.634 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-27T09:04:52.634 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-27T09:04:52.634 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-27T09:04:52.634 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-27T09:04:52.634 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-27T09:04:52.634 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-27T09:04:52.634 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-27T09:04:52.634 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-27T09:04:52.635 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-27T09:04:52.635 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-27T09:04:52.635 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-27T09:04:52.635 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-27T09:04:52.635 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-27T09:04:52.635 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-27T09:04:52.635 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-27T09:04:52.635 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-27T09:04:52.635 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-27T09:04:52.635 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-27T09:04:52.635 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-27T09:04:52.635 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-27T09:04:52.635 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-27T09:04:52.635 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-27T09:04:52.636 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-27T09:04:52.636 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-27T09:04:52.636 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-27T09:04:52.636 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-27T09:04:52.636 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-27T09:04:52.636 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-27T09:04:52.636 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-27T09:04:52.636 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-27T09:04:52.636 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-27T09:04:52.636 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-27T09:04:52.636 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-27T09:04:52.636 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-27T09:04:52.636 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-27T09:04:52.636 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-27T09:04:52.637 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-27T09:04:52.637 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-27T09:04:52.637 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-27T09:04:52.637 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-27T09:04:52.637 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-27T09:04:52.637 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-27T09:04:52.637 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-27T09:04:52.637 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-27T09:04:52.637 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-27T09:04:52.637 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-27T09:04:52.637 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-27T09:04:52.637 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-27T09:04:52.637 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-27T09:04:52.638 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-27T09:04:52.638 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-27T09:04:52.638 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-27T09:04:52.638 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-27T09:04:52.638 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-27T09:04:52.638 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-27T09:04:52.638 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-27T09:04:52.638 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-27T09:04:52.638 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-27T09:04:52.638 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-27T09:04:52.638 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-27T09:04:52.638 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-27T09:04:52.638 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-27T09:04:52.639 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-27T09:04:52.639 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-27T09:04:52.639 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-27T09:04:52.639 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-27T09:04:52.639 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-27T09:04:52.639 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-27T09:04:52.639 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-27T09:04:52.639 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-27T09:04:52.639 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-27T09:04:52.639 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-27T09:04:52.639 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-27T09:04:52.639 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-27T09:04:52.639 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-27T09:04:52.639 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-27T09:04:52.640 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-27T09:04:52.640 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-27T09:04:52.640 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-27T09:04:52.640 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-27T09:04:52.640 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-27T09:04:52.640 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-27T09:04:52.640 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-27T09:04:52.640 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-27T09:04:52.640 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-27T09:04:52.640 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-27T09:04:52.640 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-27T09:04:52.640 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-27T09:04:52.640 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-27T09:04:52.640 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-27T09:04:52.641 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-27T09:04:52.641 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-27T09:04:52.641 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-27T09:04:52.641 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-27T09:04:52.641 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-27T09:04:52.641 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-27T09:04:52.641 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-27T09:04:52.641 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-27T09:04:52.641 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-27T09:04:52.641 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-27T09:04:52.641 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-27T09:04:52.641 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-27T09:04:52.641 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-27T09:04:52.642 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-27T09:04:52.642 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-27T09:04:52.642 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-27T09:04:52.642 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-27T09:04:52.642 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-27T09:04:52.642 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-27T09:04:52.642 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-27T09:04:52.642 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-27T09:04:52.642 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-27T09:04:52.642 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-27T09:04:52.642 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-27T09:04:52.642 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-27T09:04:52.642 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-27T09:04:52.643 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-27T09:04:52.643 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-27T09:04:52.643 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-27T09:04:52.643 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-27T09:04:52.643 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-27T09:04:52.643 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-27T09:04:52.643 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-27T09:04:52.643 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-27T09:04:52.643 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-27T09:04:52.643 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-27T09:04:52.643 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-27T09:04:52.643 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-27T09:04:52.643 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-27T09:04:52.644 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-27T09:04:52.644 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-27T09:04:52.644 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-27T09:04:52.644 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-27T09:04:52.644 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-27T09:04:52.644 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-27T09:04:52.644 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-27T09:04:52.644 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-27T09:04:52.644 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-27T09:04:52.644 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-27T09:04:52.644 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-27T09:04:52.644 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-27T09:04:52.644 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-27T09:04:52.644 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-27T09:04:52.645 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-27T09:04:52.645 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-27T09:04:52.645 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-27T09:04:52.645 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-27T09:04:52.645 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-27T09:04:52.645 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-27T09:04:52.645 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-27T09:04:52.645 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-27T09:04:52.645 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-27T09:04:52.645 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-27T09:04:52.645 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-27T09:04:52.645 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-27T09:04:52.646 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-27T09:04:52.646 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-27T09:04:52.646 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-27T09:04:52.646 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-27T09:04:52.646 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-27T09:04:52.646 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-27T09:04:52.646 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-27T09:04:52.646 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-27T09:04:52.646 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-27T09:04:52.646 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-27T09:04:52.646 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-27T09:04:52.646 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-27T09:04:52.646 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-27T09:04:52.647 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-27T09:04:52.647 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-27T09:04:52.647 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-27T09:04:52.647 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-27T09:04:52.647 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-27T09:04:52.647 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-27T09:04:52.647 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-27T09:04:52.647 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-27T09:04:52.647 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-27T09:04:52.647 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-27T09:04:52.647 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-27T09:04:52.647 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-27T09:04:52.647 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-27T09:04:52.647 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-27T09:04:52.648 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-27T09:04:52.648 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-27T09:04:52.648 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-27T09:04:52.648 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-27T09:04:52.648 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-27T09:04:52.648 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-27T09:04:52.648 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-27T09:04:52.648 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-27T09:04:52.648 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-27T09:04:52.648 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-27T09:04:52.648 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-27T09:04:52.648 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-27T09:04:52.648 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-27T09:04:52.648 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-27T09:04:52.649 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-27T09:04:52.649 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-27T09:04:52.649 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-27T09:04:52.649 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-27T09:04:52.649 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-27T09:04:52.649 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-27T09:04:52.649 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-27T09:04:52.649 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-27T09:04:52.649 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-27T09:04:52.649 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-27T09:04:52.649 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-27T09:04:52.649 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-27T09:04:52.649 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-27T09:04:52.649 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-27T09:04:52.650 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-27T09:04:52.650 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-27T09:04:52.650 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-27T09:04:52.650 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-27T09:04:52.650 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-27T09:04:52.650 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-27T09:04:52.650 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-27T09:04:52.650 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-27T09:04:52.650 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-27T09:04:52.650 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-27T09:04:52.650 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-27T09:04:52.650 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-27T09:04:52.650 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-27T09:04:52.651 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-27T09:04:52.651 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-27T09:04:52.651 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-27T09:04:52.651 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-27T09:04:52.651 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-27T09:04:52.651 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-27T09:04:52.651 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-27T09:04:52.651 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-27T09:04:52.651 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-27T09:04:52.651 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-27T09:04:52.651 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-27T09:04:52.651 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-27T09:04:52.651 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-27T09:04:52.651 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-27T09:04:52.652 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-27T09:04:52.652 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-27T09:04:52.652 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-27T09:04:52.652 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-27T09:04:52.652 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-27T09:04:52.652 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-27T09:04:52.652 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-27T09:04:52.652 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-27T09:04:52.652 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-27T09:04:52.652 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-27T09:04:52.652 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-27T09:04:52.652 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-27T09:04:52.652 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-27T09:04:52.653 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-27T09:04:52.653 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-27T09:04:52.653 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-27T09:04:52.653 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-27T09:04:52.653 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-27T09:04:52.653 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-27T09:04:52.653 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-27T09:04:52.653 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-27T09:04:52.653 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-27T09:04:52.653 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-27T09:04:52.653 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-27T09:04:52.653 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-27T09:04:52.653 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-27T09:04:52.654 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-27T09:04:52.654 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-27T09:04:52.654 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-27T09:04:52.654 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-27T09:04:52.654 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-27T09:04:52.654 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-27T09:04:52.654 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-27T09:04:52.654 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-27T09:04:52.654 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-27T09:04:52.654 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-27T09:04:52.654 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-27T09:04:52.654 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-27T09:04:52.654 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-27T09:04:52.655 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-27T09:04:52.655 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-27T09:04:52.655 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-27T09:04:52.655 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-27T09:04:52.655 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-27T09:04:52.655 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-27T09:04:52.655 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-27T09:04:52.655 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-27T09:04:52.655 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-27T09:04:52.655 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-27T09:04:52.655 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-27T09:04:52.655 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-27T09:04:52.655 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-27T09:04:52.656 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-27T09:04:52.656 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-27T09:04:52.656 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-27T09:04:52.656 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-27T09:04:52.656 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-27T09:04:52.656 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-27T09:04:52.656 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-27T09:04:52.656 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-27T09:04:52.656 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-27T09:04:52.656 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-27T09:04:52.656 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-27T09:04:52.656 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-27T09:04:52.656 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-27T09:04:52.656 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-27T09:04:52.657 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-27T09:04:52.657 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-27T09:04:52.657 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-27T09:04:52.657 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-27T09:04:52.657 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-27T09:04:52.657 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-27T09:04:52.657 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-27T09:04:52.657 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-27T09:04:52.657 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-27T09:04:52.657 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-27T09:04:52.657 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-27T09:04:52.657 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-27T09:04:52.657 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-27T09:04:52.658 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-27T09:04:52.658 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-27T09:04:52.658 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-27T09:04:52.658 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-27T09:04:52.658 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-27T09:04:52.658 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-27T09:04:52.658 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-27T09:04:52.658 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-27T09:04:52.658 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-27T09:04:52.658 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-27T09:04:52.658 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-27T09:04:52.658 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-27T09:04:52.658 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-27T09:04:52.658 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-27T09:04:52.659 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-27T09:04:52.659 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-27T09:04:52.659 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-27T09:04:52.659 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-27T09:04:52.659 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-27T09:04:52.659 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-27T09:04:52.659 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-27T09:04:52.659 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-27T09:04:52.659 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-27T09:04:52.659 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-27T09:04:52.659 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-27T09:04:52.659 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-27T09:04:52.659 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-27T09:04:52.660 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-27T09:04:52.660 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-27T09:04:52.660 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-27T09:04:52.660 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-27T09:04:52.660 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-27T09:04:52.660 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-27T09:04:52.660 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-27T09:04:52.660 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-27T09:04:52.660 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-27T09:04:52.660 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-27T09:04:52.660 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-27T09:04:52.660 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-27T09:04:52.660 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-27T09:04:52.661 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-27T09:04:52.661 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-27T09:04:52.661 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-27T09:04:52.661 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-27T09:04:52.661 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-27T09:04:52.661 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-27T09:04:52.661 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-27T09:04:52.661 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-27T09:04:52.661 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-27T09:04:52.661 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-27T09:04:52.661 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-27T09:04:52.661 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-27T09:04:52.661 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-27T09:04:52.662 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-27T09:04:52.662 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-27T09:04:52.662 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-27T09:04:52.662 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-27T09:04:52.662 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-27T09:04:52.662 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-27T09:04:52.662 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-27T09:04:52.662 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-27T09:04:52.662 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-27T09:04:52.662 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-27T09:04:52.662 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-27T09:04:52.662 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-27T09:04:52.663 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-27T09:04:52.663 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-27T09:04:52.663 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-27T09:04:52.663 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-27T09:04:52.663 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-27T09:04:52.663 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-27T09:04:52.663 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-27T09:04:52.663 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-27T09:04:52.663 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-27T09:04:52.663 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-27T09:04:52.663 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-27T09:04:52.663 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-27T09:04:52.663 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-27T09:04:52.664 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-27T09:04:52.664 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-27T09:04:52.664 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-27T09:04:52.664 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-27T09:04:52.664 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-27T09:04:52.664 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-27T09:04:52.664 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-27T09:04:52.664 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-27T09:04:52.664 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-27T09:04:52.664 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-27T09:04:52.664 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-27T09:04:52.664 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-27T09:04:52.664 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-27T09:04:52.664 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-27T09:04:52.665 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-27T09:04:52.665 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-27T09:04:52.665 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-27T09:04:52.665 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-27T09:04:52.665 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-27T09:04:52.665 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-27T09:04:52.665 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-27T09:04:52.665 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-27T09:04:52.665 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-27T09:04:52.665 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-27T09:04:52.665 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-27T09:04:52.665 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-27T09:04:52.665 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-27T09:04:52.665 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-27T09:04:52.666 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-27T09:04:52.666 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-27T09:04:52.666 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-27T09:04:52.666 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-27T09:04:52.666 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-27T09:04:52.666 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-27T09:04:52.666 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-27T09:04:52.666 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-27T09:04:52.666 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-27T09:04:52.666 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-27T09:04:52.666 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-27T09:04:52.666 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-27T09:04:52.667 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-27T09:04:52.667 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-27T09:04:52.667 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-27T09:04:52.667 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-27T09:04:52.667 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-27T09:04:52.667 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-27T09:04:52.667 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-27T09:04:52.667 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-27T09:04:52.667 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-27T09:04:52.667 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-27T09:04:52.667 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-27T09:04:52.667 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-27T09:04:52.667 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-27T09:04:52.668 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-27T09:04:52.668 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-27T09:04:52.668 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-27T09:04:52.668 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-27T09:04:52.668 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-27T09:04:52.668 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-27T09:04:52.668 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-27T09:04:52.668 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-27T09:04:52.668 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-27T09:04:52.668 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-27T09:04:52.668 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-27T09:04:52.668 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-27T09:04:52.668 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-27T09:04:52.669 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-27T09:04:52.669 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-27T09:04:52.669 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-27T09:04:52.669 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-27T09:04:52.669 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-27T09:04:52.669 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-27T09:04:52.669 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-27T09:04:52.669 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-27T09:04:52.669 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-27T09:04:52.669 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-27T09:04:52.669 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-27T09:04:52.669 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-27T09:04:52.669 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-27T09:04:52.670 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-27T09:04:52.670 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-27T09:04:52.670 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-27T09:04:52.670 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-27T09:04:52.670 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-27T09:04:52.670 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-27T09:04:52.670 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-27T09:04:52.670 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-27T09:04:52.670 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-27T09:04:52.670 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-27T09:04:52.670 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-27T09:04:52.670 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-27T09:04:52.670 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-27T09:04:52.671 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-27T09:04:52.671 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-27T09:04:52.671 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-27T09:04:52.671 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-27T09:04:52.671 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-27T09:04:52.671 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-27T09:04:52.671 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-27T09:04:52.671 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-27T09:04:52.671 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-27T09:04:52.671 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-27T09:04:52.671 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-27T09:04:52.671 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-27T09:04:52.671 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-27T09:04:52.672 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-27T09:04:52.672 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-27T09:04:52.672 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-27T09:04:52.672 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-27T09:04:52.672 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-27T09:04:52.672 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-27T09:04:52.672 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-27T09:04:52.672 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-27T09:04:52.672 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-27T09:04:52.672 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-27T09:04:52.672 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-27T09:04:52.672 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-27T09:04:52.672 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-27T09:04:52.673 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-27T09:04:52.673 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-27T09:04:52.673 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-27T09:04:52.673 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-27T09:04:52.673 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-27T09:04:52.673 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-27T09:04:52.673 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-27T09:04:52.673 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-27T09:04:52.673 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-27T09:04:52.673 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-27T09:04:52.673 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-27T09:04:52.673 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-27T09:04:52.673 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-27T09:04:52.674 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-27T09:04:52.674 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-27T09:04:52.674 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-27T09:04:52.674 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-27T09:04:52.674 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-27T09:04:52.674 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-27T09:04:52.674 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-27T09:04:52.674 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-27T09:04:52.674 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-27T09:04:52.674 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-27T09:04:52.674 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-27T09:04:52.674 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-27T09:04:52.674 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-27T09:04:52.675 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-27T09:04:52.675 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-27T09:04:52.675 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-27T09:04:52.675 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-27T09:04:52.675 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-27T09:04:52.675 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-27T09:04:52.675 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-27T09:04:52.675 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-27T09:04:52.675 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-27T09:04:52.675 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-27T09:04:52.675 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-27T09:04:52.675 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-27T09:04:52.675 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-27T09:04:52.675 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-27T09:04:52.676 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-27T09:04:52.676 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-27T09:04:52.676 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-27T09:04:52.676 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-27T09:04:52.676 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-27T09:04:52.676 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-27T09:04:52.676 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-27T09:04:52.676 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-27T09:04:52.676 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-27T09:04:52.676 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-27T09:04:52.676 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-27T09:04:52.676 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-27T09:04:52.676 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-27T09:04:52.676 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-27T09:04:52.677 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-27T09:04:52.677 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-27T09:04:52.677 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-27T09:04:52.677 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-27T09:04:52.677 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-27T09:04:52.677 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-27T09:04:52.677 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-27T09:04:52.677 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-27T09:04:52.677 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-27T09:04:52.677 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-27T09:04:52.677 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-27T09:04:52.677 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-27T09:04:52.677 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-27T09:04:52.677 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-27T09:04:52.678 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-27T09:04:52.678 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-27T09:04:52.678 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-27T09:04:52.678 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-27T09:04:52.678 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-27T09:04:52.678 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-27T09:04:52.678 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-27T09:04:52.678 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-27T09:04:52.678 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-27T09:04:52.678 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-27T09:04:52.678 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-27T09:04:52.678 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-27T09:04:52.678 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-27T09:04:52.679 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-27T09:04:52.679 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-27T09:04:52.679 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-27T09:04:52.679 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-27T09:04:52.679 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-27T09:04:52.679 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-27T09:04:52.679 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-27T09:04:52.679 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-27T09:04:52.679 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-27T09:04:52.679 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-27T09:04:52.679 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-27T09:04:52.679 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-27T09:04:52.679 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-27T09:04:52.679 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-27T09:04:52.680 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-27T09:04:52.680 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-27T09:04:52.680 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-27T09:04:52.680 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-27T09:04:52.680 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-27T09:04:52.680 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-27T09:04:52.680 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-27T09:04:52.680 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-27T09:04:52.680 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-27T09:04:52.680 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-27T09:04:52.680 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-27T09:04:52.680 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-27T09:04:52.680 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-27T09:04:52.680 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-27T09:04:52.681 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-27T09:04:52.681 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-27T09:04:52.681 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-27T09:04:52.681 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-27T09:04:52.681 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-27T09:04:52.681 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-27T09:04:52.681 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-27T09:04:52.681 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-27T09:04:52.681 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-27T09:04:52.681 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-27T09:04:52.681 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-27T09:04:52.681 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-27T09:04:52.681 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-27T09:04:52.681 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-27T09:04:52.682 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-27T09:04:52.682 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-27T09:04:52.682 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-27T09:04:52.682 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-27T09:04:52.682 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-27T09:04:52.682 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-27T09:04:52.682 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-27T09:04:52.682 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-27T09:04:52.682 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-27T09:04:52.682 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-27T09:04:52.682 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-27T09:04:52.682 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-27T09:04:52.682 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-27T09:04:52.683 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-27T09:04:52.683 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-27T09:04:52.683 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-27T09:04:52.683 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-27T09:04:52.683 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-27T09:04:52.683 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-27T09:04:52.683 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-27T09:04:52.683 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-27T09:04:52.683 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-27T09:04:52.683 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-27T09:04:52.683 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-27T09:04:52.683 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-27T09:04:52.683 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-27T09:04:52.683 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-27T09:04:52.684 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-27T09:04:52.684 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-27T09:04:52.684 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-27T09:04:52.684 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-27T09:04:52.684 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-27T09:04:52.684 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-27T09:04:52.684 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-27T09:04:52.684 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-27T09:04:52.684 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-27T09:04:52.684 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-27T09:04:52.684 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-27T09:04:52.684 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-27T09:04:52.684 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-27T09:04:52.684 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-27T09:04:52.685 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-27T09:04:52.685 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-27T09:04:52.685 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-27T09:04:52.685 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-27T09:04:52.685 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-27T09:04:52.685 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-27T09:04:52.685 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-27T09:04:52.685 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-27T09:04:52.685 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-27T09:04:52.685 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-27T09:04:52.685 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-27T09:04:52.685 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-27T09:04:52.685 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-27T09:04:52.685 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-27T09:04:52.686 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-27T09:04:52.686 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-27T09:04:52.686 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-27T09:04:52.686 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-27T09:04:52.686 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-27T09:04:52.686 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-27T09:04:52.686 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-27T09:04:52.686 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-27T09:04:52.686 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-27T09:04:52.686 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-27T09:04:52.686 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-27T09:04:52.686 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-27T09:04:52.686 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-27T09:04:52.687 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-27T09:04:52.687 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-27T09:04:52.687 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-27T09:04:52.687 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-27T09:04:52.687 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-27T09:04:52.687 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-27T09:04:52.687 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-27T09:04:52.687 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-27T09:04:52.687 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-27T09:04:52.687 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-27T09:04:52.687 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-27T09:04:52.687 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-27T09:04:52.687 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-27T09:04:52.687 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-27T09:04:52.688 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-27T09:04:52.688 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-27T09:04:52.688 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-27T09:04:52.688 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-27T09:04:52.688 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-27T09:04:52.688 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-27T09:04:52.688 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-27T09:04:52.688 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-27T09:04:52.688 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-27T09:04:52.688 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-27T09:04:52.688 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-27T09:04:52.688 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-27T09:04:52.688 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-27T09:04:52.689 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-27T09:04:52.689 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-27T09:04:52.689 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-27T09:04:52.689 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-27T09:04:52.689 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-27T09:04:52.689 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-27T09:04:52.689 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-27T09:04:52.689 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-27T09:04:52.689 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-27T09:04:52.689 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-27T09:04:52.689 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-27T09:04:52.689 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-27T09:04:52.689 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-27T09:04:52.689 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-27T09:04:52.690 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-27T09:04:52.690 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-27T09:04:52.690 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-27T09:04:52.690 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-27T09:04:52.690 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-27T09:04:52.690 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-27T09:04:52.690 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-27T09:04:52.690 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-27T09:04:52.690 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-27T09:04:52.690 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-27T09:04:52.690 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-27T09:04:52.690 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-27T09:04:52.690 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-27T09:04:52.691 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-27T09:04:52.691 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-27T09:04:52.691 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-27T09:04:52.691 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-27T09:04:52.691 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-27T09:04:52.691 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-27T09:04:52.691 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-27T09:04:52.691 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-27T09:04:52.691 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-27T09:04:52.691 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-27T09:04:52.691 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-27T09:04:52.691 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-27T09:04:52.691 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-27T09:04:52.691 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-27T09:04:52.692 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-27T09:04:52.692 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-27T09:04:52.692 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-27T09:04:52.692 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-27T09:04:52.692 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-27T09:04:52.692 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-27T09:04:52.692 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-27T09:04:52.692 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-27T09:04:52.692 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-27T09:04:52.692 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-27T09:04:52.692 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-27T09:04:52.692 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-27T09:04:52.692 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-27T09:04:52.692 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-27T09:04:52.693 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-27T09:04:52.693 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-27T09:04:52.693 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-27T09:04:52.693 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-27T09:04:52.693 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-27T09:04:52.693 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-27T09:04:52.693 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-27T09:04:52.693 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-27T09:04:52.693 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-27T09:04:52.693 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-27T09:04:52.693 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-27T09:04:52.693 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-27T09:04:52.693 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-27T09:04:52.693 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-27T09:04:52.694 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-27T09:04:52.694 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-27T09:04:52.694 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-27T09:04:52.694 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-27T09:04:52.694 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-27T09:04:52.694 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-27T09:04:52.694 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-27T09:04:52.694 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-27T09:04:52.694 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-27T09:04:52.694 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-27T09:04:52.694 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-27T09:04:52.694 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-27T09:04:52.694 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-27T09:04:52.694 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-27T09:04:52.695 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-27T09:04:52.695 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-27T09:04:52.695 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-27T09:04:52.695 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-27T09:04:52.695 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-27T09:04:52.695 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-27T09:04:52.695 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-27T09:04:52.695 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-27T09:04:52.695 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-27T09:04:52.695 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-27T09:04:52.695 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-27T09:04:52.695 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-27T09:04:52.695 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-27T09:04:52.695 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-27T09:04:52.696 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-27T09:04:52.696 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-27T09:04:52.696 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-27T09:04:52.696 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-27T09:04:52.696 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-27T09:04:52.696 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-27T09:04:52.696 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-27T09:04:52.696 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-27T09:04:52.696 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-27T09:04:52.696 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-27T09:04:52.696 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-27T09:04:52.696 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-27T09:04:52.696 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-27T09:04:52.696 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-27T09:04:52.697 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-27T09:04:52.697 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-27T09:04:52.697 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-27T09:04:52.697 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-27T09:04:52.697 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-27T09:04:52.697 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-27T09:04:52.697 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-27T09:04:52.697 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-27T09:04:52.697 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-27T09:04:52.697 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-27T09:04:52.697 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-27T09:04:52.697 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-27T09:04:52.697 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-27T09:04:52.698 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-27T09:04:52.698 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-27T09:04:52.698 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-27T09:04:52.698 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-27T09:04:52.698 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-27T09:04:52.698 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-27T09:04:52.698 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-27T09:04:52.698 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-27T09:04:52.698 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-27T09:04:52.698 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-27T09:04:52.698 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-27T09:04:52.698 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-27T09:04:52.698 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-27T09:04:52.699 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-27T09:04:52.699 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-27T09:04:52.699 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-27T09:04:52.699 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-27T09:04:52.699 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-27T09:04:52.699 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-27T09:04:52.699 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-27T09:04:52.699 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-27T09:04:52.699 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-27T09:04:52.699 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-27T09:04:52.699 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-27T09:04:52.699 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-27T09:04:52.699 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-27T09:04:52.699 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-27T09:04:52.700 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-27T09:04:52.700 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-27T09:04:52.700 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-27T09:04:52.700 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-27T09:04:52.700 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-27T09:04:52.700 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-27T09:04:52.700 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-27T09:04:52.700 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-27T09:04:52.700 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-27T09:04:52.700 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-27T09:04:52.700 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-27T09:04:52.700 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-27T09:04:52.700 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-27T09:04:52.701 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-27T09:04:52.701 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-27T09:04:52.701 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-27T09:04:52.701 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-27T09:04:52.701 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-27T09:04:52.701 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-27T09:04:52.701 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-27T09:04:52.701 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-27T09:04:52.701 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-27T09:04:52.701 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-27T09:04:52.701 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-27T09:04:52.701 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-27T09:04:52.701 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-27T09:04:52.701 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-27T09:04:52.702 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-27T09:04:52.702 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-27T09:04:52.702 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-27T09:04:52.702 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-27T09:04:52.702 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-27T09:04:52.702 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-27T09:04:52.702 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-27T09:04:52.702 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-27T09:04:52.702 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-27T09:04:52.702 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-27T09:04:52.702 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-27T09:04:52.702 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-27T09:04:52.702 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-27T09:04:52.703 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-27T09:04:52.703 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-27T09:04:52.703 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-27T09:04:52.703 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-27T09:04:52.703 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-27T09:04:52.703 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-27T09:04:52.703 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-27T09:04:52.703 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-27T09:04:52.703 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-27T09:04:52.703 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-27T09:04:52.703 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-27T09:04:52.703 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-27T09:04:52.703 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-27T09:04:52.704 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-27T09:04:52.704 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-27T09:04:52.704 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-27T09:04:52.704 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-27T09:04:52.704 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-27T09:04:52.704 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-27T09:04:52.704 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-27T09:04:52.704 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-27T09:04:52.704 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-27T09:04:52.704 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-27T09:04:52.704 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-27T09:04:52.704 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-27T09:04:52.704 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-27T09:04:52.704 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-27T09:04:52.705 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-27T09:04:52.705 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-27T09:04:52.705 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-27T09:04:52.705 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-27T09:04:52.705 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-27T09:04:52.705 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-27T09:04:52.705 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-27T09:04:52.705 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-27T09:04:52.705 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-27T09:04:52.705 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-27T09:04:52.705 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-27T09:04:52.705 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-27T09:04:52.705 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-27T09:04:52.705 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-27T09:04:52.706 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-27T09:04:52.706 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-27T09:04:52.706 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-27T09:04:52.706 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-27T09:04:52.706 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-27T09:04:52.706 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T09:04:52.710 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-27T09:04:52.710 DEBUG:teuthology.orchestra.run.smithi119:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-27T09:04:52.735 INFO:teuthology.orchestra.run.smithi119.stdout:postinstall scriptlet (using /bin/sh): 2024-05-27T09:04:52.736 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T09:04:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T09:04:52.736 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2024-05-27T09:04:52.736 INFO:teuthology.orchestra.run.smithi119.stdout:for file in vmlinuz System.map config; do 2024-05-27T09:04:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-27T09:04:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-27T09:04:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: fi 2024-05-27T09:04:52.736 INFO:teuthology.orchestra.run.smithi119.stdout:done 2024-05-27T09:04:52.736 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-27T09:04:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-27T09:04:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T09:04:52.736 INFO:teuthology.orchestra.run.smithi119.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T09:04:52.736 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2024-05-27T09:04:52.737 INFO:teuthology.orchestra.run.smithi119.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-27T09:04:52.737 INFO:teuthology.orchestra.run.smithi119.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-27T09:04:52.737 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2024-05-27T09:04:52.737 INFO:teuthology.orchestra.run.smithi119.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-27T09:04:52.737 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T09:04:52.737 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-27T09:04:52.737 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2024-05-27T09:04:52.738 DEBUG:teuthology.orchestra.run.smithi119:> sudo rpm -qi grub2-tools 2024-05-27T09:04:52.814 INFO:teuthology.orchestra.run.smithi119.stdout:Name : grub2-tools 2024-05-27T09:04:52.815 INFO:teuthology.orchestra.run.smithi119.stdout:Epoch : 1 2024-05-27T09:04:52.815 INFO:teuthology.orchestra.run.smithi119.stdout:Version : 2.06 2024-05-27T09:04:52.815 INFO:teuthology.orchestra.run.smithi119.stdout:Release : 68.el9 2024-05-27T09:04:52.815 INFO:teuthology.orchestra.run.smithi119.stdout:Architecture: x86_64 2024-05-27T09:04:52.815 INFO:teuthology.orchestra.run.smithi119.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-27T09:04:52.815 INFO:teuthology.orchestra.run.smithi119.stdout:Group : Unspecified 2024-05-27T09:04:52.815 INFO:teuthology.orchestra.run.smithi119.stdout:Size : 8264254 2024-05-27T09:04:52.815 INFO:teuthology.orchestra.run.smithi119.stdout:License : GPLv3+ 2024-05-27T09:04:52.815 INFO:teuthology.orchestra.run.smithi119.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-27T09:04:52.815 INFO:teuthology.orchestra.run.smithi119.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-27T09:04:52.815 INFO:teuthology.orchestra.run.smithi119.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-27T09:04:52.815 INFO:teuthology.orchestra.run.smithi119.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-27T09:04:52.815 INFO:teuthology.orchestra.run.smithi119.stdout:Packager : builder@centos.org 2024-05-27T09:04:52.816 INFO:teuthology.orchestra.run.smithi119.stdout:Vendor : CentOS 2024-05-27T09:04:52.816 INFO:teuthology.orchestra.run.smithi119.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-27T09:04:52.816 INFO:teuthology.orchestra.run.smithi119.stdout:Summary : Support tools for GRUB. 2024-05-27T09:04:52.816 INFO:teuthology.orchestra.run.smithi119.stdout:Description : 2024-05-27T09:04:52.816 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:04:52.816 INFO:teuthology.orchestra.run.smithi119.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-27T09:04:52.816 INFO:teuthology.orchestra.run.smithi119.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-27T09:04:52.816 INFO:teuthology.orchestra.run.smithi119.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-27T09:04:52.816 INFO:teuthology.orchestra.run.smithi119.stdout:hardware devices. 2024-05-27T09:04:52.816 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:04:52.816 INFO:teuthology.orchestra.run.smithi119.stdout:This subpackage provides tools for support of all platforms. 2024-05-27T09:04:52.817 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-27T09:04:52.817 INFO:teuthology.task.kernel:Updating grub on smithi119 to boot 6.9.0-gb9459a26b088 2024-05-27T09:04:52.817 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-27T09:04:53.410 INFO:teuthology.orchestra.run.smithi119.stderr:Generating grub configuration file ... 2024-05-27T09:04:54.848 INFO:teuthology.orchestra.run.smithi119.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-27T09:04:54.877 INFO:teuthology.orchestra.run.smithi119.stderr:done 2024-05-27T09:04:54.879 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2024-05-27T09:04:54.896 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.7AWHRXWSW4 2024-05-27T09:04:54.896 DEBUG:teuthology.orchestra.run.smithi119:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.7AWHRXWSW4 2024-05-27T09:04:54.975 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.7AWHRXWSW4 2024-05-27T09:04:55.113 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.7AWHRXWSW4 is 6KB 2024-05-27T09:04:55.162 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.7AWHRXWSW4 2024-05-27T09:04:55.179 DEBUG:teuthology.orchestra.run.smithi119:> sudo /bin/ls /boot/loader/entries || true 2024-05-27T09:04:55.261 INFO:teuthology.orchestra.run.smithi119.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-27T09:04:55.261 INFO:teuthology.orchestra.run.smithi119.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-27T09:04:55.262 INFO:teuthology.orchestra.run.smithi119.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-27T09:04:55.262 INFO:teuthology.orchestra.run.smithi119.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-27T09:04:55.263 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-27T09:04:55.375 DEBUG:teuthology.orchestra.run.smithi119:> sudo shutdown -r now 2024-05-27T09:04:59.984 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi138.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-27T09:04:59.985 DEBUG:teuthology.orchestra.run.smithi138:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-27T09:05:00.072 INFO:teuthology.orchestra.run.smithi138.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-27T09:05:00.072 INFO:teuthology.orchestra.run.smithi138.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-27T09:05:00.073 INFO:teuthology.orchestra.run.smithi138.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-27T09:05:00.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-27T09:05:00.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-27T09:05:00.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-27T09:05:00.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-27T09:05:00.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-27T09:05:00.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-27T09:05:00.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-27T09:05:00.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-27T09:05:00.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-27T09:05:00.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-27T09:05:00.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-27T09:05:00.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-27T09:05:00.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-27T09:05:00.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-27T09:05:00.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-27T09:05:00.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-27T09:05:00.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-27T09:05:00.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-27T09:05:00.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-27T09:05:00.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-27T09:05:00.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-27T09:05:00.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-27T09:05:00.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-27T09:05:00.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-27T09:05:00.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-27T09:05:00.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-27T09:05:00.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-27T09:05:00.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-27T09:05:00.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-27T09:05:00.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-27T09:05:00.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-27T09:05:00.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-27T09:05:00.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-27T09:05:00.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-27T09:05:00.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-27T09:05:00.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-27T09:05:00.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-27T09:05:00.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-27T09:05:00.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-27T09:05:00.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-27T09:05:00.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-27T09:05:00.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-27T09:05:00.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-27T09:05:00.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-27T09:05:00.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-27T09:05:00.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-27T09:05:00.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-27T09:05:00.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-27T09:05:00.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-27T09:05:00.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-27T09:05:00.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-27T09:05:00.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-27T09:05:00.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-27T09:05:00.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-27T09:05:00.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-27T09:05:00.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-27T09:05:00.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-27T09:05:00.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-27T09:05:00.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-27T09:05:00.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-27T09:05:00.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-27T09:05:00.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-27T09:05:00.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-27T09:05:00.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-27T09:05:00.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-27T09:05:00.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-27T09:05:00.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-27T09:05:00.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-27T09:05:00.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-27T09:05:00.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-27T09:05:00.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-27T09:05:00.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-27T09:05:00.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-27T09:05:00.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-27T09:05:00.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-27T09:05:00.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-27T09:05:00.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-27T09:05:00.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-27T09:05:00.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-27T09:05:00.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-27T09:05:00.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-27T09:05:00.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-27T09:05:00.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-27T09:05:00.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-27T09:05:00.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-27T09:05:00.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-27T09:05:00.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-27T09:05:00.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-27T09:05:00.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-27T09:05:00.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-27T09:05:00.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-27T09:05:00.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-27T09:05:00.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-27T09:05:00.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-27T09:05:00.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-27T09:05:00.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-27T09:05:00.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-27T09:05:00.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-27T09:05:00.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-27T09:05:00.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-27T09:05:00.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-27T09:05:00.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-27T09:05:00.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-27T09:05:00.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-27T09:05:00.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-27T09:05:00.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-27T09:05:00.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-27T09:05:00.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-27T09:05:00.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-27T09:05:00.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-27T09:05:00.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-27T09:05:00.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-27T09:05:00.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-27T09:05:00.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-27T09:05:00.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-27T09:05:00.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-27T09:05:00.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-27T09:05:00.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-27T09:05:00.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-27T09:05:00.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-27T09:05:00.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-27T09:05:00.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-27T09:05:00.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-27T09:05:00.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-27T09:05:00.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-27T09:05:00.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-27T09:05:00.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-27T09:05:00.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-27T09:05:00.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-27T09:05:00.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-27T09:05:00.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-27T09:05:00.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-27T09:05:00.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-27T09:05:00.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-27T09:05:00.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-27T09:05:00.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-27T09:05:00.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-27T09:05:00.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-27T09:05:00.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-27T09:05:00.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-27T09:05:00.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-27T09:05:00.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-27T09:05:00.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-27T09:05:00.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-27T09:05:00.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-27T09:05:00.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-27T09:05:00.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-27T09:05:00.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-27T09:05:00.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-27T09:05:00.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-27T09:05:00.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-27T09:05:00.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-27T09:05:00.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-27T09:05:00.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-27T09:05:00.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-27T09:05:00.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-27T09:05:00.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-27T09:05:00.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-27T09:05:00.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-27T09:05:00.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-27T09:05:00.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-27T09:05:00.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-27T09:05:00.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-27T09:05:00.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-27T09:05:00.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-27T09:05:00.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-27T09:05:00.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-27T09:05:00.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-27T09:05:00.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-27T09:05:00.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-27T09:05:00.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-27T09:05:00.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-27T09:05:00.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-27T09:05:00.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-27T09:05:00.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-27T09:05:00.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-27T09:05:00.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-27T09:05:00.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-27T09:05:00.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-27T09:05:00.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-27T09:05:00.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-27T09:05:00.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-27T09:05:00.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-27T09:05:00.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-27T09:05:00.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-27T09:05:00.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-27T09:05:00.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-27T09:05:00.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-27T09:05:00.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-27T09:05:00.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-27T09:05:00.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-27T09:05:00.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-27T09:05:00.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-27T09:05:00.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-27T09:05:00.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-27T09:05:00.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-27T09:05:00.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-27T09:05:00.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-27T09:05:00.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-27T09:05:00.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-27T09:05:00.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-27T09:05:00.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-27T09:05:00.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-27T09:05:00.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-27T09:05:00.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-27T09:05:00.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-27T09:05:00.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-27T09:05:00.090 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-27T09:05:00.090 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-27T09:05:00.090 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-27T09:05:00.090 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-27T09:05:00.090 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-27T09:05:00.090 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-27T09:05:00.090 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-27T09:05:00.090 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-27T09:05:00.090 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-27T09:05:00.091 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-27T09:05:00.091 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-27T09:05:00.091 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-27T09:05:00.091 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-27T09:05:00.091 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-27T09:05:00.091 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-27T09:05:00.091 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-27T09:05:00.092 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-27T09:05:00.092 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-27T09:05:00.092 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-27T09:05:00.092 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-27T09:05:00.092 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-27T09:05:00.092 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-27T09:05:00.092 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-27T09:05:00.092 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-27T09:05:00.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-27T09:05:00.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-27T09:05:00.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-27T09:05:00.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-27T09:05:00.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-27T09:05:00.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-27T09:05:00.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-27T09:05:00.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-27T09:05:00.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-27T09:05:00.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-27T09:05:00.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-27T09:05:00.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-27T09:05:00.094 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-27T09:05:00.094 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-27T09:05:00.094 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-27T09:05:00.094 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-27T09:05:00.094 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-27T09:05:00.094 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-27T09:05:00.094 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-27T09:05:00.094 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-27T09:05:00.094 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-27T09:05:00.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-27T09:05:00.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-27T09:05:00.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-27T09:05:00.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-27T09:05:00.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-27T09:05:00.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-27T09:05:00.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-27T09:05:00.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-27T09:05:00.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-27T09:05:00.096 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-27T09:05:00.096 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-27T09:05:00.096 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-27T09:05:00.096 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-27T09:05:00.096 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-27T09:05:00.096 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-27T09:05:00.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-27T09:05:00.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-27T09:05:00.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-27T09:05:00.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-27T09:05:00.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-27T09:05:00.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-27T09:05:00.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-27T09:05:00.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-27T09:05:00.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-27T09:05:00.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-27T09:05:00.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-27T09:05:00.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-27T09:05:00.098 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-27T09:05:00.098 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-27T09:05:00.098 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-27T09:05:00.098 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-27T09:05:00.098 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-27T09:05:00.098 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-27T09:05:00.098 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-27T09:05:00.098 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-27T09:05:00.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-27T09:05:00.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-27T09:05:00.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-27T09:05:00.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-27T09:05:00.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-27T09:05:00.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-27T09:05:00.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-27T09:05:00.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-27T09:05:00.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-27T09:05:00.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-27T09:05:00.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-27T09:05:00.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-27T09:05:00.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-27T09:05:00.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-27T09:05:00.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-27T09:05:00.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-27T09:05:00.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-27T09:05:00.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-27T09:05:00.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-27T09:05:00.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-27T09:05:00.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-27T09:05:00.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-27T09:05:00.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-27T09:05:00.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-27T09:05:00.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-27T09:05:00.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-27T09:05:00.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-27T09:05:00.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-27T09:05:00.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-27T09:05:00.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-27T09:05:00.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-27T09:05:00.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-27T09:05:00.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-27T09:05:00.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-27T09:05:00.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-27T09:05:00.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-27T09:05:00.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-27T09:05:00.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-27T09:05:00.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-27T09:05:00.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-27T09:05:00.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-27T09:05:00.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-27T09:05:00.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-27T09:05:00.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-27T09:05:00.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-27T09:05:00.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-27T09:05:00.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-27T09:05:00.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-27T09:05:00.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-27T09:05:00.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-27T09:05:00.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-27T09:05:00.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-27T09:05:00.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-27T09:05:00.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-27T09:05:00.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-27T09:05:00.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-27T09:05:00.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-27T09:05:00.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-27T09:05:00.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-27T09:05:00.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-27T09:05:00.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-27T09:05:00.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-27T09:05:00.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-27T09:05:00.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-27T09:05:00.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-27T09:05:00.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-27T09:05:00.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-27T09:05:00.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-27T09:05:00.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-27T09:05:00.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-27T09:05:00.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-27T09:05:00.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-27T09:05:00.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-27T09:05:00.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-27T09:05:00.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-27T09:05:00.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-27T09:05:00.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-27T09:05:00.106 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-27T09:05:00.106 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-27T09:05:00.106 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-27T09:05:00.106 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-27T09:05:00.106 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-27T09:05:00.106 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-27T09:05:00.106 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-27T09:05:00.106 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-27T09:05:00.106 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-27T09:05:00.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-27T09:05:00.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-27T09:05:00.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-27T09:05:00.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-27T09:05:00.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-27T09:05:00.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-27T09:05:00.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-27T09:05:00.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-27T09:05:00.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-27T09:05:00.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-27T09:05:00.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-27T09:05:00.108 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-27T09:05:00.108 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-27T09:05:00.108 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-27T09:05:00.108 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-27T09:05:00.108 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-27T09:05:00.108 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-27T09:05:00.108 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-27T09:05:00.108 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-27T09:05:00.109 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-27T09:05:00.109 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-27T09:05:00.109 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-27T09:05:00.109 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-27T09:05:00.109 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-27T09:05:00.109 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-27T09:05:00.109 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-27T09:05:00.109 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-27T09:05:00.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-27T09:05:00.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-27T09:05:00.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-27T09:05:00.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-27T09:05:00.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-27T09:05:00.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-27T09:05:00.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-27T09:05:00.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-27T09:05:00.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-27T09:05:00.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-27T09:05:00.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-27T09:05:00.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-27T09:05:00.111 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-27T09:05:00.111 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-27T09:05:00.111 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-27T09:05:00.111 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-27T09:05:00.111 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-27T09:05:00.111 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-27T09:05:00.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-27T09:05:00.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-27T09:05:00.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-27T09:05:00.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-27T09:05:00.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-27T09:05:00.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-27T09:05:00.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-27T09:05:00.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-27T09:05:00.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-27T09:05:00.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-27T09:05:00.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-27T09:05:00.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-27T09:05:00.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-27T09:05:00.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-27T09:05:00.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-27T09:05:00.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-27T09:05:00.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-27T09:05:00.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-27T09:05:00.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-27T09:05:00.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-27T09:05:00.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-27T09:05:00.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-27T09:05:00.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-27T09:05:00.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-27T09:05:00.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-27T09:05:00.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-27T09:05:00.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-27T09:05:00.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-27T09:05:00.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-27T09:05:00.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-27T09:05:00.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-27T09:05:00.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-27T09:05:00.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-27T09:05:00.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-27T09:05:00.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-27T09:05:00.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-27T09:05:00.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-27T09:05:00.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-27T09:05:00.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-27T09:05:00.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-27T09:05:00.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-27T09:05:00.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-27T09:05:00.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-27T09:05:00.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-27T09:05:00.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-27T09:05:00.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-27T09:05:00.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-27T09:05:00.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-27T09:05:00.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-27T09:05:00.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-27T09:05:00.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-27T09:05:00.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-27T09:05:00.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-27T09:05:00.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-27T09:05:00.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-27T09:05:00.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-27T09:05:00.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-27T09:05:00.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-27T09:05:00.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-27T09:05:00.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-27T09:05:00.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-27T09:05:00.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-27T09:05:00.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-27T09:05:00.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-27T09:05:00.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-27T09:05:00.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-27T09:05:00.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-27T09:05:00.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-27T09:05:00.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-27T09:05:00.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-27T09:05:00.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-27T09:05:00.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-27T09:05:00.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-27T09:05:00.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-27T09:05:00.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-27T09:05:00.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-27T09:05:00.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-27T09:05:00.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-27T09:05:00.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-27T09:05:00.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-27T09:05:00.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-27T09:05:00.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-27T09:05:00.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-27T09:05:00.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-27T09:05:00.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-27T09:05:00.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-27T09:05:00.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-27T09:05:00.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-27T09:05:00.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-27T09:05:00.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-27T09:05:00.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-27T09:05:00.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-27T09:05:00.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-27T09:05:00.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-27T09:05:00.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-27T09:05:00.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-27T09:05:00.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-27T09:05:00.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-27T09:05:00.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-27T09:05:00.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-27T09:05:00.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-27T09:05:00.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-27T09:05:00.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-27T09:05:00.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-27T09:05:00.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-27T09:05:00.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-27T09:05:00.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-27T09:05:00.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-27T09:05:00.122 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-27T09:05:00.122 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-27T09:05:00.122 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-27T09:05:00.122 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-27T09:05:00.122 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-27T09:05:00.122 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-27T09:05:00.123 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-27T09:05:00.123 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-27T09:05:00.123 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-27T09:05:00.123 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-27T09:05:00.123 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-27T09:05:00.123 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-27T09:05:00.123 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-27T09:05:00.123 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-27T09:05:00.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-27T09:05:00.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-27T09:05:00.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-27T09:05:00.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-27T09:05:00.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-27T09:05:00.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-27T09:05:00.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-27T09:05:00.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-27T09:05:00.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-27T09:05:00.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-27T09:05:00.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-27T09:05:00.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-27T09:05:00.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-27T09:05:00.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-27T09:05:00.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-27T09:05:00.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-27T09:05:00.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-27T09:05:00.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-27T09:05:00.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-27T09:05:00.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-27T09:05:00.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-27T09:05:00.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-27T09:05:00.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-27T09:05:00.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-27T09:05:00.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-27T09:05:00.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-27T09:05:00.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-27T09:05:00.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-27T09:05:00.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-27T09:05:00.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-27T09:05:00.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-27T09:05:00.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-27T09:05:00.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-27T09:05:00.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-27T09:05:00.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-27T09:05:00.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-27T09:05:00.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-27T09:05:00.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-27T09:05:00.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-27T09:05:00.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-27T09:05:00.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-27T09:05:00.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-27T09:05:00.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-27T09:05:00.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-27T09:05:00.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-27T09:05:00.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-27T09:05:00.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-27T09:05:00.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-27T09:05:00.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-27T09:05:00.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-27T09:05:00.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-27T09:05:00.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-27T09:05:00.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-27T09:05:00.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-27T09:05:00.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-27T09:05:00.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-27T09:05:00.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-27T09:05:00.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-27T09:05:00.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-27T09:05:00.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-27T09:05:00.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-27T09:05:00.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-27T09:05:00.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-27T09:05:00.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-27T09:05:00.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-27T09:05:00.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-27T09:05:00.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-27T09:05:00.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-27T09:05:00.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-27T09:05:00.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-27T09:05:00.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-27T09:05:00.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-27T09:05:00.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-27T09:05:00.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-27T09:05:00.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-27T09:05:00.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-27T09:05:00.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-27T09:05:00.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-27T09:05:00.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-27T09:05:00.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-27T09:05:00.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-27T09:05:00.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-27T09:05:00.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-27T09:05:00.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-27T09:05:00.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-27T09:05:00.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-27T09:05:00.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-27T09:05:00.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-27T09:05:00.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-27T09:05:00.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-27T09:05:00.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-27T09:05:00.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-27T09:05:00.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-27T09:05:00.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-27T09:05:00.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-27T09:05:00.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-27T09:05:00.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-27T09:05:00.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-27T09:05:00.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-27T09:05:00.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-27T09:05:00.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-27T09:05:00.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-27T09:05:00.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-27T09:05:00.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-27T09:05:00.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-27T09:05:00.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-27T09:05:00.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-27T09:05:00.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-27T09:05:00.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-27T09:05:00.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-27T09:05:00.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-27T09:05:00.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-27T09:05:00.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-27T09:05:00.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-27T09:05:00.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-27T09:05:00.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-27T09:05:00.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-27T09:05:00.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-27T09:05:00.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-27T09:05:00.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-27T09:05:00.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-27T09:05:00.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-27T09:05:00.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-27T09:05:00.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-27T09:05:00.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-27T09:05:00.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-27T09:05:00.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-27T09:05:00.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-27T09:05:00.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-27T09:05:00.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-27T09:05:00.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-27T09:05:00.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-27T09:05:00.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-27T09:05:00.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-27T09:05:00.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-27T09:05:00.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-27T09:05:00.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-27T09:05:00.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-27T09:05:00.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-27T09:05:00.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-27T09:05:00.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-27T09:05:00.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-27T09:05:00.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-27T09:05:00.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-27T09:05:00.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-27T09:05:00.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-27T09:05:00.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-27T09:05:00.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-27T09:05:00.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-27T09:05:00.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-27T09:05:00.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-27T09:05:00.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-27T09:05:00.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-27T09:05:00.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-27T09:05:00.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-27T09:05:00.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-27T09:05:00.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-27T09:05:00.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-27T09:05:00.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-27T09:05:00.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-27T09:05:00.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-27T09:05:00.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-27T09:05:00.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-27T09:05:00.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-27T09:05:00.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-27T09:05:00.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-27T09:05:00.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-27T09:05:00.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-27T09:05:00.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-27T09:05:00.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-27T09:05:00.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-27T09:05:00.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-27T09:05:00.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-27T09:05:00.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-27T09:05:00.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-27T09:05:00.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-27T09:05:00.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-27T09:05:00.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-27T09:05:00.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-27T09:05:00.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-27T09:05:00.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-27T09:05:00.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-27T09:05:00.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-27T09:05:00.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-27T09:05:00.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-27T09:05:00.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-27T09:05:00.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-27T09:05:00.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-27T09:05:00.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-27T09:05:00.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-27T09:05:00.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-27T09:05:00.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-27T09:05:00.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-27T09:05:00.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-27T09:05:00.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-27T09:05:00.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-27T09:05:00.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-27T09:05:00.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-27T09:05:00.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-27T09:05:00.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-27T09:05:00.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-27T09:05:00.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-27T09:05:00.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-27T09:05:00.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-27T09:05:00.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-27T09:05:00.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-27T09:05:00.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-27T09:05:00.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-27T09:05:00.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-27T09:05:00.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-27T09:05:00.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-27T09:05:00.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-27T09:05:00.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-27T09:05:00.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-27T09:05:00.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-27T09:05:00.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-27T09:05:00.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-27T09:05:00.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-27T09:05:00.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-27T09:05:00.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-27T09:05:00.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-27T09:05:00.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-27T09:05:00.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-27T09:05:00.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-27T09:05:00.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-27T09:05:00.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-27T09:05:00.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-27T09:05:00.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-27T09:05:00.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-27T09:05:00.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-27T09:05:00.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-27T09:05:00.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-27T09:05:00.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-27T09:05:00.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-27T09:05:00.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-27T09:05:00.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-27T09:05:00.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-27T09:05:00.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-27T09:05:00.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-27T09:05:00.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-27T09:05:00.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-27T09:05:00.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-27T09:05:00.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-27T09:05:00.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-27T09:05:00.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-27T09:05:00.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-27T09:05:00.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-27T09:05:00.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-27T09:05:00.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-27T09:05:00.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-27T09:05:00.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-27T09:05:00.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-27T09:05:00.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-27T09:05:00.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-27T09:05:00.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-27T09:05:00.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-27T09:05:00.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-27T09:05:00.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-27T09:05:00.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-27T09:05:00.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-27T09:05:00.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-27T09:05:00.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-27T09:05:00.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-27T09:05:00.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-27T09:05:00.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-27T09:05:00.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-27T09:05:00.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-27T09:05:00.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-27T09:05:00.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-27T09:05:00.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-27T09:05:00.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-27T09:05:00.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-27T09:05:00.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-27T09:05:00.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-27T09:05:00.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-27T09:05:00.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-27T09:05:00.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-27T09:05:00.146 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-27T09:05:00.146 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-27T09:05:00.146 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-27T09:05:00.146 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-27T09:05:00.146 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-27T09:05:00.146 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-27T09:05:00.146 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-27T09:05:00.146 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-27T09:05:00.147 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-27T09:05:00.147 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-27T09:05:00.147 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-27T09:05:00.147 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-27T09:05:00.147 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-27T09:05:00.147 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-27T09:05:00.147 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-27T09:05:00.147 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-27T09:05:00.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-27T09:05:00.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-27T09:05:00.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-27T09:05:00.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-27T09:05:00.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-27T09:05:00.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-27T09:05:00.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-27T09:05:00.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-27T09:05:00.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-27T09:05:00.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-27T09:05:00.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-27T09:05:00.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-27T09:05:00.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-27T09:05:00.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-27T09:05:00.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-27T09:05:00.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-27T09:05:00.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-27T09:05:00.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-27T09:05:00.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-27T09:05:00.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-27T09:05:00.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-27T09:05:00.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-27T09:05:00.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-27T09:05:00.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-27T09:05:00.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-27T09:05:00.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-27T09:05:00.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-27T09:05:00.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-27T09:05:00.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-27T09:05:00.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-27T09:05:00.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-27T09:05:00.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-27T09:05:00.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-27T09:05:00.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-27T09:05:00.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-27T09:05:00.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-27T09:05:00.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-27T09:05:00.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-27T09:05:00.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-27T09:05:00.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-27T09:05:00.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-27T09:05:00.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-27T09:05:00.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-27T09:05:00.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-27T09:05:00.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-27T09:05:00.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-27T09:05:00.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-27T09:05:00.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-27T09:05:00.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-27T09:05:00.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-27T09:05:00.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-27T09:05:00.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-27T09:05:00.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-27T09:05:00.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-27T09:05:00.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-27T09:05:00.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-27T09:05:00.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-27T09:05:00.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-27T09:05:00.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-27T09:05:00.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-27T09:05:00.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-27T09:05:00.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-27T09:05:00.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-27T09:05:00.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-27T09:05:00.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-27T09:05:00.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-27T09:05:00.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-27T09:05:00.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-27T09:05:00.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-27T09:05:00.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-27T09:05:00.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-27T09:05:00.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-27T09:05:00.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-27T09:05:00.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-27T09:05:00.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-27T09:05:00.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-27T09:05:00.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-27T09:05:00.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-27T09:05:00.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-27T09:05:00.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-27T09:05:00.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-27T09:05:00.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-27T09:05:00.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-27T09:05:00.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-27T09:05:00.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-27T09:05:00.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-27T09:05:00.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-27T09:05:00.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-27T09:05:00.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-27T09:05:00.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-27T09:05:00.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-27T09:05:00.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-27T09:05:00.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-27T09:05:00.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-27T09:05:00.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-27T09:05:00.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-27T09:05:00.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-27T09:05:00.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-27T09:05:00.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-27T09:05:00.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-27T09:05:00.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-27T09:05:00.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-27T09:05:00.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-27T09:05:00.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-27T09:05:00.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-27T09:05:00.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-27T09:05:00.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-27T09:05:00.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-27T09:05:00.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-27T09:05:00.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-27T09:05:00.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-27T09:05:00.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-27T09:05:00.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-27T09:05:00.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-27T09:05:00.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-27T09:05:00.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-27T09:05:00.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-27T09:05:00.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-27T09:05:00.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-27T09:05:00.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-27T09:05:00.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-27T09:05:00.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-27T09:05:00.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-27T09:05:00.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-27T09:05:00.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-27T09:05:00.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-27T09:05:00.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-27T09:05:00.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-27T09:05:00.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-27T09:05:00.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-27T09:05:00.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-27T09:05:00.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-27T09:05:00.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-27T09:05:00.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-27T09:05:00.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-27T09:05:00.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-27T09:05:00.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-27T09:05:00.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-27T09:05:00.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-27T09:05:00.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-27T09:05:00.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-27T09:05:00.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-27T09:05:00.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-27T09:05:00.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-27T09:05:00.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-27T09:05:00.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-27T09:05:00.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-27T09:05:00.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-27T09:05:00.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-27T09:05:00.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-27T09:05:00.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-27T09:05:00.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-27T09:05:00.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-27T09:05:00.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-27T09:05:00.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-27T09:05:00.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-27T09:05:00.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-27T09:05:00.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-27T09:05:00.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-27T09:05:00.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-27T09:05:00.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-27T09:05:00.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-27T09:05:00.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-27T09:05:00.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-27T09:05:00.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-27T09:05:00.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-27T09:05:00.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-27T09:05:00.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-27T09:05:00.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-27T09:05:00.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-27T09:05:00.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-27T09:05:00.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-27T09:05:00.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-27T09:05:00.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-27T09:05:00.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-27T09:05:00.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-27T09:05:00.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-27T09:05:00.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-27T09:05:00.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-27T09:05:00.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-27T09:05:00.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-27T09:05:00.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-27T09:05:00.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-27T09:05:00.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-27T09:05:00.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-27T09:05:00.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-27T09:05:00.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-27T09:05:00.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-27T09:05:00.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-27T09:05:00.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-27T09:05:00.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-27T09:05:00.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-27T09:05:00.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-27T09:05:00.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-27T09:05:00.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-27T09:05:00.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-27T09:05:00.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-27T09:05:00.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-27T09:05:00.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-27T09:05:00.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-27T09:05:00.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-27T09:05:00.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-27T09:05:00.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-27T09:05:00.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-27T09:05:00.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-27T09:05:00.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-27T09:05:00.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-27T09:05:00.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-27T09:05:00.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-27T09:05:00.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-27T09:05:00.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-27T09:05:00.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-27T09:05:00.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-27T09:05:00.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-27T09:05:00.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-27T09:05:00.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-27T09:05:00.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-27T09:05:00.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-27T09:05:00.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-27T09:05:00.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-27T09:05:00.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-27T09:05:00.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-27T09:05:00.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-27T09:05:00.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-27T09:05:00.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-27T09:05:00.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-27T09:05:00.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-27T09:05:00.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-27T09:05:00.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-27T09:05:00.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-27T09:05:00.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-27T09:05:00.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-27T09:05:00.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-27T09:05:00.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-27T09:05:00.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-27T09:05:00.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-27T09:05:00.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-27T09:05:00.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-27T09:05:00.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-27T09:05:00.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-27T09:05:00.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-27T09:05:00.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-27T09:05:00.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-27T09:05:00.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-27T09:05:00.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-27T09:05:00.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-27T09:05:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-27T09:05:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-27T09:05:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-27T09:05:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-27T09:05:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-27T09:05:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-27T09:05:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-27T09:05:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-27T09:05:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-27T09:05:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-27T09:05:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-27T09:05:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-27T09:05:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-27T09:05:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-27T09:05:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-27T09:05:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-27T09:05:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-27T09:05:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-27T09:05:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-27T09:05:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-27T09:05:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-27T09:05:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-27T09:05:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-27T09:05:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-27T09:05:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-27T09:05:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-27T09:05:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-27T09:05:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-27T09:05:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-27T09:05:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-27T09:05:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-27T09:05:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-27T09:05:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-27T09:05:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-27T09:05:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-27T09:05:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-27T09:05:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-27T09:05:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-27T09:05:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-27T09:05:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-27T09:05:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-27T09:05:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-27T09:05:00.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-27T09:05:00.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-27T09:05:00.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-27T09:05:00.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-27T09:05:00.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-27T09:05:00.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-27T09:05:00.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-27T09:05:00.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-27T09:05:00.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-27T09:05:00.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-27T09:05:00.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-27T09:05:00.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-27T09:05:00.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-27T09:05:00.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-27T09:05:00.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-27T09:05:00.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-27T09:05:00.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-27T09:05:00.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-27T09:05:00.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-27T09:05:00.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-27T09:05:00.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-27T09:05:00.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-27T09:05:00.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-27T09:05:00.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-27T09:05:00.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-27T09:05:00.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-27T09:05:00.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-27T09:05:00.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-27T09:05:00.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-27T09:05:00.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-27T09:05:00.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-27T09:05:00.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-27T09:05:00.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-27T09:05:00.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-27T09:05:00.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-27T09:05:00.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-27T09:05:00.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-27T09:05:00.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-27T09:05:00.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-27T09:05:00.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-27T09:05:00.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-27T09:05:00.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-27T09:05:00.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-27T09:05:00.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-27T09:05:00.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-27T09:05:00.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-27T09:05:00.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-27T09:05:00.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-27T09:05:00.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-27T09:05:00.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-27T09:05:00.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-27T09:05:00.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-27T09:05:00.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-27T09:05:00.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-27T09:05:00.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-27T09:05:00.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-27T09:05:00.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-27T09:05:00.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-27T09:05:00.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-27T09:05:00.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-27T09:05:00.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-27T09:05:00.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-27T09:05:00.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-27T09:05:00.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-27T09:05:00.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-27T09:05:00.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-27T09:05:00.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-27T09:05:00.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-27T09:05:00.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-27T09:05:00.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-27T09:05:00.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-27T09:05:00.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-27T09:05:00.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-27T09:05:00.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-27T09:05:00.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-27T09:05:00.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-27T09:05:00.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-27T09:05:00.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-27T09:05:00.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-27T09:05:00.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-27T09:05:00.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-27T09:05:00.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-27T09:05:00.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-27T09:05:00.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-27T09:05:00.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-27T09:05:00.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-27T09:05:00.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-27T09:05:00.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-27T09:05:00.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-27T09:05:00.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-27T09:05:00.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-27T09:05:00.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-27T09:05:00.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-27T09:05:00.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-27T09:05:00.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-27T09:05:00.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-27T09:05:00.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-27T09:05:00.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-27T09:05:00.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-27T09:05:00.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-27T09:05:00.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-27T09:05:00.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-27T09:05:00.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-27T09:05:00.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-27T09:05:00.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-27T09:05:00.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-27T09:05:00.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-27T09:05:00.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-27T09:05:00.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-27T09:05:00.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-27T09:05:00.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-27T09:05:00.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-27T09:05:00.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-27T09:05:00.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-27T09:05:00.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-27T09:05:00.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-27T09:05:00.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-27T09:05:00.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-27T09:05:00.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-27T09:05:00.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-27T09:05:00.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-27T09:05:00.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-27T09:05:00.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-27T09:05:00.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-27T09:05:00.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-27T09:05:00.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-27T09:05:00.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-27T09:05:00.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-27T09:05:00.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-27T09:05:00.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-27T09:05:00.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-27T09:05:00.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-27T09:05:00.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-27T09:05:00.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-27T09:05:00.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-27T09:05:00.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-27T09:05:00.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-27T09:05:00.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-27T09:05:00.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-27T09:05:00.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-27T09:05:00.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-27T09:05:00.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-27T09:05:00.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-27T09:05:00.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-27T09:05:00.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-27T09:05:00.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-27T09:05:00.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-27T09:05:00.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-27T09:05:00.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-27T09:05:00.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-27T09:05:00.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-27T09:05:00.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-27T09:05:00.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-27T09:05:00.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-27T09:05:00.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-27T09:05:00.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-27T09:05:00.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-27T09:05:00.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-27T09:05:00.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-27T09:05:00.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-27T09:05:00.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-27T09:05:00.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-27T09:05:00.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-27T09:05:00.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-27T09:05:00.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-27T09:05:00.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-27T09:05:00.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-27T09:05:00.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-27T09:05:00.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-27T09:05:00.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-27T09:05:00.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-27T09:05:00.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-27T09:05:00.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-27T09:05:00.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-27T09:05:00.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-27T09:05:00.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-27T09:05:00.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-27T09:05:00.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-27T09:05:00.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-27T09:05:00.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-27T09:05:00.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-27T09:05:00.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-27T09:05:00.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-27T09:05:00.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-27T09:05:00.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-27T09:05:00.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-27T09:05:00.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-27T09:05:00.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-27T09:05:00.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-27T09:05:00.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-27T09:05:00.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-27T09:05:00.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-27T09:05:00.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-27T09:05:00.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-27T09:05:00.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-27T09:05:00.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-27T09:05:00.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-27T09:05:00.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-27T09:05:00.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-27T09:05:00.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-27T09:05:00.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-27T09:05:00.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-27T09:05:00.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-27T09:05:00.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-27T09:05:00.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-27T09:05:00.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-27T09:05:00.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-27T09:05:00.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-27T09:05:00.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-27T09:05:00.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-27T09:05:00.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-27T09:05:00.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-27T09:05:00.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-27T09:05:00.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-27T09:05:00.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-27T09:05:00.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-27T09:05:00.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-27T09:05:00.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-27T09:05:00.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-27T09:05:00.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-27T09:05:00.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-27T09:05:00.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-27T09:05:00.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-27T09:05:00.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-27T09:05:00.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-27T09:05:00.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-27T09:05:00.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-27T09:05:00.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-27T09:05:00.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-27T09:05:00.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-27T09:05:00.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-27T09:05:00.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-27T09:05:00.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-27T09:05:00.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-27T09:05:00.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-27T09:05:00.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-27T09:05:00.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-27T09:05:00.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-27T09:05:00.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-27T09:05:00.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-27T09:05:00.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-27T09:05:00.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-27T09:05:00.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-27T09:05:00.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-27T09:05:00.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-27T09:05:00.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-27T09:05:00.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-27T09:05:00.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-27T09:05:00.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-27T09:05:00.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-27T09:05:00.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-27T09:05:00.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-27T09:05:00.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-27T09:05:00.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-27T09:05:00.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-27T09:05:00.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-27T09:05:00.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-27T09:05:00.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-27T09:05:00.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-27T09:05:00.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-27T09:05:00.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-27T09:05:00.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-27T09:05:00.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-27T09:05:00.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-27T09:05:00.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-27T09:05:00.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-27T09:05:00.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-27T09:05:00.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-27T09:05:00.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-27T09:05:00.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-27T09:05:00.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-27T09:05:00.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-27T09:05:00.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-27T09:05:00.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-27T09:05:00.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-27T09:05:00.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-27T09:05:00.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-27T09:05:00.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-27T09:05:00.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-27T09:05:00.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-27T09:05:00.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-27T09:05:00.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-27T09:05:00.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-27T09:05:00.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-27T09:05:00.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-27T09:05:00.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-27T09:05:00.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-27T09:05:00.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-27T09:05:00.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-27T09:05:00.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-27T09:05:00.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-27T09:05:00.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-27T09:05:00.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-27T09:05:00.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-27T09:05:00.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-27T09:05:00.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-27T09:05:00.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-27T09:05:00.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-27T09:05:00.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-27T09:05:00.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-27T09:05:00.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-27T09:05:00.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-27T09:05:00.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-27T09:05:00.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-27T09:05:00.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-27T09:05:00.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-27T09:05:00.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-27T09:05:00.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-27T09:05:00.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-27T09:05:00.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-27T09:05:00.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-27T09:05:00.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-27T09:05:00.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-27T09:05:00.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-27T09:05:00.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-27T09:05:00.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-27T09:05:00.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-27T09:05:00.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-27T09:05:00.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-27T09:05:00.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-27T09:05:00.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-27T09:05:00.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-27T09:05:00.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-27T09:05:00.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-27T09:05:00.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-27T09:05:00.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-27T09:05:00.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-27T09:05:00.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-27T09:05:00.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-27T09:05:00.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-27T09:05:00.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-27T09:05:00.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-27T09:05:00.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-27T09:05:00.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-27T09:05:00.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-27T09:05:00.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-27T09:05:00.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-27T09:05:00.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-27T09:05:00.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-27T09:05:00.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-27T09:05:00.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-27T09:05:00.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-27T09:05:00.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-27T09:05:00.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-27T09:05:00.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-27T09:05:00.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-27T09:05:00.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-27T09:05:00.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-27T09:05:00.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-27T09:05:00.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-27T09:05:00.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-27T09:05:00.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-27T09:05:00.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-27T09:05:00.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-27T09:05:00.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-27T09:05:00.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-27T09:05:00.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-27T09:05:00.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-27T09:05:00.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-27T09:05:00.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-27T09:05:00.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-27T09:05:00.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-27T09:05:00.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-27T09:05:00.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-27T09:05:00.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-27T09:05:00.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-27T09:05:00.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-27T09:05:00.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-27T09:05:00.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-27T09:05:00.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-27T09:05:00.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-27T09:05:00.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-27T09:05:00.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-27T09:05:00.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-27T09:05:00.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-27T09:05:00.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-27T09:05:00.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-27T09:05:00.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-27T09:05:00.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-27T09:05:00.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-27T09:05:00.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-27T09:05:00.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-27T09:05:00.196 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-27T09:05:00.196 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-27T09:05:00.196 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-27T09:05:00.196 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-27T09:05:00.196 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-27T09:05:00.196 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-27T09:05:00.196 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-27T09:05:00.196 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-27T09:05:00.196 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-27T09:05:00.196 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-27T09:05:00.196 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-27T09:05:00.196 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-27T09:05:00.196 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-27T09:05:00.197 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-27T09:05:00.197 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-27T09:05:00.197 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-27T09:05:00.197 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-27T09:05:00.197 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-27T09:05:00.197 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-27T09:05:00.197 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-27T09:05:00.197 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-27T09:05:00.197 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-27T09:05:00.197 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-27T09:05:00.197 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-27T09:05:00.197 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-27T09:05:00.197 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-27T09:05:00.197 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-27T09:05:00.197 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-27T09:05:00.198 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-27T09:05:00.198 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-27T09:05:00.198 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-27T09:05:00.198 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-27T09:05:00.198 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-27T09:05:00.198 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-27T09:05:00.198 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-27T09:05:00.198 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-27T09:05:00.198 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-27T09:05:00.198 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-27T09:05:00.198 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-27T09:05:00.198 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-27T09:05:00.198 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-27T09:05:00.199 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-27T09:05:00.199 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-27T09:05:00.199 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-27T09:05:00.199 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-27T09:05:00.199 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-27T09:05:00.199 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-27T09:05:00.199 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-27T09:05:00.199 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-27T09:05:00.199 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-27T09:05:00.199 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-27T09:05:00.199 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-27T09:05:00.199 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-27T09:05:00.199 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-27T09:05:00.199 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-27T09:05:00.200 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-27T09:05:00.200 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-27T09:05:00.200 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-27T09:05:00.200 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-27T09:05:00.200 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-27T09:05:00.200 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-27T09:05:00.200 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-27T09:05:00.200 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-27T09:05:00.200 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-27T09:05:00.200 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-27T09:05:00.200 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-27T09:05:00.200 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-27T09:05:00.200 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-27T09:05:00.200 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-27T09:05:00.200 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-27T09:05:00.201 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-27T09:05:00.201 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-27T09:05:00.201 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-27T09:05:00.201 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-27T09:05:00.201 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-27T09:05:00.201 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-27T09:05:00.201 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-27T09:05:00.201 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-27T09:05:00.201 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-27T09:05:00.201 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-27T09:05:00.201 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-27T09:05:00.201 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-27T09:05:00.201 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-27T09:05:00.201 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-27T09:05:00.201 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-27T09:05:00.202 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-27T09:05:00.202 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-27T09:05:00.202 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-27T09:05:00.202 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-27T09:05:00.202 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-27T09:05:00.202 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-27T09:05:00.202 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-27T09:05:00.202 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-27T09:05:00.202 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-27T09:05:00.202 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-27T09:05:00.202 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-27T09:05:00.202 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-27T09:05:00.202 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-27T09:05:00.203 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-27T09:05:00.203 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-27T09:05:00.203 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-27T09:05:00.203 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-27T09:05:00.203 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-27T09:05:00.203 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-27T09:05:00.203 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-27T09:05:00.203 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-27T09:05:00.203 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-27T09:05:00.203 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-27T09:05:00.203 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-27T09:05:00.203 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-27T09:05:00.203 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-27T09:05:00.203 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-27T09:05:00.204 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-27T09:05:00.204 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-27T09:05:00.204 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-27T09:05:00.204 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-27T09:05:00.204 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-27T09:05:00.204 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-27T09:05:00.204 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-27T09:05:00.204 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-27T09:05:00.204 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-27T09:05:00.204 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-27T09:05:00.204 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-27T09:05:00.204 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-27T09:05:00.204 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-27T09:05:00.204 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-27T09:05:00.205 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-27T09:05:00.205 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-27T09:05:00.205 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-27T09:05:00.205 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-27T09:05:00.205 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-27T09:05:00.205 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-27T09:05:00.205 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-27T09:05:00.205 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-27T09:05:00.205 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-27T09:05:00.205 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-27T09:05:00.205 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-27T09:05:00.205 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-27T09:05:00.205 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-27T09:05:00.206 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-27T09:05:00.206 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-27T09:05:00.206 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-27T09:05:00.206 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-27T09:05:00.206 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-27T09:05:00.206 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-27T09:05:00.206 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-27T09:05:00.206 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-27T09:05:00.206 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-27T09:05:00.206 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-27T09:05:00.206 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-27T09:05:00.206 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-27T09:05:00.206 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-27T09:05:00.206 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-27T09:05:00.207 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-27T09:05:00.207 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-27T09:05:00.207 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-27T09:05:00.207 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-27T09:05:00.207 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-27T09:05:00.207 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-27T09:05:00.207 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-27T09:05:00.207 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-27T09:05:00.207 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-27T09:05:00.207 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-27T09:05:00.207 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-27T09:05:00.207 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-27T09:05:00.207 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-27T09:05:00.208 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-27T09:05:00.208 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-27T09:05:00.208 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-27T09:05:00.208 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-27T09:05:00.208 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-27T09:05:00.208 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-27T09:05:00.208 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-27T09:05:00.208 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-27T09:05:00.208 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-27T09:05:00.208 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-27T09:05:00.208 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-27T09:05:00.208 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-27T09:05:00.208 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-27T09:05:00.208 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-27T09:05:00.209 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-27T09:05:00.209 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-27T09:05:00.209 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-27T09:05:00.209 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-27T09:05:00.209 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-27T09:05:00.209 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-27T09:05:00.209 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-27T09:05:00.209 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-27T09:05:00.209 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-27T09:05:00.209 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-27T09:05:00.209 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-27T09:05:00.209 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-27T09:05:00.209 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-27T09:05:00.209 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-27T09:05:00.210 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-27T09:05:00.210 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-27T09:05:00.210 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-27T09:05:00.210 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-27T09:05:00.210 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-27T09:05:00.210 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-27T09:05:00.210 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-27T09:05:00.210 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-27T09:05:00.210 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-27T09:05:00.210 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-27T09:05:00.210 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-27T09:05:00.210 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-27T09:05:00.210 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-27T09:05:00.211 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-27T09:05:00.211 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-27T09:05:00.211 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-27T09:05:00.211 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-27T09:05:00.211 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-27T09:05:00.211 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-27T09:05:00.211 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-27T09:05:00.211 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-27T09:05:00.211 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-27T09:05:00.211 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-27T09:05:00.211 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-27T09:05:00.211 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-27T09:05:00.211 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-27T09:05:00.211 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-27T09:05:00.212 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-27T09:05:00.212 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-27T09:05:00.212 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-27T09:05:00.212 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-27T09:05:00.212 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-27T09:05:00.212 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-27T09:05:00.212 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-27T09:05:00.212 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-27T09:05:00.212 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-27T09:05:00.212 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-27T09:05:00.212 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-27T09:05:00.212 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-27T09:05:00.212 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-27T09:05:00.212 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-27T09:05:00.213 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-27T09:05:00.213 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-27T09:05:00.213 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-27T09:05:00.213 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-27T09:05:00.213 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-27T09:05:00.213 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-27T09:05:00.213 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-27T09:05:00.213 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-27T09:05:00.213 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-27T09:05:00.213 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-27T09:05:00.213 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-27T09:05:00.213 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-27T09:05:00.213 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-27T09:05:00.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-27T09:05:00.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-27T09:05:00.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-27T09:05:00.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-27T09:05:00.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-27T09:05:00.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-27T09:05:00.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-27T09:05:00.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-27T09:05:00.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-27T09:05:00.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-27T09:05:00.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-27T09:05:00.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-27T09:05:00.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-27T09:05:00.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-27T09:05:00.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-27T09:05:00.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-27T09:05:00.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-27T09:05:00.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-27T09:05:00.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-27T09:05:00.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-27T09:05:00.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-27T09:05:00.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-27T09:05:00.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-27T09:05:00.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-27T09:05:00.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-27T09:05:00.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-27T09:05:00.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-27T09:05:00.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-27T09:05:00.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-27T09:05:00.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-27T09:05:00.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-27T09:05:00.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-27T09:05:00.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-27T09:05:00.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-27T09:05:00.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-27T09:05:00.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-27T09:05:00.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-27T09:05:00.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-27T09:05:00.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-27T09:05:00.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-27T09:05:00.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-27T09:05:00.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-27T09:05:00.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-27T09:05:00.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-27T09:05:00.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-27T09:05:00.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-27T09:05:00.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-27T09:05:00.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-27T09:05:00.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-27T09:05:00.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-27T09:05:00.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-27T09:05:00.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-27T09:05:00.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-27T09:05:00.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-27T09:05:00.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-27T09:05:00.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-27T09:05:00.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-27T09:05:00.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-27T09:05:00.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-27T09:05:00.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-27T09:05:00.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-27T09:05:00.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-27T09:05:00.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-27T09:05:00.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-27T09:05:00.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-27T09:05:00.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-27T09:05:00.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-27T09:05:00.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-27T09:05:00.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-27T09:05:00.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-27T09:05:00.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-27T09:05:00.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-27T09:05:00.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-27T09:05:00.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-27T09:05:00.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-27T09:05:00.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-27T09:05:00.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-27T09:05:00.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-27T09:05:00.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-27T09:05:00.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-27T09:05:00.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-27T09:05:00.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-27T09:05:00.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-27T09:05:00.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-27T09:05:00.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-27T09:05:00.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-27T09:05:00.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-27T09:05:00.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-27T09:05:00.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-27T09:05:00.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-27T09:05:00.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-27T09:05:00.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-27T09:05:00.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-27T09:05:00.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-27T09:05:00.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-27T09:05:00.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-27T09:05:00.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-27T09:05:00.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-27T09:05:00.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-27T09:05:00.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-27T09:05:00.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-27T09:05:00.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-27T09:05:00.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-27T09:05:00.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-27T09:05:00.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-27T09:05:00.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-27T09:05:00.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-27T09:05:00.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-27T09:05:00.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-27T09:05:00.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-27T09:05:00.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-27T09:05:00.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-27T09:05:00.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-27T09:05:00.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-27T09:05:00.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-27T09:05:00.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-27T09:05:00.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-27T09:05:00.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-27T09:05:00.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-27T09:05:00.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-27T09:05:00.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-27T09:05:00.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-27T09:05:00.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-27T09:05:00.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-27T09:05:00.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-27T09:05:00.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-27T09:05:00.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-27T09:05:00.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-27T09:05:00.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-27T09:05:00.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-27T09:05:00.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-27T09:05:00.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-27T09:05:00.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-27T09:05:00.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-27T09:05:00.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-27T09:05:00.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-27T09:05:00.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-27T09:05:00.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-27T09:05:00.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-27T09:05:00.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-27T09:05:00.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-27T09:05:00.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-27T09:05:00.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-27T09:05:00.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-27T09:05:00.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-27T09:05:00.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-27T09:05:00.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-27T09:05:00.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-27T09:05:00.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-27T09:05:00.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-27T09:05:00.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-27T09:05:00.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-27T09:05:00.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-27T09:05:00.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-27T09:05:00.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-27T09:05:00.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-27T09:05:00.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-27T09:05:00.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-27T09:05:00.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-27T09:05:00.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-27T09:05:00.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-27T09:05:00.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-27T09:05:00.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-27T09:05:00.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-27T09:05:00.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-27T09:05:00.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-27T09:05:00.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-27T09:05:00.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-27T09:05:00.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-27T09:05:00.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-27T09:05:00.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-27T09:05:00.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-27T09:05:00.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-27T09:05:00.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-27T09:05:00.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-27T09:05:00.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-27T09:05:00.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-27T09:05:00.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-27T09:05:00.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-27T09:05:00.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-27T09:05:00.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-27T09:05:00.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-27T09:05:00.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-27T09:05:00.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-27T09:05:00.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-27T09:05:00.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-27T09:05:00.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-27T09:05:00.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-27T09:05:00.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-27T09:05:00.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-27T09:05:00.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-27T09:05:00.228 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-27T09:05:00.228 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T09:05:00.231 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-27T09:05:00.231 DEBUG:teuthology.orchestra.run.smithi138:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-27T09:05:00.256 INFO:teuthology.orchestra.run.smithi138.stdout:postinstall scriptlet (using /bin/sh): 2024-05-27T09:05:00.256 INFO:teuthology.orchestra.run.smithi138.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T09:05:00.256 INFO:teuthology.orchestra.run.smithi138.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T09:05:00.256 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-05-27T09:05:00.256 INFO:teuthology.orchestra.run.smithi138.stdout:for file in vmlinuz System.map config; do 2024-05-27T09:05:00.256 INFO:teuthology.orchestra.run.smithi138.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-27T09:05:00.256 INFO:teuthology.orchestra.run.smithi138.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-27T09:05:00.256 INFO:teuthology.orchestra.run.smithi138.stdout: fi 2024-05-27T09:05:00.256 INFO:teuthology.orchestra.run.smithi138.stdout:done 2024-05-27T09:05:00.256 INFO:teuthology.orchestra.run.smithi138.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-27T09:05:00.256 INFO:teuthology.orchestra.run.smithi138.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-27T09:05:00.257 INFO:teuthology.orchestra.run.smithi138.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T09:05:00.257 INFO:teuthology.orchestra.run.smithi138.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T09:05:00.257 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-05-27T09:05:00.257 INFO:teuthology.orchestra.run.smithi138.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-27T09:05:00.257 INFO:teuthology.orchestra.run.smithi138.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-27T09:05:00.257 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-05-27T09:05:00.257 INFO:teuthology.orchestra.run.smithi138.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-27T09:05:00.257 INFO:teuthology.orchestra.run.smithi138.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T09:05:00.257 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-27T09:05:00.257 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-05-27T09:05:00.258 DEBUG:teuthology.orchestra.run.smithi138:> sudo rpm -qi grub2-tools 2024-05-27T09:05:00.334 INFO:teuthology.orchestra.run.smithi138.stdout:Name : grub2-tools 2024-05-27T09:05:00.334 INFO:teuthology.orchestra.run.smithi138.stdout:Epoch : 1 2024-05-27T09:05:00.334 INFO:teuthology.orchestra.run.smithi138.stdout:Version : 2.06 2024-05-27T09:05:00.334 INFO:teuthology.orchestra.run.smithi138.stdout:Release : 68.el9 2024-05-27T09:05:00.334 INFO:teuthology.orchestra.run.smithi138.stdout:Architecture: x86_64 2024-05-27T09:05:00.334 INFO:teuthology.orchestra.run.smithi138.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-27T09:05:00.334 INFO:teuthology.orchestra.run.smithi138.stdout:Group : Unspecified 2024-05-27T09:05:00.334 INFO:teuthology.orchestra.run.smithi138.stdout:Size : 8264254 2024-05-27T09:05:00.334 INFO:teuthology.orchestra.run.smithi138.stdout:License : GPLv3+ 2024-05-27T09:05:00.335 INFO:teuthology.orchestra.run.smithi138.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-27T09:05:00.335 INFO:teuthology.orchestra.run.smithi138.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-27T09:05:00.335 INFO:teuthology.orchestra.run.smithi138.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-27T09:05:00.335 INFO:teuthology.orchestra.run.smithi138.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-27T09:05:00.335 INFO:teuthology.orchestra.run.smithi138.stdout:Packager : builder@centos.org 2024-05-27T09:05:00.335 INFO:teuthology.orchestra.run.smithi138.stdout:Vendor : CentOS 2024-05-27T09:05:00.335 INFO:teuthology.orchestra.run.smithi138.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-27T09:05:00.335 INFO:teuthology.orchestra.run.smithi138.stdout:Summary : Support tools for GRUB. 2024-05-27T09:05:00.335 INFO:teuthology.orchestra.run.smithi138.stdout:Description : 2024-05-27T09:05:00.335 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:05:00.335 INFO:teuthology.orchestra.run.smithi138.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-27T09:05:00.335 INFO:teuthology.orchestra.run.smithi138.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-27T09:05:00.335 INFO:teuthology.orchestra.run.smithi138.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-27T09:05:00.335 INFO:teuthology.orchestra.run.smithi138.stdout:hardware devices. 2024-05-27T09:05:00.335 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:05:00.336 INFO:teuthology.orchestra.run.smithi138.stdout:This subpackage provides tools for support of all platforms. 2024-05-27T09:05:00.336 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-27T09:05:00.336 INFO:teuthology.task.kernel:Updating grub on smithi138 to boot 6.9.0-gb9459a26b088 2024-05-27T09:05:00.337 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-27T09:05:00.927 INFO:teuthology.orchestra.run.smithi138.stderr:Generating grub configuration file ... 2024-05-27T09:05:02.360 INFO:teuthology.orchestra.run.smithi138.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-27T09:05:02.386 INFO:teuthology.orchestra.run.smithi138.stderr:done 2024-05-27T09:05:02.389 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-05-27T09:05:02.404 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.VGblxmRnQl 2024-05-27T09:05:02.405 DEBUG:teuthology.orchestra.run.smithi138:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.VGblxmRnQl 2024-05-27T09:05:02.469 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.VGblxmRnQl 2024-05-27T09:05:02.605 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.VGblxmRnQl is 6KB 2024-05-27T09:05:02.624 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.VGblxmRnQl 2024-05-27T09:05:02.638 DEBUG:teuthology.orchestra.run.smithi138:> sudo /bin/ls /boot/loader/entries || true 2024-05-27T09:05:02.699 INFO:teuthology.orchestra.run.smithi138.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-27T09:05:02.700 INFO:teuthology.orchestra.run.smithi138.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-27T09:05:02.700 INFO:teuthology.orchestra.run.smithi138.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-27T09:05:02.700 INFO:teuthology.orchestra.run.smithi138.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-27T09:05:02.701 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-27T09:05:02.860 DEBUG:teuthology.orchestra.run.smithi138:> sudo shutdown -r now 2024-05-27T09:05:25.400 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-05-27T09:05:25.401 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-05-27T09:05:25.402 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:05:32.865 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-05-27T09:05:32.865 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-27T09:05:32.866 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:05:43.966 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-05-27T09:05:51.390 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-05-27T09:05:52.969 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-05-27T09:05:52.970 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:05:56.030 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-05-27T09:06:00.394 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-27T09:06:00.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:06:03.454 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-05-27T09:06:08.034 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-05-27T09:06:08.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:06:15.457 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-27T09:06:15.458 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:06:33.886 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-05-27T09:06:39.897 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-05-27T09:06:40.471 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-05-27T09:06:40.471 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "6.9.0-gb9459a26b088"... 2024-05-27T09:06:40.471 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2024-05-27T09:06:40.511 INFO:teuthology.orchestra.run.smithi119.stdout:6.9.0-gb9459a26b088 2024-05-27T09:06:40.511 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-27T09:06:40.511 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-27T09:06:40.511 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-27T09:06:41.512 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-05-27T09:06:41.513 DEBUG:teuthology.orchestra.run.smithi119:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-27T09:06:41.621 INFO:teuthology.orchestra.run.smithi119.stdout:ttyS1 2024-05-27T09:06:41.685 DEBUG:teuthology.parallel:result is None 2024-05-27T09:06:48.888 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-27T09:06:48.889 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:06:50.171 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-05-27T09:06:50.752 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-27T09:06:50.752 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "6.9.0-gb9459a26b088"... 2024-05-27T09:06:50.752 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-05-27T09:06:50.795 INFO:teuthology.orchestra.run.smithi138.stdout:6.9.0-gb9459a26b088 2024-05-27T09:06:50.795 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-27T09:06:50.795 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-27T09:06:50.795 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-27T09:06:51.796 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-05-27T09:06:51.796 DEBUG:teuthology.orchestra.run.smithi138:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-27T09:06:51.988 INFO:teuthology.orchestra.run.smithi138.stdout:ttyS1 2024-05-27T09:06:52.051 DEBUG:teuthology.parallel:result is None 2024-05-27T09:06:52.052 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-27T09:06:52.059 INFO:teuthology.task.internal:Creating test directory... 2024-05-27T09:06:52.060 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T09:06:52.072 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T09:06:52.151 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-27T09:06:52.157 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-27T09:06:52.168 INFO:teuthology.task.internal:Creating archive directory... 2024-05-27T09:06:52.168 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T09:06:52.176 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T09:06:52.258 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-27T09:06:52.265 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-27T09:06:52.265 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T09:06:52.279 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T09:06:52.385 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T09:06:52.411 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T09:06:52.412 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T09:06:52.463 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T09:06:52.468 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-27T09:06:52.474 INFO:teuthology.task.internal:Configuring sudo... 2024-05-27T09:06:52.475 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T09:06:52.484 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T09:06:52.591 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-27T09:06:52.601 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-27T09:06:52.601 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T09:06:52.611 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T09:06:52.679 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T09:06:52.795 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T09:06:52.935 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T09:06:53.034 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T09:06:53.159 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-27T09:06:53.160 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T09:06:53.280 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T09:06:53.347 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T09:06:53.442 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T09:06:53.512 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T09:06:53.611 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-27T09:06:53.611 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T09:06:53.678 DEBUG:teuthology.orchestra.run.smithi119:> sudo service rsyslog restart 2024-05-27T09:06:53.686 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-05-27T09:06:53.797 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T09:06:53.825 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T09:06:54.077 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-27T09:06:54.085 INFO:teuthology.task.internal:Starting timer... 2024-05-27T09:06:54.085 INFO:teuthology.run_tasks:Running task pcp... 2024-05-27T09:06:54.095 INFO:teuthology.run_tasks:Running task selinux... 2024-05-27T09:06:54.103 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-27T09:06:54.103 DEBUG:teuthology.orchestra.run.smithi119:> sudo service auditd rotate 2024-05-27T09:06:54.208 INFO:teuthology.orchestra.run.smithi119.stdout:Rotating logs: 2024-05-27T09:06:54.212 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2024-05-27T09:06:54.327 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2024-05-27T09:06:54.331 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-27T09:06:54.332 DEBUG:teuthology.orchestra.run.smithi119:> /usr/sbin/getenforce 2024-05-27T09:06:54.391 INFO:teuthology.orchestra.run.smithi119.stdout:Permissive 2024-05-27T09:06:54.392 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2024-05-27T09:06:54.438 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2024-05-27T09:06:54.439 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi119.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi138.front.sepia.ceph.com': 'permissive'} 2024-05-27T09:06:54.439 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T09:06:54.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:06:54.514 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T09:06:54.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:06:54.592 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-27T09:06:54.592 DEBUG:teuthology.orchestra.run.smithi119:> sudo /usr/sbin/setenforce permissive 2024-05-27T09:06:54.667 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2024-05-27T09:06:54.744 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-27T09:06:54.753 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-27T09:06:54.756 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-27T09:06:54.881 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-27T09:06:54.892 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-27T09:06:54.893 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi119.front.sepia.ceph.com,smithi138.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-27T09:13:21.113 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi119.front.sepia.ceph.com'), Remote(name='ubuntu@smithi138.front.sepia.ceph.com')] 2024-05-27T09:13:21.115 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-05-27T09:13:21.116 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:13:21.227 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-05-27T09:13:21.380 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-05-27T09:13:21.380 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-27T09:13:21.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T09:13:21.489 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-05-27T09:13:21.638 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-27T09:13:21.639 INFO:teuthology.run_tasks:Running task clock... 2024-05-27T09:13:21.650 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-27T09:13:21.650 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T09:13:21.651 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T09:13:21.659 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T09:13:21.659 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T09:13:21.743 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T09:13:21.745 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T09:13:21.786 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T09:13:21.788 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T09:13:21.854 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2024-05-27T09:13:21.863 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: ntpd: command not found 2024-05-27T09:13:21.886 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2024-05-27T09:13:21.893 INFO:teuthology.orchestra.run.smithi119.stdout:506 Cannot talk to daemon 2024-05-27T09:13:21.927 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T09:13:21.934 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T09:13:21.970 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T09:13:21.971 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T09:13:22.096 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-05-27T09:13:22.105 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T09:13:22.105 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-05-27T09:13:22.105 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:13:22.106 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:13:22.106 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:13:22.106 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:13:22.110 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-05-27T09:13:22.118 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T09:13:22.118 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-05-27T09:13:22.118 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:13:22.118 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:13:22.119 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:13:22.119 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:13:22.120 INFO:teuthology.run_tasks:Running task install... 2024-05-27T09:13:22.128 DEBUG:teuthology.task.install:project ceph 2024-05-27T09:13:22.129 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d'}} 2024-05-27T09:13:22.129 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v18.2.0', 'flavor': 'default', 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d'} 2024-05-27T09:13:22.129 INFO:teuthology.task.install:Using flavor: default 2024-05-27T09:13:22.135 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-27T09:13:22.135 INFO:teuthology.task.install:extra packages: [] 2024-05-27T09:13:22.135 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d', 'tag': 'v18.2.0', 'wait_for_package': False} 2024-05-27T09:13:22.136 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-05-27T09:13:22.136 INFO:teuthology.packaging:ref: None 2024-05-27T09:13:22.136 INFO:teuthology.packaging:tag: v18.2.0 2024-05-27T09:13:22.136 INFO:teuthology.packaging:branch: None 2024-05-27T09:13:22.136 INFO:teuthology.packaging:sha1: a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:13:22.575 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph-ci v18.2.0^{} -> 5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-05-27T09:13:22.576 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-05-27T09:13:22.578 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d', 'tag': 'v18.2.0', 'wait_for_package': False} 2024-05-27T09:13:22.578 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-05-27T09:13:22.578 INFO:teuthology.packaging:ref: None 2024-05-27T09:13:22.578 INFO:teuthology.packaging:tag: v18.2.0 2024-05-27T09:13:22.579 INFO:teuthology.packaging:branch: None 2024-05-27T09:13:22.579 INFO:teuthology.packaging:sha1: a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:13:22.579 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-05-27T09:13:22.800 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/ 2024-05-27T09:13:22.800 INFO:teuthology.task.install.rpm:Package version is 18.2.0-0 2024-05-27T09:13:22.813 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/ 2024-05-27T09:13:22.813 INFO:teuthology.task.install.rpm:Package version is 18.2.0-0 2024-05-27T09:13:22.845 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T09:13:22.846 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-27T09:13:22.846 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T09:13:22.853 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T09:13:22.854 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-27T09:13:22.854 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T09:13:22.938 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-27T09:13:22.938 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-05-27T09:13:22.938 INFO:teuthology.packaging:ref: None 2024-05-27T09:13:22.938 INFO:teuthology.packaging:tag: v18.2.0 2024-05-27T09:13:22.939 INFO:teuthology.packaging:branch: None 2024-05-27T09:13:22.939 INFO:teuthology.packaging:sha1: a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:13:22.939 DEBUG:teuthology.orchestra.run.smithi119:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T09:13:22.942 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-27T09:13:22.943 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-05-27T09:13:22.943 INFO:teuthology.packaging:ref: None 2024-05-27T09:13:22.943 INFO:teuthology.packaging:tag: v18.2.0 2024-05-27T09:13:22.943 INFO:teuthology.packaging:branch: None 2024-05-27T09:13:22.943 INFO:teuthology.packaging:sha1: a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:13:22.943 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T09:13:23.068 DEBUG:teuthology.orchestra.run.smithi119:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T09:13:23.072 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T09:13:23.182 DEBUG:teuthology.orchestra.run.smithi119:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T09:13:23.185 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T09:13:23.259 INFO:teuthology.orchestra.run.smithi119.stdout:check_obsoletes = 1 2024-05-27T09:13:23.261 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2024-05-27T09:13:23.264 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum clean all 2024-05-27T09:13:23.266 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2024-05-27T09:13:23.579 INFO:teuthology.orchestra.run.smithi119.stdout:45 files removed 2024-05-27T09:13:23.590 INFO:teuthology.orchestra.run.smithi138.stdout:45 files removed 2024-05-27T09:13:23.618 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-radosgw 2024-05-27T09:13:23.628 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw 2024-05-27T09:13:24.191 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 471 kB/s | 76 kB 00:00 2024-05-27T09:13:24.210 INFO:teuthology.orchestra.run.smithi119.stdout:ceph packages for x86_64 467 kB/s | 76 kB 00:00 2024-05-27T09:13:24.420 INFO:teuthology.orchestra.run.smithi119.stdout:ceph noarch packages 60 kB/s | 9.3 kB 00:00 2024-05-27T09:13:24.426 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 50 kB/s | 9.3 kB 00:00 2024-05-27T09:13:24.620 INFO:teuthology.orchestra.run.smithi119.stdout:ceph source packages 14 kB/s | 2.2 kB 00:00 2024-05-27T09:13:24.635 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 13 kB/s | 2.2 kB 00:00 2024-05-27T09:13:26.287 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.1 MB 00:01 2024-05-27T09:13:26.735 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.1 MB 00:02 2024-05-27T09:13:29.730 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 8.4 MB/s | 19 MB 00:02 2024-05-27T09:13:30.294 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 8.2 MB/s | 19 MB 00:02 2024-05-27T09:13:34.728 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-05-27T09:13:35.877 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.2 MB 00:02 2024-05-27T09:13:37.312 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 17 kB 00:01 2024-05-27T09:13:37.497 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 17 kB 00:00 2024-05-27T09:13:38.425 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 28 MB/s | 21 MB 00:00 2024-05-27T09:13:39.821 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 9.1 MB/s | 21 MB 00:02 2024-05-27T09:13:44.533 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-05-27T09:13:46.164 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-05-27T09:13:47.060 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:13:47.079 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:13:47.079 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-05-27T09:13:47.079 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:13:47.079 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-05-27T09:13:47.080 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw x86_64 2:18.2.0-0.el9 ceph 7.6 M 2024-05-27T09:13:47.080 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2024-05-27T09:13:47.080 INFO:teuthology.orchestra.run.smithi119.stdout: librados2 x86_64 2:18.2.0-0.el9 ceph 3.3 M 2024-05-27T09:13:47.080 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1 x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-05-27T09:13:47.080 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-05-27T09:13:47.080 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T09:13:47.080 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base x86_64 2:18.2.0-0.el9 ceph 5.2 M 2024-05-27T09:13:47.080 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common x86_64 2:18.2.0-0.el9 ceph 18 M 2024-05-27T09:13:47.080 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux x86_64 2:18.2.0-0.el9 ceph 24 k 2024-05-27T09:13:47.080 INFO:teuthology.orchestra.run.smithi119.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-05-27T09:13:47.080 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T09:13:47.081 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T09:13:47.081 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T09:13:47.081 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2 x86_64 2:18.2.0-0.el9 ceph 653 k 2024-05-27T09:13:47.081 INFO:teuthology.orchestra.run.smithi119.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T09:13:47.081 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T09:13:47.081 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T09:13:47.081 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1 x86_64 2:18.2.0-0.el9 ceph 474 k 2024-05-27T09:13:47.081 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T09:13:47.081 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2 x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-05-27T09:13:47.081 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T09:13:47.081 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T09:13:47.081 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T09:13:47.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse x86_64 2:18.2.0-0.el9 ceph 45 k 2024-05-27T09:13:47.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common x86_64 2:18.2.0-0.el9 ceph 119 k 2024-05-27T09:13:47.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs x86_64 2:18.2.0-0.el9 ceph 155 k 2024-05-27T09:13:47.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T09:13:47.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados x86_64 2:18.2.0-0.el9 ceph 321 k 2024-05-27T09:13:47.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd x86_64 2:18.2.0-0.el9 ceph 297 k 2024-05-27T09:13:47.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw x86_64 2:18.2.0-0.el9 ceph 99 k 2024-05-27T09:13:47.082 INFO:teuthology.orchestra.run.smithi119.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T09:13:47.082 INFO:teuthology.orchestra.run.smithi119.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T09:13:47.082 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:13:47.082 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-05-27T09:13:47.083 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:13:47.083 INFO:teuthology.orchestra.run.smithi119.stdout:Install 28 Packages 2024-05-27T09:13:47.083 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 2 Packages 2024-05-27T09:13:47.083 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:13:47.083 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 53 M 2024-05-27T09:13:47.083 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-05-27T09:13:47.859 INFO:teuthology.orchestra.run.smithi119.stdout:(1/30): ceph-base-18.2.0-0.el9.x86_64.rpm 8.2 MB/s | 5.2 MB 00:00 2024-05-27T09:13:47.901 INFO:teuthology.orchestra.run.smithi119.stdout:(2/30): ceph-selinux-18.2.0-0.el9.x86_64.rpm 594 kB/s | 24 kB 00:00 2024-05-27T09:13:48.001 INFO:teuthology.orchestra.run.smithi119.stdout:(3/30): libcephfs2-18.2.0-0.el9.x86_64.rpm 6.4 MB/s | 653 kB 00:00 2024-05-27T09:13:48.076 INFO:teuthology.orchestra.run.smithi119.stdout:(4/30): libradosstriper1-18.2.0-0.el9.x86_64.rp 6.2 MB/s | 474 kB 00:00 2024-05-27T09:13:48.302 INFO:teuthology.orchestra.run.smithi119.stdout:(5/30): ceph-radosgw-18.2.0-0.el9.x86_64.rpm 7.1 MB/s | 7.6 MB 00:01 2024-05-27T09:13:48.344 INFO:teuthology.orchestra.run.smithi119.stdout:(6/30): python3-ceph-argparse-18.2.0-0.el9.x86_ 1.1 MB/s | 45 kB 00:00 2024-05-27T09:13:48.402 INFO:teuthology.orchestra.run.smithi119.stdout:(7/30): python3-ceph-common-18.2.0-0.el9.x86_64 2.0 MB/s | 119 kB 00:00 2024-05-27T09:13:48.511 INFO:teuthology.orchestra.run.smithi119.stdout:(8/30): librgw2-18.2.0-0.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-05-27T09:13:48.536 INFO:teuthology.orchestra.run.smithi119.stdout:(9/30): python3-cephfs-18.2.0-0.el9.x86_64.rpm 1.1 MB/s | 155 kB 00:00 2024-05-27T09:13:48.578 INFO:teuthology.orchestra.run.smithi119.stdout:(10/30): python3-rados-18.2.0-0.el9.x86_64.rpm 4.7 MB/s | 321 kB 00:00 2024-05-27T09:13:48.612 INFO:teuthology.orchestra.run.smithi119.stdout:(11/30): python3-rbd-18.2.0-0.el9.x86_64.rpm 3.9 MB/s | 297 kB 00:00 2024-05-27T09:13:48.646 INFO:teuthology.orchestra.run.smithi119.stdout:(12/30): python3-rgw-18.2.0-0.el9.x86_64.rpm 1.5 MB/s | 99 kB 00:00 2024-05-27T09:13:48.746 INFO:teuthology.orchestra.run.smithi119.stdout:(13/30): boost-program-options-1.75.0-8.el9.x86 808 kB/s | 107 kB 00:00 2024-05-27T09:13:48.871 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:13:48.904 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:13:48.905 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-05-27T09:13:48.905 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:13:48.905 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-27T09:13:48.905 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:18.2.0-0.el9 ceph 7.6 M 2024-05-27T09:13:48.906 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2024-05-27T09:13:48.906 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:18.2.0-0.el9 ceph 3.3 M 2024-05-27T09:13:48.906 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-05-27T09:13:48.907 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-05-27T09:13:48.907 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T09:13:48.907 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:18.2.0-0.el9 ceph 5.2 M 2024-05-27T09:13:48.907 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:18.2.0-0.el9 ceph 18 M 2024-05-27T09:13:48.908 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:18.2.0-0.el9 ceph 24 k 2024-05-27T09:13:48.908 INFO:teuthology.orchestra.run.smithi138.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-05-27T09:13:48.908 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T09:13:48.908 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T09:13:48.909 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T09:13:48.909 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:18.2.0-0.el9 ceph 653 k 2024-05-27T09:13:48.909 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T09:13:48.909 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T09:13:48.910 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T09:13:48.910 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:18.2.0-0.el9 ceph 474 k 2024-05-27T09:13:48.910 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T09:13:48.910 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-05-27T09:13:48.910 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T09:13:48.911 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T09:13:48.911 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T09:13:48.911 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_64 2:18.2.0-0.el9 ceph 45 k 2024-05-27T09:13:48.911 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:18.2.0-0.el9 ceph 119 k 2024-05-27T09:13:48.911 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:18.2.0-0.el9 ceph 155 k 2024-05-27T09:13:48.911 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T09:13:48.911 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:18.2.0-0.el9 ceph 321 k 2024-05-27T09:13:48.911 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:18.2.0-0.el9 ceph 297 k 2024-05-27T09:13:48.911 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:18.2.0-0.el9 ceph 99 k 2024-05-27T09:13:48.912 INFO:teuthology.orchestra.run.smithi138.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T09:13:48.912 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T09:13:48.912 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:13:48.912 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-27T09:13:48.912 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:13:48.913 INFO:teuthology.orchestra.run.smithi138.stdout:Install 28 Packages 2024-05-27T09:13:48.913 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 2 Packages 2024-05-27T09:13:48.913 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:13:48.914 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 53 M 2024-05-27T09:13:48.914 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-27T09:13:48.974 INFO:teuthology.orchestra.run.smithi119.stdout:(14/30): ceph-common-18.2.0-0.el9.x86_64.rpm 10 MB/s | 18 MB 00:01 2024-05-27T09:13:48.997 INFO:teuthology.orchestra.run.smithi119.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 181 kB/s | 45 kB 00:00 2024-05-27T09:13:49.024 INFO:teuthology.orchestra.run.smithi119.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 427 kB/s | 160 kB 00:00 2024-05-27T09:13:49.072 INFO:teuthology.orchestra.run.smithi119.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 859 kB/s | 42 kB 00:00 2024-05-27T09:13:49.106 INFO:teuthology.orchestra.run.smithi119.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-27T09:13:49.306 INFO:teuthology.orchestra.run.smithi119.stdout:(19/30): librdkafka-1.6.1-102.el9.x86_64.rpm 1.9 MB/s | 662 kB 00:00 2024-05-27T09:13:49.333 INFO:teuthology.orchestra.run.smithi138.stdout:(1/30): ceph-base-18.2.0-0.el9.x86_64.rpm 18 MB/s | 5.2 MB 00:00 2024-05-27T09:13:49.358 INFO:teuthology.orchestra.run.smithi138.stdout:(2/30): ceph-selinux-18.2.0-0.el9.x86_64.rpm 983 kB/s | 24 kB 00:00 2024-05-27T09:13:49.434 INFO:teuthology.orchestra.run.smithi138.stdout:(3/30): libcephfs2-18.2.0-0.el9.x86_64.rpm 8.6 MB/s | 653 kB 00:00 2024-05-27T09:13:49.483 INFO:teuthology.orchestra.run.smithi138.stdout:(4/30): libradosstriper1-18.2.0-0.el9.x86_64.rp 9.4 MB/s | 474 kB 00:00 2024-05-27T09:13:49.590 INFO:teuthology.orchestra.run.smithi119.stdout:(20/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 640 kB/s | 309 kB 00:00 2024-05-27T09:13:49.709 INFO:teuthology.orchestra.run.smithi138.stdout:(5/30): ceph-radosgw-18.2.0-0.el9.x86_64.rpm 11 MB/s | 7.6 MB 00:00 2024-05-27T09:13:49.742 INFO:teuthology.orchestra.run.smithi138.stdout:(6/30): python3-ceph-argparse-18.2.0-0.el9.x86_ 1.4 MB/s | 45 kB 00:00 2024-05-27T09:13:49.781 INFO:teuthology.orchestra.run.smithi119.stdout:(21/30): libarrow-9.0.0-11.el9.x86_64.rpm 9.3 MB/s | 4.4 MB 00:00 2024-05-27T09:13:49.806 INFO:teuthology.orchestra.run.smithi119.stdout:(22/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 123 kB/s | 26 kB 00:00 2024-05-27T09:13:49.826 INFO:teuthology.orchestra.run.smithi138.stdout:(7/30): python3-ceph-common-18.2.0-0.el9.x86_64 1.4 MB/s | 119 kB 00:00 2024-05-27T09:13:49.860 INFO:teuthology.orchestra.run.smithi138.stdout:(8/30): python3-cephfs-18.2.0-0.el9.x86_64.rpm 4.7 MB/s | 155 kB 00:00 2024-05-27T09:13:49.902 INFO:teuthology.orchestra.run.smithi138.stdout:(9/30): python3-rados-18.2.0-0.el9.x86_64.rpm 7.7 MB/s | 321 kB 00:00 2024-05-27T09:13:49.915 INFO:teuthology.orchestra.run.smithi119.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 457 kB/s | 49 kB 00:00 2024-05-27T09:13:49.944 INFO:teuthology.orchestra.run.smithi138.stdout:(10/30): python3-rbd-18.2.0-0.el9.x86_64.rpm 7.1 MB/s | 297 kB 00:00 2024-05-27T09:13:49.965 INFO:teuthology.orchestra.run.smithi119.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 431 kB/s | 67 kB 00:00 2024-05-27T09:13:49.977 INFO:teuthology.orchestra.run.smithi138.stdout:(11/30): python3-rgw-18.2.0-0.el9.x86_64.rpm 3.0 MB/s | 99 kB 00:00 2024-05-27T09:13:50.103 INFO:teuthology.orchestra.run.smithi138.stdout:(12/30): librgw2-18.2.0-0.el9.x86_64.rpm 7.2 MB/s | 4.4 MB 00:00 2024-05-27T09:13:50.115 INFO:teuthology.orchestra.run.smithi119.stdout:(25/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.1 MB/s | 839 kB 00:00 2024-05-27T09:13:50.149 INFO:teuthology.orchestra.run.smithi119.stdout:(26/30): re2-20211101-3.el9.x86_64.rpm 1.0 MB/s | 192 kB 00:00 2024-05-27T09:13:50.258 INFO:teuthology.orchestra.run.smithi119.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-27T09:13:50.359 INFO:teuthology.orchestra.run.smithi119.stdout:(28/30): librados2-18.2.0-0.el9.x86_64.rpm 16 MB/s | 3.3 MB 00:00 2024-05-27T09:13:50.384 INFO:teuthology.orchestra.run.smithi119.stdout:(29/30): fmt-8.1.1-5.el9.x86_64.rpm 84 kB/s | 111 kB 00:01 2024-05-27T09:13:50.461 INFO:teuthology.orchestra.run.smithi138.stdout:(13/30): ceph-common-18.2.0-0.el9.x86_64.rpm 13 MB/s | 18 MB 00:01 2024-05-27T09:13:50.487 INFO:teuthology.orchestra.run.smithi138.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 210 kB/s | 107 kB 00:00 2024-05-27T09:13:50.492 INFO:teuthology.orchestra.run.smithi119.stdout:(30/30): librbd1-18.2.0-0.el9.x86_64.rpm 13 MB/s | 3.0 MB 00:00 2024-05-27T09:13:50.501 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:13:50.501 INFO:teuthology.orchestra.run.smithi119.stdout:Total 15 MB/s | 53 MB 00:03 2024-05-27T09:13:50.687 INFO:teuthology.orchestra.run.smithi138.stdout:(15/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 275 kB/s | 160 kB 00:00 2024-05-27T09:13:50.725 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-05-27T09:13:50.804 INFO:teuthology.orchestra.run.smithi138.stdout:(16/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 133 kB/s | 45 kB 00:00 2024-05-27T09:13:50.808 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-05-27T09:13:50.809 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-05-27T09:13:50.908 INFO:teuthology.orchestra.run.smithi138.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.3 MB/s | 292 kB 00:00 2024-05-27T09:13:50.954 INFO:teuthology.orchestra.run.smithi138.stdout:(18/30): librdkafka-1.6.1-102.el9.x86_64.rpm 1.4 MB/s | 662 kB 00:00 2024-05-27T09:13:50.988 INFO:teuthology.orchestra.run.smithi138.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 230 kB/s | 42 kB 00:00 2024-05-27T09:13:51.055 INFO:teuthology.orchestra.run.smithi138.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 759 kB/s | 111 kB 00:00 2024-05-27T09:13:51.097 INFO:teuthology.orchestra.run.smithi138.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 639 kB/s | 26 kB 00:00 2024-05-27T09:13:51.131 INFO:teuthology.orchestra.run.smithi138.stdout:(22/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.7 MB/s | 309 kB 00:00 2024-05-27T09:13:51.156 INFO:teuthology.orchestra.run.smithi138.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 838 kB/s | 49 kB 00:00 2024-05-27T09:13:51.181 INFO:teuthology.orchestra.run.smithi138.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-05-27T09:13:51.232 INFO:teuthology.orchestra.run.smithi138.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 3.8 MB/s | 192 kB 00:00 2024-05-27T09:13:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.6 MB/s | 839 kB 00:00 2024-05-27T09:13:51.407 INFO:teuthology.orchestra.run.smithi138.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2024-05-27T09:13:51.499 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-05-27T09:13:51.500 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-05-27T09:13:51.525 INFO:teuthology.orchestra.run.smithi138.stdout:(28/30): libarrow-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 4.4 MB 00:00 2024-05-27T09:13:51.633 INFO:teuthology.orchestra.run.smithi138.stdout:(29/30): librados2-18.2.0-0.el9.x86_64.rpm 10 MB/s | 3.3 MB 00:00 2024-05-27T09:13:51.733 INFO:teuthology.orchestra.run.smithi138.stdout:(30/30): librbd1-18.2.0-0.el9.x86_64.rpm 9.2 MB/s | 3.0 MB 00:00 2024-05-27T09:13:51.738 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:13:51.739 INFO:teuthology.orchestra.run.smithi138.stdout:Total 19 MB/s | 53 MB 00:02 2024-05-27T09:13:51.977 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-27T09:13:52.058 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-27T09:13:52.058 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-27T09:13:52.749 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-27T09:13:52.750 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-27T09:13:52.960 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-05-27T09:13:53.057 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-05-27T09:13:53.091 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-05-27T09:13:53.183 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-05-27T09:13:53.592 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-05-27T09:13:53.705 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librados2-2:18.2.0-0.el9.x86_64 5/32 2024-05-27T09:13:53.874 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:18.2.0-0.el9.x86_64 5/32 2024-05-27T09:13:53.991 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rados-2:18.2.0-0.el9.x86_64 6/32 2024-05-27T09:13:54.023 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-05-27T09:13:54.107 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-05-27T09:13:54.159 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-05-27T09:13:54.231 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 9/32 2024-05-27T09:13:54.323 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-27T09:13:54.335 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-05-27T09:13:54.367 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-05-27T09:13:54.450 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-05-27T09:13:54.462 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-05-27T09:13:54.500 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-05-27T09:13:54.635 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-05-27T09:13:54.647 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-05-27T09:13:54.687 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-05-27T09:13:54.740 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-05-27T09:13:54.776 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-05-27T09:13:54.817 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-05-27T09:13:54.865 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-05-27T09:13:54.914 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-05-27T09:13:55.072 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-05-27T09:13:55.180 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:18.2.0-0.el9.x86_64 5/32 2024-05-27T09:13:55.319 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:18.2.0-0.el9.x86_64 5/32 2024-05-27T09:13:55.321 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-05-27T09:13:55.356 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-05-27T09:13:55.432 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:18.2.0-0.el9.x86_64 6/32 2024-05-27T09:13:55.464 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-05-27T09:13:55.471 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-05-27T09:13:55.505 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rbd-2:18.2.0-0.el9.x86_64 21/32 2024-05-27T09:13:55.549 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-05-27T09:13:55.561 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-05-27T09:13:55.583 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-05-27T09:13:55.647 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 9/32 2024-05-27T09:13:55.744 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-05-27T09:13:55.775 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-05-27T09:13:55.902 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-05-27T09:13:56.062 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-05-27T09:13:56.084 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-05-27T09:13:56.111 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-05-27T09:13:56.127 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-05-27T09:13:56.179 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-05-27T09:13:56.194 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-05-27T09:13:56.226 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-05-27T09:13:56.266 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-05-27T09:13:56.296 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rgw-2:18.2.0-0.el9.x86_64 25/32 2024-05-27T09:13:56.321 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-common-2:18.2.0-0.el9.x86_64 26/32 2024-05-27T09:13:56.335 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-05-27T09:13:56.391 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-05-27T09:13:56.799 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-05-27T09:13:56.848 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-05-27T09:13:56.955 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-05-27T09:13:56.997 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:18.2.0-0.el9.x86_64 21/32 2024-05-27T09:13:57.054 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-05-27T09:13:57.540 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-05-27T09:13:57.567 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-05-27T09:13:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-05-27T09:13:57.786 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:18.2.0-0.el9.x86_64 25/32 2024-05-27T09:13:57.829 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:18.2.0-0.el9.x86_64 26/32 2024-05-27T09:13:58.827 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-05-27T09:13:58.868 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-05-27T09:13:59.544 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-05-27T09:13:59.589 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-05-27T09:13:59.738 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-05-27T09:13:59.738 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T09:13:59.738 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T09:13:59.738 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:13:59.773 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-05-27T09:14:00.275 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-05-27T09:14:00.318 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-05-27T09:14:00.978 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-05-27T09:14:01.022 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-05-27T09:14:01.215 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-05-27T09:14:01.216 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T09:14:01.216 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T09:14:01.216 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:01.256 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-05-27T09:14:10.790 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-05-27T09:14:10.791 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /sys 2024-05-27T09:14:10.791 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /proc 2024-05-27T09:14:10.791 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /mnt 2024-05-27T09:14:10.791 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /var/tmp 2024-05-27T09:14:10.791 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /home 2024-05-27T09:14:10.791 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /root 2024-05-27T09:14:10.791 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /tmp 2024-05-27T09:14:10.791 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:11.662 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-05-27T09:14:11.733 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-05-27T09:14:11.733 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:14:11.733 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T09:14:11.734 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T09:14:11.734 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T09:14:11.734 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:11.739 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-05-27T09:14:11.881 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-05-27T09:14:11.881 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-05-27T09:14:12.193 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-05-27T09:14:12.193 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2024-05-27T09:14:12.194 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2024-05-27T09:14:12.194 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2024-05-27T09:14:12.194 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2024-05-27T09:14:12.194 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2024-05-27T09:14:12.194 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /root 2024-05-27T09:14:12.194 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2024-05-27T09:14:12.194 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:13.083 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-05-27T09:14:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-05-27T09:14:13.158 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:14:13.158 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T09:14:13.158 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T09:14:13.158 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T09:14:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-05-27T09:14:13.283 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-05-27T09:14:13.283 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-05-27T09:14:13.448 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-05-27T09:14:13.448 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-base-2:18.2.0-0.el9.x86_64 1/32 2024-05-27T09:14:13.448 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-common-2:18.2.0-0.el9.x86_64 2/32 2024-05-27T09:14:13.448 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-radosgw-2:18.2.0-0.el9.x86_64 3/32 2024-05-27T09:14:13.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-selinux-2:18.2.0-0.el9.x86_64 4/32 2024-05-27T09:14:13.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs2-2:18.2.0-0.el9.x86_64 5/32 2024-05-27T09:14:13.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libradosstriper1-2:18.2.0-0.el9.x86_64 6/32 2024-05-27T09:14:13.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librgw2-2:18.2.0-0.el9.x86_64 7/32 2024-05-27T09:14:13.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 8/32 2024-05-27T09:14:13.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-common-2:18.2.0-0.el9.x86_64 9/32 2024-05-27T09:14:13.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-05-27T09:14:13.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rados-2:18.2.0-0.el9.x86_64 11/32 2024-05-27T09:14:13.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rbd-2:18.2.0-0.el9.x86_64 12/32 2024-05-27T09:14:13.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rgw-2:18.2.0-0.el9.x86_64 13/32 2024-05-27T09:14:13.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-05-27T09:14:13.449 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-05-27T09:14:13.450 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-05-27T09:14:13.450 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-05-27T09:14:13.450 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-05-27T09:14:13.450 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-05-27T09:14:13.450 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-05-27T09:14:13.450 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-05-27T09:14:13.450 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-05-27T09:14:13.450 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-05-27T09:14:13.450 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-05-27T09:14:13.450 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-05-27T09:14:13.450 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-05-27T09:14:13.450 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-05-27T09:14:13.451 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-05-27T09:14:13.451 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:18.2.0-0.el9.x86_64 29/32 2024-05-27T09:14:13.451 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-05-27T09:14:13.451 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:18.2.0-0.el9.x86_64 31/32 2024-05-27T09:14:14.120 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-05-27T09:14:14.120 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:14.120 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2024-05-27T09:14:14.120 INFO:teuthology.orchestra.run.smithi119.stdout: librados2-2:18.2.0-0.el9.x86_64 librbd1-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:14.120 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-05-27T09:14:14.120 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T09:14:14.120 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:14.120 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:14.120 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: fmt-8.1.1-5.el9.x86_64 2024-05-27T09:14:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T09:14:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T09:14:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T09:14:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T09:14:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T09:14:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T09:14:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T09:14:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T09:14:14.121 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T09:14:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T09:14:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T09:14:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T09:14:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T09:14:14.122 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:14.122 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:14:14.460 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-test 2024-05-27T09:14:14.926 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-05-27T09:14:14.927 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:18.2.0-0.el9.x86_64 1/32 2024-05-27T09:14:14.927 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:18.2.0-0.el9.x86_64 2/32 2024-05-27T09:14:14.927 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:18.2.0-0.el9.x86_64 3/32 2024-05-27T09:14:14.927 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:18.2.0-0.el9.x86_64 4/32 2024-05-27T09:14:14.927 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:18.2.0-0.el9.x86_64 5/32 2024-05-27T09:14:14.927 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:18.2.0-0.el9.x86_64 6/32 2024-05-27T09:14:14.927 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:18.2.0-0.el9.x86_64 7/32 2024-05-27T09:14:14.927 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 8/32 2024-05-27T09:14:14.927 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:18.2.0-0.el9.x86_64 9/32 2024-05-27T09:14:14.927 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-05-27T09:14:14.927 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:18.2.0-0.el9.x86_64 11/32 2024-05-27T09:14:14.927 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:18.2.0-0.el9.x86_64 12/32 2024-05-27T09:14:14.928 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:18.2.0-0.el9.x86_64 13/32 2024-05-27T09:14:14.928 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-05-27T09:14:14.928 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-05-27T09:14:14.928 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-05-27T09:14:14.928 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-05-27T09:14:14.928 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-05-27T09:14:14.928 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-05-27T09:14:14.928 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-05-27T09:14:14.928 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-05-27T09:14:14.928 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-05-27T09:14:14.928 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-05-27T09:14:14.928 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-05-27T09:14:14.928 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-05-27T09:14:14.929 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-05-27T09:14:14.929 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-05-27T09:14:14.929 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-05-27T09:14:14.929 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:18.2.0-0.el9.x86_64 29/32 2024-05-27T09:14:14.930 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-05-27T09:14:14.930 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:18.2.0-0.el9.x86_64 31/32 2024-05-27T09:14:15.227 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:00:31 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:14:15.363 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:14:15.364 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:14:15.364 INFO:teuthology.orchestra.run.smithi119.stdout: Package Architecture Version Repository Size 2024-05-27T09:14:15.364 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:14:15.364 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-05-27T09:14:15.364 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test x86_64 2:18.2.0-0.el9 ceph 40 M 2024-05-27T09:14:15.364 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-05-27T09:14:15.364 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite x86_64 2:18.2.0-0.el9 ceph 161 k 2024-05-27T09:14:15.364 INFO:teuthology.orchestra.run.smithi119.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T09:14:15.364 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T09:14:15.364 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:15.365 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-05-27T09:14:15.365 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:14:15.365 INFO:teuthology.orchestra.run.smithi119.stdout:Install 4 Packages 2024-05-27T09:14:15.365 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:15.365 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 40 M 2024-05-27T09:14:15.365 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 165 M 2024-05-27T09:14:15.366 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-05-27T09:14:15.570 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-05-27T09:14:15.571 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:15.571 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2024-05-27T09:14:15.571 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:18.2.0-0.el9.x86_64 librbd1-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:15.571 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-27T09:14:15.571 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T09:14:15.571 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:15.571 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:15.571 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:15.571 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:15.571 INFO:teuthology.orchestra.run.smithi138.stdout: fmt-8.1.1-5.el9.x86_64 2024-05-27T09:14:15.571 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T09:14:15.571 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T09:14:15.572 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T09:14:15.572 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:15.572 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T09:14:15.572 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T09:14:15.572 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T09:14:15.572 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:15.572 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T09:14:15.572 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:15.572 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T09:14:15.572 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T09:14:15.572 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T09:14:15.572 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:15.572 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:15.572 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:15.573 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T09:14:15.573 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:15.573 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:15.573 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:15.573 INFO:teuthology.orchestra.run.smithi138.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T09:14:15.573 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T09:14:15.573 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:15.573 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:14:15.652 INFO:teuthology.orchestra.run.smithi119.stdout:(1/4): libcephsqlite-18.2.0-0.el9.x86_64.rpm 1.1 MB/s | 161 kB 00:00 2024-05-27T09:14:15.811 INFO:teuthology.orchestra.run.smithi119.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 402 kB/s | 64 kB 00:00 2024-05-27T09:14:15.878 INFO:teuthology.orchestra.run.smithi119.stdout:(3/4): socat-1.7.4.1-5.el9.x86_64.rpm 828 kB/s | 305 kB 00:00 2024-05-27T09:14:15.957 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-test 2024-05-27T09:14:16.711 INFO:teuthology.orchestra.run.smithi119.stdout:(4/4): ceph-test-18.2.0-0.el9.x86_64.rpm 33 MB/s | 40 MB 00:01 2024-05-27T09:14:16.715 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:14:16.716 INFO:teuthology.orchestra.run.smithi119.stdout:Total 30 MB/s | 40 MB 00:01 2024-05-27T09:14:16.740 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:30 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:14:16.794 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-05-27T09:14:16.848 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-05-27T09:14:16.848 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-05-27T09:14:16.875 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:14:16.876 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:14:16.876 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-05-27T09:14:16.876 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:14:16.876 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-27T09:14:16.877 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:18.2.0-0.el9 ceph 40 M 2024-05-27T09:14:16.877 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-05-27T09:14:16.877 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:18.2.0-0.el9 ceph 161 k 2024-05-27T09:14:16.877 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T09:14:16.877 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T09:14:16.877 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:16.877 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-27T09:14:16.877 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:14:16.878 INFO:teuthology.orchestra.run.smithi138.stdout:Install 4 Packages 2024-05-27T09:14:16.878 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:16.878 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 40 M 2024-05-27T09:14:16.878 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 165 M 2024-05-27T09:14:16.878 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-27T09:14:17.012 INFO:teuthology.orchestra.run.smithi138.stdout:(1/4): libcephsqlite-18.2.0-0.el9.x86_64.rpm 1.7 MB/s | 161 kB 00:00 2024-05-27T09:14:17.179 INFO:teuthology.orchestra.run.smithi138.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 383 kB/s | 64 kB 00:00 2024-05-27T09:14:17.204 INFO:teuthology.orchestra.run.smithi138.stdout:(3/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 305 kB 00:00 2024-05-27T09:14:17.213 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-05-27T09:14:17.214 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-05-27T09:14:17.733 INFO:teuthology.orchestra.run.smithi138.stdout:(4/4): ceph-test-18.2.0-0.el9.x86_64.rpm 49 MB/s | 40 MB 00:00 2024-05-27T09:14:17.737 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:14:17.738 INFO:teuthology.orchestra.run.smithi138.stdout:Total 47 MB/s | 40 MB 00:00 2024-05-27T09:14:17.786 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-05-27T09:14:17.813 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-27T09:14:17.867 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-27T09:14:17.867 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-27T09:14:17.871 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-27T09:14:17.940 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-27T09:14:17.972 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephsqlite-2:18.2.0-0.el9.x86_64 3/4 2024-05-27T09:14:18.262 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-27T09:14:18.263 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-27T09:14:18.838 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-27T09:14:18.989 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-27T09:14:19.047 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-27T09:14:19.073 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:18.2.0-0.el9.x86_64 3/4 2024-05-27T09:14:22.194 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephsqlite-2:18.2.0-0.el9.x86_64 3/4 2024-05-27T09:14:22.289 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-test-2:18.2.0-0.el9.x86_64 4/4 2024-05-27T09:14:23.291 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:18.2.0-0.el9.x86_64 3/4 2024-05-27T09:14:23.415 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:18.2.0-0.el9.x86_64 4/4 2024-05-27T09:14:24.378 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-test-2:18.2.0-0.el9.x86_64 4/4 2024-05-27T09:14:24.378 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-test-2:18.2.0-0.el9.x86_64 1/4 2024-05-27T09:14:24.378 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephsqlite-2:18.2.0-0.el9.x86_64 2/4 2024-05-27T09:14:24.378 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-27T09:14:24.701 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-27T09:14:24.701 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:24.701 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-05-27T09:14:24.701 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test-2:18.2.0-0.el9.x86_64 libcephsqlite-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:24.701 INFO:teuthology.orchestra.run.smithi119.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T09:14:24.702 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:24.702 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:14:25.009 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph 2024-05-27T09:14:25.776 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-test-2:18.2.0-0.el9.x86_64 4/4 2024-05-27T09:14:25.777 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:18.2.0-0.el9.x86_64 1/4 2024-05-27T09:14:25.777 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:18.2.0-0.el9.x86_64 2/4 2024-05-27T09:14:25.777 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-27T09:14:25.792 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:00:41 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:14:25.934 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:14:25.936 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:14:25.936 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-05-27T09:14:25.937 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:14:25.937 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-05-27T09:14:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: ceph x86_64 2:18.2.0-0.el9 ceph 6.4 k 2024-05-27T09:14:25.937 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-05-27T09:14:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds x86_64 2:18.2.0-0.el9 ceph 2.1 M 2024-05-27T09:14:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr x86_64 2:18.2.0-0.el9 ceph 1.4 M 2024-05-27T09:14:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core noarch 2:18.2.0-0.el9 ceph-noarch 240 k 2024-05-27T09:14:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-05-27T09:14:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd x86_64 2:18.2.0-0.el9 ceph 18 M 2024-05-27T09:14:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T09:14:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T09:14:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T09:14:25.937 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T09:14:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T09:14:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T09:14:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T09:14:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T09:14:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T09:14:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T09:14:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T09:14:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T09:14:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T09:14:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T09:14:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T09:14:25.938 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T09:14:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T09:14:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T09:14:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T09:14:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T09:14:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T09:14:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T09:14:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T09:14:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T09:14:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T09:14:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T09:14:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T09:14:25.939 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:25.939 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-05-27T09:14:25.940 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:14:25.940 INFO:teuthology.orchestra.run.smithi119.stdout:Install 33 Packages 2024-05-27T09:14:25.940 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:25.940 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 30 M 2024-05-27T09:14:25.940 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 109 M 2024-05-27T09:14:25.940 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-05-27T09:14:26.047 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-27T09:14:26.047 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:26.047 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-27T09:14:26.047 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:18.2.0-0.el9.x86_64 libcephsqlite-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:26.047 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T09:14:26.047 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:26.047 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:14:26.473 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph 2024-05-27T09:14:26.664 INFO:teuthology.orchestra.run.smithi119.stdout:(1/33): ceph-18.2.0-0.el9.x86_64.rpm 40 kB/s | 6.4 kB 00:00 2024-05-27T09:14:26.764 INFO:teuthology.orchestra.run.smithi119.stdout:(2/33): ceph-mds-18.2.0-0.el9.x86_64.rpm 8.1 MB/s | 2.1 MB 00:00 2024-05-27T09:14:26.806 INFO:teuthology.orchestra.run.smithi119.stdout:(3/33): ceph-mgr-18.2.0-0.el9.x86_64.rpm 4.8 MB/s | 1.4 MB 00:00 2024-05-27T09:14:26.873 INFO:teuthology.orchestra.run.smithi119.stdout:(4/33): ceph-mgr-modules-core-18.2.0-0.el9.noar 3.6 MB/s | 240 kB 00:00 2024-05-27T09:14:27.024 INFO:teuthology.orchestra.run.smithi119.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-05-27T09:14:27.116 INFO:teuthology.orchestra.run.smithi119.stdout:(6/33): ceph-mon-18.2.0-0.el9.x86_64.rpm 9.8 MB/s | 4.4 MB 00:00 2024-05-27T09:14:27.191 INFO:teuthology.orchestra.run.smithi119.stdout:(7/33): python3-cryptography-36.0.1-4.el9.x86_6 7.4 MB/s | 1.2 MB 00:00 2024-05-27T09:14:27.233 INFO:teuthology.orchestra.run.smithi119.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 3.2 MB/s | 135 kB 00:00 2024-05-27T09:14:27.258 INFO:teuthology.orchestra.run.smithi119.stdout:(9/33): python3-ply-3.11-14.el9.noarch.rpm 755 kB/s | 106 kB 00:00 2024-05-27T09:14:27.271 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:41 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:14:27.292 INFO:teuthology.orchestra.run.smithi119.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 2.1 MB/s | 125 kB 00:00 2024-05-27T09:14:27.392 INFO:teuthology.orchestra.run.smithi119.stdout:(11/33): python3-urllib3-1.26.5-5.el9.noarch.rp 1.6 MB/s | 215 kB 00:00 2024-05-27T09:14:27.406 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:14:27.409 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:14:27.409 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-05-27T09:14:27.409 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:14:27.409 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-27T09:14:27.409 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:18.2.0-0.el9 ceph 6.4 k 2024-05-27T09:14:27.409 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-05-27T09:14:27.409 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:18.2.0-0.el9 ceph 2.1 M 2024-05-27T09:14:27.409 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:18.2.0-0.el9 ceph 1.4 M 2024-05-27T09:14:27.409 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:18.2.0-0.el9 ceph-noarch 240 k 2024-05-27T09:14:27.409 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-05-27T09:14:27.410 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:18.2.0-0.el9 ceph 18 M 2024-05-27T09:14:27.410 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T09:14:27.410 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T09:14:27.410 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T09:14:27.410 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T09:14:27.410 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T09:14:27.410 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T09:14:27.410 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T09:14:27.410 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T09:14:27.410 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T09:14:27.410 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T09:14:27.410 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T09:14:27.410 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T09:14:27.411 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T09:14:27.411 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T09:14:27.411 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T09:14:27.411 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T09:14:27.411 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T09:14:27.411 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T09:14:27.411 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T09:14:27.411 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T09:14:27.411 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T09:14:27.411 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T09:14:27.411 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T09:14:27.411 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T09:14:27.411 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T09:14:27.411 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T09:14:27.412 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T09:14:27.412 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:27.412 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-27T09:14:27.412 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:14:27.412 INFO:teuthology.orchestra.run.smithi138.stdout:Install 33 Packages 2024-05-27T09:14:27.412 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:27.412 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 30 M 2024-05-27T09:14:27.413 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 109 M 2024-05-27T09:14:27.413 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-27T09:14:27.501 INFO:teuthology.orchestra.run.smithi119.stdout:(12/33): python3-mako-1.1.4-6.el9.noarch.rpm 829 kB/s | 172 kB 00:00 2024-05-27T09:14:27.526 INFO:teuthology.orchestra.run.smithi119.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 264 kB/s | 35 kB 00:00 2024-05-27T09:14:27.668 INFO:teuthology.orchestra.run.smithi119.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 307 kB/s | 51 kB 00:00 2024-05-27T09:14:27.685 INFO:teuthology.orchestra.run.smithi119.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 375 kB/s | 42 kB 00:00 2024-05-27T09:14:27.848 INFO:teuthology.orchestra.run.smithi138.stdout:(1/33): ceph-18.2.0-0.el9.x86_64.rpm 38 kB/s | 6.4 kB 00:00 2024-05-27T09:14:27.877 INFO:teuthology.orchestra.run.smithi119.stdout:(16/33): ceph-osd-18.2.0-0.el9.x86_64.rpm 16 MB/s | 18 MB 00:01 2024-05-27T09:14:27.907 INFO:teuthology.orchestra.run.smithi138.stdout:(2/33): ceph-mgr-18.2.0-0.el9.x86_64.rpm 6.4 MB/s | 1.4 MB 00:00 2024-05-27T09:14:27.965 INFO:teuthology.orchestra.run.smithi138.stdout:(3/33): ceph-mds-18.2.0-0.el9.x86_64.rpm 7.4 MB/s | 2.1 MB 00:00 2024-05-27T09:14:27.999 INFO:teuthology.orchestra.run.smithi138.stdout:(4/33): ceph-mgr-modules-core-18.2.0-0.el9.noar 7.2 MB/s | 240 kB 00:00 2024-05-27T09:14:28.124 INFO:teuthology.orchestra.run.smithi138.stdout:(5/33): ceph-mon-18.2.0-0.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-05-27T09:14:28.208 INFO:teuthology.orchestra.run.smithi138.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-05-27T09:14:28.249 INFO:teuthology.orchestra.run.smithi138.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-05-27T09:14:28.283 INFO:teuthology.orchestra.run.smithi138.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 4.0 MB/s | 135 kB 00:00 2024-05-27T09:14:28.333 INFO:teuthology.orchestra.run.smithi138.stdout:(9/33): python3-cryptography-36.0.1-4.el9.x86_6 5.9 MB/s | 1.2 MB 00:00 2024-05-27T09:14:28.359 INFO:teuthology.orchestra.run.smithi138.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 1.6 MB/s | 125 kB 00:00 2024-05-27T09:14:28.392 INFO:teuthology.orchestra.run.smithi138.stdout:(11/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-05-27T09:14:28.394 INFO:teuthology.orchestra.run.smithi119.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 60 kB/s | 43 kB 00:00 2024-05-27T09:14:28.501 INFO:teuthology.orchestra.run.smithi138.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 324 kB/s | 35 kB 00:00 2024-05-27T09:14:28.519 INFO:teuthology.orchestra.run.smithi119.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 86 kB/s | 11 kB 00:00 2024-05-27T09:14:28.526 INFO:teuthology.orchestra.run.smithi138.stdout:(13/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-05-27T09:14:28.602 INFO:teuthology.orchestra.run.smithi138.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 510 kB/s | 51 kB 00:00 2024-05-27T09:14:28.628 INFO:teuthology.orchestra.run.smithi119.stdout:(19/33): python3-jaraco-classes-3.2.1-5.el9.noa 164 kB/s | 18 kB 00:00 2024-05-27T09:14:28.643 INFO:teuthology.orchestra.run.smithi138.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 358 kB/s | 42 kB 00:00 2024-05-27T09:14:28.661 INFO:teuthology.orchestra.run.smithi119.stdout:(20/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 176 kB/s | 172 kB 00:00 2024-05-27T09:14:28.728 INFO:teuthology.orchestra.run.smithi119.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 233 kB/s | 23 kB 00:00 2024-05-27T09:14:28.762 INFO:teuthology.orchestra.run.smithi119.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 195 kB/s | 19 kB 00:00 2024-05-27T09:14:28.829 INFO:teuthology.orchestra.run.smithi119.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 197 kB/s | 20 kB 00:00 2024-05-27T09:14:28.862 INFO:teuthology.orchestra.run.smithi119.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 464 kB/s | 46 kB 00:00 2024-05-27T09:14:28.902 INFO:teuthology.orchestra.run.smithi138.stdout:(16/33): ceph-osd-18.2.0-0.el9.x86_64.rpm 18 MB/s | 18 MB 00:00 2024-05-27T09:14:28.988 INFO:teuthology.orchestra.run.smithi119.stdout:(25/33): python3-cherrypy-18.6.1-2.el9.noarch.r 362 kB/s | 358 kB 00:00 2024-05-27T09:14:29.013 INFO:teuthology.orchestra.run.smithi119.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 430 kB/s | 79 kB 00:00 2024-05-27T09:14:29.097 INFO:teuthology.orchestra.run.smithi119.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 152 kB/s | 16 kB 00:00 2024-05-27T09:14:29.205 INFO:teuthology.orchestra.run.smithi119.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 470 kB/s | 90 kB 00:00 2024-05-27T09:14:29.230 INFO:teuthology.orchestra.run.smithi119.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 269 kB/s | 36 kB 00:00 2024-05-27T09:14:29.281 INFO:teuthology.orchestra.run.smithi119.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 652 kB/s | 272 kB 00:00 2024-05-27T09:14:29.352 INFO:teuthology.orchestra.run.smithi138.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 58 kB/s | 43 kB 00:00 2024-05-27T09:14:29.406 INFO:teuthology.orchestra.run.smithi119.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 160 kB/s | 20 kB 00:00 2024-05-27T09:14:29.478 INFO:teuthology.orchestra.run.smithi138.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 86 kB/s | 11 kB 00:00 2024-05-27T09:14:29.565 INFO:teuthology.orchestra.run.smithi119.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 641 kB/s | 230 kB 00:00 2024-05-27T09:14:29.611 INFO:teuthology.orchestra.run.smithi138.stdout:(19/33): python3-jaraco-classes-3.2.1-5.el9.noa 133 kB/s | 18 kB 00:00 2024-05-27T09:14:29.628 INFO:teuthology.orchestra.run.smithi138.stdout:(20/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 175 kB/s | 172 kB 00:00 2024-05-27T09:14:29.737 INFO:teuthology.orchestra.run.smithi138.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 187 kB/s | 23 kB 00:00 2024-05-27T09:14:29.779 INFO:teuthology.orchestra.run.smithi138.stdout:(22/33): python3-cherrypy-18.6.1-2.el9.noarch.r 474 kB/s | 358 kB 00:00 2024-05-27T09:14:29.821 INFO:teuthology.orchestra.run.smithi138.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 101 kB/s | 19 kB 00:00 2024-05-27T09:14:29.840 INFO:teuthology.orchestra.run.smithi119.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 702 kB/s | 427 kB 00:00 2024-05-27T09:14:29.852 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:14:29.852 INFO:teuthology.orchestra.run.smithi119.stdout:Total 7.7 MB/s | 30 MB 00:03 2024-05-27T09:14:29.863 INFO:teuthology.orchestra.run.smithi138.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 159 kB/s | 20 kB 00:00 2024-05-27T09:14:29.946 INFO:teuthology.orchestra.run.smithi138.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 373 kB/s | 46 kB 00:00 2024-05-27T09:14:30.005 INFO:teuthology.orchestra.run.smithi138.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 432 kB/s | 79 kB 00:00 2024-05-27T09:14:30.122 INFO:teuthology.orchestra.run.smithi138.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.0 MB/s | 272 kB 00:00 2024-05-27T09:14:30.164 INFO:teuthology.orchestra.run.smithi138.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 76 kB/s | 16 kB 00:00 2024-05-27T09:14:30.189 INFO:teuthology.orchestra.run.smithi138.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 491 kB/s | 90 kB 00:00 2024-05-27T09:14:30.205 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-05-27T09:14:30.291 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-05-27T09:14:30.292 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-05-27T09:14:30.297 INFO:teuthology.orchestra.run.smithi138.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 271 kB/s | 36 kB 00:00 2024-05-27T09:14:30.331 INFO:teuthology.orchestra.run.smithi138.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-05-27T09:14:30.373 INFO:teuthology.orchestra.run.smithi138.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.3 MB/s | 427 kB 00:00 2024-05-27T09:14:30.448 INFO:teuthology.orchestra.run.smithi138.stdout:(33/33): python3-zc-lockfile-2.0-10.el9.noarch. 138 kB/s | 20 kB 00:00 2024-05-27T09:14:30.460 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:14:30.460 INFO:teuthology.orchestra.run.smithi138.stdout:Total 9.8 MB/s | 30 MB 00:03 2024-05-27T09:14:30.756 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-05-27T09:14:30.757 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-05-27T09:14:30.823 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-27T09:14:30.910 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-27T09:14:30.910 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-27T09:14:31.374 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-27T09:14:31.375 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-27T09:14:32.053 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-05-27T09:14:32.136 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-05-27T09:14:32.187 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-05-27T09:14:32.240 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-05-27T09:14:32.399 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-05-27T09:14:32.500 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-05-27T09:14:32.560 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-05-27T09:14:32.600 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-05-27T09:14:32.638 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-05-27T09:14:32.674 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-05-27T09:14:32.717 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-05-27T09:14:32.750 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-27T09:14:32.776 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-05-27T09:14:32.838 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-05-27T09:14:32.869 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-05-27T09:14:32.874 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-05-27T09:14:32.921 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-05-27T09:14:32.924 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-05-27T09:14:32.973 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-05-27T09:14:33.071 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-05-27T09:14:33.103 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-05-27T09:14:33.205 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-05-27T09:14:33.268 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-05-27T09:14:33.276 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-05-27T09:14:33.308 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-05-27T09:14:33.345 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-05-27T09:14:33.387 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-05-27T09:14:33.390 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-05-27T09:14:33.454 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-05-27T09:14:33.458 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-05-27T09:14:33.497 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-05-27T09:14:33.531 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-05-27T09:14:33.561 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-05-27T09:14:33.600 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-05-27T09:14:33.636 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-05-27T09:14:33.645 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-05-27T09:14:33.702 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-05-27T09:14:33.834 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-05-27T09:14:33.902 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-05-27T09:14:33.968 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-05-27T09:14:34.042 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-05-27T09:14:34.077 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-05-27T09:14:34.145 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-05-27T09:14:34.196 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-05-27T09:14:34.235 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-05-27T09:14:34.249 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-05-27T09:14:34.288 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-05-27T09:14:34.356 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-05-27T09:14:34.406 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-05-27T09:14:34.441 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-05-27T09:14:34.597 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 28/33 2024-05-27T09:14:34.631 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-05-27T09:14:34.706 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-05-27T09:14:34.706 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-05-27T09:14:34.706 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:14:34.706 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T09:14:34.706 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T09:14:34.706 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T09:14:34.707 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:34.776 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-05-27T09:14:34.885 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-05-27T09:14:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-05-27T09:14:35.065 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-05-27T09:14:35.221 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-05-27T09:14:35.409 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 28/33 2024-05-27T09:14:35.447 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-05-27T09:14:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-05-27T09:14:35.523 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:14:35.523 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T09:14:35.523 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T09:14:35.523 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T09:14:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:36.357 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-05-27T09:14:36.433 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-05-27T09:14:36.434 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:14:36.434 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T09:14:36.434 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T09:14:36.434 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T09:14:36.434 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:36.932 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-05-27T09:14:37.007 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-05-27T09:14:37.007 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:14:37.007 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T09:14:37.007 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T09:14:37.007 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T09:14:37.007 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:37.240 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-05-27T09:14:37.241 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-05-27T09:14:37.308 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-05-27T09:14:37.309 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:14:37.309 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T09:14:37.309 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T09:14:37.309 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T09:14:37.309 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:37.317 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-05-27T09:14:37.317 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:14:37.317 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T09:14:37.317 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T09:14:37.317 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T09:14:37.317 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:37.384 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-2:18.2.0-0.el9.x86_64 33/33 2024-05-27T09:14:37.511 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-2:18.2.0-0.el9.x86_64 33/33 2024-05-27T09:14:37.511 INFO:teuthology.orchestra.run.smithi119.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-27T09:14:37.512 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:37.815 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-05-27T09:14:37.884 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-05-27T09:14:37.884 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:14:37.884 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T09:14:37.884 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T09:14:37.884 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T09:14:37.884 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:38.108 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-05-27T09:14:38.176 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-05-27T09:14:38.176 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:14:38.176 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T09:14:38.176 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T09:14:38.176 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T09:14:38.176 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:38.239 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:18.2.0-0.el9.x86_64 33/33 2024-05-27T09:14:38.392 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-2:18.2.0-0.el9.x86_64 33/33 2024-05-27T09:14:38.392 INFO:teuthology.orchestra.run.smithi138.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-27T09:14:38.392 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:39.425 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-2:18.2.0-0.el9.x86_64 1/33 2024-05-27T09:14:39.425 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mds-2:18.2.0-0.el9.x86_64 2/33 2024-05-27T09:14:39.425 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-2:18.2.0-0.el9.x86_64 3/33 2024-05-27T09:14:39.426 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mon-2:18.2.0-0.el9.x86_64 4/33 2024-05-27T09:14:39.426 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-osd-2:18.2.0-0.el9.x86_64 5/33 2024-05-27T09:14:39.426 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 6/33 2024-05-27T09:14:39.426 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-05-27T09:14:39.426 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-05-27T09:14:39.426 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-05-27T09:14:39.426 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-05-27T09:14:39.426 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-05-27T09:14:39.426 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-05-27T09:14:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-05-27T09:14:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-05-27T09:14:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-05-27T09:14:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-05-27T09:14:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-05-27T09:14:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-05-27T09:14:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-05-27T09:14:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-05-27T09:14:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-05-27T09:14:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-05-27T09:14:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-05-27T09:14:39.427 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-05-27T09:14:39.428 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-05-27T09:14:39.428 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-05-27T09:14:39.428 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-05-27T09:14:39.428 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-05-27T09:14:39.428 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-05-27T09:14:39.428 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-05-27T09:14:39.428 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-05-27T09:14:39.428 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-05-27T09:14:40.387 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:18.2.0-0.el9.x86_64 1/33 2024-05-27T09:14:40.387 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:18.2.0-0.el9.x86_64 2/33 2024-05-27T09:14:40.387 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:18.2.0-0.el9.x86_64 3/33 2024-05-27T09:14:40.387 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:18.2.0-0.el9.x86_64 4/33 2024-05-27T09:14:40.388 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:18.2.0-0.el9.x86_64 5/33 2024-05-27T09:14:40.388 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 6/33 2024-05-27T09:14:40.388 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-05-27T09:14:40.388 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-05-27T09:14:40.388 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-05-27T09:14:40.388 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-05-27T09:14:40.389 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-05-27T09:14:40.389 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-05-27T09:14:40.389 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-05-27T09:14:40.389 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-05-27T09:14:40.389 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-05-27T09:14:40.389 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-05-27T09:14:40.390 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-05-27T09:14:40.390 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-05-27T09:14:40.390 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-05-27T09:14:40.390 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-05-27T09:14:40.390 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-05-27T09:14:40.390 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-05-27T09:14:40.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-05-27T09:14:40.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-05-27T09:14:40.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-05-27T09:14:40.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-05-27T09:14:40.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-05-27T09:14:40.391 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-05-27T09:14:40.392 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-05-27T09:14:40.392 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-05-27T09:14:40.392 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-05-27T09:14:40.392 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-05-27T09:14:40.475 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-05-27T09:14:40.475 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:40.475 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-05-27T09:14:40.476 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:40.476 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:40.476 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:40.476 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 2024-05-27T09:14:40.476 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:40.477 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:40.477 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T09:14:40.477 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T09:14:40.477 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T09:14:40.477 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T09:14:40.477 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T09:14:40.478 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T09:14:40.478 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T09:14:40.478 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T09:14:40.478 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T09:14:40.478 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T09:14:40.479 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T09:14:40.479 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T09:14:40.479 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T09:14:40.479 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T09:14:40.479 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T09:14:40.480 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T09:14:40.480 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T09:14:40.480 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T09:14:40.480 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T09:14:40.480 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T09:14:40.480 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T09:14:40.481 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T09:14:40.481 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T09:14:40.481 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T09:14:40.481 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T09:14:40.481 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T09:14:40.482 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T09:14:40.482 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:40.482 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:14:40.909 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-base 2024-05-27T09:14:41.421 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-05-27T09:14:41.421 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:41.422 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-27T09:14:41.422 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:41.422 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:41.422 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:41.422 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 2024-05-27T09:14:41.422 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:41.422 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:41.422 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T09:14:41.422 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T09:14:41.422 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T09:14:41.423 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T09:14:41.423 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T09:14:41.423 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T09:14:41.423 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T09:14:41.423 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T09:14:41.423 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T09:14:41.423 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T09:14:41.423 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T09:14:41.423 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T09:14:41.424 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T09:14:41.424 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T09:14:41.424 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T09:14:41.424 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T09:14:41.424 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T09:14:41.424 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T09:14:41.424 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T09:14:41.424 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T09:14:41.424 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T09:14:41.424 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T09:14:41.424 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T09:14:41.425 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T09:14:41.425 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T09:14:41.425 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T09:14:41.425 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T09:14:41.425 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:41.425 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:14:41.684 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:00:57 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:14:41.770 INFO:teuthology.orchestra.run.smithi119.stdout:Package ceph-base-2:18.2.0-0.el9.x86_64 is already installed. 2024-05-27T09:14:41.814 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:14:41.814 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-05-27T09:14:41.815 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:14:41.864 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-base 2024-05-27T09:14:41.896 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install cephadm 2024-05-27T09:14:42.639 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:00:58 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:14:42.675 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:56 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:14:42.764 INFO:teuthology.orchestra.run.smithi138.stdout:Package ceph-base-2:18.2.0-0.el9.x86_64 is already installed. 2024-05-27T09:14:42.769 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:14:42.770 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:14:42.770 INFO:teuthology.orchestra.run.smithi119.stdout: Package Architecture Version Repository Size 2024-05-27T09:14:42.770 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:14:42.770 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-05-27T09:14:42.770 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm noarch 2:18.2.0-0.el9 ceph-noarch 209 k 2024-05-27T09:14:42.770 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:42.770 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-05-27T09:14:42.770 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:14:42.770 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2024-05-27T09:14:42.771 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:42.771 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 209 k 2024-05-27T09:14:42.771 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 200 k 2024-05-27T09:14:42.771 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-05-27T09:14:42.807 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:14:42.808 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-05-27T09:14:42.808 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:14:42.882 INFO:teuthology.orchestra.run.smithi119.stdout:cephadm-18.2.0-0.el9.noarch.rpm 1.8 MB/s | 209 kB 00:00 2024-05-27T09:14:42.882 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:14:42.883 INFO:teuthology.orchestra.run.smithi119.stdout:Total 1.8 MB/s | 209 kB 00:00 2024-05-27T09:14:42.883 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-05-27T09:14:42.884 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install cephadm 2024-05-27T09:14:42.891 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-05-27T09:14:42.891 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-05-27T09:14:42.895 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-05-27T09:14:42.896 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-05-27T09:14:42.999 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-05-27T09:14:43.659 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:57 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:14:43.665 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-05-27T09:14:43.786 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-05-27T09:14:43.790 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:14:43.791 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:14:43.791 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-05-27T09:14:43.791 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:14:43.791 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-27T09:14:43.791 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:18.2.0-0.el9 ceph-noarch 209 k 2024-05-27T09:14:43.791 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:43.791 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-27T09:14:43.792 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:14:43.792 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-05-27T09:14:43.792 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:43.792 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 209 k 2024-05-27T09:14:43.792 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 200 k 2024-05-27T09:14:43.792 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-27T09:14:43.874 INFO:teuthology.orchestra.run.smithi138.stdout:cephadm-18.2.0-0.el9.noarch.rpm 2.5 MB/s | 209 kB 00:00 2024-05-27T09:14:43.875 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:14:43.875 INFO:teuthology.orchestra.run.smithi138.stdout:Total 2.5 MB/s | 209 kB 00:00 2024-05-27T09:14:43.875 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-27T09:14:43.881 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-27T09:14:43.881 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-27T09:14:43.885 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-27T09:14:43.885 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-27T09:14:43.974 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-27T09:14:44.258 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-05-27T09:14:44.541 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-05-27T09:14:44.541 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:44.541 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-05-27T09:14:44.541 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm-2:18.2.0-0.el9.noarch 2024-05-27T09:14:44.541 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:44.541 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:14:44.745 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-05-27T09:14:44.819 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-immutable-object-cache 2024-05-27T09:14:44.947 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-05-27T09:14:45.344 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-05-27T09:14:45.581 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:01 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:14:45.666 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-05-27T09:14:45.666 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:45.666 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-27T09:14:45.666 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:18.2.0-0.el9.noarch 2024-05-27T09:14:45.666 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:45.666 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:14:45.713 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:14:45.714 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:14:45.714 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repo Size 2024-05-27T09:14:45.714 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:14:45.714 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-05-27T09:14:45.715 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-0.el9 ceph 142 k 2024-05-27T09:14:45.715 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:45.715 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-05-27T09:14:45.715 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:14:45.715 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2024-05-27T09:14:45.715 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:45.715 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 142 k 2024-05-27T09:14:45.715 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 399 k 2024-05-27T09:14:45.715 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-05-27T09:14:45.797 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-immutable-object-cache-18.2.0-0.el9.x86_64 1.7 MB/s | 142 kB 00:00 2024-05-27T09:14:45.797 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:14:45.798 INFO:teuthology.orchestra.run.smithi119.stdout:Total 1.7 MB/s | 142 kB 00:00 2024-05-27T09:14:45.798 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-05-27T09:14:45.808 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-05-27T09:14:45.808 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-05-27T09:14:45.877 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-05-27T09:14:45.878 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-05-27T09:14:45.955 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-immutable-object-cache 2024-05-27T09:14:46.044 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-05-27T09:14:46.115 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:14:46.212 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:14:46.212 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:14:46.212 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T09:14:46.212 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:46.746 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:00 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:14:46.885 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:14:46.885 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:14:46.885 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-05-27T09:14:46.885 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:14:46.885 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-27T09:14:46.885 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-0.el9 ceph 142 k 2024-05-27T09:14:46.885 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:46.885 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-27T09:14:46.886 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:14:46.886 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-05-27T09:14:46.886 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:46.886 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 142 k 2024-05-27T09:14:46.886 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 399 k 2024-05-27T09:14:46.886 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-27T09:14:46.955 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-immutable-object-cache-18.2.0-0.el9.x86_64 2.0 MB/s | 142 kB 00:00 2024-05-27T09:14:46.955 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:14:46.956 INFO:teuthology.orchestra.run.smithi138.stdout:Total 2.0 MB/s | 142 kB 00:00 2024-05-27T09:14:46.956 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-27T09:14:46.965 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-27T09:14:46.965 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-27T09:14:47.030 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-27T09:14:47.030 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-27T09:14:47.217 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-27T09:14:47.273 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:14:47.349 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:14:47.350 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:14:47.350 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T09:14:47.350 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:47.464 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:14:47.464 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:47.464 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-05-27T09:14:47.464 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:47.464 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:47.464 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:14:47.762 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-mgr 2024-05-27T09:14:48.538 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:04 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:14:48.598 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:14:48.598 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:48.598 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-27T09:14:48.598 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 2024-05-27T09:14:48.598 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:48.599 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:14:48.623 INFO:teuthology.orchestra.run.smithi119.stdout:Package ceph-mgr-2:18.2.0-0.el9.x86_64 is already installed. 2024-05-27T09:14:48.666 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:14:48.667 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-05-27T09:14:48.667 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:14:48.746 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-mgr-dashboard 2024-05-27T09:14:48.916 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr 2024-05-27T09:14:49.512 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:05 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:14:49.645 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:14:49.646 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:14:49.646 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-05-27T09:14:49.646 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:14:49.647 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-05-27T09:14:49.647 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard noarch 2:18.2.0-0.el9 ceph-noarch 1.7 M 2024-05-27T09:14:49.647 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-05-27T09:14:49.647 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards noarch 2:18.2.0-0.el9 ceph-noarch 23 k 2024-05-27T09:14:49.647 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts noarch 2:18.2.0-0.el9 ceph-noarch 15 k 2024-05-27T09:14:49.647 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-05-27T09:14:49.647 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T09:14:49.647 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T09:14:49.647 INFO:teuthology.orchestra.run.smithi119.stdout:Installing weak dependencies: 2024-05-27T09:14:49.647 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-05-27T09:14:49.647 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:49.647 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-05-27T09:14:49.647 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:14:49.648 INFO:teuthology.orchestra.run.smithi119.stdout:Install 7 Packages 2024-05-27T09:14:49.648 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:49.648 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 2.0 M 2024-05-27T09:14:49.648 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 8.1 M 2024-05-27T09:14:49.648 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-05-27T09:14:49.721 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:03 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:14:49.807 INFO:teuthology.orchestra.run.smithi138.stdout:Package ceph-mgr-2:18.2.0-0.el9.x86_64 is already installed. 2024-05-27T09:14:49.850 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:14:49.851 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-05-27T09:14:49.851 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:14:49.880 INFO:teuthology.orchestra.run.smithi119.stdout:(1/7): ceph-prometheus-alerts-18.2.0-0.el9.noar 82 kB/s | 15 kB 00:00 2024-05-27T09:14:49.910 INFO:teuthology.orchestra.run.smithi119.stdout:(2/7): ceph-grafana-dashboards-18.2.0-0.el9.noa 110 kB/s | 23 kB 00:00 2024-05-27T09:14:49.929 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-dashboard 2024-05-27T09:14:49.969 INFO:teuthology.orchestra.run.smithi119.stdout:(3/7): ceph-mgr-dashboard-18.2.0-0.el9.noarch.r 6.3 MB/s | 1.7 MB 00:00 2024-05-27T09:14:50.094 INFO:teuthology.orchestra.run.smithi119.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 42 kB/s | 9.0 kB 00:00 2024-05-27T09:14:50.137 INFO:teuthology.orchestra.run.smithi119.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 181 kB/s | 41 kB 00:00 2024-05-27T09:14:50.161 INFO:teuthology.orchestra.run.smithi119.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 160 kB/s | 31 kB 00:00 2024-05-27T09:14:50.312 INFO:teuthology.orchestra.run.smithi119.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 873 kB/s | 188 kB 00:00 2024-05-27T09:14:50.316 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:14:50.317 INFO:teuthology.orchestra.run.smithi119.stdout:Total 3.0 MB/s | 2.0 MB 00:00 2024-05-27T09:14:50.318 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-05-27T09:14:50.346 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-05-27T09:14:50.346 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-05-27T09:14:50.457 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-05-27T09:14:50.458 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-05-27T09:14:50.680 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:04 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:14:50.814 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:14:50.815 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:14:50.815 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-05-27T09:14:50.815 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:14:50.815 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-27T09:14:50.815 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch 2:18.2.0-0.el9 ceph-noarch 1.7 M 2024-05-27T09:14:50.815 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-05-27T09:14:50.815 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch 2:18.2.0-0.el9 ceph-noarch 23 k 2024-05-27T09:14:50.815 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch 2:18.2.0-0.el9 ceph-noarch 15 k 2024-05-27T09:14:50.816 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-05-27T09:14:50.816 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T09:14:50.816 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T09:14:50.816 INFO:teuthology.orchestra.run.smithi138.stdout:Installing weak dependencies: 2024-05-27T09:14:50.816 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-05-27T09:14:50.816 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:50.816 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-27T09:14:50.816 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:14:50.816 INFO:teuthology.orchestra.run.smithi138.stdout:Install 7 Packages 2024-05-27T09:14:50.816 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:50.817 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 2.0 M 2024-05-27T09:14:50.817 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 8.1 M 2024-05-27T09:14:50.817 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-27T09:14:50.860 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-05-27T09:14:50.922 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-05-27T09:14:50.980 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-05-27T09:14:51.082 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-05-27T09:14:51.120 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-05-27T09:14:51.150 INFO:teuthology.orchestra.run.smithi138.stdout:(1/7): ceph-grafana-dashboards-18.2.0-0.el9.noa 131 kB/s | 23 kB 00:00 2024-05-27T09:14:51.165 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 5/7 2024-05-27T09:14:51.175 INFO:teuthology.orchestra.run.smithi138.stdout:(2/7): ceph-prometheus-alerts-18.2.0-0.el9.noar 73 kB/s | 15 kB 00:00 2024-05-27T09:14:51.250 INFO:teuthology.orchestra.run.smithi138.stdout:(3/7): ceph-mgr-dashboard-18.2.0-0.el9.noarch.r 6.1 MB/s | 1.7 MB 00:00 2024-05-27T09:14:51.466 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 6/7 2024-05-27T09:14:51.508 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-05-27T09:14:51.634 INFO:teuthology.orchestra.run.smithi138.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 19 kB/s | 9.0 kB 00:00 2024-05-27T09:14:51.734 INFO:teuthology.orchestra.run.smithi138.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 73 kB/s | 41 kB 00:00 2024-05-27T09:14:51.834 INFO:teuthology.orchestra.run.smithi138.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 53 kB/s | 31 kB 00:00 2024-05-27T09:14:52.176 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-05-27T09:14:52.176 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 1/7 2024-05-27T09:14:52.176 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2/7 2024-05-27T09:14:52.176 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 3/7 2024-05-27T09:14:52.176 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-05-27T09:14:52.177 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-05-27T09:14:52.177 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-05-27T09:14:52.318 INFO:teuthology.orchestra.run.smithi138.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 276 kB/s | 188 kB 00:00 2024-05-27T09:14:52.325 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:14:52.325 INFO:teuthology.orchestra.run.smithi138.stdout:Total 1.3 MB/s | 2.0 MB 00:01 2024-05-27T09:14:52.326 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-27T09:14:52.354 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-27T09:14:52.355 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-27T09:14:52.468 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-27T09:14:52.468 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-27T09:14:52.533 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-05-27T09:14:52.533 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:52.533 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-05-27T09:14:52.533 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 2024-05-27T09:14:52.533 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2024-05-27T09:14:52.533 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 2024-05-27T09:14:52.533 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-05-27T09:14:52.533 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-05-27T09:14:52.533 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T09:14:52.533 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T09:14:52.534 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:52.534 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:14:52.825 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-27T09:14:52.862 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-27T09:14:52.928 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-05-27T09:14:52.975 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-05-27T09:14:53.059 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-05-27T09:14:53.134 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-05-27T09:14:53.180 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 5/7 2024-05-27T09:14:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 6/7 2024-05-27T09:14:53.576 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-05-27T09:14:53.585 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:09 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:14:53.714 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:14:53.715 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:14:53.715 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-05-27T09:14:53.715 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:14:53.716 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-05-27T09:14:53.716 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-0.el9 ceph-noarch 7.4 M 2024-05-27T09:14:53.716 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-05-27T09:14:53.716 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T09:14:53.716 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T09:14:53.716 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T09:14:53.716 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T09:14:53.716 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T09:14:53.716 INFO:teuthology.orchestra.run.smithi119.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T09:14:53.716 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T09:14:53.716 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T09:14:53.716 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T09:14:53.716 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T09:14:53.717 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T09:14:53.717 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:53.717 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-05-27T09:14:53.717 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:14:53.717 INFO:teuthology.orchestra.run.smithi119.stdout:Install 12 Packages 2024-05-27T09:14:53.717 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:14:53.717 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 38 M 2024-05-27T09:14:53.717 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 209 M 2024-05-27T09:14:53.718 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-05-27T09:14:54.178 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-05-27T09:14:54.178 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 1/7 2024-05-27T09:14:54.178 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2/7 2024-05-27T09:14:54.178 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 3/7 2024-05-27T09:14:54.178 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-05-27T09:14:54.178 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-05-27T09:14:54.178 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-05-27T09:14:54.469 INFO:teuthology.orchestra.run.smithi119.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.0-0. 19 MB/s | 7.4 MB 00:00 2024-05-27T09:14:54.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-05-27T09:14:54.546 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:54.546 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-27T09:14:54.546 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 2024-05-27T09:14:54.546 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2024-05-27T09:14:54.546 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 2024-05-27T09:14:54.546 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-05-27T09:14:54.546 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-05-27T09:14:54.546 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T09:14:54.546 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T09:14:54.547 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:54.547 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:14:54.727 INFO:teuthology.orchestra.run.smithi119.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 127 kB/s | 33 kB 00:00 2024-05-27T09:14:54.752 INFO:teuthology.orchestra.run.smithi119.stdout:(3/12): libquadmath-11.4.1-3.el9.x86_64.rpm 293 kB/s | 195 kB 00:00 2024-05-27T09:14:54.852 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-27T09:14:55.011 INFO:teuthology.orchestra.run.smithi119.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 70 kB/s | 18 kB 00:00 2024-05-27T09:14:55.195 INFO:teuthology.orchestra.run.smithi119.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 193 kB/s | 35 kB 00:00 2024-05-27T09:14:55.612 INFO:teuthology.orchestra.run.smithi119.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.4 MB/s | 3.0 MB 00:00 2024-05-27T09:14:55.645 INFO:teuthology.orchestra.run.smithi119.stdout:(7/12): libgfortran-11.4.1-3.el9.x86_64.rpm 515 kB/s | 803 kB 00:01 2024-05-27T09:14:55.660 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:09 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:14:55.771 INFO:teuthology.orchestra.run.smithi119.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.5 MB/s | 244 kB 00:00 2024-05-27T09:14:55.796 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:14:55.797 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:14:55.797 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-05-27T09:14:55.797 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:14:55.797 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-27T09:14:55.797 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-0.el9 ceph-noarch 7.4 M 2024-05-27T09:14:55.798 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-05-27T09:14:55.798 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T09:14:55.798 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T09:14:55.798 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T09:14:55.798 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T09:14:55.798 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T09:14:55.798 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T09:14:55.798 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T09:14:55.798 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T09:14:55.798 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T09:14:55.798 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T09:14:55.798 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T09:14:55.799 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:55.799 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-27T09:14:55.799 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:14:55.799 INFO:teuthology.orchestra.run.smithi138.stdout:Install 12 Packages 2024-05-27T09:14:55.799 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:14:55.799 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 38 M 2024-05-27T09:14:55.799 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 209 M 2024-05-27T09:14:55.800 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-27T09:14:55.888 INFO:teuthology.orchestra.run.smithi119.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.1 MB/s | 250 kB 00:00 2024-05-27T09:14:56.142 INFO:teuthology.orchestra.run.smithi138.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-05-27T09:14:56.191 INFO:teuthology.orchestra.run.smithi138.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 803 kB 00:00 2024-05-27T09:14:56.230 INFO:teuthology.orchestra.run.smithi119.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 4.6 MB/s | 4.7 MB 00:01 2024-05-27T09:14:56.317 INFO:teuthology.orchestra.run.smithi138.stdout:(3/12): ceph-mgr-diskprediction-local-18.2.0-0. 23 MB/s | 7.4 MB 00:00 2024-05-27T09:14:56.342 INFO:teuthology.orchestra.run.smithi138.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 163 kB/s | 33 kB 00:00 2024-05-27T09:14:56.409 INFO:teuthology.orchestra.run.smithi138.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 200 kB/s | 18 kB 00:00 2024-05-27T09:14:56.434 INFO:teuthology.orchestra.run.smithi138.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 387 kB/s | 35 kB 00:00 2024-05-27T09:14:56.568 INFO:teuthology.orchestra.run.smithi138.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.8 MB/s | 244 kB 00:00 2024-05-27T09:14:56.660 INFO:teuthology.orchestra.run.smithi138.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 6.4 MB/s | 3.0 MB 00:00 2024-05-27T09:14:56.735 INFO:teuthology.orchestra.run.smithi138.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 3.4 MB/s | 250 kB 00:00 2024-05-27T09:14:56.902 INFO:teuthology.orchestra.run.smithi138.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 10 MB/s | 4.7 MB 00:00 2024-05-27T09:14:57.008 INFO:teuthology.orchestra.run.smithi138.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-05-27T09:14:57.319 INFO:teuthology.orchestra.run.smithi138.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-05-27T09:14:57.322 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:14:57.322 INFO:teuthology.orchestra.run.smithi138.stdout:Total 25 MB/s | 38 MB 00:01 2024-05-27T09:14:57.380 INFO:teuthology.orchestra.run.smithi119.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.9 MB/s | 5.1 MB 00:01 2024-05-27T09:14:57.951 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-27T09:14:58.026 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-27T09:14:58.027 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-27T09:14:58.691 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-27T09:14:58.691 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-27T09:14:59.653 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-27T09:14:59.725 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-27T09:14:59.789 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-27T09:14:59.936 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-27T09:14:59.979 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-27T09:15:00.264 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-27T09:15:00.301 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-27T09:15:00.428 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-27T09:15:01.329 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-27T09:15:01.472 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-27T09:15:03.159 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-27T09:15:04.343 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-27T09:15:04.556 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-05-27T09:15:07.226 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-05-27T09:15:07.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 1/12 2024-05-27T09:15:07.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-27T09:15:07.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-27T09:15:07.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-27T09:15:07.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-27T09:15:07.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-27T09:15:07.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-27T09:15:07.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-27T09:15:07.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-27T09:15:07.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-27T09:15:07.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-27T09:15:07.696 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-27T09:15:07.696 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:07.696 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-27T09:15:07.696 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarch 2024-05-27T09:15:07.697 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T09:15:07.697 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T09:15:07.697 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T09:15:07.697 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T09:15:07.697 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T09:15:07.697 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T09:15:07.697 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T09:15:07.697 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T09:15:07.697 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T09:15:07.697 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T09:15:07.697 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T09:15:07.697 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:07.697 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:15:08.203 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-rook 2024-05-27T09:15:08.829 INFO:teuthology.orchestra.run.smithi119.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.3 MB/s | 16 MB 00:12 2024-05-27T09:15:08.831 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:15:08.832 INFO:teuthology.orchestra.run.smithi119.stdout:Total 2.5 MB/s | 38 MB 00:15 2024-05-27T09:15:08.979 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:22 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:15:09.104 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:15:09.106 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:09.106 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-05-27T09:15:09.106 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:09.106 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-27T09:15:09.106 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch 2:18.2.0-0.el9 ceph-noarch 47 k 2024-05-27T09:15:09.106 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-05-27T09:15:09.106 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T09:15:09.106 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T09:15:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T09:15:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T09:15:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T09:15:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T09:15:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T09:15:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T09:15:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T09:15:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T09:15:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T09:15:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T09:15:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:09.107 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-27T09:15:09.107 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:09.107 INFO:teuthology.orchestra.run.smithi138.stdout:Install 13 Packages 2024-05-27T09:15:09.108 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:09.108 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 2.2 M 2024-05-27T09:15:09.108 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 26 M 2024-05-27T09:15:09.108 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-27T09:15:09.410 INFO:teuthology.orchestra.run.smithi138.stdout:(1/13): ceph-mgr-rook-18.2.0-0.el9.noarch.rpm 476 kB/s | 47 kB 00:00 2024-05-27T09:15:09.446 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-05-27T09:15:09.522 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-05-27T09:15:09.522 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-05-27T09:15:09.927 INFO:teuthology.orchestra.run.smithi138.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 31 kB/s | 19 kB 00:00 2024-05-27T09:15:10.028 INFO:teuthology.orchestra.run.smithi138.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 36 kB/s | 26 kB 00:00 2024-05-27T09:15:10.104 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-05-27T09:15:10.104 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-05-27T09:15:10.336 INFO:teuthology.orchestra.run.smithi138.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 240 kB/s | 222 kB 00:00 2024-05-27T09:15:10.820 INFO:teuthology.orchestra.run.smithi138.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 178 kB/s | 159 kB 00:00 2024-05-27T09:15:10.996 INFO:teuthology.orchestra.run.smithi138.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 289 kB/s | 279 kB 00:00 2024-05-27T09:15:11.029 INFO:teuthology.orchestra.run.smithi138.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 78 kB/s | 54 kB 00:00 2024-05-27T09:15:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-05-27T09:15:11.159 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-27T09:15:11.188 INFO:teuthology.orchestra.run.smithi138.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 88 kB/s | 32 kB 00:00 2024-05-27T09:15:11.208 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-27T09:15:11.280 INFO:teuthology.orchestra.run.smithi138.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 53 kB/s | 15 kB 00:00 2024-05-27T09:15:11.352 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-27T09:15:11.405 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-27T09:15:11.431 INFO:teuthology.orchestra.run.smithi138.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 396 kB/s | 59 kB 00:00 2024-05-27T09:15:11.467 INFO:teuthology.orchestra.run.smithi138.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 475 kB/s | 206 kB 00:00 2024-05-27T09:15:11.539 INFO:teuthology.orchestra.run.smithi138.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 2.9 MB/s | 1.0 MB 00:00 2024-05-27T09:15:11.569 INFO:teuthology.orchestra.run.smithi138.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 675 kB/s | 90 kB 00:00 2024-05-27T09:15:11.573 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:15:11.579 INFO:teuthology.orchestra.run.smithi138.stdout:Total 913 kB/s | 2.2 MB 00:02 2024-05-27T09:15:11.660 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-27T09:15:11.694 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-27T09:15:11.790 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-27T09:15:11.828 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-27T09:15:11.837 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-27T09:15:11.838 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-27T09:15:11.981 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-27T09:15:11.982 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-27T09:15:12.651 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-27T09:15:12.726 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-27T09:15:12.874 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-27T09:15:12.875 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-27T09:15:12.950 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-27T09:15:13.024 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-27T09:15:13.063 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-27T09:15:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-27T09:15:13.244 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-27T09:15:13.388 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-27T09:15:13.461 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-27T09:15:14.369 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-27T09:15:14.429 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-27T09:15:14.449 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-27T09:15:14.463 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-27T09:15:14.516 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-27T09:15:14.547 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-05-27T09:15:15.544 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-05-27T09:15:15.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:18.2.0-0.el9.noarch 1/13 2024-05-27T09:15:15.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-27T09:15:15.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-27T09:15:15.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-27T09:15:15.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-27T09:15:15.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-27T09:15:15.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-27T09:15:15.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-27T09:15:15.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-27T09:15:15.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-27T09:15:15.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-27T09:15:15.546 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-27T09:15:15.615 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-27T09:15:15.651 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-05-27T09:15:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-27T09:15:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:15.963 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-27T09:15:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:18.2.0-0.el9.noarch 2024-05-27T09:15:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T09:15:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T09:15:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T09:15:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T09:15:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T09:15:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T09:15:15.964 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T09:15:15.964 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T09:15:15.964 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T09:15:15.964 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T09:15:15.964 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T09:15:15.964 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T09:15:15.964 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:15.964 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:15:16.283 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-cephadm 2024-05-27T09:15:17.054 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:31 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:15:17.179 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:15:17.180 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-05-27T09:15:17.180 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:17.180 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-27T09:15:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch 2:18.2.0-0.el9 ceph-noarch 127 k 2024-05-27T09:15:17.181 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-05-27T09:15:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T09:15:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T09:15:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T09:15:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T09:15:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T09:15:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:17.181 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-27T09:15:17.181 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:17.181 INFO:teuthology.orchestra.run.smithi138.stdout:Install 6 Packages 2024-05-27T09:15:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:17.182 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 7.0 M 2024-05-27T09:15:17.182 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 33 M 2024-05-27T09:15:17.182 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-27T09:15:17.456 INFO:teuthology.orchestra.run.smithi138.stdout:(1/6): ceph-mgr-cephadm-18.2.0-0.el9.noarch.rpm 1.1 MB/s | 127 kB 00:00 2024-05-27T09:15:17.756 INFO:teuthology.orchestra.run.smithi138.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.8 MB/s | 548 kB 00:00 2024-05-27T09:15:17.831 INFO:teuthology.orchestra.run.smithi138.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 787 kB/s | 58 kB 00:00 2024-05-27T09:15:17.882 INFO:teuthology.orchestra.run.smithi138.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 1.1 MB/s | 51 kB 00:00 2024-05-27T09:15:18.200 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-05-27T09:15:18.200 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 1/12 2024-05-27T09:15:18.201 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-27T09:15:18.201 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-27T09:15:18.201 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-27T09:15:18.201 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-27T09:15:18.201 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-27T09:15:18.201 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-27T09:15:18.201 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-27T09:15:18.201 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-27T09:15:18.201 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-27T09:15:18.201 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-27T09:15:18.248 INFO:teuthology.orchestra.run.smithi138.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 274 kB/s | 248 kB 00:00 2024-05-27T09:15:18.636 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-27T09:15:18.636 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:18.636 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-05-27T09:15:18.636 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarch 2024-05-27T09:15:18.636 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T09:15:18.636 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T09:15:18.636 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T09:15:18.636 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T09:15:18.636 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T09:15:18.636 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T09:15:18.636 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T09:15:18.637 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T09:15:18.637 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T09:15:18.637 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T09:15:18.637 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T09:15:18.637 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:18.637 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:15:18.824 INFO:teuthology.orchestra.run.smithi138.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.0 MB/s | 6.0 MB 00:01 2024-05-27T09:15:18.826 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:15:18.827 INFO:teuthology.orchestra.run.smithi138.stdout:Total 4.2 MB/s | 7.0 MB 00:01 2024-05-27T09:15:18.956 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-27T09:15:18.996 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-27T09:15:18.996 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-27T09:15:19.147 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-mgr-rook 2024-05-27T09:15:19.148 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-27T09:15:19.149 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-27T09:15:19.576 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-27T09:15:19.791 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-27T09:15:19.857 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-27T09:15:19.929 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:35 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:15:20.058 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:15:20.059 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:20.059 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-05-27T09:15:20.059 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:20.059 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-05-27T09:15:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook noarch 2:18.2.0-0.el9 ceph-noarch 47 k 2024-05-27T09:15:20.060 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-05-27T09:15:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T09:15:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T09:15:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T09:15:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T09:15:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T09:15:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T09:15:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T09:15:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T09:15:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T09:15:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T09:15:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T09:15:20.061 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T09:15:20.061 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:20.061 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-05-27T09:15:20.061 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:20.061 INFO:teuthology.orchestra.run.smithi119.stdout:Install 13 Packages 2024-05-27T09:15:20.061 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:20.062 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 2.2 M 2024-05-27T09:15:20.062 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 26 M 2024-05-27T09:15:20.062 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-05-27T09:15:20.223 INFO:teuthology.orchestra.run.smithi119.stdout:(1/13): ceph-mgr-rook-18.2.0-0.el9.noarch.rpm 700 kB/s | 47 kB 00:00 2024-05-27T09:15:20.256 INFO:teuthology.orchestra.run.smithi119.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 253 kB/s | 26 kB 00:00 2024-05-27T09:15:20.281 INFO:teuthology.orchestra.run.smithi119.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 152 kB/s | 19 kB 00:00 2024-05-27T09:15:20.374 INFO:teuthology.orchestra.run.smithi119.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-05-27T09:15:20.424 INFO:teuthology.orchestra.run.smithi119.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.9 MB/s | 279 kB 00:00 2024-05-27T09:15:20.449 INFO:teuthology.orchestra.run.smithi119.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 721 kB/s | 54 kB 00:00 2024-05-27T09:15:20.474 INFO:teuthology.orchestra.run.smithi119.stdout:(7/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 885 kB/s | 222 kB 00:00 2024-05-27T09:15:20.549 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-27T09:15:20.575 INFO:teuthology.orchestra.run.smithi119.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 120 kB/s | 15 kB 00:00 2024-05-27T09:15:20.600 INFO:teuthology.orchestra.run.smithi119.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 184 kB/s | 32 kB 00:00 2024-05-27T09:15:20.657 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-27T09:15:20.692 INFO:teuthology.orchestra.run.smithi119.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 950 kB/s | 206 kB 00:00 2024-05-27T09:15:20.717 INFO:teuthology.orchestra.run.smithi119.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 508 kB/s | 59 kB 00:00 2024-05-27T09:15:20.745 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-05-27T09:15:20.759 INFO:teuthology.orchestra.run.smithi119.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.3 MB/s | 90 kB 00:00 2024-05-27T09:15:20.781 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-05-27T09:15:20.859 INFO:teuthology.orchestra.run.smithi119.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.6 MB/s | 1.0 MB 00:00 2024-05-27T09:15:20.860 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:15:20.861 INFO:teuthology.orchestra.run.smithi119.stdout:Total 2.7 MB/s | 2.2 MB 00:00 2024-05-27T09:15:21.065 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-05-27T09:15:21.111 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-05-27T09:15:21.111 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-05-27T09:15:21.257 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-05-27T09:15:21.257 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-05-27T09:15:21.630 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-05-27T09:15:21.630 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 1/6 2024-05-27T09:15:21.630 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-27T09:15:21.630 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-05-27T09:15:21.630 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-27T09:15:21.631 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-27T09:15:21.902 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-05-27T09:15:21.972 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-27T09:15:21.972 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:21.972 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-27T09:15:21.972 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 2024-05-27T09:15:21.972 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T09:15:21.972 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T09:15:21.972 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T09:15:21.972 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T09:15:21.972 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T09:15:21.973 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:21.973 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:15:22.117 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-27T09:15:22.196 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-27T09:15:22.269 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-27T09:15:22.315 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-27T09:15:22.327 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-fuse 2024-05-27T09:15:22.370 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-27T09:15:22.496 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-27T09:15:22.630 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-27T09:15:22.705 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-27T09:15:23.105 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:37 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:15:23.234 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:15:23.235 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:23.235 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-05-27T09:15:23.235 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:23.235 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-27T09:15:23.235 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:18.2.0-0.el9 ceph 835 k 2024-05-27T09:15:23.235 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-05-27T09:15:23.235 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T09:15:23.235 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:23.236 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-27T09:15:23.236 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:23.236 INFO:teuthology.orchestra.run.smithi138.stdout:Install 2 Packages 2024-05-27T09:15:23.236 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:23.236 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 915 k 2024-05-27T09:15:23.236 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 2.7 M 2024-05-27T09:15:23.236 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-27T09:15:23.412 INFO:teuthology.orchestra.run.smithi138.stdout:(1/2): ceph-fuse-18.2.0-0.el9.x86_64.rpm 7.6 MB/s | 835 kB 00:00 2024-05-27T09:15:23.503 INFO:teuthology.orchestra.run.smithi138.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 404 kB/s | 80 kB 00:00 2024-05-27T09:15:23.505 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:15:23.506 INFO:teuthology.orchestra.run.smithi138.stdout:Total 3.3 MB/s | 915 kB 00:00 2024-05-27T09:15:23.543 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-27T09:15:23.618 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-27T09:15:23.618 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-27T09:15:23.619 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-27T09:15:23.687 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-27T09:15:23.721 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-27T09:15:23.726 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-27T09:15:23.726 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-27T09:15:23.773 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-27T09:15:23.797 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-05-27T09:15:24.036 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-27T09:15:24.212 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-27T09:15:24.291 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-05-27T09:15:24.690 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-05-27T09:15:24.690 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-rook-2:18.2.0-0.el9.noarch 1/13 2024-05-27T09:15:24.690 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-27T09:15:24.690 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-27T09:15:24.691 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-27T09:15:24.691 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-27T09:15:24.691 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-27T09:15:24.691 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-27T09:15:24.691 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-27T09:15:24.691 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-27T09:15:24.691 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-27T09:15:24.691 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-27T09:15:24.691 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-27T09:15:25.115 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-27T09:15:25.116 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:25.116 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-05-27T09:15:25.116 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook-2:18.2.0-0.el9.noarch 2024-05-27T09:15:25.116 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T09:15:25.116 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T09:15:25.116 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T09:15:25.116 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T09:15:25.116 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T09:15:25.116 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T09:15:25.116 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T09:15:25.116 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T09:15:25.116 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T09:15:25.116 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T09:15:25.116 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T09:15:25.117 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T09:15:25.117 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:25.117 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:15:25.213 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-05-27T09:15:25.213 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:18.2.0-0.el9.x86_64 1/2 2024-05-27T09:15:25.451 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-mgr-cephadm 2024-05-27T09:15:25.528 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-27T09:15:25.528 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:25.528 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-27T09:15:25.528 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:18.2.0-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-27T09:15:25.528 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:25.528 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:15:25.892 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librados-devel 2024-05-27T09:15:26.281 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:42 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:15:26.412 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:15:26.413 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:26.413 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-05-27T09:15:26.413 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:26.413 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-05-27T09:15:26.413 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm noarch 2:18.2.0-0.el9 ceph-noarch 127 k 2024-05-27T09:15:26.414 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-05-27T09:15:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T09:15:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T09:15:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T09:15:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T09:15:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T09:15:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:26.414 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-05-27T09:15:26.414 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:26.414 INFO:teuthology.orchestra.run.smithi119.stdout:Install 6 Packages 2024-05-27T09:15:26.414 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:26.415 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 7.0 M 2024-05-27T09:15:26.415 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 33 M 2024-05-27T09:15:26.415 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-05-27T09:15:26.628 INFO:teuthology.orchestra.run.smithi119.stdout:(1/6): ceph-mgr-cephadm-18.2.0-0.el9.noarch.rpm 1.5 MB/s | 127 kB 00:00 2024-05-27T09:15:26.705 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:40 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:15:26.754 INFO:teuthology.orchestra.run.smithi119.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-05-27T09:15:26.855 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:15:26.856 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:26.856 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-05-27T09:15:26.856 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:26.856 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-27T09:15:26.856 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:18.2.0-0.el9 ceph 126 k 2024-05-27T09:15:26.856 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:26.857 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-27T09:15:26.857 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:26.857 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-05-27T09:15:26.857 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:26.857 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 126 k 2024-05-27T09:15:26.857 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 456 k 2024-05-27T09:15:26.858 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-27T09:15:26.932 INFO:teuthology.orchestra.run.smithi138.stdout:librados-devel-18.2.0-0.el9.x86_64.rpm 1.7 MB/s | 126 kB 00:00 2024-05-27T09:15:27.016 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:15:27.016 INFO:teuthology.orchestra.run.smithi138.stdout:Total 1.7 MB/s | 126 kB 00:00 2024-05-27T09:15:27.016 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-27T09:15:27.017 INFO:teuthology.orchestra.run.smithi119.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-05-27T09:15:27.017 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-27T09:15:27.018 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-27T09:15:27.018 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-27T09:15:27.020 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-27T09:15:27.063 INFO:teuthology.orchestra.run.smithi119.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 187 kB/s | 58 kB 00:00 2024-05-27T09:15:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-27T09:15:27.288 INFO:teuthology.orchestra.run.smithi119.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 157 kB/s | 51 kB 00:00 2024-05-27T09:15:27.341 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:27.530 INFO:teuthology.orchestra.run.smithi119.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 609 kB/s | 548 kB 00:00 2024-05-27T09:15:27.530 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:15:27.531 INFO:teuthology.orchestra.run.smithi119.stdout:Total 6.2 MB/s | 7.0 MB 00:01 2024-05-27T09:15:27.655 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-05-27T09:15:27.694 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-05-27T09:15:27.694 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-05-27T09:15:27.845 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-05-27T09:15:27.847 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-05-27T09:15:27.902 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:28.191 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:28.192 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:28.192 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-27T09:15:28.192 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:18.2.0-0.el9.x86_64 2024-05-27T09:15:28.192 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:28.192 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:15:28.263 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-05-27T09:15:28.491 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-27T09:15:28.499 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install libcephfs2 2024-05-27T09:15:28.564 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-27T09:15:29.230 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-27T09:15:29.293 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:43 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:15:29.328 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-27T09:15:29.382 INFO:teuthology.orchestra.run.smithi138.stdout:Package libcephfs2-2:18.2.0-0.el9.x86_64 is already installed. 2024-05-27T09:15:29.408 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-05-27T09:15:29.427 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:15:29.428 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-05-27T09:15:29.428 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:15:29.435 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-05-27T09:15:29.500 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install libcephfs-devel 2024-05-27T09:15:30.250 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:44 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:15:30.300 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-05-27T09:15:30.300 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 1/6 2024-05-27T09:15:30.300 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-27T09:15:30.300 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-05-27T09:15:30.300 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-27T09:15:30.300 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-27T09:15:30.381 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:15:30.382 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:30.382 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-05-27T09:15:30.382 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:30.382 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-27T09:15:30.382 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:18.2.0-0.el9 ceph 30 k 2024-05-27T09:15:30.382 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:30.383 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-27T09:15:30.383 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:30.383 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-05-27T09:15:30.383 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:30.383 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 30 k 2024-05-27T09:15:30.383 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 137 k 2024-05-27T09:15:30.383 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-27T09:15:30.494 INFO:teuthology.orchestra.run.smithi138.stdout:libcephfs-devel-18.2.0-0.el9.x86_64.rpm 275 kB/s | 30 kB 00:00 2024-05-27T09:15:30.495 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:15:30.495 INFO:teuthology.orchestra.run.smithi138.stdout:Total 271 kB/s | 30 kB 00:00 2024-05-27T09:15:30.495 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-27T09:15:30.500 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-27T09:15:30.500 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-27T09:15:30.519 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-27T09:15:30.519 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-27T09:15:30.625 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-27T09:15:30.625 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:30.625 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-05-27T09:15:30.625 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 2024-05-27T09:15:30.625 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T09:15:30.625 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T09:15:30.625 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T09:15:30.625 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T09:15:30.625 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T09:15:30.625 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:30.625 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:15:30.649 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-27T09:15:30.842 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:30.944 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-fuse 2024-05-27T09:15:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:31.625 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:31.625 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:31.625 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-27T09:15:31.625 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:18.2.0-0.el9.x86_64 2024-05-27T09:15:31.625 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:31.625 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:15:31.707 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:47 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:15:31.851 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:15:31.852 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:31.852 INFO:teuthology.orchestra.run.smithi119.stdout: Package Architecture Version Repository Size 2024-05-27T09:15:31.852 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:31.852 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-05-27T09:15:31.852 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse x86_64 2:18.2.0-0.el9 ceph 835 k 2024-05-27T09:15:31.853 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-05-27T09:15:31.853 INFO:teuthology.orchestra.run.smithi119.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T09:15:31.853 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:31.853 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-05-27T09:15:31.853 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:31.853 INFO:teuthology.orchestra.run.smithi119.stdout:Install 2 Packages 2024-05-27T09:15:31.853 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:31.853 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 915 k 2024-05-27T09:15:31.853 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 2.7 M 2024-05-27T09:15:31.853 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-05-27T09:15:31.873 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librados2 2024-05-27T09:15:32.007 INFO:teuthology.orchestra.run.smithi119.stdout:(1/2): ceph-fuse-18.2.0-0.el9.x86_64.rpm 9.3 MB/s | 835 kB 00:00 2024-05-27T09:15:32.057 INFO:teuthology.orchestra.run.smithi119.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 582 kB/s | 80 kB 00:00 2024-05-27T09:15:32.058 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:15:32.058 INFO:teuthology.orchestra.run.smithi119.stdout:Total 4.3 MB/s | 915 kB 00:00 2024-05-27T09:15:32.093 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-05-27T09:15:32.163 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-05-27T09:15:32.163 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-05-27T09:15:32.268 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-05-27T09:15:32.269 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-05-27T09:15:32.525 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-05-27T09:15:32.645 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:46 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:15:32.677 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-27T09:15:32.732 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-2:18.2.0-0.el9.x86_64 is already installed. 2024-05-27T09:15:32.757 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-05-27T09:15:32.775 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:15:32.776 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-05-27T09:15:32.776 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:15:32.850 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librbd1 2024-05-27T09:15:33.570 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-05-27T09:15:33.570 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-fuse-2:18.2.0-0.el9.x86_64 1/2 2024-05-27T09:15:33.605 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:47 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:15:33.693 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-2:18.2.0-0.el9.x86_64 is already installed. 2024-05-27T09:15:33.736 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:15:33.737 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-05-27T09:15:33.737 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:15:33.823 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rados 2024-05-27T09:15:33.853 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-27T09:15:33.854 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:33.854 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-05-27T09:15:33.854 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse-2:18.2.0-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-27T09:15:33.854 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:33.854 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:15:34.180 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install librados-devel 2024-05-27T09:15:34.575 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:48 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:15:34.660 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rados-2:18.2.0-0.el9.x86_64 is already installed. 2024-05-27T09:15:34.702 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:15:34.703 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-05-27T09:15:34.703 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:15:34.769 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rgw 2024-05-27T09:15:34.969 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:50 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:15:35.099 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:15:35.100 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:35.100 INFO:teuthology.orchestra.run.smithi119.stdout: Package Architecture Version Repository Size 2024-05-27T09:15:35.100 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:35.100 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-05-27T09:15:35.100 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel x86_64 2:18.2.0-0.el9 ceph 126 k 2024-05-27T09:15:35.100 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:35.100 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-05-27T09:15:35.100 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:35.101 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2024-05-27T09:15:35.101 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:35.101 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 126 k 2024-05-27T09:15:35.101 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 456 k 2024-05-27T09:15:35.101 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-05-27T09:15:35.179 INFO:teuthology.orchestra.run.smithi119.stdout:librados-devel-18.2.0-0.el9.x86_64.rpm 1.6 MB/s | 126 kB 00:00 2024-05-27T09:15:35.179 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:15:35.179 INFO:teuthology.orchestra.run.smithi119.stdout:Total 1.6 MB/s | 126 kB 00:00 2024-05-27T09:15:35.180 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-05-27T09:15:35.186 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-05-27T09:15:35.186 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-05-27T09:15:35.260 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-05-27T09:15:35.260 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-05-27T09:15:35.430 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-05-27T09:15:35.529 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:49 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:15:35.537 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:35.619 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rgw-2:18.2.0-0.el9.x86_64 is already installed. 2024-05-27T09:15:35.664 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:15:35.665 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-05-27T09:15:35.665 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:15:35.739 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-cephfs 2024-05-27T09:15:36.099 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:36.320 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:36.320 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:36.320 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-05-27T09:15:36.320 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel-2:18.2.0-0.el9.x86_64 2024-05-27T09:15:36.320 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:36.320 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:15:36.525 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:50 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:15:36.615 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-cephfs-2:18.2.0-0.el9.x86_64 is already installed. 2024-05-27T09:15:36.661 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:15:36.662 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-05-27T09:15:36.662 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:15:36.666 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install libcephfs2 2024-05-27T09:15:36.731 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rbd 2024-05-27T09:15:37.450 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:53 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:15:37.455 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:51 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:15:37.542 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rbd-2:18.2.0-0.el9.x86_64 is already installed. 2024-05-27T09:15:37.545 INFO:teuthology.orchestra.run.smithi119.stdout:Package libcephfs2-2:18.2.0-0.el9.x86_64 is already installed. 2024-05-27T09:15:37.584 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:15:37.585 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-05-27T09:15:37.586 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:15:37.589 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:15:37.590 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-05-27T09:15:37.590 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:15:37.649 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-fuse 2024-05-27T09:15:37.670 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install libcephfs-devel 2024-05-27T09:15:38.398 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:52 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:15:38.455 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:54 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:15:38.529 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:15:38.530 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:38.530 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-05-27T09:15:38.531 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:38.531 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-27T09:15:38.531 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:18.2.0-0.el9 ceph 86 k 2024-05-27T09:15:38.531 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:38.531 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-27T09:15:38.531 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:38.531 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-05-27T09:15:38.531 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:38.531 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 86 k 2024-05-27T09:15:38.531 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 230 k 2024-05-27T09:15:38.531 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-27T09:15:38.585 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:15:38.586 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:38.586 INFO:teuthology.orchestra.run.smithi119.stdout: Package Architecture Version Repository Size 2024-05-27T09:15:38.586 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:38.586 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-05-27T09:15:38.586 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel x86_64 2:18.2.0-0.el9 ceph 30 k 2024-05-27T09:15:38.586 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:38.586 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-05-27T09:15:38.586 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:38.587 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2024-05-27T09:15:38.587 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:38.587 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 30 k 2024-05-27T09:15:38.587 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 137 k 2024-05-27T09:15:38.587 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-05-27T09:15:38.607 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-fuse-18.2.0-0.el9.x86_64.rpm 1.1 MB/s | 86 kB 00:00 2024-05-27T09:15:38.608 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:15:38.609 INFO:teuthology.orchestra.run.smithi138.stdout:Total 1.1 MB/s | 86 kB 00:00 2024-05-27T09:15:38.609 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-27T09:15:38.617 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-27T09:15:38.617 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-27T09:15:38.665 INFO:teuthology.orchestra.run.smithi119.stdout:libcephfs-devel-18.2.0-0.el9.x86_64.rpm 394 kB/s | 30 kB 00:00 2024-05-27T09:15:38.666 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:15:38.666 INFO:teuthology.orchestra.run.smithi119.stdout:Total 383 kB/s | 30 kB 00:00 2024-05-27T09:15:38.667 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-05-27T09:15:38.671 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-05-27T09:15:38.671 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-05-27T09:15:38.689 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-05-27T09:15:38.689 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-05-27T09:15:38.691 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-27T09:15:38.692 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-27T09:15:38.795 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-05-27T09:15:38.855 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-27T09:15:38.955 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:38.957 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:39.429 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:39.438 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:39.700 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:39.701 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:39.701 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-05-27T09:15:39.701 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel-2:18.2.0-0.el9.x86_64 2024-05-27T09:15:39.701 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:39.701 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:15:39.748 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:39.748 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:39.748 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-27T09:15:39.748 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:18.2.0-0.el9.x86_64 2024-05-27T09:15:39.748 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:39.748 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:15:39.935 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install librados2 2024-05-27T09:15:40.143 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-mirror 2024-05-27T09:15:40.713 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:56 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:15:40.800 INFO:teuthology.orchestra.run.smithi119.stdout:Package librados2-2:18.2.0-0.el9.x86_64 is already installed. 2024-05-27T09:15:40.843 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:15:40.844 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-05-27T09:15:40.844 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:15:40.924 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install librbd1 2024-05-27T09:15:40.926 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:54 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:15:41.060 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:15:41.060 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:41.060 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-05-27T09:15:41.060 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:41.061 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-27T09:15:41.061 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-05-27T09:15:41.061 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:41.061 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-27T09:15:41.061 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:41.061 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-05-27T09:15:41.061 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:41.062 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 3.0 M 2024-05-27T09:15:41.062 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 12 M 2024-05-27T09:15:41.062 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-27T09:15:41.233 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-mirror-18.2.0-0.el9.x86_64.rpm 17 MB/s | 3.0 MB 00:00 2024-05-27T09:15:41.234 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:15:41.234 INFO:teuthology.orchestra.run.smithi138.stdout:Total 17 MB/s | 3.0 MB 00:00 2024-05-27T09:15:41.235 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-27T09:15:41.250 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-27T09:15:41.250 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-27T09:15:41.356 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-27T09:15:41.357 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-27T09:15:41.679 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:57 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:15:41.777 INFO:teuthology.orchestra.run.smithi119.stdout:Package librbd1-2:18.2.0-0.el9.x86_64 is already installed. 2024-05-27T09:15:41.820 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:15:41.821 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-05-27T09:15:41.821 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:15:41.844 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-27T09:15:41.905 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install python3-rados 2024-05-27T09:15:42.060 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:42.130 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:42.131 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:15:42.131 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T09:15:42.131 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T09:15:42.131 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T09:15:42.131 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:42.671 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:58 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:15:42.757 INFO:teuthology.orchestra.run.smithi119.stdout:Package python3-rados-2:18.2.0-0.el9.x86_64 is already installed. 2024-05-27T09:15:42.801 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:15:42.802 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-05-27T09:15:42.803 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:15:42.881 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install python3-rgw 2024-05-27T09:15:43.473 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:43.473 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:43.473 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-27T09:15:43.473 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:18.2.0-0.el9.x86_64 2024-05-27T09:15:43.473 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:43.473 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:15:43.638 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:59 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:15:43.741 INFO:teuthology.orchestra.run.smithi119.stdout:Package python3-rgw-2:18.2.0-0.el9.x86_64 is already installed. 2024-05-27T09:15:43.786 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:15:43.787 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-05-27T09:15:43.787 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:15:43.829 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-nbd 2024-05-27T09:15:43.865 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install python3-cephfs 2024-05-27T09:15:44.609 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:02:00 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:15:44.632 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:58 ago on Mon 27 May 2024 09:13:46 AM UTC. 2024-05-27T09:15:44.696 INFO:teuthology.orchestra.run.smithi119.stdout:Package python3-cephfs-2:18.2.0-0.el9.x86_64 is already installed. 2024-05-27T09:15:44.740 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:15:44.741 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-05-27T09:15:44.741 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:15:44.765 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-27T09:15:44.765 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:44.765 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-05-27T09:15:44.765 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:44.766 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-27T09:15:44.766 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:18.2.0-0.el9 ceph 169 k 2024-05-27T09:15:44.766 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:44.766 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-27T09:15:44.766 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-05-27T09:15:44.766 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-05-27T09:15:44.766 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:44.767 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 169 k 2024-05-27T09:15:44.767 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 490 k 2024-05-27T09:15:44.767 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-27T09:15:44.808 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install python3-rbd 2024-05-27T09:15:44.875 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-nbd-18.2.0-0.el9.x86_64.rpm 1.5 MB/s | 169 kB 00:00 2024-05-27T09:15:44.876 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:15:44.877 INFO:teuthology.orchestra.run.smithi138.stdout:Total 1.5 MB/s | 169 kB 00:00 2024-05-27T09:15:44.877 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-27T09:15:44.888 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-27T09:15:44.888 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-27T09:15:44.972 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-27T09:15:44.972 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-27T09:15:45.168 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-27T09:15:45.267 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:45.579 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:02:01 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:15:45.664 INFO:teuthology.orchestra.run.smithi119.stdout:Package python3-rbd-2:18.2.0-0.el9.x86_64 is already installed. 2024-05-27T09:15:45.706 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:15:45.707 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-05-27T09:15:45.707 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:15:45.779 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install rbd-fuse 2024-05-27T09:15:45.867 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:46.186 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:46.186 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:46.186 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-27T09:15:46.186 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:18.2.0-0.el9.x86_64 2024-05-27T09:15:46.186 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:15:46.186 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-27T09:15:46.533 DEBUG:teuthology.parallel:result is None 2024-05-27T09:15:46.544 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:02:02 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:15:46.677 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:15:46.678 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:46.678 INFO:teuthology.orchestra.run.smithi119.stdout: Package Architecture Version Repository Size 2024-05-27T09:15:46.678 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:46.678 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-05-27T09:15:46.678 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse x86_64 2:18.2.0-0.el9 ceph 86 k 2024-05-27T09:15:46.678 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:46.679 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-05-27T09:15:46.679 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:46.679 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2024-05-27T09:15:46.679 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:46.679 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 86 k 2024-05-27T09:15:46.679 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 230 k 2024-05-27T09:15:46.680 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-05-27T09:15:46.752 INFO:teuthology.orchestra.run.smithi119.stdout:rbd-fuse-18.2.0-0.el9.x86_64.rpm 1.2 MB/s | 86 kB 00:00 2024-05-27T09:15:46.753 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:15:46.753 INFO:teuthology.orchestra.run.smithi119.stdout:Total 1.1 MB/s | 86 kB 00:00 2024-05-27T09:15:46.753 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-05-27T09:15:46.762 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-05-27T09:15:46.762 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-05-27T09:15:46.836 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-05-27T09:15:46.837 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-05-27T09:15:47.008 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-05-27T09:15:47.141 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:47.618 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:47.908 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:47.909 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:47.909 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-05-27T09:15:47.909 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse-2:18.2.0-0.el9.x86_64 2024-05-27T09:15:47.909 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:47.909 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:15:48.271 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install rbd-mirror 2024-05-27T09:15:49.045 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:02:05 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:15:49.186 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:15:49.187 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:49.187 INFO:teuthology.orchestra.run.smithi119.stdout: Package Architecture Version Repository Size 2024-05-27T09:15:49.187 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:49.187 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-05-27T09:15:49.187 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-05-27T09:15:49.187 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:49.187 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-05-27T09:15:49.187 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:49.188 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2024-05-27T09:15:49.188 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:49.188 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 3.0 M 2024-05-27T09:15:49.188 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 12 M 2024-05-27T09:15:49.188 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-05-27T09:15:49.314 INFO:teuthology.orchestra.run.smithi119.stdout:rbd-mirror-18.2.0-0.el9.x86_64.rpm 24 MB/s | 3.0 MB 00:00 2024-05-27T09:15:49.315 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:15:49.315 INFO:teuthology.orchestra.run.smithi119.stdout:Total 23 MB/s | 3.0 MB 00:00 2024-05-27T09:15:49.315 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-05-27T09:15:49.326 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-05-27T09:15:49.327 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-05-27T09:15:49.417 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-05-27T09:15:49.418 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-05-27T09:15:49.889 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-05-27T09:15:49.956 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:50.029 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:50.029 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T09:15:50.029 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T09:15:50.029 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T09:15:50.029 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T09:15:50.030 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:51.379 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:51.379 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:51.379 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-05-27T09:15:51.379 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror-2:18.2.0-0.el9.x86_64 2024-05-27T09:15:51.379 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:51.379 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:15:51.681 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install rbd-nbd 2024-05-27T09:15:52.467 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:02:08 ago on Mon 27 May 2024 09:13:44 AM UTC. 2024-05-27T09:15:52.597 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-05-27T09:15:52.598 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:52.598 INFO:teuthology.orchestra.run.smithi119.stdout: Package Architecture Version Repository Size 2024-05-27T09:15:52.598 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:52.598 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-05-27T09:15:52.598 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd x86_64 2:18.2.0-0.el9 ceph 169 k 2024-05-27T09:15:52.599 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:52.599 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-05-27T09:15:52.599 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2024-05-27T09:15:52.599 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2024-05-27T09:15:52.599 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:52.599 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 169 k 2024-05-27T09:15:52.599 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 490 k 2024-05-27T09:15:52.599 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-05-27T09:15:52.681 INFO:teuthology.orchestra.run.smithi119.stdout:rbd-nbd-18.2.0-0.el9.x86_64.rpm 2.0 MB/s | 169 kB 00:00 2024-05-27T09:15:52.681 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:15:52.682 INFO:teuthology.orchestra.run.smithi119.stdout:Total 2.0 MB/s | 169 kB 00:00 2024-05-27T09:15:52.682 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-05-27T09:15:52.692 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-05-27T09:15:52.692 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-05-27T09:15:52.766 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-05-27T09:15:52.767 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-05-27T09:15:52.963 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-05-27T09:15:53.082 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:53.666 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:53.974 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-05-27T09:15:53.974 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:53.974 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-05-27T09:15:53.974 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd-2:18.2.0-0.el9.x86_64 2024-05-27T09:15:53.974 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:15:53.975 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-05-27T09:15:54.286 DEBUG:teuthology.parallel:result is None 2024-05-27T09:15:54.286 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-05-27T09:15:54.286 INFO:teuthology.packaging:ref: None 2024-05-27T09:15:54.286 INFO:teuthology.packaging:tag: v18.2.0 2024-05-27T09:15:54.286 INFO:teuthology.packaging:branch: None 2024-05-27T09:15:54.286 INFO:teuthology.packaging:sha1: a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:15:54.287 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-05-27T09:15:54.441 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T09:15:54.512 INFO:teuthology.orchestra.run.smithi119.stdout:18.2.0-0.el9 2024-05-27T09:15:54.513 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-0.el9 2024-05-27T09:15:54.513 INFO:teuthology.task.install:The correct ceph version 18.2.0-0 is installed. 2024-05-27T09:15:54.515 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-05-27T09:15:54.515 INFO:teuthology.packaging:ref: None 2024-05-27T09:15:54.515 INFO:teuthology.packaging:tag: v18.2.0 2024-05-27T09:15:54.515 INFO:teuthology.packaging:branch: None 2024-05-27T09:15:54.515 INFO:teuthology.packaging:sha1: a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:15:54.515 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-05-27T09:15:54.659 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T09:15:54.724 INFO:teuthology.orchestra.run.smithi138.stdout:18.2.0-0.el9 2024-05-27T09:15:54.724 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-0.el9 2024-05-27T09:15:54.724 INFO:teuthology.task.install:The correct ceph version 18.2.0-0 is installed. 2024-05-27T09:15:54.726 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-27T09:15:54.727 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-27T09:15:54.727 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T09:15:54.813 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-27T09:15:54.813 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T09:15:54.902 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-27T09:15:54.903 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-27T09:15:54.903 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T09:15:54.987 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T09:15:55.151 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-27T09:15:55.151 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T09:15:55.242 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T09:15:55.368 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-27T09:15:55.368 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-27T09:15:55.369 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T09:15:55.457 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T09:15:55.585 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-27T09:15:55.585 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T09:15:55.674 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T09:15:55.798 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-27T09:15:55.799 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-27T09:15:55.799 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T09:15:55.887 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T09:15:55.968 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-27T09:15:55.968 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T09:15:56.057 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T09:15:56.137 INFO:teuthology.run_tasks:Running task print... 2024-05-27T09:15:56.148 INFO:teuthology.task.print:**** done install task... 2024-05-27T09:15:56.148 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-27T09:15:56.256 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.io/ceph/ceph:v18.2.0', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d'} 2024-05-27T09:15:56.256 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v18.2.0 2024-05-27T09:15:56.257 INFO:tasks.cephadm:Cluster fsid is b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:15:56.257 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-27T09:15:56.257 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-05-27T09:15:56.257 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi119': '172.21.15.119', 'mon.smithi138': '172.21.15.138'} 2024-05-27T09:15:56.257 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-27T09:15:56.258 DEBUG:teuthology.orchestra.run.smithi119:> sudo hostname $(hostname -s) 2024-05-27T09:15:56.347 DEBUG:teuthology.orchestra.run.smithi138:> sudo hostname $(hostname -s) 2024-05-27T09:15:56.437 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-05-27T09:15:56.437 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:15:56.584 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79602/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3868-ga6507388', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3868.ga6507388'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-21 12:50:55.456707', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240521.105740-debug', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'a6507388dd5057528934822c0163b0c347ef1d5d', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79604/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3868-ga6507388', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3868.ga6507388'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-21 12:34:17.608165', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240521.105740-debug', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240521.105740-debug/a6507388dd5057528934822c0163b0c347ef1d5d/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-27T09:15:56.822 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref reef, sha1 19f3c03194952f81fb0d3dd9621f2ed4b14b4e1d from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-05-27T09:15:56.824 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/reef/19f3c03194952f81fb0d3dd9621f2ed4b14b4e1d/centos/9/x86_64/flavors/default/cephadm 2024-05-27T09:15:56.825 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/reef/19f3c03194952f81fb0d3dd9621f2ed4b14b4e1d/centos/9/x86_64/flavors/default/cephadm 2024-05-27T09:15:56.825 DEBUG:teuthology.orchestra.run.smithi119:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/19f3c03194952f81fb0d3dd9621f2ed4b14b4e1d/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-27T09:15:57.514 INFO:teuthology.orchestra.run.smithi119.stdout:-rw-r--r--. 1 ubuntu ubuntu 215333 May 27 09:15 /home/ubuntu/cephtest/cephadm 2024-05-27T09:15:57.515 DEBUG:teuthology.orchestra.run.smithi138:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/19f3c03194952f81fb0d3dd9621f2ed4b14b4e1d/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-27T09:15:58.208 INFO:teuthology.orchestra.run.smithi138.stdout:-rw-r--r--. 1 ubuntu ubuntu 215333 May 27 09:15 /home/ubuntu/cephtest/cephadm 2024-05-27T09:15:58.209 DEBUG:teuthology.orchestra.run.smithi119:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-27T09:15:58.268 DEBUG:teuthology.orchestra.run.smithi138:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-27T09:15:58.341 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v18.2.0 on all hosts... 2024-05-27T09:15:58.342 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 pull 2024-05-27T09:15:58.349 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 pull 2024-05-27T09:15:59.406 INFO:teuthology.orchestra.run.smithi138.stderr:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-05-27T09:15:59.474 INFO:teuthology.orchestra.run.smithi119.stderr:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-05-27T09:16:30.708 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:16:30.709 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-05-27T09:16:30.709 INFO:teuthology.orchestra.run.smithi119.stdout: "image_id": "dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946", 2024-05-27T09:16:30.709 INFO:teuthology.orchestra.run.smithi119.stdout: "repo_digests": [ 2024-05-27T09:16:30.709 INFO:teuthology.orchestra.run.smithi119.stdout: "quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007", 2024-05-27T09:16:30.709 INFO:teuthology.orchestra.run.smithi119.stdout: "quay.io/ceph/ceph@sha256:d58cf65589d0abf9d5261cd46fb62be3bfb29098febc78c0fcdf116a15274d27" 2024-05-27T09:16:30.709 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-05-27T09:16:30.709 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:16:36.141 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-27T09:16:36.142 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-05-27T09:16:36.142 INFO:teuthology.orchestra.run.smithi138.stdout: "image_id": "dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946", 2024-05-27T09:16:36.142 INFO:teuthology.orchestra.run.smithi138.stdout: "repo_digests": [ 2024-05-27T09:16:36.142 INFO:teuthology.orchestra.run.smithi138.stdout: "quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007", 2024-05-27T09:16:36.142 INFO:teuthology.orchestra.run.smithi138.stdout: "quay.io/ceph/ceph@sha256:d58cf65589d0abf9d5261cd46fb62be3bfb29098febc78c0fcdf116a15274d27" 2024-05-27T09:16:36.142 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-05-27T09:16:36.142 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-27T09:16:36.177 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph 2024-05-27T09:16:36.271 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph 2024-05-27T09:16:36.363 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 777 /etc/ceph 2024-05-27T09:16:36.449 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /etc/ceph 2024-05-27T09:16:36.525 INFO:tasks.cephadm:Writing seed config... 2024-05-27T09:16:36.526 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-05-27T09:16:36.526 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-05-27T09:16:36.526 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-27T09:16:36.526 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-27T09:16:36.526 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-27T09:16:36.526 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-27T09:16:36.526 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-27T09:16:36.526 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-27T09:16:36.527 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-27T09:16:36.527 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-27T09:16:36.527 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-27T09:16:36.527 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-27T09:16:36.527 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-27T09:16:36.527 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-27T09:16:36.527 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-27T09:16:36.527 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-27T09:16:36.527 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-27T09:16:36.527 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-27T09:16:36.527 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-27T09:16:36.527 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-27T09:16:36.527 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-27T09:16:36.527 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-27T09:16:36.527 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-27T09:16:36.528 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-05-27T09:16:36.528 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-27T09:16:36.528 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-27T09:16:36.528 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-27T09:16:36.528 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-27T09:16:36.528 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-27T09:16:36.528 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-27T09:16:36.528 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-27T09:16:36.528 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-27T09:16:36.528 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-27T09:16:36.528 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-27T09:16:36.528 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-27T09:16:36.528 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-27T09:16:36.528 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-27T09:16:36.528 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-27T09:16:36.528 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-27T09:16:36.529 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-27T09:16:36.529 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-27T09:16:36.529 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-27T09:16:36.529 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-27T09:16:36.529 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-27T09:16:36.586 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b966b06e-1c09-11ef-bc9b-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-27T09:16:36.587 DEBUG:teuthology.orchestra.run.smithi119:mon.smithi119> sudo journalctl -f -n 0 -u ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@mon.smithi119.service 2024-05-27T09:16:36.633 INFO:tasks.cephadm:Bootstrapping... 2024-05-27T09:16:36.633 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 -v bootstrap --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.119 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-27T09:16:36.866 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-05-27T09:16:36.866 INFO:teuthology.orchestra.run.smithi119.stdout:cephadm ['--image', 'quay.io/ceph/ceph:v18.2.0', '-v', 'bootstrap', '--fsid', 'b966b06e-1c09-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.119', '--skip-admin-label'] 2024-05-27T09:16:36.923 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stdout 5.0.2 2024-05-27T09:16:36.923 INFO:teuthology.orchestra.run.smithi119.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-27T09:16:36.923 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying podman|docker is present... 2024-05-27T09:16:36.972 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stdout 5.0.2 2024-05-27T09:16:36.972 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying lvm2 is present... 2024-05-27T09:16:36.972 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying time synchronization is in place... 2024-05-27T09:16:36.987 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-27T09:16:36.987 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-27T09:16:37.000 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-27T09:16:37.000 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2024-05-27T09:16:37.015 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout enabled 2024-05-27T09:16:37.028 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout active 2024-05-27T09:16:37.028 INFO:teuthology.orchestra.run.smithi119.stdout:Unit chronyd.service is enabled and running 2024-05-27T09:16:37.028 INFO:teuthology.orchestra.run.smithi119.stdout:Repeating the final host check... 2024-05-27T09:16:37.078 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stdout 5.0.2 2024-05-27T09:16:37.078 INFO:teuthology.orchestra.run.smithi119.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-27T09:16:37.078 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl is present 2024-05-27T09:16:37.078 INFO:teuthology.orchestra.run.smithi119.stdout:lvcreate is present 2024-05-27T09:16:37.092 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-27T09:16:37.092 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-27T09:16:37.107 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-27T09:16:37.108 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2024-05-27T09:16:37.129 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout enabled 2024-05-27T09:16:37.150 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout active 2024-05-27T09:16:37.151 INFO:teuthology.orchestra.run.smithi119.stdout:Unit chronyd.service is enabled and running 2024-05-27T09:16:37.151 INFO:teuthology.orchestra.run.smithi119.stdout:Host looks OK 2024-05-27T09:16:37.151 INFO:teuthology.orchestra.run.smithi119.stdout:Cluster fsid: b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:16:37.152 INFO:teuthology.orchestra.run.smithi119.stdout:Acquiring lock 140250015224112 on /run/cephadm/b966b06e-1c09-11ef-bc9b-c7b262605968.lock 2024-05-27T09:16:37.153 INFO:teuthology.orchestra.run.smithi119.stdout:Lock 140250015224112 acquired on /run/cephadm/b966b06e-1c09-11ef-bc9b-c7b262605968.lock 2024-05-27T09:16:37.154 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying IP 172.21.15.119 port 3300 ... 2024-05-27T09:16:37.155 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying IP 172.21.15.119 port 6789 ... 2024-05-27T09:16:37.155 INFO:teuthology.orchestra.run.smithi119.stdout:Base mon IP(s) is [172.21.15.119:3300, 172.21.15.119:6789], mon addrv is [v2:172.21.15.119:3300,v1:172.21.15.119:6789] 2024-05-27T09:16:37.170 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.119 metric 100 2024-05-27T09:16:37.170 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.119 metric 100 2024-05-27T09:16:37.181 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-27T09:16:37.182 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-27T09:16:37.182 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1797sec hoplimit 64 pref medium 2024-05-27T09:16:37.192 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-27T09:16:37.193 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-27T09:16:37.193 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-27T09:16:37.193 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-27T09:16:37.193 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-05-27T09:16:37.193 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-27T09:16:37.196 INFO:teuthology.orchestra.run.smithi119.stdout:Mon IP `172.21.15.119` is in CIDR network `172.21.0.0/20` 2024-05-27T09:16:37.196 INFO:teuthology.orchestra.run.smithi119.stdout:Mon IP `172.21.15.119` is in CIDR network `172.21.0.0/20` 2024-05-27T09:16:37.196 INFO:teuthology.orchestra.run.smithi119.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-27T09:16:37.199 INFO:teuthology.orchestra.run.smithi119.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-27T09:16:37.201 INFO:teuthology.orchestra.run.smithi119.stdout:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-05-27T09:16:38.169 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stdout dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 2024-05-27T09:16:38.169 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Trying to pull quay.io/ceph/ceph:v18.2.0... 2024-05-27T09:16:38.169 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Getting image source signatures 2024-05-27T09:16:38.169 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Copying blob sha256:46af8f5390d4e94fc57efb422ccb97bb53dfe5b948546bfc191b46557eb2dbd9 2024-05-27T09:16:38.169 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Copying blob sha256:3bd20aeff60302f668275dc2005d10679ae56492967a3a5a54fd3dde85333aec 2024-05-27T09:16:38.169 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Copying config sha256:dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 2024-05-27T09:16:38.170 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-27T09:16:39.380 INFO:teuthology.orchestra.run.smithi119.stdout:ceph: stdout ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable) 2024-05-27T09:16:39.380 INFO:teuthology.orchestra.run.smithi119.stdout:Ceph version: ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable) 2024-05-27T09:16:39.380 INFO:teuthology.orchestra.run.smithi119.stdout:Extracting ceph user uid/gid from container image... 2024-05-27T09:16:49.098 INFO:teuthology.orchestra.run.smithi119.stdout:stat: stdout 167 167 2024-05-27T09:16:49.098 INFO:teuthology.orchestra.run.smithi119.stdout:Creating initial keys... 2024-05-27T09:16:50.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph-authtool: stdout AQCBT1RmKpImIhAAmAN3mHounhO5luiTEW1tFQ== 2024-05-27T09:16:51.342 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph-authtool: stdout AQCCT1RmEax5KhAAIjlhnnMqvZIE4IbFe6WPig== 2024-05-27T09:16:52.517 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph-authtool: stdout AQCDT1Rmb+80MhAAceg3Pl0oPdE1ccK3rp8UDw== 2024-05-27T09:16:52.520 INFO:teuthology.orchestra.run.smithi119.stdout:Creating initial monmap... 2024-05-27T09:16:53.668 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-27T09:16:53.669 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-05-27T09:16:53.669 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:16:53.669 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-27T09:16:53.669 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool for smithi119 [v2:172.21.15.119:3300,v1:172.21.15.119:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-27T09:16:53.669 INFO:teuthology.orchestra.run.smithi119.stdout:setting min_mon_release = pacific 2024-05-27T09:16:53.669 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: set fsid to b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:16:53.669 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-27T09:16:53.669 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:16:53.670 INFO:teuthology.orchestra.run.smithi119.stdout:Creating mon... 2024-05-27T09:16:55.684 INFO:teuthology.orchestra.run.smithi119.stdout:create mon.smithi119 on 2024-05-27T09:16:56.055 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-27T09:16:56.300 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-27T09:16:56.600 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b966b06e-1c09-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-b966b06e-1c09-11ef-bc9b-c7b262605968.target. 2024-05-27T09:16:56.600 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b966b06e-1c09-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-b966b06e-1c09-11ef-bc9b-c7b262605968.target. 2024-05-27T09:16:56.978 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@mon.smithi119 2024-05-27T09:16:56.978 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@mon.smithi119.service: Unit ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@mon.smithi119.service not loaded. 2024-05-27T09:16:57.283 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b966b06e-1c09-11ef-bc9b-c7b262605968.target.wants/ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@mon.smithi119.service → /etc/systemd/system/ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@.service. 2024-05-27T09:16:57.932 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:16:57 smithi119 podman[28689]: 2024-05-27 09:16:57.919482994 +0000 UTC m=+0.373564001 container init 472e07abdf320991201cf56c61addae9b3b42bdc0b37f349eef9855f8b8a9438 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, ceph=True, GIT_BRANCH=HEAD, RELEASE=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1) 2024-05-27T09:16:57.933 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:16:57 smithi119 podman[28689]: 2024-05-27 09:16:57.931599245 +0000 UTC m=+0.385680218 container start 472e07abdf320991201cf56c61addae9b3b42bdc0b37f349eef9855f8b8a9438 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20231212, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, ceph=True, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-05-27T09:16:58.078 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T09:16:58.078 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout disabled 2024-05-27T09:16:58.098 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T09:16:58.098 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2024-05-27T09:16:58.099 INFO:teuthology.orchestra.run.smithi119.stdout:firewalld.service is not enabled 2024-05-27T09:16:58.099 INFO:teuthology.orchestra.run.smithi119.stdout:Not possible to enable service . firewalld.service is not available 2024-05-27T09:16:58.100 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mon to start... 2024-05-27T09:16:58.100 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mon... 2024-05-27T09:16:58.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:16:58 smithi119 ceph-mon[28708]: mkfs b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:16:58.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:16:58 smithi119 ceph-mon[28708]: mon.smithi119 is new leader, mons smithi119 in quorum (ranks 0) 2024-05-27T09:16:58.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:16:58 smithi119 bash[28689]: 472e07abdf320991201cf56c61addae9b3b42bdc0b37f349eef9855f8b8a9438 2024-05-27T09:16:58.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:16:58 smithi119 systemd[1]: Started Ceph mon.smithi119 for b966b06e-1c09-11ef-bc9b-c7b262605968. 2024-05-27T09:16:59.152 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:16:59 smithi119 ceph-mon[28708]: mon.smithi119 is new leader, mons smithi119 in quorum (ranks 0) 2024-05-27T09:16:59.153 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:16:59 smithi119 ceph-mon[28708]: monmap e1: 1 mons at {smithi119=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0]} removed_ranks: {} 2024-05-27T09:16:59.153 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:16:59 smithi119 ceph-mon[28708]: fsmap 2024-05-27T09:16:59.153 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:16:59 smithi119 ceph-mon[28708]: osdmap e1: 0 total, 0 up, 0 in 2024-05-27T09:16:59.153 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:16:59 smithi119 ceph-mon[28708]: mgrmap e1: no daemons active 2024-05-27T09:16:59.153 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:16:59 smithi119 ceph-mon[28708]: from='client.? 172.21.15.119:0/3114602409' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-27T09:16:59.630 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout cluster: 2024-05-27T09:16:59.630 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout id: b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:16:59.631 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-27T09:16:59.631 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-05-27T09:16:59.631 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout services: 2024-05-27T09:16:59.631 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi119 (age 0.846675s) 2024-05-27T09:16:59.631 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-27T09:16:59.631 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-27T09:16:59.631 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-05-27T09:16:59.631 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout data: 2024-05-27T09:16:59.631 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-27T09:16:59.631 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-27T09:16:59.631 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-27T09:16:59.631 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout pgs: 2024-05-27T09:16:59.631 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-05-27T09:16:59.631 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.759+0000 7fd31b400700 1 Processor -- start 2024-05-27T09:16:59.632 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.760+0000 7fd31b400700 1 -- start start 2024-05-27T09:16:59.632 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.760+0000 7fd31b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd314107d00 0x7fd314108110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:16:59.632 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.760+0000 7fd31b400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd314108650 con 0x7fd314107d00 2024-05-27T09:16:59.632 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.761+0000 7fd318a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd314107d00 0x7fd314108110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:16:59.632 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.761+0000 7fd318a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd314107d00 0x7fd314108110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60854/0 (socket says 172.21.15.119:60854) 2024-05-27T09:16:59.632 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.761+0000 7fd318a00700 1 -- 172.21.15.119:0/3519370630 learned_addr learned my addr 172.21.15.119:0/3519370630 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:16:59.632 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.762+0000 7fd318a00700 1 -- 172.21.15.119:0/3519370630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd314108790 con 0x7fd314107d00 2024-05-27T09:16:59.632 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.762+0000 7fd318a00700 1 --2- 172.21.15.119:0/3519370630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd314107d00 0x7fd314108110 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd3080098a0 tx=0x7fd308009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5b34b9345aa56f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:16:59.632 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.762+0000 7fd313400700 1 -- 172.21.15.119:0/3519370630 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd308004180 con 0x7fd314107d00 2024-05-27T09:16:59.632 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.762+0000 7fd313400700 1 -- 172.21.15.119:0/3519370630 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3080042e0 con 0x7fd314107d00 2024-05-27T09:16:59.632 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.762+0000 7fd313400700 1 -- 172.21.15.119:0/3519370630 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd3080036a0 con 0x7fd314107d00 2024-05-27T09:16:59.632 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.763+0000 7fd31b400700 1 -- 172.21.15.119:0/3519370630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd314107d00 msgr2=0x7fd314108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:16:59.633 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.763+0000 7fd31b400700 1 --2- 172.21.15.119:0/3519370630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd314107d00 0x7fd314108110 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd3080098a0 tx=0x7fd308009bb0 comp rx=0 tx=0).stop 2024-05-27T09:16:59.633 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.763+0000 7fd31b400700 1 -- 172.21.15.119:0/3519370630 shutdown_connections 2024-05-27T09:16:59.633 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.763+0000 7fd31b400700 1 --2- 172.21.15.119:0/3519370630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd314107d00 0x7fd314108110 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:16:59.633 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.763+0000 7fd31b400700 1 -- 172.21.15.119:0/3519370630 >> 172.21.15.119:0/3519370630 conn(0x7fd31407b220 msgr2=0x7fd31407b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:16:59.633 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.763+0000 7fd31b400700 1 -- 172.21.15.119:0/3519370630 shutdown_connections 2024-05-27T09:16:59.633 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.763+0000 7fd31b400700 1 -- 172.21.15.119:0/3519370630 wait complete. 2024-05-27T09:16:59.633 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.764+0000 7fd31b400700 1 Processor -- start 2024-05-27T09:16:59.633 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.764+0000 7fd31b400700 1 -- start start 2024-05-27T09:16:59.633 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.765+0000 7fd31b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd314107d00 0x7fd31419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:16:59.633 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.765+0000 7fd31b400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31419b750 con 0x7fd314107d00 2024-05-27T09:16:59.633 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.765+0000 7fd318a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd314107d00 0x7fd31419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:16:59.633 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.765+0000 7fd318a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd314107d00 0x7fd31419b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60868/0 (socket says 172.21.15.119:60868) 2024-05-27T09:16:59.633 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.765+0000 7fd318a00700 1 -- 172.21.15.119:0/3114602409 learned_addr learned my addr 172.21.15.119:0/3114602409 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:16:59.634 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.766+0000 7fd318a00700 1 -- 172.21.15.119:0/3114602409 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd308009550 con 0x7fd314107d00 2024-05-27T09:16:59.634 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.766+0000 7fd318a00700 1 --2- 172.21.15.119:0/3114602409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd314107d00 0x7fd31419b210 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd314108a30 tx=0x7fd308004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:16:59.634 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.766+0000 7fd311600700 1 -- 172.21.15.119:0/3114602409 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd308003ce0 con 0x7fd314107d00 2024-05-27T09:16:59.634 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.766+0000 7fd311600700 1 -- 172.21.15.119:0/3114602409 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd308020070 con 0x7fd314107d00 2024-05-27T09:16:59.634 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.766+0000 7fd31b400700 1 -- 172.21.15.119:0/3114602409 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd31419b950 con 0x7fd314107d00 2024-05-27T09:16:59.634 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.766+0000 7fd311600700 1 -- 172.21.15.119:0/3114602409 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd308011630 con 0x7fd314107d00 2024-05-27T09:16:59.634 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.766+0000 7fd31b400700 1 -- 172.21.15.119:0/3114602409 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd31419bd70 con 0x7fd314107d00 2024-05-27T09:16:59.634 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.767+0000 7fd311600700 1 -- 172.21.15.119:0/3114602409 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd308022030 con 0x7fd314107d00 2024-05-27T09:16:59.634 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.767+0000 7fd311600700 1 -- 172.21.15.119:0/3114602409 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd308025d00 con 0x7fd314107d00 2024-05-27T09:16:59.634 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.768+0000 7fd31b400700 1 -- 172.21.15.119:0/3114602409 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd314194a50 con 0x7fd314107d00 2024-05-27T09:16:59.634 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.771+0000 7fd311600700 1 -- 172.21.15.119:0/3114602409 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fd308025480 con 0x7fd314107d00 2024-05-27T09:16:59.634 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.850+0000 7fd31b400700 1 -- 172.21.15.119:0/3114602409 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fd31402cc30 con 0x7fd314107d00 2024-05-27T09:16:59.634 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.850+0000 7fd311600700 1 -- 172.21.15.119:0/3114602409 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7fd308011c20 con 0x7fd314107d00 2024-05-27T09:16:59.635 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.852+0000 7fd31b400700 1 -- 172.21.15.119:0/3114602409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd314107d00 msgr2=0x7fd31419b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:16:59.635 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.852+0000 7fd31b400700 1 --2- 172.21.15.119:0/3114602409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd314107d00 0x7fd31419b210 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd314108a30 tx=0x7fd308004750 comp rx=0 tx=0).stop 2024-05-27T09:16:59.635 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.852+0000 7fd31b400700 1 -- 172.21.15.119:0/3114602409 shutdown_connections 2024-05-27T09:16:59.635 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.852+0000 7fd31b400700 1 --2- 172.21.15.119:0/3114602409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd314107d00 0x7fd31419b210 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:16:59.635 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.852+0000 7fd31b400700 1 -- 172.21.15.119:0/3114602409 >> 172.21.15.119:0/3114602409 conn(0x7fd31407b220 msgr2=0x7fd31418f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:16:59.635 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.852+0000 7fd31b400700 1 -- 172.21.15.119:0/3114602409 shutdown_connections 2024-05-27T09:16:59.635 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:16:58.852+0000 7fd31b400700 1 -- 172.21.15.119:0/3114602409 wait complete. 2024-05-27T09:16:59.635 INFO:teuthology.orchestra.run.smithi119.stdout:mon is available 2024-05-27T09:16:59.635 INFO:teuthology.orchestra.run.smithi119.stdout:Assimilating anything we can from ceph.conf... 2024-05-27T09:17:00.811 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:00 smithi119 ceph-mon[28708]: from='client.? 172.21.15.119:0/2149055716' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-27T09:17:00.811 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:00 smithi119 ceph-mon[28708]: from='client.? 172.21.15.119:0/2149055716' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-27T09:17:01.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-05-27T09:17:01.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [global] 2024-05-27T09:17:01.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout fsid = b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:17:01.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.119:3300,v1:172.21.15.119:6789] 2024-05-27T09:17:01.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-27T09:17:01.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-27T09:17:01.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-27T09:17:01.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-27T09:17:01.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-05-27T09:17:01.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-27T09:17:01.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-27T09:17:01.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-05-27T09:17:01.190 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [osd] 2024-05-27T09:17:01.190 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-27T09:17:01.190 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-27T09:17:01.190 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-27T09:17:01.190 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-27T09:17:01.190 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-27T09:17:01.190 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.408+0000 7fe99e800700 1 Processor -- start 2024-05-27T09:17:01.190 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.408+0000 7fe99e800700 1 -- start start 2024-05-27T09:17:01.190 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.408+0000 7fe99e800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9981064b0 0x7fe998108890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:01.191 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.408+0000 7fe99e800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9980744b0 con 0x7fe9981064b0 2024-05-27T09:17:01.191 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.409+0000 7fe99d400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9981064b0 0x7fe998108890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:01.191 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.409+0000 7fe99d400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9981064b0 0x7fe998108890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46986/0 (socket says 172.21.15.119:46986) 2024-05-27T09:17:01.191 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.409+0000 7fe99d400700 1 -- 172.21.15.119:0/2437269944 learned_addr learned my addr 172.21.15.119:0/2437269944 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:01.191 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.410+0000 7fe99d400700 1 -- 172.21.15.119:0/2437269944 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9980745f0 con 0x7fe9981064b0 2024-05-27T09:17:01.191 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.410+0000 7fe99d400700 1 --2- 172.21.15.119:0/2437269944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9981064b0 0x7fe998108890 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0098a0 tx=0x7fe98c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995b19a4f277418b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:01.191 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.410+0000 7fe997e00700 1 -- 172.21.15.119:0/2437269944 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe98c004180 con 0x7fe9981064b0 2024-05-27T09:17:01.191 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.410+0000 7fe997e00700 1 -- 172.21.15.119:0/2437269944 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98c0042e0 con 0x7fe9981064b0 2024-05-27T09:17:01.191 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.410+0000 7fe997e00700 1 -- 172.21.15.119:0/2437269944 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe98c0036a0 con 0x7fe9981064b0 2024-05-27T09:17:01.191 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.411+0000 7fe99e800700 1 -- 172.21.15.119:0/2437269944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9981064b0 msgr2=0x7fe998108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:01.191 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.411+0000 7fe99e800700 1 --2- 172.21.15.119:0/2437269944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9981064b0 0x7fe998108890 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0098a0 tx=0x7fe98c009bb0 comp rx=0 tx=0).stop 2024-05-27T09:17:01.192 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.411+0000 7fe99e800700 1 -- 172.21.15.119:0/2437269944 shutdown_connections 2024-05-27T09:17:01.192 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.411+0000 7fe99e800700 1 --2- 172.21.15.119:0/2437269944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9981064b0 0x7fe998108890 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:01.192 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.411+0000 7fe99e800700 1 -- 172.21.15.119:0/2437269944 >> 172.21.15.119:0/2437269944 conn(0x7fe9980ffeb0 msgr2=0x7fe9981022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:01.192 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.411+0000 7fe99e800700 1 -- 172.21.15.119:0/2437269944 shutdown_connections 2024-05-27T09:17:01.192 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.411+0000 7fe99e800700 1 -- 172.21.15.119:0/2437269944 wait complete. 2024-05-27T09:17:01.192 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.412+0000 7fe99e800700 1 Processor -- start 2024-05-27T09:17:01.192 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.413+0000 7fe99e800700 1 -- start start 2024-05-27T09:17:01.192 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.413+0000 7fe99e800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9981064b0 0x7fe998198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:01.192 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.413+0000 7fe99e800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9981993d0 con 0x7fe9981064b0 2024-05-27T09:17:01.192 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.413+0000 7fe99d400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9981064b0 0x7fe998198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:01.192 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.413+0000 7fe99d400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9981064b0 0x7fe998198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47000/0 (socket says 172.21.15.119:47000) 2024-05-27T09:17:01.192 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.413+0000 7fe99d400700 1 -- 172.21.15.119:0/2149055716 learned_addr learned my addr 172.21.15.119:0/2149055716 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:01.192 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.414+0000 7fe99d400700 1 -- 172.21.15.119:0/2149055716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c009550 con 0x7fe9981064b0 2024-05-27T09:17:01.192 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.414+0000 7fe99d400700 1 --2- 172.21.15.119:0/2149055716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9981064b0 0x7fe998198e90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe98c004000 tx=0x7fe98c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:01.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.414+0000 7fe996000700 1 -- 172.21.15.119:0/2149055716 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe98c003ce0 con 0x7fe9981064b0 2024-05-27T09:17:01.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.414+0000 7fe996000700 1 -- 172.21.15.119:0/2149055716 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98c020070 con 0x7fe9981064b0 2024-05-27T09:17:01.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.414+0000 7fe996000700 1 -- 172.21.15.119:0/2149055716 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe98c011630 con 0x7fe9981064b0 2024-05-27T09:17:01.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.414+0000 7fe99e800700 1 -- 172.21.15.119:0/2149055716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9981995d0 con 0x7fe9981064b0 2024-05-27T09:17:01.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.415+0000 7fe99e800700 1 -- 172.21.15.119:0/2149055716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9981999f0 con 0x7fe9981064b0 2024-05-27T09:17:01.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.415+0000 7fe996000700 1 -- 172.21.15.119:0/2149055716 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe98c022030 con 0x7fe9981064b0 2024-05-27T09:17:01.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.415+0000 7fe996000700 1 -- 172.21.15.119:0/2149055716 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe98c025d40 con 0x7fe9981064b0 2024-05-27T09:17:01.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.416+0000 7fe99e800700 1 -- 172.21.15.119:0/2149055716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe998192c20 con 0x7fe9981064b0 2024-05-27T09:17:01.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.418+0000 7fe996000700 1 -- 172.21.15.119:0/2149055716 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fe98c025480 con 0x7fe9981064b0 2024-05-27T09:17:01.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.496+0000 7fe99e800700 1 -- 172.21.15.119:0/2149055716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fe99802ce60 con 0x7fe9981064b0 2024-05-27T09:17:01.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.502+0000 7fe996000700 1 -- 172.21.15.119:0/2149055716 <== mon.0 v2:172.21.15.119:3300/0 7 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe98c011bd0 con 0x7fe9981064b0 2024-05-27T09:17:01.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.502+0000 7fe996000700 1 -- 172.21.15.119:0/2149055716 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7fe98c036020 con 0x7fe9981064b0 2024-05-27T09:17:01.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.504+0000 7fe99e800700 1 -- 172.21.15.119:0/2149055716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9981064b0 msgr2=0x7fe998198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:01.194 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.504+0000 7fe99e800700 1 --2- 172.21.15.119:0/2149055716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9981064b0 0x7fe998198e90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe98c004000 tx=0x7fe98c004750 comp rx=0 tx=0).stop 2024-05-27T09:17:01.194 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.505+0000 7fe99e800700 1 -- 172.21.15.119:0/2149055716 shutdown_connections 2024-05-27T09:17:01.194 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.505+0000 7fe99e800700 1 --2- 172.21.15.119:0/2149055716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9981064b0 0x7fe998198e90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:01.194 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.505+0000 7fe99e800700 1 -- 172.21.15.119:0/2149055716 >> 172.21.15.119:0/2149055716 conn(0x7fe9980ffeb0 msgr2=0x7fe998101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:01.194 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.505+0000 7fe99e800700 1 -- 172.21.15.119:0/2149055716 shutdown_connections 2024-05-27T09:17:01.194 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:00.505+0000 7fe99e800700 1 -- 172.21.15.119:0/2149055716 wait complete. 2024-05-27T09:17:01.194 INFO:teuthology.orchestra.run.smithi119.stdout:Generating new minimal ceph.conf... 2024-05-27T09:17:02.156 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:01 smithi119 ceph-mon[28708]: from='client.? 172.21.15.119:0/2456958893' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:17:02.446 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.746+0000 7f93f3e00700 1 Processor -- start 2024-05-27T09:17:02.447 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.746+0000 7f93f3e00700 1 -- start start 2024-05-27T09:17:02.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.746+0000 7f93f3e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93ec106610 0x7f93ec1089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:02.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.746+0000 7f93f3e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec074510 con 0x7f93ec106610 2024-05-27T09:17:02.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.747+0000 7f93f1400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93ec106610 0x7f93ec1089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:02.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.747+0000 7f93f1400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93ec106610 0x7f93ec1089f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47004/0 (socket says 172.21.15.119:47004) 2024-05-27T09:17:02.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.747+0000 7f93f1400700 1 -- 172.21.15.119:0/1481339284 learned_addr learned my addr 172.21.15.119:0/1481339284 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:02.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.748+0000 7f93f1400700 1 -- 172.21.15.119:0/1481339284 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ec074650 con 0x7f93ec106610 2024-05-27T09:17:02.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.748+0000 7f93f1400700 1 --2- 172.21.15.119:0/1481339284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93ec106610 0x7f93ec1089f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f93e00098a0 tx=0x7f93e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b0ce19a8934adb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:02.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.748+0000 7f93ebe00700 1 -- 172.21.15.119:0/1481339284 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f93e0004180 con 0x7f93ec106610 2024-05-27T09:17:02.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.748+0000 7f93ebe00700 1 -- 172.21.15.119:0/1481339284 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f93e00042e0 con 0x7f93ec106610 2024-05-27T09:17:02.451 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.749+0000 7f93ebe00700 1 -- 172.21.15.119:0/1481339284 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f93e0011a50 con 0x7f93ec106610 2024-05-27T09:17:02.452 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.749+0000 7f93f3e00700 1 -- 172.21.15.119:0/1481339284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93ec106610 msgr2=0x7f93ec1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:02.452 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.749+0000 7f93f3e00700 1 --2- 172.21.15.119:0/1481339284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93ec106610 0x7f93ec1089f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f93e00098a0 tx=0x7f93e0009bb0 comp rx=0 tx=0).stop 2024-05-27T09:17:02.452 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.749+0000 7f93f3e00700 1 -- 172.21.15.119:0/1481339284 shutdown_connections 2024-05-27T09:17:02.452 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.749+0000 7f93f3e00700 1 --2- 172.21.15.119:0/1481339284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93ec106610 0x7f93ec1089f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:02.452 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.749+0000 7f93f3e00700 1 -- 172.21.15.119:0/1481339284 >> 172.21.15.119:0/1481339284 conn(0x7f93ec100070 msgr2=0x7f93ec102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:02.452 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.750+0000 7f93f3e00700 1 -- 172.21.15.119:0/1481339284 shutdown_connections 2024-05-27T09:17:02.452 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.750+0000 7f93f3e00700 1 -- 172.21.15.119:0/1481339284 wait complete. 2024-05-27T09:17:02.452 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.751+0000 7f93f3e00700 1 Processor -- start 2024-05-27T09:17:02.452 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.751+0000 7f93f3e00700 1 -- start start 2024-05-27T09:17:02.452 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.751+0000 7f93f3e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93ec106610 0x7f93ec105f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:02.452 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.751+0000 7f93f3e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec1064b0 con 0x7f93ec106610 2024-05-27T09:17:02.453 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.751+0000 7f93f1400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93ec106610 0x7f93ec105f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:02.453 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.751+0000 7f93f1400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93ec106610 0x7f93ec105f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47020/0 (socket says 172.21.15.119:47020) 2024-05-27T09:17:02.453 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.751+0000 7f93f1400700 1 -- 172.21.15.119:0/2456958893 learned_addr learned my addr 172.21.15.119:0/2456958893 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:02.453 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.752+0000 7f93f1400700 1 -- 172.21.15.119:0/2456958893 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e0009550 con 0x7f93ec106610 2024-05-27T09:17:02.453 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.752+0000 7f93f1400700 1 --2- 172.21.15.119:0/2456958893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93ec106610 0x7f93ec105f70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f93e0000c00 tx=0x7f93e0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:02.453 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.752+0000 7f93ea000700 1 -- 172.21.15.119:0/2456958893 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f93e0003b90 con 0x7f93ec106610 2024-05-27T09:17:02.453 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.752+0000 7f93ea000700 1 -- 172.21.15.119:0/2456958893 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f93e00248b0 con 0x7f93ec106610 2024-05-27T09:17:02.453 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.752+0000 7f93ea000700 1 -- 172.21.15.119:0/2456958893 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f93e001a8e0 con 0x7f93ec106610 2024-05-27T09:17:02.453 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.752+0000 7f93f3e00700 1 -- 172.21.15.119:0/2456958893 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93ec1045c0 con 0x7f93ec106610 2024-05-27T09:17:02.453 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.752+0000 7f93f3e00700 1 -- 172.21.15.119:0/2456958893 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ec1049e0 con 0x7f93ec106610 2024-05-27T09:17:02.453 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.753+0000 7f93ea000700 1 -- 172.21.15.119:0/2456958893 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f93e002d3f0 con 0x7f93ec106610 2024-05-27T09:17:02.453 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.753+0000 7f93ea000700 1 -- 172.21.15.119:0/2456958893 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f93e00194e0 con 0x7f93ec106610 2024-05-27T09:17:02.453 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.754+0000 7f93f3e00700 1 -- 172.21.15.119:0/2456958893 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93ec109b80 con 0x7f93ec106610 2024-05-27T09:17:02.453 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.756+0000 7f93ea000700 1 -- 172.21.15.119:0/2456958893 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f93e0024a20 con 0x7f93ec106610 2024-05-27T09:17:02.454 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.834+0000 7f93f3e00700 1 -- 172.21.15.119:0/2456958893 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f93ec105040 con 0x7f93ec106610 2024-05-27T09:17:02.454 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.835+0000 7f93ea000700 1 -- 172.21.15.119:0/2456958893 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7f93e0035090 con 0x7f93ec106610 2024-05-27T09:17:02.454 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.836+0000 7f93f3e00700 1 -- 172.21.15.119:0/2456958893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93ec106610 msgr2=0x7f93ec105f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:02.454 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.836+0000 7f93f3e00700 1 --2- 172.21.15.119:0/2456958893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93ec106610 0x7f93ec105f70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f93e0000c00 tx=0x7f93e0011ab0 comp rx=0 tx=0).stop 2024-05-27T09:17:02.454 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.836+0000 7f93f3e00700 1 -- 172.21.15.119:0/2456958893 shutdown_connections 2024-05-27T09:17:02.454 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.836+0000 7f93f3e00700 1 --2- 172.21.15.119:0/2456958893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93ec106610 0x7f93ec105f70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:02.454 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.836+0000 7f93f3e00700 1 -- 172.21.15.119:0/2456958893 >> 172.21.15.119:0/2456958893 conn(0x7f93ec100070 msgr2=0x7f93ec101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:02.454 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.836+0000 7f93f3e00700 1 -- 172.21.15.119:0/2456958893 shutdown_connections 2024-05-27T09:17:02.454 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:01.836+0000 7f93f3e00700 1 -- 172.21.15.119:0/2456958893 wait complete. 2024-05-27T09:17:02.454 INFO:teuthology.orchestra.run.smithi119.stdout:Restarting the monitor... 2024-05-27T09:17:02.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:02 smithi119 systemd[1]: Stopping Ceph mon.smithi119 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:17:03.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:02 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119[28704]: 2024-05-27T09:17:02.919+0000 7fe94d200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi119 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T09:17:03.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:02 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119[28704]: 2024-05-27T09:17:02.919+0000 7fe94d200700 -1 mon.smithi119@0(leader) e1 *** Got Signal Terminated *** 2024-05-27T09:17:03.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:03 smithi119 podman[28961]: 2024-05-27 09:17:03.065548033 +0000 UTC m=+0.274179610 container died 472e07abdf320991201cf56c61addae9b3b42bdc0b37f349eef9855f8b8a9438 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD, io.buildah.version=1.29.1) 2024-05-27T09:17:03.665 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:03 smithi119 podman[28961]: 2024-05-27 09:17:03.464473338 +0000 UTC m=+0.673104973 container cleanup 472e07abdf320991201cf56c61addae9b3b42bdc0b37f349eef9855f8b8a9438 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119, org.label-schema.build-date=20231212, GIT_BRANCH=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, ceph=True, org.label-schema.schema-version=1.0, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T09:17:03.666 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:03 smithi119 bash[28961]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119 2024-05-27T09:17:03.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:03 smithi119 podman[28974]: 2024-05-27 09:17:03.665002969 +0000 UTC m=+0.576196104 container remove 472e07abdf320991201cf56c61addae9b3b42bdc0b37f349eef9855f8b8a9438 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=HEAD, GIT_BRANCH=HEAD) 2024-05-27T09:17:04.425 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 systemd[1]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@mon.smithi119.service: Deactivated successfully. 2024-05-27T09:17:04.425 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 systemd[1]: Stopped Ceph mon.smithi119 for b966b06e-1c09-11ef-bc9b-c7b262605968. 2024-05-27T09:17:04.425 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 systemd[1]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@mon.smithi119.service: Consumed 1.795s CPU time. 2024-05-27T09:17:04.425 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 systemd[1]: Starting Ceph mon.smithi119 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:17:04.728 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 podman[29064]: 2024-05-27 09:17:04.329863189 +0000 UTC m=+0.034465214 image pull dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 quay.io/ceph/ceph:v18.2.0 2024-05-27T09:17:04.728 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 podman[29064]: 2024-05-27 09:17:04.44954492 +0000 UTC m=+0.154146962 container create ba998b789a1c708b9fa6ba9962892e24c8488192c708f59682be39479f0dd0c8 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, org.label-schema.build-date=20231212, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T09:17:04.728 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 podman[29064]: 2024-05-27 09:17:04.727874193 +0000 UTC m=+0.432476238 container init ba998b789a1c708b9fa6ba9962892e24c8488192c708f59682be39479f0dd0c8 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119, RELEASE=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20231212, maintainer=Guillaume Abrioux ) 2024-05-27T09:17:04.838 INFO:teuthology.orchestra.run.smithi119.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-27T09:17:04.991 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 podman[29064]: 2024-05-27 09:17:04.738560855 +0000 UTC m=+0.443162927 container start ba998b789a1c708b9fa6ba9962892e24c8488192c708f59682be39479f0dd0c8 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=-18.2.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=HEAD, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS) 2024-05-27T09:17:04.991 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: set uid:gid to 167:167 (ceph:ceph) 2024-05-27T09:17:04.991 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable), process ceph-mon, pid 2 2024-05-27T09:17:04.991 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: pidfile_write: ignore empty --pid-file 2024-05-27T09:17:04.991 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: load: jerasure load: lrc 2024-05-27T09:17:04.992 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: RocksDB version: 7.9.2 2024-05-27T09:17:04.992 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Git sha 0 2024-05-27T09:17:04.992 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Compile date 2023-08-03 19:21:13 2024-05-27T09:17:04.992 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: DB SUMMARY 2024-05-27T09:17:04.992 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: DB Session ID: 847R3X7A2HXDB0XF035N 2024-05-27T09:17:04.992 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: CURRENT file: CURRENT 2024-05-27T09:17:04.993 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: IDENTITY file: IDENTITY 2024-05-27T09:17:04.993 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-27T09:17:04.993 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi119/store.db dir, Total Num: 1, files: 000008.sst 2024-05-27T09:17:04.993 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi119/store.db: 000009.log size: 98288 ; 2024-05-27T09:17:04.993 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.error_if_exists: 0 2024-05-27T09:17:04.993 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.create_if_missing: 0 2024-05-27T09:17:04.994 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.paranoid_checks: 1 2024-05-27T09:17:04.994 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-27T09:17:04.994 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-27T09:17:04.994 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-27T09:17:04.994 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.env: 0x5637b6075720 2024-05-27T09:17:04.994 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.fs: PosixFileSystem 2024-05-27T09:17:04.995 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.info_log: 0x5637c9b792c0 2024-05-27T09:17:04.995 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_file_opening_threads: 16 2024-05-27T09:17:04.995 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.statistics: (nil) 2024-05-27T09:17:04.995 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.use_fsync: 0 2024-05-27T09:17:04.995 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_log_file_size: 0 2024-05-27T09:17:04.995 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-27T09:17:04.996 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-27T09:17:04.996 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.keep_log_file_num: 1000 2024-05-27T09:17:04.996 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.recycle_log_file_num: 0 2024-05-27T09:17:04.996 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.allow_fallocate: 1 2024-05-27T09:17:04.996 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.allow_mmap_reads: 0 2024-05-27T09:17:04.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.allow_mmap_writes: 0 2024-05-27T09:17:04.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.use_direct_reads: 0 2024-05-27T09:17:04.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-27T09:17:04.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.create_missing_column_families: 0 2024-05-27T09:17:04.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.db_log_dir: 2024-05-27T09:17:04.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.wal_dir: 2024-05-27T09:17:05.000 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-27T09:17:05.000 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-27T09:17:05.000 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-27T09:17:05.000 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-27T09:17:05.000 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-27T09:17:05.000 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-27T09:17:05.001 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.advise_random_on_open: 1 2024-05-27T09:17:05.001 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.db_write_buffer_size: 0 2024-05-27T09:17:05.001 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.write_buffer_manager: 0x5637c8dea3c0 2024-05-27T09:17:05.001 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-27T09:17:05.001 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-27T09:17:05.001 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-27T09:17:05.002 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.rate_limiter: (nil) 2024-05-27T09:17:05.002 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-27T09:17:05.002 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.wal_recovery_mode: 2 2024-05-27T09:17:05.002 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.enable_thread_tracking: 0 2024-05-27T09:17:05.002 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.enable_pipelined_write: 0 2024-05-27T09:17:05.002 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.unordered_write: 0 2024-05-27T09:17:05.003 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-27T09:17:05.003 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-27T09:17:05.003 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-27T09:17:05.003 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-27T09:17:05.003 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.row_cache: None 2024-05-27T09:17:05.003 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.wal_filter: None 2024-05-27T09:17:05.004 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-27T09:17:05.004 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.allow_ingest_behind: 0 2024-05-27T09:17:05.004 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.two_write_queues: 0 2024-05-27T09:17:05.004 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.manual_wal_flush: 0 2024-05-27T09:17:05.004 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.wal_compression: 0 2024-05-27T09:17:05.004 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.atomic_flush: 0 2024-05-27T09:17:05.005 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-27T09:17:05.005 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-27T09:17:05.005 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-27T09:17:05.006 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.log_readahead_size: 0 2024-05-27T09:17:05.006 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-27T09:17:05.006 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.best_efforts_recovery: 0 2024-05-27T09:17:05.006 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-27T09:17:05.006 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-27T09:17:05.007 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.allow_data_in_errors: 0 2024-05-27T09:17:05.007 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.db_host_id: __hostname__ 2024-05-27T09:17:05.007 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-27T09:17:05.007 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_background_jobs: 2 2024-05-27T09:17:05.007 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_background_compactions: -1 2024-05-27T09:17:05.007 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_subcompactions: 1 2024-05-27T09:17:05.007 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-27T09:17:05.008 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-27T09:17:05.008 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-27T09:17:05.008 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_total_wal_size: 0 2024-05-27T09:17:05.008 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-27T09:17:05.008 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-27T09:17:05.008 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-27T09:17:05.009 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-27T09:17:05.009 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_open_files: -1 2024-05-27T09:17:05.009 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.bytes_per_sync: 0 2024-05-27T09:17:05.009 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-27T09:17:05.009 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-27T09:17:05.010 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compaction_readahead_size: 0 2024-05-27T09:17:05.010 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_background_flushes: -1 2024-05-27T09:17:05.010 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Compression algorithms supported: 2024-05-27T09:17:05.010 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-27T09:17:05.010 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: kZSTD supported: 0 2024-05-27T09:17:05.011 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: kXpressCompression supported: 0 2024-05-27T09:17:05.011 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: kLZ4HCCompression supported: 1 2024-05-27T09:17:05.011 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: kZlibCompression supported: 1 2024-05-27T09:17:05.011 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: kSnappyCompression supported: 1 2024-05-27T09:17:05.011 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: kLZ4Compression supported: 1 2024-05-27T09:17:05.011 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: kBZip2Compression supported: 0 2024-05-27T09:17:05.011 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-27T09:17:05.012 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-27T09:17:05.012 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi119/store.db/MANIFEST-000010 2024-05-27T09:17:05.012 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-27T09:17:05.012 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-27T09:17:05.012 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.merge_operator: 2024-05-27T09:17:05.012 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compaction_filter: None 2024-05-27T09:17:05.013 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compaction_filter_factory: None 2024-05-27T09:17:05.013 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.sst_partitioner_factory: None 2024-05-27T09:17:05.013 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-27T09:17:05.013 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-27T09:17:05.013 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5637c9b793c0) 2024-05-27T09:17:05.014 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: cache_index_and_filter_blocks: 1 2024-05-27T09:17:05.014 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-27T09:17:05.014 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-27T09:17:05.014 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: pin_top_level_index_and_filter: 1 2024-05-27T09:17:05.014 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: index_type: 0 2024-05-27T09:17:05.014 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: data_block_index_type: 0 2024-05-27T09:17:05.015 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: index_shortening: 1 2024-05-27T09:17:05.015 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-27T09:17:05.015 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: checksum: 4 2024-05-27T09:17:05.015 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: no_block_cache: 0 2024-05-27T09:17:05.015 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: block_cache: 0x5637c8e8b1f0 2024-05-27T09:17:05.015 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: block_cache_name: BinnedLRUCache 2024-05-27T09:17:05.016 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: block_cache_options: 2024-05-27T09:17:05.016 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: capacity : 536870912 2024-05-27T09:17:05.016 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: num_shard_bits : 4 2024-05-27T09:17:05.016 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: strict_capacity_limit : 0 2024-05-27T09:17:05.016 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: high_pri_pool_ratio: 0.000 2024-05-27T09:17:05.016 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: block_cache_compressed: (nil) 2024-05-27T09:17:05.017 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: persistent_cache: (nil) 2024-05-27T09:17:05.017 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: block_size: 4096 2024-05-27T09:17:05.017 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: block_size_deviation: 10 2024-05-27T09:17:05.017 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: block_restart_interval: 16 2024-05-27T09:17:05.017 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: index_block_restart_interval: 1 2024-05-27T09:17:05.017 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: metadata_block_size: 4096 2024-05-27T09:17:05.018 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: partition_filters: 0 2024-05-27T09:17:05.018 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: use_delta_encoding: 1 2024-05-27T09:17:05.018 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: filter_policy: bloomfilter 2024-05-27T09:17:05.018 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: whole_key_filtering: 1 2024-05-27T09:17:05.018 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: verify_compression: 0 2024-05-27T09:17:05.018 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: read_amp_bytes_per_bit: 0 2024-05-27T09:17:05.019 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: format_version: 5 2024-05-27T09:17:05.019 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: enable_index_compression: 1 2024-05-27T09:17:05.019 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: block_align: 0 2024-05-27T09:17:05.019 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: max_auto_readahead_size: 262144 2024-05-27T09:17:05.019 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: prepopulate_block_cache: 0 2024-05-27T09:17:05.019 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: initial_auto_readahead_size: 8192 2024-05-27T09:17:05.020 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: num_file_reads_for_auto_readahead: 2 2024-05-27T09:17:05.020 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.write_buffer_size: 33554432 2024-05-27T09:17:05.020 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_write_buffer_number: 2 2024-05-27T09:17:05.020 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compression: NoCompression 2024-05-27T09:17:05.020 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.bottommost_compression: Disabled 2024-05-27T09:17:05.021 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.prefix_extractor: nullptr 2024-05-27T09:17:05.021 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-27T09:17:05.021 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.num_levels: 7 2024-05-27T09:17:05.021 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-27T09:17:05.021 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-27T09:17:05.021 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-27T09:17:05.022 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-27T09:17:05.022 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-27T09:17:05.022 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-27T09:17:05.022 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-27T09:17:05.022 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-27T09:17:05.022 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-27T09:17:05.023 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-27T09:17:05.023 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-27T09:17:05.023 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-27T09:17:05.023 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-27T09:17:05.023 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compression_opts.level: 32767 2024-05-27T09:17:05.023 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compression_opts.strategy: 0 2024-05-27T09:17:05.024 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-27T09:17:05.024 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-27T09:17:05.024 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-27T09:17:05.024 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-27T09:17:05.024 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compression_opts.enabled: false 2024-05-27T09:17:05.024 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-27T09:17:05.025 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-27T09:17:05.025 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-27T09:17:05.025 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-27T09:17:05.025 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.target_file_size_base: 67108864 2024-05-27T09:17:05.025 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-27T09:17:05.026 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-27T09:17:05.026 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-27T09:17:05.026 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-27T09:17:05.026 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-27T09:17:05.027 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-27T09:17:05.027 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-27T09:17:05.027 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-27T09:17:05.027 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-27T09:17:05.027 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-27T09:17:05.027 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-27T09:17:05.028 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-27T09:17:05.028 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-27T09:17:05.028 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-27T09:17:05.028 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.arena_block_size: 1048576 2024-05-27T09:17:05.029 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-27T09:17:05.029 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-27T09:17:05.029 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.disable_auto_compactions: 0 2024-05-27T09:17:05.029 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-27T09:17:05.029 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-27T09:17:05.030 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-27T09:17:05.030 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-27T09:17:05.030 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-27T09:17:05.030 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-27T09:17:05.031 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-27T09:17:05.031 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-27T09:17:05.031 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-27T09:17:05.031 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-27T09:17:05.032 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.table_properties_collectors: 2024-05-27T09:17:05.032 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.inplace_update_support: 0 2024-05-27T09:17:05.032 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-27T09:17:05.032 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-27T09:17:05.032 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-27T09:17:05.033 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-27T09:17:05.033 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.bloom_locality: 0 2024-05-27T09:17:05.033 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.max_successive_merges: 0 2024-05-27T09:17:05.033 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-27T09:17:05.033 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.paranoid_file_checks: 0 2024-05-27T09:17:05.034 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.force_consistency_checks: 1 2024-05-27T09:17:05.034 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.report_bg_io_stats: 0 2024-05-27T09:17:05.034 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.ttl: 2592000 2024-05-27T09:17:05.034 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-27T09:17:05.035 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-27T09:17:05.035 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-27T09:17:05.035 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.enable_blob_files: false 2024-05-27T09:17:05.035 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.min_blob_size: 0 2024-05-27T09:17:05.036 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.blob_file_size: 268435456 2024-05-27T09:17:05.036 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-27T09:17:05.036 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-27T09:17:05.036 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-27T09:17:05.036 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-27T09:17:05.037 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-27T09:17:05.037 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.blob_file_starting_level: 0 2024-05-27T09:17:05.037 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-27T09:17:05.037 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi119/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-27T09:17:05.037 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-27T09:17:05.038 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: da0f174e-a9ee-4728-a3bb-037f69355945 2024-05-27T09:17:05.038 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716801424795459, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-27T09:17:05.038 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-27T09:17:05.038 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716801424798102, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93539, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 314, "table_properties": {"data_size": 91561, "index_size": 260, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84170, "raw_average_value_size": 308, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716801424, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "da0f174e-a9ee-4728-a3bb-037f69355945", "db_session_id": "847R3X7A2HXDB0XF035N", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-27T09:17:05.039 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716801424798445, "job": 1, "event": "recovery_finished"} 2024-05-27T09:17:05.039 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-27T09:17:05.039 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi119/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-27T09:17:05.039 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5637c8f28000 2024-05-27T09:17:05.039 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: DB pointer 0x5637c8f14000 2024-05-27T09:17:05.040 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-27T09:17:05.040 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-27T09:17:05.040 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: ** DB Stats ** 2024-05-27T09:17:05.040 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T09:17:05.040 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-27T09:17:05.041 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T09:17:05.041 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T09:17:05.041 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-27T09:17:05.041 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T09:17:05.042 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T09:17:05.042 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: 2024-05-27T09:17:05.042 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: ** Compaction Stats [default] ** 2024-05-27T09:17:05.042 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T09:17:05.043 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-27T09:17:05.043 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: L0 2/0 93.22 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 60.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T09:17:05.043 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Sum 2/0 93.22 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 60.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T09:17:05.043 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 60.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T09:17:05.043 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: 2024-05-27T09:17:05.044 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: ** Compaction Stats [default] ** 2024-05-27T09:17:05.044 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T09:17:05.044 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-27T09:17:05.044 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 60.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T09:17:05.044 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: 2024-05-27T09:17:05.045 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-27T09:17:05.045 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: 2024-05-27T09:17:05.045 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T09:17:05.045 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-27T09:17:05.046 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-27T09:17:05.046 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-27T09:17:05.046 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-27T09:17:05.046 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-27T09:17:05.046 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Cumulative compaction: 0.00 GB write, 9.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T09:17:05.047 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Interval compaction: 0.00 GB write, 9.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T09:17:05.047 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-27T09:17:05.047 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Block cache BinnedLRUCache@0x5637c8e8b1f0#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-05-27T09:17:05.047 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-05-27T09:17:05.048 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: 2024-05-27T09:17:05.048 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-27T09:17:05.048 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: starting mon.smithi119 rank 0 at public addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] at bind addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi119 fsid b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:17:05.048 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: mon.smithi119@-1(???) e1 preinit fsid b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:17:05.048 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: mon.smithi119@-1(???).mds e1 new map 2024-05-27T09:17:05.048 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: mon.smithi119@-1(???).mds e1 print_map 2024-05-27T09:17:05.048 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: e1 2024-05-27T09:17:05.048 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T09:17:05.048 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:17:05.048 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: legacy client fscid: -1 2024-05-27T09:17:05.048 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: 2024-05-27T09:17:05.049 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: No filesystems configured 2024-05-27T09:17:05.049 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: mon.smithi119@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-27T09:17:05.049 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: mon.smithi119@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T09:17:05.049 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: mon.smithi119@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T09:17:05.049 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: mon.smithi119@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T09:17:05.049 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: mon.smithi119@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-27T09:17:05.049 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 bash[29064]: ba998b789a1c708b9fa6ba9962892e24c8488192c708f59682be39479f0dd0c8 2024-05-27T09:17:05.049 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 systemd[1]: Started Ceph mon.smithi119 for b966b06e-1c09-11ef-bc9b-c7b262605968. 2024-05-27T09:17:05.049 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: mon.smithi119 is new leader, mons smithi119 in quorum (ranks 0) 2024-05-27T09:17:05.049 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: monmap e1: 1 mons at {smithi119=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0]} removed_ranks: {} 2024-05-27T09:17:05.049 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: fsmap 2024-05-27T09:17:05.049 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: osdmap e1: 0 total, 0 up, 0 in 2024-05-27T09:17:05.049 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:04 smithi119 ceph-mon[29082]: mgrmap e1: no daemons active 2024-05-27T09:17:06.197 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.532+0000 7ff732800700 1 Processor -- start 2024-05-27T09:17:06.198 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.532+0000 7ff732800700 1 -- start start 2024-05-27T09:17:06.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.533+0000 7ff732800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c107d00 0x7ff72c108110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:06.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.533+0000 7ff732800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c108650 con 0x7ff72c107d00 2024-05-27T09:17:06.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.533+0000 7ff72b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c107d00 0x7ff72c108110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:06.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.533+0000 7ff72b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c107d00 0x7ff72c108110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47026/0 (socket says 172.21.15.119:47026) 2024-05-27T09:17:06.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.533+0000 7ff72b400700 1 -- 172.21.15.119:0/4204947533 learned_addr learned my addr 172.21.15.119:0/4204947533 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:06.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.534+0000 7ff72b400700 1 -- 172.21.15.119:0/4204947533 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72c108790 con 0x7ff72c107d00 2024-05-27T09:17:06.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.534+0000 7ff72b400700 1 --2- 172.21.15.119:0/4204947533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c107d00 0x7ff72c108110 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0098a0 tx=0x7ff71c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a738696837b8b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:06.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.535+0000 7ff72a000700 1 -- 172.21.15.119:0/4204947533 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff71c004180 con 0x7ff72c107d00 2024-05-27T09:17:06.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.535+0000 7ff72a000700 1 -- 172.21.15.119:0/4204947533 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff71c0042e0 con 0x7ff72c107d00 2024-05-27T09:17:06.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.535+0000 7ff72a000700 1 -- 172.21.15.119:0/4204947533 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff71c011a50 con 0x7ff72c107d00 2024-05-27T09:17:06.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.536+0000 7ff732800700 1 -- 172.21.15.119:0/4204947533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c107d00 msgr2=0x7ff72c108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:06.200 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.536+0000 7ff732800700 1 --2- 172.21.15.119:0/4204947533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c107d00 0x7ff72c108110 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0098a0 tx=0x7ff71c009bb0 comp rx=0 tx=0).stop 2024-05-27T09:17:06.201 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.536+0000 7ff732800700 1 -- 172.21.15.119:0/4204947533 shutdown_connections 2024-05-27T09:17:06.201 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.536+0000 7ff732800700 1 --2- 172.21.15.119:0/4204947533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c107d00 0x7ff72c108110 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:06.201 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.536+0000 7ff732800700 1 -- 172.21.15.119:0/4204947533 >> 172.21.15.119:0/4204947533 conn(0x7ff72c07b220 msgr2=0x7ff72c07b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:06.201 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.536+0000 7ff732800700 1 -- 172.21.15.119:0/4204947533 shutdown_connections 2024-05-27T09:17:06.201 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.536+0000 7ff732800700 1 -- 172.21.15.119:0/4204947533 wait complete. 2024-05-27T09:17:06.203 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.537+0000 7ff732800700 1 Processor -- start 2024-05-27T09:17:06.203 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.537+0000 7ff732800700 1 -- start start 2024-05-27T09:17:06.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.538+0000 7ff732800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c107d00 0x7ff72c19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:06.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.538+0000 7ff732800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c19bfb0 con 0x7ff72c107d00 2024-05-27T09:17:06.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.538+0000 7ff72b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c107d00 0x7ff72c19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:06.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.538+0000 7ff72b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c107d00 0x7ff72c19ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47032/0 (socket says 172.21.15.119:47032) 2024-05-27T09:17:06.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.538+0000 7ff72b400700 1 -- 172.21.15.119:0/2617480093 learned_addr learned my addr 172.21.15.119:0/2617480093 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:06.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.538+0000 7ff72b400700 1 -- 172.21.15.119:0/2617480093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71c009550 con 0x7ff72c107d00 2024-05-27T09:17:06.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.539+0000 7ff72b400700 1 --2- 172.21.15.119:0/2617480093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c107d00 0x7ff72c19ba70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff71c000c00 tx=0x7ff71c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:06.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.539+0000 7ff723e00700 1 -- 172.21.15.119:0/2617480093 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff71c003b90 con 0x7ff72c107d00 2024-05-27T09:17:06.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.539+0000 7ff723e00700 1 -- 172.21.15.119:0/2617480093 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff71c0248b0 con 0x7ff72c107d00 2024-05-27T09:17:06.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.539+0000 7ff732800700 1 -- 172.21.15.119:0/2617480093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff72c19c1b0 con 0x7ff72c107d00 2024-05-27T09:17:06.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.539+0000 7ff723e00700 1 -- 172.21.15.119:0/2617480093 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff71c01a8e0 con 0x7ff72c107d00 2024-05-27T09:17:06.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.539+0000 7ff732800700 1 -- 172.21.15.119:0/2617480093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff72c19c5d0 con 0x7ff72c107d00 2024-05-27T09:17:06.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.539+0000 7ff723e00700 1 -- 172.21.15.119:0/2617480093 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff71c02d3f0 con 0x7ff72c107d00 2024-05-27T09:17:06.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.539+0000 7ff723e00700 1 -- 172.21.15.119:0/2617480093 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff71c024440 con 0x7ff72c107d00 2024-05-27T09:17:06.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.540+0000 7ff732800700 1 -- 172.21.15.119:0/2617480093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72c062380 con 0x7ff72c107d00 2024-05-27T09:17:06.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.544+0000 7ff723e00700 1 -- 172.21.15.119:0/2617480093 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7ff71c024db0 con 0x7ff72c107d00 2024-05-27T09:17:06.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.618+0000 7ff732800700 1 -- 172.21.15.119:0/2617480093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7ff72c109b70 con 0x7ff72c107d00 2024-05-27T09:17:06.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.622+0000 7ff723e00700 1 -- 172.21.15.119:0/2617480093 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff71c024600 con 0x7ff72c107d00 2024-05-27T09:17:06.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.623+0000 7ff732800700 1 -- 172.21.15.119:0/2617480093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c107d00 msgr2=0x7ff72c19ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:06.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.623+0000 7ff732800700 1 --2- 172.21.15.119:0/2617480093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c107d00 0x7ff72c19ba70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff71c000c00 tx=0x7ff71c011ab0 comp rx=0 tx=0).stop 2024-05-27T09:17:06.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.623+0000 7ff732800700 1 -- 172.21.15.119:0/2617480093 shutdown_connections 2024-05-27T09:17:06.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.623+0000 7ff732800700 1 --2- 172.21.15.119:0/2617480093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c107d00 0x7ff72c19ba70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:06.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.623+0000 7ff732800700 1 -- 172.21.15.119:0/2617480093 >> 172.21.15.119:0/2617480093 conn(0x7ff72c07b220 msgr2=0x7ff72c105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:06.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.624+0000 7ff732800700 1 -- 172.21.15.119:0/2617480093 shutdown_connections 2024-05-27T09:17:06.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:05.624+0000 7ff732800700 1 -- 172.21.15.119:0/2617480093 wait complete. 2024-05-27T09:17:06.205 INFO:teuthology.orchestra.run.smithi119.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-27T09:17:06.233 INFO:teuthology.orchestra.run.smithi119.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-27T09:17:06.233 INFO:teuthology.orchestra.run.smithi119.stdout:Creating mgr... 2024-05-27T09:17:06.234 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-27T09:17:06.235 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-27T09:17:06.236 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying port 0.0.0.0:8443 ... 2024-05-27T09:17:06.694 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@mgr.smithi119.ncwmog 2024-05-27T09:17:06.695 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@mgr.smithi119.ncwmog.service: Unit ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@mgr.smithi119.ncwmog.service not loaded. 2024-05-27T09:17:06.959 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b966b06e-1c09-11ef-bc9b-c7b262605968.target.wants/ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@mgr.smithi119.ncwmog.service → /etc/systemd/system/ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@.service. 2024-05-27T09:17:06.996 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:06 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/2617480093' entity='client.admin' 2024-05-27T09:17:07.719 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T09:17:07.720 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout disabled 2024-05-27T09:17:07.737 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T09:17:07.737 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2024-05-27T09:17:07.738 INFO:teuthology.orchestra.run.smithi119.stdout:firewalld.service is not enabled 2024-05-27T09:17:07.738 INFO:teuthology.orchestra.run.smithi119.stdout:Not possible to enable service . firewalld.service is not available 2024-05-27T09:17:07.760 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T09:17:07.760 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout disabled 2024-05-27T09:17:07.780 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T09:17:07.781 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2024-05-27T09:17:07.781 INFO:teuthology.orchestra.run.smithi119.stdout:firewalld.service is not enabled 2024-05-27T09:17:07.781 INFO:teuthology.orchestra.run.smithi119.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-05-27T09:17:07.782 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mgr to start... 2024-05-27T09:17:07.782 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mgr... 2024-05-27T09:17:08.735 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:08 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/975187946' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T09:17:09.158 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-05-27T09:17:09.158 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2024-05-27T09:17:09.158 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsid": "b966b06e-1c09-11ef-bc9b-c7b262605968", 2024-05-27T09:17:09.158 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T09:17:09.158 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T09:17:09.158 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T09:17:09.158 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T09:17:09.159 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:09.159 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T09:17:09.159 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T09:17:09.159 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 0 2024-05-27T09:17:09.159 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-05-27T09:17:09.159 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T09:17:09.161 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "smithi119" 2024-05-27T09:17:09.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-05-27T09:17:09.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-05-27T09:17:09.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T09:17:09.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T09:17:09.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-27T09:17:09.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T09:17:09.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:09.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T09:17:09.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T09:17:09.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T09:17:09.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T09:17:09.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T09:17:09.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T09:17:09.163 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T09:17:09.163 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T09:17:09.169 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:09.169 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T09:17:09.169 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T09:17:09.169 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T09:17:09.169 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T09:17:09.169 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T09:17:09.169 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T09:17:09.170 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T09:17:09.170 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T09:17:09.171 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T09:17:09.171 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:09.171 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T09:17:09.171 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T09:17:09.171 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T09:17:09.171 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T09:17:09.171 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:09.171 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T09:17:09.171 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-27T09:17:09.171 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T09:17:09.172 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T09:17:09.172 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T09:17:09.172 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T09:17:09.172 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T09:17:09.172 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-05-27T09:17:09.172 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T09:17:09.172 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:09.172 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T09:17:09.172 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T09:17:09.172 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T09:16:58.006487+0000", 2024-05-27T09:17:09.172 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T09:17:09.172 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:09.172 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T09:17:09.173 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2024-05-27T09:17:09.173 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.374+0000 7f53c3200700 1 Processor -- start 2024-05-27T09:17:09.173 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.374+0000 7f53c3200700 1 -- start start 2024-05-27T09:17:09.173 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.375+0000 7f53c3200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53bc07acc0 0x7f53bc0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:09.173 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.375+0000 7f53c3200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc079700 con 0x7f53bc07acc0 2024-05-27T09:17:09.173 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.375+0000 7f53bbe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53bc07acc0 0x7f53bc0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:09.173 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.375+0000 7f53bbe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53bc07acc0 0x7f53bc0791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47064/0 (socket says 172.21.15.119:47064) 2024-05-27T09:17:09.173 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.375+0000 7f53bbe00700 1 -- 172.21.15.119:0/1448859216 learned_addr learned my addr 172.21.15.119:0/1448859216 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:09.173 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.376+0000 7f53bbe00700 1 -- 172.21.15.119:0/1448859216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53bc079840 con 0x7f53bc07acc0 2024-05-27T09:17:09.173 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.376+0000 7f53bbe00700 1 --2- 172.21.15.119:0/1448859216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53bc07acc0 0x7f53bc0791c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0098a0 tx=0x7f53ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5e58fe86d3b79b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:09.173 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.376+0000 7f53baa00700 1 -- 172.21.15.119:0/1448859216 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f53ac004180 con 0x7f53bc07acc0 2024-05-27T09:17:09.173 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.376+0000 7f53baa00700 1 -- 172.21.15.119:0/1448859216 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f53ac0042e0 con 0x7f53bc07acc0 2024-05-27T09:17:09.173 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.376+0000 7f53baa00700 1 -- 172.21.15.119:0/1448859216 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f53ac011a50 con 0x7f53bc07acc0 2024-05-27T09:17:09.174 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.377+0000 7f53c3200700 1 -- 172.21.15.119:0/1448859216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53bc07acc0 msgr2=0x7f53bc0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:09.174 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.377+0000 7f53c3200700 1 --2- 172.21.15.119:0/1448859216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53bc07acc0 0x7f53bc0791c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0098a0 tx=0x7f53ac009bb0 comp rx=0 tx=0).stop 2024-05-27T09:17:09.174 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.377+0000 7f53c3200700 1 -- 172.21.15.119:0/1448859216 shutdown_connections 2024-05-27T09:17:09.174 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.377+0000 7f53c3200700 1 --2- 172.21.15.119:0/1448859216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53bc07acc0 0x7f53bc0791c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:09.174 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.377+0000 7f53c3200700 1 -- 172.21.15.119:0/1448859216 >> 172.21.15.119:0/1448859216 conn(0x7f53bc101160 msgr2=0x7f53bc103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:09.174 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.377+0000 7f53c3200700 1 -- 172.21.15.119:0/1448859216 shutdown_connections 2024-05-27T09:17:09.174 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.377+0000 7f53c3200700 1 -- 172.21.15.119:0/1448859216 wait complete. 2024-05-27T09:17:09.174 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.378+0000 7f53c3200700 1 Processor -- start 2024-05-27T09:17:09.174 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.378+0000 7f53c3200700 1 -- start start 2024-05-27T09:17:09.174 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.379+0000 7f53c3200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53bc07acc0 0x7f53bc19bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:09.174 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.379+0000 7f53c3200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc19c010 con 0x7f53bc07acc0 2024-05-27T09:17:09.174 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.379+0000 7f53bbe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53bc07acc0 0x7f53bc19bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:09.174 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.379+0000 7f53bbe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53bc07acc0 0x7f53bc19bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47070/0 (socket says 172.21.15.119:47070) 2024-05-27T09:17:09.175 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.379+0000 7f53bbe00700 1 -- 172.21.15.119:0/975187946 learned_addr learned my addr 172.21.15.119:0/975187946 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:09.175 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.379+0000 7f53bbe00700 1 -- 172.21.15.119:0/975187946 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ac009550 con 0x7f53bc07acc0 2024-05-27T09:17:09.175 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.380+0000 7f53bbe00700 1 --2- 172.21.15.119:0/975187946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53bc07acc0 0x7f53bc19bad0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f53ac006af0 tx=0x7f53ac0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:09.175 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.380+0000 7f53b8c00700 1 -- 172.21.15.119:0/975187946 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f53ac003a00 con 0x7f53bc07acc0 2024-05-27T09:17:09.175 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.380+0000 7f53b8c00700 1 -- 172.21.15.119:0/975187946 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f53ac003b60 con 0x7f53bc07acc0 2024-05-27T09:17:09.175 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.380+0000 7f53c3200700 1 -- 172.21.15.119:0/975187946 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53bc19c210 con 0x7f53bc07acc0 2024-05-27T09:17:09.175 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.380+0000 7f53c3200700 1 -- 172.21.15.119:0/975187946 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53bc19c630 con 0x7f53bc07acc0 2024-05-27T09:17:09.175 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.380+0000 7f53b8c00700 1 -- 172.21.15.119:0/975187946 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f53ac019710 con 0x7f53bc07acc0 2024-05-27T09:17:09.175 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.380+0000 7f53b8c00700 1 -- 172.21.15.119:0/975187946 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f53ac019d00 con 0x7f53bc07acc0 2024-05-27T09:17:09.175 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.381+0000 7f53b8c00700 1 -- 172.21.15.119:0/975187946 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f53ac024730 con 0x7f53bc07acc0 2024-05-27T09:17:09.175 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.381+0000 7f53c3200700 1 -- 172.21.15.119:0/975187946 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53bc062380 con 0x7f53bc07acc0 2024-05-27T09:17:09.175 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.384+0000 7f53b8c00700 1 -- 172.21.15.119:0/975187946 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f53ac01abc0 con 0x7f53bc07acc0 2024-05-27T09:17:09.175 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.463+0000 7f53c3200700 1 -- 172.21.15.119:0/975187946 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f53bc19f160 con 0x7f53bc07acc0 2024-05-27T09:17:09.176 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.464+0000 7f53b8c00700 1 -- 172.21.15.119:0/975187946 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f53ac024940 con 0x7f53bc07acc0 2024-05-27T09:17:09.176 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.465+0000 7f53c3200700 1 -- 172.21.15.119:0/975187946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53bc07acc0 msgr2=0x7f53bc19bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:09.176 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.465+0000 7f53c3200700 1 --2- 172.21.15.119:0/975187946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53bc07acc0 0x7f53bc19bad0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f53ac006af0 tx=0x7f53ac0047c0 comp rx=0 tx=0).stop 2024-05-27T09:17:09.176 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.465+0000 7f53c3200700 1 -- 172.21.15.119:0/975187946 shutdown_connections 2024-05-27T09:17:09.176 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.465+0000 7f53c3200700 1 --2- 172.21.15.119:0/975187946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53bc07acc0 0x7f53bc19bad0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:09.176 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.465+0000 7f53c3200700 1 -- 172.21.15.119:0/975187946 >> 172.21.15.119:0/975187946 conn(0x7f53bc101160 msgr2=0x7f53bc102840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:09.176 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.466+0000 7f53c3200700 1 -- 172.21.15.119:0/975187946 shutdown_connections 2024-05-27T09:17:09.176 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:08.466+0000 7f53c3200700 1 -- 172.21.15.119:0/975187946 wait complete. 2024-05-27T09:17:09.176 INFO:teuthology.orchestra.run.smithi119.stdout:mgr not available, waiting (1/15)... 2024-05-27T09:17:12.202 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:11 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/2258787653' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T09:17:12.464 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-05-27T09:17:12.465 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2024-05-27T09:17:12.465 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsid": "b966b06e-1c09-11ef-bc9b-c7b262605968", 2024-05-27T09:17:12.465 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T09:17:12.465 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T09:17:12.465 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T09:17:12.465 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T09:17:12.465 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:12.465 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T09:17:12.465 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T09:17:12.465 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 0 2024-05-27T09:17:12.465 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-05-27T09:17:12.465 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T09:17:12.466 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "smithi119" 2024-05-27T09:17:12.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-05-27T09:17:12.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-05-27T09:17:12.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T09:17:12.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T09:17:12.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-27T09:17:12.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T09:17:12.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:12.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T09:17:12.474 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T09:17:12.474 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T09:17:12.474 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T09:17:12.474 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T09:17:12.474 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T09:17:12.474 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T09:17:12.474 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T09:17:12.474 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:12.474 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T09:17:12.474 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T09:17:12.474 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T09:17:12.474 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T09:17:12.475 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T09:17:12.475 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T09:17:12.475 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T09:17:12.475 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T09:17:12.475 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T09:17:12.475 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:12.475 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T09:17:12.475 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T09:17:12.475 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T09:17:12.475 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T09:17:12.475 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:12.475 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T09:17:12.475 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-27T09:17:12.475 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T09:17:12.476 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T09:17:12.476 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T09:17:12.476 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T09:17:12.476 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T09:17:12.476 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-05-27T09:17:12.476 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T09:17:12.476 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:12.476 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T09:17:12.476 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T09:17:12.476 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T09:16:58.006487+0000", 2024-05-27T09:17:12.476 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T09:17:12.476 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:12.476 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T09:17:12.476 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2024-05-27T09:17:12.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.754+0000 7ff855a00700 1 Processor -- start 2024-05-27T09:17:12.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.754+0000 7ff855a00700 1 -- start start 2024-05-27T09:17:12.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.755+0000 7ff855a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff850104a00 0x7ff850104e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:12.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.755+0000 7ff855a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff850105350 con 0x7ff850104a00 2024-05-27T09:17:12.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.756+0000 7ff84ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff850104a00 0x7ff850104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:12.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.756+0000 7ff84ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff850104a00 0x7ff850104e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58132/0 (socket says 172.21.15.119:58132) 2024-05-27T09:17:12.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.756+0000 7ff84ea00700 1 -- 172.21.15.119:0/2904505110 learned_addr learned my addr 172.21.15.119:0/2904505110 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:12.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.756+0000 7ff84ea00700 1 -- 172.21.15.119:0/2904505110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff850105490 con 0x7ff850104a00 2024-05-27T09:17:12.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.756+0000 7ff84ea00700 1 --2- 172.21.15.119:0/2904505110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff850104a00 0x7ff850104e10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff8400098a0 tx=0x7ff840009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f33d14c891cb8c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:12.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.757+0000 7ff84d600700 1 -- 172.21.15.119:0/2904505110 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff840004180 con 0x7ff850104a00 2024-05-27T09:17:12.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.757+0000 7ff84d600700 1 -- 172.21.15.119:0/2904505110 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff8400042e0 con 0x7ff850104a00 2024-05-27T09:17:12.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.757+0000 7ff84d600700 1 -- 172.21.15.119:0/2904505110 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff840011a50 con 0x7ff850104a00 2024-05-27T09:17:12.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.758+0000 7ff855a00700 1 -- 172.21.15.119:0/2904505110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff850104a00 msgr2=0x7ff850104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:12.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.758+0000 7ff855a00700 1 --2- 172.21.15.119:0/2904505110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff850104a00 0x7ff850104e10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff8400098a0 tx=0x7ff840009bb0 comp rx=0 tx=0).stop 2024-05-27T09:17:12.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.758+0000 7ff855a00700 1 -- 172.21.15.119:0/2904505110 shutdown_connections 2024-05-27T09:17:12.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.758+0000 7ff855a00700 1 --2- 172.21.15.119:0/2904505110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff850104a00 0x7ff850104e10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:12.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.758+0000 7ff855a00700 1 -- 172.21.15.119:0/2904505110 >> 172.21.15.119:0/2904505110 conn(0x7ff850100070 msgr2=0x7ff850102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:12.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.758+0000 7ff855a00700 1 -- 172.21.15.119:0/2904505110 shutdown_connections 2024-05-27T09:17:12.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.758+0000 7ff855a00700 1 -- 172.21.15.119:0/2904505110 wait complete. 2024-05-27T09:17:12.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.760+0000 7ff855a00700 1 Processor -- start 2024-05-27T09:17:12.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.760+0000 7ff855a00700 1 -- start start 2024-05-27T09:17:12.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.760+0000 7ff855a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff850104a00 0x7ff8501975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:12.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.760+0000 7ff855a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff850197b20 con 0x7ff850104a00 2024-05-27T09:17:12.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.761+0000 7ff84ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff850104a00 0x7ff8501975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:12.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.761+0000 7ff84ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff850104a00 0x7ff8501975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58148/0 (socket says 172.21.15.119:58148) 2024-05-27T09:17:12.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.761+0000 7ff84ea00700 1 -- 172.21.15.119:0/2258787653 learned_addr learned my addr 172.21.15.119:0/2258787653 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:12.479 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.761+0000 7ff84ea00700 1 -- 172.21.15.119:0/2258787653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff840009550 con 0x7ff850104a00 2024-05-27T09:17:12.479 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.761+0000 7ff84ea00700 1 --2- 172.21.15.119:0/2258787653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff850104a00 0x7ff8501975e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff840000c00 tx=0x7ff840011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:12.479 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.762+0000 7ff847400700 1 -- 172.21.15.119:0/2258787653 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff840003b90 con 0x7ff850104a00 2024-05-27T09:17:12.479 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.762+0000 7ff855a00700 1 -- 172.21.15.119:0/2258787653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff850197d20 con 0x7ff850104a00 2024-05-27T09:17:12.479 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.762+0000 7ff847400700 1 -- 172.21.15.119:0/2258787653 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff8400248b0 con 0x7ff850104a00 2024-05-27T09:17:12.479 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.762+0000 7ff847400700 1 -- 172.21.15.119:0/2258787653 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff84001a8e0 con 0x7ff850104a00 2024-05-27T09:17:12.479 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.762+0000 7ff855a00700 1 -- 172.21.15.119:0/2258787653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff850198140 con 0x7ff850104a00 2024-05-27T09:17:12.479 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.762+0000 7ff847400700 1 -- 172.21.15.119:0/2258787653 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff84002d3f0 con 0x7ff850104a00 2024-05-27T09:17:12.479 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.763+0000 7ff847400700 1 -- 172.21.15.119:0/2258787653 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff840019400 con 0x7ff850104a00 2024-05-27T09:17:12.479 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.763+0000 7ff855a00700 1 -- 172.21.15.119:0/2258787653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff850191330 con 0x7ff850104a00 2024-05-27T09:17:12.479 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.768+0000 7ff847400700 1 -- 172.21.15.119:0/2258787653 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7ff840024db0 con 0x7ff850104a00 2024-05-27T09:17:12.479 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.849+0000 7ff855a00700 1 -- 172.21.15.119:0/2258787653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff850062380 con 0x7ff850104a00 2024-05-27T09:17:12.479 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.850+0000 7ff847400700 1 -- 172.21.15.119:0/2258787653 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7ff840035090 con 0x7ff850104a00 2024-05-27T09:17:12.480 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.851+0000 7ff855a00700 1 -- 172.21.15.119:0/2258787653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff850104a00 msgr2=0x7ff8501975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:12.480 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.851+0000 7ff855a00700 1 --2- 172.21.15.119:0/2258787653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff850104a00 0x7ff8501975e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff840000c00 tx=0x7ff840011ab0 comp rx=0 tx=0).stop 2024-05-27T09:17:12.480 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.852+0000 7ff855a00700 1 -- 172.21.15.119:0/2258787653 shutdown_connections 2024-05-27T09:17:12.480 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.852+0000 7ff855a00700 1 --2- 172.21.15.119:0/2258787653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff850104a00 0x7ff8501975e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:12.480 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.852+0000 7ff855a00700 1 -- 172.21.15.119:0/2258787653 >> 172.21.15.119:0/2258787653 conn(0x7ff850100070 msgr2=0x7ff85018e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:12.480 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.852+0000 7ff855a00700 1 -- 172.21.15.119:0/2258787653 shutdown_connections 2024-05-27T09:17:12.480 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:11.852+0000 7ff855a00700 1 -- 172.21.15.119:0/2258787653 wait complete. 2024-05-27T09:17:12.480 INFO:teuthology.orchestra.run.smithi119.stdout:mgr not available, waiting (2/15)... 2024-05-27T09:17:15.908 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:15 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/3241041641' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T09:17:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-05-27T09:17:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2024-05-27T09:17:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsid": "b966b06e-1c09-11ef-bc9b-c7b262605968", 2024-05-27T09:17:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T09:17:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T09:17:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T09:17:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T09:17:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:15.924 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T09:17:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T09:17:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 0 2024-05-27T09:17:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-05-27T09:17:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T09:17:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "smithi119" 2024-05-27T09:17:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-05-27T09:17:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-05-27T09:17:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T09:17:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T09:17:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-27T09:17:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T09:17:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T09:17:15.925 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T09:17:15.926 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T09:17:15.926 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T09:17:15.926 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T09:17:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T09:17:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T09:17:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T09:17:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T09:17:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T09:17:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T09:17:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T09:17:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T09:17:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T09:17:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T09:17:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T09:17:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T09:17:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T09:17:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T09:17:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T09:17:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T09:17:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T09:17:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-27T09:17:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T09:17:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T09:17:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T09:17:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T09:17:15.934 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T09:17:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-05-27T09:17:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T09:17:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T09:17:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T09:17:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T09:16:58.006487+0000", 2024-05-27T09:17:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T09:17:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T09:17:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2024-05-27T09:17:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.085+0000 7f2483e00700 1 Processor -- start 2024-05-27T09:17:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.085+0000 7f2483e00700 1 -- start start 2024-05-27T09:17:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.085+0000 7f2483e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c105b10 0x7f247c105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:15.935 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.085+0000 7f2483e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247c106460 con 0x7f247c105b10 2024-05-27T09:17:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.086+0000 7f2481400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c105b10 0x7f247c105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.086+0000 7f2481400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c105b10 0x7f247c105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58162/0 (socket says 172.21.15.119:58162) 2024-05-27T09:17:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.086+0000 7f2481400700 1 -- 172.21.15.119:0/1351502385 learned_addr learned my addr 172.21.15.119:0/1351502385 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.086+0000 7f2481400700 1 -- 172.21.15.119:0/1351502385 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247c1065a0 con 0x7f247c105b10 2024-05-27T09:17:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.087+0000 7f2481400700 1 --2- 172.21.15.119:0/1351502385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c105b10 0x7f247c105f20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f24700098a0 tx=0x7f2470009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e19a69ad3ca5e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.087+0000 7f247be00700 1 -- 172.21.15.119:0/1351502385 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2470004180 con 0x7f247c105b10 2024-05-27T09:17:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.087+0000 7f247be00700 1 -- 172.21.15.119:0/1351502385 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f24700042e0 con 0x7f247c105b10 2024-05-27T09:17:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.087+0000 7f247be00700 1 -- 172.21.15.119:0/1351502385 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2470011a50 con 0x7f247c105b10 2024-05-27T09:17:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.088+0000 7f2483e00700 1 -- 172.21.15.119:0/1351502385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c105b10 msgr2=0x7f247c105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.088+0000 7f2483e00700 1 --2- 172.21.15.119:0/1351502385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c105b10 0x7f247c105f20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f24700098a0 tx=0x7f2470009bb0 comp rx=0 tx=0).stop 2024-05-27T09:17:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.088+0000 7f2483e00700 1 -- 172.21.15.119:0/1351502385 shutdown_connections 2024-05-27T09:17:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.088+0000 7f2483e00700 1 --2- 172.21.15.119:0/1351502385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c105b10 0x7f247c105f20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:15.936 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.088+0000 7f2483e00700 1 -- 172.21.15.119:0/1351502385 >> 172.21.15.119:0/1351502385 conn(0x7f247c101160 msgr2=0x7f247c103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.088+0000 7f2483e00700 1 -- 172.21.15.119:0/1351502385 shutdown_connections 2024-05-27T09:17:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.088+0000 7f2483e00700 1 -- 172.21.15.119:0/1351502385 wait complete. 2024-05-27T09:17:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.089+0000 7f2483e00700 1 Processor -- start 2024-05-27T09:17:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.090+0000 7f2483e00700 1 -- start start 2024-05-27T09:17:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.090+0000 7f2483e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c105b10 0x7f247c07ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.090+0000 7f2483e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247c07b070 con 0x7f247c105b10 2024-05-27T09:17:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.090+0000 7f2481400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c105b10 0x7f247c07ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.090+0000 7f2481400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c105b10 0x7f247c07ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58168/0 (socket says 172.21.15.119:58168) 2024-05-27T09:17:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.090+0000 7f2481400700 1 -- 172.21.15.119:0/3241041641 learned_addr learned my addr 172.21.15.119:0/3241041641 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.091+0000 7f2481400700 1 -- 172.21.15.119:0/3241041641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2470009550 con 0x7f247c105b10 2024-05-27T09:17:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.091+0000 7f2481400700 1 --2- 172.21.15.119:0/3241041641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c105b10 0x7f247c07ab30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2470000c00 tx=0x7f2470011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.091+0000 7f247a000700 1 -- 172.21.15.119:0/3241041641 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2470003b90 con 0x7f247c105b10 2024-05-27T09:17:15.937 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.091+0000 7f247a000700 1 -- 172.21.15.119:0/3241041641 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f24700248b0 con 0x7f247c105b10 2024-05-27T09:17:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.091+0000 7f2483e00700 1 -- 172.21.15.119:0/3241041641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f247c079180 con 0x7f247c105b10 2024-05-27T09:17:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.091+0000 7f247a000700 1 -- 172.21.15.119:0/3241041641 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f247001a8e0 con 0x7f247c105b10 2024-05-27T09:17:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.092+0000 7f2483e00700 1 -- 172.21.15.119:0/3241041641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f247c0795a0 con 0x7f247c105b10 2024-05-27T09:17:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.092+0000 7f247a000700 1 -- 172.21.15.119:0/3241041641 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f247002d3f0 con 0x7f247c105b10 2024-05-27T09:17:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.092+0000 7f247a000700 1 -- 172.21.15.119:0/3241041641 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2470019520 con 0x7f247c105b10 2024-05-27T09:17:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.093+0000 7f2483e00700 1 -- 172.21.15.119:0/3241041641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f247c062380 con 0x7f247c105b10 2024-05-27T09:17:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.096+0000 7f247a000700 1 -- 172.21.15.119:0/3241041641 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f2470024db0 con 0x7f247c105b10 2024-05-27T09:17:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.172+0000 7f2483e00700 1 -- 172.21.15.119:0/3241041641 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f247c107980 con 0x7f247c105b10 2024-05-27T09:17:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.172+0000 7f247a000700 1 -- 172.21.15.119:0/3241041641 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1247 (secure 0 0 0) 0x7f2470024a20 con 0x7f247c105b10 2024-05-27T09:17:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.174+0000 7f2483e00700 1 -- 172.21.15.119:0/3241041641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c105b10 msgr2=0x7f247c07ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.174+0000 7f2483e00700 1 --2- 172.21.15.119:0/3241041641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c105b10 0x7f247c07ab30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2470000c00 tx=0x7f2470011ab0 comp rx=0 tx=0).stop 2024-05-27T09:17:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.174+0000 7f2483e00700 1 -- 172.21.15.119:0/3241041641 shutdown_connections 2024-05-27T09:17:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.174+0000 7f2483e00700 1 --2- 172.21.15.119:0/3241041641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c105b10 0x7f247c07ab30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:15.939 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.174+0000 7f2483e00700 1 -- 172.21.15.119:0/3241041641 >> 172.21.15.119:0/3241041641 conn(0x7f247c101160 msgr2=0x7f247c101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:15.939 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.174+0000 7f2483e00700 1 -- 172.21.15.119:0/3241041641 shutdown_connections 2024-05-27T09:17:15.939 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:15.174+0000 7f2483e00700 1 -- 172.21.15.119:0/3241041641 wait complete. 2024-05-27T09:17:15.939 INFO:teuthology.orchestra.run.smithi119.stdout:mgr not available, waiting (3/15)... 2024-05-27T09:17:17.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:16 smithi119 ceph-mon[29082]: Activating manager daemon smithi119.ncwmog 2024-05-27T09:17:17.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:16 smithi119 ceph-mon[29082]: mgrmap e2: smithi119.ncwmog(active, starting, since 0.0117069s) 2024-05-27T09:17:17.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:16 smithi119 ceph-mon[29082]: from='mgr.14100 172.21.15.119:0/697398323' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T09:17:17.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:16 smithi119 ceph-mon[29082]: from='mgr.14100 172.21.15.119:0/697398323' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T09:17:17.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:16 smithi119 ceph-mon[29082]: from='mgr.14100 172.21.15.119:0/697398323' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T09:17:17.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:16 smithi119 ceph-mon[29082]: from='mgr.14100 172.21.15.119:0/697398323' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi119"}]: dispatch 2024-05-27T09:17:17.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:16 smithi119 ceph-mon[29082]: from='mgr.14100 172.21.15.119:0/697398323' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr metadata", "who": "smithi119.ncwmog", "id": "smithi119.ncwmog"}]: dispatch 2024-05-27T09:17:17.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:16 smithi119 ceph-mon[29082]: Manager daemon smithi119.ncwmog is now available 2024-05-27T09:17:17.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:16 smithi119 ceph-mon[29082]: from='mgr.14100 172.21.15.119:0/697398323' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi119.ncwmog/mirror_snapshot_schedule"}]: dispatch 2024-05-27T09:17:17.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:16 smithi119 ceph-mon[29082]: from='mgr.14100 172.21.15.119:0/697398323' entity='mgr.smithi119.ncwmog' 2024-05-27T09:17:17.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:16 smithi119 ceph-mon[29082]: from='mgr.14100 172.21.15.119:0/697398323' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi119.ncwmog/trash_purge_schedule"}]: dispatch 2024-05-27T09:17:17.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:16 smithi119 ceph-mon[29082]: from='mgr.14100 172.21.15.119:0/697398323' entity='mgr.smithi119.ncwmog' 2024-05-27T09:17:17.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:16 smithi119 ceph-mon[29082]: from='mgr.14100 172.21.15.119:0/697398323' entity='mgr.smithi119.ncwmog' 2024-05-27T09:17:18.412 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:18 smithi119 ceph-mon[29082]: mgrmap e3: smithi119.ncwmog(active, since 1.01941s) 2024-05-27T09:17:19.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:19 smithi119 ceph-mon[29082]: mgrmap e4: smithi119.ncwmog(active, since 2s) 2024-05-27T09:17:19.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:19 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/639157990' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T09:17:19.544 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-05-27T09:17:19.545 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2024-05-27T09:17:19.545 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsid": "b966b06e-1c09-11ef-bc9b-c7b262605968", 2024-05-27T09:17:19.545 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T09:17:19.545 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T09:17:19.545 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T09:17:19.546 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T09:17:19.546 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:19.547 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T09:17:19.547 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T09:17:19.547 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 0 2024-05-27T09:17:19.547 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-05-27T09:17:19.547 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T09:17:19.547 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "smithi119" 2024-05-27T09:17:19.547 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-05-27T09:17:19.547 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-05-27T09:17:19.547 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T09:17:19.547 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T09:17:19.547 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-27T09:17:19.548 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T09:17:19.548 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:19.549 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T09:17:19.549 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T09:17:19.549 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T09:17:19.549 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T09:17:19.549 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T09:17:19.549 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T09:17:19.549 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T09:17:19.549 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T09:17:19.553 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:19.553 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T09:17:19.553 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T09:17:19.553 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T09:17:19.553 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T09:17:19.553 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T09:17:19.553 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T09:17:19.553 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T09:17:19.553 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T09:17:19.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T09:17:19.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:19.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T09:17:19.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T09:17:19.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T09:17:19.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T09:17:19.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:19.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T09:17:19.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-27T09:17:19.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T09:17:19.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T09:17:19.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T09:17:19.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T09:17:19.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T09:17:19.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2024-05-27T09:17:19.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T09:17:19.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:19.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T09:17:19.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T09:17:19.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T09:16:58.006487+0000", 2024-05-27T09:17:19.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T09:17:19.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2024-05-27T09:17:19.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T09:17:19.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2024-05-27T09:17:19.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.568+0000 7fdfd2e00700 1 Processor -- start 2024-05-27T09:17:19.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.568+0000 7fdfd2e00700 1 -- start start 2024-05-27T09:17:19.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.569+0000 7fdfd2e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc107d10 0x7fdfcc108120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:19.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.569+0000 7fdfd2e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc108660 con 0x7fdfcc107d10 2024-05-27T09:17:19.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.569+0000 7fdfcbe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc107d10 0x7fdfcc108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:19.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.569+0000 7fdfcbe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc107d10 0x7fdfcc108120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58224/0 (socket says 172.21.15.119:58224) 2024-05-27T09:17:19.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.569+0000 7fdfcbe00700 1 -- 172.21.15.119:0/2588274879 learned_addr learned my addr 172.21.15.119:0/2588274879 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:19.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.570+0000 7fdfcbe00700 1 -- 172.21.15.119:0/2588274879 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc1087a0 con 0x7fdfcc107d10 2024-05-27T09:17:19.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.570+0000 7fdfcbe00700 1 --2- 172.21.15.119:0/2588274879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc107d10 0x7fdfcc108120 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0098a0 tx=0x7fdfbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613940bee6afde89 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:19.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.571+0000 7fdfcaa00700 1 -- 172.21.15.119:0/2588274879 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdfbc004180 con 0x7fdfcc107d10 2024-05-27T09:17:19.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.571+0000 7fdfcaa00700 1 -- 172.21.15.119:0/2588274879 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdfbc0042e0 con 0x7fdfcc107d10 2024-05-27T09:17:19.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.571+0000 7fdfcaa00700 1 -- 172.21.15.119:0/2588274879 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdfbc011a50 con 0x7fdfcc107d10 2024-05-27T09:17:19.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.571+0000 7fdfd2e00700 1 -- 172.21.15.119:0/2588274879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc107d10 msgr2=0x7fdfcc108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:19.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.571+0000 7fdfd2e00700 1 --2- 172.21.15.119:0/2588274879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc107d10 0x7fdfcc108120 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0098a0 tx=0x7fdfbc009bb0 comp rx=0 tx=0).stop 2024-05-27T09:17:19.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.572+0000 7fdfd2e00700 1 -- 172.21.15.119:0/2588274879 shutdown_connections 2024-05-27T09:17:19.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.572+0000 7fdfd2e00700 1 --2- 172.21.15.119:0/2588274879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc107d10 0x7fdfcc108120 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:19.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.572+0000 7fdfd2e00700 1 -- 172.21.15.119:0/2588274879 >> 172.21.15.119:0/2588274879 conn(0x7fdfcc103570 msgr2=0x7fdfcc105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:19.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.572+0000 7fdfd2e00700 1 -- 172.21.15.119:0/2588274879 shutdown_connections 2024-05-27T09:17:19.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.572+0000 7fdfd2e00700 1 -- 172.21.15.119:0/2588274879 wait complete. 2024-05-27T09:17:19.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.573+0000 7fdfd2e00700 1 Processor -- start 2024-05-27T09:17:19.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.573+0000 7fdfd2e00700 1 -- start start 2024-05-27T09:17:19.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.573+0000 7fdfd2e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc107d10 0x7fdfcc19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:19.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.573+0000 7fdfd2e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc19bf60 con 0x7fdfcc107d10 2024-05-27T09:17:19.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.573+0000 7fdfcbe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc107d10 0x7fdfcc19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:19.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.573+0000 7fdfcbe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc107d10 0x7fdfcc19ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58236/0 (socket says 172.21.15.119:58236) 2024-05-27T09:17:19.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.573+0000 7fdfcbe00700 1 -- 172.21.15.119:0/639157990 learned_addr learned my addr 172.21.15.119:0/639157990 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:19.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.574+0000 7fdfcbe00700 1 -- 172.21.15.119:0/639157990 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfbc009550 con 0x7fdfcc107d10 2024-05-27T09:17:19.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.574+0000 7fdfcbe00700 1 --2- 172.21.15.119:0/639157990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc107d10 0x7fdfcc19ba20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc000c00 tx=0x7fdfbc011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:19.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.574+0000 7fdfc8c00700 1 -- 172.21.15.119:0/639157990 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdfbc003b90 con 0x7fdfcc107d10 2024-05-27T09:17:19.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.574+0000 7fdfc8c00700 1 -- 172.21.15.119:0/639157990 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdfbc0248b0 con 0x7fdfcc107d10 2024-05-27T09:17:19.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.574+0000 7fdfc8c00700 1 -- 172.21.15.119:0/639157990 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdfbc01a8e0 con 0x7fdfcc107d10 2024-05-27T09:17:19.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.574+0000 7fdfd2e00700 1 -- 172.21.15.119:0/639157990 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfcc19c160 con 0x7fdfcc107d10 2024-05-27T09:17:19.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.575+0000 7fdfd2e00700 1 -- 172.21.15.119:0/639157990 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfcc19c580 con 0x7fdfcc107d10 2024-05-27T09:17:19.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.576+0000 7fdfc8c00700 1 -- 172.21.15.119:0/639157990 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7fdfbc024a20 con 0x7fdfcc107d10 2024-05-27T09:17:19.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.576+0000 7fdfd2e00700 1 -- 172.21.15.119:0/639157990 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfcc10c1a0 con 0x7fdfcc107d10 2024-05-27T09:17:19.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.576+0000 7fdfc8c00700 1 --2- 172.21.15.119:0/639157990 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fdfb00383f0 0x7fdfb003a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:19.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.576+0000 7fdfc8c00700 1 -- 172.21.15.119:0/639157990 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdfbc04c130 con 0x7fdfcc107d10 2024-05-27T09:17:19.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.577+0000 7fdfcb400700 1 --2- 172.21.15.119:0/639157990 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fdfb00383f0 0x7fdfb003a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:19.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.577+0000 7fdfcb400700 1 --2- 172.21.15.119:0/639157990 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fdfb00383f0 0x7fdfb003a8a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8006fd0 tx=0x7fdfb8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:19.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.582+0000 7fdfc8c00700 1 -- 172.21.15.119:0/639157990 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fdfbc024db0 con 0x7fdfcc107d10 2024-05-27T09:17:19.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.879+0000 7fdfd2e00700 1 -- 172.21.15.119:0/639157990 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fdfcc0623d0 con 0x7fdfcc107d10 2024-05-27T09:17:19.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.880+0000 7fdfc8c00700 1 -- 172.21.15.119:0/639157990 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fdfbc018350 con 0x7fdfcc107d10 2024-05-27T09:17:19.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.882+0000 7fdfd2e00700 1 -- 172.21.15.119:0/639157990 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fdfb00383f0 msgr2=0x7fdfb003a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:19.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.882+0000 7fdfd2e00700 1 --2- 172.21.15.119:0/639157990 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fdfb00383f0 0x7fdfb003a8a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8006fd0 tx=0x7fdfb8006e40 comp rx=0 tx=0).stop 2024-05-27T09:17:19.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.882+0000 7fdfd2e00700 1 -- 172.21.15.119:0/639157990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc107d10 msgr2=0x7fdfcc19ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:19.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.882+0000 7fdfd2e00700 1 --2- 172.21.15.119:0/639157990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc107d10 0x7fdfcc19ba20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc000c00 tx=0x7fdfbc011ab0 comp rx=0 tx=0).stop 2024-05-27T09:17:19.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.883+0000 7fdfd2e00700 1 -- 172.21.15.119:0/639157990 shutdown_connections 2024-05-27T09:17:19.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.883+0000 7fdfd2e00700 1 --2- 172.21.15.119:0/639157990 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fdfb00383f0 0x7fdfb003a8a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:19.564 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.883+0000 7fdfd2e00700 1 --2- 172.21.15.119:0/639157990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc107d10 0x7fdfcc19ba20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:19.564 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.883+0000 7fdfd2e00700 1 -- 172.21.15.119:0/639157990 >> 172.21.15.119:0/639157990 conn(0x7fdfcc103570 msgr2=0x7fdfcc105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:19.564 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.883+0000 7fdfd2e00700 1 -- 172.21.15.119:0/639157990 shutdown_connections 2024-05-27T09:17:19.564 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:18.883+0000 7fdfd2e00700 1 -- 172.21.15.119:0/639157990 wait complete. 2024-05-27T09:17:19.564 INFO:teuthology.orchestra.run.smithi119.stdout:mgr is available 2024-05-27T09:17:21.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:20 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/2857464366' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-27T09:17:21.358 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-05-27T09:17:21.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [global] 2024-05-27T09:17:21.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout fsid = b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:17:21.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-27T09:17:21.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-27T09:17:21.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-27T09:17:21.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-27T09:17:21.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-05-27T09:17:21.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-27T09:17:21.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-27T09:17:21.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2024-05-27T09:17:21.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [osd] 2024-05-27T09:17:21.359 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-27T09:17:21.360 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-27T09:17:21.360 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-27T09:17:21.360 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-27T09:17:21.360 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-27T09:17:21.360 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.205+0000 7fcc5ae00700 1 Processor -- start 2024-05-27T09:17:21.360 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.205+0000 7fcc5ae00700 1 -- start start 2024-05-27T09:17:21.360 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.206+0000 7fcc5ae00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc54107d00 0x7fcc54108110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:21.360 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.206+0000 7fcc5ae00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc54108650 con 0x7fcc54107d00 2024-05-27T09:17:21.360 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.206+0000 7fcc53e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc54107d00 0x7fcc54108110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:21.360 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.206+0000 7fcc53e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc54107d00 0x7fcc54108110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59210/0 (socket says 172.21.15.119:59210) 2024-05-27T09:17:21.360 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.206+0000 7fcc53e00700 1 -- 172.21.15.119:0/1799654741 learned_addr learned my addr 172.21.15.119:0/1799654741 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:21.360 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.207+0000 7fcc53e00700 1 -- 172.21.15.119:0/1799654741 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc54108790 con 0x7fcc54107d00 2024-05-27T09:17:21.360 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.207+0000 7fcc53e00700 1 --2- 172.21.15.119:0/1799654741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc54107d00 0x7fcc54108110 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcc440098a0 tx=0x7fcc44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27363e161773dfe1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:21.361 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.207+0000 7fcc52a00700 1 -- 172.21.15.119:0/1799654741 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc44004180 con 0x7fcc54107d00 2024-05-27T09:17:21.361 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.207+0000 7fcc52a00700 1 -- 172.21.15.119:0/1799654741 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc440042e0 con 0x7fcc54107d00 2024-05-27T09:17:21.361 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.207+0000 7fcc52a00700 1 -- 172.21.15.119:0/1799654741 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc44011a50 con 0x7fcc54107d00 2024-05-27T09:17:21.361 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.208+0000 7fcc5ae00700 1 -- 172.21.15.119:0/1799654741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc54107d00 msgr2=0x7fcc54108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:21.361 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.208+0000 7fcc5ae00700 1 --2- 172.21.15.119:0/1799654741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc54107d00 0x7fcc54108110 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcc440098a0 tx=0x7fcc44009bb0 comp rx=0 tx=0).stop 2024-05-27T09:17:21.361 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.208+0000 7fcc5ae00700 1 -- 172.21.15.119:0/1799654741 shutdown_connections 2024-05-27T09:17:21.361 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.208+0000 7fcc5ae00700 1 --2- 172.21.15.119:0/1799654741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc54107d00 0x7fcc54108110 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:21.361 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.208+0000 7fcc5ae00700 1 -- 172.21.15.119:0/1799654741 >> 172.21.15.119:0/1799654741 conn(0x7fcc5407b220 msgr2=0x7fcc5407b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:21.361 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.208+0000 7fcc5ae00700 1 -- 172.21.15.119:0/1799654741 shutdown_connections 2024-05-27T09:17:21.361 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.208+0000 7fcc5ae00700 1 -- 172.21.15.119:0/1799654741 wait complete. 2024-05-27T09:17:21.361 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.209+0000 7fcc5ae00700 1 Processor -- start 2024-05-27T09:17:21.361 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.209+0000 7fcc5ae00700 1 -- start start 2024-05-27T09:17:21.361 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.210+0000 7fcc5ae00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc54107d00 0x7fcc5419ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:21.362 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.210+0000 7fcc5ae00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5419bfb0 con 0x7fcc54107d00 2024-05-27T09:17:21.362 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.210+0000 7fcc53e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc54107d00 0x7fcc5419ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:21.362 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.210+0000 7fcc53e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc54107d00 0x7fcc5419ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59214/0 (socket says 172.21.15.119:59214) 2024-05-27T09:17:21.362 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.210+0000 7fcc53e00700 1 -- 172.21.15.119:0/2857464366 learned_addr learned my addr 172.21.15.119:0/2857464366 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:21.362 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.210+0000 7fcc53e00700 1 -- 172.21.15.119:0/2857464366 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc44009550 con 0x7fcc54107d00 2024-05-27T09:17:21.362 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.211+0000 7fcc53e00700 1 --2- 172.21.15.119:0/2857464366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc54107d00 0x7fcc5419ba70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcc44000c00 tx=0x7fcc44011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:21.362 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.211+0000 7fcc50c00700 1 -- 172.21.15.119:0/2857464366 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc44003b90 con 0x7fcc54107d00 2024-05-27T09:17:21.362 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.211+0000 7fcc50c00700 1 -- 172.21.15.119:0/2857464366 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc440248b0 con 0x7fcc54107d00 2024-05-27T09:17:21.362 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.211+0000 7fcc50c00700 1 -- 172.21.15.119:0/2857464366 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc4401a8e0 con 0x7fcc54107d00 2024-05-27T09:17:21.362 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.211+0000 7fcc5ae00700 1 -- 172.21.15.119:0/2857464366 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc5419c1b0 con 0x7fcc54107d00 2024-05-27T09:17:21.362 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.211+0000 7fcc5ae00700 1 -- 172.21.15.119:0/2857464366 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5419c5d0 con 0x7fcc54107d00 2024-05-27T09:17:21.362 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.212+0000 7fcc50c00700 1 -- 172.21.15.119:0/2857464366 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7fcc44024440 con 0x7fcc54107d00 2024-05-27T09:17:21.363 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.213+0000 7fcc5ae00700 1 -- 172.21.15.119:0/2857464366 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc54062380 con 0x7fcc54107d00 2024-05-27T09:17:21.363 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.213+0000 7fcc50c00700 1 --2- 172.21.15.119:0/2857464366 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fcc380383f0 0x7fcc3803a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:21.363 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.213+0000 7fcc50c00700 1 -- 172.21.15.119:0/2857464366 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcc4404c060 con 0x7fcc54107d00 2024-05-27T09:17:21.363 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.213+0000 7fcc53400700 1 --2- 172.21.15.119:0/2857464366 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fcc380383f0 0x7fcc3803a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:21.363 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.214+0000 7fcc53400700 1 --2- 172.21.15.119:0/2857464366 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fcc380383f0 0x7fcc3803a8a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcc40006fd0 tx=0x7fcc40006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:21.363 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.218+0000 7fcc50c00700 1 -- 172.21.15.119:0/2857464366 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fcc440189a0 con 0x7fcc54107d00 2024-05-27T09:17:21.363 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.434+0000 7fcc5ae00700 1 -- 172.21.15.119:0/2857464366 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fcc5419f430 con 0x7fcc54107d00 2024-05-27T09:17:21.363 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.436+0000 7fcc50c00700 1 -- 172.21.15.119:0/2857464366 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fcc44018350 con 0x7fcc54107d00 2024-05-27T09:17:21.363 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.438+0000 7fcc5ae00700 1 -- 172.21.15.119:0/2857464366 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fcc380383f0 msgr2=0x7fcc3803a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:21.363 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.438+0000 7fcc5ae00700 1 --2- 172.21.15.119:0/2857464366 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fcc380383f0 0x7fcc3803a8a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcc40006fd0 tx=0x7fcc40006e40 comp rx=0 tx=0).stop 2024-05-27T09:17:21.363 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.439+0000 7fcc5ae00700 1 -- 172.21.15.119:0/2857464366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc54107d00 msgr2=0x7fcc5419ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:21.363 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.439+0000 7fcc5ae00700 1 --2- 172.21.15.119:0/2857464366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc54107d00 0x7fcc5419ba70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcc44000c00 tx=0x7fcc44011ab0 comp rx=0 tx=0).stop 2024-05-27T09:17:21.363 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.439+0000 7fcc5ae00700 1 -- 172.21.15.119:0/2857464366 shutdown_connections 2024-05-27T09:17:21.363 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.439+0000 7fcc5ae00700 1 --2- 172.21.15.119:0/2857464366 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fcc380383f0 0x7fcc3803a8a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:21.364 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.439+0000 7fcc5ae00700 1 --2- 172.21.15.119:0/2857464366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc54107d00 0x7fcc5419ba70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:21.364 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.439+0000 7fcc5ae00700 1 -- 172.21.15.119:0/2857464366 >> 172.21.15.119:0/2857464366 conn(0x7fcc5407b220 msgr2=0x7fcc54105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:21.364 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.439+0000 7fcc5ae00700 1 -- 172.21.15.119:0/2857464366 shutdown_connections 2024-05-27T09:17:21.364 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:20.439+0000 7fcc5ae00700 1 -- 172.21.15.119:0/2857464366 wait complete. 2024-05-27T09:17:21.364 INFO:teuthology.orchestra.run.smithi119.stdout:Enabling cephadm module... 2024-05-27T09:17:22.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:22 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/408895781' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-27T09:17:23.013 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.953+0000 7f1297000700 1 Processor -- start 2024-05-27T09:17:23.013 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.954+0000 7f1297000700 1 -- start start 2024-05-27T09:17:23.013 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.954+0000 7f1297000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1290105b10 0x7f1290105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:23.013 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.954+0000 7f1297000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1290106460 con 0x7f1290105b10 2024-05-27T09:17:23.013 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.954+0000 7f128fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1290105b10 0x7f1290105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:23.013 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.955+0000 7f128fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1290105b10 0x7f1290105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59222/0 (socket says 172.21.15.119:59222) 2024-05-27T09:17:23.013 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.955+0000 7f128fe00700 1 -- 172.21.15.119:0/1131068998 learned_addr learned my addr 172.21.15.119:0/1131068998 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:23.013 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.955+0000 7f128fe00700 1 -- 172.21.15.119:0/1131068998 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12901065a0 con 0x7f1290105b10 2024-05-27T09:17:23.013 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.955+0000 7f128fe00700 1 --2- 172.21.15.119:0/1131068998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1290105b10 0x7f1290105f20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f12800098a0 tx=0x7f1280009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9456608c946231d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:23.013 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.956+0000 7f128ea00700 1 -- 172.21.15.119:0/1131068998 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1280004180 con 0x7f1290105b10 2024-05-27T09:17:23.013 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.956+0000 7f128ea00700 1 -- 172.21.15.119:0/1131068998 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f12800042e0 con 0x7f1290105b10 2024-05-27T09:17:23.014 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.956+0000 7f128ea00700 1 -- 172.21.15.119:0/1131068998 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1280011a50 con 0x7f1290105b10 2024-05-27T09:17:23.014 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.956+0000 7f1297000700 1 -- 172.21.15.119:0/1131068998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1290105b10 msgr2=0x7f1290105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:23.014 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.956+0000 7f1297000700 1 --2- 172.21.15.119:0/1131068998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1290105b10 0x7f1290105f20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f12800098a0 tx=0x7f1280009bb0 comp rx=0 tx=0).stop 2024-05-27T09:17:23.014 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.956+0000 7f1297000700 1 -- 172.21.15.119:0/1131068998 shutdown_connections 2024-05-27T09:17:23.014 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.956+0000 7f1297000700 1 --2- 172.21.15.119:0/1131068998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1290105b10 0x7f1290105f20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:23.014 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.956+0000 7f1297000700 1 -- 172.21.15.119:0/1131068998 >> 172.21.15.119:0/1131068998 conn(0x7f1290101180 msgr2=0x7f1290103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:23.014 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.957+0000 7f1297000700 1 -- 172.21.15.119:0/1131068998 shutdown_connections 2024-05-27T09:17:23.014 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.957+0000 7f1297000700 1 -- 172.21.15.119:0/1131068998 wait complete. 2024-05-27T09:17:23.014 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.957+0000 7f1297000700 1 Processor -- start 2024-05-27T09:17:23.014 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.958+0000 7f1297000700 1 -- start start 2024-05-27T09:17:23.014 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.958+0000 7f1297000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1290105b10 0x7f129019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:23.014 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.958+0000 7f1297000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129019bf80 con 0x7f1290105b10 2024-05-27T09:17:23.014 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.958+0000 7f128fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1290105b10 0x7f129019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:23.014 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.958+0000 7f128fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1290105b10 0x7f129019ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59230/0 (socket says 172.21.15.119:59230) 2024-05-27T09:17:23.015 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.958+0000 7f128fe00700 1 -- 172.21.15.119:0/408895781 learned_addr learned my addr 172.21.15.119:0/408895781 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:23.015 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.958+0000 7f128fe00700 1 -- 172.21.15.119:0/408895781 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1280009550 con 0x7f1290105b10 2024-05-27T09:17:23.015 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.959+0000 7f128fe00700 1 --2- 172.21.15.119:0/408895781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1290105b10 0x7f129019ba40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1280000c00 tx=0x7f1280011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:23.015 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.959+0000 7f128cc00700 1 -- 172.21.15.119:0/408895781 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1280003b90 con 0x7f1290105b10 2024-05-27T09:17:23.015 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.959+0000 7f128cc00700 1 -- 172.21.15.119:0/408895781 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f12800248b0 con 0x7f1290105b10 2024-05-27T09:17:23.015 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.959+0000 7f128cc00700 1 -- 172.21.15.119:0/408895781 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f128001a8e0 con 0x7f1290105b10 2024-05-27T09:17:23.015 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.959+0000 7f1297000700 1 -- 172.21.15.119:0/408895781 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f129019c180 con 0x7f1290105b10 2024-05-27T09:17:23.015 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.959+0000 7f1297000700 1 -- 172.21.15.119:0/408895781 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129019c5a0 con 0x7f1290105b10 2024-05-27T09:17:23.015 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.960+0000 7f128cc00700 1 -- 172.21.15.119:0/408895781 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7f1280024a20 con 0x7f1290105b10 2024-05-27T09:17:23.015 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.960+0000 7f1297000700 1 -- 172.21.15.119:0/408895781 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1290062380 con 0x7f1290105b10 2024-05-27T09:17:23.015 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.961+0000 7f128cc00700 1 --2- 172.21.15.119:0/408895781 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7f12740383f0 0x7f127403a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:23.015 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.961+0000 7f128cc00700 1 -- 172.21.15.119:0/408895781 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f128004c110 con 0x7f1290105b10 2024-05-27T09:17:23.015 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.961+0000 7f128f400700 1 --2- 172.21.15.119:0/408895781 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7f12740383f0 0x7f127403a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:23.016 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.962+0000 7f128f400700 1 --2- 172.21.15.119:0/408895781 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7f12740383f0 0x7f127403a8a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f127c006fd0 tx=0x7f127c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:23.016 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:21.966+0000 7f128cc00700 1 -- 172.21.15.119:0/408895781 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1280024db0 con 0x7f1290105b10 2024-05-27T09:17:23.016 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:22.237+0000 7f1297000700 1 -- 172.21.15.119:0/408895781 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f129019f740 con 0x7f1290105b10 2024-05-27T09:17:23.016 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:22.295+0000 7f128cc00700 1 -- 172.21.15.119:0/408895781 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f128002d910 con 0x7f1290105b10 2024-05-27T09:17:23.016 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:22.297+0000 7f128cc00700 1 -- 172.21.15.119:0/408895781 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f128004eb40 con 0x7f1290105b10 2024-05-27T09:17:23.016 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:22.300+0000 7f1297000700 1 -- 172.21.15.119:0/408895781 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7f12740383f0 msgr2=0x7f127403a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:23.016 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:22.300+0000 7f1297000700 1 --2- 172.21.15.119:0/408895781 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7f12740383f0 0x7f127403a8a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f127c006fd0 tx=0x7f127c006e40 comp rx=0 tx=0).stop 2024-05-27T09:17:23.016 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:22.300+0000 7f1297000700 1 -- 172.21.15.119:0/408895781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1290105b10 msgr2=0x7f129019ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:23.016 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:22.300+0000 7f1297000700 1 --2- 172.21.15.119:0/408895781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1290105b10 0x7f129019ba40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1280000c00 tx=0x7f1280011ab0 comp rx=0 tx=0).stop 2024-05-27T09:17:23.016 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:22.301+0000 7f1297000700 1 -- 172.21.15.119:0/408895781 shutdown_connections 2024-05-27T09:17:23.016 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:22.301+0000 7f1297000700 1 --2- 172.21.15.119:0/408895781 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7f12740383f0 0x7f127403a8a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:23.016 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:22.301+0000 7f1297000700 1 --2- 172.21.15.119:0/408895781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1290105b10 0x7f129019ba40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:23.016 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:22.301+0000 7f1297000700 1 -- 172.21.15.119:0/408895781 >> 172.21.15.119:0/408895781 conn(0x7f1290101180 msgr2=0x7f1290101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:23.016 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:22.301+0000 7f1297000700 1 -- 172.21.15.119:0/408895781 shutdown_connections 2024-05-27T09:17:23.017 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:22.301+0000 7f1297000700 1 -- 172.21.15.119:0/408895781 wait complete. 2024-05-27T09:17:23.428 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:23 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/408895781' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-27T09:17:23.428 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:23 smithi119 ceph-mon[29082]: mgrmap e5: smithi119.ncwmog(active, since 6s) 2024-05-27T09:17:24.409 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:24 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/3749877717' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-27T09:17:24.419 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2024-05-27T09:17:24.419 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-27T09:17:24.419 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-27T09:17:24.419 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "active_name": "smithi119.ncwmog", 2024-05-27T09:17:24.419 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-27T09:17:24.419 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2024-05-27T09:17:24.419 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.580+0000 7f0103200700 1 Processor -- start 2024-05-27T09:17:24.420 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.580+0000 7f0103200700 1 -- start start 2024-05-27T09:17:24.420 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.580+0000 7f0103200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f010407ad00 0x7f0104079200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:24.420 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.580+0000 7f0103200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0104079740 con 0x7f010407ad00 2024-05-27T09:17:24.420 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.581+0000 7f0101e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f010407ad00 0x7f0104079200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:24.420 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.581+0000 7f0101e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f010407ad00 0x7f0104079200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59242/0 (socket says 172.21.15.119:59242) 2024-05-27T09:17:24.420 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.581+0000 7f0101e00700 1 -- 172.21.15.119:0/498657826 learned_addr learned my addr 172.21.15.119:0/498657826 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:24.420 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.581+0000 7f0101e00700 1 -- 172.21.15.119:0/498657826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0104079880 con 0x7f010407ad00 2024-05-27T09:17:24.420 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.582+0000 7f0101e00700 1 --2- 172.21.15.119:0/498657826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f010407ad00 0x7f0104079200 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f00f40098a0 tx=0x7f00f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935972f55db13e27 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:24.420 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.582+0000 7f0100a00700 1 -- 172.21.15.119:0/498657826 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00f4004180 con 0x7f010407ad00 2024-05-27T09:17:24.420 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.582+0000 7f0100a00700 1 -- 172.21.15.119:0/498657826 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f00f40042e0 con 0x7f010407ad00 2024-05-27T09:17:24.420 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.582+0000 7f0100a00700 1 -- 172.21.15.119:0/498657826 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00f4011a50 con 0x7f010407ad00 2024-05-27T09:17:24.420 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.582+0000 7f0103200700 1 -- 172.21.15.119:0/498657826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f010407ad00 msgr2=0x7f0104079200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:24.420 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.582+0000 7f0103200700 1 --2- 172.21.15.119:0/498657826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f010407ad00 0x7f0104079200 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f00f40098a0 tx=0x7f00f4009bb0 comp rx=0 tx=0).stop 2024-05-27T09:17:24.421 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.583+0000 7f0103200700 1 -- 172.21.15.119:0/498657826 shutdown_connections 2024-05-27T09:17:24.421 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.583+0000 7f0103200700 1 --2- 172.21.15.119:0/498657826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f010407ad00 0x7f0104079200 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:24.421 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.583+0000 7f0103200700 1 -- 172.21.15.119:0/498657826 >> 172.21.15.119:0/498657826 conn(0x7f0104100fb0 msgr2=0x7f01041033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:24.421 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.583+0000 7f0103200700 1 -- 172.21.15.119:0/498657826 shutdown_connections 2024-05-27T09:17:24.421 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.583+0000 7f0103200700 1 -- 172.21.15.119:0/498657826 wait complete. 2024-05-27T09:17:24.421 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.584+0000 7f0103200700 1 Processor -- start 2024-05-27T09:17:24.421 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.584+0000 7f0103200700 1 -- start start 2024-05-27T09:17:24.421 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.584+0000 7f0103200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f010407ad00 0x7f010419b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:24.421 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.584+0000 7f0103200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010419bdc0 con 0x7f010407ad00 2024-05-27T09:17:24.421 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.584+0000 7f0101e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f010407ad00 0x7f010419b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:24.421 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.585+0000 7f0101e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f010407ad00 0x7f010419b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59244/0 (socket says 172.21.15.119:59244) 2024-05-27T09:17:24.421 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.585+0000 7f0101e00700 1 -- 172.21.15.119:0/3749877717 learned_addr learned my addr 172.21.15.119:0/3749877717 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:24.421 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.585+0000 7f0101e00700 1 -- 172.21.15.119:0/3749877717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f4009550 con 0x7f010407ad00 2024-05-27T09:17:24.421 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.585+0000 7f0101e00700 1 --2- 172.21.15.119:0/3749877717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f010407ad00 0x7f010419b880 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f00f4000c00 tx=0x7f00f4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:24.422 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.585+0000 7f00faa00700 1 -- 172.21.15.119:0/3749877717 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00f4003b90 con 0x7f010407ad00 2024-05-27T09:17:24.422 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.585+0000 7f00faa00700 1 -- 172.21.15.119:0/3749877717 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f00f40248b0 con 0x7f010407ad00 2024-05-27T09:17:24.422 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.585+0000 7f00faa00700 1 -- 172.21.15.119:0/3749877717 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00f401a8e0 con 0x7f010407ad00 2024-05-27T09:17:24.422 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.585+0000 7f0103200700 1 -- 172.21.15.119:0/3749877717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010419bfc0 con 0x7f010407ad00 2024-05-27T09:17:24.422 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.586+0000 7f0103200700 1 -- 172.21.15.119:0/3749877717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010419c3e0 con 0x7f010407ad00 2024-05-27T09:17:24.422 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.587+0000 7f00faa00700 1 -- 172.21.15.119:0/3749877717 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f00f4024a20 con 0x7f010407ad00 2024-05-27T09:17:24.422 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.587+0000 7f0103200700 1 -- 172.21.15.119:0/3749877717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0104062380 con 0x7f010407ad00 2024-05-27T09:17:24.422 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.587+0000 7f00faa00700 1 --2- 172.21.15.119:0/3749877717 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7f00e8038440 0x7f00e803a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:24.422 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.587+0000 7f00faa00700 1 -- 172.21.15.119:0/3749877717 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f00f404c050 con 0x7f010407ad00 2024-05-27T09:17:24.422 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.587+0000 7f0101400700 1 -- 172.21.15.119:0/3749877717 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7f00e8038440 msgr2=0x7f00e803a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1743331737 2024-05-27T09:17:24.422 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.587+0000 7f0101400700 1 --2- 172.21.15.119:0/3749877717 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7f00e8038440 0x7f00e803a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T09:17:24.422 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.592+0000 7f00faa00700 1 -- 172.21.15.119:0/3749877717 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f00f40189a0 con 0x7f010407ad00 2024-05-27T09:17:24.423 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.788+0000 7f0101400700 1 -- 172.21.15.119:0/3749877717 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7f00e8038440 msgr2=0x7f00e803a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1743331737 2024-05-27T09:17:24.423 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.788+0000 7f0101400700 1 --2- 172.21.15.119:0/3749877717 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7f00e8038440 0x7f00e803a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T09:17:24.423 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.850+0000 7f0103200700 1 -- 172.21.15.119:0/3749877717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f010419f560 con 0x7f010407ad00 2024-05-27T09:17:24.423 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.851+0000 7f00faa00700 1 -- 172.21.15.119:0/3749877717 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f00f4018350 con 0x7f010407ad00 2024-05-27T09:17:24.423 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.854+0000 7f0103200700 1 -- 172.21.15.119:0/3749877717 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7f00e8038440 msgr2=0x7f00e803a8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T09:17:24.423 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.854+0000 7f0103200700 1 --2- 172.21.15.119:0/3749877717 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7f00e8038440 0x7f00e803a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:24.423 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.854+0000 7f0103200700 1 -- 172.21.15.119:0/3749877717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f010407ad00 msgr2=0x7f010419b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:24.423 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.854+0000 7f0103200700 1 --2- 172.21.15.119:0/3749877717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f010407ad00 0x7f010419b880 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f00f4000c00 tx=0x7f00f4011ab0 comp rx=0 tx=0).stop 2024-05-27T09:17:24.423 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.854+0000 7f0103200700 1 -- 172.21.15.119:0/3749877717 shutdown_connections 2024-05-27T09:17:24.423 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.855+0000 7f0103200700 1 --2- 172.21.15.119:0/3749877717 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7f00e8038440 0x7f00e803a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:24.423 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.855+0000 7f0103200700 1 --2- 172.21.15.119:0/3749877717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f010407ad00 0x7f010419b880 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:24.423 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.855+0000 7f0103200700 1 -- 172.21.15.119:0/3749877717 >> 172.21.15.119:0/3749877717 conn(0x7f0104100fb0 msgr2=0x7f01041026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:24.423 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.855+0000 7f0103200700 1 -- 172.21.15.119:0/3749877717 shutdown_connections 2024-05-27T09:17:24.423 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:23.855+0000 7f0103200700 1 -- 172.21.15.119:0/3749877717 wait complete. 2024-05-27T09:17:24.424 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for the mgr to restart... 2024-05-27T09:17:24.424 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mgr epoch 5... 2024-05-27T09:17:31.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:31 smithi119 ceph-mon[29082]: Active manager daemon smithi119.ncwmog restarted 2024-05-27T09:17:31.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:31 smithi119 ceph-mon[29082]: Activating manager daemon smithi119.ncwmog 2024-05-27T09:17:31.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:31 smithi119 ceph-mon[29082]: osdmap e2: 0 total, 0 up, 0 in 2024-05-27T09:17:31.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:31 smithi119 ceph-mon[29082]: mgrmap e6: smithi119.ncwmog(active, starting, since 0.00945328s) 2024-05-27T09:17:31.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:31 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi119"}]: dispatch 2024-05-27T09:17:31.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:31 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr metadata", "who": "smithi119.ncwmog", "id": "smithi119.ncwmog"}]: dispatch 2024-05-27T09:17:31.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:31 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T09:17:31.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:31 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T09:17:31.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:31 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T09:17:31.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:31 smithi119 ceph-mon[29082]: Manager daemon smithi119.ncwmog is now available 2024-05-27T09:17:31.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:31 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' 2024-05-27T09:17:31.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:31 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' 2024-05-27T09:17:32.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:32 smithi119 ceph-mon[29082]: Found migration_current of "None". Setting to last migration. 2024-05-27T09:17:32.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:32 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:17:32.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:32 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:17:32.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:32 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:17:32.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:32 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi119.ncwmog/mirror_snapshot_schedule"}]: dispatch 2024-05-27T09:17:32.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:32 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi119.ncwmog/trash_purge_schedule"}]: dispatch 2024-05-27T09:17:32.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:32 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' 2024-05-27T09:17:32.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:32 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' 2024-05-27T09:17:32.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:32 smithi119 ceph-mon[29082]: mgrmap e7: smithi119.ncwmog(active, since 1.01677s) 2024-05-27T09:17:32.679 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2024-05-27T09:17:32.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-27T09:17:32.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-27T09:17:32.680 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2024-05-27T09:17:32.681 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.967+0000 7fdf69a00700 1 Processor -- start 2024-05-27T09:17:32.682 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.967+0000 7fdf69a00700 1 -- start start 2024-05-27T09:17:32.682 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.968+0000 7fdf69a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf6407ace0 0x7fdf640791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:32.682 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.968+0000 7fdf69a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf64079720 con 0x7fdf6407ace0 2024-05-27T09:17:32.682 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.968+0000 7fdf62a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf6407ace0 0x7fdf640791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:32.682 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.968+0000 7fdf62a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf6407ace0 0x7fdf640791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59256/0 (socket says 172.21.15.119:59256) 2024-05-27T09:17:32.682 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.968+0000 7fdf62a00700 1 -- 172.21.15.119:0/4169261935 learned_addr learned my addr 172.21.15.119:0/4169261935 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:32.682 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.969+0000 7fdf62a00700 1 -- 172.21.15.119:0/4169261935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf64079860 con 0x7fdf6407ace0 2024-05-27T09:17:32.682 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.969+0000 7fdf62a00700 1 --2- 172.21.15.119:0/4169261935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf6407ace0 0x7fdf640791e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdf540098a0 tx=0x7fdf54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4240260179ff0b7d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:32.687 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.969+0000 7fdf61600700 1 -- 172.21.15.119:0/4169261935 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf54004180 con 0x7fdf6407ace0 2024-05-27T09:17:32.688 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.970+0000 7fdf61600700 1 -- 172.21.15.119:0/4169261935 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdf540042e0 con 0x7fdf6407ace0 2024-05-27T09:17:32.688 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.970+0000 7fdf61600700 1 -- 172.21.15.119:0/4169261935 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf54011a50 con 0x7fdf6407ace0 2024-05-27T09:17:32.688 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.970+0000 7fdf69a00700 1 -- 172.21.15.119:0/4169261935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf6407ace0 msgr2=0x7fdf640791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:32.688 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.970+0000 7fdf69a00700 1 --2- 172.21.15.119:0/4169261935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf6407ace0 0x7fdf640791e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdf540098a0 tx=0x7fdf54009bb0 comp rx=0 tx=0).stop 2024-05-27T09:17:32.688 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.970+0000 7fdf69a00700 1 -- 172.21.15.119:0/4169261935 shutdown_connections 2024-05-27T09:17:32.688 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.970+0000 7fdf69a00700 1 --2- 172.21.15.119:0/4169261935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf6407ace0 0x7fdf640791e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:32.688 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.970+0000 7fdf69a00700 1 -- 172.21.15.119:0/4169261935 >> 172.21.15.119:0/4169261935 conn(0x7fdf64101110 msgr2=0x7fdf64103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:32.688 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.971+0000 7fdf69a00700 1 -- 172.21.15.119:0/4169261935 shutdown_connections 2024-05-27T09:17:32.688 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.971+0000 7fdf69a00700 1 -- 172.21.15.119:0/4169261935 wait complete. 2024-05-27T09:17:32.688 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.972+0000 7fdf69a00700 1 Processor -- start 2024-05-27T09:17:32.688 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.972+0000 7fdf69a00700 1 -- start start 2024-05-27T09:17:32.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.972+0000 7fdf69a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf6407ace0 0x7fdf6419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:32.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.972+0000 7fdf69a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6419bf10 con 0x7fdf6407ace0 2024-05-27T09:17:32.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.973+0000 7fdf62a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf6407ace0 0x7fdf6419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:32.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.973+0000 7fdf62a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf6407ace0 0x7fdf6419b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59264/0 (socket says 172.21.15.119:59264) 2024-05-27T09:17:32.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.973+0000 7fdf62a00700 1 -- 172.21.15.119:0/3626544538 learned_addr learned my addr 172.21.15.119:0/3626544538 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:32.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.973+0000 7fdf62a00700 1 -- 172.21.15.119:0/3626544538 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf54009550 con 0x7fdf6407ace0 2024-05-27T09:17:32.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.973+0000 7fdf62a00700 1 --2- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf6407ace0 0x7fdf6419b9d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdf54000c00 tx=0x7fdf54011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:32.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.973+0000 7fdf5b400700 1 -- 172.21.15.119:0/3626544538 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf54003b90 con 0x7fdf6407ace0 2024-05-27T09:17:32.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.973+0000 7fdf5b400700 1 -- 172.21.15.119:0/3626544538 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdf540248b0 con 0x7fdf6407ace0 2024-05-27T09:17:32.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.973+0000 7fdf5b400700 1 -- 172.21.15.119:0/3626544538 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf5401a8e0 con 0x7fdf6407ace0 2024-05-27T09:17:32.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.974+0000 7fdf69a00700 1 -- 172.21.15.119:0/3626544538 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf6419c110 con 0x7fdf6407ace0 2024-05-27T09:17:32.689 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.974+0000 7fdf69a00700 1 -- 172.21.15.119:0/3626544538 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf6419c530 con 0x7fdf6407ace0 2024-05-27T09:17:32.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.975+0000 7fdf5b400700 1 -- 172.21.15.119:0/3626544538 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fdf54024a20 con 0x7fdf6407ace0 2024-05-27T09:17:32.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.975+0000 7fdf5b400700 1 --2- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fdf48038440 0x7fdf4803a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:32.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.975+0000 7fdf5b400700 1 -- 172.21.15.119:0/3626544538 --> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf4803b000 con 0x7fdf48038440 2024-05-27T09:17:32.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.975+0000 7fdf5b400700 1 -- 172.21.15.119:0/3626544538 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdf5404c080 con 0x7fdf6407ace0 2024-05-27T09:17:32.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.975+0000 7fdf62000700 1 -- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fdf48038440 msgr2=0x7fdf4803a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1743331737 2024-05-27T09:17:32.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:24.975+0000 7fdf62000700 1 --2- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fdf48038440 0x7fdf4803a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T09:17:32.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:25.176+0000 7fdf62000700 1 -- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fdf48038440 msgr2=0x7fdf4803a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1743331737 2024-05-27T09:17:32.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:25.176+0000 7fdf62000700 1 --2- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fdf48038440 0x7fdf4803a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T09:17:32.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:25.577+0000 7fdf62000700 1 -- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fdf48038440 msgr2=0x7fdf4803a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1743331737 2024-05-27T09:17:32.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:25.577+0000 7fdf62000700 1 --2- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fdf48038440 0x7fdf4803a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-27T09:17:32.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:26.378+0000 7fdf62000700 1 -- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fdf48038440 msgr2=0x7fdf4803a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1743331737 2024-05-27T09:17:32.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:26.378+0000 7fdf62000700 1 --2- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fdf48038440 0x7fdf4803a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-27T09:17:32.690 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:27.980+0000 7fdf62000700 1 -- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fdf48038440 msgr2=0x7fdf4803a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1743331737 2024-05-27T09:17:32.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:27.980+0000 7fdf62000700 1 --2- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fdf48038440 0x7fdf4803a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-27T09:17:32.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:30.961+0000 7fdf5b400700 1 -- 172.21.15.119:0/3626544538 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mgrmap(e 6) v1 ==== 44851+0+0 (secure 0 0 0) 0x7fdf5402d910 con 0x7fdf6407ace0 2024-05-27T09:17:32.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:30.962+0000 7fdf5b400700 1 -- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fdf48038440 msgr2=0x7fdf4803a8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T09:17:32.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:30.962+0000 7fdf5b400700 1 --2- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fdf48038440 0x7fdf4803a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:32.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:31.966+0000 7fdf5b400700 1 -- 172.21.15.119:0/3626544538 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 7) v1 ==== 44978+0+0 (secure 0 0 0) 0x7fdf5404dc30 con 0x7fdf6407ace0 2024-05-27T09:17:32.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:31.967+0000 7fdf5b400700 1 --2- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fdf4803bd10 0x7fdf4803e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:32.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:31.967+0000 7fdf5b400700 1 -- 172.21.15.119:0/3626544538 --> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf4803b000 con 0x7fdf4803bd10 2024-05-27T09:17:32.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:31.969+0000 7fdf62000700 1 --2- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fdf4803bd10 0x7fdf4803e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:32.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:31.973+0000 7fdf62000700 1 --2- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fdf4803bd10 0x7fdf4803e0f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdf50003a10 tx=0x7fdf500092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:32.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:31.975+0000 7fdf5b400700 1 -- 172.21.15.119:0/3626544538 <== mgr.14120 v2:172.21.15.119:6800/1553098156 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fdf4803b000 con 0x7fdf4803bd10 2024-05-27T09:17:32.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:31.982+0000 7fdf69a00700 1 -- 172.21.15.119:0/3626544538 --> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fdf6419cd10 con 0x7fdf4803bd10 2024-05-27T09:17:32.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:31.982+0000 7fdf5b400700 1 -- 172.21.15.119:0/3626544538 <== mgr.14120 v2:172.21.15.119:6800/1553098156 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fdf6419cd10 con 0x7fdf4803bd10 2024-05-27T09:17:32.691 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:31.983+0000 7fdf69a00700 1 -- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fdf4803bd10 msgr2=0x7fdf4803e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:32.692 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:31.983+0000 7fdf69a00700 1 --2- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fdf4803bd10 0x7fdf4803e0f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdf50003a10 tx=0x7fdf500092b0 comp rx=0 tx=0).stop 2024-05-27T09:17:32.692 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:31.983+0000 7fdf69a00700 1 -- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf6407ace0 msgr2=0x7fdf6419b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:32.692 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:31.983+0000 7fdf69a00700 1 --2- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf6407ace0 0x7fdf6419b9d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdf54000c00 tx=0x7fdf54011ab0 comp rx=0 tx=0).stop 2024-05-27T09:17:32.693 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:31.983+0000 7fdf69a00700 1 -- 172.21.15.119:0/3626544538 shutdown_connections 2024-05-27T09:17:32.693 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:31.983+0000 7fdf69a00700 1 --2- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fdf4803bd10 0x7fdf4803e0f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:32.693 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:31.983+0000 7fdf69a00700 1 --2- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf6407ace0 0x7fdf6419b9d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:32.693 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:31.983+0000 7fdf69a00700 1 --2- 172.21.15.119:0/3626544538 >> [v2:172.21.15.119:6800/1743331737,v1:172.21.15.119:6801/1743331737] conn(0x7fdf48038440 0x7fdf4803a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:32.693 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:31.983+0000 7fdf69a00700 1 -- 172.21.15.119:0/3626544538 >> 172.21.15.119:0/3626544538 conn(0x7fdf64101110 msgr2=0x7fdf64101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:32.693 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:31.983+0000 7fdf69a00700 1 -- 172.21.15.119:0/3626544538 shutdown_connections 2024-05-27T09:17:32.693 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:31.983+0000 7fdf69a00700 1 -- 172.21.15.119:0/3626544538 wait complete. 2024-05-27T09:17:32.693 INFO:teuthology.orchestra.run.smithi119.stdout:mgr epoch 5 is available 2024-05-27T09:17:32.693 INFO:teuthology.orchestra.run.smithi119.stdout:Setting orchestrator backend to cephadm... 2024-05-27T09:17:33.235 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:33 smithi119 ceph-mon[29082]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-27T09:17:33.235 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:33 smithi119 ceph-mon[29082]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-27T09:17:33.235 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:33 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:17:34.163 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:34 smithi119 ceph-mon[29082]: [27/May/2024:09:17:32] ENGINE Bus STARTING 2024-05-27T09:17:34.163 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:34 smithi119 ceph-mon[29082]: [27/May/2024:09:17:32] ENGINE Serving on https://172.21.15.119:7150 2024-05-27T09:17:34.163 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:34 smithi119 ceph-mon[29082]: [27/May/2024:09:17:32] ENGINE Serving on http://172.21.15.119:8765 2024-05-27T09:17:34.163 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:34 smithi119 ceph-mon[29082]: [27/May/2024:09:17:32] ENGINE Bus STARTED 2024-05-27T09:17:34.163 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:34 smithi119 ceph-mon[29082]: mgrmap e8: smithi119.ncwmog(active, since 2s) 2024-05-27T09:17:34.163 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:34 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' 2024-05-27T09:17:34.163 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:34 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:17:34.176 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.280+0000 7f0f22400700 1 Processor -- start 2024-05-27T09:17:34.176 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.280+0000 7f0f22400700 1 -- start start 2024-05-27T09:17:34.176 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.280+0000 7f0f22400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f1c0802e0 0x7f0f1c10ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:34.176 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.280+0000 7f0f22400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c10f210 con 0x7f0f1c0802e0 2024-05-27T09:17:34.176 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.281+0000 7f0f1b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f1c0802e0 0x7f0f1c10ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:34.176 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.281+0000 7f0f1b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f1c0802e0 0x7f0f1c10ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55178/0 (socket says 172.21.15.119:55178) 2024-05-27T09:17:34.176 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.281+0000 7f0f1b400700 1 -- 172.21.15.119:0/599306456 learned_addr learned my addr 172.21.15.119:0/599306456 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:34.176 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.281+0000 7f0f1b400700 1 -- 172.21.15.119:0/599306456 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1c10f350 con 0x7f0f1c0802e0 2024-05-27T09:17:34.176 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.281+0000 7f0f1b400700 1 --2- 172.21.15.119:0/599306456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f1c0802e0 0x7f0f1c10ecd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c0098a0 tx=0x7f0f0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725acb0a8426ce14 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:34.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.282+0000 7f0f1aa00700 1 -- 172.21.15.119:0/599306456 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f0c004180 con 0x7f0f1c0802e0 2024-05-27T09:17:34.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.282+0000 7f0f1aa00700 1 -- 172.21.15.119:0/599306456 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0f0c0042e0 con 0x7f0f1c0802e0 2024-05-27T09:17:34.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.282+0000 7f0f1aa00700 1 -- 172.21.15.119:0/599306456 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f0c011a50 con 0x7f0f1c0802e0 2024-05-27T09:17:34.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.282+0000 7f0f22400700 1 -- 172.21.15.119:0/599306456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f1c0802e0 msgr2=0x7f0f1c10ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:34.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.282+0000 7f0f22400700 1 --2- 172.21.15.119:0/599306456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f1c0802e0 0x7f0f1c10ecd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c0098a0 tx=0x7f0f0c009bb0 comp rx=0 tx=0).stop 2024-05-27T09:17:34.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.283+0000 7f0f22400700 1 -- 172.21.15.119:0/599306456 shutdown_connections 2024-05-27T09:17:34.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.283+0000 7f0f22400700 1 --2- 172.21.15.119:0/599306456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f1c0802e0 0x7f0f1c10ecd0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:34.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.283+0000 7f0f22400700 1 -- 172.21.15.119:0/599306456 >> 172.21.15.119:0/599306456 conn(0x7f0f1c07b220 msgr2=0x7f0f1c07d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:34.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.283+0000 7f0f22400700 1 -- 172.21.15.119:0/599306456 shutdown_connections 2024-05-27T09:17:34.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.283+0000 7f0f22400700 1 -- 172.21.15.119:0/599306456 wait complete. 2024-05-27T09:17:34.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.284+0000 7f0f22400700 1 Processor -- start 2024-05-27T09:17:34.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.284+0000 7f0f22400700 1 -- start start 2024-05-27T09:17:34.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.284+0000 7f0f22400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f1c0802e0 0x7f0f1c19fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:34.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.284+0000 7f0f22400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c1a0370 con 0x7f0f1c0802e0 2024-05-27T09:17:34.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.285+0000 7f0f1b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f1c0802e0 0x7f0f1c19fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:34.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.285+0000 7f0f1b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f1c0802e0 0x7f0f1c19fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55188/0 (socket says 172.21.15.119:55188) 2024-05-27T09:17:34.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.285+0000 7f0f1b400700 1 -- 172.21.15.119:0/2150526404 learned_addr learned my addr 172.21.15.119:0/2150526404 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:34.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.285+0000 7f0f1b400700 1 -- 172.21.15.119:0/2150526404 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c009550 con 0x7f0f1c0802e0 2024-05-27T09:17:34.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.285+0000 7f0f1b400700 1 --2- 172.21.15.119:0/2150526404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f1c0802e0 0x7f0f1c19fe30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c006af0 tx=0x7f0f0c011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:34.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.285+0000 7f0f18c00700 1 -- 172.21.15.119:0/2150526404 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f0c003b90 con 0x7f0f1c0802e0 2024-05-27T09:17:34.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.285+0000 7f0f18c00700 1 -- 172.21.15.119:0/2150526404 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0f0c01a8a0 con 0x7f0f1c0802e0 2024-05-27T09:17:34.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.285+0000 7f0f18c00700 1 -- 172.21.15.119:0/2150526404 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f0c019880 con 0x7f0f1c0802e0 2024-05-27T09:17:34.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.286+0000 7f0f22400700 1 -- 172.21.15.119:0/2150526404 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f1c1a0570 con 0x7f0f1c0802e0 2024-05-27T09:17:34.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.286+0000 7f0f22400700 1 -- 172.21.15.119:0/2150526404 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f1c1a0990 con 0x7f0f1c0802e0 2024-05-27T09:17:34.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.287+0000 7f0f22400700 1 -- 172.21.15.119:0/2150526404 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f1c062380 con 0x7f0f1c0802e0 2024-05-27T09:17:34.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.287+0000 7f0f18c00700 1 -- 172.21.15.119:0/2150526404 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f0f0c01a430 con 0x7f0f1c0802e0 2024-05-27T09:17:34.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.287+0000 7f0f18c00700 1 --2- 172.21.15.119:0/2150526404 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f0f000383b0 0x7f0f0003a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:34.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.287+0000 7f0f18c00700 1 -- 172.21.15.119:0/2150526404 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0f0c04c7a0 con 0x7f0f1c0802e0 2024-05-27T09:17:34.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.287+0000 7f0f13e00700 1 --2- 172.21.15.119:0/2150526404 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f0f000383b0 0x7f0f0003a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:34.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.288+0000 7f0f13e00700 1 --2- 172.21.15.119:0/2150526404 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f0f000383b0 0x7f0f0003a860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0f04006fd0 tx=0x7f0f04006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:34.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.292+0000 7f0f18c00700 1 -- 172.21.15.119:0/2150526404 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0f0c0189a0 con 0x7f0f1c0802e0 2024-05-27T09:17:34.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.520+0000 7f0f22400700 1 -- 172.21.15.119:0/2150526404 --> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f1c07d580 con 0x7f0f000383b0 2024-05-27T09:17:34.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.539+0000 7f0f18c00700 1 -- 172.21.15.119:0/2150526404 <== mgr.14120 v2:172.21.15.119:6800/1553098156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0f1c07d580 con 0x7f0f000383b0 2024-05-27T09:17:34.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.541+0000 7f0f22400700 1 -- 172.21.15.119:0/2150526404 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f0f000383b0 msgr2=0x7f0f0003a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:34.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.541+0000 7f0f22400700 1 --2- 172.21.15.119:0/2150526404 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f0f000383b0 0x7f0f0003a860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0f04006fd0 tx=0x7f0f04006e40 comp rx=0 tx=0).stop 2024-05-27T09:17:34.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.541+0000 7f0f22400700 1 -- 172.21.15.119:0/2150526404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f1c0802e0 msgr2=0x7f0f1c19fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:34.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.541+0000 7f0f22400700 1 --2- 172.21.15.119:0/2150526404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f1c0802e0 0x7f0f1c19fe30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c006af0 tx=0x7f0f0c011a90 comp rx=0 tx=0).stop 2024-05-27T09:17:34.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.541+0000 7f0f22400700 1 -- 172.21.15.119:0/2150526404 shutdown_connections 2024-05-27T09:17:34.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.541+0000 7f0f22400700 1 --2- 172.21.15.119:0/2150526404 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f0f000383b0 0x7f0f0003a860 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:34.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.541+0000 7f0f22400700 1 --2- 172.21.15.119:0/2150526404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f1c0802e0 0x7f0f1c19fe30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:34.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.542+0000 7f0f22400700 1 -- 172.21.15.119:0/2150526404 >> 172.21.15.119:0/2150526404 conn(0x7f0f1c07b220 msgr2=0x7f0f1c07c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:34.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.542+0000 7f0f22400700 1 -- 172.21.15.119:0/2150526404 shutdown_connections 2024-05-27T09:17:34.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:33.542+0000 7f0f22400700 1 -- 172.21.15.119:0/2150526404 wait complete. 2024-05-27T09:17:35.364 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:35 smithi119 ceph-mon[29082]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:17:35.696 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-27T09:17:35.696 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.760+0000 7f170a400700 1 Processor -- start 2024-05-27T09:17:35.696 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.760+0000 7f170a400700 1 -- start start 2024-05-27T09:17:35.696 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.760+0000 7f170a400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1704105b10 0x7f1704105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:35.696 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.760+0000 7f170a400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1704106460 con 0x7f1704105b10 2024-05-27T09:17:35.696 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.761+0000 7f1703400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1704105b10 0x7f1704105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:35.696 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.761+0000 7f1703400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1704105b10 0x7f1704105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55190/0 (socket says 172.21.15.119:55190) 2024-05-27T09:17:35.696 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.761+0000 7f1703400700 1 -- 172.21.15.119:0/1624767607 learned_addr learned my addr 172.21.15.119:0/1624767607 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:35.696 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.761+0000 7f1703400700 1 -- 172.21.15.119:0/1624767607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17041065a0 con 0x7f1704105b10 2024-05-27T09:17:35.696 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.762+0000 7f1703400700 1 --2- 172.21.15.119:0/1624767607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1704105b10 0x7f1704105f20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f16f40098a0 tx=0x7f16f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9219874f4ba12ce3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:35.697 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.762+0000 7f1702000700 1 -- 172.21.15.119:0/1624767607 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16f4004180 con 0x7f1704105b10 2024-05-27T09:17:35.697 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.762+0000 7f1702000700 1 -- 172.21.15.119:0/1624767607 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f16f40042e0 con 0x7f1704105b10 2024-05-27T09:17:35.697 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.762+0000 7f1702000700 1 -- 172.21.15.119:0/1624767607 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16f4011a50 con 0x7f1704105b10 2024-05-27T09:17:35.697 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.763+0000 7f170a400700 1 -- 172.21.15.119:0/1624767607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1704105b10 msgr2=0x7f1704105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:35.701 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.763+0000 7f170a400700 1 --2- 172.21.15.119:0/1624767607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1704105b10 0x7f1704105f20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f16f40098a0 tx=0x7f16f4009bb0 comp rx=0 tx=0).stop 2024-05-27T09:17:35.701 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.763+0000 7f170a400700 1 -- 172.21.15.119:0/1624767607 shutdown_connections 2024-05-27T09:17:35.701 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.763+0000 7f170a400700 1 --2- 172.21.15.119:0/1624767607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1704105b10 0x7f1704105f20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:35.701 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.763+0000 7f170a400700 1 -- 172.21.15.119:0/1624767607 >> 172.21.15.119:0/1624767607 conn(0x7f1704101180 msgr2=0x7f1704103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:35.701 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.763+0000 7f170a400700 1 -- 172.21.15.119:0/1624767607 shutdown_connections 2024-05-27T09:17:35.701 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.763+0000 7f170a400700 1 -- 172.21.15.119:0/1624767607 wait complete. 2024-05-27T09:17:35.701 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.764+0000 7f170a400700 1 Processor -- start 2024-05-27T09:17:35.701 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.764+0000 7f170a400700 1 -- start start 2024-05-27T09:17:35.701 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.765+0000 7f170a400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1704105b10 0x7f170419ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:35.701 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.765+0000 7f170a400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170419bf50 con 0x7f1704105b10 2024-05-27T09:17:35.701 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.765+0000 7f1703400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1704105b10 0x7f170419ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:35.702 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.765+0000 7f1703400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1704105b10 0x7f170419ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55200/0 (socket says 172.21.15.119:55200) 2024-05-27T09:17:35.702 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.765+0000 7f1703400700 1 -- 172.21.15.119:0/4010703996 learned_addr learned my addr 172.21.15.119:0/4010703996 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:35.702 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.765+0000 7f1703400700 1 -- 172.21.15.119:0/4010703996 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f4009550 con 0x7f1704105b10 2024-05-27T09:17:35.702 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.766+0000 7f1703400700 1 --2- 172.21.15.119:0/4010703996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1704105b10 0x7f170419ba10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f16f4000c00 tx=0x7f16f4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:35.702 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.766+0000 7f16fbe00700 1 -- 172.21.15.119:0/4010703996 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16f4003b90 con 0x7f1704105b10 2024-05-27T09:17:35.702 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.766+0000 7f16fbe00700 1 -- 172.21.15.119:0/4010703996 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f16f40248b0 con 0x7f1704105b10 2024-05-27T09:17:35.702 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.766+0000 7f16fbe00700 1 -- 172.21.15.119:0/4010703996 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16f401a8e0 con 0x7f1704105b10 2024-05-27T09:17:35.702 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.766+0000 7f170a400700 1 -- 172.21.15.119:0/4010703996 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f170419c150 con 0x7f1704105b10 2024-05-27T09:17:35.702 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.766+0000 7f170a400700 1 -- 172.21.15.119:0/4010703996 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170419c570 con 0x7f1704105b10 2024-05-27T09:17:35.702 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.767+0000 7f16fbe00700 1 -- 172.21.15.119:0/4010703996 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f16f4024a20 con 0x7f1704105b10 2024-05-27T09:17:35.702 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.767+0000 7f170a400700 1 -- 172.21.15.119:0/4010703996 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1704062380 con 0x7f1704105b10 2024-05-27T09:17:35.702 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.768+0000 7f16fbe00700 1 --2- 172.21.15.119:0/4010703996 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f16e8038440 0x7f16e803a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:35.703 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.768+0000 7f16fbe00700 1 -- 172.21.15.119:0/4010703996 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f16f404c560 con 0x7f1704105b10 2024-05-27T09:17:35.703 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.768+0000 7f1702a00700 1 --2- 172.21.15.119:0/4010703996 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f16e8038440 0x7f16e803a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:35.703 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.768+0000 7f1702a00700 1 --2- 172.21.15.119:0/4010703996 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f16e8038440 0x7f16e803a8f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f16f0006fd0 tx=0x7f16f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:35.703 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.773+0000 7f16fbe00700 1 -- 172.21.15.119:0/4010703996 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f16f4024db0 con 0x7f1704105b10 2024-05-27T09:17:35.703 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.991+0000 7f170a400700 1 -- 172.21.15.119:0/4010703996 --> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f1704102610 con 0x7f16e8038440 2024-05-27T09:17:35.703 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.992+0000 7f16fbe00700 1 -- 172.21.15.119:0/4010703996 <== mgr.14120 v2:172.21.15.119:6800/1553098156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f1704102610 con 0x7f16e8038440 2024-05-27T09:17:35.703 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.995+0000 7f170a400700 1 -- 172.21.15.119:0/4010703996 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f16e8038440 msgr2=0x7f16e803a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:35.703 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.995+0000 7f170a400700 1 --2- 172.21.15.119:0/4010703996 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f16e8038440 0x7f16e803a8f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f16f0006fd0 tx=0x7f16f0006e40 comp rx=0 tx=0).stop 2024-05-27T09:17:35.703 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.995+0000 7f170a400700 1 -- 172.21.15.119:0/4010703996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1704105b10 msgr2=0x7f170419ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:35.703 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.995+0000 7f170a400700 1 --2- 172.21.15.119:0/4010703996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1704105b10 0x7f170419ba10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f16f4000c00 tx=0x7f16f4011ab0 comp rx=0 tx=0).stop 2024-05-27T09:17:35.703 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.995+0000 7f170a400700 1 -- 172.21.15.119:0/4010703996 shutdown_connections 2024-05-27T09:17:35.703 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.995+0000 7f170a400700 1 --2- 172.21.15.119:0/4010703996 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f16e8038440 0x7f16e803a8f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:35.703 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.995+0000 7f170a400700 1 --2- 172.21.15.119:0/4010703996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1704105b10 0x7f170419ba10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:35.704 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.995+0000 7f170a400700 1 -- 172.21.15.119:0/4010703996 >> 172.21.15.119:0/4010703996 conn(0x7f1704101180 msgr2=0x7f1704101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:35.704 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.995+0000 7f170a400700 1 -- 172.21.15.119:0/4010703996 shutdown_connections 2024-05-27T09:17:35.704 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:34.995+0000 7f170a400700 1 -- 172.21.15.119:0/4010703996 wait complete. 2024-05-27T09:17:35.704 INFO:teuthology.orchestra.run.smithi119.stdout:Generating ssh key... 2024-05-27T09:17:36.308 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:36 smithi119 ceph-mon[29082]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:17:37.446 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.307+0000 7f95da000700 1 Processor -- start 2024-05-27T09:17:37.447 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.307+0000 7f95da000700 1 -- start start 2024-05-27T09:17:37.447 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.307+0000 7f95da000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95d40802e0 0x7f95d410ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:37.447 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.308+0000 7f95da000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d410f210 con 0x7f95d40802e0 2024-05-27T09:17:37.447 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.308+0000 7f95d3400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95d40802e0 0x7f95d410ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:37.447 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.308+0000 7f95d3400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95d40802e0 0x7f95d410ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55206/0 (socket says 172.21.15.119:55206) 2024-05-27T09:17:37.447 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.308+0000 7f95d3400700 1 -- 172.21.15.119:0/2338621214 learned_addr learned my addr 172.21.15.119:0/2338621214 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:37.447 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.308+0000 7f95d3400700 1 -- 172.21.15.119:0/2338621214 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d410f350 con 0x7f95d40802e0 2024-05-27T09:17:37.447 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.309+0000 7f95d3400700 1 --2- 172.21.15.119:0/2338621214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95d40802e0 0x7f95d410ecd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f95c40098a0 tx=0x7f95c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6be4ecb3558b88f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:37.447 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.309+0000 7f95d2000700 1 -- 172.21.15.119:0/2338621214 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f95c4004180 con 0x7f95d40802e0 2024-05-27T09:17:37.447 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.309+0000 7f95d2000700 1 -- 172.21.15.119:0/2338621214 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95c40042e0 con 0x7f95d40802e0 2024-05-27T09:17:37.447 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.309+0000 7f95d2000700 1 -- 172.21.15.119:0/2338621214 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f95c4011a50 con 0x7f95d40802e0 2024-05-27T09:17:37.447 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.310+0000 7f95da000700 1 -- 172.21.15.119:0/2338621214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95d40802e0 msgr2=0x7f95d410ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:37.447 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.310+0000 7f95da000700 1 --2- 172.21.15.119:0/2338621214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95d40802e0 0x7f95d410ecd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f95c40098a0 tx=0x7f95c4009bb0 comp rx=0 tx=0).stop 2024-05-27T09:17:37.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.310+0000 7f95da000700 1 -- 172.21.15.119:0/2338621214 shutdown_connections 2024-05-27T09:17:37.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.310+0000 7f95da000700 1 --2- 172.21.15.119:0/2338621214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95d40802e0 0x7f95d410ecd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:37.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.310+0000 7f95da000700 1 -- 172.21.15.119:0/2338621214 >> 172.21.15.119:0/2338621214 conn(0x7f95d407b220 msgr2=0x7f95d407d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:37.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.310+0000 7f95da000700 1 -- 172.21.15.119:0/2338621214 shutdown_connections 2024-05-27T09:17:37.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.310+0000 7f95da000700 1 -- 172.21.15.119:0/2338621214 wait complete. 2024-05-27T09:17:37.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.311+0000 7f95da000700 1 Processor -- start 2024-05-27T09:17:37.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.311+0000 7f95da000700 1 -- start start 2024-05-27T09:17:37.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.312+0000 7f95da000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95d40802e0 0x7f95d419fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:37.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.312+0000 7f95da000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d41a03a0 con 0x7f95d40802e0 2024-05-27T09:17:37.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.312+0000 7f95d3400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95d40802e0 0x7f95d419fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:37.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.312+0000 7f95d3400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95d40802e0 0x7f95d419fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55210/0 (socket says 172.21.15.119:55210) 2024-05-27T09:17:37.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.312+0000 7f95d3400700 1 -- 172.21.15.119:0/3355507450 learned_addr learned my addr 172.21.15.119:0/3355507450 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:37.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.312+0000 7f95d3400700 1 -- 172.21.15.119:0/3355507450 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c4009550 con 0x7f95d40802e0 2024-05-27T09:17:37.448 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.312+0000 7f95d3400700 1 --2- 172.21.15.119:0/3355507450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95d40802e0 0x7f95d419fe60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f95c4000c00 tx=0x7f95c4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:37.449 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.313+0000 7f95cbe00700 1 -- 172.21.15.119:0/3355507450 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f95c4003b90 con 0x7f95d40802e0 2024-05-27T09:17:37.449 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.313+0000 7f95cbe00700 1 -- 172.21.15.119:0/3355507450 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f95c40248b0 con 0x7f95d40802e0 2024-05-27T09:17:37.449 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.313+0000 7f95da000700 1 -- 172.21.15.119:0/3355507450 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d41a05a0 con 0x7f95d40802e0 2024-05-27T09:17:37.449 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.313+0000 7f95cbe00700 1 -- 172.21.15.119:0/3355507450 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f95c401a8e0 con 0x7f95d40802e0 2024-05-27T09:17:37.449 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.313+0000 7f95da000700 1 -- 172.21.15.119:0/3355507450 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d41a09c0 con 0x7f95d40802e0 2024-05-27T09:17:37.449 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.314+0000 7f95cbe00700 1 -- 172.21.15.119:0/3355507450 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f95c401ab50 con 0x7f95d40802e0 2024-05-27T09:17:37.449 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.314+0000 7f95da000700 1 -- 172.21.15.119:0/3355507450 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b4005320 con 0x7f95d40802e0 2024-05-27T09:17:37.449 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.315+0000 7f95cbe00700 1 --2- 172.21.15.119:0/3355507450 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f95b8038490 0x7f95b803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:37.449 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.315+0000 7f95cbe00700 1 -- 172.21.15.119:0/3355507450 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f95c404c600 con 0x7f95d40802e0 2024-05-27T09:17:37.449 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.315+0000 7f95d2a00700 1 --2- 172.21.15.119:0/3355507450 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f95b8038490 0x7f95b803a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:37.449 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.316+0000 7f95d2a00700 1 --2- 172.21.15.119:0/3355507450 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f95b8038490 0x7f95b803a940 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f95c0006fd0 tx=0x7f95c0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:37.449 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.320+0000 7f95cbe00700 1 -- 172.21.15.119:0/3355507450 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f95c40233a0 con 0x7f95d40802e0 2024-05-27T09:17:37.449 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.533+0000 7f95da000700 1 -- 172.21.15.119:0/3355507450 --> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f95b4000bf0 con 0x7f95b8038490 2024-05-27T09:17:37.449 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.799+0000 7f95cbe00700 1 -- 172.21.15.119:0/3355507450 <== mgr.14120 v2:172.21.15.119:6800/1553098156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f95b4000bf0 con 0x7f95b8038490 2024-05-27T09:17:37.450 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.802+0000 7f95da000700 1 -- 172.21.15.119:0/3355507450 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f95b8038490 msgr2=0x7f95b803a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:37.450 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.802+0000 7f95da000700 1 --2- 172.21.15.119:0/3355507450 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f95b8038490 0x7f95b803a940 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f95c0006fd0 tx=0x7f95c0006e40 comp rx=0 tx=0).stop 2024-05-27T09:17:37.450 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.802+0000 7f95da000700 1 -- 172.21.15.119:0/3355507450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95d40802e0 msgr2=0x7f95d419fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:37.450 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.802+0000 7f95da000700 1 --2- 172.21.15.119:0/3355507450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95d40802e0 0x7f95d419fe60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f95c4000c00 tx=0x7f95c4011ab0 comp rx=0 tx=0).stop 2024-05-27T09:17:37.450 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.803+0000 7f95da000700 1 -- 172.21.15.119:0/3355507450 shutdown_connections 2024-05-27T09:17:37.450 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.803+0000 7f95da000700 1 --2- 172.21.15.119:0/3355507450 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f95b8038490 0x7f95b803a940 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:37.450 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.803+0000 7f95da000700 1 --2- 172.21.15.119:0/3355507450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95d40802e0 0x7f95d419fe60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:37.450 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.803+0000 7f95da000700 1 -- 172.21.15.119:0/3355507450 >> 172.21.15.119:0/3355507450 conn(0x7f95d407b220 msgr2=0x7f95d407c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:37.450 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.803+0000 7f95da000700 1 -- 172.21.15.119:0/3355507450 shutdown_connections 2024-05-27T09:17:37.450 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:36.803+0000 7f95da000700 1 -- 172.21.15.119:0/3355507450 wait complete. 2024-05-27T09:17:38.074 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:37 smithi119 ceph-mon[29082]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:17:38.074 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:37 smithi119 ceph-mon[29082]: Generating ssh key... 2024-05-27T09:17:38.074 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:37 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' 2024-05-27T09:17:38.074 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:37 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' 2024-05-27T09:17:38.939 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:38 smithi119 ceph-mon[29082]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:17:38.948 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDO7MhqggAUNaILv17jDDnNMfX9/7VgW7sAxuzPWPIocS16LhcM7fG2NND/RFGy0NzJCJN8LFpZ1rKc0FkChEUugUu2B+Gv7VkDZBjlHYhZgteTHDZYHUFOdYtjgfbjZi7qFLQDosu/m+kUTIuN8cWw0WIOSX9QqmTyqpGYu3GoahgqaVrHMy7HV4GcTVEsMzU9g0sKlcRcklK4xIp0HFBamSXwM8mitDsVsVjy463crmSNh4TY5usla10D+nF5cc+dQtlSi0pRKjk5xW/o4/wHpF4gTegsUzxPA2YhOUC3E22nXHrjrwl51aq8Vd5wJo4O9FZk1WPjbFjgJzYgjs2LKP5dbonREqrZ1x/iU28ly6Pjdia9RNlksk6XqvYTypc+Vryr/4V6BeDfXsgsRursR+h7cWxbtHLwM87X8/2KRCwNQdb/OSdX6DbDZF7RLK85AhSp6lM1WA5hUUAkTWKORjf2lXuvxz2T/YU5QXST7+5xFgmIiDgb+lg6pPFuf/0= ceph-b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:17:38.954 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.072+0000 7f0fe8000700 1 Processor -- start 2024-05-27T09:17:38.954 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.073+0000 7f0fe8000700 1 -- start start 2024-05-27T09:17:38.954 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.073+0000 7f0fe8000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fe0104460 0x7f0fe0104870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:38.955 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.073+0000 7f0fe8000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe00744b0 con 0x7f0fe0104460 2024-05-27T09:17:38.955 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.073+0000 7f0fe5600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fe0104460 0x7f0fe0104870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:38.955 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.074+0000 7f0fe5600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fe0104460 0x7f0fe0104870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55214/0 (socket says 172.21.15.119:55214) 2024-05-27T09:17:38.955 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.074+0000 7f0fe5600700 1 -- 172.21.15.119:0/1622796731 learned_addr learned my addr 172.21.15.119:0/1622796731 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:38.955 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.074+0000 7f0fe5600700 1 -- 172.21.15.119:0/1622796731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe00745f0 con 0x7f0fe0104460 2024-05-27T09:17:38.955 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.074+0000 7f0fe5600700 1 --2- 172.21.15.119:0/1622796731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fe0104460 0x7f0fe0104870 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4009e80 tx=0x7f0fd400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e59a393049ed875 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:38.955 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.075+0000 7f0fdfe00700 1 -- 172.21.15.119:0/1622796731 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0fd400e070 con 0x7f0fe0104460 2024-05-27T09:17:38.955 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.075+0000 7f0fdfe00700 1 -- 172.21.15.119:0/1622796731 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0fd4003e90 con 0x7f0fe0104460 2024-05-27T09:17:38.955 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.075+0000 7f0fdfe00700 1 -- 172.21.15.119:0/1622796731 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0fd4004380 con 0x7f0fe0104460 2024-05-27T09:17:38.955 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.075+0000 7f0fe8000700 1 -- 172.21.15.119:0/1622796731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fe0104460 msgr2=0x7f0fe0104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:38.955 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.075+0000 7f0fe8000700 1 --2- 172.21.15.119:0/1622796731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fe0104460 0x7f0fe0104870 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4009e80 tx=0x7f0fd400b330 comp rx=0 tx=0).stop 2024-05-27T09:17:38.956 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.075+0000 7f0fe8000700 1 -- 172.21.15.119:0/1622796731 shutdown_connections 2024-05-27T09:17:38.956 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.075+0000 7f0fe8000700 1 --2- 172.21.15.119:0/1622796731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fe0104460 0x7f0fe0104870 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:38.956 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.075+0000 7f0fe8000700 1 -- 172.21.15.119:0/1622796731 >> 172.21.15.119:0/1622796731 conn(0x7f0fe0100070 msgr2=0x7f0fe0102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:38.956 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.076+0000 7f0fe8000700 1 -- 172.21.15.119:0/1622796731 shutdown_connections 2024-05-27T09:17:38.956 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.076+0000 7f0fe8000700 1 -- 172.21.15.119:0/1622796731 wait complete. 2024-05-27T09:17:38.956 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.077+0000 7f0fe8000700 1 Processor -- start 2024-05-27T09:17:38.956 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.077+0000 7f0fe8000700 1 -- start start 2024-05-27T09:17:38.956 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.077+0000 7f0fe8000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fe0104460 0x7f0fe019ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:38.956 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.077+0000 7f0fe8000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe019bf50 con 0x7f0fe0104460 2024-05-27T09:17:38.956 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.077+0000 7f0fe5600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fe0104460 0x7f0fe019ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:38.956 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.077+0000 7f0fe5600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fe0104460 0x7f0fe019ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55220/0 (socket says 172.21.15.119:55220) 2024-05-27T09:17:38.956 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.077+0000 7f0fe5600700 1 -- 172.21.15.119:0/3258775731 learned_addr learned my addr 172.21.15.119:0/3258775731 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:38.957 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.078+0000 7f0fe5600700 1 -- 172.21.15.119:0/3258775731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd4009b30 con 0x7f0fe0104460 2024-05-27T09:17:38.957 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.078+0000 7f0fe5600700 1 --2- 172.21.15.119:0/3258775731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fe0104460 0x7f0fe019ba10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4009530 tx=0x7f0fd4004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:38.957 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.078+0000 7f0fde000700 1 -- 172.21.15.119:0/3258775731 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0fd400e040 con 0x7f0fe0104460 2024-05-27T09:17:38.957 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.078+0000 7f0fde000700 1 -- 172.21.15.119:0/3258775731 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0fd4003c30 con 0x7f0fe0104460 2024-05-27T09:17:38.957 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.078+0000 7f0fde000700 1 -- 172.21.15.119:0/3258775731 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0fd401b740 con 0x7f0fe0104460 2024-05-27T09:17:38.957 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.078+0000 7f0fe8000700 1 -- 172.21.15.119:0/3258775731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe019c150 con 0x7f0fe0104460 2024-05-27T09:17:38.957 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.079+0000 7f0fe8000700 1 -- 172.21.15.119:0/3258775731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe019c5d0 con 0x7f0fe0104460 2024-05-27T09:17:38.957 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.080+0000 7f0fe8000700 1 -- 172.21.15.119:0/3258775731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe00623f0 con 0x7f0fe0104460 2024-05-27T09:17:38.957 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.080+0000 7f0fde000700 1 -- 172.21.15.119:0/3258775731 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f0fd4019040 con 0x7f0fe0104460 2024-05-27T09:17:38.957 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.080+0000 7f0fde000700 1 --2- 172.21.15.119:0/3258775731 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f0fc8038440 0x7f0fc803a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:38.958 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.080+0000 7f0fde000700 1 -- 172.21.15.119:0/3258775731 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0fd404bd10 con 0x7f0fe0104460 2024-05-27T09:17:38.958 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.080+0000 7f0fe4c00700 1 --2- 172.21.15.119:0/3258775731 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f0fc8038440 0x7f0fc803a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:38.958 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.081+0000 7f0fe4c00700 1 --2- 172.21.15.119:0/3258775731 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f0fc8038440 0x7f0fc803a8f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc006fd0 tx=0x7f0fcc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:38.958 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.085+0000 7f0fde000700 1 -- 172.21.15.119:0/3258775731 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0fe00623f0 con 0x7f0fe0104460 2024-05-27T09:17:38.958 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.304+0000 7f0fe8000700 1 -- 172.21.15.119:0/3258775731 --> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fe0062380 con 0x7f0fc8038440 2024-05-27T09:17:38.958 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.306+0000 7f0fde000700 1 -- 172.21.15.119:0/3258775731 <== mgr.14120 v2:172.21.15.119:6800/1553098156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f0fe0062380 con 0x7f0fc8038440 2024-05-27T09:17:38.958 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.309+0000 7f0fe8000700 1 -- 172.21.15.119:0/3258775731 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f0fc8038440 msgr2=0x7f0fc803a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:38.958 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.309+0000 7f0fe8000700 1 --2- 172.21.15.119:0/3258775731 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f0fc8038440 0x7f0fc803a8f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc006fd0 tx=0x7f0fcc006e40 comp rx=0 tx=0).stop 2024-05-27T09:17:38.958 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.309+0000 7f0fe8000700 1 -- 172.21.15.119:0/3258775731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fe0104460 msgr2=0x7f0fe019ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:38.958 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.309+0000 7f0fe8000700 1 --2- 172.21.15.119:0/3258775731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fe0104460 0x7f0fe019ba10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4009530 tx=0x7f0fd4004920 comp rx=0 tx=0).stop 2024-05-27T09:17:38.958 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.309+0000 7f0fe8000700 1 -- 172.21.15.119:0/3258775731 shutdown_connections 2024-05-27T09:17:38.958 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.309+0000 7f0fe8000700 1 --2- 172.21.15.119:0/3258775731 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f0fc8038440 0x7f0fc803a8f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:38.958 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.309+0000 7f0fe8000700 1 --2- 172.21.15.119:0/3258775731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fe0104460 0x7f0fe019ba10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:38.959 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.309+0000 7f0fe8000700 1 -- 172.21.15.119:0/3258775731 >> 172.21.15.119:0/3258775731 conn(0x7f0fe0100070 msgr2=0x7f0fe0190020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:38.959 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.309+0000 7f0fe8000700 1 -- 172.21.15.119:0/3258775731 shutdown_connections 2024-05-27T09:17:38.959 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:38.309+0000 7f0fe8000700 1 -- 172.21.15.119:0/3258775731 wait complete. 2024-05-27T09:17:38.959 INFO:teuthology.orchestra.run.smithi119.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-27T09:17:38.959 INFO:teuthology.orchestra.run.smithi119.stdout:Adding key to root@localhost authorized_keys... 2024-05-27T09:17:38.959 INFO:teuthology.orchestra.run.smithi119.stdout:Adding host smithi119... 2024-05-27T09:17:41.127 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:40 smithi119 ceph-mon[29082]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi119", "addr": "172.21.15.119", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:17:41.995 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:41 smithi119 ceph-mon[29082]: Deploying cephadm binary to smithi119 2024-05-27T09:17:44.797 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout Added host 'smithi119' with addr '172.21.15.119' 2024-05-27T09:17:44.798 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.576+0000 7fcd96000700 1 Processor -- start 2024-05-27T09:17:44.798 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.576+0000 7fcd96000700 1 -- start start 2024-05-27T09:17:44.798 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.576+0000 7fcd96000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9007bdd0 0x7fcd9007a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:44.798 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.576+0000 7fcd96000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9007a810 con 0x7fcd9007bdd0 2024-05-27T09:17:44.798 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.577+0000 7fcd8f400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9007bdd0 0x7fcd9007a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:44.798 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.577+0000 7fcd8f400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9007bdd0 0x7fcd9007a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56608/0 (socket says 172.21.15.119:56608) 2024-05-27T09:17:44.798 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.577+0000 7fcd8f400700 1 -- 172.21.15.119:0/3010935434 learned_addr learned my addr 172.21.15.119:0/3010935434 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:44.798 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.577+0000 7fcd8f400700 1 -- 172.21.15.119:0/3010935434 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9007a950 con 0x7fcd9007bdd0 2024-05-27T09:17:44.798 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.577+0000 7fcd8f400700 1 --2- 172.21.15.119:0/3010935434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9007bdd0 0x7fcd9007a2d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009e80 tx=0x7fcd8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7afd3c75248f79 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:44.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.578+0000 7fcd8ea00700 1 -- 172.21.15.119:0/3010935434 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcd8000e070 con 0x7fcd9007bdd0 2024-05-27T09:17:44.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.578+0000 7fcd8ea00700 1 -- 172.21.15.119:0/3010935434 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcd80003e90 con 0x7fcd9007bdd0 2024-05-27T09:17:44.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.578+0000 7fcd8ea00700 1 -- 172.21.15.119:0/3010935434 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcd80004380 con 0x7fcd9007bdd0 2024-05-27T09:17:44.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.578+0000 7fcd96000700 1 -- 172.21.15.119:0/3010935434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9007bdd0 msgr2=0x7fcd9007a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:44.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.578+0000 7fcd96000700 1 --2- 172.21.15.119:0/3010935434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9007bdd0 0x7fcd9007a2d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009e80 tx=0x7fcd8000b330 comp rx=0 tx=0).stop 2024-05-27T09:17:44.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.578+0000 7fcd96000700 1 -- 172.21.15.119:0/3010935434 shutdown_connections 2024-05-27T09:17:44.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.579+0000 7fcd96000700 1 --2- 172.21.15.119:0/3010935434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9007bdd0 0x7fcd9007a2d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:44.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.579+0000 7fcd96000700 1 -- 172.21.15.119:0/3010935434 >> 172.21.15.119:0/3010935434 conn(0x7fcd90101180 msgr2=0x7fcd90103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:44.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.579+0000 7fcd96000700 1 -- 172.21.15.119:0/3010935434 shutdown_connections 2024-05-27T09:17:44.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.579+0000 7fcd96000700 1 -- 172.21.15.119:0/3010935434 wait complete. 2024-05-27T09:17:44.799 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.580+0000 7fcd96000700 1 Processor -- start 2024-05-27T09:17:44.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.580+0000 7fcd96000700 1 -- start start 2024-05-27T09:17:44.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.580+0000 7fcd96000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9007bdd0 0x7fcd9019ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:44.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.580+0000 7fcd96000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9019bfc0 con 0x7fcd9007bdd0 2024-05-27T09:17:44.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.580+0000 7fcd8f400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9007bdd0 0x7fcd9019ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:44.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.580+0000 7fcd8f400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9007bdd0 0x7fcd9019ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56620/0 (socket says 172.21.15.119:56620) 2024-05-27T09:17:44.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.580+0000 7fcd8f400700 1 -- 172.21.15.119:0/2433219877 learned_addr learned my addr 172.21.15.119:0/2433219877 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:44.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.581+0000 7fcd8f400700 1 -- 172.21.15.119:0/2433219877 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd80009b30 con 0x7fcd9007bdd0 2024-05-27T09:17:44.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.581+0000 7fcd8f400700 1 --2- 172.21.15.119:0/2433219877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9007bdd0 0x7fcd9019ba80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000b7b0 tx=0x7fcd80003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:44.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.581+0000 7fcd8cc00700 1 -- 172.21.15.119:0/2433219877 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcd8000e040 con 0x7fcd9007bdd0 2024-05-27T09:17:44.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.581+0000 7fcd8cc00700 1 -- 172.21.15.119:0/2433219877 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcd800048a0 con 0x7fcd9007bdd0 2024-05-27T09:17:44.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.581+0000 7fcd96000700 1 -- 172.21.15.119:0/2433219877 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9019c1c0 con 0x7fcd9007bdd0 2024-05-27T09:17:44.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.581+0000 7fcd8cc00700 1 -- 172.21.15.119:0/2433219877 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcd8001b630 con 0x7fcd9007bdd0 2024-05-27T09:17:44.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.582+0000 7fcd96000700 1 -- 172.21.15.119:0/2433219877 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9019c640 con 0x7fcd9007bdd0 2024-05-27T09:17:44.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.583+0000 7fcd8cc00700 1 -- 172.21.15.119:0/2433219877 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fcd80019040 con 0x7fcd9007bdd0 2024-05-27T09:17:44.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.583+0000 7fcd96000700 1 -- 172.21.15.119:0/2433219877 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd900623f0 con 0x7fcd9007bdd0 2024-05-27T09:17:44.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.583+0000 7fcd8cc00700 1 --2- 172.21.15.119:0/2433219877 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fcd74038400 0x7fcd7403a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:44.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.583+0000 7fcd8cc00700 1 -- 172.21.15.119:0/2433219877 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcd8004bc70 con 0x7fcd9007bdd0 2024-05-27T09:17:44.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.584+0000 7fcd86a00700 1 --2- 172.21.15.119:0/2433219877 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fcd74038400 0x7fcd7403a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:44.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.584+0000 7fcd86a00700 1 --2- 172.21.15.119:0/2433219877 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fcd74038400 0x7fcd7403a8b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcd78006fd0 tx=0x7fcd78006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:44.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.588+0000 7fcd8cc00700 1 -- 172.21.15.119:0/2433219877 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fcd900623f0 con 0x7fcd9007bdd0 2024-05-27T09:17:44.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:39.806+0000 7fcd96000700 1 -- 172.21.15.119:0/2433219877 --> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi119", "addr": "172.21.15.119", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd90062380 con 0x7fcd74038400 2024-05-27T09:17:44.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:44.041+0000 7fcd8cc00700 1 -- 172.21.15.119:0/2433219877 <== mgr.14120 v2:172.21.15.119:6800/1553098156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fcd90062380 con 0x7fcd74038400 2024-05-27T09:17:44.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:44.045+0000 7fcd96000700 1 -- 172.21.15.119:0/2433219877 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fcd74038400 msgr2=0x7fcd7403a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:44.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:44.045+0000 7fcd96000700 1 --2- 172.21.15.119:0/2433219877 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fcd74038400 0x7fcd7403a8b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcd78006fd0 tx=0x7fcd78006e40 comp rx=0 tx=0).stop 2024-05-27T09:17:44.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:44.045+0000 7fcd96000700 1 -- 172.21.15.119:0/2433219877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9007bdd0 msgr2=0x7fcd9019ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:44.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:44.045+0000 7fcd96000700 1 --2- 172.21.15.119:0/2433219877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9007bdd0 0x7fcd9019ba80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000b7b0 tx=0x7fcd80003d00 comp rx=0 tx=0).stop 2024-05-27T09:17:44.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:44.045+0000 7fcd96000700 1 -- 172.21.15.119:0/2433219877 shutdown_connections 2024-05-27T09:17:44.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:44.045+0000 7fcd96000700 1 --2- 172.21.15.119:0/2433219877 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fcd74038400 0x7fcd7403a8b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:44.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:44.045+0000 7fcd96000700 1 --2- 172.21.15.119:0/2433219877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9007bdd0 0x7fcd9019ba80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:44.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:44.045+0000 7fcd96000700 1 -- 172.21.15.119:0/2433219877 >> 172.21.15.119:0/2433219877 conn(0x7fcd90101180 msgr2=0x7fcd90101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:44.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:44.046+0000 7fcd96000700 1 -- 172.21.15.119:0/2433219877 shutdown_connections 2024-05-27T09:17:44.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:44.046+0000 7fcd96000700 1 -- 172.21.15.119:0/2433219877 wait complete. 2024-05-27T09:17:44.802 INFO:teuthology.orchestra.run.smithi119.stdout:Deploying mon service with default placement... 2024-05-27T09:17:45.375 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:45 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' 2024-05-27T09:17:45.375 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:45 smithi119 ceph-mon[29082]: Added host smithi119 2024-05-27T09:17:45.375 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:45 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:17:46.592 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-27T09:17:46.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.620+0000 7ff1cea00700 1 Processor -- start 2024-05-27T09:17:46.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.620+0000 7ff1cea00700 1 -- start start 2024-05-27T09:17:46.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.620+0000 7ff1cea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1c8104400 0x7ff1c8104810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:46.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.620+0000 7ff1cea00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c8074510 con 0x7ff1c8104400 2024-05-27T09:17:46.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.621+0000 7ff1c7e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1c8104400 0x7ff1c8104810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:46.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.621+0000 7ff1c7e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1c8104400 0x7ff1c8104810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56626/0 (socket says 172.21.15.119:56626) 2024-05-27T09:17:46.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.621+0000 7ff1c7e00700 1 -- 172.21.15.119:0/3399743376 learned_addr learned my addr 172.21.15.119:0/3399743376 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:46.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.621+0000 7ff1c7e00700 1 -- 172.21.15.119:0/3399743376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c8074650 con 0x7ff1c8104400 2024-05-27T09:17:46.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.621+0000 7ff1c7e00700 1 --2- 172.21.15.119:0/3399743376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1c8104400 0x7ff1c8104810 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80098a0 tx=0x7ff1b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0b024c275f5340 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:46.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.622+0000 7ff1c6a00700 1 -- 172.21.15.119:0/3399743376 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1b8004180 con 0x7ff1c8104400 2024-05-27T09:17:46.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.622+0000 7ff1c6a00700 1 -- 172.21.15.119:0/3399743376 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff1b80042e0 con 0x7ff1c8104400 2024-05-27T09:17:46.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.622+0000 7ff1c6a00700 1 -- 172.21.15.119:0/3399743376 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1b8011a50 con 0x7ff1c8104400 2024-05-27T09:17:46.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.622+0000 7ff1cea00700 1 -- 172.21.15.119:0/3399743376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1c8104400 msgr2=0x7ff1c8104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:46.594 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.622+0000 7ff1cea00700 1 --2- 172.21.15.119:0/3399743376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1c8104400 0x7ff1c8104810 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80098a0 tx=0x7ff1b8009bb0 comp rx=0 tx=0).stop 2024-05-27T09:17:46.594 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.622+0000 7ff1cea00700 1 -- 172.21.15.119:0/3399743376 shutdown_connections 2024-05-27T09:17:46.594 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.622+0000 7ff1cea00700 1 --2- 172.21.15.119:0/3399743376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1c8104400 0x7ff1c8104810 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:46.594 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.622+0000 7ff1cea00700 1 -- 172.21.15.119:0/3399743376 >> 172.21.15.119:0/3399743376 conn(0x7ff1c8100070 msgr2=0x7ff1c8102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:46.594 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.623+0000 7ff1cea00700 1 -- 172.21.15.119:0/3399743376 shutdown_connections 2024-05-27T09:17:46.594 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.623+0000 7ff1cea00700 1 -- 172.21.15.119:0/3399743376 wait complete. 2024-05-27T09:17:46.594 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.624+0000 7ff1cea00700 1 Processor -- start 2024-05-27T09:17:46.594 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.624+0000 7ff1cea00700 1 -- start start 2024-05-27T09:17:46.594 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.624+0000 7ff1cea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1c8104400 0x7ff1c819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:46.594 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.624+0000 7ff1cea00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c819bf60 con 0x7ff1c8104400 2024-05-27T09:17:46.594 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.624+0000 7ff1c7e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1c8104400 0x7ff1c819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:46.594 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.624+0000 7ff1c7e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1c8104400 0x7ff1c819ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56634/0 (socket says 172.21.15.119:56634) 2024-05-27T09:17:46.595 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.624+0000 7ff1c7e00700 1 -- 172.21.15.119:0/2740848875 learned_addr learned my addr 172.21.15.119:0/2740848875 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:46.595 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.625+0000 7ff1c7e00700 1 -- 172.21.15.119:0/2740848875 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b8009550 con 0x7ff1c8104400 2024-05-27T09:17:46.595 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.625+0000 7ff1c7e00700 1 --2- 172.21.15.119:0/2740848875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1c8104400 0x7ff1c819ba20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8000c00 tx=0x7ff1b8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:46.595 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.625+0000 7ff1c4c00700 1 -- 172.21.15.119:0/2740848875 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1b8003b90 con 0x7ff1c8104400 2024-05-27T09:17:46.595 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.625+0000 7ff1c4c00700 1 -- 172.21.15.119:0/2740848875 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff1b80248b0 con 0x7ff1c8104400 2024-05-27T09:17:46.595 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.625+0000 7ff1cea00700 1 -- 172.21.15.119:0/2740848875 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c819c160 con 0x7ff1c8104400 2024-05-27T09:17:46.595 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.626+0000 7ff1c4c00700 1 -- 172.21.15.119:0/2740848875 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1b801a8e0 con 0x7ff1c8104400 2024-05-27T09:17:46.595 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.626+0000 7ff1cea00700 1 -- 172.21.15.119:0/2740848875 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c819c580 con 0x7ff1c8104400 2024-05-27T09:17:46.595 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.627+0000 7ff1c4c00700 1 -- 172.21.15.119:0/2740848875 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7ff1b801ab50 con 0x7ff1c8104400 2024-05-27T09:17:46.595 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.627+0000 7ff1cea00700 1 -- 172.21.15.119:0/2740848875 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c8195260 con 0x7ff1c8104400 2024-05-27T09:17:46.595 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.627+0000 7ff1c4c00700 1 --2- 172.21.15.119:0/2740848875 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7ff1ac038440 0x7ff1ac03a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:46.595 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.628+0000 7ff1c4c00700 1 -- 172.21.15.119:0/2740848875 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff1b804c3b0 con 0x7ff1c8104400 2024-05-27T09:17:46.595 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.628+0000 7ff1c7400700 1 --2- 172.21.15.119:0/2740848875 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7ff1ac038440 0x7ff1ac03a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:46.595 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.629+0000 7ff1c7400700 1 --2- 172.21.15.119:0/2740848875 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7ff1ac038440 0x7ff1ac03a8f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4006fd0 tx=0x7ff1b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:46.596 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.633+0000 7ff1c4c00700 1 -- 172.21.15.119:0/2740848875 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff1b8018530 con 0x7ff1c8104400 2024-05-27T09:17:46.596 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.863+0000 7ff1cea00700 1 -- 172.21.15.119:0/2740848875 --> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1c8061190 con 0x7ff1ac038440 2024-05-27T09:17:46.596 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.871+0000 7ff1c4c00700 1 -- 172.21.15.119:0/2740848875 <== mgr.14120 v2:172.21.15.119:6800/1553098156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff1c8061190 con 0x7ff1ac038440 2024-05-27T09:17:46.596 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.874+0000 7ff1cea00700 1 -- 172.21.15.119:0/2740848875 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7ff1ac038440 msgr2=0x7ff1ac03a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:46.596 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.874+0000 7ff1cea00700 1 --2- 172.21.15.119:0/2740848875 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7ff1ac038440 0x7ff1ac03a8f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4006fd0 tx=0x7ff1b4006e40 comp rx=0 tx=0).stop 2024-05-27T09:17:46.596 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.874+0000 7ff1cea00700 1 -- 172.21.15.119:0/2740848875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1c8104400 msgr2=0x7ff1c819ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:46.596 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.874+0000 7ff1cea00700 1 --2- 172.21.15.119:0/2740848875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1c8104400 0x7ff1c819ba20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8000c00 tx=0x7ff1b8011ab0 comp rx=0 tx=0).stop 2024-05-27T09:17:46.596 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.874+0000 7ff1cea00700 1 -- 172.21.15.119:0/2740848875 shutdown_connections 2024-05-27T09:17:46.596 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.874+0000 7ff1cea00700 1 --2- 172.21.15.119:0/2740848875 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7ff1ac038440 0x7ff1ac03a8f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:46.596 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.874+0000 7ff1cea00700 1 --2- 172.21.15.119:0/2740848875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1c8104400 0x7ff1c819ba20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:46.596 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.874+0000 7ff1cea00700 1 -- 172.21.15.119:0/2740848875 >> 172.21.15.119:0/2740848875 conn(0x7ff1c8100070 msgr2=0x7ff1c818ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:46.596 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.875+0000 7ff1cea00700 1 -- 172.21.15.119:0/2740848875 shutdown_connections 2024-05-27T09:17:46.596 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:45.875+0000 7ff1cea00700 1 -- 172.21.15.119:0/2740848875 wait complete. 2024-05-27T09:17:46.597 INFO:teuthology.orchestra.run.smithi119.stdout:Deploying mgr service with default placement... 2024-05-27T09:17:47.120 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:46 smithi119 ceph-mon[29082]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:17:47.120 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:46 smithi119 ceph-mon[29082]: Saving service mon spec with placement count:5 2024-05-27T09:17:47.120 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:46 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' 2024-05-27T09:17:48.221 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:47 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' 2024-05-27T09:17:48.221 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:47 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' 2024-05-27T09:17:48.221 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:47 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' 2024-05-27T09:17:48.600 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-27T09:17:48.600 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.567+0000 7fbea2e00700 1 Processor -- start 2024-05-27T09:17:48.600 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.567+0000 7fbea2e00700 1 -- start start 2024-05-27T09:17:48.600 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.567+0000 7fbea2e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe9c105b10 0x7fbe9c105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:48.600 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.567+0000 7fbea2e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c106460 con 0x7fbe9c105b10 2024-05-27T09:17:48.600 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.568+0000 7fbe9be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe9c105b10 0x7fbe9c105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:48.600 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.568+0000 7fbe9be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe9c105b10 0x7fbe9c105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56644/0 (socket says 172.21.15.119:56644) 2024-05-27T09:17:48.600 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.568+0000 7fbe9be00700 1 -- 172.21.15.119:0/3814237727 learned_addr learned my addr 172.21.15.119:0/3814237727 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:48.600 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.569+0000 7fbe9be00700 1 -- 172.21.15.119:0/3814237727 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9c1065a0 con 0x7fbe9c105b10 2024-05-27T09:17:48.601 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.569+0000 7fbe9be00700 1 --2- 172.21.15.119:0/3814237727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe9c105b10 0x7fbe9c105f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009e80 tx=0x7fbe8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9826dc3dde1d222 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:48.601 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.569+0000 7fbe9b400700 1 -- 172.21.15.119:0/3814237727 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe8c00e070 con 0x7fbe9c105b10 2024-05-27T09:17:48.601 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.569+0000 7fbe9b400700 1 -- 172.21.15.119:0/3814237727 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbe8c00b8c0 con 0x7fbe9c105b10 2024-05-27T09:17:48.601 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.569+0000 7fbe9b400700 1 -- 172.21.15.119:0/3814237727 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe8c0044d0 con 0x7fbe9c105b10 2024-05-27T09:17:48.601 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.570+0000 7fbea2e00700 1 -- 172.21.15.119:0/3814237727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe9c105b10 msgr2=0x7fbe9c105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:48.601 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.570+0000 7fbea2e00700 1 --2- 172.21.15.119:0/3814237727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe9c105b10 0x7fbe9c105f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009e80 tx=0x7fbe8c00b330 comp rx=0 tx=0).stop 2024-05-27T09:17:48.601 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.570+0000 7fbea2e00700 1 -- 172.21.15.119:0/3814237727 shutdown_connections 2024-05-27T09:17:48.601 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.570+0000 7fbea2e00700 1 --2- 172.21.15.119:0/3814237727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe9c105b10 0x7fbe9c105f20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:48.601 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.570+0000 7fbea2e00700 1 -- 172.21.15.119:0/3814237727 >> 172.21.15.119:0/3814237727 conn(0x7fbe9c101160 msgr2=0x7fbe9c103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:48.601 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.571+0000 7fbea2e00700 1 -- 172.21.15.119:0/3814237727 shutdown_connections 2024-05-27T09:17:48.602 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.571+0000 7fbea2e00700 1 -- 172.21.15.119:0/3814237727 wait complete. 2024-05-27T09:17:48.602 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.571+0000 7fbea2e00700 1 Processor -- start 2024-05-27T09:17:48.602 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.572+0000 7fbea2e00700 1 -- start start 2024-05-27T09:17:48.602 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.572+0000 7fbea2e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe9c105b10 0x7fbe9c07ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:48.602 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.572+0000 7fbea2e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c07b070 con 0x7fbe9c105b10 2024-05-27T09:17:48.602 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.572+0000 7fbe9be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe9c105b10 0x7fbe9c07ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:48.602 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.573+0000 7fbe9be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe9c105b10 0x7fbe9c07ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56652/0 (socket says 172.21.15.119:56652) 2024-05-27T09:17:48.602 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.573+0000 7fbe9be00700 1 -- 172.21.15.119:0/653278180 learned_addr learned my addr 172.21.15.119:0/653278180 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:48.602 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.573+0000 7fbe9be00700 1 -- 172.21.15.119:0/653278180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8c009b30 con 0x7fbe9c105b10 2024-05-27T09:17:48.602 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.573+0000 7fbe9be00700 1 --2- 172.21.15.119:0/653278180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe9c105b10 0x7fbe9c07ab30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0072a0 tx=0x7fbe8c0049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:48.602 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.574+0000 7fbe99600700 1 -- 172.21.15.119:0/653278180 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe8c00e040 con 0x7fbe9c105b10 2024-05-27T09:17:48.603 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.574+0000 7fbea2e00700 1 -- 172.21.15.119:0/653278180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9c079180 con 0x7fbe9c105b10 2024-05-27T09:17:48.603 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.574+0000 7fbe99600700 1 -- 172.21.15.119:0/653278180 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbe8c00bd80 con 0x7fbe9c105b10 2024-05-27T09:17:48.603 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.574+0000 7fbe99600700 1 -- 172.21.15.119:0/653278180 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe8c0126c0 con 0x7fbe9c105b10 2024-05-27T09:17:48.603 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.574+0000 7fbea2e00700 1 -- 172.21.15.119:0/653278180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9c079600 con 0x7fbe9c105b10 2024-05-27T09:17:48.603 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.575+0000 7fbe99600700 1 -- 172.21.15.119:0/653278180 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fbe8c019040 con 0x7fbe9c105b10 2024-05-27T09:17:48.603 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.575+0000 7fbea2e00700 1 -- 172.21.15.119:0/653278180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe9c062380 con 0x7fbe9c105b10 2024-05-27T09:17:48.603 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.575+0000 7fbe99600700 1 --2- 172.21.15.119:0/653278180 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fbe80038400 0x7fbe8003a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:48.603 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.576+0000 7fbe99600700 1 -- 172.21.15.119:0/653278180 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbe8c04bf80 con 0x7fbe9c105b10 2024-05-27T09:17:48.603 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.576+0000 7fbe93e00700 1 --2- 172.21.15.119:0/653278180 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fbe80038400 0x7fbe8003a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:48.603 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.577+0000 7fbe93e00700 1 --2- 172.21.15.119:0/653278180 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fbe80038400 0x7fbe8003a8b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbe84006fd0 tx=0x7fbe84006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:48.603 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.581+0000 7fbe99600700 1 -- 172.21.15.119:0/653278180 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbe8c011750 con 0x7fbe9c105b10 2024-05-27T09:17:48.603 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.824+0000 7fbea2e00700 1 -- 172.21.15.119:0/653278180 --> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe9c102600 con 0x7fbe80038400 2024-05-27T09:17:48.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.829+0000 7fbe99600700 1 -- 172.21.15.119:0/653278180 <== mgr.14120 v2:172.21.15.119:6800/1553098156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fbe9c102600 con 0x7fbe80038400 2024-05-27T09:17:48.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.833+0000 7fbea2e00700 1 -- 172.21.15.119:0/653278180 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fbe80038400 msgr2=0x7fbe8003a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:48.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.833+0000 7fbea2e00700 1 --2- 172.21.15.119:0/653278180 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fbe80038400 0x7fbe8003a8b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbe84006fd0 tx=0x7fbe84006e40 comp rx=0 tx=0).stop 2024-05-27T09:17:48.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.833+0000 7fbea2e00700 1 -- 172.21.15.119:0/653278180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe9c105b10 msgr2=0x7fbe9c07ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:48.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.833+0000 7fbea2e00700 1 --2- 172.21.15.119:0/653278180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe9c105b10 0x7fbe9c07ab30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0072a0 tx=0x7fbe8c0049f0 comp rx=0 tx=0).stop 2024-05-27T09:17:48.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.833+0000 7fbea2e00700 1 -- 172.21.15.119:0/653278180 shutdown_connections 2024-05-27T09:17:48.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.833+0000 7fbea2e00700 1 --2- 172.21.15.119:0/653278180 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fbe80038400 0x7fbe8003a8b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:48.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.833+0000 7fbea2e00700 1 --2- 172.21.15.119:0/653278180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe9c105b10 0x7fbe9c07ab30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:48.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.833+0000 7fbea2e00700 1 -- 172.21.15.119:0/653278180 >> 172.21.15.119:0/653278180 conn(0x7fbe9c101160 msgr2=0x7fbe9c101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:48.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.833+0000 7fbea2e00700 1 -- 172.21.15.119:0/653278180 shutdown_connections 2024-05-27T09:17:48.604 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:47.833+0000 7fbea2e00700 1 -- 172.21.15.119:0/653278180 wait complete. 2024-05-27T09:17:48.604 INFO:teuthology.orchestra.run.smithi119.stdout:Deploying crash service with default placement... 2024-05-27T09:17:49.177 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:48 smithi119 ceph-mon[29082]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:17:49.177 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:48 smithi119 ceph-mon[29082]: Saving service mgr spec with placement count:2 2024-05-27T09:17:50.548 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-05-27T09:17:50.548 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.176+0000 7f1cefe00700 1 Processor -- start 2024-05-27T09:17:50.548 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.176+0000 7f1cefe00700 1 -- start start 2024-05-27T09:17:50.548 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.177+0000 7f1cefe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ce8079c90 0x7f1ce8108730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:50.548 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.177+0000 7f1cefe00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce8108c70 con 0x7f1ce8079c90 2024-05-27T09:17:50.548 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.177+0000 7f1ced400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ce8079c90 0x7f1ce8108730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:50.548 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.177+0000 7f1ced400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ce8079c90 0x7f1ce8108730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40016/0 (socket says 172.21.15.119:40016) 2024-05-27T09:17:50.548 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.177+0000 7f1ced400700 1 -- 172.21.15.119:0/2062635867 learned_addr learned my addr 172.21.15.119:0/2062635867 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:50.553 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.177+0000 7f1ced400700 1 -- 172.21.15.119:0/2062635867 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce8108db0 con 0x7f1ce8079c90 2024-05-27T09:17:50.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.177+0000 7f1ced400700 1 --2- 172.21.15.119:0/2062635867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ce8079c90 0x7f1ce8108730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc0098a0 tx=0x7f1cdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b960a5d00da787 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:50.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.177+0000 7f1ce7e00700 1 -- 172.21.15.119:0/2062635867 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1cdc004180 con 0x7f1ce8079c90 2024-05-27T09:17:50.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.177+0000 7f1ce7e00700 1 -- 172.21.15.119:0/2062635867 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1cdc0042e0 con 0x7f1ce8079c90 2024-05-27T09:17:50.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.177+0000 7f1ce7e00700 1 -- 172.21.15.119:0/2062635867 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1cdc011af0 con 0x7f1ce8079c90 2024-05-27T09:17:50.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.178+0000 7f1cefe00700 1 -- 172.21.15.119:0/2062635867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ce8079c90 msgr2=0x7f1ce8108730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:50.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.178+0000 7f1cefe00700 1 --2- 172.21.15.119:0/2062635867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ce8079c90 0x7f1ce8108730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc0098a0 tx=0x7f1cdc009bb0 comp rx=0 tx=0).stop 2024-05-27T09:17:50.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.178+0000 7f1cefe00700 1 -- 172.21.15.119:0/2062635867 shutdown_connections 2024-05-27T09:17:50.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.178+0000 7f1cefe00700 1 --2- 172.21.15.119:0/2062635867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ce8079c90 0x7f1ce8108730 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:50.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.178+0000 7f1cefe00700 1 -- 172.21.15.119:0/2062635867 >> 172.21.15.119:0/2062635867 conn(0x7f1ce8100060 msgr2=0x7f1ce81024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:50.554 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.180+0000 7f1cefe00700 1 -- 172.21.15.119:0/2062635867 shutdown_connections 2024-05-27T09:17:50.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.180+0000 7f1cefe00700 1 -- 172.21.15.119:0/2062635867 wait complete. 2024-05-27T09:17:50.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.180+0000 7f1cefe00700 1 Processor -- start 2024-05-27T09:17:50.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.180+0000 7f1cefe00700 1 -- start start 2024-05-27T09:17:50.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.180+0000 7f1cefe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ce8079c90 0x7f1ce819bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:50.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.180+0000 7f1cefe00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce819c080 con 0x7f1ce8079c90 2024-05-27T09:17:50.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.180+0000 7f1ced400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ce8079c90 0x7f1ce819bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:50.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.180+0000 7f1ced400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ce8079c90 0x7f1ce819bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40024/0 (socket says 172.21.15.119:40024) 2024-05-27T09:17:50.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.180+0000 7f1ced400700 1 -- 172.21.15.119:0/1119884533 learned_addr learned my addr 172.21.15.119:0/1119884533 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:50.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.181+0000 7f1ced400700 1 -- 172.21.15.119:0/1119884533 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc009550 con 0x7f1ce8079c90 2024-05-27T09:17:50.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.181+0000 7f1ced400700 1 --2- 172.21.15.119:0/1119884533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ce8079c90 0x7f1ce819bb40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc011670 tx=0x7f1cdc011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:50.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.181+0000 7f1ce6000700 1 -- 172.21.15.119:0/1119884533 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1cdc0039f0 con 0x7f1ce8079c90 2024-05-27T09:17:50.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.181+0000 7f1ce6000700 1 -- 172.21.15.119:0/1119884533 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1cdc003b50 con 0x7f1ce8079c90 2024-05-27T09:17:50.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.181+0000 7f1ce6000700 1 -- 172.21.15.119:0/1119884533 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1cdc0247c0 con 0x7f1ce8079c90 2024-05-27T09:17:50.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.182+0000 7f1cefe00700 1 -- 172.21.15.119:0/1119884533 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce819c280 con 0x7f1ce8079c90 2024-05-27T09:17:50.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.182+0000 7f1cefe00700 1 -- 172.21.15.119:0/1119884533 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce819c6a0 con 0x7f1ce8079c90 2024-05-27T09:17:50.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.183+0000 7f1ce6000700 1 -- 172.21.15.119:0/1119884533 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f1cdc01ac00 con 0x7f1ce8079c90 2024-05-27T09:17:50.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.183+0000 7f1cefe00700 1 -- 172.21.15.119:0/1119884533 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce8109c00 con 0x7f1ce8079c90 2024-05-27T09:17:50.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.183+0000 7f1ce6000700 1 --2- 172.21.15.119:0/1119884533 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f1cd0038440 0x7f1cd003a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:50.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.183+0000 7f1ce6000700 1 -- 172.21.15.119:0/1119884533 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1cdc04c380 con 0x7f1ce8079c90 2024-05-27T09:17:50.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.184+0000 7f1ceca00700 1 --2- 172.21.15.119:0/1119884533 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f1cd0038440 0x7f1cd003a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:50.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.185+0000 7f1ceca00700 1 --2- 172.21.15.119:0/1119884533 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f1cd0038440 0x7f1cd003a8f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4006fd0 tx=0x7f1cd4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:50.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.190+0000 7f1ce6000700 1 -- 172.21.15.119:0/1119884533 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1cdc010380 con 0x7f1ce8079c90 2024-05-27T09:17:50.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.440+0000 7f1cefe00700 1 -- 172.21.15.119:0/1119884533 --> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ce819f500 con 0x7f1cd0038440 2024-05-27T09:17:50.557 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.449+0000 7f1ce6000700 1 -- 172.21.15.119:0/1119884533 <== mgr.14120 v2:172.21.15.119:6800/1553098156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f1ce819f500 con 0x7f1cd0038440 2024-05-27T09:17:50.557 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.451+0000 7f1cefe00700 1 -- 172.21.15.119:0/1119884533 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f1cd0038440 msgr2=0x7f1cd003a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:50.557 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.452+0000 7f1cefe00700 1 --2- 172.21.15.119:0/1119884533 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f1cd0038440 0x7f1cd003a8f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4006fd0 tx=0x7f1cd4006e40 comp rx=0 tx=0).stop 2024-05-27T09:17:50.557 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.452+0000 7f1cefe00700 1 -- 172.21.15.119:0/1119884533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ce8079c90 msgr2=0x7f1ce819bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:50.557 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.452+0000 7f1cefe00700 1 --2- 172.21.15.119:0/1119884533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ce8079c90 0x7f1ce819bb40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc011670 tx=0x7f1cdc011b50 comp rx=0 tx=0).stop 2024-05-27T09:17:50.557 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.452+0000 7f1cefe00700 1 -- 172.21.15.119:0/1119884533 shutdown_connections 2024-05-27T09:17:50.557 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.452+0000 7f1cefe00700 1 --2- 172.21.15.119:0/1119884533 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f1cd0038440 0x7f1cd003a8f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:50.557 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.452+0000 7f1cefe00700 1 --2- 172.21.15.119:0/1119884533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ce8079c90 0x7f1ce819bb40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:50.557 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.452+0000 7f1cefe00700 1 -- 172.21.15.119:0/1119884533 >> 172.21.15.119:0/1119884533 conn(0x7f1ce8100060 msgr2=0x7f1ce8101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:50.557 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.452+0000 7f1cefe00700 1 -- 172.21.15.119:0/1119884533 shutdown_connections 2024-05-27T09:17:50.557 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:49.452+0000 7f1cefe00700 1 -- 172.21.15.119:0/1119884533 wait complete. 2024-05-27T09:17:50.557 INFO:teuthology.orchestra.run.smithi119.stdout:Deploying ceph-exporter service with default placement... 2024-05-27T09:17:50.711 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:50 smithi119 ceph-mon[29082]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:17:50.711 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:50 smithi119 ceph-mon[29082]: Saving service crash spec with placement * 2024-05-27T09:17:50.711 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:50 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' 2024-05-27T09:17:51.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:51 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' 2024-05-27T09:17:51.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:51 smithi119 ceph-mon[29082]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:17:51.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:51 smithi119 ceph-mon[29082]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:17:51.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:51 smithi119 ceph-mon[29082]: Saving service ceph-exporter spec with placement * 2024-05-27T09:17:51.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:51 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' 2024-05-27T09:17:52.063 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-05-27T09:17:52.063 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.128+0000 7fe460c00700 1 Processor -- start 2024-05-27T09:17:52.063 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.128+0000 7fe460c00700 1 -- start start 2024-05-27T09:17:52.064 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.129+0000 7fe460c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45c07acc0 0x7fe45c0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:52.064 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.129+0000 7fe460c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c079700 con 0x7fe45c07acc0 2024-05-27T09:17:52.064 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.129+0000 7fe45b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45c07acc0 0x7fe45c0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:52.064 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.129+0000 7fe45b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45c07acc0 0x7fe45c0791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40030/0 (socket says 172.21.15.119:40030) 2024-05-27T09:17:52.064 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.129+0000 7fe45b400700 1 -- 172.21.15.119:0/1459401009 learned_addr learned my addr 172.21.15.119:0/1459401009 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:52.064 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.130+0000 7fe45b400700 1 -- 172.21.15.119:0/1459401009 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45c079840 con 0x7fe45c07acc0 2024-05-27T09:17:52.064 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.130+0000 7fe45b400700 1 --2- 172.21.15.119:0/1459401009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45c07acc0 0x7fe45c0791c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe44c0098a0 tx=0x7fe44c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85d70aa1bad6009 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:52.064 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.130+0000 7fe45aa00700 1 -- 172.21.15.119:0/1459401009 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe44c004180 con 0x7fe45c07acc0 2024-05-27T09:17:52.064 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.130+0000 7fe45aa00700 1 -- 172.21.15.119:0/1459401009 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe44c0042e0 con 0x7fe45c07acc0 2024-05-27T09:17:52.064 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.130+0000 7fe45aa00700 1 -- 172.21.15.119:0/1459401009 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe44c011af0 con 0x7fe45c07acc0 2024-05-27T09:17:52.064 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.131+0000 7fe460c00700 1 -- 172.21.15.119:0/1459401009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45c07acc0 msgr2=0x7fe45c0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:52.064 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.131+0000 7fe460c00700 1 --2- 172.21.15.119:0/1459401009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45c07acc0 0x7fe45c0791c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe44c0098a0 tx=0x7fe44c009bb0 comp rx=0 tx=0).stop 2024-05-27T09:17:52.064 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.131+0000 7fe460c00700 1 -- 172.21.15.119:0/1459401009 shutdown_connections 2024-05-27T09:17:52.065 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.131+0000 7fe460c00700 1 --2- 172.21.15.119:0/1459401009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45c07acc0 0x7fe45c0791c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:52.065 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.131+0000 7fe460c00700 1 -- 172.21.15.119:0/1459401009 >> 172.21.15.119:0/1459401009 conn(0x7fe45c100fb0 msgr2=0x7fe45c1033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:52.065 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.132+0000 7fe460c00700 1 -- 172.21.15.119:0/1459401009 shutdown_connections 2024-05-27T09:17:52.065 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.132+0000 7fe460c00700 1 -- 172.21.15.119:0/1459401009 wait complete. 2024-05-27T09:17:52.065 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.133+0000 7fe460c00700 1 Processor -- start 2024-05-27T09:17:52.065 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.133+0000 7fe460c00700 1 -- start start 2024-05-27T09:17:52.065 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.133+0000 7fe460c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45c07acc0 0x7fe45c19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:52.065 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.133+0000 7fe460c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c19be00 con 0x7fe45c07acc0 2024-05-27T09:17:52.065 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.133+0000 7fe45b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45c07acc0 0x7fe45c19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:52.065 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.134+0000 7fe45b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45c07acc0 0x7fe45c19b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40038/0 (socket says 172.21.15.119:40038) 2024-05-27T09:17:52.065 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.134+0000 7fe45b400700 1 -- 172.21.15.119:0/90465981 learned_addr learned my addr 172.21.15.119:0/90465981 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:52.065 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.134+0000 7fe45b400700 1 -- 172.21.15.119:0/90465981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44c009550 con 0x7fe45c07acc0 2024-05-27T09:17:52.066 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.134+0000 7fe45b400700 1 --2- 172.21.15.119:0/90465981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45c07acc0 0x7fe45c19b8c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe44c000c00 tx=0x7fe44c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:52.066 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.135+0000 7fe458c00700 1 -- 172.21.15.119:0/90465981 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe44c003a70 con 0x7fe45c07acc0 2024-05-27T09:17:52.066 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.135+0000 7fe458c00700 1 -- 172.21.15.119:0/90465981 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe44c003bd0 con 0x7fe45c07acc0 2024-05-27T09:17:52.066 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.135+0000 7fe458c00700 1 -- 172.21.15.119:0/90465981 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe44c0238c0 con 0x7fe45c07acc0 2024-05-27T09:17:52.066 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.135+0000 7fe460c00700 1 -- 172.21.15.119:0/90465981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45c19c000 con 0x7fe45c07acc0 2024-05-27T09:17:52.066 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.135+0000 7fe460c00700 1 -- 172.21.15.119:0/90465981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe45c19c420 con 0x7fe45c07acc0 2024-05-27T09:17:52.066 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.136+0000 7fe458c00700 1 -- 172.21.15.119:0/90465981 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fe44c023b80 con 0x7fe45c07acc0 2024-05-27T09:17:52.066 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.136+0000 7fe460c00700 1 -- 172.21.15.119:0/90465981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe45c062380 con 0x7fe45c07acc0 2024-05-27T09:17:52.066 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.136+0000 7fe458c00700 1 --2- 172.21.15.119:0/90465981 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fe4400383f0 0x7fe44003a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:52.066 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.136+0000 7fe458c00700 1 -- 172.21.15.119:0/90465981 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe44c04c290 con 0x7fe45c07acc0 2024-05-27T09:17:52.066 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.137+0000 7fe453e00700 1 --2- 172.21.15.119:0/90465981 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fe4400383f0 0x7fe44003a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:52.066 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.137+0000 7fe453e00700 1 --2- 172.21.15.119:0/90465981 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fe4400383f0 0x7fe44003a8a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe444006fd0 tx=0x7fe444006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:52.067 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.142+0000 7fe458c00700 1 -- 172.21.15.119:0/90465981 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe44c023e30 con 0x7fe45c07acc0 2024-05-27T09:17:52.067 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.376+0000 7fe460c00700 1 -- 172.21.15.119:0/90465981 --> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fe45c1032e0 con 0x7fe4400383f0 2024-05-27T09:17:52.067 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.387+0000 7fe458c00700 1 -- 172.21.15.119:0/90465981 <== mgr.14120 v2:172.21.15.119:6800/1553098156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fe45c1032e0 con 0x7fe4400383f0 2024-05-27T09:17:52.067 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.390+0000 7fe460c00700 1 -- 172.21.15.119:0/90465981 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fe4400383f0 msgr2=0x7fe44003a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:52.067 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.390+0000 7fe460c00700 1 --2- 172.21.15.119:0/90465981 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fe4400383f0 0x7fe44003a8a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe444006fd0 tx=0x7fe444006e40 comp rx=0 tx=0).stop 2024-05-27T09:17:52.067 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.390+0000 7fe460c00700 1 -- 172.21.15.119:0/90465981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45c07acc0 msgr2=0x7fe45c19b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:52.067 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.390+0000 7fe460c00700 1 --2- 172.21.15.119:0/90465981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45c07acc0 0x7fe45c19b8c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe44c000c00 tx=0x7fe44c011b50 comp rx=0 tx=0).stop 2024-05-27T09:17:52.067 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.390+0000 7fe460c00700 1 -- 172.21.15.119:0/90465981 shutdown_connections 2024-05-27T09:17:52.067 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.390+0000 7fe460c00700 1 --2- 172.21.15.119:0/90465981 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fe4400383f0 0x7fe44003a8a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:52.067 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.390+0000 7fe460c00700 1 --2- 172.21.15.119:0/90465981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45c07acc0 0x7fe45c19b8c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:52.067 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.390+0000 7fe460c00700 1 -- 172.21.15.119:0/90465981 >> 172.21.15.119:0/90465981 conn(0x7fe45c100fb0 msgr2=0x7fe45c1026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:52.067 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.390+0000 7fe460c00700 1 -- 172.21.15.119:0/90465981 shutdown_connections 2024-05-27T09:17:52.067 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:51.390+0000 7fe460c00700 1 -- 172.21.15.119:0/90465981 wait complete. 2024-05-27T09:17:52.068 INFO:teuthology.orchestra.run.smithi119.stdout:Deploying prometheus service with default placement... 2024-05-27T09:17:53.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:53 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' 2024-05-27T09:17:53.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:53 smithi119 ceph-mon[29082]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:17:53.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:53 smithi119 ceph-mon[29082]: Saving service prometheus spec with placement count:1 2024-05-27T09:17:53.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:53 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' 2024-05-27T09:17:53.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:53 smithi119 ceph-mon[29082]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:17:53.526 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-05-27T09:17:53.527 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.618+0000 7f3407c00700 1 Processor -- start 2024-05-27T09:17:53.527 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.618+0000 7f3407c00700 1 -- start start 2024-05-27T09:17:53.527 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.619+0000 7f3407c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3400079cf0 0x7f34001086d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:53.528 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.619+0000 7f3407c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3400108c10 con 0x7f3400079cf0 2024-05-27T09:17:53.528 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.620+0000 7f3405200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3400079cf0 0x7f34001086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:53.528 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.620+0000 7f3405200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3400079cf0 0x7f34001086d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40042/0 (socket says 172.21.15.119:40042) 2024-05-27T09:17:53.528 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.620+0000 7f3405200700 1 -- 172.21.15.119:0/4239956800 learned_addr learned my addr 172.21.15.119:0/4239956800 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:53.529 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.620+0000 7f3405200700 1 -- 172.21.15.119:0/4239956800 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3400108d50 con 0x7f3400079cf0 2024-05-27T09:17:53.529 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.621+0000 7f3405200700 1 --2- 172.21.15.119:0/4239956800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3400079cf0 0x7f34001086d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f33f40098a0 tx=0x7f33f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a62bc0cc6e78b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:53.529 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.621+0000 7f33ff400700 1 -- 172.21.15.119:0/4239956800 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33f4004180 con 0x7f3400079cf0 2024-05-27T09:17:53.529 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.621+0000 7f33ff400700 1 -- 172.21.15.119:0/4239956800 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f33f40042e0 con 0x7f3400079cf0 2024-05-27T09:17:53.529 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.621+0000 7f33ff400700 1 -- 172.21.15.119:0/4239956800 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33f4011af0 con 0x7f3400079cf0 2024-05-27T09:17:53.530 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.622+0000 7f3407c00700 1 -- 172.21.15.119:0/4239956800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3400079cf0 msgr2=0x7f34001086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:53.530 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.622+0000 7f3407c00700 1 --2- 172.21.15.119:0/4239956800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3400079cf0 0x7f34001086d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f33f40098a0 tx=0x7f33f4009bb0 comp rx=0 tx=0).stop 2024-05-27T09:17:53.531 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.622+0000 7f3407c00700 1 -- 172.21.15.119:0/4239956800 shutdown_connections 2024-05-27T09:17:53.531 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.622+0000 7f3407c00700 1 --2- 172.21.15.119:0/4239956800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3400079cf0 0x7f34001086d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:53.531 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.622+0000 7f3407c00700 1 -- 172.21.15.119:0/4239956800 >> 172.21.15.119:0/4239956800 conn(0x7f3400100060 msgr2=0x7f34001024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:53.531 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.622+0000 7f3407c00700 1 -- 172.21.15.119:0/4239956800 shutdown_connections 2024-05-27T09:17:53.531 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.622+0000 7f3407c00700 1 -- 172.21.15.119:0/4239956800 wait complete. 2024-05-27T09:17:53.531 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.623+0000 7f3407c00700 1 Processor -- start 2024-05-27T09:17:53.531 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.623+0000 7f3407c00700 1 -- start start 2024-05-27T09:17:53.531 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.624+0000 7f3407c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34001998e0 0x7f3400199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:53.532 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.624+0000 7f3407c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340019a230 con 0x7f34001998e0 2024-05-27T09:17:53.532 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.624+0000 7f3405200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34001998e0 0x7f3400199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:53.532 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.624+0000 7f3405200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34001998e0 0x7f3400199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40054/0 (socket says 172.21.15.119:40054) 2024-05-27T09:17:53.533 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.624+0000 7f3405200700 1 -- 172.21.15.119:0/4058224828 learned_addr learned my addr 172.21.15.119:0/4058224828 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:53.533 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.624+0000 7f3405200700 1 -- 172.21.15.119:0/4058224828 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f4009550 con 0x7f34001998e0 2024-05-27T09:17:53.533 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.625+0000 7f3405200700 1 --2- 172.21.15.119:0/4058224828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34001998e0 0x7f3400199cf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f33f4011670 tx=0x7f33f4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:53.533 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.625+0000 7f33fd600700 1 -- 172.21.15.119:0/4058224828 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33f40039f0 con 0x7f34001998e0 2024-05-27T09:17:53.533 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.625+0000 7f33fd600700 1 -- 172.21.15.119:0/4058224828 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f33f4003b50 con 0x7f34001998e0 2024-05-27T09:17:53.534 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.625+0000 7f33fd600700 1 -- 172.21.15.119:0/4058224828 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33f40247c0 con 0x7f34001998e0 2024-05-27T09:17:53.534 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.625+0000 7f3407c00700 1 -- 172.21.15.119:0/4058224828 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340019a430 con 0x7f34001998e0 2024-05-27T09:17:53.534 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.625+0000 7f3407c00700 1 -- 172.21.15.119:0/4058224828 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f340019d090 con 0x7f34001998e0 2024-05-27T09:17:53.534 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.626+0000 7f33fd600700 1 -- 172.21.15.119:0/4058224828 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f33f401ac00 con 0x7f34001998e0 2024-05-27T09:17:53.534 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.626+0000 7f33fd600700 1 --2- 172.21.15.119:0/4058224828 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f33e8038440 0x7f33e803a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:53.534 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.627+0000 7f33fd600700 1 -- 172.21.15.119:0/4058224828 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f33f404c460 con 0x7f34001998e0 2024-05-27T09:17:53.534 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.627+0000 7f3407c00700 1 -- 172.21.15.119:0/4058224828 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3400079cf0 con 0x7f34001998e0 2024-05-27T09:17:53.535 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.627+0000 7f33ffe00700 1 --2- 172.21.15.119:0/4058224828 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f33e8038440 0x7f33e803a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:53.535 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.628+0000 7f33ffe00700 1 --2- 172.21.15.119:0/4058224828 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f33e8038440 0x7f33e803a8f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f33ec006fd0 tx=0x7f33ec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:53.535 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.636+0000 7f33fd600700 1 -- 172.21.15.119:0/4058224828 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f33f402d3f0 con 0x7f34001998e0 2024-05-27T09:17:53.535 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.863+0000 7f3407c00700 1 -- 172.21.15.119:0/4058224828 --> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f34001023d0 con 0x7f33e8038440 2024-05-27T09:17:53.536 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.872+0000 7f33fd600700 1 -- 172.21.15.119:0/4058224828 <== mgr.14120 v2:172.21.15.119:6800/1553098156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f34001023d0 con 0x7f33e8038440 2024-05-27T09:17:53.536 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.875+0000 7f3407c00700 1 -- 172.21.15.119:0/4058224828 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f33e8038440 msgr2=0x7f33e803a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:53.536 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.875+0000 7f3407c00700 1 --2- 172.21.15.119:0/4058224828 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f33e8038440 0x7f33e803a8f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f33ec006fd0 tx=0x7f33ec006e40 comp rx=0 tx=0).stop 2024-05-27T09:17:53.537 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.875+0000 7f3407c00700 1 -- 172.21.15.119:0/4058224828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34001998e0 msgr2=0x7f3400199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:53.537 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.875+0000 7f3407c00700 1 --2- 172.21.15.119:0/4058224828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34001998e0 0x7f3400199cf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f33f4011670 tx=0x7f33f4011b50 comp rx=0 tx=0).stop 2024-05-27T09:17:53.537 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.875+0000 7f3407c00700 1 -- 172.21.15.119:0/4058224828 shutdown_connections 2024-05-27T09:17:53.537 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.875+0000 7f3407c00700 1 --2- 172.21.15.119:0/4058224828 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f33e8038440 0x7f33e803a8f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:53.537 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.875+0000 7f3407c00700 1 --2- 172.21.15.119:0/4058224828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34001998e0 0x7f3400199cf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:53.537 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.875+0000 7f3407c00700 1 -- 172.21.15.119:0/4058224828 >> 172.21.15.119:0/4058224828 conn(0x7f3400100060 msgr2=0x7f34001017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:53.537 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.876+0000 7f3407c00700 1 -- 172.21.15.119:0/4058224828 shutdown_connections 2024-05-27T09:17:53.538 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:52.876+0000 7f3407c00700 1 -- 172.21.15.119:0/4058224828 wait complete. 2024-05-27T09:17:53.538 INFO:teuthology.orchestra.run.smithi119.stdout:Deploying grafana service with default placement... 2024-05-27T09:17:55.406 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-05-27T09:17:55.407 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.358+0000 7ffbc4000700 1 Processor -- start 2024-05-27T09:17:55.407 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.359+0000 7ffbc4000700 1 -- start start 2024-05-27T09:17:55.407 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.359+0000 7ffbc4000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbbc07ace0 0x7ffbbc0791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:55.407 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.359+0000 7ffbc4000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc079720 con 0x7ffbbc07ace0 2024-05-27T09:17:55.407 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.359+0000 7ffbc1600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbbc07ace0 0x7ffbbc0791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:55.407 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.359+0000 7ffbc1600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbbc07ace0 0x7ffbbc0791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40066/0 (socket says 172.21.15.119:40066) 2024-05-27T09:17:55.407 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.359+0000 7ffbc1600700 1 -- 172.21.15.119:0/1725018041 learned_addr learned my addr 172.21.15.119:0/1725018041 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:55.407 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.360+0000 7ffbc1600700 1 -- 172.21.15.119:0/1725018041 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc079860 con 0x7ffbbc07ace0 2024-05-27T09:17:55.407 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.360+0000 7ffbc1600700 1 --2- 172.21.15.119:0/1725018041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbbc07ace0 0x7ffbbc0791e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009e80 tx=0x7ffbac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c617963f54d9e72 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:55.407 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.360+0000 7ffbbbe00700 1 -- 172.21.15.119:0/1725018041 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffbac00e070 con 0x7ffbbc07ace0 2024-05-27T09:17:55.407 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.360+0000 7ffbbbe00700 1 -- 172.21.15.119:0/1725018041 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffbac00b8c0 con 0x7ffbbc07ace0 2024-05-27T09:17:55.407 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.360+0000 7ffbbbe00700 1 -- 172.21.15.119:0/1725018041 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffbac0044d0 con 0x7ffbbc07ace0 2024-05-27T09:17:55.408 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.361+0000 7ffbc4000700 1 -- 172.21.15.119:0/1725018041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbbc07ace0 msgr2=0x7ffbbc0791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:55.408 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.361+0000 7ffbc4000700 1 --2- 172.21.15.119:0/1725018041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbbc07ace0 0x7ffbbc0791e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009e80 tx=0x7ffbac00b330 comp rx=0 tx=0).stop 2024-05-27T09:17:55.408 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.361+0000 7ffbc4000700 1 -- 172.21.15.119:0/1725018041 shutdown_connections 2024-05-27T09:17:55.408 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.361+0000 7ffbc4000700 1 --2- 172.21.15.119:0/1725018041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbbc07ace0 0x7ffbbc0791e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:55.408 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.361+0000 7ffbc4000700 1 -- 172.21.15.119:0/1725018041 >> 172.21.15.119:0/1725018041 conn(0x7ffbbc101160 msgr2=0x7ffbbc103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:55.408 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.361+0000 7ffbc4000700 1 -- 172.21.15.119:0/1725018041 shutdown_connections 2024-05-27T09:17:55.408 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.361+0000 7ffbc4000700 1 -- 172.21.15.119:0/1725018041 wait complete. 2024-05-27T09:17:55.408 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.362+0000 7ffbc4000700 1 Processor -- start 2024-05-27T09:17:55.408 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.363+0000 7ffbc4000700 1 -- start start 2024-05-27T09:17:55.408 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.363+0000 7ffbc4000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbbc07ace0 0x7ffbbc19baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:55.408 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.363+0000 7ffbc4000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc19bfe0 con 0x7ffbbc07ace0 2024-05-27T09:17:55.408 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.363+0000 7ffbc1600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbbc07ace0 0x7ffbbc19baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:55.408 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.363+0000 7ffbc1600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbbc07ace0 0x7ffbbc19baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40080/0 (socket says 172.21.15.119:40080) 2024-05-27T09:17:55.409 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.363+0000 7ffbc1600700 1 -- 172.21.15.119:0/704310664 learned_addr learned my addr 172.21.15.119:0/704310664 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:55.409 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.364+0000 7ffbc1600700 1 -- 172.21.15.119:0/704310664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbac009b30 con 0x7ffbbc07ace0 2024-05-27T09:17:55.409 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.364+0000 7ffbc1600700 1 --2- 172.21.15.119:0/704310664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbbc07ace0 0x7ffbbc19baa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ffbac004070 tx=0x7ffbac0049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:55.409 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.364+0000 7ffbba000700 1 -- 172.21.15.119:0/704310664 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffbac00e040 con 0x7ffbbc07ace0 2024-05-27T09:17:55.409 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.364+0000 7ffbba000700 1 -- 172.21.15.119:0/704310664 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffbac00bc40 con 0x7ffbbc07ace0 2024-05-27T09:17:55.409 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.364+0000 7ffbc4000700 1 -- 172.21.15.119:0/704310664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbbc19c1e0 con 0x7ffbbc07ace0 2024-05-27T09:17:55.409 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.364+0000 7ffbba000700 1 -- 172.21.15.119:0/704310664 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffbac0126b0 con 0x7ffbbc07ace0 2024-05-27T09:17:55.409 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.364+0000 7ffbc4000700 1 -- 172.21.15.119:0/704310664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbbc19c660 con 0x7ffbbc07ace0 2024-05-27T09:17:55.409 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.366+0000 7ffbc4000700 1 -- 172.21.15.119:0/704310664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbbc0623f0 con 0x7ffbbc07ace0 2024-05-27T09:17:55.409 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.366+0000 7ffbba000700 1 -- 172.21.15.119:0/704310664 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7ffbac019040 con 0x7ffbbc07ace0 2024-05-27T09:17:55.409 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.366+0000 7ffbba000700 1 --2- 172.21.15.119:0/704310664 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7ffba0038440 0x7ffba003a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:55.409 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.366+0000 7ffbba000700 1 -- 172.21.15.119:0/704310664 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ffbac04bcf0 con 0x7ffbbc07ace0 2024-05-27T09:17:55.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.366+0000 7ffbc0c00700 1 --2- 172.21.15.119:0/704310664 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7ffba0038440 0x7ffba003a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:55.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.367+0000 7ffbc0c00700 1 --2- 172.21.15.119:0/704310664 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7ffba0038440 0x7ffba003a8f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffba8006fd0 tx=0x7ffba8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:55.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.371+0000 7ffbba000700 1 -- 172.21.15.119:0/704310664 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ffbbc0623f0 con 0x7ffbbc07ace0 2024-05-27T09:17:55.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.604+0000 7ffbc4000700 1 -- 172.21.15.119:0/704310664 --> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7ffbbc062380 con 0x7ffba0038440 2024-05-27T09:17:55.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.612+0000 7ffbba000700 1 -- 172.21.15.119:0/704310664 <== mgr.14120 v2:172.21.15.119:6800/1553098156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7ffbbc062380 con 0x7ffba0038440 2024-05-27T09:17:55.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.614+0000 7ffbc4000700 1 -- 172.21.15.119:0/704310664 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7ffba0038440 msgr2=0x7ffba003a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:55.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.614+0000 7ffbc4000700 1 --2- 172.21.15.119:0/704310664 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7ffba0038440 0x7ffba003a8f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffba8006fd0 tx=0x7ffba8006e40 comp rx=0 tx=0).stop 2024-05-27T09:17:55.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.614+0000 7ffbc4000700 1 -- 172.21.15.119:0/704310664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbbc07ace0 msgr2=0x7ffbbc19baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:55.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.614+0000 7ffbc4000700 1 --2- 172.21.15.119:0/704310664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbbc07ace0 0x7ffbbc19baa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ffbac004070 tx=0x7ffbac0049f0 comp rx=0 tx=0).stop 2024-05-27T09:17:55.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.615+0000 7ffbc4000700 1 -- 172.21.15.119:0/704310664 shutdown_connections 2024-05-27T09:17:55.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.615+0000 7ffbc4000700 1 --2- 172.21.15.119:0/704310664 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7ffba0038440 0x7ffba003a8f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:55.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.615+0000 7ffbc4000700 1 --2- 172.21.15.119:0/704310664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbbc07ace0 0x7ffbbc19baa0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:55.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.615+0000 7ffbc4000700 1 -- 172.21.15.119:0/704310664 >> 172.21.15.119:0/704310664 conn(0x7ffbbc101160 msgr2=0x7ffbbc101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:55.411 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.615+0000 7ffbc4000700 1 -- 172.21.15.119:0/704310664 shutdown_connections 2024-05-27T09:17:55.411 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:54.615+0000 7ffbc4000700 1 -- 172.21.15.119:0/704310664 wait complete. 2024-05-27T09:17:55.411 INFO:teuthology.orchestra.run.smithi119.stdout:Deploying node-exporter service with default placement... 2024-05-27T09:17:55.720 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:55 smithi119 ceph-mon[29082]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:17:55.720 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:55 smithi119 ceph-mon[29082]: Saving service grafana spec with placement count:1 2024-05-27T09:17:55.720 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:55 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' 2024-05-27T09:17:55.720 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:55 smithi119 ceph-mon[29082]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:17:57.253 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-05-27T09:17:57.253 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.411+0000 7f3e54200700 1 Processor -- start 2024-05-27T09:17:57.253 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.411+0000 7f3e54200700 1 -- start start 2024-05-27T09:17:57.254 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.411+0000 7f3e54200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e4c0802e0 0x7f3e4c10ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:57.254 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.411+0000 7f3e54200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c10f210 con 0x7f3e4c0802e0 2024-05-27T09:17:57.254 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.412+0000 7f3e51800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e4c0802e0 0x7f3e4c10ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:57.254 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.412+0000 7f3e51800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e4c0802e0 0x7f3e4c10ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40088/0 (socket says 172.21.15.119:40088) 2024-05-27T09:17:57.254 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.412+0000 7f3e51800700 1 -- 172.21.15.119:0/2354451998 learned_addr learned my addr 172.21.15.119:0/2354451998 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:57.255 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.413+0000 7f3e51800700 1 -- 172.21.15.119:0/2354451998 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4c10f350 con 0x7f3e4c0802e0 2024-05-27T09:17:57.255 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.413+0000 7f3e51800700 1 --2- 172.21.15.119:0/2354451998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e4c0802e0 0x7f3e4c10ecd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3e400098a0 tx=0x7f3e40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b192b2d265ec2baf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:57.256 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.413+0000 7f3e4be00700 1 -- 172.21.15.119:0/2354451998 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e40004180 con 0x7f3e4c0802e0 2024-05-27T09:17:57.256 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.413+0000 7f3e4be00700 1 -- 172.21.15.119:0/2354451998 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3e400042e0 con 0x7f3e4c0802e0 2024-05-27T09:17:57.256 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.413+0000 7f3e4be00700 1 -- 172.21.15.119:0/2354451998 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e40011af0 con 0x7f3e4c0802e0 2024-05-27T09:17:57.256 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.414+0000 7f3e54200700 1 -- 172.21.15.119:0/2354451998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e4c0802e0 msgr2=0x7f3e4c10ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:57.256 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.414+0000 7f3e54200700 1 --2- 172.21.15.119:0/2354451998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e4c0802e0 0x7f3e4c10ecd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3e400098a0 tx=0x7f3e40009bb0 comp rx=0 tx=0).stop 2024-05-27T09:17:57.256 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.414+0000 7f3e54200700 1 -- 172.21.15.119:0/2354451998 shutdown_connections 2024-05-27T09:17:57.256 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.414+0000 7f3e54200700 1 --2- 172.21.15.119:0/2354451998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e4c0802e0 0x7f3e4c10ecd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:57.256 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.414+0000 7f3e54200700 1 -- 172.21.15.119:0/2354451998 >> 172.21.15.119:0/2354451998 conn(0x7f3e4c07b220 msgr2=0x7f3e4c07d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:57.256 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.414+0000 7f3e54200700 1 -- 172.21.15.119:0/2354451998 shutdown_connections 2024-05-27T09:17:57.256 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.414+0000 7f3e54200700 1 -- 172.21.15.119:0/2354451998 wait complete. 2024-05-27T09:17:57.256 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.415+0000 7f3e54200700 1 Processor -- start 2024-05-27T09:17:57.258 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.416+0000 7f3e54200700 1 -- start start 2024-05-27T09:17:57.259 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.416+0000 7f3e54200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e4c0802e0 0x7f3e4c19fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:57.259 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.416+0000 7f3e54200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c1a03e0 con 0x7f3e4c0802e0 2024-05-27T09:17:57.259 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.416+0000 7f3e51800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e4c0802e0 0x7f3e4c19fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:57.259 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.416+0000 7f3e51800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e4c0802e0 0x7f3e4c19fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40104/0 (socket says 172.21.15.119:40104) 2024-05-27T09:17:57.259 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.416+0000 7f3e51800700 1 -- 172.21.15.119:0/493644617 learned_addr learned my addr 172.21.15.119:0/493644617 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:57.259 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.417+0000 7f3e51800700 1 -- 172.21.15.119:0/493644617 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e40009550 con 0x7f3e4c0802e0 2024-05-27T09:17:57.259 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.417+0000 7f3e51800700 1 --2- 172.21.15.119:0/493644617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e4c0802e0 0x7f3e4c19fea0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e40006ba0 tx=0x7f3e40011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:57.259 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.417+0000 7f3e4a000700 1 -- 172.21.15.119:0/493644617 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e40003b00 con 0x7f3e4c0802e0 2024-05-27T09:17:57.259 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.417+0000 7f3e4a000700 1 -- 172.21.15.119:0/493644617 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3e4001a8e0 con 0x7f3e4c0802e0 2024-05-27T09:17:57.259 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.417+0000 7f3e54200700 1 -- 172.21.15.119:0/493644617 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e4c1a05e0 con 0x7f3e4c0802e0 2024-05-27T09:17:57.259 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.417+0000 7f3e4a000700 1 -- 172.21.15.119:0/493644617 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e40019990 con 0x7f3e4c0802e0 2024-05-27T09:17:57.259 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.417+0000 7f3e54200700 1 -- 172.21.15.119:0/493644617 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e4c1a0a00 con 0x7f3e4c0802e0 2024-05-27T09:17:57.259 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.419+0000 7f3e54200700 1 -- 172.21.15.119:0/493644617 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e4c110200 con 0x7f3e4c0802e0 2024-05-27T09:17:57.259 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.419+0000 7f3e4a000700 1 -- 172.21.15.119:0/493644617 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f3e40019c00 con 0x7f3e4c0802e0 2024-05-27T09:17:57.260 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.419+0000 7f3e4a000700 1 --2- 172.21.15.119:0/493644617 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f3e340383f0 0x7f3e3403a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:57.260 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.419+0000 7f3e4a000700 1 -- 172.21.15.119:0/493644617 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3e4004c5f0 con 0x7f3e4c0802e0 2024-05-27T09:17:57.260 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.419+0000 7f3e50e00700 1 --2- 172.21.15.119:0/493644617 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f3e340383f0 0x7f3e3403a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:57.260 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.420+0000 7f3e50e00700 1 --2- 172.21.15.119:0/493644617 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f3e340383f0 0x7f3e3403a8a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3e38006fd0 tx=0x7f3e38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:57.261 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.424+0000 7f3e4a000700 1 -- 172.21.15.119:0/493644617 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3e40018710 con 0x7f3e4c0802e0 2024-05-27T09:17:57.261 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.651+0000 7f3e54200700 1 -- 172.21.15.119:0/493644617 --> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e4c1a3860 con 0x7f3e340383f0 2024-05-27T09:17:57.262 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.659+0000 7f3e4a000700 1 -- 172.21.15.119:0/493644617 <== mgr.14120 v2:172.21.15.119:6800/1553098156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f3e4c1a3860 con 0x7f3e340383f0 2024-05-27T09:17:57.262 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.662+0000 7f3e54200700 1 -- 172.21.15.119:0/493644617 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f3e340383f0 msgr2=0x7f3e3403a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:57.262 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.662+0000 7f3e54200700 1 --2- 172.21.15.119:0/493644617 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f3e340383f0 0x7f3e3403a8a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3e38006fd0 tx=0x7f3e38006e40 comp rx=0 tx=0).stop 2024-05-27T09:17:57.262 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.662+0000 7f3e54200700 1 -- 172.21.15.119:0/493644617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e4c0802e0 msgr2=0x7f3e4c19fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:57.262 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.662+0000 7f3e54200700 1 --2- 172.21.15.119:0/493644617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e4c0802e0 0x7f3e4c19fea0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e40006ba0 tx=0x7f3e40011b50 comp rx=0 tx=0).stop 2024-05-27T09:17:57.262 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.662+0000 7f3e54200700 1 -- 172.21.15.119:0/493644617 shutdown_connections 2024-05-27T09:17:57.262 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.662+0000 7f3e54200700 1 --2- 172.21.15.119:0/493644617 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f3e340383f0 0x7f3e3403a8a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:57.262 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.662+0000 7f3e54200700 1 --2- 172.21.15.119:0/493644617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e4c0802e0 0x7f3e4c19fea0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:57.262 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.662+0000 7f3e54200700 1 -- 172.21.15.119:0/493644617 >> 172.21.15.119:0/493644617 conn(0x7f3e4c07b220 msgr2=0x7f3e4c07c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:57.262 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.662+0000 7f3e54200700 1 -- 172.21.15.119:0/493644617 shutdown_connections 2024-05-27T09:17:57.262 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:56.662+0000 7f3e54200700 1 -- 172.21.15.119:0/493644617 wait complete. 2024-05-27T09:17:57.262 INFO:teuthology.orchestra.run.smithi119.stdout:Deploying alertmanager service with default placement... 2024-05-27T09:17:57.911 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:57 smithi119 ceph-mon[29082]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:17:57.911 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:57 smithi119 ceph-mon[29082]: Saving service node-exporter spec with placement * 2024-05-27T09:17:57.911 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:57 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' 2024-05-27T09:17:57.911 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:57 smithi119 ceph-mon[29082]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:17:58.827 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-05-27T09:17:58.832 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.909+0000 7f265aa00700 1 Processor -- start 2024-05-27T09:17:58.832 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.909+0000 7f265aa00700 1 -- start start 2024-05-27T09:17:58.832 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.909+0000 7f265aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2654105b10 0x7f2654105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:58.832 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.909+0000 7f265aa00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2654106460 con 0x7f2654105b10 2024-05-27T09:17:58.832 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.910+0000 7f2653e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2654105b10 0x7f2654105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:58.832 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.910+0000 7f2653e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2654105b10 0x7f2654105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40120/0 (socket says 172.21.15.119:40120) 2024-05-27T09:17:58.832 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.910+0000 7f2653e00700 1 -- 172.21.15.119:0/2503276086 learned_addr learned my addr 172.21.15.119:0/2503276086 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:58.832 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.910+0000 7f2653e00700 1 -- 172.21.15.119:0/2503276086 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26541065a0 con 0x7f2654105b10 2024-05-27T09:17:58.832 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.911+0000 7f2653e00700 1 --2- 172.21.15.119:0/2503276086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2654105b10 0x7f2654105f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f26440098a0 tx=0x7f2644009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed88aba9995d2148 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:58.833 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.911+0000 7f2652a00700 1 -- 172.21.15.119:0/2503276086 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2644004180 con 0x7f2654105b10 2024-05-27T09:17:58.833 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.911+0000 7f2652a00700 1 -- 172.21.15.119:0/2503276086 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f26440042e0 con 0x7f2654105b10 2024-05-27T09:17:58.833 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.911+0000 7f2652a00700 1 -- 172.21.15.119:0/2503276086 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2644011af0 con 0x7f2654105b10 2024-05-27T09:17:58.833 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.912+0000 7f265aa00700 1 -- 172.21.15.119:0/2503276086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2654105b10 msgr2=0x7f2654105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:58.833 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.912+0000 7f265aa00700 1 --2- 172.21.15.119:0/2503276086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2654105b10 0x7f2654105f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f26440098a0 tx=0x7f2644009bb0 comp rx=0 tx=0).stop 2024-05-27T09:17:58.833 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.912+0000 7f265aa00700 1 -- 172.21.15.119:0/2503276086 shutdown_connections 2024-05-27T09:17:58.833 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.912+0000 7f265aa00700 1 --2- 172.21.15.119:0/2503276086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2654105b10 0x7f2654105f20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:58.833 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.912+0000 7f265aa00700 1 -- 172.21.15.119:0/2503276086 >> 172.21.15.119:0/2503276086 conn(0x7f2654101180 msgr2=0x7f2654103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:58.833 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.912+0000 7f265aa00700 1 -- 172.21.15.119:0/2503276086 shutdown_connections 2024-05-27T09:17:58.833 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.912+0000 7f265aa00700 1 -- 172.21.15.119:0/2503276086 wait complete. 2024-05-27T09:17:58.833 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.913+0000 7f265aa00700 1 Processor -- start 2024-05-27T09:17:58.833 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.913+0000 7f265aa00700 1 -- start start 2024-05-27T09:17:58.833 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.914+0000 7f265aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2654105b10 0x7f265419ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:58.834 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.914+0000 7f265aa00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265419bf90 con 0x7f2654105b10 2024-05-27T09:17:58.834 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.914+0000 7f2653e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2654105b10 0x7f265419ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:58.834 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.914+0000 7f2653e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2654105b10 0x7f265419ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40128/0 (socket says 172.21.15.119:40128) 2024-05-27T09:17:58.836 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.914+0000 7f2653e00700 1 -- 172.21.15.119:0/511822111 learned_addr learned my addr 172.21.15.119:0/511822111 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:17:58.837 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.914+0000 7f2653e00700 1 -- 172.21.15.119:0/511822111 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2644009550 con 0x7f2654105b10 2024-05-27T09:17:58.837 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.915+0000 7f2653e00700 1 --2- 172.21.15.119:0/511822111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2654105b10 0x7f265419ba50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2644011ad0 tx=0x7f2644011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:58.837 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.915+0000 7f2650c00700 1 -- 172.21.15.119:0/511822111 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2644003ab0 con 0x7f2654105b10 2024-05-27T09:17:58.837 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.915+0000 7f2650c00700 1 -- 172.21.15.119:0/511822111 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2644003c10 con 0x7f2654105b10 2024-05-27T09:17:58.837 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.915+0000 7f2650c00700 1 -- 172.21.15.119:0/511822111 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26440198e0 con 0x7f2654105b10 2024-05-27T09:17:58.837 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.915+0000 7f265aa00700 1 -- 172.21.15.119:0/511822111 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265419c190 con 0x7f2654105b10 2024-05-27T09:17:58.837 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.915+0000 7f265aa00700 1 -- 172.21.15.119:0/511822111 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265419c5b0 con 0x7f2654105b10 2024-05-27T09:17:58.837 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.916+0000 7f2650c00700 1 -- 172.21.15.119:0/511822111 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f2644019ba0 con 0x7f2654105b10 2024-05-27T09:17:58.837 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.916+0000 7f265aa00700 1 -- 172.21.15.119:0/511822111 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2654062380 con 0x7f2654105b10 2024-05-27T09:17:58.837 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.917+0000 7f2650c00700 1 --2- 172.21.15.119:0/511822111 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f26380383f0 0x7f263803a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:17:58.837 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.917+0000 7f2650c00700 1 -- 172.21.15.119:0/511822111 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f264404c1a0 con 0x7f2654105b10 2024-05-27T09:17:58.838 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.917+0000 7f2653400700 1 --2- 172.21.15.119:0/511822111 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f26380383f0 0x7f263803a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:17:58.838 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.918+0000 7f2653400700 1 --2- 172.21.15.119:0/511822111 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f26380383f0 0x7f263803a8a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2640006fd0 tx=0x7f2640006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:17:58.838 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:57.922+0000 7f2650c00700 1 -- 172.21.15.119:0/511822111 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f264401a430 con 0x7f2654105b10 2024-05-27T09:17:58.838 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:58.147+0000 7f265aa00700 1 -- 172.21.15.119:0/511822111 --> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f2654102610 con 0x7f26380383f0 2024-05-27T09:17:58.838 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:58.156+0000 7f2650c00700 1 -- 172.21.15.119:0/511822111 <== mgr.14120 v2:172.21.15.119:6800/1553098156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f2654102610 con 0x7f26380383f0 2024-05-27T09:17:58.838 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:58.160+0000 7f265aa00700 1 -- 172.21.15.119:0/511822111 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f26380383f0 msgr2=0x7f263803a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:58.838 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:58.160+0000 7f265aa00700 1 --2- 172.21.15.119:0/511822111 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f26380383f0 0x7f263803a8a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2640006fd0 tx=0x7f2640006e40 comp rx=0 tx=0).stop 2024-05-27T09:17:58.838 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:58.160+0000 7f265aa00700 1 -- 172.21.15.119:0/511822111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2654105b10 msgr2=0x7f265419ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:17:58.838 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:58.160+0000 7f265aa00700 1 --2- 172.21.15.119:0/511822111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2654105b10 0x7f265419ba50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2644011ad0 tx=0x7f2644011bb0 comp rx=0 tx=0).stop 2024-05-27T09:17:58.838 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:58.160+0000 7f265aa00700 1 -- 172.21.15.119:0/511822111 shutdown_connections 2024-05-27T09:17:58.838 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:58.160+0000 7f265aa00700 1 --2- 172.21.15.119:0/511822111 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f26380383f0 0x7f263803a8a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:58.838 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:58.160+0000 7f265aa00700 1 --2- 172.21.15.119:0/511822111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2654105b10 0x7f265419ba50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:17:58.838 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:58.160+0000 7f265aa00700 1 -- 172.21.15.119:0/511822111 >> 172.21.15.119:0/511822111 conn(0x7f2654101180 msgr2=0x7f2654101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:17:58.839 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:58.160+0000 7f265aa00700 1 -- 172.21.15.119:0/511822111 shutdown_connections 2024-05-27T09:17:58.839 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:58.160+0000 7f265aa00700 1 -- 172.21.15.119:0/511822111 wait complete. 2024-05-27T09:17:59.465 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:59 smithi119 ceph-mon[29082]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:17:59.466 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:59 smithi119 ceph-mon[29082]: Saving service alertmanager spec with placement count:1 2024-05-27T09:17:59.466 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:17:59 smithi119 ceph-mon[29082]: from='mgr.14120 172.21.15.119:0/4081645768' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:00.298 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:00 smithi119 ceph-mon[29082]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:18:00.298 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:00 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/1263608844' entity='client.admin' 2024-05-27T09:18:00.313 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.465+0000 7f943f600700 1 Processor -- start 2024-05-27T09:18:00.313 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.465+0000 7f943f600700 1 -- start start 2024-05-27T09:18:00.313 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.465+0000 7f943f600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9438104a00 0x7f9438104e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:00.314 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.465+0000 7f943f600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9438105350 con 0x7f9438104a00 2024-05-27T09:18:00.314 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.466+0000 7f943cc00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9438104a00 0x7f9438104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:00.314 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.466+0000 7f943cc00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9438104a00 0x7f9438104e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54632/0 (socket says 172.21.15.119:54632) 2024-05-27T09:18:00.314 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.466+0000 7f943cc00700 1 -- 172.21.15.119:0/1591633145 learned_addr learned my addr 172.21.15.119:0/1591633145 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:00.314 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.466+0000 7f943cc00700 1 -- 172.21.15.119:0/1591633145 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9438105490 con 0x7f9438104a00 2024-05-27T09:18:00.314 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.467+0000 7f943cc00700 1 --2- 172.21.15.119:0/1591633145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9438104a00 0x7f9438104e10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f942c0098a0 tx=0x7f942c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9709952b910a3e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:00.314 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.467+0000 7f9437400700 1 -- 172.21.15.119:0/1591633145 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f942c004180 con 0x7f9438104a00 2024-05-27T09:18:00.314 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.467+0000 7f9437400700 1 -- 172.21.15.119:0/1591633145 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f942c0042e0 con 0x7f9438104a00 2024-05-27T09:18:00.314 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.467+0000 7f9437400700 1 -- 172.21.15.119:0/1591633145 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f942c011af0 con 0x7f9438104a00 2024-05-27T09:18:00.314 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.467+0000 7f943f600700 1 -- 172.21.15.119:0/1591633145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9438104a00 msgr2=0x7f9438104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:00.314 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.467+0000 7f943f600700 1 --2- 172.21.15.119:0/1591633145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9438104a00 0x7f9438104e10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f942c0098a0 tx=0x7f942c009bb0 comp rx=0 tx=0).stop 2024-05-27T09:18:00.314 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.468+0000 7f943f600700 1 -- 172.21.15.119:0/1591633145 shutdown_connections 2024-05-27T09:18:00.315 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.468+0000 7f943f600700 1 --2- 172.21.15.119:0/1591633145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9438104a00 0x7f9438104e10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:00.315 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.468+0000 7f943f600700 1 -- 172.21.15.119:0/1591633145 >> 172.21.15.119:0/1591633145 conn(0x7f9438100070 msgr2=0x7f9438102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:00.315 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.468+0000 7f943f600700 1 -- 172.21.15.119:0/1591633145 shutdown_connections 2024-05-27T09:18:00.315 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.468+0000 7f943f600700 1 -- 172.21.15.119:0/1591633145 wait complete. 2024-05-27T09:18:00.315 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.469+0000 7f943f600700 1 Processor -- start 2024-05-27T09:18:00.315 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.469+0000 7f943f600700 1 -- start start 2024-05-27T09:18:00.315 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.469+0000 7f943f600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9438104a00 0x7f943818fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:00.315 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.469+0000 7f943f600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9438190120 con 0x7f9438104a00 2024-05-27T09:18:00.315 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.470+0000 7f943cc00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9438104a00 0x7f943818fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:00.315 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.470+0000 7f943cc00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9438104a00 0x7f943818fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54644/0 (socket says 172.21.15.119:54644) 2024-05-27T09:18:00.315 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.470+0000 7f943cc00700 1 -- 172.21.15.119:0/1263608844 learned_addr learned my addr 172.21.15.119:0/1263608844 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:00.315 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.470+0000 7f943cc00700 1 -- 172.21.15.119:0/1263608844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c009550 con 0x7f9438104a00 2024-05-27T09:18:00.315 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.470+0000 7f943cc00700 1 --2- 172.21.15.119:0/1263608844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9438104a00 0x7f943818fbe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f942c011ad0 tx=0x7f942c011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:00.316 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.471+0000 7f9435600700 1 -- 172.21.15.119:0/1263608844 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f942c003ab0 con 0x7f9438104a00 2024-05-27T09:18:00.316 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.471+0000 7f9435600700 1 -- 172.21.15.119:0/1263608844 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f942c003c10 con 0x7f9438104a00 2024-05-27T09:18:00.316 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.471+0000 7f9435600700 1 -- 172.21.15.119:0/1263608844 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f942c0198e0 con 0x7f9438104a00 2024-05-27T09:18:00.316 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.471+0000 7f943f600700 1 -- 172.21.15.119:0/1263608844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943818e230 con 0x7f9438104a00 2024-05-27T09:18:00.316 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.471+0000 7f943f600700 1 -- 172.21.15.119:0/1263608844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943818e650 con 0x7f9438104a00 2024-05-27T09:18:00.316 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.472+0000 7f9435600700 1 -- 172.21.15.119:0/1263608844 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f942c019ba0 con 0x7f9438104a00 2024-05-27T09:18:00.316 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.472+0000 7f943f600700 1 -- 172.21.15.119:0/1263608844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9438062380 con 0x7f9438104a00 2024-05-27T09:18:00.316 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.472+0000 7f9435600700 1 --2- 172.21.15.119:0/1263608844 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f94200383f0 0x7f942003a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:00.316 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.473+0000 7f9435600700 1 -- 172.21.15.119:0/1263608844 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f942c04c260 con 0x7f9438104a00 2024-05-27T09:18:00.316 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.473+0000 7f9437e00700 1 --2- 172.21.15.119:0/1263608844 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f94200383f0 0x7f942003a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:00.316 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.473+0000 7f9437e00700 1 --2- 172.21.15.119:0/1263608844 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f94200383f0 0x7f942003a8a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9424006fd0 tx=0x7f9424006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:00.316 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.477+0000 7f9435600700 1 -- 172.21.15.119:0/1263608844 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f942c00b3e0 con 0x7f9438104a00 2024-05-27T09:18:00.316 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.694+0000 7f943f600700 1 -- 172.21.15.119:0/1263608844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f9438106900 con 0x7f9438104a00 2024-05-27T09:18:00.316 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.699+0000 7f9435600700 1 -- 172.21.15.119:0/1263608844 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f942c010380 con 0x7f9438104a00 2024-05-27T09:18:00.317 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.702+0000 7f943f600700 1 -- 172.21.15.119:0/1263608844 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f94200383f0 msgr2=0x7f942003a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:00.317 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.702+0000 7f943f600700 1 --2- 172.21.15.119:0/1263608844 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f94200383f0 0x7f942003a8a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9424006fd0 tx=0x7f9424006e40 comp rx=0 tx=0).stop 2024-05-27T09:18:00.317 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.702+0000 7f943f600700 1 -- 172.21.15.119:0/1263608844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9438104a00 msgr2=0x7f943818fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:00.317 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.702+0000 7f943f600700 1 --2- 172.21.15.119:0/1263608844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9438104a00 0x7f943818fbe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f942c011ad0 tx=0x7f942c011bb0 comp rx=0 tx=0).stop 2024-05-27T09:18:00.317 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.703+0000 7f943f600700 1 -- 172.21.15.119:0/1263608844 shutdown_connections 2024-05-27T09:18:00.317 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.703+0000 7f943f600700 1 --2- 172.21.15.119:0/1263608844 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f94200383f0 0x7f942003a8a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:00.317 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.703+0000 7f943f600700 1 --2- 172.21.15.119:0/1263608844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9438104a00 0x7f943818fbe0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:00.317 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.703+0000 7f943f600700 1 -- 172.21.15.119:0/1263608844 >> 172.21.15.119:0/1263608844 conn(0x7f9438100070 msgr2=0x7f94380740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:00.317 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.703+0000 7f943f600700 1 -- 172.21.15.119:0/1263608844 shutdown_connections 2024-05-27T09:18:00.317 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:17:59.703+0000 7f943f600700 1 -- 172.21.15.119:0/1263608844 wait complete. 2024-05-27T09:18:01.809 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.903+0000 7fa0a1800700 1 Processor -- start 2024-05-27T09:18:01.810 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.903+0000 7fa0a1800700 1 -- start start 2024-05-27T09:18:01.810 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.903+0000 7fa0a1800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa09c07ace0 0x7fa09c0791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:01.810 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.903+0000 7fa0a1800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c079720 con 0x7fa09c07ace0 2024-05-27T09:18:01.810 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.904+0000 7fa09aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa09c07ace0 0x7fa09c0791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:01.810 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.904+0000 7fa09aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa09c07ace0 0x7fa09c0791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54656/0 (socket says 172.21.15.119:54656) 2024-05-27T09:18:01.810 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.904+0000 7fa09aa00700 1 -- 172.21.15.119:0/159504450 learned_addr learned my addr 172.21.15.119:0/159504450 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:01.812 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.904+0000 7fa09aa00700 1 -- 172.21.15.119:0/159504450 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c079860 con 0x7fa09c07ace0 2024-05-27T09:18:01.813 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.905+0000 7fa09aa00700 1 --2- 172.21.15.119:0/159504450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa09c07ace0 0x7fa09c0791e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0098a0 tx=0x7fa08c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d5b21361e889ce server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:01.813 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.905+0000 7fa099600700 1 -- 172.21.15.119:0/159504450 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa08c004180 con 0x7fa09c07ace0 2024-05-27T09:18:01.813 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.905+0000 7fa099600700 1 -- 172.21.15.119:0/159504450 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa08c0042e0 con 0x7fa09c07ace0 2024-05-27T09:18:01.813 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.905+0000 7fa099600700 1 -- 172.21.15.119:0/159504450 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa08c011af0 con 0x7fa09c07ace0 2024-05-27T09:18:01.813 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.906+0000 7fa0a1800700 1 -- 172.21.15.119:0/159504450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa09c07ace0 msgr2=0x7fa09c0791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:01.813 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.906+0000 7fa0a1800700 1 --2- 172.21.15.119:0/159504450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa09c07ace0 0x7fa09c0791e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0098a0 tx=0x7fa08c009bb0 comp rx=0 tx=0).stop 2024-05-27T09:18:01.813 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.906+0000 7fa0a1800700 1 -- 172.21.15.119:0/159504450 shutdown_connections 2024-05-27T09:18:01.813 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.906+0000 7fa0a1800700 1 --2- 172.21.15.119:0/159504450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa09c07ace0 0x7fa09c0791e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:01.813 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.906+0000 7fa0a1800700 1 -- 172.21.15.119:0/159504450 >> 172.21.15.119:0/159504450 conn(0x7fa09c101160 msgr2=0x7fa09c103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:01.813 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.906+0000 7fa0a1800700 1 -- 172.21.15.119:0/159504450 shutdown_connections 2024-05-27T09:18:01.813 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.906+0000 7fa0a1800700 1 -- 172.21.15.119:0/159504450 wait complete. 2024-05-27T09:18:01.813 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.907+0000 7fa0a1800700 1 Processor -- start 2024-05-27T09:18:01.814 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.908+0000 7fa0a1800700 1 -- start start 2024-05-27T09:18:01.814 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.908+0000 7fa0a1800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa09c07ace0 0x7fa09c19baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:01.814 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.908+0000 7fa0a1800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c19c030 con 0x7fa09c07ace0 2024-05-27T09:18:01.814 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.908+0000 7fa09aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa09c07ace0 0x7fa09c19baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:01.814 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.908+0000 7fa09aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa09c07ace0 0x7fa09c19baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54668/0 (socket says 172.21.15.119:54668) 2024-05-27T09:18:01.814 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.908+0000 7fa09aa00700 1 -- 172.21.15.119:0/1193496170 learned_addr learned my addr 172.21.15.119:0/1193496170 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:01.814 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.909+0000 7fa09aa00700 1 -- 172.21.15.119:0/1193496170 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08c009550 con 0x7fa09c07ace0 2024-05-27T09:18:01.814 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.909+0000 7fa09aa00700 1 --2- 172.21.15.119:0/1193496170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa09c07ace0 0x7fa09c19baf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa08c000c00 tx=0x7fa08c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:01.817 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.909+0000 7fa093400700 1 -- 172.21.15.119:0/1193496170 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa08c003a70 con 0x7fa09c07ace0 2024-05-27T09:18:01.817 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.909+0000 7fa093400700 1 -- 172.21.15.119:0/1193496170 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa08c003bd0 con 0x7fa09c07ace0 2024-05-27T09:18:01.817 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.909+0000 7fa093400700 1 -- 172.21.15.119:0/1193496170 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa08c023700 con 0x7fa09c07ace0 2024-05-27T09:18:01.817 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.909+0000 7fa0a1800700 1 -- 172.21.15.119:0/1193496170 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09c19c230 con 0x7fa09c07ace0 2024-05-27T09:18:01.817 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.909+0000 7fa0a1800700 1 -- 172.21.15.119:0/1193496170 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09c19c650 con 0x7fa09c07ace0 2024-05-27T09:18:01.817 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.911+0000 7fa093400700 1 -- 172.21.15.119:0/1193496170 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fa08c01ab70 con 0x7fa09c07ace0 2024-05-27T09:18:01.817 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.911+0000 7fa0a1800700 1 -- 172.21.15.119:0/1193496170 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa09c10be40 con 0x7fa09c07ace0 2024-05-27T09:18:01.817 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.911+0000 7fa093400700 1 --2- 172.21.15.119:0/1193496170 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fa080038440 0x7fa08003a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:01.818 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.911+0000 7fa093400700 1 -- 172.21.15.119:0/1193496170 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa08c04c150 con 0x7fa09c07ace0 2024-05-27T09:18:01.818 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.912+0000 7fa09a000700 1 --2- 172.21.15.119:0/1193496170 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fa080038440 0x7fa08003a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:01.818 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.912+0000 7fa09a000700 1 --2- 172.21.15.119:0/1193496170 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fa080038440 0x7fa08003a8f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa088006fd0 tx=0x7fa088006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:01.818 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:00.917+0000 7fa093400700 1 -- 172.21.15.119:0/1193496170 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa08c01a430 con 0x7fa09c07ace0 2024-05-27T09:18:01.818 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:01.133+0000 7fa0a1800700 1 -- 172.21.15.119:0/1193496170 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fa09c10c000 con 0x7fa09c07ace0 2024-05-27T09:18:01.818 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:01.137+0000 7fa093400700 1 -- 172.21.15.119:0/1193496170 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fa08c02c350 con 0x7fa09c07ace0 2024-05-27T09:18:01.818 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:01.139+0000 7fa0a1800700 1 -- 172.21.15.119:0/1193496170 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fa080038440 msgr2=0x7fa08003a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:01.818 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:01.139+0000 7fa0a1800700 1 --2- 172.21.15.119:0/1193496170 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fa080038440 0x7fa08003a8f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa088006fd0 tx=0x7fa088006e40 comp rx=0 tx=0).stop 2024-05-27T09:18:01.818 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:01.139+0000 7fa0a1800700 1 -- 172.21.15.119:0/1193496170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa09c07ace0 msgr2=0x7fa09c19baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:01.818 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:01.139+0000 7fa0a1800700 1 --2- 172.21.15.119:0/1193496170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa09c07ace0 0x7fa09c19baf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa08c000c00 tx=0x7fa08c011b50 comp rx=0 tx=0).stop 2024-05-27T09:18:01.818 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:01.140+0000 7fa0a1800700 1 -- 172.21.15.119:0/1193496170 shutdown_connections 2024-05-27T09:18:01.818 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:01.140+0000 7fa0a1800700 1 --2- 172.21.15.119:0/1193496170 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7fa080038440 0x7fa08003a8f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:01.819 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:01.140+0000 7fa0a1800700 1 --2- 172.21.15.119:0/1193496170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa09c07ace0 0x7fa09c19baf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:01.819 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:01.140+0000 7fa0a1800700 1 -- 172.21.15.119:0/1193496170 >> 172.21.15.119:0/1193496170 conn(0x7fa09c101160 msgr2=0x7fa09c102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:01.819 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:01.140+0000 7fa0a1800700 1 -- 172.21.15.119:0/1193496170 shutdown_connections 2024-05-27T09:18:01.819 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:01.140+0000 7fa0a1800700 1 -- 172.21.15.119:0/1193496170 wait complete. 2024-05-27T09:18:01.819 INFO:teuthology.orchestra.run.smithi119.stdout:Enabling the dashboard module... 2024-05-27T09:18:02.390 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:02 smithi119 ceph-mon[29082]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:18:02.391 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:02 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/1193496170' entity='client.admin' 2024-05-27T09:18:03.442 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:03 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/4185782674' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-27T09:18:03.882 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.389+0000 7f0690800700 1 Processor -- start 2024-05-27T09:18:03.883 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.389+0000 7f0690800700 1 -- start start 2024-05-27T09:18:03.883 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.390+0000 7f0690800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f068807bdd0 0x7f068807a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:03.883 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.390+0000 7f0690800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068807a810 con 0x7f068807bdd0 2024-05-27T09:18:03.883 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.390+0000 7f068de00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f068807bdd0 0x7f068807a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:03.883 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.390+0000 7f068de00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f068807bdd0 0x7f068807a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54678/0 (socket says 172.21.15.119:54678) 2024-05-27T09:18:03.883 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.390+0000 7f068de00700 1 -- 172.21.15.119:0/3313814151 learned_addr learned my addr 172.21.15.119:0/3313814151 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:03.883 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.391+0000 7f068de00700 1 -- 172.21.15.119:0/3313814151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068807a950 con 0x7f068807bdd0 2024-05-27T09:18:03.883 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.391+0000 7f068de00700 1 --2- 172.21.15.119:0/3313814151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f068807bdd0 0x7f068807a2d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f067c0098a0 tx=0x7f067c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a693b17cba86bb8b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:03.883 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.391+0000 7f068ca00700 1 -- 172.21.15.119:0/3313814151 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f067c004180 con 0x7f068807bdd0 2024-05-27T09:18:03.883 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.392+0000 7f068ca00700 1 -- 172.21.15.119:0/3313814151 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f067c0042e0 con 0x7f068807bdd0 2024-05-27T09:18:03.883 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.392+0000 7f068ca00700 1 -- 172.21.15.119:0/3313814151 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f067c011af0 con 0x7f068807bdd0 2024-05-27T09:18:03.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.392+0000 7f0690800700 1 -- 172.21.15.119:0/3313814151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f068807bdd0 msgr2=0x7f068807a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:03.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.392+0000 7f0690800700 1 --2- 172.21.15.119:0/3313814151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f068807bdd0 0x7f068807a2d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f067c0098a0 tx=0x7f067c009bb0 comp rx=0 tx=0).stop 2024-05-27T09:18:03.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.392+0000 7f0690800700 1 -- 172.21.15.119:0/3313814151 shutdown_connections 2024-05-27T09:18:03.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.392+0000 7f0690800700 1 --2- 172.21.15.119:0/3313814151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f068807bdd0 0x7f068807a2d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:03.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.392+0000 7f0690800700 1 -- 172.21.15.119:0/3313814151 >> 172.21.15.119:0/3313814151 conn(0x7f0688101180 msgr2=0x7f0688103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:03.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.392+0000 7f0690800700 1 -- 172.21.15.119:0/3313814151 shutdown_connections 2024-05-27T09:18:03.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.393+0000 7f0690800700 1 -- 172.21.15.119:0/3313814151 wait complete. 2024-05-27T09:18:03.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.393+0000 7f0690800700 1 Processor -- start 2024-05-27T09:18:03.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.394+0000 7f0690800700 1 -- start start 2024-05-27T09:18:03.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.394+0000 7f0690800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f068807bdd0 0x7f068819baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:03.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.394+0000 7f0690800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068819bfe0 con 0x7f068807bdd0 2024-05-27T09:18:03.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.394+0000 7f068de00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f068807bdd0 0x7f068819baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:03.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.394+0000 7f068de00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f068807bdd0 0x7f068819baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54688/0 (socket says 172.21.15.119:54688) 2024-05-27T09:18:03.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.394+0000 7f068de00700 1 -- 172.21.15.119:0/4185782674 learned_addr learned my addr 172.21.15.119:0/4185782674 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:03.885 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.395+0000 7f068de00700 1 -- 172.21.15.119:0/4185782674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c009550 con 0x7f068807bdd0 2024-05-27T09:18:03.885 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.395+0000 7f068de00700 1 --2- 172.21.15.119:0/4185782674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f068807bdd0 0x7f068819baa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f067c000c00 tx=0x7f067c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:03.885 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.395+0000 7f0682a00700 1 -- 172.21.15.119:0/4185782674 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f067c003a70 con 0x7f068807bdd0 2024-05-27T09:18:03.885 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.395+0000 7f0682a00700 1 -- 172.21.15.119:0/4185782674 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f067c003bd0 con 0x7f068807bdd0 2024-05-27T09:18:03.885 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.395+0000 7f0682a00700 1 -- 172.21.15.119:0/4185782674 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f067c0238c0 con 0x7f068807bdd0 2024-05-27T09:18:03.885 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.395+0000 7f0690800700 1 -- 172.21.15.119:0/4185782674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068819c1e0 con 0x7f068807bdd0 2024-05-27T09:18:03.885 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.395+0000 7f0690800700 1 -- 172.21.15.119:0/4185782674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068819c600 con 0x7f068807bdd0 2024-05-27T09:18:03.885 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.396+0000 7f0682a00700 1 -- 172.21.15.119:0/4185782674 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f067c023b80 con 0x7f068807bdd0 2024-05-27T09:18:03.885 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.396+0000 7f0690800700 1 -- 172.21.15.119:0/4185782674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f068810be60 con 0x7f068807bdd0 2024-05-27T09:18:03.885 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.397+0000 7f0682a00700 1 --2- 172.21.15.119:0/4185782674 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f06700383f0 0x7f067003a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:03.885 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.397+0000 7f0682a00700 1 -- 172.21.15.119:0/4185782674 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f067c04c2d0 con 0x7f068807bdd0 2024-05-27T09:18:03.885 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.397+0000 7f068d400700 1 --2- 172.21.15.119:0/4185782674 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f06700383f0 0x7f067003a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:03.885 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.398+0000 7f068d400700 1 --2- 172.21.15.119:0/4185782674 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f06700383f0 0x7f067003a8a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0678006fd0 tx=0x7f0678006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:03.886 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.402+0000 7f0682a00700 1 -- 172.21.15.119:0/4185782674 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f067c0193c0 con 0x7f068807bdd0 2024-05-27T09:18:03.886 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:02.672+0000 7f0690800700 1 -- 172.21.15.119:0/4185782674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f068810c020 con 0x7f068807bdd0 2024-05-27T09:18:03.886 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:03.148+0000 7f0682a00700 1 -- 172.21.15.119:0/4185782674 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f067c019740 con 0x7f068807bdd0 2024-05-27T09:18:03.886 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:03.151+0000 7f0682a00700 1 -- 172.21.15.119:0/4185782674 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f067c02cbb0 con 0x7f068807bdd0 2024-05-27T09:18:03.886 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:03.154+0000 7f0690800700 1 -- 172.21.15.119:0/4185782674 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f06700383f0 msgr2=0x7f067003a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:03.886 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:03.154+0000 7f0690800700 1 --2- 172.21.15.119:0/4185782674 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f06700383f0 0x7f067003a8a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0678006fd0 tx=0x7f0678006e40 comp rx=0 tx=0).stop 2024-05-27T09:18:03.886 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:03.154+0000 7f0690800700 1 -- 172.21.15.119:0/4185782674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f068807bdd0 msgr2=0x7f068819baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:03.886 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:03.154+0000 7f0690800700 1 --2- 172.21.15.119:0/4185782674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f068807bdd0 0x7f068819baa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f067c000c00 tx=0x7f067c011b50 comp rx=0 tx=0).stop 2024-05-27T09:18:03.886 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:03.154+0000 7f0690800700 1 -- 172.21.15.119:0/4185782674 shutdown_connections 2024-05-27T09:18:03.886 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:03.154+0000 7f0690800700 1 --2- 172.21.15.119:0/4185782674 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f06700383f0 0x7f067003a8a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:03.886 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:03.154+0000 7f0690800700 1 --2- 172.21.15.119:0/4185782674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f068807bdd0 0x7f068819baa0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:03.886 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:03.154+0000 7f0690800700 1 -- 172.21.15.119:0/4185782674 >> 172.21.15.119:0/4185782674 conn(0x7f0688101180 msgr2=0x7f0688102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:03.886 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:03.154+0000 7f0690800700 1 -- 172.21.15.119:0/4185782674 shutdown_connections 2024-05-27T09:18:03.887 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:03.154+0000 7f0690800700 1 -- 172.21.15.119:0/4185782674 wait complete. 2024-05-27T09:18:04.306 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:04 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/4185782674' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-27T09:18:04.306 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:04 smithi119 ceph-mon[29082]: mgrmap e9: smithi119.ncwmog(active, since 32s) 2024-05-27T09:18:05.392 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:05 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/3349316665' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-27T09:18:05.403 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2024-05-27T09:18:05.409 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-27T09:18:05.409 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-27T09:18:05.409 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "active_name": "smithi119.ncwmog", 2024-05-27T09:18:05.409 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-27T09:18:05.409 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2024-05-27T09:18:05.409 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.459+0000 7f7235800700 1 Processor -- start 2024-05-27T09:18:05.409 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.459+0000 7f7235800700 1 -- start start 2024-05-27T09:18:05.409 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.459+0000 7f7235800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f723007ad00 0x7f7230079200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:05.409 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.459+0000 7f7235800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7230079740 con 0x7f723007ad00 2024-05-27T09:18:05.409 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.460+0000 7f722ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f723007ad00 0x7f7230079200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:05.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.460+0000 7f722ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f723007ad00 0x7f7230079200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54704/0 (socket says 172.21.15.119:54704) 2024-05-27T09:18:05.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.460+0000 7f722ea00700 1 -- 172.21.15.119:0/543492232 learned_addr learned my addr 172.21.15.119:0/543492232 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:05.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.460+0000 7f722ea00700 1 -- 172.21.15.119:0/543492232 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7230079880 con 0x7f723007ad00 2024-05-27T09:18:05.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.461+0000 7f722ea00700 1 --2- 172.21.15.119:0/543492232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f723007ad00 0x7f7230079200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f72200098a0 tx=0x7f7220009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c9870298590d3c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:05.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.461+0000 7f722d600700 1 -- 172.21.15.119:0/543492232 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7220004180 con 0x7f723007ad00 2024-05-27T09:18:05.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.461+0000 7f722d600700 1 -- 172.21.15.119:0/543492232 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f72200042e0 con 0x7f723007ad00 2024-05-27T09:18:05.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.461+0000 7f722d600700 1 -- 172.21.15.119:0/543492232 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7220011af0 con 0x7f723007ad00 2024-05-27T09:18:05.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.462+0000 7f7235800700 1 -- 172.21.15.119:0/543492232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f723007ad00 msgr2=0x7f7230079200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:05.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.462+0000 7f7235800700 1 --2- 172.21.15.119:0/543492232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f723007ad00 0x7f7230079200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f72200098a0 tx=0x7f7220009bb0 comp rx=0 tx=0).stop 2024-05-27T09:18:05.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.462+0000 7f7235800700 1 -- 172.21.15.119:0/543492232 shutdown_connections 2024-05-27T09:18:05.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.462+0000 7f7235800700 1 --2- 172.21.15.119:0/543492232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f723007ad00 0x7f7230079200 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:05.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.462+0000 7f7235800700 1 -- 172.21.15.119:0/543492232 >> 172.21.15.119:0/543492232 conn(0x7f7230101160 msgr2=0x7f7230103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:05.410 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.462+0000 7f7235800700 1 -- 172.21.15.119:0/543492232 shutdown_connections 2024-05-27T09:18:05.411 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.462+0000 7f7235800700 1 -- 172.21.15.119:0/543492232 wait complete. 2024-05-27T09:18:05.411 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.463+0000 7f7235800700 1 Processor -- start 2024-05-27T09:18:05.411 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.463+0000 7f7235800700 1 -- start start 2024-05-27T09:18:05.411 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.463+0000 7f7235800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f723007ad00 0x7f723019bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:05.411 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.463+0000 7f7235800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723019c010 con 0x7f723007ad00 2024-05-27T09:18:05.411 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.464+0000 7f722ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f723007ad00 0x7f723019bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:05.411 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.464+0000 7f722ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f723007ad00 0x7f723019bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54708/0 (socket says 172.21.15.119:54708) 2024-05-27T09:18:05.411 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.464+0000 7f722ea00700 1 -- 172.21.15.119:0/3349316665 learned_addr learned my addr 172.21.15.119:0/3349316665 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:05.411 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.464+0000 7f722ea00700 1 -- 172.21.15.119:0/3349316665 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7220009550 con 0x7f723007ad00 2024-05-27T09:18:05.411 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.464+0000 7f722ea00700 1 --2- 172.21.15.119:0/3349316665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f723007ad00 0x7f723019bad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7220000c00 tx=0x7f7220011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:05.411 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.465+0000 7f7227400700 1 -- 172.21.15.119:0/3349316665 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7220003a70 con 0x7f723007ad00 2024-05-27T09:18:05.411 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.465+0000 7f7235800700 1 -- 172.21.15.119:0/3349316665 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f723019c210 con 0x7f723007ad00 2024-05-27T09:18:05.411 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.465+0000 7f7227400700 1 -- 172.21.15.119:0/3349316665 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7220003bd0 con 0x7f723007ad00 2024-05-27T09:18:05.412 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.465+0000 7f7227400700 1 -- 172.21.15.119:0/3349316665 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f72200238c0 con 0x7f723007ad00 2024-05-27T09:18:05.412 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.465+0000 7f7235800700 1 -- 172.21.15.119:0/3349316665 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f723019c630 con 0x7f723007ad00 2024-05-27T09:18:05.412 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.466+0000 7f7227400700 1 -- 172.21.15.119:0/3349316665 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f7220023b80 con 0x7f723007ad00 2024-05-27T09:18:05.412 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.466+0000 7f7235800700 1 -- 172.21.15.119:0/3349316665 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7230062380 con 0x7f723007ad00 2024-05-27T09:18:05.412 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.466+0000 7f7227400700 1 --2- 172.21.15.119:0/3349316665 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f7214038440 0x7f721403a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:05.412 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.466+0000 7f7227400700 1 -- 172.21.15.119:0/3349316665 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f722004c320 con 0x7f723007ad00 2024-05-27T09:18:05.412 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.466+0000 7f722e000700 1 -- 172.21.15.119:0/3349316665 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f7214038440 msgr2=0x7f721403a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1553098156 2024-05-27T09:18:05.412 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.466+0000 7f722e000700 1 --2- 172.21.15.119:0/3349316665 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f7214038440 0x7f721403a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T09:18:05.412 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.471+0000 7f7227400700 1 -- 172.21.15.119:0/3349316665 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f722002b300 con 0x7f723007ad00 2024-05-27T09:18:05.412 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.667+0000 7f722e000700 1 -- 172.21.15.119:0/3349316665 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f7214038440 msgr2=0x7f721403a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1553098156 2024-05-27T09:18:05.412 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.667+0000 7f722e000700 1 --2- 172.21.15.119:0/3349316665 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f7214038440 0x7f721403a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T09:18:05.412 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.742+0000 7f7235800700 1 -- 172.21.15.119:0/3349316665 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f723019f7b0 con 0x7f723007ad00 2024-05-27T09:18:05.412 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.743+0000 7f7227400700 1 -- 172.21.15.119:0/3349316665 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f7220010b30 con 0x7f723007ad00 2024-05-27T09:18:05.413 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.746+0000 7f7235800700 1 -- 172.21.15.119:0/3349316665 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f7214038440 msgr2=0x7f721403a8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T09:18:05.413 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.746+0000 7f7235800700 1 --2- 172.21.15.119:0/3349316665 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f7214038440 0x7f721403a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:05.413 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.746+0000 7f7235800700 1 -- 172.21.15.119:0/3349316665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f723007ad00 msgr2=0x7f723019bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:05.413 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.746+0000 7f7235800700 1 --2- 172.21.15.119:0/3349316665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f723007ad00 0x7f723019bad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7220000c00 tx=0x7f7220011b50 comp rx=0 tx=0).stop 2024-05-27T09:18:05.413 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.746+0000 7f7235800700 1 -- 172.21.15.119:0/3349316665 shutdown_connections 2024-05-27T09:18:05.413 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.746+0000 7f7235800700 1 --2- 172.21.15.119:0/3349316665 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f7214038440 0x7f721403a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:05.413 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.746+0000 7f7235800700 1 --2- 172.21.15.119:0/3349316665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f723007ad00 0x7f723019bad0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:05.413 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.746+0000 7f7235800700 1 -- 172.21.15.119:0/3349316665 >> 172.21.15.119:0/3349316665 conn(0x7f7230101160 msgr2=0x7f7230102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:05.413 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.746+0000 7f7235800700 1 -- 172.21.15.119:0/3349316665 shutdown_connections 2024-05-27T09:18:05.413 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:04.746+0000 7f7235800700 1 -- 172.21.15.119:0/3349316665 wait complete. 2024-05-27T09:18:05.413 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for the mgr to restart... 2024-05-27T09:18:05.413 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mgr epoch 9... 2024-05-27T09:18:12.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:11 smithi119 ceph-mon[29082]: Active manager daemon smithi119.ncwmog restarted 2024-05-27T09:18:12.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:11 smithi119 ceph-mon[29082]: Activating manager daemon smithi119.ncwmog 2024-05-27T09:18:12.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:11 smithi119 ceph-mon[29082]: osdmap e3: 0 total, 0 up, 0 in 2024-05-27T09:18:12.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:11 smithi119 ceph-mon[29082]: mgrmap e10: smithi119.ncwmog(active, starting, since 0.00898033s) 2024-05-27T09:18:12.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:11 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi119"}]: dispatch 2024-05-27T09:18:12.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:11 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr metadata", "who": "smithi119.ncwmog", "id": "smithi119.ncwmog"}]: dispatch 2024-05-27T09:18:12.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:11 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T09:18:12.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:11 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T09:18:12.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:11 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T09:18:12.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:11 smithi119 ceph-mon[29082]: Manager daemon smithi119.ncwmog is now available 2024-05-27T09:18:13.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:12 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:18:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:12 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:18:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:12 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi119.ncwmog/mirror_snapshot_schedule"}]: dispatch 2024-05-27T09:18:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:12 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi119.ncwmog/trash_purge_schedule"}]: dispatch 2024-05-27T09:18:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:12 smithi119 ceph-mon[29082]: [27/May/2024:09:18:12] ENGINE Bus STARTING 2024-05-27T09:18:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:12 smithi119 ceph-mon[29082]: mgrmap e11: smithi119.ncwmog(active, since 1.01557s) 2024-05-27T09:18:14.085 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:13 smithi119 ceph-mon[29082]: [27/May/2024:09:18:12] ENGINE Serving on http://172.21.15.119:8765 2024-05-27T09:18:14.085 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:13 smithi119 ceph-mon[29082]: [27/May/2024:09:18:12] ENGINE Serving on https://172.21.15.119:7150 2024-05-27T09:18:14.085 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:13 smithi119 ceph-mon[29082]: [27/May/2024:09:18:12] ENGINE Bus STARTED 2024-05-27T09:18:14.085 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:13 smithi119 ceph-mon[29082]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-27T09:18:14.086 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:13 smithi119 ceph-mon[29082]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-27T09:18:14.086 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:13 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:14.104 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2024-05-27T09:18:14.104 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-27T09:18:14.104 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-27T09:18:14.104 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2024-05-27T09:18:14.105 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.949+0000 7f9488a00700 1 Processor -- start 2024-05-27T09:18:14.105 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.949+0000 7f9488a00700 1 -- start start 2024-05-27T09:18:14.105 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.950+0000 7f9488a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9484106650 0x7f9484108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:14.105 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.950+0000 7f9488a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94840744d0 con 0x7f9484106650 2024-05-27T09:18:14.105 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.950+0000 7f9481e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9484106650 0x7f9484108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:14.105 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.950+0000 7f9481e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9484106650 0x7f9484108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54714/0 (socket says 172.21.15.119:54714) 2024-05-27T09:18:14.106 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.950+0000 7f9481e00700 1 -- 172.21.15.119:0/3427610612 learned_addr learned my addr 172.21.15.119:0/3427610612 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:14.106 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.950+0000 7f9481e00700 1 -- 172.21.15.119:0/3427610612 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9484074610 con 0x7f9484106650 2024-05-27T09:18:14.106 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.951+0000 7f9481e00700 1 --2- 172.21.15.119:0/3427610612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9484106650 0x7f9484108a30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9474009e80 tx=0x7f947400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a40dbb02767207 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:14.106 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.951+0000 7f9481400700 1 -- 172.21.15.119:0/3427610612 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f947400e070 con 0x7f9484106650 2024-05-27T09:18:14.106 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.951+0000 7f9481400700 1 -- 172.21.15.119:0/3427610612 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f947400b8c0 con 0x7f9484106650 2024-05-27T09:18:14.106 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.951+0000 7f9481400700 1 -- 172.21.15.119:0/3427610612 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f94740044d0 con 0x7f9484106650 2024-05-27T09:18:14.107 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.952+0000 7f9488a00700 1 -- 172.21.15.119:0/3427610612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9484106650 msgr2=0x7f9484108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:14.107 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.952+0000 7f9488a00700 1 --2- 172.21.15.119:0/3427610612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9484106650 0x7f9484108a30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9474009e80 tx=0x7f947400b330 comp rx=0 tx=0).stop 2024-05-27T09:18:14.107 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.952+0000 7f9488a00700 1 -- 172.21.15.119:0/3427610612 shutdown_connections 2024-05-27T09:18:14.107 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.952+0000 7f9488a00700 1 --2- 172.21.15.119:0/3427610612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9484106650 0x7f9484108a30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:14.107 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.952+0000 7f9488a00700 1 -- 172.21.15.119:0/3427610612 >> 172.21.15.119:0/3427610612 conn(0x7f9484100070 msgr2=0x7f9484102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:14.107 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.952+0000 7f9488a00700 1 -- 172.21.15.119:0/3427610612 shutdown_connections 2024-05-27T09:18:14.107 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.952+0000 7f9488a00700 1 -- 172.21.15.119:0/3427610612 wait complete. 2024-05-27T09:18:14.107 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.953+0000 7f9488a00700 1 Processor -- start 2024-05-27T09:18:14.108 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.953+0000 7f9488a00700 1 -- start start 2024-05-27T09:18:14.108 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.954+0000 7f9488a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9484106650 0x7f948419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:14.108 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.954+0000 7f9488a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948419bee0 con 0x7f9484106650 2024-05-27T09:18:14.108 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.954+0000 7f9481e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9484106650 0x7f948419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:14.108 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.954+0000 7f9481e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9484106650 0x7f948419b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54720/0 (socket says 172.21.15.119:54720) 2024-05-27T09:18:14.108 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.954+0000 7f9481e00700 1 -- 172.21.15.119:0/3377044865 learned_addr learned my addr 172.21.15.119:0/3377044865 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:14.109 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.954+0000 7f9481e00700 1 -- 172.21.15.119:0/3377044865 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9474009b30 con 0x7f9484106650 2024-05-27T09:18:14.109 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.955+0000 7f9481e00700 1 --2- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9484106650 0x7f948419b9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9474004070 tx=0x7f94740049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:14.109 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.955+0000 7f947b400700 1 -- 172.21.15.119:0/3377044865 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f947400e040 con 0x7f9484106650 2024-05-27T09:18:14.109 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.955+0000 7f947b400700 1 -- 172.21.15.119:0/3377044865 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f947400be00 con 0x7f9484106650 2024-05-27T09:18:14.109 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.955+0000 7f9488a00700 1 -- 172.21.15.119:0/3377044865 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948419c0e0 con 0x7f9484106650 2024-05-27T09:18:14.109 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.955+0000 7f947b400700 1 -- 172.21.15.119:0/3377044865 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9474012770 con 0x7f9484106650 2024-05-27T09:18:14.109 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.955+0000 7f9488a00700 1 -- 172.21.15.119:0/3377044865 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948419c560 con 0x7f9484106650 2024-05-27T09:18:14.110 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.956+0000 7f947b400700 1 -- 172.21.15.119:0/3377044865 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f9474019040 con 0x7f9484106650 2024-05-27T09:18:14.110 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.957+0000 7f947b400700 1 --2- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f9468038440 0x7f946803a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:14.110 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.957+0000 7f947b400700 1 -- 172.21.15.119:0/3377044865 --> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f946803b000 con 0x7f9468038440 2024-05-27T09:18:14.110 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.957+0000 7f947b400700 1 -- 172.21.15.119:0/3377044865 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f947404bde0 con 0x7f9484106650 2024-05-27T09:18:14.111 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.957+0000 7f9479400700 1 -- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f9468038440 msgr2=0x7f946803a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1553098156 2024-05-27T09:18:14.111 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:05.957+0000 7f9479400700 1 --2- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f9468038440 0x7f946803a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T09:18:14.111 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:06.158+0000 7f9479400700 1 -- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f9468038440 msgr2=0x7f946803a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1553098156 2024-05-27T09:18:14.111 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:06.158+0000 7f9479400700 1 --2- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f9468038440 0x7f946803a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T09:18:14.111 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:06.559+0000 7f9479400700 1 -- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f9468038440 msgr2=0x7f946803a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1553098156 2024-05-27T09:18:14.111 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:06.559+0000 7f9479400700 1 --2- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f9468038440 0x7f946803a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-27T09:18:14.112 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:07.360+0000 7f9479400700 1 -- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f9468038440 msgr2=0x7f946803a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1553098156 2024-05-27T09:18:14.112 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:07.360+0000 7f9479400700 1 --2- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f9468038440 0x7f946803a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-27T09:18:14.112 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:08.962+0000 7f9479400700 1 -- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f9468038440 msgr2=0x7f946803a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/1553098156 2024-05-27T09:18:14.112 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:08.962+0000 7f9479400700 1 --2- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f9468038440 0x7f946803a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-27T09:18:14.112 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:11.831+0000 7f947b400700 1 -- 172.21.15.119:0/3377044865 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mgrmap(e 10) v1 ==== 44864+0+0 (secure 0 0 0) 0x7f947401bc00 con 0x7f9484106650 2024-05-27T09:18:14.112 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:11.831+0000 7f947b400700 1 -- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f9468038440 msgr2=0x7f946803a8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T09:18:14.112 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:11.831+0000 7f947b400700 1 --2- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f9468038440 0x7f946803a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:14.113 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:12.835+0000 7f947b400700 1 -- 172.21.15.119:0/3377044865 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 11) v1 ==== 44991+0+0 (secure 0 0 0) 0x7f9474017cd0 con 0x7f9484106650 2024-05-27T09:18:14.113 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:12.835+0000 7f947b400700 1 --2- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f946803bea0 0x7f946803e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:14.113 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:12.835+0000 7f947b400700 1 -- 172.21.15.119:0/3377044865 --> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f946803b000 con 0x7f946803bea0 2024-05-27T09:18:14.113 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:12.836+0000 7f9479400700 1 --2- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f946803bea0 0x7f946803e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:14.113 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:12.837+0000 7f9479400700 1 --2- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f946803bea0 0x7f946803e280 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f946c003a10 tx=0x7f946c0092b0 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:14.113 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:12.839+0000 7f947b400700 1 -- 172.21.15.119:0/3377044865 <== mgr.14164 v2:172.21.15.119:6800/2118907438 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f946803b000 con 0x7f946803bea0 2024-05-27T09:18:14.114 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:12.848+0000 7f9488a00700 1 -- 172.21.15.119:0/3377044865 --> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f9460002630 con 0x7f946803bea0 2024-05-27T09:18:14.114 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:12.849+0000 7f947b400700 1 -- 172.21.15.119:0/3377044865 <== mgr.14164 v2:172.21.15.119:6800/2118907438 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f9460002630 con 0x7f946803bea0 2024-05-27T09:18:14.114 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:12.849+0000 7f9488a00700 1 -- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f946803bea0 msgr2=0x7f946803e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:14.114 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:12.849+0000 7f9488a00700 1 --2- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f946803bea0 0x7f946803e280 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f946c003a10 tx=0x7f946c0092b0 comp rx=0 tx=0).stop 2024-05-27T09:18:14.114 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:12.849+0000 7f9488a00700 1 -- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9484106650 msgr2=0x7f948419b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:14.114 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:12.849+0000 7f9488a00700 1 --2- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9484106650 0x7f948419b9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9474004070 tx=0x7f94740049f0 comp rx=0 tx=0).stop 2024-05-27T09:18:14.114 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:12.849+0000 7f9488a00700 1 -- 172.21.15.119:0/3377044865 shutdown_connections 2024-05-27T09:18:14.115 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:12.849+0000 7f9488a00700 1 --2- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f946803bea0 0x7f946803e280 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:14.115 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:12.849+0000 7f9488a00700 1 --2- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9484106650 0x7f948419b9a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:14.115 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:12.849+0000 7f9488a00700 1 --2- 172.21.15.119:0/3377044865 >> [v2:172.21.15.119:6800/1553098156,v1:172.21.15.119:6801/1553098156] conn(0x7f9468038440 0x7f946803a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:14.115 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:12.849+0000 7f9488a00700 1 -- 172.21.15.119:0/3377044865 >> 172.21.15.119:0/3377044865 conn(0x7f9484100070 msgr2=0x7f9484100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:14.116 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:12.849+0000 7f9488a00700 1 -- 172.21.15.119:0/3377044865 shutdown_connections 2024-05-27T09:18:14.116 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:12.849+0000 7f9488a00700 1 -- 172.21.15.119:0/3377044865 wait complete. 2024-05-27T09:18:14.116 INFO:teuthology.orchestra.run.smithi119.stdout:mgr epoch 9 is available 2024-05-27T09:18:14.116 INFO:teuthology.orchestra.run.smithi119.stdout:Generating a dashboard self-signed certificate... 2024-05-27T09:18:14.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:14 smithi119 ceph-mon[29082]: mgrmap e12: smithi119.ncwmog(active, since 2s) 2024-05-27T09:18:15.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-27T09:18:15.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.649+0000 7fea87a00700 1 Processor -- start 2024-05-27T09:18:15.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.650+0000 7fea87a00700 1 -- start start 2024-05-27T09:18:15.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.650+0000 7fea87a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea80106610 0x7fea801089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:15.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.650+0000 7fea87a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea80074510 con 0x7fea80106610 2024-05-27T09:18:15.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.651+0000 7fea85000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea80106610 0x7fea801089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:15.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.651+0000 7fea85000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea80106610 0x7fea801089f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50088/0 (socket says 172.21.15.119:50088) 2024-05-27T09:18:15.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.651+0000 7fea85000700 1 -- 172.21.15.119:0/166001822 learned_addr learned my addr 172.21.15.119:0/166001822 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:15.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.651+0000 7fea85000700 1 -- 172.21.15.119:0/166001822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea80074650 con 0x7fea80106610 2024-05-27T09:18:15.722 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.651+0000 7fea85000700 1 --2- 172.21.15.119:0/166001822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea80106610 0x7fea801089f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fea740098a0 tx=0x7fea74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88641288a26b8c67 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:15.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.652+0000 7fea7f400700 1 -- 172.21.15.119:0/166001822 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fea74004180 con 0x7fea80106610 2024-05-27T09:18:15.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.652+0000 7fea7f400700 1 -- 172.21.15.119:0/166001822 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fea740042e0 con 0x7fea80106610 2024-05-27T09:18:15.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.652+0000 7fea7f400700 1 -- 172.21.15.119:0/166001822 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fea74011af0 con 0x7fea80106610 2024-05-27T09:18:15.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.653+0000 7fea87a00700 1 -- 172.21.15.119:0/166001822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea80106610 msgr2=0x7fea801089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:15.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.653+0000 7fea87a00700 1 --2- 172.21.15.119:0/166001822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea80106610 0x7fea801089f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fea740098a0 tx=0x7fea74009bb0 comp rx=0 tx=0).stop 2024-05-27T09:18:15.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.653+0000 7fea87a00700 1 -- 172.21.15.119:0/166001822 shutdown_connections 2024-05-27T09:18:15.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.653+0000 7fea87a00700 1 --2- 172.21.15.119:0/166001822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea80106610 0x7fea801089f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:15.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.653+0000 7fea87a00700 1 -- 172.21.15.119:0/166001822 >> 172.21.15.119:0/166001822 conn(0x7fea80100070 msgr2=0x7fea80102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:15.723 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.653+0000 7fea87a00700 1 -- 172.21.15.119:0/166001822 shutdown_connections 2024-05-27T09:18:15.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.653+0000 7fea87a00700 1 -- 172.21.15.119:0/166001822 wait complete. 2024-05-27T09:18:15.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.655+0000 7fea87a00700 1 Processor -- start 2024-05-27T09:18:15.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.655+0000 7fea87a00700 1 -- start start 2024-05-27T09:18:15.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.655+0000 7fea87a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea80106610 0x7fea80105f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:15.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.655+0000 7fea87a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea801064b0 con 0x7fea80106610 2024-05-27T09:18:15.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.655+0000 7fea85000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea80106610 0x7fea80105f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:15.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.655+0000 7fea85000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea80106610 0x7fea80105f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50092/0 (socket says 172.21.15.119:50092) 2024-05-27T09:18:15.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.655+0000 7fea85000700 1 -- 172.21.15.119:0/948473174 learned_addr learned my addr 172.21.15.119:0/948473174 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:15.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.656+0000 7fea85000700 1 -- 172.21.15.119:0/948473174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea74009550 con 0x7fea80106610 2024-05-27T09:18:15.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.656+0000 7fea85000700 1 --2- 172.21.15.119:0/948473174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea80106610 0x7fea80105f70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fea74006ba0 tx=0x7fea74011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:15.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.656+0000 7fea7d600700 1 -- 172.21.15.119:0/948473174 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fea74003b00 con 0x7fea80106610 2024-05-27T09:18:15.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.657+0000 7fea7d600700 1 -- 172.21.15.119:0/948473174 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fea7401a8e0 con 0x7fea80106610 2024-05-27T09:18:15.724 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.657+0000 7fea87a00700 1 -- 172.21.15.119:0/948473174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea801045c0 con 0x7fea80106610 2024-05-27T09:18:15.725 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.657+0000 7fea7d600700 1 -- 172.21.15.119:0/948473174 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fea74019990 con 0x7fea80106610 2024-05-27T09:18:15.725 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.657+0000 7fea87a00700 1 -- 172.21.15.119:0/948473174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea801049e0 con 0x7fea80106610 2024-05-27T09:18:15.725 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.658+0000 7fea7d600700 1 -- 172.21.15.119:0/948473174 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 12) v1 ==== 45097+0+0 (secure 0 0 0) 0x7fea74019c50 con 0x7fea80106610 2024-05-27T09:18:15.725 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.658+0000 7fea87a00700 1 -- 172.21.15.119:0/948473174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea80109bf0 con 0x7fea80106610 2024-05-27T09:18:15.725 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.659+0000 7fea7d600700 1 --2- 172.21.15.119:0/948473174 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fea680383f0 0x7fea6803a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:15.725 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.659+0000 7fea7d600700 1 -- 172.21.15.119:0/948473174 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fea7404c7c0 con 0x7fea80106610 2024-05-27T09:18:15.725 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.660+0000 7fea7fe00700 1 --2- 172.21.15.119:0/948473174 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fea680383f0 0x7fea6803a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:15.725 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.660+0000 7fea7fe00700 1 --2- 172.21.15.119:0/948473174 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fea680383f0 0x7fea6803a8a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fea6c006fd0 tx=0x7fea6c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:15.725 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.665+0000 7fea7d600700 1 -- 172.21.15.119:0/948473174 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fea7402b700 con 0x7fea80106610 2024-05-27T09:18:15.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:14.887+0000 7fea87a00700 1 -- 172.21.15.119:0/948473174 --> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fea80102390 con 0x7fea680383f0 2024-05-27T09:18:15.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:15.034+0000 7fea7d600700 1 -- 172.21.15.119:0/948473174 <== mgr.14164 v2:172.21.15.119:6800/2118907438 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fea80102390 con 0x7fea680383f0 2024-05-27T09:18:15.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:15.037+0000 7fea87a00700 1 -- 172.21.15.119:0/948473174 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fea680383f0 msgr2=0x7fea6803a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:15.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:15.037+0000 7fea87a00700 1 --2- 172.21.15.119:0/948473174 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fea680383f0 0x7fea6803a8a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fea6c006fd0 tx=0x7fea6c006e40 comp rx=0 tx=0).stop 2024-05-27T09:18:15.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:15.037+0000 7fea87a00700 1 -- 172.21.15.119:0/948473174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea80106610 msgr2=0x7fea80105f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:15.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:15.037+0000 7fea87a00700 1 --2- 172.21.15.119:0/948473174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea80106610 0x7fea80105f70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fea74006ba0 tx=0x7fea74011b50 comp rx=0 tx=0).stop 2024-05-27T09:18:15.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:15.037+0000 7fea87a00700 1 -- 172.21.15.119:0/948473174 shutdown_connections 2024-05-27T09:18:15.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:15.037+0000 7fea87a00700 1 --2- 172.21.15.119:0/948473174 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fea680383f0 0x7fea6803a8a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:15.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:15.037+0000 7fea87a00700 1 --2- 172.21.15.119:0/948473174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea80106610 0x7fea80105f70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:15.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:15.037+0000 7fea87a00700 1 -- 172.21.15.119:0/948473174 >> 172.21.15.119:0/948473174 conn(0x7fea80100070 msgr2=0x7fea80101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:15.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:15.038+0000 7fea87a00700 1 -- 172.21.15.119:0/948473174 shutdown_connections 2024-05-27T09:18:15.726 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:15.038+0000 7fea87a00700 1 -- 172.21.15.119:0/948473174 wait complete. 2024-05-27T09:18:15.727 INFO:teuthology.orchestra.run.smithi119.stdout:Creating initial admin user... 2024-05-27T09:18:16.310 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:16 smithi119 ceph-mon[29082]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:18:16.310 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:16 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:16.310 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:16 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:16.310 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:16 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:17.734 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Tlbsj5rh.XKBWt94svSgLeeSiCMGZZUG3gIyEcQLRF4CL6PG.Hid2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1716801496, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-27T09:18:17.735 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.309+0000 7f99c9a00700 1 Processor -- start 2024-05-27T09:18:17.735 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.309+0000 7f99c9a00700 1 -- start start 2024-05-27T09:18:17.735 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.309+0000 7f99c9a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c407ace0 0x7f99c40791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:17.735 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.309+0000 7f99c9a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c4079720 con 0x7f99c407ace0 2024-05-27T09:18:17.736 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.310+0000 7f99c2a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c407ace0 0x7f99c40791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:17.736 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.310+0000 7f99c2a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c407ace0 0x7f99c40791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50100/0 (socket says 172.21.15.119:50100) 2024-05-27T09:18:17.736 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.310+0000 7f99c2a00700 1 -- 172.21.15.119:0/313078121 learned_addr learned my addr 172.21.15.119:0/313078121 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:17.736 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.310+0000 7f99c2a00700 1 -- 172.21.15.119:0/313078121 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c4079860 con 0x7f99c407ace0 2024-05-27T09:18:17.736 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.311+0000 7f99c2a00700 1 --2- 172.21.15.119:0/313078121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c407ace0 0x7f99c40791e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f99b40098a0 tx=0x7f99b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f692d33e69b440 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:17.737 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.311+0000 7f99c1600700 1 -- 172.21.15.119:0/313078121 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99b4004180 con 0x7f99c407ace0 2024-05-27T09:18:17.737 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.311+0000 7f99c1600700 1 -- 172.21.15.119:0/313078121 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f99b40042e0 con 0x7f99c407ace0 2024-05-27T09:18:17.737 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.311+0000 7f99c1600700 1 -- 172.21.15.119:0/313078121 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99b4011af0 con 0x7f99c407ace0 2024-05-27T09:18:17.737 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.312+0000 7f99c9a00700 1 -- 172.21.15.119:0/313078121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c407ace0 msgr2=0x7f99c40791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:17.737 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.312+0000 7f99c9a00700 1 --2- 172.21.15.119:0/313078121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c407ace0 0x7f99c40791e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f99b40098a0 tx=0x7f99b4009bb0 comp rx=0 tx=0).stop 2024-05-27T09:18:17.738 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.312+0000 7f99c9a00700 1 -- 172.21.15.119:0/313078121 shutdown_connections 2024-05-27T09:18:17.738 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.312+0000 7f99c9a00700 1 --2- 172.21.15.119:0/313078121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c407ace0 0x7f99c40791e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:17.738 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.312+0000 7f99c9a00700 1 -- 172.21.15.119:0/313078121 >> 172.21.15.119:0/313078121 conn(0x7f99c4101160 msgr2=0x7f99c4103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:17.738 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.312+0000 7f99c9a00700 1 -- 172.21.15.119:0/313078121 shutdown_connections 2024-05-27T09:18:17.738 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.312+0000 7f99c9a00700 1 -- 172.21.15.119:0/313078121 wait complete. 2024-05-27T09:18:17.739 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.313+0000 7f99c9a00700 1 Processor -- start 2024-05-27T09:18:17.739 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.313+0000 7f99c9a00700 1 -- start start 2024-05-27T09:18:17.739 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.314+0000 7f99c9a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c407ace0 0x7f99c419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:17.739 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.314+0000 7f99c9a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c419c030 con 0x7f99c407ace0 2024-05-27T09:18:17.739 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.314+0000 7f99c2a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c407ace0 0x7f99c419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:17.739 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.314+0000 7f99c2a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c407ace0 0x7f99c419baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50102/0 (socket says 172.21.15.119:50102) 2024-05-27T09:18:17.740 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.314+0000 7f99c2a00700 1 -- 172.21.15.119:0/344932085 learned_addr learned my addr 172.21.15.119:0/344932085 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:17.740 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.314+0000 7f99c2a00700 1 -- 172.21.15.119:0/344932085 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b4009550 con 0x7f99c407ace0 2024-05-27T09:18:17.740 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.315+0000 7f99c2a00700 1 --2- 172.21.15.119:0/344932085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c407ace0 0x7f99c419baf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f99b4000c00 tx=0x7f99b4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:17.740 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.315+0000 7f99bb400700 1 -- 172.21.15.119:0/344932085 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99b4003a70 con 0x7f99c407ace0 2024-05-27T09:18:17.740 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.315+0000 7f99bb400700 1 -- 172.21.15.119:0/344932085 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f99b4003bd0 con 0x7f99c407ace0 2024-05-27T09:18:17.740 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.315+0000 7f99c9a00700 1 -- 172.21.15.119:0/344932085 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c419c230 con 0x7f99c407ace0 2024-05-27T09:18:17.741 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.315+0000 7f99bb400700 1 -- 172.21.15.119:0/344932085 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99b4023700 con 0x7f99c407ace0 2024-05-27T09:18:17.741 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.315+0000 7f99c9a00700 1 -- 172.21.15.119:0/344932085 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c419c650 con 0x7f99c407ace0 2024-05-27T09:18:17.741 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.316+0000 7f99c9a00700 1 -- 172.21.15.119:0/344932085 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c4062380 con 0x7f99c407ace0 2024-05-27T09:18:17.741 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.317+0000 7f99bb400700 1 -- 172.21.15.119:0/344932085 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 12) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f99b401a430 con 0x7f99c407ace0 2024-05-27T09:18:17.741 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.317+0000 7f99bb400700 1 --2- 172.21.15.119:0/344932085 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f99a8038490 0x7f99a803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:17.741 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.317+0000 7f99bb400700 1 -- 172.21.15.119:0/344932085 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f99b404c290 con 0x7f99c407ace0 2024-05-27T09:18:17.742 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.317+0000 7f99c2000700 1 --2- 172.21.15.119:0/344932085 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f99a8038490 0x7f99a803a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:17.742 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.318+0000 7f99c2000700 1 --2- 172.21.15.119:0/344932085 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f99a8038490 0x7f99a803a940 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f99b0006fd0 tx=0x7f99b0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:17.742 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.322+0000 7f99bb400700 1 -- 172.21.15.119:0/344932085 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f99b40193c0 con 0x7f99c407ace0 2024-05-27T09:18:17.742 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.566+0000 7f99c9a00700 1 -- 172.21.15.119:0/344932085 --> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f99c419f090 con 0x7f99a8038490 2024-05-27T09:18:17.742 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.831+0000 7f99bb400700 1 -- 172.21.15.119:0/344932085 <== mgr.14164 v2:172.21.15.119:6800/2118907438 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f99b0009e20 con 0x7f99a8038490 2024-05-27T09:18:17.742 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.834+0000 7f99c9a00700 1 -- 172.21.15.119:0/344932085 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f99a8038490 msgr2=0x7f99a803a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:17.743 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.834+0000 7f99c9a00700 1 --2- 172.21.15.119:0/344932085 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f99a8038490 0x7f99a803a940 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f99b0006fd0 tx=0x7f99b0006e40 comp rx=0 tx=0).stop 2024-05-27T09:18:17.743 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.834+0000 7f99c9a00700 1 -- 172.21.15.119:0/344932085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c407ace0 msgr2=0x7f99c419baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:17.743 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.834+0000 7f99c9a00700 1 --2- 172.21.15.119:0/344932085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c407ace0 0x7f99c419baf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f99b4000c00 tx=0x7f99b4011b50 comp rx=0 tx=0).stop 2024-05-27T09:18:17.743 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.834+0000 7f99c9a00700 1 -- 172.21.15.119:0/344932085 shutdown_connections 2024-05-27T09:18:17.743 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.834+0000 7f99c9a00700 1 --2- 172.21.15.119:0/344932085 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f99a8038490 0x7f99a803a940 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:17.743 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.834+0000 7f99c9a00700 1 --2- 172.21.15.119:0/344932085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c407ace0 0x7f99c419baf0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:17.744 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.834+0000 7f99c9a00700 1 -- 172.21.15.119:0/344932085 >> 172.21.15.119:0/344932085 conn(0x7f99c4101160 msgr2=0x7f99c4102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:17.744 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.834+0000 7f99c9a00700 1 -- 172.21.15.119:0/344932085 shutdown_connections 2024-05-27T09:18:17.744 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:16.834+0000 7f99c9a00700 1 -- 172.21.15.119:0/344932085 wait complete. 2024-05-27T09:18:17.744 INFO:teuthology.orchestra.run.smithi119.stdout:Fetching dashboard port number... 2024-05-27T09:18:17.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:17 smithi119 ceph-mon[29082]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:18:17.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:17 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:18.916 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:18 smithi119 ceph-mon[29082]: mgrmap e13: smithi119.ncwmog(active, since 6s) 2024-05-27T09:18:19.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 8443 2024-05-27T09:18:19.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.664+0000 7facbc800700 1 Processor -- start 2024-05-27T09:18:19.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.665+0000 7facbc800700 1 -- start start 2024-05-27T09:18:19.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.665+0000 7facbc800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facb407be10 0x7facb407a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:19.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.665+0000 7facbc800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb407a850 con 0x7facb407be10 2024-05-27T09:18:19.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.665+0000 7facb9e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facb407be10 0x7facb407a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:19.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.666+0000 7facb9e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facb407be10 0x7facb407a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50110/0 (socket says 172.21.15.119:50110) 2024-05-27T09:18:19.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.666+0000 7facb9e00700 1 -- 172.21.15.119:0/4278821055 learned_addr learned my addr 172.21.15.119:0/4278821055 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:19.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.666+0000 7facb9e00700 1 -- 172.21.15.119:0/4278821055 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb407a990 con 0x7facb407be10 2024-05-27T09:18:19.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.666+0000 7facb9e00700 1 --2- 172.21.15.119:0/4278821055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facb407be10 0x7facb407a310 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faca80098a0 tx=0x7faca8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3a022a03e0dd2a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:19.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.667+0000 7facb8a00700 1 -- 172.21.15.119:0/4278821055 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faca8004180 con 0x7facb407be10 2024-05-27T09:18:19.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.667+0000 7facb8a00700 1 -- 172.21.15.119:0/4278821055 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faca80042e0 con 0x7facb407be10 2024-05-27T09:18:19.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.667+0000 7facb8a00700 1 -- 172.21.15.119:0/4278821055 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faca8011af0 con 0x7facb407be10 2024-05-27T09:18:19.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.667+0000 7facbc800700 1 -- 172.21.15.119:0/4278821055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facb407be10 msgr2=0x7facb407a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:19.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.667+0000 7facbc800700 1 --2- 172.21.15.119:0/4278821055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facb407be10 0x7facb407a310 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faca80098a0 tx=0x7faca8009bb0 comp rx=0 tx=0).stop 2024-05-27T09:18:19.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.668+0000 7facbc800700 1 -- 172.21.15.119:0/4278821055 shutdown_connections 2024-05-27T09:18:19.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.668+0000 7facbc800700 1 --2- 172.21.15.119:0/4278821055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facb407be10 0x7facb407a310 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:19.478 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.668+0000 7facbc800700 1 -- 172.21.15.119:0/4278821055 >> 172.21.15.119:0/4278821055 conn(0x7facb4101180 msgr2=0x7facb4103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:19.479 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.668+0000 7facbc800700 1 -- 172.21.15.119:0/4278821055 shutdown_connections 2024-05-27T09:18:19.479 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.668+0000 7facbc800700 1 -- 172.21.15.119:0/4278821055 wait complete. 2024-05-27T09:18:19.479 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.669+0000 7facbc800700 1 Processor -- start 2024-05-27T09:18:19.479 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.669+0000 7facbc800700 1 -- start start 2024-05-27T09:18:19.479 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.669+0000 7facbc800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facb407be10 0x7facb419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:19.479 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.669+0000 7facbc800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb419c010 con 0x7facb407be10 2024-05-27T09:18:19.479 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.669+0000 7facb9e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facb407be10 0x7facb419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:19.479 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.670+0000 7facb9e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facb407be10 0x7facb419bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50112/0 (socket says 172.21.15.119:50112) 2024-05-27T09:18:19.480 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.670+0000 7facb9e00700 1 -- 172.21.15.119:0/3985183556 learned_addr learned my addr 172.21.15.119:0/3985183556 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:19.480 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.670+0000 7facb9e00700 1 -- 172.21.15.119:0/3985183556 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca8009550 con 0x7facb407be10 2024-05-27T09:18:19.480 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.670+0000 7facb9e00700 1 --2- 172.21.15.119:0/3985183556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facb407be10 0x7facb419bad0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faca8000c00 tx=0x7faca8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:19.480 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.670+0000 7facaea00700 1 -- 172.21.15.119:0/3985183556 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faca8003a70 con 0x7facb407be10 2024-05-27T09:18:19.480 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.670+0000 7facaea00700 1 -- 172.21.15.119:0/3985183556 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faca8003bd0 con 0x7facb407be10 2024-05-27T09:18:19.480 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.670+0000 7facbc800700 1 -- 172.21.15.119:0/3985183556 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facb419c210 con 0x7facb407be10 2024-05-27T09:18:19.480 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.670+0000 7facaea00700 1 -- 172.21.15.119:0/3985183556 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faca80238c0 con 0x7facb407be10 2024-05-27T09:18:19.480 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.671+0000 7facbc800700 1 -- 172.21.15.119:0/3985183556 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb419c630 con 0x7facb407be10 2024-05-27T09:18:19.480 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.672+0000 7facaea00700 1 -- 172.21.15.119:0/3985183556 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7faca8023b80 con 0x7facb407be10 2024-05-27T09:18:19.480 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.672+0000 7facbc800700 1 -- 172.21.15.119:0/3985183556 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facb4062380 con 0x7facb407be10 2024-05-27T09:18:19.480 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.672+0000 7facaea00700 1 --2- 172.21.15.119:0/3985183556 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fac9c0384e0 0x7fac9c03a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:19.480 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.672+0000 7facaea00700 1 -- 172.21.15.119:0/3985183556 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7faca804c500 con 0x7facb407be10 2024-05-27T09:18:19.480 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.672+0000 7facb9400700 1 --2- 172.21.15.119:0/3985183556 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fac9c0384e0 0x7fac9c03a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:19.481 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.673+0000 7facb9400700 1 --2- 172.21.15.119:0/3985183556 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fac9c0384e0 0x7fac9c03a990 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faca4006fd0 tx=0x7faca4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:19.481 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.677+0000 7facaea00700 1 -- 172.21.15.119:0/3985183556 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7faca8010970 con 0x7facb407be10 2024-05-27T09:18:19.481 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.894+0000 7facbc800700 1 -- 172.21.15.119:0/3985183556 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7facb419f520 con 0x7facb407be10 2024-05-27T09:18:19.481 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.894+0000 7facaea00700 1 -- 172.21.15.119:0/3985183556 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7faca802c350 con 0x7facb407be10 2024-05-27T09:18:19.481 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.898+0000 7facbc800700 1 -- 172.21.15.119:0/3985183556 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fac9c0384e0 msgr2=0x7fac9c03a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:19.481 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.898+0000 7facbc800700 1 --2- 172.21.15.119:0/3985183556 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fac9c0384e0 0x7fac9c03a990 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faca4006fd0 tx=0x7faca4006e40 comp rx=0 tx=0).stop 2024-05-27T09:18:19.481 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.898+0000 7facbc800700 1 -- 172.21.15.119:0/3985183556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facb407be10 msgr2=0x7facb419bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:19.481 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.898+0000 7facbc800700 1 --2- 172.21.15.119:0/3985183556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facb407be10 0x7facb419bad0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faca8000c00 tx=0x7faca8011b50 comp rx=0 tx=0).stop 2024-05-27T09:18:19.481 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.898+0000 7facbc800700 1 -- 172.21.15.119:0/3985183556 shutdown_connections 2024-05-27T09:18:19.481 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.898+0000 7facbc800700 1 --2- 172.21.15.119:0/3985183556 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fac9c0384e0 0x7fac9c03a990 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:19.481 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.898+0000 7facbc800700 1 --2- 172.21.15.119:0/3985183556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7facb407be10 0x7facb419bad0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:19.481 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.898+0000 7facbc800700 1 -- 172.21.15.119:0/3985183556 >> 172.21.15.119:0/3985183556 conn(0x7facb4101180 msgr2=0x7facb4102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:19.481 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.898+0000 7facbc800700 1 -- 172.21.15.119:0/3985183556 shutdown_connections 2024-05-27T09:18:19.481 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:18.898+0000 7facbc800700 1 -- 172.21.15.119:0/3985183556 wait complete. 2024-05-27T09:18:19.505 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T09:18:19.505 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout disabled 2024-05-27T09:18:19.527 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T09:18:19.527 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2024-05-27T09:18:19.527 INFO:teuthology.orchestra.run.smithi119.stdout:firewalld.service is not enabled 2024-05-27T09:18:19.528 INFO:teuthology.orchestra.run.smithi119.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-27T09:18:19.535 INFO:teuthology.orchestra.run.smithi119.stdout:Ceph Dashboard is now available at: 2024-05-27T09:18:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:18:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: URL: https://smithi119.front.sepia.ceph.com:8443/ 2024-05-27T09:18:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: User: admin 2024-05-27T09:18:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: Password: qiqi1brosz 2024-05-27T09:18:19.535 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:18:19.536 INFO:teuthology.orchestra.run.smithi119.stdout:Saving cluster configuration to /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config directory 2024-05-27T09:18:19.538 INFO:teuthology.orchestra.run.smithi119.stdout:Enabling autotune for osd_memory_target 2024-05-27T09:18:20.148 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:19 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/3985183556' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-27T09:18:21.072 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.147+0000 7fcc77600700 1 Processor -- start 2024-05-27T09:18:21.072 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.147+0000 7fcc77600700 1 -- start start 2024-05-27T09:18:21.073 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.148+0000 7fcc77600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc70105b70 0x7fcc70105f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:21.073 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.148+0000 7fcc77600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc701064c0 con 0x7fcc70105b70 2024-05-27T09:18:21.073 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.148+0000 7fcc74c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc70105b70 0x7fcc70105f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:21.073 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.148+0000 7fcc74c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc70105b70 0x7fcc70105f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38106/0 (socket says 172.21.15.119:38106) 2024-05-27T09:18:21.073 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.148+0000 7fcc74c00700 1 -- 172.21.15.119:0/1690744133 learned_addr learned my addr 172.21.15.119:0/1690744133 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:21.073 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.149+0000 7fcc74c00700 1 -- 172.21.15.119:0/1690744133 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc70106600 con 0x7fcc70105b70 2024-05-27T09:18:21.073 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.149+0000 7fcc74c00700 1 --2- 172.21.15.119:0/1690744133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc70105b70 0x7fcc70105f80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcc600098a0 tx=0x7fcc60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2f966bebbb1bbb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:21.073 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.149+0000 7fcc6fe00700 1 -- 172.21.15.119:0/1690744133 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc60004180 con 0x7fcc70105b70 2024-05-27T09:18:21.073 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.149+0000 7fcc6fe00700 1 -- 172.21.15.119:0/1690744133 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcc600042e0 con 0x7fcc70105b70 2024-05-27T09:18:21.073 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.149+0000 7fcc6fe00700 1 -- 172.21.15.119:0/1690744133 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc60011af0 con 0x7fcc70105b70 2024-05-27T09:18:21.074 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.150+0000 7fcc77600700 1 -- 172.21.15.119:0/1690744133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc70105b70 msgr2=0x7fcc70105f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:21.074 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.150+0000 7fcc77600700 1 --2- 172.21.15.119:0/1690744133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc70105b70 0x7fcc70105f80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcc600098a0 tx=0x7fcc60009bb0 comp rx=0 tx=0).stop 2024-05-27T09:18:21.074 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.150+0000 7fcc77600700 1 -- 172.21.15.119:0/1690744133 shutdown_connections 2024-05-27T09:18:21.074 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.150+0000 7fcc77600700 1 --2- 172.21.15.119:0/1690744133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc70105b70 0x7fcc70105f80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:21.074 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.150+0000 7fcc77600700 1 -- 172.21.15.119:0/1690744133 >> 172.21.15.119:0/1690744133 conn(0x7fcc701011a0 msgr2=0x7fcc701035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:21.074 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.150+0000 7fcc77600700 1 -- 172.21.15.119:0/1690744133 shutdown_connections 2024-05-27T09:18:21.074 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.150+0000 7fcc77600700 1 -- 172.21.15.119:0/1690744133 wait complete. 2024-05-27T09:18:21.074 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.151+0000 7fcc77600700 1 Processor -- start 2024-05-27T09:18:21.074 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.152+0000 7fcc77600700 1 -- start start 2024-05-27T09:18:21.074 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.152+0000 7fcc77600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc70105b70 0x7fcc70199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:21.074 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.152+0000 7fcc77600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc70199dc0 con 0x7fcc70105b70 2024-05-27T09:18:21.075 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.152+0000 7fcc74c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc70105b70 0x7fcc70199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:21.075 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.152+0000 7fcc74c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc70105b70 0x7fcc70199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38112/0 (socket says 172.21.15.119:38112) 2024-05-27T09:18:21.075 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.152+0000 7fcc74c00700 1 -- 172.21.15.119:0/2309811636 learned_addr learned my addr 172.21.15.119:0/2309811636 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:21.075 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.153+0000 7fcc74c00700 1 -- 172.21.15.119:0/2309811636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc60009550 con 0x7fcc70105b70 2024-05-27T09:18:21.075 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.153+0000 7fcc74c00700 1 --2- 172.21.15.119:0/2309811636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc70105b70 0x7fcc70199880 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcc60011ad0 tx=0x7fcc60011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:21.075 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.153+0000 7fcc6e000700 1 -- 172.21.15.119:0/2309811636 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc60003ab0 con 0x7fcc70105b70 2024-05-27T09:18:21.075 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.153+0000 7fcc6e000700 1 -- 172.21.15.119:0/2309811636 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcc60003c10 con 0x7fcc70105b70 2024-05-27T09:18:21.075 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.153+0000 7fcc77600700 1 -- 172.21.15.119:0/2309811636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc70199fc0 con 0x7fcc70105b70 2024-05-27T09:18:21.075 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.153+0000 7fcc6e000700 1 -- 172.21.15.119:0/2309811636 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc600198e0 con 0x7fcc70105b70 2024-05-27T09:18:21.075 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.153+0000 7fcc77600700 1 -- 172.21.15.119:0/2309811636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7019a3e0 con 0x7fcc70105b70 2024-05-27T09:18:21.075 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.154+0000 7fcc6e000700 1 -- 172.21.15.119:0/2309811636 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7fcc60019b50 con 0x7fcc70105b70 2024-05-27T09:18:21.075 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.154+0000 7fcc77600700 1 -- 172.21.15.119:0/2309811636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc70062380 con 0x7fcc70105b70 2024-05-27T09:18:21.075 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.155+0000 7fcc6e000700 1 --2- 172.21.15.119:0/2309811636 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fcc54038530 0x7fcc5403a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:21.076 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.155+0000 7fcc6e000700 1 -- 172.21.15.119:0/2309811636 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcc6004c410 con 0x7fcc70105b70 2024-05-27T09:18:21.076 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.155+0000 7fcc67e00700 1 --2- 172.21.15.119:0/2309811636 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fcc54038530 0x7fcc5403a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:21.076 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.156+0000 7fcc67e00700 1 --2- 172.21.15.119:0/2309811636 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fcc54038530 0x7fcc5403a9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c006fd0 tx=0x7fcc5c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:21.076 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.160+0000 7fcc6e000700 1 -- 172.21.15.119:0/2309811636 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fcc6001a430 con 0x7fcc70105b70 2024-05-27T09:18:21.076 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.375+0000 7fcc77600700 1 -- 172.21.15.119:0/2309811636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fcc7019d2b0 con 0x7fcc70105b70 2024-05-27T09:18:21.076 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.376+0000 7fcc6e000700 1 -- 172.21.15.119:0/2309811636 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fcc60018350 con 0x7fcc70105b70 2024-05-27T09:18:21.076 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.379+0000 7fcc77600700 1 -- 172.21.15.119:0/2309811636 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fcc54038530 msgr2=0x7fcc5403a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:21.076 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.379+0000 7fcc77600700 1 --2- 172.21.15.119:0/2309811636 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fcc54038530 0x7fcc5403a9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c006fd0 tx=0x7fcc5c006e40 comp rx=0 tx=0).stop 2024-05-27T09:18:21.076 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.379+0000 7fcc77600700 1 -- 172.21.15.119:0/2309811636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc70105b70 msgr2=0x7fcc70199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:21.076 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.379+0000 7fcc77600700 1 --2- 172.21.15.119:0/2309811636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc70105b70 0x7fcc70199880 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcc60011ad0 tx=0x7fcc60011bb0 comp rx=0 tx=0).stop 2024-05-27T09:18:21.076 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.379+0000 7fcc77600700 1 -- 172.21.15.119:0/2309811636 shutdown_connections 2024-05-27T09:18:21.076 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.379+0000 7fcc77600700 1 --2- 172.21.15.119:0/2309811636 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fcc54038530 0x7fcc5403a9e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:21.076 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.379+0000 7fcc77600700 1 --2- 172.21.15.119:0/2309811636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc70105b70 0x7fcc70199880 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:21.077 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.379+0000 7fcc77600700 1 -- 172.21.15.119:0/2309811636 >> 172.21.15.119:0/2309811636 conn(0x7fcc701011a0 msgr2=0x7fcc70101f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:21.077 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.379+0000 7fcc77600700 1 -- 172.21.15.119:0/2309811636 shutdown_connections 2024-05-27T09:18:21.077 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:20.379+0000 7fcc77600700 1 -- 172.21.15.119:0/2309811636 wait complete. 2024-05-27T09:18:22.737 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.661+0000 7f0d07800700 1 Processor -- start 2024-05-27T09:18:22.737 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.661+0000 7f0d07800700 1 -- start start 2024-05-27T09:18:22.737 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.661+0000 7f0d07800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d00105b10 0x7f0d00105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:22.737 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.661+0000 7f0d07800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d00106460 con 0x7f0d00105b10 2024-05-27T09:18:22.738 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.662+0000 7f0d04e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d00105b10 0x7f0d00105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:22.738 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.662+0000 7f0d04e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d00105b10 0x7f0d00105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38118/0 (socket says 172.21.15.119:38118) 2024-05-27T09:18:22.738 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.662+0000 7f0d04e00700 1 -- 172.21.15.119:0/3702324238 learned_addr learned my addr 172.21.15.119:0/3702324238 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:22.738 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.662+0000 7f0d04e00700 1 -- 172.21.15.119:0/3702324238 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d001065a0 con 0x7f0d00105b10 2024-05-27T09:18:22.738 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.662+0000 7f0d04e00700 1 --2- 172.21.15.119:0/3702324238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d00105b10 0x7f0d00105f20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40098a0 tx=0x7f0cf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3699cbee7bfd16f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:22.738 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.663+0000 7f0cff400700 1 -- 172.21.15.119:0/3702324238 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0cf4004180 con 0x7f0d00105b10 2024-05-27T09:18:22.738 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.663+0000 7f0cff400700 1 -- 172.21.15.119:0/3702324238 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0cf40042e0 con 0x7f0d00105b10 2024-05-27T09:18:22.738 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.663+0000 7f0cff400700 1 -- 172.21.15.119:0/3702324238 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0cf4011af0 con 0x7f0d00105b10 2024-05-27T09:18:22.738 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.663+0000 7f0d07800700 1 -- 172.21.15.119:0/3702324238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d00105b10 msgr2=0x7f0d00105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:22.738 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.663+0000 7f0d07800700 1 --2- 172.21.15.119:0/3702324238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d00105b10 0x7f0d00105f20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40098a0 tx=0x7f0cf4009bb0 comp rx=0 tx=0).stop 2024-05-27T09:18:22.738 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.664+0000 7f0d07800700 1 -- 172.21.15.119:0/3702324238 shutdown_connections 2024-05-27T09:18:22.739 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.664+0000 7f0d07800700 1 --2- 172.21.15.119:0/3702324238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d00105b10 0x7f0d00105f20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:22.739 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.664+0000 7f0d07800700 1 -- 172.21.15.119:0/3702324238 >> 172.21.15.119:0/3702324238 conn(0x7f0d00101160 msgr2=0x7f0d00103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:22.739 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.664+0000 7f0d07800700 1 -- 172.21.15.119:0/3702324238 shutdown_connections 2024-05-27T09:18:22.739 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.664+0000 7f0d07800700 1 -- 172.21.15.119:0/3702324238 wait complete. 2024-05-27T09:18:22.739 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.665+0000 7f0d07800700 1 Processor -- start 2024-05-27T09:18:22.739 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.665+0000 7f0d07800700 1 -- start start 2024-05-27T09:18:22.739 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.665+0000 7f0d07800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d00105b10 0x7f0d0019ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:22.739 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.665+0000 7f0d07800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0019bfb0 con 0x7f0d00105b10 2024-05-27T09:18:22.739 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.665+0000 7f0d04e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d00105b10 0x7f0d0019ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:22.739 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.665+0000 7f0d04e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d00105b10 0x7f0d0019ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38122/0 (socket says 172.21.15.119:38122) 2024-05-27T09:18:22.739 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.665+0000 7f0d04e00700 1 -- 172.21.15.119:0/3033991395 learned_addr learned my addr 172.21.15.119:0/3033991395 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:22.739 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.666+0000 7f0d04e00700 1 -- 172.21.15.119:0/3033991395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf4009550 con 0x7f0d00105b10 2024-05-27T09:18:22.740 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.666+0000 7f0d04e00700 1 --2- 172.21.15.119:0/3033991395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d00105b10 0x7f0d0019ba70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4006ba0 tx=0x7f0cf4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:22.740 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.666+0000 7f0cfd600700 1 -- 172.21.15.119:0/3033991395 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0cf4003b00 con 0x7f0d00105b10 2024-05-27T09:18:22.740 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.666+0000 7f0cfd600700 1 -- 172.21.15.119:0/3033991395 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0cf401a8e0 con 0x7f0d00105b10 2024-05-27T09:18:22.740 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.666+0000 7f0cfd600700 1 -- 172.21.15.119:0/3033991395 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0cf40199c0 con 0x7f0d00105b10 2024-05-27T09:18:22.740 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.666+0000 7f0d07800700 1 -- 172.21.15.119:0/3033991395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d0019c1b0 con 0x7f0d00105b10 2024-05-27T09:18:22.740 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.667+0000 7f0d07800700 1 -- 172.21.15.119:0/3033991395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0019c5d0 con 0x7f0d00105b10 2024-05-27T09:18:22.740 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.667+0000 7f0cfd600700 1 -- 172.21.15.119:0/3033991395 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f0cf4019c30 con 0x7f0d00105b10 2024-05-27T09:18:22.740 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.668+0000 7f0d07800700 1 -- 172.21.15.119:0/3033991395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d00062380 con 0x7f0d00105b10 2024-05-27T09:18:22.740 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.668+0000 7f0cfd600700 1 --2- 172.21.15.119:0/3033991395 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f0ce80384e0 0x7f0ce803a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:22.740 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.668+0000 7f0cfd600700 1 -- 172.21.15.119:0/3033991395 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0cf404c830 con 0x7f0d00105b10 2024-05-27T09:18:22.740 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.668+0000 7f0cffe00700 1 --2- 172.21.15.119:0/3033991395 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f0ce80384e0 0x7f0ce803a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:22.740 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.669+0000 7f0cffe00700 1 --2- 172.21.15.119:0/3033991395 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f0ce80384e0 0x7f0ce803a990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0cec006fd0 tx=0x7f0cec006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:22.740 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:21.673+0000 7f0cfd600700 1 -- 172.21.15.119:0/3033991395 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0cf401aa50 con 0x7f0d00105b10 2024-05-27T09:18:22.741 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:22.000+0000 7f0d07800700 1 -- 172.21.15.119:0/3033991395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f0d0019f430 con 0x7f0d00105b10 2024-05-27T09:18:22.741 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:22.006+0000 7f0cfd600700 1 -- 172.21.15.119:0/3033991395 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7f0cf4018350 con 0x7f0d00105b10 2024-05-27T09:18:22.741 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:22.009+0000 7f0d07800700 1 -- 172.21.15.119:0/3033991395 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f0ce80384e0 msgr2=0x7f0ce803a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:22.741 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:22.009+0000 7f0d07800700 1 --2- 172.21.15.119:0/3033991395 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f0ce80384e0 0x7f0ce803a990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0cec006fd0 tx=0x7f0cec006e40 comp rx=0 tx=0).stop 2024-05-27T09:18:22.741 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:22.009+0000 7f0d07800700 1 -- 172.21.15.119:0/3033991395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d00105b10 msgr2=0x7f0d0019ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:22.741 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:22.009+0000 7f0d07800700 1 --2- 172.21.15.119:0/3033991395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d00105b10 0x7f0d0019ba70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4006ba0 tx=0x7f0cf4011b50 comp rx=0 tx=0).stop 2024-05-27T09:18:22.741 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:22.009+0000 7f0d07800700 1 -- 172.21.15.119:0/3033991395 shutdown_connections 2024-05-27T09:18:22.741 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:22.009+0000 7f0d07800700 1 --2- 172.21.15.119:0/3033991395 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f0ce80384e0 0x7f0ce803a990 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:22.741 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:22.009+0000 7f0d07800700 1 --2- 172.21.15.119:0/3033991395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d00105b10 0x7f0d0019ba70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:22.741 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:22.009+0000 7f0d07800700 1 -- 172.21.15.119:0/3033991395 >> 172.21.15.119:0/3033991395 conn(0x7f0d00101160 msgr2=0x7f0d00101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:22.741 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:22.009+0000 7f0d07800700 1 -- 172.21.15.119:0/3033991395 shutdown_connections 2024-05-27T09:18:22.741 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2024-05-27T09:18:22.009+0000 7f0d07800700 1 -- 172.21.15.119:0/3033991395 wait complete. 2024-05-27T09:18:22.742 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-27T09:18:22.742 INFO:teuthology.orchestra.run.smithi119.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-27T09:18:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:18:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-27T09:18:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:18:22.742 INFO:teuthology.orchestra.run.smithi119.stdout:Or, if you are only running a single cluster on this host: 2024-05-27T09:18:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:18:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-27T09:18:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:18:22.742 INFO:teuthology.orchestra.run.smithi119.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-27T09:18:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:18:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: ceph telemetry on 2024-05-27T09:18:22.742 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:18:22.743 INFO:teuthology.orchestra.run.smithi119.stdout:For more information see: 2024-05-27T09:18:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:18:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-27T09:18:22.743 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:18:22.743 INFO:teuthology.orchestra.run.smithi119.stdout:Bootstrap complete. 2024-05-27T09:18:22.811 INFO:tasks.cephadm:Fetching config... 2024-05-27T09:18:22.811 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-27T09:18:22.811 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-27T09:18:22.871 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-27T09:18:22.872 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-27T09:18:22.872 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-27T09:18:22.914 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-27T09:18:22.915 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-27T09:18:22.915 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/keyring of=/dev/stdout 2024-05-27T09:18:23.024 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-27T09:18:23.024 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-27T09:18:23.025 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-27T09:18:23.119 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-27T09:18:23.119 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDO7MhqggAUNaILv17jDDnNMfX9/7VgW7sAxuzPWPIocS16LhcM7fG2NND/RFGy0NzJCJN8LFpZ1rKc0FkChEUugUu2B+Gv7VkDZBjlHYhZgteTHDZYHUFOdYtjgfbjZi7qFLQDosu/m+kUTIuN8cWw0WIOSX9QqmTyqpGYu3GoahgqaVrHMy7HV4GcTVEsMzU9g0sKlcRcklK4xIp0HFBamSXwM8mitDsVsVjy463crmSNh4TY5usla10D+nF5cc+dQtlSi0pRKjk5xW/o4/wHpF4gTegsUzxPA2YhOUC3E22nXHrjrwl51aq8Vd5wJo4O9FZk1WPjbFjgJzYgjs2LKP5dbonREqrZ1x/iU28ly6Pjdia9RNlksk6XqvYTypc+Vryr/4V6BeDfXsgsRursR+h7cWxbtHLwM87X8/2KRCwNQdb/OSdX6DbDZF7RLK85AhSp6lM1WA5hUUAkTWKORjf2lXuvxz2T/YU5QXST7+5xFgmIiDgb+lg6pPFuf/0= ceph-b966b06e-1c09-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-27T09:18:23.219 INFO:teuthology.orchestra.run.smithi119.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDO7MhqggAUNaILv17jDDnNMfX9/7VgW7sAxuzPWPIocS16LhcM7fG2NND/RFGy0NzJCJN8LFpZ1rKc0FkChEUugUu2B+Gv7VkDZBjlHYhZgteTHDZYHUFOdYtjgfbjZi7qFLQDosu/m+kUTIuN8cWw0WIOSX9QqmTyqpGYu3GoahgqaVrHMy7HV4GcTVEsMzU9g0sKlcRcklK4xIp0HFBamSXwM8mitDsVsVjy463crmSNh4TY5usla10D+nF5cc+dQtlSi0pRKjk5xW/o4/wHpF4gTegsUzxPA2YhOUC3E22nXHrjrwl51aq8Vd5wJo4O9FZk1WPjbFjgJzYgjs2LKP5dbonREqrZ1x/iU28ly6Pjdia9RNlksk6XqvYTypc+Vryr/4V6BeDfXsgsRursR+h7cWxbtHLwM87X8/2KRCwNQdb/OSdX6DbDZF7RLK85AhSp6lM1WA5hUUAkTWKORjf2lXuvxz2T/YU5QXST7+5xFgmIiDgb+lg6pPFuf/0= ceph-b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:18:23.246 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDO7MhqggAUNaILv17jDDnNMfX9/7VgW7sAxuzPWPIocS16LhcM7fG2NND/RFGy0NzJCJN8LFpZ1rKc0FkChEUugUu2B+Gv7VkDZBjlHYhZgteTHDZYHUFOdYtjgfbjZi7qFLQDosu/m+kUTIuN8cWw0WIOSX9QqmTyqpGYu3GoahgqaVrHMy7HV4GcTVEsMzU9g0sKlcRcklK4xIp0HFBamSXwM8mitDsVsVjy463crmSNh4TY5usla10D+nF5cc+dQtlSi0pRKjk5xW/o4/wHpF4gTegsUzxPA2YhOUC3E22nXHrjrwl51aq8Vd5wJo4O9FZk1WPjbFjgJzYgjs2LKP5dbonREqrZ1x/iU28ly6Pjdia9RNlksk6XqvYTypc+Vryr/4V6BeDfXsgsRursR+h7cWxbtHLwM87X8/2KRCwNQdb/OSdX6DbDZF7RLK85AhSp6lM1WA5hUUAkTWKORjf2lXuvxz2T/YU5QXST7+5xFgmIiDgb+lg6pPFuf/0= ceph-b966b06e-1c09-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-27T09:18:23.320 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:23 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/3033991395' entity='client.admin' 2024-05-27T09:18:23.363 INFO:teuthology.orchestra.run.smithi138.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDO7MhqggAUNaILv17jDDnNMfX9/7VgW7sAxuzPWPIocS16LhcM7fG2NND/RFGy0NzJCJN8LFpZ1rKc0FkChEUugUu2B+Gv7VkDZBjlHYhZgteTHDZYHUFOdYtjgfbjZi7qFLQDosu/m+kUTIuN8cWw0WIOSX9QqmTyqpGYu3GoahgqaVrHMy7HV4GcTVEsMzU9g0sKlcRcklK4xIp0HFBamSXwM8mitDsVsVjy463crmSNh4TY5usla10D+nF5cc+dQtlSi0pRKjk5xW/o4/wHpF4gTegsUzxPA2YhOUC3E22nXHrjrwl51aq8Vd5wJo4O9FZk1WPjbFjgJzYgjs2LKP5dbonREqrZ1x/iU28ly6Pjdia9RNlksk6XqvYTypc+Vryr/4V6BeDfXsgsRursR+h7cWxbtHLwM87X8/2KRCwNQdb/OSdX6DbDZF7RLK85AhSp6lM1WA5hUUAkTWKORjf2lXuvxz2T/YU5QXST7+5xFgmIiDgb+lg6pPFuf/0= ceph-b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:18:23.412 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-27T09:18:23.774 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:18:26.326 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:26 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:26.326 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:26 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:26.326 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:26 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd/host:smithi119", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:18:26.326 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:26 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:26.327 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:26 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi119", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-27T09:18:26.327 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:26 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi119", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-27T09:18:26.327 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:26 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:18:26.327 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:26 smithi119 ceph-mon[29082]: Deploying daemon ceph-exporter.smithi119 on smithi119 2024-05-27T09:18:26.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.338+0000 7f3d3c600700 1 -- 172.21.15.119:0/1347750724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d340fe6b0 msgr2=0x7f3d340feac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:26.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.338+0000 7f3d3c600700 1 --2- 172.21.15.119:0/1347750724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d340fe6b0 0x7f3d340feac0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3d28009910 tx=0x7f3d28009c20 comp rx=0 tx=0).stop 2024-05-27T09:18:26.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.338+0000 7f3d3c600700 1 -- 172.21.15.119:0/1347750724 shutdown_connections 2024-05-27T09:18:26.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.338+0000 7f3d3c600700 1 --2- 172.21.15.119:0/1347750724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d340fe6b0 0x7f3d340feac0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:26.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.338+0000 7f3d3c600700 1 -- 172.21.15.119:0/1347750724 >> 172.21.15.119:0/1347750724 conn(0x7f3d340fa2e0 msgr2=0x7f3d340fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:26.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.338+0000 7f3d3c600700 1 -- 172.21.15.119:0/1347750724 shutdown_connections 2024-05-27T09:18:26.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.338+0000 7f3d3c600700 1 -- 172.21.15.119:0/1347750724 wait complete. 2024-05-27T09:18:26.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.339+0000 7f3d3c600700 1 Processor -- start 2024-05-27T09:18:26.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.339+0000 7f3d3c600700 1 -- start start 2024-05-27T09:18:26.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.340+0000 7f3d3c600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d340fe6b0 0x7f3d341971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:26.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.340+0000 7f3d3c600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d341976e0 con 0x7f3d340fe6b0 2024-05-27T09:18:26.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.340+0000 7f3d39c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d340fe6b0 0x7f3d341971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:26.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.340+0000 7f3d39c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d340fe6b0 0x7f3d341971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38140/0 (socket says 172.21.15.119:38140) 2024-05-27T09:18:26.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.340+0000 7f3d39c00700 1 -- 172.21.15.119:0/3822077096 learned_addr learned my addr 172.21.15.119:0/3822077096 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:26.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.340+0000 7f3d39c00700 1 -- 172.21.15.119:0/3822077096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d280095f0 con 0x7f3d340fe6b0 2024-05-27T09:18:26.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.341+0000 7f3d39c00700 1 --2- 172.21.15.119:0/3822077096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d340fe6b0 0x7f3d341971a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3d280092e0 tx=0x7f3d28009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:26.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.341+0000 7f3d32000700 1 -- 172.21.15.119:0/3822077096 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3d2801d070 con 0x7f3d340fe6b0 2024-05-27T09:18:26.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.341+0000 7f3d32000700 1 -- 172.21.15.119:0/3822077096 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3d2800be10 con 0x7f3d340fe6b0 2024-05-27T09:18:26.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.341+0000 7f3d32000700 1 -- 172.21.15.119:0/3822077096 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3d2800f460 con 0x7f3d340fe6b0 2024-05-27T09:18:26.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.341+0000 7f3d3c600700 1 -- 172.21.15.119:0/3822077096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d341978e0 con 0x7f3d340fe6b0 2024-05-27T09:18:26.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.341+0000 7f3d3c600700 1 -- 172.21.15.119:0/3822077096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d34197d80 con 0x7f3d340fe6b0 2024-05-27T09:18:26.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.343+0000 7f3d32000700 1 -- 172.21.15.119:0/3822077096 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f3d28022470 con 0x7f3d340fe6b0 2024-05-27T09:18:26.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.343+0000 7f3d3c600700 1 -- 172.21.15.119:0/3822077096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d34190d00 con 0x7f3d340fe6b0 2024-05-27T09:18:26.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.343+0000 7f3d32000700 1 --2- 172.21.15.119:0/3822077096 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f3d1c0384e0 0x7f3d1c03a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:26.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.343+0000 7f3d32000700 1 -- 172.21.15.119:0/3822077096 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3d2804c510 con 0x7f3d340fe6b0 2024-05-27T09:18:26.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.344+0000 7f3d39200700 1 --2- 172.21.15.119:0/3822077096 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f3d1c0384e0 0x7f3d1c03a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:26.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.344+0000 7f3d39200700 1 --2- 172.21.15.119:0/3822077096 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f3d1c0384e0 0x7f3d1c03a990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3d20006fd0 tx=0x7f3d20006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:26.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.349+0000 7f3d32000700 1 -- 172.21.15.119:0/3822077096 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3d2800b960 con 0x7f3d340fe6b0 2024-05-27T09:18:26.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.566+0000 7f3d3c600700 1 -- 172.21.15.119:0/3822077096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f3d3402cc30 con 0x7f3d340fe6b0 2024-05-27T09:18:26.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.577+0000 7f3d32000700 1 -- 172.21.15.119:0/3822077096 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f3d28027070 con 0x7f3d340fe6b0 2024-05-27T09:18:26.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.580+0000 7f3d3c600700 1 -- 172.21.15.119:0/3822077096 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f3d1c0384e0 msgr2=0x7f3d1c03a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:26.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.580+0000 7f3d3c600700 1 --2- 172.21.15.119:0/3822077096 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f3d1c0384e0 0x7f3d1c03a990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3d20006fd0 tx=0x7f3d20006e40 comp rx=0 tx=0).stop 2024-05-27T09:18:26.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.580+0000 7f3d3c600700 1 -- 172.21.15.119:0/3822077096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d340fe6b0 msgr2=0x7f3d341971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:26.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.580+0000 7f3d3c600700 1 --2- 172.21.15.119:0/3822077096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d340fe6b0 0x7f3d341971a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3d280092e0 tx=0x7f3d28009e60 comp rx=0 tx=0).stop 2024-05-27T09:18:26.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.581+0000 7f3d3c600700 1 -- 172.21.15.119:0/3822077096 shutdown_connections 2024-05-27T09:18:26.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.581+0000 7f3d3c600700 1 --2- 172.21.15.119:0/3822077096 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f3d1c0384e0 0x7f3d1c03a990 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:26.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.581+0000 7f3d3c600700 1 --2- 172.21.15.119:0/3822077096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d340fe6b0 0x7f3d341971a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:26.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.581+0000 7f3d3c600700 1 -- 172.21.15.119:0/3822077096 >> 172.21.15.119:0/3822077096 conn(0x7f3d340fa2e0 msgr2=0x7f3d340faf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:26.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.582+0000 7f3d3c600700 1 -- 172.21.15.119:0/3822077096 shutdown_connections 2024-05-27T09:18:26.583 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:26.582+0000 7f3d3c600700 1 -- 172.21.15.119:0/3822077096 wait complete. 2024-05-27T09:18:27.589 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-27T09:18:27.590 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-27T09:18:27.917 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:27 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/3822077096' entity='client.admin' 2024-05-27T09:18:28.061 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:18:30.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.825+0000 7f4fdd800700 1 -- 172.21.15.119:0/1560153135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd8074d80 msgr2=0x7f4fd80731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:30.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.825+0000 7f4fdd800700 1 --2- 172.21.15.119:0/1560153135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd8074d80 0x7f4fd80731e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009910 tx=0x7f4fc8009c20 comp rx=0 tx=0).stop 2024-05-27T09:18:30.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.825+0000 7f4fdd800700 1 -- 172.21.15.119:0/1560153135 shutdown_connections 2024-05-27T09:18:30.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.825+0000 7f4fdd800700 1 --2- 172.21.15.119:0/1560153135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd8074d80 0x7f4fd80731e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:30.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.825+0000 7f4fdd800700 1 -- 172.21.15.119:0/1560153135 >> 172.21.15.119:0/1560153135 conn(0x7f4fd80fb3d0 msgr2=0x7f4fd80fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:30.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.826+0000 7f4fdd800700 1 -- 172.21.15.119:0/1560153135 shutdown_connections 2024-05-27T09:18:30.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.826+0000 7f4fdd800700 1 -- 172.21.15.119:0/1560153135 wait complete. 2024-05-27T09:18:30.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.827+0000 7f4fdd800700 1 Processor -- start 2024-05-27T09:18:30.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.827+0000 7f4fdd800700 1 -- start start 2024-05-27T09:18:30.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.827+0000 7f4fdd800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd8074d80 0x7f4fd819b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:30.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.827+0000 7f4fdd800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd819bad0 con 0x7f4fd8074d80 2024-05-27T09:18:30.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.828+0000 7f4fd6a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd8074d80 0x7f4fd819b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:30.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.828+0000 7f4fd6a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd8074d80 0x7f4fd819b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57418/0 (socket says 172.21.15.119:57418) 2024-05-27T09:18:30.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.828+0000 7f4fd6a00700 1 -- 172.21.15.119:0/4248689471 learned_addr learned my addr 172.21.15.119:0/4248689471 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:30.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.828+0000 7f4fd6a00700 1 -- 172.21.15.119:0/4248689471 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc80095f0 con 0x7f4fd8074d80 2024-05-27T09:18:30.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.828+0000 7f4fd6a00700 1 --2- 172.21.15.119:0/4248689471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd8074d80 0x7f4fd819b590 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4fc80092e0 tx=0x7f4fc8009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:30.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.829+0000 7f4fcf400700 1 -- 172.21.15.119:0/4248689471 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4fc801d070 con 0x7f4fd8074d80 2024-05-27T09:18:30.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.829+0000 7f4fdd800700 1 -- 172.21.15.119:0/4248689471 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd819bcd0 con 0x7f4fd8074d80 2024-05-27T09:18:30.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.829+0000 7f4fcf400700 1 -- 172.21.15.119:0/4248689471 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4fc800be10 con 0x7f4fd8074d80 2024-05-27T09:18:30.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.829+0000 7f4fcf400700 1 -- 172.21.15.119:0/4248689471 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4fc800f460 con 0x7f4fd8074d80 2024-05-27T09:18:30.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.829+0000 7f4fdd800700 1 -- 172.21.15.119:0/4248689471 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd819c170 con 0x7f4fd8074d80 2024-05-27T09:18:30.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.830+0000 7f4fdd800700 1 -- 172.21.15.119:0/4248689471 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd8195120 con 0x7f4fd8074d80 2024-05-27T09:18:30.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.831+0000 7f4fcf400700 1 -- 172.21.15.119:0/4248689471 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f4fc8022470 con 0x7f4fd8074d80 2024-05-27T09:18:30.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.831+0000 7f4fcf400700 1 --2- 172.21.15.119:0/4248689471 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f4fbc038490 0x7f4fbc03a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:30.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.831+0000 7f4fcf400700 1 -- 172.21.15.119:0/4248689471 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4fc804c560 con 0x7f4fd8074d80 2024-05-27T09:18:30.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.831+0000 7f4fd6000700 1 --2- 172.21.15.119:0/4248689471 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f4fbc038490 0x7f4fbc03a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:30.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.832+0000 7f4fd6000700 1 --2- 172.21.15.119:0/4248689471 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f4fbc038490 0x7f4fbc03a940 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4006fd0 tx=0x7f4fc4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:30.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:30.837+0000 7f4fcf400700 1 -- 172.21.15.119:0/4248689471 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4fc800b960 con 0x7f4fd8074d80 2024-05-27T09:18:31.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:31.065+0000 7f4fdd800700 1 -- 172.21.15.119:0/4248689471 --> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fd8061190 con 0x7f4fbc038490 2024-05-27T09:18:31.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:31.071+0000 7f4fcf400700 1 -- 172.21.15.119:0/4248689471 <== mgr.14164 v2:172.21.15.119:6800/2118907438 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4fd8061190 con 0x7f4fbc038490 2024-05-27T09:18:31.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:31.073+0000 7f4fdd800700 1 -- 172.21.15.119:0/4248689471 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f4fbc038490 msgr2=0x7f4fbc03a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:31.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:31.073+0000 7f4fdd800700 1 --2- 172.21.15.119:0/4248689471 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f4fbc038490 0x7f4fbc03a940 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4006fd0 tx=0x7f4fc4006e40 comp rx=0 tx=0).stop 2024-05-27T09:18:31.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:31.073+0000 7f4fdd800700 1 -- 172.21.15.119:0/4248689471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd8074d80 msgr2=0x7f4fd819b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:31.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:31.073+0000 7f4fdd800700 1 --2- 172.21.15.119:0/4248689471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd8074d80 0x7f4fd819b590 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4fc80092e0 tx=0x7f4fc8009e60 comp rx=0 tx=0).stop 2024-05-27T09:18:31.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:31.073+0000 7f4fdd800700 1 -- 172.21.15.119:0/4248689471 shutdown_connections 2024-05-27T09:18:31.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:31.073+0000 7f4fdd800700 1 --2- 172.21.15.119:0/4248689471 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f4fbc038490 0x7f4fbc03a940 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:31.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:31.073+0000 7f4fdd800700 1 --2- 172.21.15.119:0/4248689471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fd8074d80 0x7f4fd819b590 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:31.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:31.073+0000 7f4fdd800700 1 -- 172.21.15.119:0/4248689471 >> 172.21.15.119:0/4248689471 conn(0x7f4fd80fb3d0 msgr2=0x7f4fd80fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:31.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:31.074+0000 7f4fdd800700 1 -- 172.21.15.119:0/4248689471 shutdown_connections 2024-05-27T09:18:31.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:31.074+0000 7f4fdd800700 1 -- 172.21.15.119:0/4248689471 wait complete. 2024-05-27T09:18:31.732 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi138 2024-05-27T09:18:31.733 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-27T09:18:31.733 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.conf 2024-05-27T09:18:31.790 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-27T09:18:31.790 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T09:18:31.884 INFO:tasks.cephadm:Adding host smithi138 to orchestrator... 2024-05-27T09:18:31.884 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph orch host add smithi138 2024-05-27T09:18:32.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:32 smithi119 ceph-mon[29082]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:18:32.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:32 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:32.288 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:18:33.353 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:33 smithi119 ceph-mon[29082]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:18:33.353 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:33 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:33.353 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:33 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:33.354 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:33 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:33.354 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:33 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:33.354 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:33 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi119", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-27T09:18:33.354 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:33 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi119", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-05-27T09:18:33.354 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:33 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:18:34.485 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:34 smithi119 ceph-mon[29082]: Deploying daemon crash.smithi119 on smithi119 2024-05-27T09:18:34.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.658+0000 7f7df7200700 1 -- 172.21.15.119:0/2188810780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7df8106270 msgr2=0x7f7df8106680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:34.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.658+0000 7f7df7200700 1 --2- 172.21.15.119:0/2188810780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7df8106270 0x7f7df8106680 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7de8009910 tx=0x7f7de8009c20 comp rx=0 tx=0).stop 2024-05-27T09:18:34.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.658+0000 7f7df7200700 1 -- 172.21.15.119:0/2188810780 shutdown_connections 2024-05-27T09:18:34.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.658+0000 7f7df7200700 1 --2- 172.21.15.119:0/2188810780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7df8106270 0x7f7df8106680 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:34.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.658+0000 7f7df7200700 1 -- 172.21.15.119:0/2188810780 >> 172.21.15.119:0/2188810780 conn(0x7f7df8075240 msgr2=0x7f7df8077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:34.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.658+0000 7f7df7200700 1 -- 172.21.15.119:0/2188810780 shutdown_connections 2024-05-27T09:18:34.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.658+0000 7f7df7200700 1 -- 172.21.15.119:0/2188810780 wait complete. 2024-05-27T09:18:34.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.659+0000 7f7df7200700 1 Processor -- start 2024-05-27T09:18:34.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.660+0000 7f7df7200700 1 -- start start 2024-05-27T09:18:34.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.660+0000 7f7df7200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7df8106270 0x7f7df819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:34.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.660+0000 7f7df7200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df819b940 con 0x7f7df8106270 2024-05-27T09:18:34.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.660+0000 7f7df5e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7df8106270 0x7f7df819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:34.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.660+0000 7f7df5e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7df8106270 0x7f7df819b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57448/0 (socket says 172.21.15.119:57448) 2024-05-27T09:18:34.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.660+0000 7f7df5e00700 1 -- 172.21.15.119:0/1327436277 learned_addr learned my addr 172.21.15.119:0/1327436277 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:34.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.661+0000 7f7df5e00700 1 -- 172.21.15.119:0/1327436277 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de80095f0 con 0x7f7df8106270 2024-05-27T09:18:34.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.661+0000 7f7df5e00700 1 --2- 172.21.15.119:0/1327436277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7df8106270 0x7f7df819b400 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7de8005b40 tx=0x7f7de8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:34.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.661+0000 7f7deea00700 1 -- 172.21.15.119:0/1327436277 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7de801d070 con 0x7f7df8106270 2024-05-27T09:18:34.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.661+0000 7f7deea00700 1 -- 172.21.15.119:0/1327436277 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7de800be10 con 0x7f7df8106270 2024-05-27T09:18:34.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.661+0000 7f7deea00700 1 -- 172.21.15.119:0/1327436277 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7de800f460 con 0x7f7df8106270 2024-05-27T09:18:34.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.662+0000 7f7df7200700 1 -- 172.21.15.119:0/1327436277 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df819bb40 con 0x7f7df8106270 2024-05-27T09:18:34.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.662+0000 7f7df7200700 1 -- 172.21.15.119:0/1327436277 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df819bfe0 con 0x7f7df8106270 2024-05-27T09:18:34.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.663+0000 7f7deea00700 1 -- 172.21.15.119:0/1327436277 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f7de80227e0 con 0x7f7df8106270 2024-05-27T09:18:34.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.663+0000 7f7df7200700 1 -- 172.21.15.119:0/1327436277 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df8194f50 con 0x7f7df8106270 2024-05-27T09:18:34.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.663+0000 7f7deea00700 1 --2- 172.21.15.119:0/1327436277 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f7ddc0384e0 0x7f7ddc03a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:34.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.663+0000 7f7deea00700 1 -- 172.21.15.119:0/1327436277 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7de804c470 con 0x7f7df8106270 2024-05-27T09:18:34.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.664+0000 7f7defe00700 1 --2- 172.21.15.119:0/1327436277 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f7ddc0384e0 0x7f7ddc03a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:34.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.664+0000 7f7defe00700 1 --2- 172.21.15.119:0/1327436277 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f7ddc0384e0 0x7f7ddc03a990 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7de0006fd0 tx=0x7f7de0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:34.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.670+0000 7f7deea00700 1 -- 172.21.15.119:0/1327436277 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7de802ba00 con 0x7f7df8106270 2024-05-27T09:18:34.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:34.895+0000 7f7df7200700 1 -- 172.21.15.119:0/1327436277 --> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}) v1 -- 0x7f7df8061190 con 0x7f7ddc0384e0 2024-05-27T09:18:35.660 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:35 smithi119 ceph-mon[29082]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:18:36.282 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:36 smithi119 ceph-mon[29082]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:18:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:37 smithi119 ceph-mon[29082]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:18:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:37 smithi119 ceph-mon[29082]: Deploying cephadm binary to smithi138 2024-05-27T09:18:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:37 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:37 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:37.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:37 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:37.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:37 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:37.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:37 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:38.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:38 smithi119 ceph-mon[29082]: Deploying daemon node-exporter.smithi119 on smithi119 2024-05-27T09:18:39.078 INFO:teuthology.orchestra.run.smithi119.stdout:Added host 'smithi138' with addr '172.21.15.138' 2024-05-27T09:18:39.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:39.075+0000 7f7deea00700 1 -- 172.21.15.119:0/1327436277 <== mgr.14164 v2:172.21.15.119:6800/2118907438 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f7df8061190 con 0x7f7ddc0384e0 2024-05-27T09:18:39.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:39.079+0000 7f7df7200700 1 -- 172.21.15.119:0/1327436277 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f7ddc0384e0 msgr2=0x7f7ddc03a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:39.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:39.079+0000 7f7df7200700 1 --2- 172.21.15.119:0/1327436277 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f7ddc0384e0 0x7f7ddc03a990 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7de0006fd0 tx=0x7f7de0006e40 comp rx=0 tx=0).stop 2024-05-27T09:18:39.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:39.079+0000 7f7df7200700 1 -- 172.21.15.119:0/1327436277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7df8106270 msgr2=0x7f7df819b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:39.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:39.079+0000 7f7df7200700 1 --2- 172.21.15.119:0/1327436277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7df8106270 0x7f7df819b400 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7de8005b40 tx=0x7f7de8004740 comp rx=0 tx=0).stop 2024-05-27T09:18:39.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:39.079+0000 7f7df7200700 1 -- 172.21.15.119:0/1327436277 shutdown_connections 2024-05-27T09:18:39.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:39.079+0000 7f7df7200700 1 --2- 172.21.15.119:0/1327436277 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f7ddc0384e0 0x7f7ddc03a990 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:39.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:39.079+0000 7f7df7200700 1 --2- 172.21.15.119:0/1327436277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7df8106270 0x7f7df819b400 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:39.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:39.079+0000 7f7df7200700 1 -- 172.21.15.119:0/1327436277 >> 172.21.15.119:0/1327436277 conn(0x7f7df8075240 msgr2=0x7f7df8075ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:39.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:39.080+0000 7f7df7200700 1 -- 172.21.15.119:0/1327436277 shutdown_connections 2024-05-27T09:18:39.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:39.080+0000 7f7df7200700 1 -- 172.21.15.119:0/1327436277 wait complete. 2024-05-27T09:18:39.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:39 smithi119 ceph-mon[29082]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:18:39.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:39 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:40.588 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:40 smithi119 ceph-mon[29082]: Added host smithi138 2024-05-27T09:18:40.638 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-05-27T09:18:41.001 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:18:41.376 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:41 smithi119 ceph-mon[29082]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:18:43.406 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:43 smithi119 ceph-mon[29082]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:18:43.406 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:43 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:43.406 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:43 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:43.406 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:43 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:43.406 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:43 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:44.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.016+0000 7f4a98800700 1 -- 172.21.15.119:0/565191490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a90074d80 msgr2=0x7f4a900731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:44.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.016+0000 7f4a98800700 1 --2- 172.21.15.119:0/565191490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a90074d80 0x7f4a900731e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4a84009910 tx=0x7f4a84009c20 comp rx=0 tx=0).stop 2024-05-27T09:18:44.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.016+0000 7f4a98800700 1 -- 172.21.15.119:0/565191490 shutdown_connections 2024-05-27T09:18:44.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.016+0000 7f4a98800700 1 --2- 172.21.15.119:0/565191490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a90074d80 0x7f4a900731e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:44.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.016+0000 7f4a98800700 1 -- 172.21.15.119:0/565191490 >> 172.21.15.119:0/565191490 conn(0x7f4a900fb3d0 msgr2=0x7f4a900fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:44.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.016+0000 7f4a98800700 1 -- 172.21.15.119:0/565191490 shutdown_connections 2024-05-27T09:18:44.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.016+0000 7f4a98800700 1 -- 172.21.15.119:0/565191490 wait complete. 2024-05-27T09:18:44.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.018+0000 7f4a98800700 1 Processor -- start 2024-05-27T09:18:44.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.018+0000 7f4a98800700 1 -- start start 2024-05-27T09:18:44.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.018+0000 7f4a98800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a90074d80 0x7f4a9019b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:44.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.018+0000 7f4a98800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9019bad0 con 0x7f4a90074d80 2024-05-27T09:18:44.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.018+0000 7f4a95e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a90074d80 0x7f4a9019b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:44.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.019+0000 7f4a95e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a90074d80 0x7f4a9019b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56574/0 (socket says 172.21.15.119:56574) 2024-05-27T09:18:44.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.019+0000 7f4a95e00700 1 -- 172.21.15.119:0/163464202 learned_addr learned my addr 172.21.15.119:0/163464202 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.019+0000 7f4a95e00700 1 -- 172.21.15.119:0/163464202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a840095f0 con 0x7f4a90074d80 2024-05-27T09:18:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.019+0000 7f4a95e00700 1 --2- 172.21.15.119:0/163464202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a90074d80 0x7f4a9019b590 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4a840092e0 tx=0x7f4a84009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.019+0000 7f4a8aa00700 1 -- 172.21.15.119:0/163464202 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4a8401d070 con 0x7f4a90074d80 2024-05-27T09:18:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.019+0000 7f4a8aa00700 1 -- 172.21.15.119:0/163464202 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4a8400be10 con 0x7f4a90074d80 2024-05-27T09:18:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.019+0000 7f4a98800700 1 -- 172.21.15.119:0/163464202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9019bcd0 con 0x7f4a90074d80 2024-05-27T09:18:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.020+0000 7f4a8aa00700 1 -- 172.21.15.119:0/163464202 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4a8400f460 con 0x7f4a90074d80 2024-05-27T09:18:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.020+0000 7f4a98800700 1 -- 172.21.15.119:0/163464202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9019c170 con 0x7f4a90074d80 2024-05-27T09:18:44.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.021+0000 7f4a8aa00700 1 -- 172.21.15.119:0/163464202 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f4a8400f5c0 con 0x7f4a90074d80 2024-05-27T09:18:44.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.021+0000 7f4a98800700 1 -- 172.21.15.119:0/163464202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a90195120 con 0x7f4a90074d80 2024-05-27T09:18:44.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.021+0000 7f4a8aa00700 1 --2- 172.21.15.119:0/163464202 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f4a780384e0 0x7f4a7803a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:44.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.022+0000 7f4a8aa00700 1 -- 172.21.15.119:0/163464202 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4a8404c490 con 0x7f4a90074d80 2024-05-27T09:18:44.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.022+0000 7f4a95400700 1 --2- 172.21.15.119:0/163464202 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f4a780384e0 0x7f4a7803a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:44.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.023+0000 7f4a95400700 1 --2- 172.21.15.119:0/163464202 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f4a780384e0 0x7f4a7803a990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4a80006fd0 tx=0x7f4a80006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:44.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.027+0000 7f4a8aa00700 1 -- 172.21.15.119:0/163464202 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4a84005c00 con 0x7f4a90074d80 2024-05-27T09:18:44.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.268+0000 7f4a98800700 1 -- 172.21.15.119:0/163464202 --> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4a90061190 con 0x7f4a780384e0 2024-05-27T09:18:44.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.270+0000 7f4a8aa00700 1 -- 172.21.15.119:0/163464202 <== mgr.14164 v2:172.21.15.119:6800/2118907438 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f4a90061190 con 0x7f4a780384e0 2024-05-27T09:18:44.271 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:18:44.271 INFO:teuthology.orchestra.run.smithi119.stdout:[{"addr": "172.21.15.119", "hostname": "smithi119", "labels": [], "status": ""}, {"addr": "172.21.15.138", "hostname": "smithi138", "labels": [], "status": ""}] 2024-05-27T09:18:44.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.273+0000 7f4a98800700 1 -- 172.21.15.119:0/163464202 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f4a780384e0 msgr2=0x7f4a7803a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:44.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.273+0000 7f4a98800700 1 --2- 172.21.15.119:0/163464202 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f4a780384e0 0x7f4a7803a990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4a80006fd0 tx=0x7f4a80006e40 comp rx=0 tx=0).stop 2024-05-27T09:18:44.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.273+0000 7f4a98800700 1 -- 172.21.15.119:0/163464202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a90074d80 msgr2=0x7f4a9019b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:44.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.273+0000 7f4a98800700 1 --2- 172.21.15.119:0/163464202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a90074d80 0x7f4a9019b590 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4a840092e0 tx=0x7f4a84009e60 comp rx=0 tx=0).stop 2024-05-27T09:18:44.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.273+0000 7f4a98800700 1 -- 172.21.15.119:0/163464202 shutdown_connections 2024-05-27T09:18:44.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.273+0000 7f4a98800700 1 --2- 172.21.15.119:0/163464202 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f4a780384e0 0x7f4a7803a990 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:44.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.273+0000 7f4a98800700 1 --2- 172.21.15.119:0/163464202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a90074d80 0x7f4a9019b590 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:44.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.273+0000 7f4a98800700 1 -- 172.21.15.119:0/163464202 >> 172.21.15.119:0/163464202 conn(0x7f4a900fb3d0 msgr2=0x7f4a900fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:44.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.273+0000 7f4a98800700 1 -- 172.21.15.119:0/163464202 shutdown_connections 2024-05-27T09:18:44.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:44.273+0000 7f4a98800700 1 -- 172.21.15.119:0/163464202 wait complete. 2024-05-27T09:18:44.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:44 smithi119 ceph-mon[29082]: Deploying daemon alertmanager.smithi119 on smithi119 2024-05-27T09:18:45.142 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-27T09:18:45.142 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-05-27T09:18:45.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:45 smithi119 ceph-mon[29082]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:18:45.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:45 smithi119 ceph-mon[29082]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T09:18:45.544 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:18:46.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:46 smithi119 ceph-mon[29082]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:18:48.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:47 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:48.984 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:48 smithi119 ceph-mon[29082]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:18:49.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.023+0000 7f6d08800700 1 -- 172.21.15.119:0/960389902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d001008d0 msgr2=0x7f6d00102cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:49.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.023+0000 7f6d08800700 1 --2- 172.21.15.119:0/960389902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d001008d0 0x7f6d00102cb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009910 tx=0x7f6cf4009c20 comp rx=0 tx=0).stop 2024-05-27T09:18:49.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.023+0000 7f6d08800700 1 -- 172.21.15.119:0/960389902 shutdown_connections 2024-05-27T09:18:49.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.023+0000 7f6d08800700 1 --2- 172.21.15.119:0/960389902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d001008d0 0x7f6d00102cb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:49.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.024+0000 7f6d08800700 1 -- 172.21.15.119:0/960389902 >> 172.21.15.119:0/960389902 conn(0x7f6d000fa2f0 msgr2=0x7f6d000fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:49.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.024+0000 7f6d08800700 1 -- 172.21.15.119:0/960389902 shutdown_connections 2024-05-27T09:18:49.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.024+0000 7f6d08800700 1 -- 172.21.15.119:0/960389902 wait complete. 2024-05-27T09:18:49.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.025+0000 7f6d08800700 1 Processor -- start 2024-05-27T09:18:49.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.025+0000 7f6d08800700 1 -- start start 2024-05-27T09:18:49.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.025+0000 7f6d08800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d001008d0 0x7f6d00100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:49.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.025+0000 7f6d08800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d00100770 con 0x7f6d001008d0 2024-05-27T09:18:49.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.025+0000 7f6d05e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d001008d0 0x7f6d00100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:49.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.025+0000 7f6d05e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d001008d0 0x7f6d00100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44176/0 (socket says 172.21.15.119:44176) 2024-05-27T09:18:49.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.025+0000 7f6d05e00700 1 -- 172.21.15.119:0/1717846157 learned_addr learned my addr 172.21.15.119:0/1717846157 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:18:49.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.026+0000 7f6d05e00700 1 -- 172.21.15.119:0/1717846157 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf40095f0 con 0x7f6d001008d0 2024-05-27T09:18:49.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.026+0000 7f6d05e00700 1 --2- 172.21.15.119:0/1717846157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d001008d0 0x7f6d00100230 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40092e0 tx=0x7f6cf4009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:49.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.026+0000 7f6cfaa00700 1 -- 172.21.15.119:0/1717846157 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6cf401d070 con 0x7f6d001008d0 2024-05-27T09:18:49.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.026+0000 7f6cfaa00700 1 -- 172.21.15.119:0/1717846157 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6cf400be10 con 0x7f6d001008d0 2024-05-27T09:18:49.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.026+0000 7f6d08800700 1 -- 172.21.15.119:0/1717846157 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d000fe880 con 0x7f6d001008d0 2024-05-27T09:18:49.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.026+0000 7f6cfaa00700 1 -- 172.21.15.119:0/1717846157 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6cf400f460 con 0x7f6d001008d0 2024-05-27T09:18:49.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.027+0000 7f6d08800700 1 -- 172.21.15.119:0/1717846157 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d000fed20 con 0x7f6d001008d0 2024-05-27T09:18:49.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.028+0000 7f6cfaa00700 1 -- 172.21.15.119:0/1717846157 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f6cf4022470 con 0x7f6d001008d0 2024-05-27T09:18:49.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.028+0000 7f6d08800700 1 -- 172.21.15.119:0/1717846157 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d00062380 con 0x7f6d001008d0 2024-05-27T09:18:49.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.028+0000 7f6cfaa00700 1 --2- 172.21.15.119:0/1717846157 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f6ce8038490 0x7f6ce803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:49.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.028+0000 7f6cfaa00700 1 -- 172.21.15.119:0/1717846157 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6cf404c4a0 con 0x7f6d001008d0 2024-05-27T09:18:49.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.029+0000 7f6d05400700 1 --2- 172.21.15.119:0/1717846157 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f6ce8038490 0x7f6ce803a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:49.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.029+0000 7f6d05400700 1 --2- 172.21.15.119:0/1717846157 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f6ce8038490 0x7f6ce803a940 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0006fd0 tx=0x7f6cf0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:49.034 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.034+0000 7f6cfaa00700 1 -- 172.21.15.119:0/1717846157 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6cf400b960 con 0x7f6d001008d0 2024-05-27T09:18:49.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.262+0000 7f6d08800700 1 -- 172.21.15.119:0/1717846157 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f6d0018e500 con 0x7f6d001008d0 2024-05-27T09:18:49.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.917+0000 7f6cfaa00700 1 -- 172.21.15.119:0/1717846157 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6cf4027020 con 0x7f6d001008d0 2024-05-27T09:18:49.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.922+0000 7f6d08800700 1 -- 172.21.15.119:0/1717846157 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f6ce8038490 msgr2=0x7f6ce803a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:49.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.922+0000 7f6d08800700 1 --2- 172.21.15.119:0/1717846157 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f6ce8038490 0x7f6ce803a940 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0006fd0 tx=0x7f6cf0006e40 comp rx=0 tx=0).stop 2024-05-27T09:18:49.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.922+0000 7f6d08800700 1 -- 172.21.15.119:0/1717846157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d001008d0 msgr2=0x7f6d00100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:49.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.922+0000 7f6d08800700 1 --2- 172.21.15.119:0/1717846157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d001008d0 0x7f6d00100230 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40092e0 tx=0x7f6cf4009e60 comp rx=0 tx=0).stop 2024-05-27T09:18:49.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.922+0000 7f6d08800700 1 -- 172.21.15.119:0/1717846157 shutdown_connections 2024-05-27T09:18:49.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.922+0000 7f6d08800700 1 --2- 172.21.15.119:0/1717846157 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f6ce8038490 0x7f6ce803a940 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:49.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.922+0000 7f6d08800700 1 --2- 172.21.15.119:0/1717846157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d001008d0 0x7f6d00100230 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:49.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.922+0000 7f6d08800700 1 -- 172.21.15.119:0/1717846157 >> 172.21.15.119:0/1717846157 conn(0x7f6d000fa2f0 msgr2=0x7f6d000fafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:49.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.923+0000 7f6d08800700 1 -- 172.21.15.119:0/1717846157 shutdown_connections 2024-05-27T09:18:49.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:18:49.923+0000 7f6d08800700 1 -- 172.21.15.119:0/1717846157 wait complete. 2024-05-27T09:18:49.924 INFO:teuthology.orchestra.run.smithi119.stderr:adjusted tunables profile to default 2024-05-27T09:18:50.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:49 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/1717846157' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-27T09:18:50.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:50 smithi119 ceph-mon[29082]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:18:50.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:50 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/1717846157' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-27T09:18:50.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:50 smithi119 ceph-mon[29082]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T09:18:51.043 INFO:tasks.cephadm:Adding mon.smithi119 on smithi119 2024-05-27T09:18:51.043 INFO:tasks.cephadm:Adding mon.smithi138 on smithi138 2024-05-27T09:18:51.043 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph orch apply mon '2;smithi119:172.21.15.119=smithi119;smithi138:172.21.15.138=smithi138' 2024-05-27T09:18:51.422 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:18:51.521 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:18:52.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:52 smithi119 ceph-mon[29082]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:18:53.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.772+0000 7f276fe00700 1 -- 172.21.15.138:0/3196598786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2768106460 msgr2=0x7f2768106870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:53.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.772+0000 7f276fe00700 1 --2- 172.21.15.138:0/3196598786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2768106460 0x7f2768106870 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f275c009910 tx=0x7f275c009c20 comp rx=0 tx=0).stop 2024-05-27T09:18:53.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.773+0000 7f276fe00700 1 -- 172.21.15.138:0/3196598786 shutdown_connections 2024-05-27T09:18:53.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.773+0000 7f276fe00700 1 --2- 172.21.15.138:0/3196598786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2768106460 0x7f2768106870 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:53.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.773+0000 7f276fe00700 1 -- 172.21.15.138:0/3196598786 >> 172.21.15.138:0/3196598786 conn(0x7f27680756e0 msgr2=0x7f2768077b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:53.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.773+0000 7f276fe00700 1 -- 172.21.15.138:0/3196598786 shutdown_connections 2024-05-27T09:18:53.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.773+0000 7f276fe00700 1 -- 172.21.15.138:0/3196598786 wait complete. 2024-05-27T09:18:53.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.774+0000 7f276fe00700 1 Processor -- start 2024-05-27T09:18:53.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.774+0000 7f276fe00700 1 -- start start 2024-05-27T09:18:53.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.775+0000 7f276fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2768106460 0x7f276819b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:53.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.775+0000 7f276fe00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276819baa0 con 0x7f2768106460 2024-05-27T09:18:53.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.775+0000 7f276d400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2768106460 0x7f276819b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:53.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.775+0000 7f276d400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2768106460 0x7f276819b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.138:43884/0 (socket says 172.21.15.138:43884) 2024-05-27T09:18:53.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.775+0000 7f276d400700 1 -- 172.21.15.138:0/718429523 learned_addr learned my addr 172.21.15.138:0/718429523 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:18:53.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.776+0000 7f276d400700 1 -- 172.21.15.138:0/718429523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275c0095f0 con 0x7f2768106460 2024-05-27T09:18:53.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.776+0000 7f276d400700 1 --2- 172.21.15.138:0/718429523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2768106460 0x7f276819b560 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f275c0092e0 tx=0x7f275c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:53.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.776+0000 7f2766000700 1 -- 172.21.15.138:0/718429523 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f275c01d070 con 0x7f2768106460 2024-05-27T09:18:53.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.776+0000 7f276fe00700 1 -- 172.21.15.138:0/718429523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f276819bca0 con 0x7f2768106460 2024-05-27T09:18:53.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.776+0000 7f2766000700 1 -- 172.21.15.138:0/718429523 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f275c00be10 con 0x7f2768106460 2024-05-27T09:18:53.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.776+0000 7f2766000700 1 -- 172.21.15.138:0/718429523 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f275c00f460 con 0x7f2768106460 2024-05-27T09:18:53.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.776+0000 7f276fe00700 1 -- 172.21.15.138:0/718429523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f276819c140 con 0x7f2768106460 2024-05-27T09:18:53.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.778+0000 7f276fe00700 1 -- 172.21.15.138:0/718429523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2768195130 con 0x7f2768106460 2024-05-27T09:18:53.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.778+0000 7f2766000700 1 -- 172.21.15.138:0/718429523 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f275c022470 con 0x7f2768106460 2024-05-27T09:18:53.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.778+0000 7f2766000700 1 --2- 172.21.15.138:0/718429523 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f2750038490 0x7f275003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:53.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.779+0000 7f2766000700 1 -- 172.21.15.138:0/718429523 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f275c04c560 con 0x7f2768106460 2024-05-27T09:18:53.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.779+0000 7f276ca00700 1 --2- 172.21.15.138:0/718429523 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f2750038490 0x7f275003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:53.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.780+0000 7f276ca00700 1 --2- 172.21.15.138:0/718429523 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f2750038490 0x7f275003a940 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2754006fd0 tx=0x7f2754006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:53.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:53.786+0000 7f2766000700 1 -- 172.21.15.138:0/718429523 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f275c00b960 con 0x7f2768106460 2024-05-27T09:18:54.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:54.011+0000 7f276fe00700 1 -- 172.21.15.138:0/718429523 --> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi119:172.21.15.119=smithi119;smithi138:172.21.15.138=smithi138", "target": ["mon-mgr", ""]}) v1 -- 0x7f2768061300 con 0x7f2750038490 2024-05-27T09:18:54.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:54.026+0000 7f2766000700 1 -- 172.21.15.138:0/718429523 <== mgr.14164 v2:172.21.15.119:6800/2118907438 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2768061300 con 0x7f2750038490 2024-05-27T09:18:54.026 INFO:teuthology.orchestra.run.smithi138.stdout:Scheduled mon update... 2024-05-27T09:18:54.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:54.028+0000 7f276fe00700 1 -- 172.21.15.138:0/718429523 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f2750038490 msgr2=0x7f275003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:54.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:54.028+0000 7f276fe00700 1 --2- 172.21.15.138:0/718429523 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f2750038490 0x7f275003a940 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2754006fd0 tx=0x7f2754006e40 comp rx=0 tx=0).stop 2024-05-27T09:18:54.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:54.028+0000 7f276fe00700 1 -- 172.21.15.138:0/718429523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2768106460 msgr2=0x7f276819b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:54.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:54.029+0000 7f276fe00700 1 --2- 172.21.15.138:0/718429523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2768106460 0x7f276819b560 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f275c0092e0 tx=0x7f275c009e60 comp rx=0 tx=0).stop 2024-05-27T09:18:54.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:54.029+0000 7f276fe00700 1 -- 172.21.15.138:0/718429523 shutdown_connections 2024-05-27T09:18:54.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:54.029+0000 7f276fe00700 1 --2- 172.21.15.138:0/718429523 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f2750038490 0x7f275003a940 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:54.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:54.029+0000 7f276fe00700 1 --2- 172.21.15.138:0/718429523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2768106460 0x7f276819b560 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:54.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:54.029+0000 7f276fe00700 1 -- 172.21.15.138:0/718429523 >> 172.21.15.138:0/718429523 conn(0x7f27680756e0 msgr2=0x7f2768077310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:54.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:54.029+0000 7f276fe00700 1 -- 172.21.15.138:0/718429523 shutdown_connections 2024-05-27T09:18:54.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:54.029+0000 7f276fe00700 1 -- 172.21.15.138:0/718429523 wait complete. 2024-05-27T09:18:54.667 DEBUG:teuthology.orchestra.run.smithi138:mon.smithi138> sudo journalctl -f -n 0 -u ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@mon.smithi138.service 2024-05-27T09:18:54.673 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-27T09:18:54.673 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T09:18:55.055 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:18:55.158 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:18:55.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:55 smithi119 ceph-mon[29082]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:18:55.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:55 smithi119 ceph-mon[29082]: from='client.14197 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi119:172.21.15.119=smithi119;smithi138:172.21.15.138=smithi138", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:18:55.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:55 smithi119 ceph-mon[29082]: Saving service mon spec with placement smithi119:172.21.15.119=smithi119;smithi138:172.21.15.138=smithi138;count:2 2024-05-27T09:18:55.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:55 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:55.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:55 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:55.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:55 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:55.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:55 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:55.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:55 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:55.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:55 smithi119 ceph-mon[29082]: Regenerating cephadm self-signed grafana TLS certificates 2024-05-27T09:18:55.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:55 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:55.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:55 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:55.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:55 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-05-27T09:18:55.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:55 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:56.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:56 smithi119 ceph-mon[29082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-05-27T09:18:56.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:56 smithi119 ceph-mon[29082]: Deploying daemon grafana.smithi119 on smithi119 2024-05-27T09:18:57.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.158+0000 7f8be5000700 1 -- 172.21.15.138:0/3010867059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8be0106460 msgr2=0x7f8be0106870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:57.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.158+0000 7f8be5000700 1 --2- 172.21.15.138:0/3010867059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8be0106460 0x7f8be0106870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009910 tx=0x7f8bd0009c20 comp rx=0 tx=0).stop 2024-05-27T09:18:57.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.158+0000 7f8be5000700 1 -- 172.21.15.138:0/3010867059 shutdown_connections 2024-05-27T09:18:57.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.158+0000 7f8be5000700 1 --2- 172.21.15.138:0/3010867059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8be0106460 0x7f8be0106870 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:57.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.158+0000 7f8be5000700 1 -- 172.21.15.138:0/3010867059 >> 172.21.15.138:0/3010867059 conn(0x7f8be00756e0 msgr2=0x7f8be0077b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:57.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.159+0000 7f8be5000700 1 -- 172.21.15.138:0/3010867059 shutdown_connections 2024-05-27T09:18:57.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.159+0000 7f8be5000700 1 -- 172.21.15.138:0/3010867059 wait complete. 2024-05-27T09:18:57.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.160+0000 7f8be5000700 1 Processor -- start 2024-05-27T09:18:57.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.160+0000 7f8be5000700 1 -- start start 2024-05-27T09:18:57.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.161+0000 7f8be5000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8be0106460 0x7f8be019b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:57.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.161+0000 7f8be5000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be019baa0 con 0x7f8be0106460 2024-05-27T09:18:57.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.161+0000 7f8bdde00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8be0106460 0x7f8be019b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:57.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.161+0000 7f8bdde00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8be0106460 0x7f8be019b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.138:43908/0 (socket says 172.21.15.138:43908) 2024-05-27T09:18:57.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.161+0000 7f8bdde00700 1 -- 172.21.15.138:0/2008522462 learned_addr learned my addr 172.21.15.138:0/2008522462 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:18:57.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.162+0000 7f8bdde00700 1 -- 172.21.15.138:0/2008522462 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd00095f0 con 0x7f8be0106460 2024-05-27T09:18:57.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.163+0000 7f8bdde00700 1 --2- 172.21.15.138:0/2008522462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8be0106460 0x7f8be019b560 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00092e0 tx=0x7f8bd0009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:57.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.164+0000 7f8bd6a00700 1 -- 172.21.15.138:0/2008522462 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8bd001d070 con 0x7f8be0106460 2024-05-27T09:18:57.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.164+0000 7f8be5000700 1 -- 172.21.15.138:0/2008522462 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be019bca0 con 0x7f8be0106460 2024-05-27T09:18:57.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.164+0000 7f8be5000700 1 -- 172.21.15.138:0/2008522462 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be019c140 con 0x7f8be0106460 2024-05-27T09:18:57.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.164+0000 7f8bd6a00700 1 -- 172.21.15.138:0/2008522462 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8bd000be10 con 0x7f8be0106460 2024-05-27T09:18:57.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.164+0000 7f8bd6a00700 1 -- 172.21.15.138:0/2008522462 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8bd0022de0 con 0x7f8be0106460 2024-05-27T09:18:57.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.165+0000 7f8be5000700 1 -- 172.21.15.138:0/2008522462 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc0005320 con 0x7f8be0106460 2024-05-27T09:18:57.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.167+0000 7f8bd6a00700 1 -- 172.21.15.138:0/2008522462 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f8bd0022690 con 0x7f8be0106460 2024-05-27T09:18:57.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.168+0000 7f8bd6a00700 1 --2- 172.21.15.138:0/2008522462 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f8bc40384e0 0x7f8bc403a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:18:57.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.168+0000 7f8bd6a00700 1 -- 172.21.15.138:0/2008522462 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8bd004c630 con 0x7f8be0106460 2024-05-27T09:18:57.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.169+0000 7f8bdd400700 1 --2- 172.21.15.138:0/2008522462 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f8bc40384e0 0x7f8bc403a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:18:57.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.170+0000 7f8bdd400700 1 --2- 172.21.15.138:0/2008522462 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f8bc40384e0 0x7f8bc403a990 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc006fd0 tx=0x7f8bcc006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:18:57.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.190+0000 7f8bd6a00700 1 -- 172.21.15.138:0/2008522462 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8bd000b960 con 0x7f8be0106460 2024-05-27T09:18:57.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.489+0000 7f8be5000700 1 -- 172.21.15.138:0/2008522462 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8bc0005190 con 0x7f8be0106460 2024-05-27T09:18:57.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.490+0000 7f8bd6a00700 1 -- 172.21.15.138:0/2008522462 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f8bd0027020 con 0x7f8be0106460 2024-05-27T09:18:57.491 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:18:57.491 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b966b06e-1c09-11ef-bc9b-c7b262605968","modified":"2024-05-27T09:16:53.014813Z","created":"2024-05-27T09:16:53.014813Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi119","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T09:18:57.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.492+0000 7f8be5000700 1 -- 172.21.15.138:0/2008522462 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f8bc40384e0 msgr2=0x7f8bc403a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:57.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.492+0000 7f8be5000700 1 --2- 172.21.15.138:0/2008522462 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f8bc40384e0 0x7f8bc403a990 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc006fd0 tx=0x7f8bcc006e40 comp rx=0 tx=0).stop 2024-05-27T09:18:57.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.492+0000 7f8be5000700 1 -- 172.21.15.138:0/2008522462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8be0106460 msgr2=0x7f8be019b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:18:57.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.492+0000 7f8be5000700 1 --2- 172.21.15.138:0/2008522462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8be0106460 0x7f8be019b560 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00092e0 tx=0x7f8bd0009e60 comp rx=0 tx=0).stop 2024-05-27T09:18:57.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.493+0000 7f8be5000700 1 -- 172.21.15.138:0/2008522462 shutdown_connections 2024-05-27T09:18:57.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.493+0000 7f8be5000700 1 --2- 172.21.15.138:0/2008522462 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f8bc40384e0 0x7f8bc403a990 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:57.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.493+0000 7f8be5000700 1 --2- 172.21.15.138:0/2008522462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8be0106460 0x7f8be019b560 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:18:57.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.493+0000 7f8be5000700 1 -- 172.21.15.138:0/2008522462 >> 172.21.15.138:0/2008522462 conn(0x7f8be00756e0 msgr2=0x7f8be0077310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:18:57.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.493+0000 7f8be5000700 1 -- 172.21.15.138:0/2008522462 shutdown_connections 2024-05-27T09:18:57.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:18:57.493+0000 7f8be5000700 1 -- 172.21.15.138:0/2008522462 wait complete. 2024-05-27T09:18:57.495 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-27T09:18:57.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:57 smithi119 ceph-mon[29082]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:18:57.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:57 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:18:58.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:58 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/2008522462' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:18:59.287 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-27T09:18:59.288 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T09:18:59.399 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:18:59 smithi119 ceph-mon[29082]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:18:59.666 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:18:59.774 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:19:01.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:01 smithi119 ceph-mon[29082]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:19:02.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.055+0000 7f8916800700 1 -- 172.21.15.138:0/1586178354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89100fe6b0 msgr2=0x7f89100feac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:02.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.055+0000 7f8916800700 1 --2- 172.21.15.138:0/1586178354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89100fe6b0 0x7f89100feac0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8900009910 tx=0x7f8900009c20 comp rx=0 tx=0).stop 2024-05-27T09:19:02.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.055+0000 7f8916800700 1 -- 172.21.15.138:0/1586178354 shutdown_connections 2024-05-27T09:19:02.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.055+0000 7f8916800700 1 --2- 172.21.15.138:0/1586178354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89100fe6b0 0x7f89100feac0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:02.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.055+0000 7f8916800700 1 -- 172.21.15.138:0/1586178354 >> 172.21.15.138:0/1586178354 conn(0x7f89100fa2e0 msgr2=0x7f89100fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:02.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.056+0000 7f8916800700 1 -- 172.21.15.138:0/1586178354 shutdown_connections 2024-05-27T09:19:02.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.056+0000 7f8916800700 1 -- 172.21.15.138:0/1586178354 wait complete. 2024-05-27T09:19:02.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.057+0000 7f8916800700 1 Processor -- start 2024-05-27T09:19:02.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.058+0000 7f8916800700 1 -- start start 2024-05-27T09:19:02.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.058+0000 7f8916800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89100fe6b0 0x7f89101971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:02.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.058+0000 7f8916800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89101976e0 con 0x7f89100fe6b0 2024-05-27T09:19:02.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.059+0000 7f890f400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89100fe6b0 0x7f89101971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:19:02.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.059+0000 7f890f400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89100fe6b0 0x7f89101971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.138:57098/0 (socket says 172.21.15.138:57098) 2024-05-27T09:19:02.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.059+0000 7f890f400700 1 -- 172.21.15.138:0/3552975110 learned_addr learned my addr 172.21.15.138:0/3552975110 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:19:02.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.060+0000 7f890f400700 1 -- 172.21.15.138:0/3552975110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89000095f0 con 0x7f89100fe6b0 2024-05-27T09:19:02.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.060+0000 7f890f400700 1 --2- 172.21.15.138:0/3552975110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89100fe6b0 0x7f89101971a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8900005b40 tx=0x7f8900004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:19:02.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.061+0000 7f8907e00700 1 -- 172.21.15.138:0/3552975110 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f890001d070 con 0x7f89100fe6b0 2024-05-27T09:19:02.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.061+0000 7f8907e00700 1 -- 172.21.15.138:0/3552975110 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f890000be10 con 0x7f89100fe6b0 2024-05-27T09:19:02.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.061+0000 7f8907e00700 1 -- 172.21.15.138:0/3552975110 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f890000f460 con 0x7f89100fe6b0 2024-05-27T09:19:02.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.061+0000 7f8916800700 1 -- 172.21.15.138:0/3552975110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89101978e0 con 0x7f89100fe6b0 2024-05-27T09:19:02.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.061+0000 7f8916800700 1 -- 172.21.15.138:0/3552975110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8910197d80 con 0x7f89100fe6b0 2024-05-27T09:19:02.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.062+0000 7f8916800700 1 -- 172.21.15.138:0/3552975110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8910190d00 con 0x7f89100fe6b0 2024-05-27T09:19:02.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.063+0000 7f8907e00700 1 -- 172.21.15.138:0/3552975110 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f89000227e0 con 0x7f89100fe6b0 2024-05-27T09:19:02.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.063+0000 7f8907e00700 1 --2- 172.21.15.138:0/3552975110 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f88f40384e0 0x7f88f403a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:02.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.063+0000 7f8907e00700 1 -- 172.21.15.138:0/3552975110 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f890004cae0 con 0x7f89100fe6b0 2024-05-27T09:19:02.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.064+0000 7f890ea00700 1 --2- 172.21.15.138:0/3552975110 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f88f40384e0 0x7f88f403a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:19:02.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.065+0000 7f890ea00700 1 --2- 172.21.15.138:0/3552975110 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f88f40384e0 0x7f88f403a990 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f88fc006fd0 tx=0x7f88fc006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:19:02.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.091+0000 7f8907e00700 1 -- 172.21.15.138:0/3552975110 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f890002be10 con 0x7f89100fe6b0 2024-05-27T09:19:02.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.392+0000 7f8916800700 1 -- 172.21.15.138:0/3552975110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f891002cc30 con 0x7f89100fe6b0 2024-05-27T09:19:02.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.393+0000 7f8907e00700 1 -- 172.21.15.138:0/3552975110 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f8900027020 con 0x7f89100fe6b0 2024-05-27T09:19:02.394 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:19:02.395 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b966b06e-1c09-11ef-bc9b-c7b262605968","modified":"2024-05-27T09:16:53.014813Z","created":"2024-05-27T09:16:53.014813Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi119","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T09:19:02.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.396+0000 7f8916800700 1 -- 172.21.15.138:0/3552975110 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f88f40384e0 msgr2=0x7f88f403a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:02.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.396+0000 7f8916800700 1 --2- 172.21.15.138:0/3552975110 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f88f40384e0 0x7f88f403a990 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f88fc006fd0 tx=0x7f88fc006e40 comp rx=0 tx=0).stop 2024-05-27T09:19:02.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.396+0000 7f8916800700 1 -- 172.21.15.138:0/3552975110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89100fe6b0 msgr2=0x7f89101971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:02.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.396+0000 7f8916800700 1 --2- 172.21.15.138:0/3552975110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89100fe6b0 0x7f89101971a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8900005b40 tx=0x7f8900004740 comp rx=0 tx=0).stop 2024-05-27T09:19:02.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.397+0000 7f8916800700 1 -- 172.21.15.138:0/3552975110 shutdown_connections 2024-05-27T09:19:02.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.397+0000 7f8916800700 1 --2- 172.21.15.138:0/3552975110 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f88f40384e0 0x7f88f403a990 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:02.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.397+0000 7f8916800700 1 --2- 172.21.15.138:0/3552975110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89100fe6b0 0x7f89101971a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:02.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.397+0000 7f8916800700 1 -- 172.21.15.138:0/3552975110 >> 172.21.15.138:0/3552975110 conn(0x7f89100fa2e0 msgr2=0x7f89100faf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:02.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.397+0000 7f8916800700 1 -- 172.21.15.138:0/3552975110 shutdown_connections 2024-05-27T09:19:02.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:02.397+0000 7f8916800700 1 -- 172.21.15.138:0/3552975110 wait complete. 2024-05-27T09:19:02.399 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-27T09:19:03.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:03 smithi119 ceph-mon[29082]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:19:03.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:03 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/3552975110' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:19:04.145 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-27T09:19:04.145 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T09:19:04.521 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:19:04.622 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:19:05.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:05 smithi119 ceph-mon[29082]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:19:06.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.710+0000 7f647be00700 1 -- 172.21.15.138:0/2718226209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64740ffe40 msgr2=0x7f6474100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:06.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.710+0000 7f647be00700 1 --2- 172.21.15.138:0/2718226209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64740ffe40 0x7f6474100250 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6468009910 tx=0x7f6468009c20 comp rx=0 tx=0).stop 2024-05-27T09:19:06.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.710+0000 7f647be00700 1 -- 172.21.15.138:0/2718226209 shutdown_connections 2024-05-27T09:19:06.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.710+0000 7f647be00700 1 --2- 172.21.15.138:0/2718226209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64740ffe40 0x7f6474100250 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:06.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.710+0000 7f647be00700 1 -- 172.21.15.138:0/2718226209 >> 172.21.15.138:0/2718226209 conn(0x7f64740fb3d0 msgr2=0x7f64740fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:06.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.710+0000 7f647be00700 1 -- 172.21.15.138:0/2718226209 shutdown_connections 2024-05-27T09:19:06.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.710+0000 7f647be00700 1 -- 172.21.15.138:0/2718226209 wait complete. 2024-05-27T09:19:06.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.711+0000 7f647be00700 1 Processor -- start 2024-05-27T09:19:06.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.711+0000 7f647be00700 1 -- start start 2024-05-27T09:19:06.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.712+0000 7f647be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64740ffe40 0x7f6474194f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:06.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.712+0000 7f647be00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64741954d0 con 0x7f64740ffe40 2024-05-27T09:19:06.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.712+0000 7f6479400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64740ffe40 0x7f6474194f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:19:06.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.712+0000 7f6479400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64740ffe40 0x7f6474194f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.138:57120/0 (socket says 172.21.15.138:57120) 2024-05-27T09:19:06.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.712+0000 7f6479400700 1 -- 172.21.15.138:0/4019991080 learned_addr learned my addr 172.21.15.138:0/4019991080 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:19:06.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.713+0000 7f6479400700 1 -- 172.21.15.138:0/4019991080 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64680095f0 con 0x7f64740ffe40 2024-05-27T09:19:06.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.714+0000 7f6479400700 1 --2- 172.21.15.138:0/4019991080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64740ffe40 0x7f6474194f90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f64680092e0 tx=0x7f6468009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:19:06.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.714+0000 7f6472000700 1 -- 172.21.15.138:0/4019991080 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f646801d070 con 0x7f64740ffe40 2024-05-27T09:19:06.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.714+0000 7f647be00700 1 -- 172.21.15.138:0/4019991080 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64741956d0 con 0x7f64740ffe40 2024-05-27T09:19:06.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.714+0000 7f6472000700 1 -- 172.21.15.138:0/4019991080 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f646800be10 con 0x7f64740ffe40 2024-05-27T09:19:06.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.714+0000 7f6472000700 1 -- 172.21.15.138:0/4019991080 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f646800f460 con 0x7f64740ffe40 2024-05-27T09:19:06.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.714+0000 7f647be00700 1 -- 172.21.15.138:0/4019991080 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6474195b70 con 0x7f64740ffe40 2024-05-27T09:19:06.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.715+0000 7f647be00700 1 -- 172.21.15.138:0/4019991080 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f647418eaa0 con 0x7f64740ffe40 2024-05-27T09:19:06.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.716+0000 7f6472000700 1 -- 172.21.15.138:0/4019991080 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f6468022470 con 0x7f64740ffe40 2024-05-27T09:19:06.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.716+0000 7f6472000700 1 --2- 172.21.15.138:0/4019991080 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f645c038490 0x7f645c03a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:06.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.716+0000 7f6472000700 1 -- 172.21.15.138:0/4019991080 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f646804c3b0 con 0x7f64740ffe40 2024-05-27T09:19:06.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.717+0000 7f6478a00700 1 --2- 172.21.15.138:0/4019991080 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f645c038490 0x7f645c03a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:19:06.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.718+0000 7f6478a00700 1 --2- 172.21.15.138:0/4019991080 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f645c038490 0x7f645c03a940 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6460006fd0 tx=0x7f6460006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:19:06.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:06.727+0000 7f6472000700 1 -- 172.21.15.138:0/4019991080 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f646800b960 con 0x7f64740ffe40 2024-05-27T09:19:07.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:07.029+0000 7f647be00700 1 -- 172.21.15.138:0/4019991080 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f647402cc30 con 0x7f64740ffe40 2024-05-27T09:19:07.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:07.030+0000 7f6472000700 1 -- 172.21.15.138:0/4019991080 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f6468027070 con 0x7f64740ffe40 2024-05-27T09:19:07.031 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:19:07.032 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b966b06e-1c09-11ef-bc9b-c7b262605968","modified":"2024-05-27T09:16:53.014813Z","created":"2024-05-27T09:16:53.014813Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi119","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T09:19:07.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:07.034+0000 7f647be00700 1 -- 172.21.15.138:0/4019991080 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f645c038490 msgr2=0x7f645c03a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:07.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:07.034+0000 7f647be00700 1 --2- 172.21.15.138:0/4019991080 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f645c038490 0x7f645c03a940 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6460006fd0 tx=0x7f6460006e40 comp rx=0 tx=0).stop 2024-05-27T09:19:07.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:07.034+0000 7f647be00700 1 -- 172.21.15.138:0/4019991080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64740ffe40 msgr2=0x7f6474194f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:07.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:07.034+0000 7f647be00700 1 --2- 172.21.15.138:0/4019991080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64740ffe40 0x7f6474194f90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f64680092e0 tx=0x7f6468009e60 comp rx=0 tx=0).stop 2024-05-27T09:19:07.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:07.034+0000 7f647be00700 1 -- 172.21.15.138:0/4019991080 shutdown_connections 2024-05-27T09:19:07.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:07.034+0000 7f647be00700 1 --2- 172.21.15.138:0/4019991080 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f645c038490 0x7f645c03a940 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:07.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:07.034+0000 7f647be00700 1 --2- 172.21.15.138:0/4019991080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64740ffe40 0x7f6474194f90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:07.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:07.034+0000 7f647be00700 1 -- 172.21.15.138:0/4019991080 >> 172.21.15.138:0/4019991080 conn(0x7f64740fb3d0 msgr2=0x7f64740fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:07.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:07.034+0000 7f647be00700 1 -- 172.21.15.138:0/4019991080 shutdown_connections 2024-05-27T09:19:07.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:07.034+0000 7f647be00700 1 -- 172.21.15.138:0/4019991080 wait complete. 2024-05-27T09:19:07.036 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-27T09:19:07.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:07 smithi119 ceph-mon[29082]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:19:07.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:07 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/4019991080' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:19:08.815 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-27T09:19:08.815 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T09:19:09.190 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:19:09.288 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:19:09.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:09 smithi119 ceph-mon[29082]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:19:11.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:11 smithi119 ceph-mon[29082]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:19:11.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.586+0000 7fdb74400700 1 -- 172.21.15.138:0/1393738264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb6c0ffe40 msgr2=0x7fdb6c100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:11.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.586+0000 7fdb74400700 1 --2- 172.21.15.138:0/1393738264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb6c0ffe40 0x7fdb6c100250 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009910 tx=0x7fdb60009c20 comp rx=0 tx=0).stop 2024-05-27T09:19:11.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.587+0000 7fdb74400700 1 -- 172.21.15.138:0/1393738264 shutdown_connections 2024-05-27T09:19:11.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.587+0000 7fdb74400700 1 --2- 172.21.15.138:0/1393738264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb6c0ffe40 0x7fdb6c100250 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:11.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.587+0000 7fdb74400700 1 -- 172.21.15.138:0/1393738264 >> 172.21.15.138:0/1393738264 conn(0x7fdb6c0fb3d0 msgr2=0x7fdb6c0fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:11.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.587+0000 7fdb74400700 1 -- 172.21.15.138:0/1393738264 shutdown_connections 2024-05-27T09:19:11.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.587+0000 7fdb74400700 1 -- 172.21.15.138:0/1393738264 wait complete. 2024-05-27T09:19:11.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.588+0000 7fdb74400700 1 Processor -- start 2024-05-27T09:19:11.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.588+0000 7fdb74400700 1 -- start start 2024-05-27T09:19:11.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.589+0000 7fdb74400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb6c0ffe40 0x7fdb6c197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:11.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.589+0000 7fdb74400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6c1976d0 con 0x7fdb6c0ffe40 2024-05-27T09:19:11.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.589+0000 7fdb71a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb6c0ffe40 0x7fdb6c197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:19:11.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.589+0000 7fdb71a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb6c0ffe40 0x7fdb6c197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.138:56438/0 (socket says 172.21.15.138:56438) 2024-05-27T09:19:11.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.589+0000 7fdb71a00700 1 -- 172.21.15.138:0/3155111022 learned_addr learned my addr 172.21.15.138:0/3155111022 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:19:11.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.590+0000 7fdb71a00700 1 -- 172.21.15.138:0/3155111022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb600095f0 con 0x7fdb6c0ffe40 2024-05-27T09:19:11.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.590+0000 7fdb71a00700 1 --2- 172.21.15.138:0/3155111022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb6c0ffe40 0x7fdb6c197190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdb600092e0 tx=0x7fdb60009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:19:11.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.590+0000 7fdb6a000700 1 -- 172.21.15.138:0/3155111022 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb6001d070 con 0x7fdb6c0ffe40 2024-05-27T09:19:11.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.591+0000 7fdb74400700 1 -- 172.21.15.138:0/3155111022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb6c1978d0 con 0x7fdb6c0ffe40 2024-05-27T09:19:11.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.591+0000 7fdb6a000700 1 -- 172.21.15.138:0/3155111022 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdb6000be10 con 0x7fdb6c0ffe40 2024-05-27T09:19:11.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.591+0000 7fdb6a000700 1 -- 172.21.15.138:0/3155111022 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb6000f460 con 0x7fdb6c0ffe40 2024-05-27T09:19:11.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.591+0000 7fdb74400700 1 -- 172.21.15.138:0/3155111022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb6c197d70 con 0x7fdb6c0ffe40 2024-05-27T09:19:11.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.592+0000 7fdb74400700 1 -- 172.21.15.138:0/3155111022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb6c190ea0 con 0x7fdb6c0ffe40 2024-05-27T09:19:11.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.592+0000 7fdb6a000700 1 -- 172.21.15.138:0/3155111022 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7fdb60022470 con 0x7fdb6c0ffe40 2024-05-27T09:19:11.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.593+0000 7fdb6a000700 1 --2- 172.21.15.138:0/3155111022 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fdb54038490 0x7fdb5403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:11.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.593+0000 7fdb6a000700 1 -- 172.21.15.138:0/3155111022 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdb6004c200 con 0x7fdb6c0ffe40 2024-05-27T09:19:11.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.593+0000 7fdb71000700 1 --2- 172.21.15.138:0/3155111022 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fdb54038490 0x7fdb5403a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:19:11.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.594+0000 7fdb71000700 1 --2- 172.21.15.138:0/3155111022 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fdb54038490 0x7fdb5403a940 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdb58006fd0 tx=0x7fdb58006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:19:11.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.608+0000 7fdb6a000700 1 -- 172.21.15.138:0/3155111022 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fdb60022790 con 0x7fdb6c0ffe40 2024-05-27T09:19:11.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.902+0000 7fdb74400700 1 -- 172.21.15.138:0/3155111022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdb6c02cc30 con 0x7fdb6c0ffe40 2024-05-27T09:19:11.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.903+0000 7fdb6a000700 1 -- 172.21.15.138:0/3155111022 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fdb60027030 con 0x7fdb6c0ffe40 2024-05-27T09:19:11.904 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:19:11.904 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b966b06e-1c09-11ef-bc9b-c7b262605968","modified":"2024-05-27T09:16:53.014813Z","created":"2024-05-27T09:16:53.014813Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi119","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T09:19:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.906+0000 7fdb74400700 1 -- 172.21.15.138:0/3155111022 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fdb54038490 msgr2=0x7fdb5403a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.906+0000 7fdb74400700 1 --2- 172.21.15.138:0/3155111022 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fdb54038490 0x7fdb5403a940 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdb58006fd0 tx=0x7fdb58006e40 comp rx=0 tx=0).stop 2024-05-27T09:19:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.906+0000 7fdb74400700 1 -- 172.21.15.138:0/3155111022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb6c0ffe40 msgr2=0x7fdb6c197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.906+0000 7fdb74400700 1 --2- 172.21.15.138:0/3155111022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb6c0ffe40 0x7fdb6c197190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdb600092e0 tx=0x7fdb60009e60 comp rx=0 tx=0).stop 2024-05-27T09:19:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.906+0000 7fdb74400700 1 -- 172.21.15.138:0/3155111022 shutdown_connections 2024-05-27T09:19:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.906+0000 7fdb74400700 1 --2- 172.21.15.138:0/3155111022 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fdb54038490 0x7fdb5403a940 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.906+0000 7fdb74400700 1 --2- 172.21.15.138:0/3155111022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb6c0ffe40 0x7fdb6c197190 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.906+0000 7fdb74400700 1 -- 172.21.15.138:0/3155111022 >> 172.21.15.138:0/3155111022 conn(0x7fdb6c0fb3d0 msgr2=0x7fdb6c0fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.906+0000 7fdb74400700 1 -- 172.21.15.138:0/3155111022 shutdown_connections 2024-05-27T09:19:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:11.906+0000 7fdb74400700 1 -- 172.21.15.138:0/3155111022 wait complete. 2024-05-27T09:19:11.908 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-27T09:19:12.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:12 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/3155111022' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:19:13.257 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:13 smithi119 ceph-mon[29082]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:19:13.731 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-27T09:19:13.731 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T09:19:14.115 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:19:14.219 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:19:14.307 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:14 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:14.307 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:14 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:14.308 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:14 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:14.308 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:14 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:14.308 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:14 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:14.308 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:14 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:15.530 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:15 smithi119 ceph-mon[29082]: Deploying daemon prometheus.smithi119 on smithi119 2024-05-27T09:19:15.530 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:15 smithi119 ceph-mon[29082]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:19:16.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.520+0000 7f352e200700 1 -- 172.21.15.138:0/416779449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3528106460 msgr2=0x7f3528106870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:16.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.520+0000 7f352e200700 1 --2- 172.21.15.138:0/416779449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3528106460 0x7f3528106870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3518009910 tx=0x7f3518009c20 comp rx=0 tx=0).stop 2024-05-27T09:19:16.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.520+0000 7f352e200700 1 -- 172.21.15.138:0/416779449 shutdown_connections 2024-05-27T09:19:16.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.520+0000 7f352e200700 1 --2- 172.21.15.138:0/416779449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3528106460 0x7f3528106870 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:16.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.520+0000 7f352e200700 1 -- 172.21.15.138:0/416779449 >> 172.21.15.138:0/416779449 conn(0x7f35280756e0 msgr2=0x7f3528077b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:16.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.520+0000 7f352e200700 1 -- 172.21.15.138:0/416779449 shutdown_connections 2024-05-27T09:19:16.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.520+0000 7f352e200700 1 -- 172.21.15.138:0/416779449 wait complete. 2024-05-27T09:19:16.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.522+0000 7f352e200700 1 Processor -- start 2024-05-27T09:19:16.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.522+0000 7f352e200700 1 -- start start 2024-05-27T09:19:16.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.522+0000 7f352e200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3528106460 0x7f352819b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:16.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.522+0000 7f352e200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352819baa0 con 0x7f3528106460 2024-05-27T09:19:16.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.523+0000 7f3527400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3528106460 0x7f352819b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:19:16.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.523+0000 7f3527400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3528106460 0x7f352819b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.138:56444/0 (socket says 172.21.15.138:56444) 2024-05-27T09:19:16.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.523+0000 7f3527400700 1 -- 172.21.15.138:0/2245120112 learned_addr learned my addr 172.21.15.138:0/2245120112 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:19:16.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.524+0000 7f3527400700 1 -- 172.21.15.138:0/2245120112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35180095f0 con 0x7f3528106460 2024-05-27T09:19:16.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.524+0000 7f3527400700 1 --2- 172.21.15.138:0/2245120112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3528106460 0x7f352819b560 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3518005b40 tx=0x7f3518004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:19:16.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.525+0000 7f351fe00700 1 -- 172.21.15.138:0/2245120112 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f351801d070 con 0x7f3528106460 2024-05-27T09:19:16.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.525+0000 7f352e200700 1 -- 172.21.15.138:0/2245120112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352819bca0 con 0x7f3528106460 2024-05-27T09:19:16.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.525+0000 7f351fe00700 1 -- 172.21.15.138:0/2245120112 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f351800be10 con 0x7f3528106460 2024-05-27T09:19:16.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.525+0000 7f351fe00700 1 -- 172.21.15.138:0/2245120112 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f351800f460 con 0x7f3528106460 2024-05-27T09:19:16.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.525+0000 7f352e200700 1 -- 172.21.15.138:0/2245120112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f352819c140 con 0x7f3528106460 2024-05-27T09:19:16.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.526+0000 7f352e200700 1 -- 172.21.15.138:0/2245120112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3528195130 con 0x7f3528106460 2024-05-27T09:19:16.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.526+0000 7f351fe00700 1 -- 172.21.15.138:0/2245120112 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f35180227e0 con 0x7f3528106460 2024-05-27T09:19:16.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.527+0000 7f351fe00700 1 --2- 172.21.15.138:0/2245120112 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f350c0384e0 0x7f350c03a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:16.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.527+0000 7f351fe00700 1 -- 172.21.15.138:0/2245120112 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f351804c390 con 0x7f3528106460 2024-05-27T09:19:16.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.528+0000 7f3526a00700 1 --2- 172.21.15.138:0/2245120112 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f350c0384e0 0x7f350c03a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:19:16.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.529+0000 7f3526a00700 1 --2- 172.21.15.138:0/2245120112 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f350c0384e0 0x7f350c03a990 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3514006fd0 tx=0x7f3514006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:19:16.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.537+0000 7f351fe00700 1 -- 172.21.15.138:0/2245120112 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f351802ba00 con 0x7f3528106460 2024-05-27T09:19:16.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.829+0000 7f352e200700 1 -- 172.21.15.138:0/2245120112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f352802cc30 con 0x7f3528106460 2024-05-27T09:19:16.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.829+0000 7f351fe00700 1 -- 172.21.15.138:0/2245120112 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f3518027030 con 0x7f3528106460 2024-05-27T09:19:16.830 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:19:16.831 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b966b06e-1c09-11ef-bc9b-c7b262605968","modified":"2024-05-27T09:16:53.014813Z","created":"2024-05-27T09:16:53.014813Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi119","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T09:19:16.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.832+0000 7f352e200700 1 -- 172.21.15.138:0/2245120112 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f350c0384e0 msgr2=0x7f350c03a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:16.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.832+0000 7f352e200700 1 --2- 172.21.15.138:0/2245120112 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f350c0384e0 0x7f350c03a990 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3514006fd0 tx=0x7f3514006e40 comp rx=0 tx=0).stop 2024-05-27T09:19:16.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.832+0000 7f352e200700 1 -- 172.21.15.138:0/2245120112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3528106460 msgr2=0x7f352819b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:16.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.832+0000 7f352e200700 1 --2- 172.21.15.138:0/2245120112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3528106460 0x7f352819b560 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3518005b40 tx=0x7f3518004740 comp rx=0 tx=0).stop 2024-05-27T09:19:16.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.832+0000 7f352e200700 1 -- 172.21.15.138:0/2245120112 shutdown_connections 2024-05-27T09:19:16.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.832+0000 7f352e200700 1 --2- 172.21.15.138:0/2245120112 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f350c0384e0 0x7f350c03a990 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:16.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.832+0000 7f352e200700 1 --2- 172.21.15.138:0/2245120112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3528106460 0x7f352819b560 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:16.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.832+0000 7f352e200700 1 -- 172.21.15.138:0/2245120112 >> 172.21.15.138:0/2245120112 conn(0x7f35280756e0 msgr2=0x7f3528077310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:16.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.833+0000 7f352e200700 1 -- 172.21.15.138:0/2245120112 shutdown_connections 2024-05-27T09:19:16.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:16.833+0000 7f352e200700 1 -- 172.21.15.138:0/2245120112 wait complete. 2024-05-27T09:19:16.835 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-27T09:19:17.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:17 smithi119 ceph-mon[29082]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:19:17.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:17 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/2245120112' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:19:17.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:17 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:18.516 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-27T09:19:18.517 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T09:19:18.892 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:19:18.992 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:19:19.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:19 smithi119 ceph-mon[29082]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:19:21.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.043+0000 7fe257200700 1 -- 172.21.15.138:0/1524585270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe258100740 msgr2=0x7fe258102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:21.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.043+0000 7fe257200700 1 --2- 172.21.15.138:0/1524585270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe258100740 0x7fe258102b20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe248009910 tx=0x7fe248009c20 comp rx=0 tx=0).stop 2024-05-27T09:19:21.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.043+0000 7fe257200700 1 -- 172.21.15.138:0/1524585270 shutdown_connections 2024-05-27T09:19:21.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.043+0000 7fe257200700 1 --2- 172.21.15.138:0/1524585270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe258100740 0x7fe258102b20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:21.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.043+0000 7fe257200700 1 -- 172.21.15.138:0/1524585270 >> 172.21.15.138:0/1524585270 conn(0x7fe2580fa0e0 msgr2=0x7fe2580fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:21.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.043+0000 7fe257200700 1 -- 172.21.15.138:0/1524585270 shutdown_connections 2024-05-27T09:19:21.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.043+0000 7fe257200700 1 -- 172.21.15.138:0/1524585270 wait complete. 2024-05-27T09:19:21.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.044+0000 7fe257200700 1 Processor -- start 2024-05-27T09:19:21.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.045+0000 7fe257200700 1 -- start start 2024-05-27T09:19:21.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.045+0000 7fe257200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe258100740 0x7fe2581000a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:21.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.045+0000 7fe257200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2581005e0 con 0x7fe258100740 2024-05-27T09:19:21.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.046+0000 7fe255e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe258100740 0x7fe2581000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:19:21.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.046+0000 7fe255e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe258100740 0x7fe2581000a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.138:45282/0 (socket says 172.21.15.138:45282) 2024-05-27T09:19:21.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.046+0000 7fe255e00700 1 -- 172.21.15.138:0/1376410496 learned_addr learned my addr 172.21.15.138:0/1376410496 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:19:21.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.046+0000 7fe255e00700 1 -- 172.21.15.138:0/1376410496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2480095f0 con 0x7fe258100740 2024-05-27T09:19:21.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.046+0000 7fe255e00700 1 --2- 172.21.15.138:0/1376410496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe258100740 0x7fe2581000a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe2480092e0 tx=0x7fe248009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:19:21.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.047+0000 7fe24ea00700 1 -- 172.21.15.138:0/1376410496 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe24801d070 con 0x7fe258100740 2024-05-27T09:19:21.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.047+0000 7fe24ea00700 1 -- 172.21.15.138:0/1376410496 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe24800be10 con 0x7fe258100740 2024-05-27T09:19:21.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.047+0000 7fe257200700 1 -- 172.21.15.138:0/1376410496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2580fe6f0 con 0x7fe258100740 2024-05-27T09:19:21.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.047+0000 7fe24ea00700 1 -- 172.21.15.138:0/1376410496 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe24800f460 con 0x7fe258100740 2024-05-27T09:19:21.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.047+0000 7fe257200700 1 -- 172.21.15.138:0/1376410496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2580feb90 con 0x7fe258100740 2024-05-27T09:19:21.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.048+0000 7fe257200700 1 -- 172.21.15.138:0/1376410496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe258062380 con 0x7fe258100740 2024-05-27T09:19:21.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.048+0000 7fe24ea00700 1 -- 172.21.15.138:0/1376410496 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7fe248022470 con 0x7fe258100740 2024-05-27T09:19:21.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.049+0000 7fe24ea00700 1 --2- 172.21.15.138:0/1376410496 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fe23c0384e0 0x7fe23c03a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:21.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.049+0000 7fe24ea00700 1 -- 172.21.15.138:0/1376410496 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe24804c510 con 0x7fe258100740 2024-05-27T09:19:21.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.049+0000 7fe255400700 1 --2- 172.21.15.138:0/1376410496 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fe23c0384e0 0x7fe23c03a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:19:21.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.050+0000 7fe255400700 1 --2- 172.21.15.138:0/1376410496 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fe23c0384e0 0x7fe23c03a990 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe244006fd0 tx=0x7fe244006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:19:21.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.055+0000 7fe24ea00700 1 -- 172.21.15.138:0/1376410496 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe24800b960 con 0x7fe258100740 2024-05-27T09:19:21.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.356+0000 7fe257200700 1 -- 172.21.15.138:0/1376410496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe25818e310 con 0x7fe258100740 2024-05-27T09:19:21.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.357+0000 7fe24ea00700 1 -- 172.21.15.138:0/1376410496 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fe248027070 con 0x7fe258100740 2024-05-27T09:19:21.358 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:19:21.359 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b966b06e-1c09-11ef-bc9b-c7b262605968","modified":"2024-05-27T09:16:53.014813Z","created":"2024-05-27T09:16:53.014813Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi119","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T09:19:21.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.361+0000 7fe257200700 1 -- 172.21.15.138:0/1376410496 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fe23c0384e0 msgr2=0x7fe23c03a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:21.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.361+0000 7fe257200700 1 --2- 172.21.15.138:0/1376410496 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fe23c0384e0 0x7fe23c03a990 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe244006fd0 tx=0x7fe244006e40 comp rx=0 tx=0).stop 2024-05-27T09:19:21.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.361+0000 7fe257200700 1 -- 172.21.15.138:0/1376410496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe258100740 msgr2=0x7fe2581000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:21.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.361+0000 7fe257200700 1 --2- 172.21.15.138:0/1376410496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe258100740 0x7fe2581000a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe2480092e0 tx=0x7fe248009e60 comp rx=0 tx=0).stop 2024-05-27T09:19:21.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.361+0000 7fe257200700 1 -- 172.21.15.138:0/1376410496 shutdown_connections 2024-05-27T09:19:21.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.361+0000 7fe257200700 1 --2- 172.21.15.138:0/1376410496 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fe23c0384e0 0x7fe23c03a990 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:21.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.361+0000 7fe257200700 1 --2- 172.21.15.138:0/1376410496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe258100740 0x7fe2581000a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:21.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.361+0000 7fe257200700 1 -- 172.21.15.138:0/1376410496 >> 172.21.15.138:0/1376410496 conn(0x7fe2580fa0e0 msgr2=0x7fe2580fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:21.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.361+0000 7fe257200700 1 -- 172.21.15.138:0/1376410496 shutdown_connections 2024-05-27T09:19:21.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:21.361+0000 7fe257200700 1 -- 172.21.15.138:0/1376410496 wait complete. 2024-05-27T09:19:21.363 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-27T09:19:21.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:21 smithi119 ceph-mon[29082]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:19:22.596 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:22 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/1376410496' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:19:23.184 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-27T09:19:23.184 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T09:19:23.491 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:23 smithi119 ceph-mon[29082]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:19:23.556 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:19:23.656 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:19:25.631 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:25 smithi119 ceph-mon[29082]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:19:26.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.168+0000 7f2425a00700 1 -- 172.21.15.138:0/2875660797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24200ffc90 msgr2=0x7f24201000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:26.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.168+0000 7f2425a00700 1 --2- 172.21.15.138:0/2875660797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24200ffc90 0x7f24201000a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2410009910 tx=0x7f2410009c20 comp rx=0 tx=0).stop 2024-05-27T09:19:26.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.169+0000 7f2425a00700 1 -- 172.21.15.138:0/2875660797 shutdown_connections 2024-05-27T09:19:26.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.169+0000 7f2425a00700 1 --2- 172.21.15.138:0/2875660797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24200ffc90 0x7f24201000a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:26.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.169+0000 7f2425a00700 1 -- 172.21.15.138:0/2875660797 >> 172.21.15.138:0/2875660797 conn(0x7f24200fb220 msgr2=0x7f24200fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:26.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.169+0000 7f2425a00700 1 -- 172.21.15.138:0/2875660797 shutdown_connections 2024-05-27T09:19:26.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.169+0000 7f2425a00700 1 -- 172.21.15.138:0/2875660797 wait complete. 2024-05-27T09:19:26.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.170+0000 7f2425a00700 1 Processor -- start 2024-05-27T09:19:26.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.170+0000 7f2425a00700 1 -- start start 2024-05-27T09:19:26.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.171+0000 7f2425a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24200ffc90 0x7f2420197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:26.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.171+0000 7f2425a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2420197890 con 0x7f24200ffc90 2024-05-27T09:19:26.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.171+0000 7f241fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24200ffc90 0x7f2420197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:19:26.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.171+0000 7f241fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24200ffc90 0x7f2420197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.138:45314/0 (socket says 172.21.15.138:45314) 2024-05-27T09:19:26.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.171+0000 7f241fe00700 1 -- 172.21.15.138:0/2042527289 learned_addr learned my addr 172.21.15.138:0/2042527289 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:19:26.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.172+0000 7f241fe00700 1 -- 172.21.15.138:0/2042527289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24100095f0 con 0x7f24200ffc90 2024-05-27T09:19:26.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.172+0000 7f241fe00700 1 --2- 172.21.15.138:0/2042527289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24200ffc90 0x7f2420197350 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2410005b40 tx=0x7f2410004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:19:26.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.173+0000 7f241cc00700 1 -- 172.21.15.138:0/2042527289 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f241001d070 con 0x7f24200ffc90 2024-05-27T09:19:26.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.173+0000 7f2425a00700 1 -- 172.21.15.138:0/2042527289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2420197a90 con 0x7f24200ffc90 2024-05-27T09:19:26.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.173+0000 7f2425a00700 1 -- 172.21.15.138:0/2042527289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2420197f30 con 0x7f24200ffc90 2024-05-27T09:19:26.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.173+0000 7f241cc00700 1 -- 172.21.15.138:0/2042527289 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f241000be10 con 0x7f24200ffc90 2024-05-27T09:19:26.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.173+0000 7f241cc00700 1 -- 172.21.15.138:0/2042527289 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f241000f460 con 0x7f24200ffc90 2024-05-27T09:19:26.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.174+0000 7f2425a00700 1 -- 172.21.15.138:0/2042527289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2400005320 con 0x7f24200ffc90 2024-05-27T09:19:26.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.175+0000 7f241cc00700 1 -- 172.21.15.138:0/2042527289 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f241000b960 con 0x7f24200ffc90 2024-05-27T09:19:26.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.175+0000 7f241cc00700 1 --2- 172.21.15.138:0/2042527289 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f24040384e0 0x7f240403a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:26.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.175+0000 7f241cc00700 1 -- 172.21.15.138:0/2042527289 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f241004c4c0 con 0x7f24200ffc90 2024-05-27T09:19:26.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.176+0000 7f241f400700 1 --2- 172.21.15.138:0/2042527289 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f24040384e0 0x7f240403a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:19:26.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.178+0000 7f241f400700 1 --2- 172.21.15.138:0/2042527289 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f24040384e0 0x7f240403a990 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f240c006fd0 tx=0x7f240c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:19:26.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.190+0000 7f241cc00700 1 -- 172.21.15.138:0/2042527289 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f241000f5c0 con 0x7f24200ffc90 2024-05-27T09:19:26.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.481+0000 7f2425a00700 1 -- 172.21.15.138:0/2042527289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2400005190 con 0x7f24200ffc90 2024-05-27T09:19:26.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.482+0000 7f241cc00700 1 -- 172.21.15.138:0/2042527289 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f2410027020 con 0x7f24200ffc90 2024-05-27T09:19:26.483 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:19:26.483 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b966b06e-1c09-11ef-bc9b-c7b262605968","modified":"2024-05-27T09:16:53.014813Z","created":"2024-05-27T09:16:53.014813Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi119","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T09:19:26.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.485+0000 7f2425a00700 1 -- 172.21.15.138:0/2042527289 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f24040384e0 msgr2=0x7f240403a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:26.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.485+0000 7f2425a00700 1 --2- 172.21.15.138:0/2042527289 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f24040384e0 0x7f240403a990 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f240c006fd0 tx=0x7f240c006e40 comp rx=0 tx=0).stop 2024-05-27T09:19:26.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.485+0000 7f2425a00700 1 -- 172.21.15.138:0/2042527289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24200ffc90 msgr2=0x7f2420197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:26.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.485+0000 7f2425a00700 1 --2- 172.21.15.138:0/2042527289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24200ffc90 0x7f2420197350 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2410005b40 tx=0x7f2410004740 comp rx=0 tx=0).stop 2024-05-27T09:19:26.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.485+0000 7f2425a00700 1 -- 172.21.15.138:0/2042527289 shutdown_connections 2024-05-27T09:19:26.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.485+0000 7f2425a00700 1 --2- 172.21.15.138:0/2042527289 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7f24040384e0 0x7f240403a990 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:26.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.485+0000 7f2425a00700 1 --2- 172.21.15.138:0/2042527289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24200ffc90 0x7f2420197350 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:26.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.485+0000 7f2425a00700 1 -- 172.21.15.138:0/2042527289 >> 172.21.15.138:0/2042527289 conn(0x7f24200fb220 msgr2=0x7f24200fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:26.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.485+0000 7f2425a00700 1 -- 172.21.15.138:0/2042527289 shutdown_connections 2024-05-27T09:19:26.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:26.485+0000 7f2425a00700 1 -- 172.21.15.138:0/2042527289 wait complete. 2024-05-27T09:19:26.487 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-27T09:19:27.619 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:27 smithi119 ceph-mon[29082]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:19:27.619 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:27 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/2042527289' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:19:27.619 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:27 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:27.620 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:27 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:27.620 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:27 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:27.620 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:27 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-05-27T09:19:27.620 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:27 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:28.166 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-27T09:19:28.167 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T09:19:28.548 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:19:28.661 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:19:28.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:28 smithi119 ceph-mon[29082]: from='mgr.14164 172.21.15.119:0/3854764723' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-05-27T09:19:28.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:28 smithi119 ceph-mon[29082]: mgrmap e14: smithi119.ncwmog(active, since 75s) 2024-05-27T09:19:30.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.910+0000 7fc923c00700 1 -- 172.21.15.138:0/1138125348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc91c102040 msgr2=0x7fc91c102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:30.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.910+0000 7fc923c00700 1 --2- 172.21.15.138:0/1138125348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc91c102040 0x7fc91c102450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc910009910 tx=0x7fc910009c20 comp rx=0 tx=0).stop 2024-05-27T09:19:30.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.910+0000 7fc923c00700 1 -- 172.21.15.138:0/1138125348 shutdown_connections 2024-05-27T09:19:30.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.910+0000 7fc923c00700 1 --2- 172.21.15.138:0/1138125348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc91c102040 0x7fc91c102450 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:30.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.910+0000 7fc923c00700 1 -- 172.21.15.138:0/1138125348 >> 172.21.15.138:0/1138125348 conn(0x7fc91c0fd700 msgr2=0x7fc91c0ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:30.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.910+0000 7fc923c00700 1 -- 172.21.15.138:0/1138125348 shutdown_connections 2024-05-27T09:19:30.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.910+0000 7fc923c00700 1 -- 172.21.15.138:0/1138125348 wait complete. 2024-05-27T09:19:30.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.912+0000 7fc923c00700 1 Processor -- start 2024-05-27T09:19:30.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.912+0000 7fc923c00700 1 -- start start 2024-05-27T09:19:30.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.912+0000 7fc923c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc91c102040 0x7fc91c1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:30.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.912+0000 7fc923c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c197710 con 0x7fc91c102040 2024-05-27T09:19:30.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.913+0000 7fc921200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc91c102040 0x7fc91c1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:19:30.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.913+0000 7fc921200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc91c102040 0x7fc91c1971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.138:40666/0 (socket says 172.21.15.138:40666) 2024-05-27T09:19:30.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.913+0000 7fc921200700 1 -- 172.21.15.138:0/2018193289 learned_addr learned my addr 172.21.15.138:0/2018193289 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:19:30.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.913+0000 7fc921200700 1 -- 172.21.15.138:0/2018193289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9100095f0 con 0x7fc91c102040 2024-05-27T09:19:30.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.914+0000 7fc921200700 1 --2- 172.21.15.138:0/2018193289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc91c102040 0x7fc91c1971d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc910005950 tx=0x7fc910009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:19:30.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.914+0000 7fc919600700 1 -- 172.21.15.138:0/2018193289 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc91001d070 con 0x7fc91c102040 2024-05-27T09:19:30.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.914+0000 7fc919600700 1 -- 172.21.15.138:0/2018193289 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc91000be10 con 0x7fc91c102040 2024-05-27T09:19:30.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.914+0000 7fc919600700 1 -- 172.21.15.138:0/2018193289 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc91000f460 con 0x7fc91c102040 2024-05-27T09:19:30.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.914+0000 7fc923c00700 1 -- 172.21.15.138:0/2018193289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc91c197910 con 0x7fc91c102040 2024-05-27T09:19:30.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.914+0000 7fc923c00700 1 -- 172.21.15.138:0/2018193289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc91c197db0 con 0x7fc91c102040 2024-05-27T09:19:30.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.915+0000 7fc923c00700 1 -- 172.21.15.138:0/2018193289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc91c190d20 con 0x7fc91c102040 2024-05-27T09:19:30.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.916+0000 7fc919600700 1 -- 172.21.15.138:0/2018193289 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 14) v1 ==== 45155+0+0 (secure 0 0 0) 0x7fc9100227c0 con 0x7fc91c102040 2024-05-27T09:19:30.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.916+0000 7fc919600700 1 --2- 172.21.15.138:0/2018193289 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fc904038530 0x7fc90403a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:30.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.916+0000 7fc919600700 1 -- 172.21.15.138:0/2018193289 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc91004c460 con 0x7fc91c102040 2024-05-27T09:19:30.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.916+0000 7fc91be00700 1 -- 172.21.15.138:0/2018193289 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fc904038530 msgr2=0x7fc90403a9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2118907438 2024-05-27T09:19:30.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.917+0000 7fc91be00700 1 --2- 172.21.15.138:0/2018193289 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fc904038530 0x7fc90403a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T09:19:30.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:30.926+0000 7fc919600700 1 -- 172.21.15.138:0/2018193289 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc91002ba00 con 0x7fc91c102040 2024-05-27T09:19:31.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:31.118+0000 7fc91be00700 1 -- 172.21.15.138:0/2018193289 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fc904038530 msgr2=0x7fc90403a9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2118907438 2024-05-27T09:19:31.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:31.118+0000 7fc91be00700 1 --2- 172.21.15.138:0/2018193289 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fc904038530 0x7fc90403a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T09:19:31.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:31.223+0000 7fc923c00700 1 -- 172.21.15.138:0/2018193289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc91c02cc30 con 0x7fc91c102040 2024-05-27T09:19:31.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:31.224+0000 7fc919600700 1 -- 172.21.15.138:0/2018193289 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fc910027030 con 0x7fc91c102040 2024-05-27T09:19:31.225 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:19:31.225 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b966b06e-1c09-11ef-bc9b-c7b262605968","modified":"2024-05-27T09:16:53.014813Z","created":"2024-05-27T09:16:53.014813Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi119","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T09:19:31.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:31.227+0000 7fc923c00700 1 -- 172.21.15.138:0/2018193289 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fc904038530 msgr2=0x7fc90403a9e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T09:19:31.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:31.227+0000 7fc923c00700 1 --2- 172.21.15.138:0/2018193289 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fc904038530 0x7fc90403a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:31.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:31.227+0000 7fc923c00700 1 -- 172.21.15.138:0/2018193289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc91c102040 msgr2=0x7fc91c1971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:31.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:31.227+0000 7fc923c00700 1 --2- 172.21.15.138:0/2018193289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc91c102040 0x7fc91c1971d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc910005950 tx=0x7fc910009e60 comp rx=0 tx=0).stop 2024-05-27T09:19:31.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:31.227+0000 7fc923c00700 1 -- 172.21.15.138:0/2018193289 shutdown_connections 2024-05-27T09:19:31.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:31.227+0000 7fc923c00700 1 --2- 172.21.15.138:0/2018193289 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fc904038530 0x7fc90403a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:31.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:31.227+0000 7fc923c00700 1 --2- 172.21.15.138:0/2018193289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc91c102040 0x7fc91c1971d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:31.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:31.227+0000 7fc923c00700 1 -- 172.21.15.138:0/2018193289 >> 172.21.15.138:0/2018193289 conn(0x7fc91c0fd700 msgr2=0x7fc91c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:31.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:31.227+0000 7fc923c00700 1 -- 172.21.15.138:0/2018193289 shutdown_connections 2024-05-27T09:19:31.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:31.228+0000 7fc923c00700 1 -- 172.21.15.138:0/2018193289 wait complete. 2024-05-27T09:19:31.229 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-27T09:19:31.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:31 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/2018193289' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:19:33.096 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-27T09:19:33.097 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T09:19:33.475 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:19:33.574 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:19:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.799+0000 7fd1bfc00700 1 -- 172.21.15.138:0/219941369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1b8106440 msgr2=0x7fd1b8106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.799+0000 7fd1bfc00700 1 --2- 172.21.15.138:0/219941369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1b8106440 0x7fd1b8106850 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009910 tx=0x7fd1ac009c20 comp rx=0 tx=0).stop 2024-05-27T09:19:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.799+0000 7fd1bfc00700 1 -- 172.21.15.138:0/219941369 shutdown_connections 2024-05-27T09:19:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.799+0000 7fd1bfc00700 1 --2- 172.21.15.138:0/219941369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1b8106440 0x7fd1b8106850 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.799+0000 7fd1bfc00700 1 -- 172.21.15.138:0/219941369 >> 172.21.15.138:0/219941369 conn(0x7fd1b80754e0 msgr2=0x7fd1b8077930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.799+0000 7fd1bfc00700 1 -- 172.21.15.138:0/219941369 shutdown_connections 2024-05-27T09:19:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.799+0000 7fd1bfc00700 1 -- 172.21.15.138:0/219941369 wait complete. 2024-05-27T09:19:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.801+0000 7fd1bfc00700 1 Processor -- start 2024-05-27T09:19:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.801+0000 7fd1bfc00700 1 -- start start 2024-05-27T09:19:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.801+0000 7fd1bfc00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1b8106440 0x7fd1b819b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.801+0000 7fd1bfc00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b819bb00 con 0x7fd1b8106440 2024-05-27T09:19:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.802+0000 7fd1bd200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1b8106440 0x7fd1b819b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:19:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.802+0000 7fd1bd200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1b8106440 0x7fd1b819b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.138:40676/0 (socket says 172.21.15.138:40676) 2024-05-27T09:19:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.802+0000 7fd1bd200700 1 -- 172.21.15.138:0/497478282 learned_addr learned my addr 172.21.15.138:0/497478282 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:19:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.802+0000 7fd1bd200700 1 -- 172.21.15.138:0/497478282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac0095f0 con 0x7fd1b8106440 2024-05-27T09:19:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.803+0000 7fd1bd200700 1 --2- 172.21.15.138:0/497478282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1b8106440 0x7fd1b819b5c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac005950 tx=0x7fd1ac009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:19:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.803+0000 7fd1b5600700 1 -- 172.21.15.138:0/497478282 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1ac01d070 con 0x7fd1b8106440 2024-05-27T09:19:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.803+0000 7fd1bfc00700 1 -- 172.21.15.138:0/497478282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b819bd00 con 0x7fd1b8106440 2024-05-27T09:19:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.803+0000 7fd1b5600700 1 -- 172.21.15.138:0/497478282 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd1ac00be10 con 0x7fd1b8106440 2024-05-27T09:19:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.803+0000 7fd1b5600700 1 -- 172.21.15.138:0/497478282 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1ac022ea0 con 0x7fd1b8106440 2024-05-27T09:19:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.803+0000 7fd1bfc00700 1 -- 172.21.15.138:0/497478282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b819c1a0 con 0x7fd1b8106440 2024-05-27T09:19:35.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.804+0000 7fd1bfc00700 1 -- 172.21.15.138:0/497478282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b8195120 con 0x7fd1b8106440 2024-05-27T09:19:35.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.805+0000 7fd1b5600700 1 -- 172.21.15.138:0/497478282 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 14) v1 ==== 45155+0+0 (secure 0 0 0) 0x7fd1ac00b960 con 0x7fd1b8106440 2024-05-27T09:19:35.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.805+0000 7fd1b5600700 1 --2- 172.21.15.138:0/497478282 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fd1a0038490 0x7fd1a003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:35.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.805+0000 7fd1b5600700 1 -- 172.21.15.138:0/497478282 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd1ac04c3e0 con 0x7fd1b8106440 2024-05-27T09:19:35.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.805+0000 7fd1b7e00700 1 -- 172.21.15.138:0/497478282 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fd1a0038490 msgr2=0x7fd1a003a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2118907438 2024-05-27T09:19:35.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.806+0000 7fd1b7e00700 1 --2- 172.21.15.138:0/497478282 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fd1a0038490 0x7fd1a003a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T09:19:35.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:35.820+0000 7fd1b5600700 1 -- 172.21.15.138:0/497478282 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd1ac02a780 con 0x7fd1b8106440 2024-05-27T09:19:36.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:36.007+0000 7fd1b7e00700 1 -- 172.21.15.138:0/497478282 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fd1a0038490 msgr2=0x7fd1a003a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2118907438 2024-05-27T09:19:36.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:36.007+0000 7fd1b7e00700 1 --2- 172.21.15.138:0/497478282 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fd1a0038490 0x7fd1a003a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T09:19:36.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:36.120+0000 7fd1bfc00700 1 -- 172.21.15.138:0/497478282 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd1b802cc30 con 0x7fd1b8106440 2024-05-27T09:19:36.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:36.121+0000 7fd1b5600700 1 -- 172.21.15.138:0/497478282 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fd1ac027030 con 0x7fd1b8106440 2024-05-27T09:19:36.122 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:19:36.122 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b966b06e-1c09-11ef-bc9b-c7b262605968","modified":"2024-05-27T09:16:53.014813Z","created":"2024-05-27T09:16:53.014813Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi119","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T09:19:36.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:36.124+0000 7fd1bfc00700 1 -- 172.21.15.138:0/497478282 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fd1a0038490 msgr2=0x7fd1a003a940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T09:19:36.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:36.124+0000 7fd1bfc00700 1 --2- 172.21.15.138:0/497478282 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fd1a0038490 0x7fd1a003a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:36.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:36.124+0000 7fd1bfc00700 1 -- 172.21.15.138:0/497478282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1b8106440 msgr2=0x7fd1b819b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:36.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:36.124+0000 7fd1bfc00700 1 --2- 172.21.15.138:0/497478282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1b8106440 0x7fd1b819b5c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac005950 tx=0x7fd1ac009e60 comp rx=0 tx=0).stop 2024-05-27T09:19:36.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:36.124+0000 7fd1bfc00700 1 -- 172.21.15.138:0/497478282 shutdown_connections 2024-05-27T09:19:36.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:36.124+0000 7fd1bfc00700 1 --2- 172.21.15.138:0/497478282 >> [v2:172.21.15.119:6800/2118907438,v1:172.21.15.119:6801/2118907438] conn(0x7fd1a0038490 0x7fd1a003a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:36.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:36.124+0000 7fd1bfc00700 1 --2- 172.21.15.138:0/497478282 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1b8106440 0x7fd1b819b5c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:36.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:36.124+0000 7fd1bfc00700 1 -- 172.21.15.138:0/497478282 >> 172.21.15.138:0/497478282 conn(0x7fd1b80754e0 msgr2=0x7fd1b8077140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:36.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:36.124+0000 7fd1bfc00700 1 -- 172.21.15.138:0/497478282 shutdown_connections 2024-05-27T09:19:36.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:36.125+0000 7fd1bfc00700 1 -- 172.21.15.138:0/497478282 wait complete. 2024-05-27T09:19:36.127 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-27T09:19:36.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:36 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/497478282' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:19:36.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:36 smithi119 ceph-mon[29082]: Active manager daemon smithi119.ncwmog restarted 2024-05-27T09:19:36.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:36 smithi119 ceph-mon[29082]: Activating manager daemon smithi119.ncwmog 2024-05-27T09:19:36.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:36 smithi119 ceph-mon[29082]: osdmap e5: 0 total, 0 up, 0 in 2024-05-27T09:19:36.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:36 smithi119 ceph-mon[29082]: mgrmap e15: smithi119.ncwmog(active, starting, since 0.0054289s) 2024-05-27T09:19:36.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:36 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi119"}]: dispatch 2024-05-27T09:19:37.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:37 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr metadata", "who": "smithi119.ncwmog", "id": "smithi119.ncwmog"}]: dispatch 2024-05-27T09:19:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:37 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T09:19:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:37 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T09:19:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:37 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T09:19:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:37 smithi119 ceph-mon[29082]: Manager daemon smithi119.ncwmog is now available 2024-05-27T09:19:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:37 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:19:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:37 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:19:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:37 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:37 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:19:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:37 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi119.ncwmog/mirror_snapshot_schedule"}]: dispatch 2024-05-27T09:19:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:37 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi119.ncwmog/trash_purge_schedule"}]: dispatch 2024-05-27T09:19:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:37 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:37.968 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-27T09:19:37.968 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T09:19:38.313 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:19:38.417 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:19:38.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:38 smithi119 ceph-mon[29082]: mgrmap e16: smithi119.ncwmog(active, since 1.02648s) 2024-05-27T09:19:38.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:38 smithi119 ceph-mon[29082]: [27/May/2024:09:19:37] ENGINE Bus STARTING 2024-05-27T09:19:38.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:38 smithi119 ceph-mon[29082]: [27/May/2024:09:19:37] ENGINE Serving on http://172.21.15.119:8765 2024-05-27T09:19:38.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:38 smithi119 ceph-mon[29082]: [27/May/2024:09:19:37] ENGINE Serving on https://172.21.15.119:7150 2024-05-27T09:19:38.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:38 smithi119 ceph-mon[29082]: [27/May/2024:09:19:37] ENGINE Bus STARTED 2024-05-27T09:19:38.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:38 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:40.168 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:39 smithi119 ceph-mon[29082]: mgrmap e17: smithi119.ncwmog(active, since 2s) 2024-05-27T09:19:40.168 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:39 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:40.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.371+0000 7fe224c00700 1 -- 172.21.15.138:0/3162573308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2200ffe40 msgr2=0x7fe220100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:40.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.371+0000 7fe224c00700 1 --2- 172.21.15.138:0/3162573308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2200ffe40 0x7fe220100250 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe210009910 tx=0x7fe210009c20 comp rx=0 tx=0).stop 2024-05-27T09:19:40.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.371+0000 7fe224c00700 1 -- 172.21.15.138:0/3162573308 shutdown_connections 2024-05-27T09:19:40.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.371+0000 7fe224c00700 1 --2- 172.21.15.138:0/3162573308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2200ffe40 0x7fe220100250 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:40.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.371+0000 7fe224c00700 1 -- 172.21.15.138:0/3162573308 >> 172.21.15.138:0/3162573308 conn(0x7fe2200fb3d0 msgr2=0x7fe2200fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:40.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.372+0000 7fe224c00700 1 -- 172.21.15.138:0/3162573308 shutdown_connections 2024-05-27T09:19:40.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.372+0000 7fe224c00700 1 -- 172.21.15.138:0/3162573308 wait complete. 2024-05-27T09:19:40.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.373+0000 7fe224c00700 1 Processor -- start 2024-05-27T09:19:40.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.373+0000 7fe224c00700 1 -- start start 2024-05-27T09:19:40.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.373+0000 7fe224c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2200ffe40 0x7fe220074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:40.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.373+0000 7fe224c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe220075030 con 0x7fe2200ffe40 2024-05-27T09:19:40.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.374+0000 7fe21de00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2200ffe40 0x7fe220074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:19:40.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.374+0000 7fe21de00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2200ffe40 0x7fe220074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.138:52294/0 (socket says 172.21.15.138:52294) 2024-05-27T09:19:40.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.374+0000 7fe21de00700 1 -- 172.21.15.138:0/193136284 learned_addr learned my addr 172.21.15.138:0/193136284 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:19:40.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.374+0000 7fe21de00700 1 -- 172.21.15.138:0/193136284 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2100095f0 con 0x7fe2200ffe40 2024-05-27T09:19:40.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.375+0000 7fe21de00700 1 --2- 172.21.15.138:0/193136284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2200ffe40 0x7fe220074af0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe210005b40 tx=0x7fe210004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:19:40.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.375+0000 7fe216a00700 1 -- 172.21.15.138:0/193136284 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe21001d070 con 0x7fe2200ffe40 2024-05-27T09:19:40.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.375+0000 7fe224c00700 1 -- 172.21.15.138:0/193136284 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe220073140 con 0x7fe2200ffe40 2024-05-27T09:19:40.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.375+0000 7fe216a00700 1 -- 172.21.15.138:0/193136284 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe21000be10 con 0x7fe2200ffe40 2024-05-27T09:19:40.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.375+0000 7fe216a00700 1 -- 172.21.15.138:0/193136284 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe21000f460 con 0x7fe2200ffe40 2024-05-27T09:19:40.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.375+0000 7fe224c00700 1 -- 172.21.15.138:0/193136284 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2200735e0 con 0x7fe2200ffe40 2024-05-27T09:19:40.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.376+0000 7fe224c00700 1 -- 172.21.15.138:0/193136284 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe220062380 con 0x7fe2200ffe40 2024-05-27T09:19:40.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.377+0000 7fe216a00700 1 -- 172.21.15.138:0/193136284 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 45199+0+0 (secure 0 0 0) 0x7fe2100227e0 con 0x7fe2200ffe40 2024-05-27T09:19:40.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.377+0000 7fe216a00700 1 --2- 172.21.15.138:0/193136284 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fe204038580 0x7fe20403aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:40.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.377+0000 7fe216a00700 1 -- 172.21.15.138:0/193136284 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fe21004c520 con 0x7fe2200ffe40 2024-05-27T09:19:40.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.377+0000 7fe21d400700 1 --2- 172.21.15.138:0/193136284 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fe204038580 0x7fe20403aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:19:40.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.378+0000 7fe21d400700 1 --2- 172.21.15.138:0/193136284 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fe204038580 0x7fe20403aa30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe20c006fd0 tx=0x7fe20c006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:19:40.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.387+0000 7fe216a00700 1 -- 172.21.15.138:0/193136284 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe21002b590 con 0x7fe2200ffe40 2024-05-27T09:19:40.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.690+0000 7fe224c00700 1 -- 172.21.15.138:0/193136284 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe22018e480 con 0x7fe2200ffe40 2024-05-27T09:19:40.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.691+0000 7fe216a00700 1 -- 172.21.15.138:0/193136284 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fe210027070 con 0x7fe2200ffe40 2024-05-27T09:19:40.692 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:19:40.692 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b966b06e-1c09-11ef-bc9b-c7b262605968","modified":"2024-05-27T09:16:53.014813Z","created":"2024-05-27T09:16:53.014813Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi119","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T09:19:40.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.694+0000 7fe224c00700 1 -- 172.21.15.138:0/193136284 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fe204038580 msgr2=0x7fe20403aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:40.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.694+0000 7fe224c00700 1 --2- 172.21.15.138:0/193136284 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fe204038580 0x7fe20403aa30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe20c006fd0 tx=0x7fe20c006e40 comp rx=0 tx=0).stop 2024-05-27T09:19:40.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.694+0000 7fe224c00700 1 -- 172.21.15.138:0/193136284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2200ffe40 msgr2=0x7fe220074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:40.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.694+0000 7fe224c00700 1 --2- 172.21.15.138:0/193136284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2200ffe40 0x7fe220074af0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe210005b40 tx=0x7fe210004740 comp rx=0 tx=0).stop 2024-05-27T09:19:40.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.694+0000 7fe224c00700 1 -- 172.21.15.138:0/193136284 shutdown_connections 2024-05-27T09:19:40.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.694+0000 7fe224c00700 1 --2- 172.21.15.138:0/193136284 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fe204038580 0x7fe20403aa30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:40.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.694+0000 7fe224c00700 1 --2- 172.21.15.138:0/193136284 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2200ffe40 0x7fe220074af0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:40.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.694+0000 7fe224c00700 1 -- 172.21.15.138:0/193136284 >> 172.21.15.138:0/193136284 conn(0x7fe2200fb3d0 msgr2=0x7fe2200fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:40.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.695+0000 7fe224c00700 1 -- 172.21.15.138:0/193136284 shutdown_connections 2024-05-27T09:19:40.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:40.695+0000 7fe224c00700 1 -- 172.21.15.138:0/193136284 wait complete. 2024-05-27T09:19:40.697 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-27T09:19:41.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:40 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/193136284' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:19:43.232 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-27T09:19:43.232 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T09:19:44.197 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:19:44.299 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T09:19:45.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:45 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:45.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:45 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:45.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:45 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:45.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:45 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:45.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:45 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:45.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:45 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:45.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:45 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:19:46.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.280+0000 7f0dfe600700 1 -- 172.21.15.138:0/2029733124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0df80ffe40 msgr2=0x7f0df8100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:46.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.280+0000 7f0dfe600700 1 --2- 172.21.15.138:0/2029733124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0df80ffe40 0x7f0df8100250 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0de8009910 tx=0x7f0de8009c20 comp rx=0 tx=0).stop 2024-05-27T09:19:46.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.280+0000 7f0dfe600700 1 -- 172.21.15.138:0/2029733124 shutdown_connections 2024-05-27T09:19:46.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.280+0000 7f0dfe600700 1 --2- 172.21.15.138:0/2029733124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0df80ffe40 0x7f0df8100250 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:46.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.280+0000 7f0dfe600700 1 -- 172.21.15.138:0/2029733124 >> 172.21.15.138:0/2029733124 conn(0x7f0df80fb3d0 msgr2=0x7f0df80fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:46.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.280+0000 7f0dfe600700 1 -- 172.21.15.138:0/2029733124 shutdown_connections 2024-05-27T09:19:46.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.280+0000 7f0dfe600700 1 -- 172.21.15.138:0/2029733124 wait complete. 2024-05-27T09:19:46.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.281+0000 7f0dfe600700 1 Processor -- start 2024-05-27T09:19:46.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.281+0000 7f0dfe600700 1 -- start start 2024-05-27T09:19:46.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.282+0000 7f0dfe600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0df80ffe40 0x7f0df8197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:46.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.282+0000 7f0dfe600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df81976d0 con 0x7f0df80ffe40 2024-05-27T09:19:46.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.282+0000 7f0df7400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0df80ffe40 0x7f0df8197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:19:46.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.282+0000 7f0df7400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0df80ffe40 0x7f0df8197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.138:52312/0 (socket says 172.21.15.138:52312) 2024-05-27T09:19:46.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.282+0000 7f0df7400700 1 -- 172.21.15.138:0/1190011489 learned_addr learned my addr 172.21.15.138:0/1190011489 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:19:46.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.283+0000 7f0df7400700 1 -- 172.21.15.138:0/1190011489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de80095f0 con 0x7f0df80ffe40 2024-05-27T09:19:46.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.283+0000 7f0df7400700 1 --2- 172.21.15.138:0/1190011489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0df80ffe40 0x7f0df8197190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0de80092e0 tx=0x7f0de8009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:19:46.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.284+0000 7f0defe00700 1 -- 172.21.15.138:0/1190011489 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0de801d070 con 0x7f0df80ffe40 2024-05-27T09:19:46.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.284+0000 7f0dfe600700 1 -- 172.21.15.138:0/1190011489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df81978d0 con 0x7f0df80ffe40 2024-05-27T09:19:46.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.284+0000 7f0defe00700 1 -- 172.21.15.138:0/1190011489 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0de800be10 con 0x7f0df80ffe40 2024-05-27T09:19:46.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.284+0000 7f0defe00700 1 -- 172.21.15.138:0/1190011489 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0de800f460 con 0x7f0df80ffe40 2024-05-27T09:19:46.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.284+0000 7f0dfe600700 1 -- 172.21.15.138:0/1190011489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df8197d70 con 0x7f0df80ffe40 2024-05-27T09:19:46.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.285+0000 7f0dfe600700 1 -- 172.21.15.138:0/1190011489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df8190ea0 con 0x7f0df80ffe40 2024-05-27T09:19:46.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.285+0000 7f0defe00700 1 -- 172.21.15.138:0/1190011489 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 45199+0+0 (secure 0 0 0) 0x7f0de800b960 con 0x7f0df80ffe40 2024-05-27T09:19:46.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.285+0000 7f0defe00700 1 --2- 172.21.15.138:0/1190011489 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f0ddc0385d0 0x7f0ddc03aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:46.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.286+0000 7f0defe00700 1 -- 172.21.15.138:0/1190011489 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f0de804c500 con 0x7f0df80ffe40 2024-05-27T09:19:46.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.286+0000 7f0df6a00700 1 --2- 172.21.15.138:0/1190011489 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f0ddc0385d0 0x7f0ddc03aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:19:46.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.287+0000 7f0df6a00700 1 --2- 172.21.15.138:0/1190011489 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f0ddc0385d0 0x7f0ddc03aa80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0de4006fd0 tx=0x7f0de4006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:19:46.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.298+0000 7f0defe00700 1 -- 172.21.15.138:0/1190011489 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0de8022790 con 0x7f0df80ffe40 2024-05-27T09:19:46.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.602+0000 7f0dfe600700 1 -- 172.21.15.138:0/1190011489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0df802cc30 con 0x7f0df80ffe40 2024-05-27T09:19:46.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.603+0000 7f0defe00700 1 -- 172.21.15.138:0/1190011489 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f0de8027070 con 0x7f0df80ffe40 2024-05-27T09:19:46.604 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:19:46.604 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b966b06e-1c09-11ef-bc9b-c7b262605968","modified":"2024-05-27T09:16:53.014813Z","created":"2024-05-27T09:16:53.014813Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi119","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T09:19:46.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.605+0000 7f0dfe600700 1 -- 172.21.15.138:0/1190011489 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f0ddc0385d0 msgr2=0x7f0ddc03aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:46.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.605+0000 7f0dfe600700 1 --2- 172.21.15.138:0/1190011489 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f0ddc0385d0 0x7f0ddc03aa80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0de4006fd0 tx=0x7f0de4006e40 comp rx=0 tx=0).stop 2024-05-27T09:19:46.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.605+0000 7f0dfe600700 1 -- 172.21.15.138:0/1190011489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0df80ffe40 msgr2=0x7f0df8197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:46.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.605+0000 7f0dfe600700 1 --2- 172.21.15.138:0/1190011489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0df80ffe40 0x7f0df8197190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0de80092e0 tx=0x7f0de8009e60 comp rx=0 tx=0).stop 2024-05-27T09:19:46.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.606+0000 7f0dfe600700 1 -- 172.21.15.138:0/1190011489 shutdown_connections 2024-05-27T09:19:46.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.606+0000 7f0dfe600700 1 --2- 172.21.15.138:0/1190011489 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f0ddc0385d0 0x7f0ddc03aa80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:46.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.606+0000 7f0dfe600700 1 --2- 172.21.15.138:0/1190011489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0df80ffe40 0x7f0df8197190 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:46.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.606+0000 7f0dfe600700 1 -- 172.21.15.138:0/1190011489 >> 172.21.15.138:0/1190011489 conn(0x7f0df80fb3d0 msgr2=0x7f0df80fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:46.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.606+0000 7f0dfe600700 1 -- 172.21.15.138:0/1190011489 shutdown_connections 2024-05-27T09:19:46.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:46.606+0000 7f0dfe600700 1 -- 172.21.15.138:0/1190011489 wait complete. 2024-05-27T09:19:46.607 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-27T09:19:47.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:46 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:47.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:46 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:47.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:46 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd/host:smithi119", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:19:47.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:46 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:19:47.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:46 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:19:47.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:46 smithi119 ceph-mon[29082]: Updating smithi119:/etc/ceph/ceph.conf 2024-05-27T09:19:47.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:46 smithi119 ceph-mon[29082]: Updating smithi138:/etc/ceph/ceph.conf 2024-05-27T09:19:47.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:46 smithi119 ceph-mon[29082]: Updating smithi138:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:19:47.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:46 smithi119 ceph-mon[29082]: Updating smithi119:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:19:47.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:46 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/1190011489' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:19:48.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:47 smithi119 ceph-mon[29082]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-05-27T09:19:48.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:47 smithi119 ceph-mon[29082]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-05-27T09:19:48.343 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-27T09:19:48.343 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T09:19:48.690 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:19:49.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:49 smithi119 ceph-mon[29082]: Updating smithi138:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T09:19:49.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:49 smithi119 ceph-mon[29082]: Updating smithi119:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T09:19:49.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:49 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:49.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:49 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:49.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:49 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:49.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:49 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:49.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:49 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:49.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:49 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi138", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-27T09:19:49.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:49 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi138", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-27T09:19:49.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:49 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:19:50.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:50 smithi119 ceph-mon[29082]: Deploying daemon ceph-exporter.smithi138 on smithi138 2024-05-27T09:19:51.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.223+0000 7f0c9da00700 1 -- 172.21.15.138:0/3181812550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98073090 msgr2=0x7f0c98073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:51.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.223+0000 7f0c9da00700 1 --2- 172.21.15.138:0/3181812550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98073090 0x7f0c98073460 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009910 tx=0x7f0c88009c20 comp rx=0 tx=0).stop 2024-05-27T09:19:51.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.223+0000 7f0c9da00700 1 -- 172.21.15.138:0/3181812550 shutdown_connections 2024-05-27T09:19:51.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.223+0000 7f0c9da00700 1 --2- 172.21.15.138:0/3181812550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98073090 0x7f0c98073460 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:51.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.223+0000 7f0c9da00700 1 -- 172.21.15.138:0/3181812550 >> 172.21.15.138:0/3181812550 conn(0x7f0c980fb7a0 msgr2=0x7f0c980fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:51.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.224+0000 7f0c9da00700 1 -- 172.21.15.138:0/3181812550 shutdown_connections 2024-05-27T09:19:51.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.224+0000 7f0c9da00700 1 -- 172.21.15.138:0/3181812550 wait complete. 2024-05-27T09:19:51.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.225+0000 7f0c9da00700 1 Processor -- start 2024-05-27T09:19:51.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.225+0000 7f0c9da00700 1 -- start start 2024-05-27T09:19:51.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.225+0000 7f0c9da00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98073090 0x7f0c98197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:51.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.225+0000 7f0c9da00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c98197890 con 0x7f0c98073090 2024-05-27T09:19:51.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.226+0000 7f0c97e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98073090 0x7f0c98197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:19:51.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.226+0000 7f0c97e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98073090 0x7f0c98197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.138:48092/0 (socket says 172.21.15.138:48092) 2024-05-27T09:19:51.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.226+0000 7f0c97e00700 1 -- 172.21.15.138:0/3163927046 learned_addr learned my addr 172.21.15.138:0/3163927046 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:19:51.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.227+0000 7f0c97e00700 1 -- 172.21.15.138:0/3163927046 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c880095f0 con 0x7f0c98073090 2024-05-27T09:19:51.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.227+0000 7f0c97e00700 1 --2- 172.21.15.138:0/3163927046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98073090 0x7f0c98197350 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0c880092e0 tx=0x7f0c88004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:19:51.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.227+0000 7f0c94c00700 1 -- 172.21.15.138:0/3163927046 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c8801d070 con 0x7f0c98073090 2024-05-27T09:19:51.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.228+0000 7f0c94c00700 1 -- 172.21.15.138:0/3163927046 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0c8800be10 con 0x7f0c98073090 2024-05-27T09:19:51.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.228+0000 7f0c94c00700 1 -- 172.21.15.138:0/3163927046 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c88022e10 con 0x7f0c98073090 2024-05-27T09:19:51.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.228+0000 7f0c9da00700 1 -- 172.21.15.138:0/3163927046 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c98197a90 con 0x7f0c98073090 2024-05-27T09:19:51.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.228+0000 7f0c9da00700 1 -- 172.21.15.138:0/3163927046 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c98197eb0 con 0x7f0c98073090 2024-05-27T09:19:51.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.229+0000 7f0c9da00700 1 -- 172.21.15.138:0/3163927046 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c98062380 con 0x7f0c98073090 2024-05-27T09:19:51.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.229+0000 7f0c94c00700 1 -- 172.21.15.138:0/3163927046 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 45199+0+0 (secure 0 0 0) 0x7f0c88022470 con 0x7f0c98073090 2024-05-27T09:19:51.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.230+0000 7f0c94c00700 1 --2- 172.21.15.138:0/3163927046 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f0c7c038530 0x7f0c7c03a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:51.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.230+0000 7f0c94c00700 1 -- 172.21.15.138:0/3163927046 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f0c8804c920 con 0x7f0c98073090 2024-05-27T09:19:51.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.230+0000 7f0c97400700 1 --2- 172.21.15.138:0/3163927046 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f0c7c038530 0x7f0c7c03a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:19:51.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.231+0000 7f0c97400700 1 --2- 172.21.15.138:0/3163927046 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f0c7c038530 0x7f0c7c03a9e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0c84006fd0 tx=0x7f0c84006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:19:51.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.245+0000 7f0c94c00700 1 -- 172.21.15.138:0/3163927046 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0c8802b980 con 0x7f0c98073090 2024-05-27T09:19:51.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.543+0000 7f0c9da00700 1 -- 172.21.15.138:0/3163927046 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0c981986b0 con 0x7f0c98073090 2024-05-27T09:19:51.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.545+0000 7f0c94c00700 1 -- 172.21.15.138:0/3163927046 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f0c88027070 con 0x7f0c98073090 2024-05-27T09:19:51.546 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:19:51.546 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b966b06e-1c09-11ef-bc9b-c7b262605968","modified":"2024-05-27T09:16:53.014813Z","created":"2024-05-27T09:16:53.014813Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi119","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T09:19:51.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.547+0000 7f0c9da00700 1 -- 172.21.15.138:0/3163927046 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f0c7c038530 msgr2=0x7f0c7c03a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:51.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.547+0000 7f0c9da00700 1 --2- 172.21.15.138:0/3163927046 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f0c7c038530 0x7f0c7c03a9e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0c84006fd0 tx=0x7f0c84006e40 comp rx=0 tx=0).stop 2024-05-27T09:19:51.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.547+0000 7f0c9da00700 1 -- 172.21.15.138:0/3163927046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98073090 msgr2=0x7f0c98197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:51.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.547+0000 7f0c9da00700 1 --2- 172.21.15.138:0/3163927046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98073090 0x7f0c98197350 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0c880092e0 tx=0x7f0c88004c80 comp rx=0 tx=0).stop 2024-05-27T09:19:51.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.547+0000 7f0c9da00700 1 -- 172.21.15.138:0/3163927046 shutdown_connections 2024-05-27T09:19:51.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.547+0000 7f0c9da00700 1 --2- 172.21.15.138:0/3163927046 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f0c7c038530 0x7f0c7c03a9e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:51.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.547+0000 7f0c9da00700 1 --2- 172.21.15.138:0/3163927046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98073090 0x7f0c98197350 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:51.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.547+0000 7f0c9da00700 1 -- 172.21.15.138:0/3163927046 >> 172.21.15.138:0/3163927046 conn(0x7f0c980fb7a0 msgr2=0x7f0c98106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:51.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.547+0000 7f0c9da00700 1 -- 172.21.15.138:0/3163927046 shutdown_connections 2024-05-27T09:19:51.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:51.548+0000 7f0c9da00700 1 -- 172.21.15.138:0/3163927046 wait complete. 2024-05-27T09:19:51.549 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-27T09:19:51.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:51 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:19:51.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:51 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/3163927046' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:19:53.522 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-27T09:19:53.522 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T09:19:54.016 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:19:55.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:55 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:55.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:55 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:55.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:55 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:55.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:55 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:55.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:55 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-27T09:19:55.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:55 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-05-27T09:19:55.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:55 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:19:55.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:55 smithi119 ceph-mon[29082]: Deploying daemon crash.smithi138 on smithi138 2024-05-27T09:19:56.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.097+0000 7fe638a00700 1 -- 172.21.15.138:0/3905286001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6341000b0 msgr2=0x7fe634100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:56.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.097+0000 7fe638a00700 1 --2- 172.21.15.138:0/3905286001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6341000b0 0x7fe634100480 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe624009910 tx=0x7fe624009c20 comp rx=0 tx=0).stop 2024-05-27T09:19:56.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.097+0000 7fe638a00700 1 -- 172.21.15.138:0/3905286001 shutdown_connections 2024-05-27T09:19:56.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.097+0000 7fe638a00700 1 --2- 172.21.15.138:0/3905286001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6341000b0 0x7fe634100480 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:56.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.097+0000 7fe638a00700 1 -- 172.21.15.138:0/3905286001 >> 172.21.15.138:0/3905286001 conn(0x7fe6340fb990 msgr2=0x7fe6340fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:56.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.097+0000 7fe638a00700 1 -- 172.21.15.138:0/3905286001 shutdown_connections 2024-05-27T09:19:56.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.097+0000 7fe638a00700 1 -- 172.21.15.138:0/3905286001 wait complete. 2024-05-27T09:19:56.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.098+0000 7fe638a00700 1 Processor -- start 2024-05-27T09:19:56.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.098+0000 7fe638a00700 1 -- start start 2024-05-27T09:19:56.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.098+0000 7fe638a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6341000b0 0x7fe634197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:56.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.099+0000 7fe638a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe634197a80 con 0x7fe6341000b0 2024-05-27T09:19:56.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.099+0000 7fe631e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6341000b0 0x7fe634197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:19:56.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.099+0000 7fe631e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6341000b0 0x7fe634197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.138:48110/0 (socket says 172.21.15.138:48110) 2024-05-27T09:19:56.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.099+0000 7fe631e00700 1 -- 172.21.15.138:0/2060881437 learned_addr learned my addr 172.21.15.138:0/2060881437 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:19:56.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.100+0000 7fe631e00700 1 -- 172.21.15.138:0/2060881437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6240095f0 con 0x7fe6341000b0 2024-05-27T09:19:56.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.100+0000 7fe631e00700 1 --2- 172.21.15.138:0/2060881437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6341000b0 0x7fe634197540 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe624005b40 tx=0x7fe624004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:19:56.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.100+0000 7fe62aa00700 1 -- 172.21.15.138:0/2060881437 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe62401d070 con 0x7fe6341000b0 2024-05-27T09:19:56.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.100+0000 7fe638a00700 1 -- 172.21.15.138:0/2060881437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe634197c80 con 0x7fe6341000b0 2024-05-27T09:19:56.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.101+0000 7fe638a00700 1 -- 172.21.15.138:0/2060881437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe634198040 con 0x7fe6341000b0 2024-05-27T09:19:56.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.101+0000 7fe62aa00700 1 -- 172.21.15.138:0/2060881437 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe62400be10 con 0x7fe6341000b0 2024-05-27T09:19:56.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.101+0000 7fe62aa00700 1 -- 172.21.15.138:0/2060881437 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe624022d90 con 0x7fe6341000b0 2024-05-27T09:19:56.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.102+0000 7fe638a00700 1 -- 172.21.15.138:0/2060881437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe634062380 con 0x7fe6341000b0 2024-05-27T09:19:56.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.102+0000 7fe62aa00700 1 -- 172.21.15.138:0/2060881437 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 45199+0+0 (secure 0 0 0) 0x7fe624022650 con 0x7fe6341000b0 2024-05-27T09:19:56.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.102+0000 7fe62aa00700 1 --2- 172.21.15.138:0/2060881437 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fe6180385d0 0x7fe61803aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:19:56.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.103+0000 7fe62aa00700 1 -- 172.21.15.138:0/2060881437 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fe62404c550 con 0x7fe6341000b0 2024-05-27T09:19:56.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.103+0000 7fe631400700 1 --2- 172.21.15.138:0/2060881437 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fe6180385d0 0x7fe61803aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:19:56.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.104+0000 7fe631400700 1 --2- 172.21.15.138:0/2060881437 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fe6180385d0 0x7fe61803aa80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe620006fd0 tx=0x7fe620006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:19:56.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.117+0000 7fe62aa00700 1 -- 172.21.15.138:0/2060881437 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe6240213d0 con 0x7fe6341000b0 2024-05-27T09:19:56.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.395+0000 7fe638a00700 1 -- 172.21.15.138:0/2060881437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe63419bfc0 con 0x7fe6341000b0 2024-05-27T09:19:56.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.396+0000 7fe62aa00700 1 -- 172.21.15.138:0/2060881437 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fe6240213d0 con 0x7fe6341000b0 2024-05-27T09:19:56.531 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:19:56.531 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b966b06e-1c09-11ef-bc9b-c7b262605968","modified":"2024-05-27T09:16:53.014813Z","created":"2024-05-27T09:16:53.014813Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi119","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T09:19:56.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.399+0000 7fe638a00700 1 -- 172.21.15.138:0/2060881437 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fe6180385d0 msgr2=0x7fe61803aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:56.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.399+0000 7fe638a00700 1 --2- 172.21.15.138:0/2060881437 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fe6180385d0 0x7fe61803aa80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe620006fd0 tx=0x7fe620006e40 comp rx=0 tx=0).stop 2024-05-27T09:19:56.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.399+0000 7fe638a00700 1 -- 172.21.15.138:0/2060881437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6341000b0 msgr2=0x7fe634197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:19:56.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.399+0000 7fe638a00700 1 --2- 172.21.15.138:0/2060881437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6341000b0 0x7fe634197540 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe624005b40 tx=0x7fe624004c80 comp rx=0 tx=0).stop 2024-05-27T09:19:56.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.399+0000 7fe638a00700 1 -- 172.21.15.138:0/2060881437 shutdown_connections 2024-05-27T09:19:56.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.399+0000 7fe638a00700 1 --2- 172.21.15.138:0/2060881437 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fe6180385d0 0x7fe61803aa80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:56.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.399+0000 7fe638a00700 1 --2- 172.21.15.138:0/2060881437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6341000b0 0x7fe634197540 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:19:56.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.399+0000 7fe638a00700 1 -- 172.21.15.138:0/2060881437 >> 172.21.15.138:0/2060881437 conn(0x7fe6340fb990 msgr2=0x7fe634103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:19:56.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.399+0000 7fe638a00700 1 -- 172.21.15.138:0/2060881437 shutdown_connections 2024-05-27T09:19:56.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:19:56.399+0000 7fe638a00700 1 -- 172.21.15.138:0/2060881437 wait complete. 2024-05-27T09:19:56.532 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-27T09:19:57.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:57 smithi119 ceph-mon[29082]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:19:57.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:57 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:19:57.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:57 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/2060881437' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:19:58.125 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-27T09:19:58.125 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T09:19:58.556 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:19:59.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:19:59 smithi119 ceph-mon[29082]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:00.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:00 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:00.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:00 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:00.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:00 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:00.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:00 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:00.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:00 smithi119 ceph-mon[29082]: Deploying daemon node-exporter.smithi138 on smithi138 2024-05-27T09:20:01.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.281+0000 7f4ded400700 1 -- 172.21.15.138:0/1533122653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4de81000b0 msgr2=0x7f4de8100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:20:01.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.281+0000 7f4ded400700 1 --2- 172.21.15.138:0/1533122653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4de81000b0 0x7f4de8100480 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009910 tx=0x7f4dd8009c20 comp rx=0 tx=0).stop 2024-05-27T09:20:01.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.281+0000 7f4ded400700 1 -- 172.21.15.138:0/1533122653 shutdown_connections 2024-05-27T09:20:01.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.281+0000 7f4ded400700 1 --2- 172.21.15.138:0/1533122653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4de81000b0 0x7f4de8100480 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:01.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.281+0000 7f4ded400700 1 -- 172.21.15.138:0/1533122653 >> 172.21.15.138:0/1533122653 conn(0x7f4de80fb990 msgr2=0x7f4de80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:20:01.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.281+0000 7f4ded400700 1 -- 172.21.15.138:0/1533122653 shutdown_connections 2024-05-27T09:20:01.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.281+0000 7f4ded400700 1 -- 172.21.15.138:0/1533122653 wait complete. 2024-05-27T09:20:01.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.283+0000 7f4ded400700 1 Processor -- start 2024-05-27T09:20:01.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.283+0000 7f4ded400700 1 -- start start 2024-05-27T09:20:01.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.284+0000 7f4ded400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4de81000b0 0x7f4de8197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:20:01.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.284+0000 7f4ded400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de8197a80 con 0x7f4de81000b0 2024-05-27T09:20:01.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.284+0000 7f4de6800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4de81000b0 0x7f4de8197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:20:01.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.284+0000 7f4de6800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4de81000b0 0x7f4de8197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.138:50196/0 (socket says 172.21.15.138:50196) 2024-05-27T09:20:01.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.284+0000 7f4de6800700 1 -- 172.21.15.138:0/990671078 learned_addr learned my addr 172.21.15.138:0/990671078 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:20:01.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.285+0000 7f4de6800700 1 -- 172.21.15.138:0/990671078 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd80095f0 con 0x7f4de81000b0 2024-05-27T09:20:01.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.286+0000 7f4de6800700 1 --2- 172.21.15.138:0/990671078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4de81000b0 0x7f4de8197540 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80092e0 tx=0x7f4dd8004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:20:01.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.286+0000 7f4ddfe00700 1 -- 172.21.15.138:0/990671078 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4dd801d070 con 0x7f4de81000b0 2024-05-27T09:20:01.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.286+0000 7f4ded400700 1 -- 172.21.15.138:0/990671078 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de8197c80 con 0x7f4de81000b0 2024-05-27T09:20:01.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.286+0000 7f4ddfe00700 1 -- 172.21.15.138:0/990671078 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4dd800be10 con 0x7f4de81000b0 2024-05-27T09:20:01.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.286+0000 7f4ddfe00700 1 -- 172.21.15.138:0/990671078 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4dd8022e10 con 0x7f4de81000b0 2024-05-27T09:20:01.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.286+0000 7f4ded400700 1 -- 172.21.15.138:0/990671078 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de81980a0 con 0x7f4de81000b0 2024-05-27T09:20:01.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.287+0000 7f4ded400700 1 -- 172.21.15.138:0/990671078 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de8062380 con 0x7f4de81000b0 2024-05-27T09:20:01.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.288+0000 7f4ddfe00700 1 -- 172.21.15.138:0/990671078 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 45199+0+0 (secure 0 0 0) 0x7f4dd8022470 con 0x7f4de81000b0 2024-05-27T09:20:01.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.288+0000 7f4ddfe00700 1 --2- 172.21.15.138:0/990671078 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4dc8038530 0x7f4dc803a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:20:01.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.288+0000 7f4ddfe00700 1 -- 172.21.15.138:0/990671078 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f4dd804ca20 con 0x7f4de81000b0 2024-05-27T09:20:01.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.289+0000 7f4ddde00700 1 --2- 172.21.15.138:0/990671078 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4dc8038530 0x7f4dc803a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:20:01.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.293+0000 7f4ddde00700 1 --2- 172.21.15.138:0/990671078 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4dc8038530 0x7f4dc803a9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0006fd0 tx=0x7f4dd0006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:20:01.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.305+0000 7f4ddfe00700 1 -- 172.21.15.138:0/990671078 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4dd802b590 con 0x7f4de81000b0 2024-05-27T09:20:01.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.645+0000 7f4ded400700 1 -- 172.21.15.138:0/990671078 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4de819c030 con 0x7f4de81000b0 2024-05-27T09:20:01.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.646+0000 7f4ddfe00700 1 -- 172.21.15.138:0/990671078 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f4dd8034300 con 0x7f4de81000b0 2024-05-27T09:20:01.647 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:20:01.647 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b966b06e-1c09-11ef-bc9b-c7b262605968","modified":"2024-05-27T09:16:53.014813Z","created":"2024-05-27T09:16:53.014813Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi119","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T09:20:01.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.649+0000 7f4ded400700 1 -- 172.21.15.138:0/990671078 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4dc8038530 msgr2=0x7f4dc803a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:20:01.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.649+0000 7f4ded400700 1 --2- 172.21.15.138:0/990671078 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4dc8038530 0x7f4dc803a9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0006fd0 tx=0x7f4dd0006e40 comp rx=0 tx=0).stop 2024-05-27T09:20:01.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.649+0000 7f4ded400700 1 -- 172.21.15.138:0/990671078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4de81000b0 msgr2=0x7f4de8197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:20:01.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.649+0000 7f4ded400700 1 --2- 172.21.15.138:0/990671078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4de81000b0 0x7f4de8197540 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80092e0 tx=0x7f4dd8004c80 comp rx=0 tx=0).stop 2024-05-27T09:20:01.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.650+0000 7f4ded400700 1 -- 172.21.15.138:0/990671078 shutdown_connections 2024-05-27T09:20:01.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.650+0000 7f4ded400700 1 --2- 172.21.15.138:0/990671078 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4dc8038530 0x7f4dc803a9e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:01.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.650+0000 7f4ded400700 1 --2- 172.21.15.138:0/990671078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4de81000b0 0x7f4de8197540 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:01.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.650+0000 7f4ded400700 1 -- 172.21.15.138:0/990671078 >> 172.21.15.138:0/990671078 conn(0x7f4de80fb990 msgr2=0x7f4de8103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:20:01.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.650+0000 7f4ded400700 1 -- 172.21.15.138:0/990671078 shutdown_connections 2024-05-27T09:20:01.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:01.650+0000 7f4ded400700 1 -- 172.21.15.138:0/990671078 wait complete. 2024-05-27T09:20:01.651 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-27T09:20:01.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:01 smithi119 ceph-mon[29082]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:01.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:01 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:02.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:02 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/990671078' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:20:03.416 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-27T09:20:03.416 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T09:20:03.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:03 smithi119 ceph-mon[29082]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:04.371 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:20:05.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:05 smithi119 ceph-mon[29082]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:06.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:06 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:20:07.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.665+0000 7f14c6800700 1 -- 172.21.15.138:0/1375573593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14c01000b0 msgr2=0x7f14c0100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:20:07.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.665+0000 7f14c6800700 1 --2- 172.21.15.138:0/1375573593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14c01000b0 0x7f14c0100480 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009910 tx=0x7f14b0009c20 comp rx=0 tx=0).stop 2024-05-27T09:20:07.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.665+0000 7f14c6800700 1 -- 172.21.15.138:0/1375573593 shutdown_connections 2024-05-27T09:20:07.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.665+0000 7f14c6800700 1 --2- 172.21.15.138:0/1375573593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14c01000b0 0x7f14c0100480 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:07.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.665+0000 7f14c6800700 1 -- 172.21.15.138:0/1375573593 >> 172.21.15.138:0/1375573593 conn(0x7f14c00fb990 msgr2=0x7f14c00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:20:07.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.665+0000 7f14c6800700 1 -- 172.21.15.138:0/1375573593 shutdown_connections 2024-05-27T09:20:07.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.665+0000 7f14c6800700 1 -- 172.21.15.138:0/1375573593 wait complete. 2024-05-27T09:20:07.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.666+0000 7f14c6800700 1 Processor -- start 2024-05-27T09:20:07.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.667+0000 7f14c6800700 1 -- start start 2024-05-27T09:20:07.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.667+0000 7f14c6800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14c01000b0 0x7f14c01953a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:20:07.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.667+0000 7f14c6800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c01958e0 con 0x7f14c01000b0 2024-05-27T09:20:07.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.668+0000 7f14bf400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14c01000b0 0x7f14c01953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:20:07.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.668+0000 7f14bf400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14c01000b0 0x7f14c01953a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.138:50218/0 (socket says 172.21.15.138:50218) 2024-05-27T09:20:07.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.668+0000 7f14bf400700 1 -- 172.21.15.138:0/1323564809 learned_addr learned my addr 172.21.15.138:0/1323564809 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:20:07.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.668+0000 7f14bf400700 1 -- 172.21.15.138:0/1323564809 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b00095f0 con 0x7f14c01000b0 2024-05-27T09:20:07.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.669+0000 7f14bf400700 1 --2- 172.21.15.138:0/1323564809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14c01000b0 0x7f14c01953a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f14b0005b40 tx=0x7f14b0004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:20:07.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.669+0000 7f14b7e00700 1 -- 172.21.15.138:0/1323564809 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f14b001d070 con 0x7f14c01000b0 2024-05-27T09:20:07.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.669+0000 7f14c6800700 1 -- 172.21.15.138:0/1323564809 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c0195ae0 con 0x7f14c01000b0 2024-05-27T09:20:07.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.669+0000 7f14c6800700 1 -- 172.21.15.138:0/1323564809 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c0195ea0 con 0x7f14c01000b0 2024-05-27T09:20:07.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.669+0000 7f14b7e00700 1 -- 172.21.15.138:0/1323564809 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f14b000be10 con 0x7f14c01000b0 2024-05-27T09:20:07.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.669+0000 7f14b7e00700 1 -- 172.21.15.138:0/1323564809 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f14b0022d90 con 0x7f14c01000b0 2024-05-27T09:20:07.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.670+0000 7f14b5600700 1 -- 172.21.15.138:0/1323564809 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14980052f0 con 0x7f14c01000b0 2024-05-27T09:20:07.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.671+0000 7f14b7e00700 1 -- 172.21.15.138:0/1323564809 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 45199+0+0 (secure 0 0 0) 0x7f14b0022650 con 0x7f14c01000b0 2024-05-27T09:20:07.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.671+0000 7f14b7e00700 1 --2- 172.21.15.138:0/1323564809 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f14a40385d0 0x7f14a403aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:20:07.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.671+0000 7f14b7e00700 1 -- 172.21.15.138:0/1323564809 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f14b004c550 con 0x7f14c01000b0 2024-05-27T09:20:07.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.672+0000 7f14bea00700 1 --2- 172.21.15.138:0/1323564809 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f14a40385d0 0x7f14a403aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:20:07.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.673+0000 7f14bea00700 1 --2- 172.21.15.138:0/1323564809 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f14a40385d0 0x7f14a403aa80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f14ac006fd0 tx=0x7f14ac006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:20:07.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.686+0000 7f14b7e00700 1 -- 172.21.15.138:0/1323564809 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f14b00213d0 con 0x7f14c01000b0 2024-05-27T09:20:07.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:07 smithi119 ceph-mon[29082]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:07.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:07 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:07.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:07 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:07.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.991+0000 7f14b5600700 1 -- 172.21.15.138:0/1323564809 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1498005160 con 0x7f14c01000b0 2024-05-27T09:20:07.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.992+0000 7f14b7e00700 1 -- 172.21.15.138:0/1323564809 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f14b00213d0 con 0x7f14c01000b0 2024-05-27T09:20:07.993 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:20:07.993 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b966b06e-1c09-11ef-bc9b-c7b262605968","modified":"2024-05-27T09:16:53.014813Z","created":"2024-05-27T09:16:53.014813Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi119","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T09:20:07.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.994+0000 7f14b5600700 1 -- 172.21.15.138:0/1323564809 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f14a40385d0 msgr2=0x7f14a403aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:20:07.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.994+0000 7f14b5600700 1 --2- 172.21.15.138:0/1323564809 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f14a40385d0 0x7f14a403aa80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f14ac006fd0 tx=0x7f14ac006e40 comp rx=0 tx=0).stop 2024-05-27T09:20:07.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.994+0000 7f14b5600700 1 -- 172.21.15.138:0/1323564809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14c01000b0 msgr2=0x7f14c01953a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:20:07.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.994+0000 7f14b5600700 1 --2- 172.21.15.138:0/1323564809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14c01000b0 0x7f14c01953a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f14b0005b40 tx=0x7f14b0004c80 comp rx=0 tx=0).stop 2024-05-27T09:20:07.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.995+0000 7f14b5600700 1 -- 172.21.15.138:0/1323564809 shutdown_connections 2024-05-27T09:20:07.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.995+0000 7f14b5600700 1 --2- 172.21.15.138:0/1323564809 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f14a40385d0 0x7f14a403aa80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:07.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.995+0000 7f14b5600700 1 --2- 172.21.15.138:0/1323564809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14c01000b0 0x7f14c01953a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:07.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.995+0000 7f14b5600700 1 -- 172.21.15.138:0/1323564809 >> 172.21.15.138:0/1323564809 conn(0x7f14c00fb990 msgr2=0x7f14c0104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:20:07.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.995+0000 7f14b5600700 1 -- 172.21.15.138:0/1323564809 shutdown_connections 2024-05-27T09:20:07.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:07.995+0000 7f14b5600700 1 -- 172.21.15.138:0/1323564809 wait complete. 2024-05-27T09:20:07.997 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-27T09:20:08.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:08 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:08.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:08 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:08.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:08 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.dxsuqf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T09:20:08.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:08 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.dxsuqf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T09:20:08.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:08 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T09:20:08.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:08 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:20:08.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:08 smithi119 ceph-mon[29082]: Deploying daemon mgr.smithi138.dxsuqf on smithi138 2024-05-27T09:20:08.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:08 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/1323564809' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:20:09.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:09 smithi119 ceph-mon[29082]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:09.815 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-27T09:20:09.815 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T09:20:10.417 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:20:11.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:10 smithi119 ceph-mon[29082]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:12.423 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:12 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:13.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.120+0000 7f73b8600700 1 -- 172.21.15.138:0/3317735105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73b01000b0 msgr2=0x7f73b0100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:20:13.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.120+0000 7f73b8600700 1 --2- 172.21.15.138:0/3317735105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73b01000b0 0x7f73b0100480 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f73a400b0a0 tx=0x7f73a400b3b0 comp rx=0 tx=0).stop 2024-05-27T09:20:13.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.120+0000 7f73b8600700 1 -- 172.21.15.138:0/3317735105 shutdown_connections 2024-05-27T09:20:13.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.120+0000 7f73b8600700 1 --2- 172.21.15.138:0/3317735105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73b01000b0 0x7f73b0100480 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:13.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.120+0000 7f73b8600700 1 -- 172.21.15.138:0/3317735105 >> 172.21.15.138:0/3317735105 conn(0x7f73b00fb990 msgr2=0x7f73b00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:20:13.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.120+0000 7f73b8600700 1 -- 172.21.15.138:0/3317735105 shutdown_connections 2024-05-27T09:20:13.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.120+0000 7f73b8600700 1 -- 172.21.15.138:0/3317735105 wait complete. 2024-05-27T09:20:13.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.122+0000 7f73b8600700 1 Processor -- start 2024-05-27T09:20:13.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.122+0000 7f73b8600700 1 -- start start 2024-05-27T09:20:13.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.123+0000 7f73b8600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73b01000b0 0x7f73b0073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:20:13.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.123+0000 7f73b8600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b0074f60 con 0x7f73b01000b0 2024-05-27T09:20:13.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.123+0000 7f73b5c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73b01000b0 0x7f73b0073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:20:13.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.123+0000 7f73b5c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73b01000b0 0x7f73b0073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.138:39378/0 (socket says 172.21.15.138:39378) 2024-05-27T09:20:13.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.124+0000 7f73b5c00700 1 -- 172.21.15.138:0/4215545472 learned_addr learned my addr 172.21.15.138:0/4215545472 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:20:13.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.124+0000 7f73b5c00700 1 -- 172.21.15.138:0/4215545472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a4009d00 con 0x7f73b01000b0 2024-05-27T09:20:13.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.125+0000 7f73b5c00700 1 --2- 172.21.15.138:0/4215545472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73b01000b0 0x7f73b0073050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f73a40062a0 tx=0x7f73a4003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:20:13.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.125+0000 7f73ae000700 1 -- 172.21.15.138:0/4215545472 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f73a400e040 con 0x7f73b01000b0 2024-05-27T09:20:13.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.125+0000 7f73b8600700 1 -- 172.21.15.138:0/4215545472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b0073590 con 0x7f73b01000b0 2024-05-27T09:20:13.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.125+0000 7f73ae000700 1 -- 172.21.15.138:0/4215545472 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f73a400bdd0 con 0x7f73b01000b0 2024-05-27T09:20:13.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.125+0000 7f73b8600700 1 -- 172.21.15.138:0/4215545472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b0073a10 con 0x7f73b01000b0 2024-05-27T09:20:13.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.125+0000 7f73ae000700 1 -- 172.21.15.138:0/4215545472 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f73a401d8f0 con 0x7f73b01000b0 2024-05-27T09:20:13.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.126+0000 7f73b8600700 1 -- 172.21.15.138:0/4215545472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b0108330 con 0x7f73b01000b0 2024-05-27T09:20:13.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.127+0000 7f73ae000700 1 -- 172.21.15.138:0/4215545472 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 45199+0+0 (secure 0 0 0) 0x7f73a4004250 con 0x7f73b01000b0 2024-05-27T09:20:13.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.127+0000 7f73ae000700 1 --2- 172.21.15.138:0/4215545472 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f73980385d0 0x7f739803aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:20:13.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.127+0000 7f73ae000700 1 -- 172.21.15.138:0/4215545472 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f73a404c360 con 0x7f73b01000b0 2024-05-27T09:20:13.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.128+0000 7f73b5200700 1 --2- 172.21.15.138:0/4215545472 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f73980385d0 0x7f739803aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:20:13.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.129+0000 7f73b5200700 1 --2- 172.21.15.138:0/4215545472 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f73980385d0 0x7f739803aa80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f739c006fd0 tx=0x7f739c006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:20:13.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.142+0000 7f73ae000700 1 -- 172.21.15.138:0/4215545472 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f73a4025750 con 0x7f73b01000b0 2024-05-27T09:20:13.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.448+0000 7f73b8600700 1 -- 172.21.15.138:0/4215545472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f73b00fe1c0 con 0x7f73b01000b0 2024-05-27T09:20:13.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.449+0000 7f73ae000700 1 -- 172.21.15.138:0/4215545472 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f73b00fe1c0 con 0x7f73b01000b0 2024-05-27T09:20:13.450 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:20:13.451 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b966b06e-1c09-11ef-bc9b-c7b262605968","modified":"2024-05-27T09:16:53.014813Z","created":"2024-05-27T09:16:53.014813Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi119","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T09:20:13.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.452+0000 7f73b8600700 1 -- 172.21.15.138:0/4215545472 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f73980385d0 msgr2=0x7f739803aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:20:13.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.452+0000 7f73b8600700 1 --2- 172.21.15.138:0/4215545472 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f73980385d0 0x7f739803aa80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f739c006fd0 tx=0x7f739c006e40 comp rx=0 tx=0).stop 2024-05-27T09:20:13.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.452+0000 7f73b8600700 1 -- 172.21.15.138:0/4215545472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73b01000b0 msgr2=0x7f73b0073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:20:13.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.452+0000 7f73b8600700 1 --2- 172.21.15.138:0/4215545472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73b01000b0 0x7f73b0073050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f73a40062a0 tx=0x7f73a4003ce0 comp rx=0 tx=0).stop 2024-05-27T09:20:13.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.452+0000 7f73b8600700 1 -- 172.21.15.138:0/4215545472 shutdown_connections 2024-05-27T09:20:13.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.452+0000 7f73b8600700 1 --2- 172.21.15.138:0/4215545472 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f73980385d0 0x7f739803aa80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:13.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.452+0000 7f73b8600700 1 --2- 172.21.15.138:0/4215545472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73b01000b0 0x7f73b0073050 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:13.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.452+0000 7f73b8600700 1 -- 172.21.15.138:0/4215545472 >> 172.21.15.138:0/4215545472 conn(0x7f73b00fb990 msgr2=0x7f73b0104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:20:13.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.452+0000 7f73b8600700 1 -- 172.21.15.138:0/4215545472 shutdown_connections 2024-05-27T09:20:13.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:13.453+0000 7f73b8600700 1 -- 172.21.15.138:0/4215545472 wait complete. 2024-05-27T09:20:13.454 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-27T09:20:13.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:13 smithi119 ceph-mon[29082]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:14.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:14 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:14.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:14 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:14.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:14 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:14.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:14 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:14.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:14 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T09:20:14.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:14 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:20:14.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:14 smithi119 ceph-mon[29082]: Deploying daemon mon.smithi138 on smithi138 2024-05-27T09:20:14.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:14 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/4215545472' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:20:15.188 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-27T09:20:15.188 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T09:20:15.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:15 smithi119 ceph-mon[29082]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:15.899 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:20:17.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:17 smithi119 ceph-mon[29082]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:17.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:17 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:19.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:18.997+0000 7f3545800700 1 -- 172.21.15.138:0/3100762193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3540075280 msgr2=0x7f3540075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:20:19.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:18.997+0000 7f3545800700 1 --2- 172.21.15.138:0/3100762193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3540075280 0x7f3540075650 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3530009910 tx=0x7f3530009c20 comp rx=0 tx=0).stop 2024-05-27T09:20:19.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:18.998+0000 7f3545800700 1 -- 172.21.15.138:0/3100762193 shutdown_connections 2024-05-27T09:20:19.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:18.998+0000 7f3545800700 1 --2- 172.21.15.138:0/3100762193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3540075280 0x7f3540075650 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:19.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:18.998+0000 7f3545800700 1 -- 172.21.15.138:0/3100762193 >> 172.21.15.138:0/3100762193 conn(0x7f35400fdba0 msgr2=0x7f35400fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:20:19.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:18.998+0000 7f3545800700 1 -- 172.21.15.138:0/3100762193 shutdown_connections 2024-05-27T09:20:19.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:18.998+0000 7f3545800700 1 -- 172.21.15.138:0/3100762193 wait complete. 2024-05-27T09:20:19.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:18.999+0000 7f3545800700 1 Processor -- start 2024-05-27T09:20:19.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:18.999+0000 7f3545800700 1 -- start start 2024-05-27T09:20:19.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:18.999+0000 7f3545800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3540075280 0x7f35401a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:20:19.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:18.999+0000 7f3545800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35401a1bf0 con 0x7f3540075280 2024-05-27T09:20:19.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.000+0000 7f353ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3540075280 0x7f35401a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:20:19.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.000+0000 7f353ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3540075280 0x7f35401a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.138:38400/0 (socket says 172.21.15.138:38400) 2024-05-27T09:20:19.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.000+0000 7f353ea00700 1 -- 172.21.15.138:0/817616099 learned_addr learned my addr 172.21.15.138:0/817616099 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:20:19.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.001+0000 7f353ea00700 1 -- 172.21.15.138:0/817616099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35300095f0 con 0x7f3540075280 2024-05-27T09:20:19.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.001+0000 7f353ea00700 1 --2- 172.21.15.138:0/817616099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3540075280 0x7f35401a16b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f35300092e0 tx=0x7f3530004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:20:19.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.002+0000 7f3537400700 1 -- 172.21.15.138:0/817616099 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f353001d070 con 0x7f3540075280 2024-05-27T09:20:19.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.002+0000 7f3545800700 1 -- 172.21.15.138:0/817616099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35401a1df0 con 0x7f3540075280 2024-05-27T09:20:19.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.002+0000 7f3545800700 1 -- 172.21.15.138:0/817616099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35401a2210 con 0x7f3540075280 2024-05-27T09:20:19.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.002+0000 7f3537400700 1 -- 172.21.15.138:0/817616099 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f353000be10 con 0x7f3540075280 2024-05-27T09:20:19.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.002+0000 7f3537400700 1 -- 172.21.15.138:0/817616099 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3530022e10 con 0x7f3540075280 2024-05-27T09:20:19.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.003+0000 7f3545800700 1 -- 172.21.15.138:0/817616099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3540062380 con 0x7f3540075280 2024-05-27T09:20:19.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.004+0000 7f3537400700 1 -- 172.21.15.138:0/817616099 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 45199+0+0 (secure 0 0 0) 0x7f3530022470 con 0x7f3540075280 2024-05-27T09:20:19.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.004+0000 7f3537400700 1 --2- 172.21.15.138:0/817616099 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f3524038530 0x7f352403a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:20:19.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.004+0000 7f3537400700 1 -- 172.21.15.138:0/817616099 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f353004c510 con 0x7f3540075280 2024-05-27T09:20:19.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.005+0000 7f353e000700 1 --2- 172.21.15.138:0/817616099 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f3524038530 0x7f352403a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:20:19.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.006+0000 7f353e000700 1 --2- 172.21.15.138:0/817616099 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f3524038530 0x7f352403a9e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f352c006fd0 tx=0x7f352c006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:20:19.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.019+0000 7f3537400700 1 -- 172.21.15.138:0/817616099 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f353002a540 con 0x7f3540075280 2024-05-27T09:20:19.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.322+0000 7f3545800700 1 -- 172.21.15.138:0/817616099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f354010da50 con 0x7f3540075280 2024-05-27T09:20:19.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.324+0000 7f3537400700 1 -- 172.21.15.138:0/817616099 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f3530027030 con 0x7f3540075280 2024-05-27T09:20:19.325 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:20:19.325 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b966b06e-1c09-11ef-bc9b-c7b262605968","modified":"2024-05-27T09:16:53.014813Z","created":"2024-05-27T09:16:53.014813Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi119","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T09:20:19.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.326+0000 7f3545800700 1 -- 172.21.15.138:0/817616099 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f3524038530 msgr2=0x7f352403a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:20:19.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.327+0000 7f3545800700 1 --2- 172.21.15.138:0/817616099 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f3524038530 0x7f352403a9e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f352c006fd0 tx=0x7f352c006e40 comp rx=0 tx=0).stop 2024-05-27T09:20:19.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.327+0000 7f3545800700 1 -- 172.21.15.138:0/817616099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3540075280 msgr2=0x7f35401a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:20:19.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.327+0000 7f3545800700 1 --2- 172.21.15.138:0/817616099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3540075280 0x7f35401a16b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f35300092e0 tx=0x7f3530004c80 comp rx=0 tx=0).stop 2024-05-27T09:20:19.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.327+0000 7f3545800700 1 -- 172.21.15.138:0/817616099 shutdown_connections 2024-05-27T09:20:19.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.327+0000 7f3545800700 1 --2- 172.21.15.138:0/817616099 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f3524038530 0x7f352403a9e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:19.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.327+0000 7f3545800700 1 --2- 172.21.15.138:0/817616099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3540075280 0x7f35401a16b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:19.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.327+0000 7f3545800700 1 -- 172.21.15.138:0/817616099 >> 172.21.15.138:0/817616099 conn(0x7f35400fdba0 msgr2=0x7f35400fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:20:19.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.327+0000 7f3545800700 1 -- 172.21.15.138:0/817616099 shutdown_connections 2024-05-27T09:20:19.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:19.327+0000 7f3545800700 1 -- 172.21.15.138:0/817616099 wait complete. 2024-05-27T09:20:19.328 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-05-27T09:20:19.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:19 smithi119 ceph-mon[29082]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:19.956 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 podman[34849]: 2024-05-27 09:20:19.914870621 +0000 UTC m=+0.637841271 container start 60216fd6707805237264b6996f23ce90dca1fe616ecf272aa322f90c92d16471 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi138, org.label-schema.build-date=20231212, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.29.1) 2024-05-27T09:20:19.956 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: set uid:gid to 167:167 (ceph:ceph) 2024-05-27T09:20:19.956 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable), process ceph-mon, pid 2 2024-05-27T09:20:19.956 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: pidfile_write: ignore empty --pid-file 2024-05-27T09:20:20.210 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: load: jerasure load: lrc 2024-05-27T09:20:20.210 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: RocksDB version: 7.9.2 2024-05-27T09:20:20.210 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Git sha 0 2024-05-27T09:20:20.210 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Compile date 2023-08-03 19:21:13 2024-05-27T09:20:20.210 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: DB SUMMARY 2024-05-27T09:20:20.210 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: DB Session ID: 819EP3KCD61NA66A2ASZ 2024-05-27T09:20:20.211 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: CURRENT file: CURRENT 2024-05-27T09:20:20.211 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: IDENTITY file: IDENTITY 2024-05-27T09:20:20.211 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-27T09:20:20.211 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi138/store.db dir, Total Num: 0, files: 2024-05-27T09:20:20.211 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi138/store.db: 000004.log size: 511 ; 2024-05-27T09:20:20.211 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.error_if_exists: 0 2024-05-27T09:20:20.211 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.create_if_missing: 0 2024-05-27T09:20:20.211 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.paranoid_checks: 1 2024-05-27T09:20:20.211 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-27T09:20:20.211 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-27T09:20:20.211 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-27T09:20:20.211 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.env: 0x55a373ad6720 2024-05-27T09:20:20.212 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.fs: PosixFileSystem 2024-05-27T09:20:20.212 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.info_log: 0x55a37ff492c0 2024-05-27T09:20:20.212 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_file_opening_threads: 16 2024-05-27T09:20:20.212 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.statistics: (nil) 2024-05-27T09:20:20.212 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.use_fsync: 0 2024-05-27T09:20:20.212 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_log_file_size: 0 2024-05-27T09:20:20.212 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-27T09:20:20.212 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-27T09:20:20.212 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.keep_log_file_num: 1000 2024-05-27T09:20:20.212 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.recycle_log_file_num: 0 2024-05-27T09:20:20.212 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.allow_fallocate: 1 2024-05-27T09:20:20.212 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.allow_mmap_reads: 0 2024-05-27T09:20:20.212 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.allow_mmap_writes: 0 2024-05-27T09:20:20.212 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.use_direct_reads: 0 2024-05-27T09:20:20.213 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-27T09:20:20.213 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.create_missing_column_families: 0 2024-05-27T09:20:20.213 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.db_log_dir: 2024-05-27T09:20:20.213 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.wal_dir: 2024-05-27T09:20:20.213 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-27T09:20:20.213 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-27T09:20:20.213 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-27T09:20:20.213 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-27T09:20:20.213 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-27T09:20:20.213 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-27T09:20:20.214 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.advise_random_on_open: 1 2024-05-27T09:20:20.214 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.db_write_buffer_size: 0 2024-05-27T09:20:20.214 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.write_buffer_manager: 0x55a37f1ba3c0 2024-05-27T09:20:20.214 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-27T09:20:20.214 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-27T09:20:20.214 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-27T09:20:20.214 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.rate_limiter: (nil) 2024-05-27T09:20:20.214 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-27T09:20:20.214 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.wal_recovery_mode: 2 2024-05-27T09:20:20.214 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.enable_thread_tracking: 0 2024-05-27T09:20:20.214 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.enable_pipelined_write: 0 2024-05-27T09:20:20.214 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.unordered_write: 0 2024-05-27T09:20:20.214 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-27T09:20:20.215 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-27T09:20:20.215 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-27T09:20:20.215 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-27T09:20:20.215 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.row_cache: None 2024-05-27T09:20:20.215 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.wal_filter: None 2024-05-27T09:20:20.215 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-27T09:20:20.215 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.allow_ingest_behind: 0 2024-05-27T09:20:20.215 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.two_write_queues: 0 2024-05-27T09:20:20.215 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.manual_wal_flush: 0 2024-05-27T09:20:20.215 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.wal_compression: 0 2024-05-27T09:20:20.215 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.atomic_flush: 0 2024-05-27T09:20:20.215 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-27T09:20:20.215 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-27T09:20:20.215 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-27T09:20:20.216 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.log_readahead_size: 0 2024-05-27T09:20:20.216 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-27T09:20:20.216 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.best_efforts_recovery: 0 2024-05-27T09:20:20.216 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-27T09:20:20.216 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-27T09:20:20.216 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.allow_data_in_errors: 0 2024-05-27T09:20:20.216 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.db_host_id: __hostname__ 2024-05-27T09:20:20.216 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-27T09:20:20.216 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_background_jobs: 2 2024-05-27T09:20:20.216 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_background_compactions: -1 2024-05-27T09:20:20.216 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_subcompactions: 1 2024-05-27T09:20:20.217 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-27T09:20:20.217 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-27T09:20:20.217 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-27T09:20:20.217 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_total_wal_size: 0 2024-05-27T09:20:20.217 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-27T09:20:20.217 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-27T09:20:20.217 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-27T09:20:20.217 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-27T09:20:20.217 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_open_files: -1 2024-05-27T09:20:20.217 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.bytes_per_sync: 0 2024-05-27T09:20:20.217 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-27T09:20:20.218 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-27T09:20:20.218 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compaction_readahead_size: 0 2024-05-27T09:20:20.218 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_background_flushes: -1 2024-05-27T09:20:20.218 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Compression algorithms supported: 2024-05-27T09:20:20.218 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-27T09:20:20.218 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: kZSTD supported: 0 2024-05-27T09:20:20.218 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: kXpressCompression supported: 0 2024-05-27T09:20:20.218 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: kLZ4HCCompression supported: 1 2024-05-27T09:20:20.218 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: kZlibCompression supported: 1 2024-05-27T09:20:20.218 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: kSnappyCompression supported: 1 2024-05-27T09:20:20.218 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: kLZ4Compression supported: 1 2024-05-27T09:20:20.218 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: kBZip2Compression supported: 0 2024-05-27T09:20:20.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-27T09:20:20.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-27T09:20:20.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi138/store.db/MANIFEST-000005 2024-05-27T09:20:20.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-27T09:20:20.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-27T09:20:20.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.merge_operator: 2024-05-27T09:20:20.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compaction_filter: None 2024-05-27T09:20:20.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compaction_filter_factory: None 2024-05-27T09:20:20.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.sst_partitioner_factory: None 2024-05-27T09:20:20.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-27T09:20:20.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-27T09:20:20.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a37ff493c0) 2024-05-27T09:20:20.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: cache_index_and_filter_blocks: 1 2024-05-27T09:20:20.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-27T09:20:20.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-27T09:20:20.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: pin_top_level_index_and_filter: 1 2024-05-27T09:20:20.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: index_type: 0 2024-05-27T09:20:20.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: data_block_index_type: 0 2024-05-27T09:20:20.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: index_shortening: 1 2024-05-27T09:20:20.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-27T09:20:20.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: checksum: 4 2024-05-27T09:20:20.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: no_block_cache: 0 2024-05-27T09:20:20.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_cache: 0x55a37f25b350 2024-05-27T09:20:20.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_cache_name: BinnedLRUCache 2024-05-27T09:20:20.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_cache_options: 2024-05-27T09:20:20.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: capacity : 536870912 2024-05-27T09:20:20.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: num_shard_bits : 4 2024-05-27T09:20:20.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: strict_capacity_limit : 0 2024-05-27T09:20:20.221 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: high_pri_pool_ratio: 0.000 2024-05-27T09:20:20.221 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_cache_compressed: (nil) 2024-05-27T09:20:20.221 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: persistent_cache: (nil) 2024-05-27T09:20:20.221 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_size: 4096 2024-05-27T09:20:20.221 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_size_deviation: 10 2024-05-27T09:20:20.221 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_restart_interval: 16 2024-05-27T09:20:20.221 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: index_block_restart_interval: 1 2024-05-27T09:20:20.221 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: metadata_block_size: 4096 2024-05-27T09:20:20.221 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: partition_filters: 0 2024-05-27T09:20:20.221 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: use_delta_encoding: 1 2024-05-27T09:20:20.221 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: filter_policy: bloomfilter 2024-05-27T09:20:20.221 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: whole_key_filtering: 1 2024-05-27T09:20:20.221 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: verify_compression: 0 2024-05-27T09:20:20.221 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: read_amp_bytes_per_bit: 0 2024-05-27T09:20:20.221 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: format_version: 5 2024-05-27T09:20:20.222 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: enable_index_compression: 1 2024-05-27T09:20:20.222 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_align: 0 2024-05-27T09:20:20.222 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: max_auto_readahead_size: 262144 2024-05-27T09:20:20.222 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: prepopulate_block_cache: 0 2024-05-27T09:20:20.222 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: initial_auto_readahead_size: 8192 2024-05-27T09:20:20.222 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: num_file_reads_for_auto_readahead: 2 2024-05-27T09:20:20.222 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.write_buffer_size: 33554432 2024-05-27T09:20:20.222 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_write_buffer_number: 2 2024-05-27T09:20:20.222 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compression: NoCompression 2024-05-27T09:20:20.222 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.bottommost_compression: Disabled 2024-05-27T09:20:20.222 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.prefix_extractor: nullptr 2024-05-27T09:20:20.222 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-27T09:20:20.222 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.num_levels: 7 2024-05-27T09:20:20.223 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-27T09:20:20.223 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-27T09:20:20.223 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-27T09:20:20.223 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-27T09:20:20.223 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-27T09:20:20.223 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-27T09:20:20.223 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-27T09:20:20.223 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-27T09:20:20.223 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-27T09:20:20.223 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-27T09:20:20.223 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-27T09:20:20.224 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-27T09:20:20.224 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-27T09:20:20.224 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compression_opts.level: 32767 2024-05-27T09:20:20.224 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compression_opts.strategy: 0 2024-05-27T09:20:20.224 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-27T09:20:20.224 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-27T09:20:20.224 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-27T09:20:20.225 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-27T09:20:20.225 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compression_opts.enabled: false 2024-05-27T09:20:20.225 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-27T09:20:20.225 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-27T09:20:20.225 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-27T09:20:20.225 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-27T09:20:20.225 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.target_file_size_base: 67108864 2024-05-27T09:20:20.225 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-27T09:20:20.225 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-27T09:20:20.225 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-27T09:20:20.225 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-27T09:20:20.225 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-27T09:20:20.225 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-27T09:20:20.225 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-27T09:20:20.226 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-27T09:20:20.226 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-27T09:20:20.226 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-27T09:20:20.226 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-27T09:20:20.226 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-27T09:20:20.226 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-27T09:20:20.226 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-27T09:20:20.226 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.arena_block_size: 1048576 2024-05-27T09:20:20.226 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-27T09:20:20.226 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-27T09:20:20.226 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.disable_auto_compactions: 0 2024-05-27T09:20:20.226 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-27T09:20:20.227 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-27T09:20:20.227 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-27T09:20:20.227 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-27T09:20:20.227 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-27T09:20:20.227 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-27T09:20:20.227 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-27T09:20:20.227 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-27T09:20:20.227 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-27T09:20:20.227 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-27T09:20:20.227 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.table_properties_collectors: 2024-05-27T09:20:20.227 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.inplace_update_support: 0 2024-05-27T09:20:20.227 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-27T09:20:20.227 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-27T09:20:20.227 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-27T09:20:20.228 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-27T09:20:20.228 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.bloom_locality: 0 2024-05-27T09:20:20.228 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.max_successive_merges: 0 2024-05-27T09:20:20.228 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-27T09:20:20.228 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.paranoid_file_checks: 0 2024-05-27T09:20:20.228 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.force_consistency_checks: 1 2024-05-27T09:20:20.228 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.report_bg_io_stats: 0 2024-05-27T09:20:20.228 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.ttl: 2592000 2024-05-27T09:20:20.228 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-27T09:20:20.228 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-27T09:20:20.228 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-27T09:20:20.228 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.enable_blob_files: false 2024-05-27T09:20:20.228 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.min_blob_size: 0 2024-05-27T09:20:20.228 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.blob_file_size: 268435456 2024-05-27T09:20:20.229 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-27T09:20:20.229 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-27T09:20:20.229 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-27T09:20:20.229 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-27T09:20:20.229 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-27T09:20:20.229 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.blob_file_starting_level: 0 2024-05-27T09:20:20.229 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-27T09:20:20.229 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi138/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-27T09:20:20.229 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-27T09:20:20.229 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 565d2627-d6ba-44a0-823b-2c4b78d6dd5d 2024-05-27T09:20:20.229 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716801619966883, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-27T09:20:20.229 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-27T09:20:20.229 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716801619967865, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716801619, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "565d2627-d6ba-44a0-823b-2c4b78d6dd5d", "db_session_id": "819EP3KCD61NA66A2ASZ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-27T09:20:20.229 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716801619968103, "job": 1, "event": "recovery_finished"} 2024-05-27T09:20:20.230 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-27T09:20:20.230 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi138/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-27T09:20:20.230 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a37f2f8000 2024-05-27T09:20:20.230 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: DB pointer 0x55a37f2e4000 2024-05-27T09:20:20.230 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: mon.smithi138 does not exist in monmap, will attempt to join an existing cluster 2024-05-27T09:20:20.230 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-27T09:20:20.230 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-27T09:20:20.230 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: ** DB Stats ** 2024-05-27T09:20:20.230 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T09:20:20.230 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-27T09:20:20.230 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T09:20:20.230 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T09:20:20.230 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-27T09:20:20.231 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T09:20:20.231 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T09:20:20.231 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-27T09:20:20.231 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: ** Compaction Stats [default] ** 2024-05-27T09:20:20.231 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T09:20:20.231 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-27T09:20:20.231 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T09:20:20.231 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T09:20:20.231 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T09:20:20.231 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-27T09:20:20.231 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: ** Compaction Stats [default] ** 2024-05-27T09:20:20.231 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T09:20:20.231 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-27T09:20:20.231 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T09:20:20.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-27T09:20:20.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-27T09:20:20.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-27T09:20:20.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T09:20:20.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-27T09:20:20.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-27T09:20:20.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-27T09:20:20.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-27T09:20:20.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-27T09:20:20.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Cumulative compaction: 0.00 GB write, 0.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T09:20:20.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Interval compaction: 0.00 GB write, 0.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T09:20:20.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-27T09:20:20.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Block cache BinnedLRUCache@0x55a37f25b350#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-05-27T09:20:20.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-27T09:20:20.232 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-27T09:20:20.233 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-27T09:20:20.233 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: using public_addr v2:172.21.15.138:0/0 -> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] 2024-05-27T09:20:20.233 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: starting mon.smithi138 rank -1 at public addrs [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] at bind addrs [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi138 fsid b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:20:20.233 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:19 smithi138 ceph-mon[34885]: mon.smithi138@-1(???) e0 preinit fsid b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:20:20.233 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: mon.smithi138@-1(synchronizing).mds e1 new map 2024-05-27T09:20:20.233 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: mon.smithi138@-1(synchronizing).mds e1 print_map 2024-05-27T09:20:20.233 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: e1 2024-05-27T09:20:20.233 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T09:20:20.234 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:20:20.234 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: legacy client fscid: -1 2024-05-27T09:20:20.234 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-27T09:20:20.234 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: No filesystems configured 2024-05-27T09:20:20.234 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: mon.smithi138@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-27T09:20:20.234 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: mon.smithi138@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-27T09:20:20.234 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: mon.smithi138@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-27T09:20:20.234 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: mon.smithi138@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-27T09:20:20.234 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: mon.smithi138@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-27T09:20:20.235 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: mon.smithi138@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-27T09:20:20.235 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: mon.smithi138@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-05-27T09:20:20.235 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: mon.smithi138@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-27T09:20:20.235 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: mon.smithi138@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T09:20:20.235 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: mon.smithi138@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T09:20:20.235 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: mon.smithi138@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T09:20:20.235 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.235 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.235 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd/host:smithi119", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:20:20.235 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:20:20.235 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:20:20.235 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: Updating smithi119:/etc/ceph/ceph.conf 2024-05-27T09:20:20.235 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: Updating smithi138:/etc/ceph/ceph.conf 2024-05-27T09:20:20.236 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: Updating smithi138:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:20:20.236 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: Updating smithi119:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:20:20.236 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='client.? 172.21.15.138:0/1190011489' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:20:20.236 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-05-27T09:20:20.236 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-05-27T09:20:20.236 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: Updating smithi138:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T09:20:20.236 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: Updating smithi119:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T09:20:20.236 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.236 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.236 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.236 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.236 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.236 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi138", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-27T09:20:20.237 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi138", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-27T09:20:20.237 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:20:20.237 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: Deploying daemon ceph-exporter.smithi138 on smithi138 2024-05-27T09:20:20.237 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:20:20.237 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='client.? 172.21.15.138:0/3163927046' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:20:20.237 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.237 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.237 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.237 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.237 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-27T09:20:20.237 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-05-27T09:20:20.237 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:20:20.237 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: Deploying daemon crash.smithi138 on smithi138 2024-05-27T09:20:20.237 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:20.237 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.238 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='client.? 172.21.15.138:0/2060881437' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:20:20.238 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:20.238 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.238 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.238 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.238 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.238 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: Deploying daemon node-exporter.smithi138 on smithi138 2024-05-27T09:20:20.238 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:20.238 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.238 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='client.? 172.21.15.138:0/990671078' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:20:20.238 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:20.238 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:20.238 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:20:20.239 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:20.239 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.239 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.239 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.239 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.239 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.dxsuqf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T09:20:20.239 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.dxsuqf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T09:20:20.239 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T09:20:20.239 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:20:20.239 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: Deploying daemon mgr.smithi138.dxsuqf on smithi138 2024-05-27T09:20:20.239 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='client.? 172.21.15.138:0/1323564809' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:20:20.239 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:20.239 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:20.239 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.240 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:20.240 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.240 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.240 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.240 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.240 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T09:20:20.240 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:20:20.240 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: Deploying daemon mon.smithi138 on smithi138 2024-05-27T09:20:20.240 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='client.? 172.21.15.138:0/4215545472' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:20:20.240 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:20.240 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:20.240 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:20.240 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:20.240 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 ceph-mon[34885]: mon.smithi138@-1(synchronizing).paxosservice(auth 1..8) refresh upgraded, format 0 -> 3 2024-05-27T09:20:20.240 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 bash[34849]: 60216fd6707805237264b6996f23ce90dca1fe616ecf272aa322f90c92d16471 2024-05-27T09:20:20.241 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:20 smithi138 systemd[1]: Started Ceph mon.smithi138 for b966b06e-1c09-11ef-bc9b-c7b262605968. 2024-05-27T09:20:21.670 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-27T09:20:21.670 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T09:20:22.030 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi138/config 2024-05-27T09:20:25.244 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi119"}]: dispatch 2024-05-27T09:20:25.244 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: mon.smithi119 calling monitor election 2024-05-27T09:20:25.244 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:20:25.244 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:25.244 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:20:25.244 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:20:25.244 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.138:0/677592363' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.dxsuqf/crt"}]: dispatch 2024-05-27T09:20:25.244 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:20:25.244 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: mon.smithi138 calling monitor election 2024-05-27T09:20:25.245 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:25.245 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:20:25.245 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:20:25.245 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:25.245 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:20:25.245 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: mon.smithi119 is new leader, mons smithi119,smithi138 in quorum (ranks 0,1) 2024-05-27T09:20:25.245 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: monmap e2: 2 mons at {smithi119=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],smithi138=[v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0]} removed_ranks: {} 2024-05-27T09:20:25.245 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: fsmap 2024-05-27T09:20:25.245 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: osdmap e5: 0 total, 0 up, 0 in 2024-05-27T09:20:25.245 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: mgrmap e17: smithi119.ncwmog(active, since 48s) 2024-05-27T09:20:25.245 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: Standby manager daemon smithi138.dxsuqf started 2024-05-27T09:20:25.245 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.138:0/677592363' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T09:20:25.245 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: overall HEALTH_OK 2024-05-27T09:20:25.246 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.138:0/677592363' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.dxsuqf/key"}]: dispatch 2024-05-27T09:20:25.246 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.138:0/677592363' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T09:20:25.246 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:25.246 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:25 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:25.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi119"}]: dispatch 2024-05-27T09:20:25.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: mon.smithi119 calling monitor election 2024-05-27T09:20:25.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:20:25.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:25.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:20:25.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:20:25.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.138:0/677592363' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.dxsuqf/crt"}]: dispatch 2024-05-27T09:20:25.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:20:25.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: mon.smithi138 calling monitor election 2024-05-27T09:20:25.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:25.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:20:25.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:20:25.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:25.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:20:25.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: mon.smithi119 is new leader, mons smithi119,smithi138 in quorum (ranks 0,1) 2024-05-27T09:20:25.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: monmap e2: 2 mons at {smithi119=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0],smithi138=[v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0]} removed_ranks: {} 2024-05-27T09:20:25.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: fsmap 2024-05-27T09:20:25.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: osdmap e5: 0 total, 0 up, 0 in 2024-05-27T09:20:25.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: mgrmap e17: smithi119.ncwmog(active, since 48s) 2024-05-27T09:20:25.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: Standby manager daemon smithi138.dxsuqf started 2024-05-27T09:20:25.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.138:0/677592363' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T09:20:25.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: overall HEALTH_OK 2024-05-27T09:20:25.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.138:0/677592363' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.dxsuqf/key"}]: dispatch 2024-05-27T09:20:25.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.138:0/677592363' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T09:20:25.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:25.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:25 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:25.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.581+0000 7ff9e9400700 1 -- 172.21.15.138:0/1707939148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9e41023b0 msgr2=0x7ff9e4102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:20:25.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.581+0000 7ff9e9400700 1 --2- 172.21.15.138:0/1707939148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9e41023b0 0x7ff9e4102780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4005fa0 tx=0x7ff9d400fe80 comp rx=0 tx=0).stop 2024-05-27T09:20:25.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.581+0000 7ff9e9400700 1 -- 172.21.15.138:0/1707939148 shutdown_connections 2024-05-27T09:20:25.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.581+0000 7ff9e9400700 1 --2- 172.21.15.138:0/1707939148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9e41023b0 0x7ff9e4102780 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:25.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.581+0000 7ff9e9400700 1 -- 172.21.15.138:0/1707939148 >> 172.21.15.138:0/1707939148 conn(0x7ff9e40fdc30 msgr2=0x7ff9e4100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:20:25.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.581+0000 7ff9e9400700 1 -- 172.21.15.138:0/1707939148 shutdown_connections 2024-05-27T09:20:25.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.581+0000 7ff9e9400700 1 -- 172.21.15.138:0/1707939148 wait complete. 2024-05-27T09:20:25.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.582+0000 7ff9e9400700 1 Processor -- start 2024-05-27T09:20:25.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.582+0000 7ff9e9400700 1 -- start start 2024-05-27T09:20:25.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.583+0000 7ff9e9400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9e41023b0 0x7ff9e41a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:20:25.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.583+0000 7ff9e9400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9e41a1ee0 0x7ff9e419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:20:25.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.583+0000 7ff9e9400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e41a2480 con 0x7ff9e41023b0 2024-05-27T09:20:25.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.583+0000 7ff9e9400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e419bed0 con 0x7ff9e41a1ee0 2024-05-27T09:20:25.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.583+0000 7ff9e1e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9e41a1ee0 0x7ff9e419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:20:25.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.583+0000 7ff9e2800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9e41023b0 0x7ff9e41a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:20:25.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.583+0000 7ff9e1e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9e41a1ee0 0x7ff9e419b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37748/0 (socket says 172.21.15.138:37748) 2024-05-27T09:20:25.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.583+0000 7ff9e1e00700 1 -- 172.21.15.138:0/191797563 learned_addr learned my addr 172.21.15.138:0/191797563 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:20:25.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.584+0000 7ff9e1e00700 1 -- 172.21.15.138:0/191797563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9e41a1ee0 msgr2=0x7ff9e419b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-27T09:20:25.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.584+0000 7ff9e1e00700 1 -- 172.21.15.138:0/191797563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9e41a1ee0 msgr2=0x7ff9e419b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-27T09:20:25.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.584+0000 7ff9e1e00700 1 --2- 172.21.15.138:0/191797563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9e41a1ee0 0x7ff9e419b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T09:20:25.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.584+0000 7ff9e1e00700 1 --2- 172.21.15.138:0/191797563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9e41a1ee0 0x7ff9e419b990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T09:20:25.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.584+0000 7ff9e2800700 1 -- 172.21.15.138:0/191797563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9e41a1ee0 msgr2=0x7ff9e419b990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T09:20:25.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.584+0000 7ff9e2800700 1 --2- 172.21.15.138:0/191797563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9e41a1ee0 0x7ff9e419b990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:25.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.584+0000 7ff9e2800700 1 -- 172.21.15.138:0/191797563 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d400f880 con 0x7ff9e41023b0 2024-05-27T09:20:25.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.585+0000 7ff9e2800700 1 --2- 172.21.15.138:0/191797563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9e41023b0 0x7ff9e41a19a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4005fd0 tx=0x7ff9d4003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:20:25.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.585+0000 7ff9db400700 1 -- 172.21.15.138:0/191797563 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9d4021070 con 0x7ff9e41023b0 2024-05-27T09:20:25.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.585+0000 7ff9db400700 1 -- 172.21.15.138:0/191797563 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff9d4003e80 con 0x7ff9e41023b0 2024-05-27T09:20:25.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.585+0000 7ff9db400700 1 -- 172.21.15.138:0/191797563 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9d4012ea0 con 0x7ff9e41023b0 2024-05-27T09:20:25.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.585+0000 7ff9e9400700 1 -- 172.21.15.138:0/191797563 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e419c100 con 0x7ff9e41023b0 2024-05-27T09:20:25.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.586+0000 7ff9e9400700 1 -- 172.21.15.138:0/191797563 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e419c5f0 con 0x7ff9e41023b0 2024-05-27T09:20:25.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.587+0000 7ff9e9400700 1 -- 172.21.15.138:0/191797563 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e4062380 con 0x7ff9e41023b0 2024-05-27T09:20:25.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.589+0000 7ff9db400700 1 -- 172.21.15.138:0/191797563 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 89860+0+0 (secure 0 0 0) 0x7ff9d40126e0 con 0x7ff9e41023b0 2024-05-27T09:20:25.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.590+0000 7ff9db400700 1 --2- 172.21.15.138:0/191797563 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff9c806c7a0 0x7ff9c806ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:20:25.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.591+0000 7ff9db400700 1 -- 172.21.15.138:0/191797563 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7ff9d4018070 con 0x7ff9e41023b0 2024-05-27T09:20:25.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.591+0000 7ff9e1e00700 1 --2- 172.21.15.138:0/191797563 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff9c806c7a0 0x7ff9c806ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:20:25.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.592+0000 7ff9e1e00700 1 --2- 172.21.15.138:0/191797563 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff9c806c7a0 0x7ff9c806ec50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0009180 tx=0x7ff9d0008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:20:25.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.603+0000 7ff9db400700 1 -- 172.21.15.138:0/191797563 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff9d4057fb0 con 0x7ff9e41023b0 2024-05-27T09:20:25.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.912+0000 7ff9e9400700 1 -- 172.21.15.138:0/191797563 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff9e41a6740 con 0x7ff9e41023b0 2024-05-27T09:20:25.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.913+0000 7ff9db400700 1 -- 172.21.15.138:0/191797563 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1026 (secure 0 0 0) 0x7ff9d402a090 con 0x7ff9e41023b0 2024-05-27T09:20:25.914 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-27T09:20:25.914 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":2,"fsid":"b966b06e-1c09-11ef-bc9b-c7b262605968","modified":"2024-05-27T09:20:20.043858Z","created":"2024-05-27T09:16:53.014813Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi119","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi138","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3300","nonce":0},{"type":"v1","addr":"172.21.15.138:6789","nonce":0}]},"addr":"172.21.15.138:6789/0","public_addr":"172.21.15.138:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-27T09:20:25.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.917+0000 7ff9e9400700 1 -- 172.21.15.138:0/191797563 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff9c806c7a0 msgr2=0x7ff9c806ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:20:25.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.917+0000 7ff9e9400700 1 --2- 172.21.15.138:0/191797563 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff9c806c7a0 0x7ff9c806ec50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0009180 tx=0x7ff9d0008040 comp rx=0 tx=0).stop 2024-05-27T09:20:25.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.917+0000 7ff9e9400700 1 -- 172.21.15.138:0/191797563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9e41023b0 msgr2=0x7ff9e41a19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:20:25.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.917+0000 7ff9e9400700 1 --2- 172.21.15.138:0/191797563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9e41023b0 0x7ff9e41a19a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4005fd0 tx=0x7ff9d4003c60 comp rx=0 tx=0).stop 2024-05-27T09:20:25.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.917+0000 7ff9e9400700 1 -- 172.21.15.138:0/191797563 shutdown_connections 2024-05-27T09:20:25.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.917+0000 7ff9e9400700 1 --2- 172.21.15.138:0/191797563 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff9c806c7a0 0x7ff9c806ec50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:25.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.917+0000 7ff9e9400700 1 --2- 172.21.15.138:0/191797563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9e41023b0 0x7ff9e41a19a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:25.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.917+0000 7ff9e9400700 1 --2- 172.21.15.138:0/191797563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9e41a1ee0 0x7ff9e419b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:25.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.917+0000 7ff9e9400700 1 -- 172.21.15.138:0/191797563 >> 172.21.15.138:0/191797563 conn(0x7ff9e40fdc30 msgr2=0x7ff9e4107e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:20:25.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.917+0000 7ff9e9400700 1 -- 172.21.15.138:0/191797563 shutdown_connections 2024-05-27T09:20:25.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:20:25.917+0000 7ff9e9400700 1 -- 172.21.15.138:0/191797563 wait complete. 2024-05-27T09:20:25.919 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 2 2024-05-27T09:20:26.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:26 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:26.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:26 smithi138 ceph-mon[34885]: mgrmap e18: smithi119.ncwmog(active, since 48s), standbys: smithi138.dxsuqf 2024-05-27T09:20:26.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:26 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr metadata", "who": "smithi138.dxsuqf", "id": "smithi138.dxsuqf"}]: dispatch 2024-05-27T09:20:26.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:26 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:26.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:26 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:26.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:26 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:20:26.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:26 smithi138 ceph-mon[34885]: from='client.? 172.21.15.138:0/191797563' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:20:26.325 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:26 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:20:26.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:26 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:26.677 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:26 smithi119 ceph-mon[29082]: mgrmap e18: smithi119.ncwmog(active, since 48s), standbys: smithi138.dxsuqf 2024-05-27T09:20:26.678 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:26 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr metadata", "who": "smithi138.dxsuqf", "id": "smithi138.dxsuqf"}]: dispatch 2024-05-27T09:20:26.678 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:26 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:26.678 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:26 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:26.678 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:26 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:20:26.678 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:26 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/191797563' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T09:20:26.678 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:26 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:20:26.728 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-27T09:20:26.728 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-05-27T09:20:27.132 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:20:27.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:27 smithi138 ceph-mon[34885]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:27.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:27 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:27.530 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:27 smithi119 ceph-mon[29082]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:27.530 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:27 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:29.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.234+0000 7ff16a800700 1 -- 172.21.15.119:0/653719662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1641000b0 msgr2=0x7ff164100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:20:29.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.234+0000 7ff16a800700 1 --2- 172.21.15.119:0/653719662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1641000b0 0x7ff164100480 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff154009910 tx=0x7ff154009c20 comp rx=0 tx=0).stop 2024-05-27T09:20:29.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.234+0000 7ff16a800700 1 -- 172.21.15.119:0/653719662 shutdown_connections 2024-05-27T09:20:29.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.234+0000 7ff16a800700 1 --2- 172.21.15.119:0/653719662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1641000b0 0x7ff164100480 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:29.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.234+0000 7ff16a800700 1 -- 172.21.15.119:0/653719662 >> 172.21.15.119:0/653719662 conn(0x7ff1640fb990 msgr2=0x7ff1640fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:20:29.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.234+0000 7ff16a800700 1 -- 172.21.15.119:0/653719662 shutdown_connections 2024-05-27T09:20:29.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.234+0000 7ff16a800700 1 -- 172.21.15.119:0/653719662 wait complete. 2024-05-27T09:20:29.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.235+0000 7ff16a800700 1 Processor -- start 2024-05-27T09:20:29.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.236+0000 7ff16a800700 1 -- start start 2024-05-27T09:20:29.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.236+0000 7ff16a800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1641000b0 0x7ff164073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:20:29.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.236+0000 7ff16a800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff164073590 0x7ff164073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:20:29.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.236+0000 7ff16a800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1641a16e0 con 0x7ff164073590 2024-05-27T09:20:29.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.236+0000 7ff16a800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1641a1820 con 0x7ff1641000b0 2024-05-27T09:20:29.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.236+0000 7ff162a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff164073590 0x7ff164073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:20:29.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.236+0000 7ff162a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff164073590 0x7ff164073a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48786/0 (socket says 172.21.15.119:48786) 2024-05-27T09:20:29.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.236+0000 7ff162a00700 1 -- 172.21.15.119:0/686180244 learned_addr learned my addr 172.21.15.119:0/686180244 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:20:29.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.237+0000 7ff163400700 1 --2- 172.21.15.119:0/686180244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1641000b0 0x7ff164073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:20:29.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.237+0000 7ff162a00700 1 -- 172.21.15.119:0/686180244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1641000b0 msgr2=0x7ff164073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:20:29.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.237+0000 7ff162a00700 1 --2- 172.21.15.119:0/686180244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1641000b0 0x7ff164073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:29.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.237+0000 7ff162a00700 1 -- 172.21.15.119:0/686180244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1540095f0 con 0x7ff164073590 2024-05-27T09:20:29.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.237+0000 7ff162a00700 1 --2- 172.21.15.119:0/686180244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff164073590 0x7ff164073a00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff15000b730 tx=0x7ff15000baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:20:29.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.238+0000 7ff15be00700 1 -- 172.21.15.119:0/686180244 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff150004200 con 0x7ff164073590 2024-05-27T09:20:29.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.238+0000 7ff16a800700 1 -- 172.21.15.119:0/686180244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1641a1960 con 0x7ff164073590 2024-05-27T09:20:29.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.238+0000 7ff15be00700 1 -- 172.21.15.119:0/686180244 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff150004360 con 0x7ff164073590 2024-05-27T09:20:29.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.238+0000 7ff15be00700 1 -- 172.21.15.119:0/686180244 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff150011be0 con 0x7ff164073590 2024-05-27T09:20:29.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.238+0000 7ff16a800700 1 -- 172.21.15.119:0/686180244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1641a1eb0 con 0x7ff164073590 2024-05-27T09:20:29.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.239+0000 7ff16a800700 1 -- 172.21.15.119:0/686180244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff164062380 con 0x7ff164073590 2024-05-27T09:20:29.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.240+0000 7ff15be00700 1 -- 172.21.15.119:0/686180244 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 89860+0+0 (secure 0 0 0) 0x7ff1500036a0 con 0x7ff164073590 2024-05-27T09:20:29.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.241+0000 7ff15be00700 1 --2- 172.21.15.119:0/686180244 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff14806c760 0x7ff14806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:20:29.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.241+0000 7ff15be00700 1 -- 172.21.15.119:0/686180244 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7ff15008a3c0 con 0x7ff164073590 2024-05-27T09:20:29.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.242+0000 7ff163400700 1 --2- 172.21.15.119:0/686180244 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff14806c760 0x7ff14806ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:20:29.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.243+0000 7ff163400700 1 --2- 172.21.15.119:0/686180244 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff14806c760 0x7ff14806ec10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff15400b5c0 tx=0x7ff1540058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:20:29.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.246+0000 7ff15be00700 1 -- 172.21.15.119:0/686180244 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff150055ae0 con 0x7ff164073590 2024-05-27T09:20:29.417 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:29 smithi119 ceph-mon[29082]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:29.417 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:29 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:29.417 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:29 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:29.417 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:29 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:20:29.417 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:29 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:20:29.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.469+0000 7ff16a800700 1 -- 172.21.15.119:0/686180244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7ff164074580 con 0x7ff164073590 2024-05-27T09:20:29.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.470+0000 7ff15be00700 1 -- 172.21.15.119:0/686180244 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+227 (secure 0 0 0) 0x7ff150059100 con 0x7ff164073590 2024-05-27T09:20:29.472 INFO:teuthology.orchestra.run.smithi119.stdout:# minimal ceph.conf for b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:20:29.472 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2024-05-27T09:20:29.472 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:20:29.472 INFO:teuthology.orchestra.run.smithi119.stdout: mon_host = [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] 2024-05-27T09:20:29.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.472+0000 7ff16a800700 1 -- 172.21.15.119:0/686180244 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff14806c760 msgr2=0x7ff14806ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:20:29.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.472+0000 7ff16a800700 1 --2- 172.21.15.119:0/686180244 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff14806c760 0x7ff14806ec10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff15400b5c0 tx=0x7ff1540058e0 comp rx=0 tx=0).stop 2024-05-27T09:20:29.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.472+0000 7ff16a800700 1 -- 172.21.15.119:0/686180244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff164073590 msgr2=0x7ff164073a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:20:29.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.472+0000 7ff16a800700 1 --2- 172.21.15.119:0/686180244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff164073590 0x7ff164073a00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff15000b730 tx=0x7ff15000baf0 comp rx=0 tx=0).stop 2024-05-27T09:20:29.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.472+0000 7ff16a800700 1 -- 172.21.15.119:0/686180244 shutdown_connections 2024-05-27T09:20:29.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.472+0000 7ff16a800700 1 --2- 172.21.15.119:0/686180244 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff14806c760 0x7ff14806ec10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:29.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.472+0000 7ff16a800700 1 --2- 172.21.15.119:0/686180244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1641000b0 0x7ff164073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:29.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.472+0000 7ff16a800700 1 --2- 172.21.15.119:0/686180244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff164073590 0x7ff164073a00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:29.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.472+0000 7ff16a800700 1 -- 172.21.15.119:0/686180244 >> 172.21.15.119:0/686180244 conn(0x7ff1640fb990 msgr2=0x7ff164105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:20:29.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.473+0000 7ff16a800700 1 -- 172.21.15.119:0/686180244 shutdown_connections 2024-05-27T09:20:29.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:29.473+0000 7ff16a800700 1 -- 172.21.15.119:0/686180244 wait complete. 2024-05-27T09:20:29.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:29 smithi138 ceph-mon[34885]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:29.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:29 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:29.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:29 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:29.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:29 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:20:29.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:29 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:20:30.275 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-27T09:20:30.275 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-27T09:20:30.275 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.conf 2024-05-27T09:20:30.356 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-27T09:20:30.356 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T09:20:30.459 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-27T09:20:30.459 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.conf 2024-05-27T09:20:30.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:30 smithi119 ceph-mon[29082]: Updating smithi119:/etc/ceph/ceph.conf 2024-05-27T09:20:30.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:30 smithi119 ceph-mon[29082]: Updating smithi138:/etc/ceph/ceph.conf 2024-05-27T09:20:30.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:30 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/686180244' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:20:30.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:30 smithi119 ceph-mon[29082]: Updating smithi119:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:20:30.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:30 smithi119 ceph-mon[29082]: Updating smithi138:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:20:30.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:30 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:30.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:30 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:30.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:30 smithi138 ceph-mon[34885]: Updating smithi119:/etc/ceph/ceph.conf 2024-05-27T09:20:30.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:30 smithi138 ceph-mon[34885]: Updating smithi138:/etc/ceph/ceph.conf 2024-05-27T09:20:30.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:30 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/686180244' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:20:30.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:30 smithi138 ceph-mon[34885]: Updating smithi119:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:20:30.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:30 smithi138 ceph-mon[34885]: Updating smithi138:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:20:30.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:30 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:30.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:30 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:30.543 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-27T09:20:30.543 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T09:20:30.660 INFO:tasks.cephadm:Deploying OSDs... 2024-05-27T09:20:30.660 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-27T09:20:30.660 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T09:20:30.699 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T09:20:30.699 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_1 2024-05-27T09:20:30.779 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T09:20:30.779 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T09:20:30.779 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-05-27T09:20:30.779 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T09:20:30.779 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T09:20:30.779 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-05-27 09:19:25.496469457 +0000 2024-05-27T09:20:30.779 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-05-27 09:12:27.948186304 +0000 2024-05-27T09:20:30.779 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-05-27 09:12:27.948186304 +0000 2024-05-27T09:20:30.779 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-05-27 09:12:27.948186304 +0000 2024-05-27T09:20:30.780 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T09:20:30.895 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-05-27T09:20:30.895 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-05-27T09:20:30.895 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000440228 s, 1.2 MB/s 2024-05-27T09:20:30.898 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T09:20:30.947 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_2 2024-05-27T09:20:30.989 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T09:20:30.989 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T09:20:30.989 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-05-27T09:20:30.989 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T09:20:30.989 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T09:20:30.989 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-05-27 09:19:25.496469457 +0000 2024-05-27T09:20:30.989 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-05-27 09:12:28.504172506 +0000 2024-05-27T09:20:30.989 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-05-27 09:12:28.504172506 +0000 2024-05-27T09:20:30.989 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-05-27 09:12:28.504172506 +0000 2024-05-27T09:20:30.990 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T09:20:31.090 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-05-27T09:20:31.091 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-05-27T09:20:31.091 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.00033391 s, 1.5 MB/s 2024-05-27T09:20:31.093 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T09:20:31.134 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_3 2024-05-27T09:20:31.216 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T09:20:31.216 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T09:20:31.216 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-05-27T09:20:31.216 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T09:20:31.217 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T09:20:31.217 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-05-27 09:19:25.497469433 +0000 2024-05-27T09:20:31.217 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-05-27 09:12:29.095157839 +0000 2024-05-27T09:20:31.217 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-05-27 09:12:29.095157839 +0000 2024-05-27T09:20:31.217 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-05-27 09:12:29.095157839 +0000 2024-05-27T09:20:31.217 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T09:20:31.325 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-05-27T09:20:31.325 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-05-27T09:20:31.325 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000392997 s, 1.3 MB/s 2024-05-27T09:20:31.328 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T09:20:31.375 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_4 2024-05-27T09:20:31.415 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T09:20:31.415 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T09:20:31.415 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-05-27T09:20:31.415 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T09:20:31.415 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T09:20:31.416 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-05-27 09:19:25.498469409 +0000 2024-05-27T09:20:31.416 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-05-27 09:12:29.706142676 +0000 2024-05-27T09:20:31.416 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-05-27 09:12:29.706142676 +0000 2024-05-27T09:20:31.416 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-05-27 09:12:29.706142676 +0000 2024-05-27T09:20:31.416 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T09:20:31.515 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:31 smithi119 ceph-mon[29082]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:31.516 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:31 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:31.516 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:31 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:31.516 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:31 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:31.516 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:31 smithi119 ceph-mon[29082]: Reconfiguring mon.smithi119 (unknown last config time)... 2024-05-27T09:20:31.516 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:31 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T09:20:31.516 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:31 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T09:20:31.516 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:31 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:20:31.516 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:31 smithi119 ceph-mon[29082]: Reconfiguring daemon mon.smithi119 on smithi119 2024-05-27T09:20:31.529 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-05-27T09:20:31.529 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-05-27T09:20:31.530 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000280368 s, 1.8 MB/s 2024-05-27T09:20:31.533 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T09:20:31.578 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-27T09:20:31.578 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T09:20:31.635 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T09:20:31.635 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-05-27T09:20:31.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:31 smithi138 ceph-mon[34885]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:31.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:31 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:31.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:31 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:31.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:31 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:31.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:31 smithi138 ceph-mon[34885]: Reconfiguring mon.smithi119 (unknown last config time)... 2024-05-27T09:20:31.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:31 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T09:20:31.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:31 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T09:20:31.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:31 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:20:31.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:31 smithi138 ceph-mon[34885]: Reconfiguring daemon mon.smithi119 on smithi119 2024-05-27T09:20:31.686 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T09:20:31.686 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T09:20:31.686 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 635 Links: 1 2024-05-27T09:20:31.686 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T09:20:31.686 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T09:20:31.686 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-05-27 09:20:18.928285591 +0000 2024-05-27T09:20:31.686 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-05-27 09:12:31.539264553 +0000 2024-05-27T09:20:31.686 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-05-27 09:12:31.539264553 +0000 2024-05-27T09:20:31.686 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-05-27 09:12:31.539264553 +0000 2024-05-27T09:20:31.687 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T09:20:31.763 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-05-27T09:20:31.764 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-05-27T09:20:31.764 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000474566 s, 1.1 MB/s 2024-05-27T09:20:31.767 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T09:20:31.823 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-05-27T09:20:31.872 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T09:20:31.872 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T09:20:31.872 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-05-27T09:20:31.872 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T09:20:31.872 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T09:20:31.872 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-05-27 09:20:18.929285570 +0000 2024-05-27T09:20:31.873 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-05-27 09:12:32.173251333 +0000 2024-05-27T09:20:31.873 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-05-27 09:12:32.173251333 +0000 2024-05-27T09:20:31.873 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-05-27 09:12:32.173251333 +0000 2024-05-27T09:20:31.873 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T09:20:31.951 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-05-27T09:20:31.951 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-05-27T09:20:31.951 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000391995 s, 1.3 MB/s 2024-05-27T09:20:31.955 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T09:20:32.011 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-05-27T09:20:32.064 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T09:20:32.064 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T09:20:32.064 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 674 Links: 1 2024-05-27T09:20:32.064 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T09:20:32.064 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T09:20:32.064 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-05-27 09:20:18.930285549 +0000 2024-05-27T09:20:32.064 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-05-27 09:12:32.782238634 +0000 2024-05-27T09:20:32.064 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-05-27 09:12:32.782238634 +0000 2024-05-27T09:20:32.064 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-05-27 09:12:32.782238634 +0000 2024-05-27T09:20:32.065 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T09:20:32.146 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-05-27T09:20:32.146 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-05-27T09:20:32.147 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000350155 s, 1.5 MB/s 2024-05-27T09:20:32.151 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T09:20:32.204 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-05-27T09:20:32.255 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T09:20:32.255 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T09:20:32.255 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 693 Links: 1 2024-05-27T09:20:32.256 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T09:20:32.256 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T09:20:32.256 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-05-27 09:20:18.931285529 +0000 2024-05-27T09:20:32.256 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-05-27 09:12:33.381226144 +0000 2024-05-27T09:20:32.256 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-05-27 09:12:33.381226144 +0000 2024-05-27T09:20:32.256 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-05-27 09:12:33.381226144 +0000 2024-05-27T09:20:32.256 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T09:20:32.336 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-05-27T09:20:32.336 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-05-27T09:20:32.336 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000373035 s, 1.4 MB/s 2024-05-27T09:20:32.340 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T09:20:32.396 INFO:tasks.cephadm:Deploying osd.0 on smithi119 with /dev/vg_nvme/lv_4... 2024-05-27T09:20:32.396 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-27T09:20:32.800 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:20:33.935 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:33 smithi138 ceph-mon[34885]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:33.935 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:33 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:33.935 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:33 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:33.935 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:33 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi119.ncwmog", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T09:20:33.935 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:33 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T09:20:33.935 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:33 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:20:33.959 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:33 smithi119 ceph-mon[29082]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:33.959 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:33 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:33.959 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:33 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:33.960 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:33 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi119.ncwmog", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T09:20:33.960 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:33 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T09:20:33.960 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:33 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:20:34.906 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:34 smithi119 ceph-mon[29082]: Reconfiguring mgr.smithi119.ncwmog (unknown last config time)... 2024-05-27T09:20:34.907 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:34 smithi119 ceph-mon[29082]: Reconfiguring daemon mgr.smithi119.ncwmog on smithi119 2024-05-27T09:20:35.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:34 smithi138 ceph-mon[34885]: Reconfiguring mgr.smithi119.ncwmog (unknown last config time)... 2024-05-27T09:20:35.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:34 smithi138 ceph-mon[34885]: Reconfiguring daemon mgr.smithi119.ncwmog on smithi119 2024-05-27T09:20:35.550 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-27T09:20:35.550 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-27T09:20:35.550 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2024-05-27T09:20:35.550 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2024-05-27T09:20:35.550 INFO:teuthology.orchestra.run.smithi119.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0337659 s, 311 MB/s 2024-05-27T09:20:35.550 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2024-05-27T09:20:35.987 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:35 smithi119 ceph-mon[29082]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:36.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:35 smithi138 ceph-mon[34885]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:36.358 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi119:vg_nvme/lv_4 2024-05-27T09:20:36.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:36 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:36.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:36 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:36.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:36 smithi138 ceph-mon[34885]: Reconfiguring ceph-exporter.smithi119 (monmap changed)... 2024-05-27T09:20:36.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:36 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi119", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-27T09:20:36.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:36 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:20:36.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:36 smithi138 ceph-mon[34885]: Reconfiguring daemon ceph-exporter.smithi119 on smithi119 2024-05-27T09:20:36.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:36 smithi138 ceph-mon[34885]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:36.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:36 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:20:36.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:36 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:36.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:36 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:36.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:36 smithi119 ceph-mon[29082]: Reconfiguring ceph-exporter.smithi119 (monmap changed)... 2024-05-27T09:20:36.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:36 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi119", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-27T09:20:36.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:36 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:20:36.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:36 smithi119 ceph-mon[29082]: Reconfiguring daemon ceph-exporter.smithi119 on smithi119 2024-05-27T09:20:36.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:36 smithi119 ceph-mon[29082]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:36.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:36 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:20:37.550 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:20:38.902 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:38 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:38.903 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:38 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:38.903 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:38 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi119", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-27T09:20:38.903 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:38 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:20:38.955 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:38 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:38.955 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:38 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:38.955 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:38 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi119", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-27T09:20:38.955 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:38 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:20:39.713 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:39 smithi119 ceph-mon[29082]: Reconfiguring crash.smithi119 (monmap changed)... 2024-05-27T09:20:39.713 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:39 smithi119 ceph-mon[29082]: Reconfiguring daemon crash.smithi119 on smithi119 2024-05-27T09:20:39.713 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:39 smithi119 ceph-mon[29082]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:39.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.870+0000 7f1fbec00700 1 -- 172.21.15.119:0/544252781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fb8101350 msgr2=0x7f1fb81017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:20:39.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.870+0000 7f1fbec00700 1 --2- 172.21.15.119:0/544252781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fb8101350 0x7f1fb81017a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4009910 tx=0x7f1fa4009c20 comp rx=0 tx=0).stop 2024-05-27T09:20:39.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.870+0000 7f1fbec00700 1 -- 172.21.15.119:0/544252781 shutdown_connections 2024-05-27T09:20:39.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.870+0000 7f1fbec00700 1 --2- 172.21.15.119:0/544252781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fb8101350 0x7f1fb81017a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:39.943 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.870+0000 7f1fbec00700 1 --2- 172.21.15.119:0/544252781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fb8100150 0x7f1fb8100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.870+0000 7f1fbec00700 1 -- 172.21.15.119:0/544252781 >> 172.21.15.119:0/544252781 conn(0x7f1fb80fb6e0 msgr2=0x7f1fb80fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:20:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.870+0000 7f1fbec00700 1 -- 172.21.15.119:0/544252781 shutdown_connections 2024-05-27T09:20:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.870+0000 7f1fbec00700 1 -- 172.21.15.119:0/544252781 wait complete. 2024-05-27T09:20:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.871+0000 7f1fbec00700 1 Processor -- start 2024-05-27T09:20:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.872+0000 7f1fbec00700 1 -- start start 2024-05-27T09:20:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.872+0000 7f1fbec00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fb8100150 0x7f1fb8197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:20:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.872+0000 7f1fbec00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fb8101350 0x7f1fb81980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:20:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.872+0000 7f1fbec00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb81986d0 con 0x7f1fb8101350 2024-05-27T09:20:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.872+0000 7f1fbec00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb8198810 con 0x7f1fb8100150 2024-05-27T09:20:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.872+0000 7f1fb7400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fb8101350 0x7f1fb81980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:20:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.872+0000 7f1fb7400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fb8101350 0x7f1fb81980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40310/0 (socket says 172.21.15.119:40310) 2024-05-27T09:20:39.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.872+0000 7f1fb7400700 1 -- 172.21.15.119:0/2405565385 learned_addr learned my addr 172.21.15.119:0/2405565385 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:20:39.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.873+0000 7f1fb7e00700 1 --2- 172.21.15.119:0/2405565385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fb8100150 0x7f1fb8197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:20:39.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.873+0000 7f1fb7400700 1 -- 172.21.15.119:0/2405565385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fb8100150 msgr2=0x7f1fb8197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:20:39.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.873+0000 7f1fb7400700 1 --2- 172.21.15.119:0/2405565385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fb8100150 0x7f1fb8197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:20:39.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.873+0000 7f1fb7400700 1 -- 172.21.15.119:0/2405565385 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa40095f0 con 0x7f1fb8101350 2024-05-27T09:20:39.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.873+0000 7f1fb7400700 1 --2- 172.21.15.119:0/2405565385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fb8101350 0x7f1fb81980b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4005b40 tx=0x7f1fa400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:20:39.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.873+0000 7f1fb4c00700 1 -- 172.21.15.119:0/2405565385 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fa401d070 con 0x7f1fb8101350 2024-05-27T09:20:39.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.873+0000 7f1fb4c00700 1 -- 172.21.15.119:0/2405565385 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1fa4022470 con 0x7f1fb8101350 2024-05-27T09:20:39.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.873+0000 7f1fbec00700 1 -- 172.21.15.119:0/2405565385 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb819d260 con 0x7f1fb8101350 2024-05-27T09:20:39.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.873+0000 7f1fb4c00700 1 -- 172.21.15.119:0/2405565385 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fa400f690 con 0x7f1fb8101350 2024-05-27T09:20:39.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.874+0000 7f1fbec00700 1 -- 172.21.15.119:0/2405565385 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb819d750 con 0x7f1fb8101350 2024-05-27T09:20:39.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.875+0000 7f1fbec00700 1 -- 172.21.15.119:0/2405565385 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb8105430 con 0x7f1fb8101350 2024-05-27T09:20:39.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.876+0000 7f1fb4c00700 1 -- 172.21.15.119:0/2405565385 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 89860+0+0 (secure 0 0 0) 0x7f1fa40225e0 con 0x7f1fb8101350 2024-05-27T09:20:39.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.877+0000 7f1fb4c00700 1 --2- 172.21.15.119:0/2405565385 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1f9c06c610 0x7f1f9c06eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:20:39.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.877+0000 7f1fb4c00700 1 -- 172.21.15.119:0/2405565385 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f1fa408bff0 con 0x7f1fb8101350 2024-05-27T09:20:39.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.877+0000 7f1fb7e00700 1 --2- 172.21.15.119:0/2405565385 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1f9c06c610 0x7f1f9c06eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:20:39.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.878+0000 7f1fb7e00700 1 --2- 172.21.15.119:0/2405565385 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1f9c06c610 0x7f1f9c06eac0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1fb81011b0 tx=0x7f1fa8008640 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:20:39.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:39.882+0000 7f1fb4c00700 1 -- 172.21.15.119:0/2405565385 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1fa40be720 con 0x7f1fb8101350 2024-05-27T09:20:40.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:20:40.109+0000 7f1fbec00700 1 -- 172.21.15.119:0/2405565385 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fb8061190 con 0x7f1f9c06c610 2024-05-27T09:20:40.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:39 smithi138 ceph-mon[34885]: Reconfiguring crash.smithi119 (monmap changed)... 2024-05-27T09:20:40.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:39 smithi138 ceph-mon[34885]: Reconfiguring daemon crash.smithi119 on smithi119 2024-05-27T09:20:40.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:39 smithi138 ceph-mon[34885]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:41.144 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:40 smithi119 ceph-mon[29082]: from='client.14254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:20:41.144 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:40 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T09:20:41.144 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:40 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T09:20:41.144 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:40 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:20:41.144 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:40 smithi119 ceph-mon[29082]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:41.144 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:40 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:41.144 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:40 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:41.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:40 smithi138 ceph-mon[34885]: from='client.14254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:20:41.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:40 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T09:20:41.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:40 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T09:20:41.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:40 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:20:41.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:40 smithi138 ceph-mon[34885]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:41.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:40 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:41.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:40 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:42.071 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:41 smithi119 ceph-mon[29082]: Reconfiguring alertmanager.smithi119 (dependencies changed)... 2024-05-27T09:20:42.072 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:41 smithi119 ceph-mon[29082]: Reconfiguring daemon alertmanager.smithi119 on smithi119 2024-05-27T09:20:42.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:41 smithi138 ceph-mon[34885]: Reconfiguring alertmanager.smithi119 (dependencies changed)... 2024-05-27T09:20:42.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:41 smithi138 ceph-mon[34885]: Reconfiguring daemon alertmanager.smithi119 on smithi119 2024-05-27T09:20:43.198 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:42 smithi119 ceph-mon[29082]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:43.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:42 smithi138 ceph-mon[34885]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:45.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:45 smithi119 ceph-mon[29082]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:45.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:45 smithi138 ceph-mon[34885]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:47.544 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:47 smithi119 ceph-mon[29082]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:47.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:47 smithi138 ceph-mon[34885]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:49.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:48 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:49.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:48 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:49.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:48 smithi119 ceph-mon[29082]: Reconfiguring grafana.smithi119 (dependencies changed)... 2024-05-27T09:20:49.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:48 smithi119 ceph-mon[29082]: Reconfiguring daemon grafana.smithi119 on smithi119 2024-05-27T09:20:49.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:48 smithi119 ceph-mon[29082]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:49.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:48 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:49.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:48 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:20:49.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:48 smithi138 ceph-mon[34885]: Reconfiguring grafana.smithi119 (dependencies changed)... 2024-05-27T09:20:49.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:48 smithi138 ceph-mon[34885]: Reconfiguring daemon grafana.smithi119 on smithi119 2024-05-27T09:20:49.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:48 smithi138 ceph-mon[34885]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:50.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:49 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/1044715830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3f78266-8de9-4e4f-85d6-7df2efa3b70e"}]: dispatch 2024-05-27T09:20:50.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:49 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/1044715830' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3f78266-8de9-4e4f-85d6-7df2efa3b70e"}]': finished 2024-05-27T09:20:50.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:49 smithi119 ceph-mon[29082]: osdmap e6: 1 total, 0 up, 1 in 2024-05-27T09:20:50.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:49 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T09:20:50.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:49 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/1044715830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3f78266-8de9-4e4f-85d6-7df2efa3b70e"}]: dispatch 2024-05-27T09:20:50.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:49 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/1044715830' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3f78266-8de9-4e4f-85d6-7df2efa3b70e"}]': finished 2024-05-27T09:20:50.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:49 smithi138 ceph-mon[34885]: osdmap e6: 1 total, 0 up, 1 in 2024-05-27T09:20:50.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:49 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T09:20:51.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:50 smithi119 ceph-mon[29082]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:51.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:50 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/3597070371' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T09:20:51.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:50 smithi138 ceph-mon[34885]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:51.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:50 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/3597070371' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T09:20:52.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:51 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:20:52.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:51 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:20:53.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:52 smithi119 ceph-mon[29082]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:53.286 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:52 smithi138 ceph-mon[34885]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:55.413 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:55 smithi119 ceph-mon[29082]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:55.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:55 smithi138 ceph-mon[34885]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:57.420 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:57 smithi119 ceph-mon[29082]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:57.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:57 smithi138 ceph-mon[34885]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:59.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:20:59 smithi138 ceph-mon[34885]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:20:59.464 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:20:59 smithi119 ceph-mon[29082]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:01.199 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:00 smithi119 ceph-mon[29082]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:01.200 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:00 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:01.200 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:00 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:01.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:00 smithi138 ceph-mon[34885]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:01.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:00 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:01.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:00 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:02.086 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:01 smithi119 ceph-mon[29082]: Reconfiguring prometheus.smithi119 (dependencies changed)... 2024-05-27T09:21:02.086 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:01 smithi119 ceph-mon[29082]: Reconfiguring daemon prometheus.smithi119 on smithi119 2024-05-27T09:21:02.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:01 smithi138 ceph-mon[34885]: Reconfiguring prometheus.smithi119 (dependencies changed)... 2024-05-27T09:21:02.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:01 smithi138 ceph-mon[34885]: Reconfiguring daemon prometheus.smithi119 on smithi119 2024-05-27T09:21:03.164 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:02 smithi119 ceph-mon[29082]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:03.165 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:02 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-27T09:21:03.165 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:02 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:21:03.165 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:02 smithi119 ceph-mon[29082]: Deploying daemon osd.0 on smithi119 2024-05-27T09:21:03.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:02 smithi138 ceph-mon[34885]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:03.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:02 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-27T09:21:03.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:02 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:21:03.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:02 smithi138 ceph-mon[34885]: Deploying daemon osd.0 on smithi119 2024-05-27T09:21:05.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:05 smithi138 ceph-mon[34885]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:05.737 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:05 smithi119 ceph-mon[29082]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:06.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:06 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:21:06.740 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:06 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:21:07.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:07 smithi119 ceph-mon[29082]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:07.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:07 smithi138 ceph-mon[34885]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:09.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:09 smithi138 ceph-mon[34885]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:09.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:09 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:09.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:09 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:09.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:09 smithi138 ceph-mon[34885]: Reconfiguring ceph-exporter.smithi138 (monmap changed)... 2024-05-27T09:21:09.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:09 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi138", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-27T09:21:09.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:09 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:21:09.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:09 smithi138 ceph-mon[34885]: Reconfiguring daemon ceph-exporter.smithi138 on smithi138 2024-05-27T09:21:09.717 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:09 smithi119 ceph-mon[29082]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:09.717 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:09 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:09.717 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:09 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:09.718 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:09 smithi119 ceph-mon[29082]: Reconfiguring ceph-exporter.smithi138 (monmap changed)... 2024-05-27T09:21:09.718 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:09 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi138", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-27T09:21:09.718 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:09 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:21:09.718 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:09 smithi119 ceph-mon[29082]: Reconfiguring daemon ceph-exporter.smithi138 on smithi138 2024-05-27T09:21:11.581 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:11 smithi119 ceph-mon[29082]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:11.581 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:11 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:11.582 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:11 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:11.582 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:11 smithi119 ceph-mon[29082]: Reconfiguring crash.smithi138 (monmap changed)... 2024-05-27T09:21:11.582 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:11 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-27T09:21:11.582 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:11 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:21:11.582 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:11 smithi119 ceph-mon[29082]: Reconfiguring daemon crash.smithi138 on smithi138 2024-05-27T09:21:11.792 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:11 smithi138 ceph-mon[34885]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:11.792 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:11 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:11.792 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:11 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:11.792 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:11 smithi138 ceph-mon[34885]: Reconfiguring crash.smithi138 (monmap changed)... 2024-05-27T09:21:11.792 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:11 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-27T09:21:11.792 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:11 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:21:11.792 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:11 smithi138 ceph-mon[34885]: Reconfiguring daemon crash.smithi138 on smithi138 2024-05-27T09:21:13.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:13 smithi119 ceph-mon[29082]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:13.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:13 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:13.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:13 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:13.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:13 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.dxsuqf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T09:21:13.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:13 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T09:21:13.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:13 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:21:13.660 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:13 smithi138 ceph-mon[34885]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:13.661 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:13 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:13.661 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:13 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:13.661 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:13 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.dxsuqf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T09:21:13.661 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:13 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T09:21:13.661 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:13 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:21:14.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:14 smithi119 ceph-mon[29082]: Reconfiguring mgr.smithi138.dxsuqf (monmap changed)... 2024-05-27T09:21:14.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:14 smithi119 ceph-mon[29082]: Reconfiguring daemon mgr.smithi138.dxsuqf on smithi138 2024-05-27T09:21:14.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:14 smithi138 ceph-mon[34885]: Reconfiguring mgr.smithi138.dxsuqf (monmap changed)... 2024-05-27T09:21:14.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:14 smithi138 ceph-mon[34885]: Reconfiguring daemon mgr.smithi138.dxsuqf on smithi138 2024-05-27T09:21:15.690 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:15 smithi138 ceph-mon[34885]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:15.691 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:15 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:15.691 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:15 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:15.691 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:15 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T09:21:15.691 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:15 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T09:21:15.691 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:15 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:21:15.871 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:15 smithi119 ceph-mon[29082]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:15.871 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:15 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:15.871 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:15 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:15.871 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:15 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T09:21:15.871 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:15 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T09:21:15.871 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:15 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:21:16.814 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:16 smithi119 ceph-mon[29082]: Reconfiguring mon.smithi138 (monmap changed)... 2024-05-27T09:21:16.814 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:16 smithi119 ceph-mon[29082]: Reconfiguring daemon mon.smithi138 on smithi138 2024-05-27T09:21:16.815 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:16 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:16.815 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:16 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:16.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:16 smithi138 ceph-mon[34885]: Reconfiguring mon.smithi138 (monmap changed)... 2024-05-27T09:21:16.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:16 smithi138 ceph-mon[34885]: Reconfiguring daemon mon.smithi138 on smithi138 2024-05-27T09:21:16.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:16 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:16.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:16 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:17.918 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:17 smithi138 ceph-mon[34885]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:17.918 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:17 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:17.918 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:17 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:17.918 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:17 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-27T09:21:17.918 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:17 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi119.front.sepia.ceph.com:9093"}]: dispatch 2024-05-27T09:21:17.918 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:17 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:17.918 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:17 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-27T09:21:17.919 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:17 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi119.front.sepia.ceph.com:3000"}]: dispatch 2024-05-27T09:21:17.919 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:17 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:17.919 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:17 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-27T09:21:17.919 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:17 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi119.front.sepia.ceph.com:9095"}]: dispatch 2024-05-27T09:21:17.919 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:17 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:17.919 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:17 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:21:17.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:17 smithi119 ceph-mon[29082]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:17.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:17 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:17.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:17 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:17.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:17 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-27T09:21:17.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:17 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi119.front.sepia.ceph.com:9093"}]: dispatch 2024-05-27T09:21:17.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:17 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:17.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:17 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-27T09:21:17.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:17 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi119.front.sepia.ceph.com:3000"}]: dispatch 2024-05-27T09:21:17.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:17 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:17.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:17 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-27T09:21:17.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:17 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi119.front.sepia.ceph.com:9095"}]: dispatch 2024-05-27T09:21:17.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:17 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:17.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:17 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:21:18.937 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:18 smithi119 ceph-mon[29082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-27T09:21:18.937 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:18 smithi119 ceph-mon[29082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi119.front.sepia.ceph.com:9093"}]: dispatch 2024-05-27T09:21:18.937 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:18 smithi119 ceph-mon[29082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-27T09:21:18.937 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:18 smithi119 ceph-mon[29082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi119.front.sepia.ceph.com:3000"}]: dispatch 2024-05-27T09:21:18.938 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:18 smithi119 ceph-mon[29082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-27T09:21:18.938 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:18 smithi119 ceph-mon[29082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi119.front.sepia.ceph.com:9095"}]: dispatch 2024-05-27T09:21:18.941 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:18 smithi138 ceph-mon[34885]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-27T09:21:18.941 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:18 smithi138 ceph-mon[34885]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi119.front.sepia.ceph.com:9093"}]: dispatch 2024-05-27T09:21:18.941 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:18 smithi138 ceph-mon[34885]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-27T09:21:18.942 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:18 smithi138 ceph-mon[34885]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi119.front.sepia.ceph.com:3000"}]: dispatch 2024-05-27T09:21:18.942 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:18 smithi138 ceph-mon[34885]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-27T09:21:18.942 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:18 smithi138 ceph-mon[34885]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi119.front.sepia.ceph.com:9095"}]: dispatch 2024-05-27T09:21:19.866 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:19 smithi119 ceph-mon[29082]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:19.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:19.941+0000 7f1fb4c00700 1 -- 172.21.15.119:0/2405565385 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1fb8061190 con 0x7f1f9c06c610 2024-05-27T09:21:19.942 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 0 on host 'smithi119' 2024-05-27T09:21:19.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:19.945+0000 7f1fbec00700 1 -- 172.21.15.119:0/2405565385 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1f9c06c610 msgr2=0x7f1f9c06eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:21:19.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:19.945+0000 7f1fbec00700 1 --2- 172.21.15.119:0/2405565385 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1f9c06c610 0x7f1f9c06eac0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1fb81011b0 tx=0x7f1fa8008640 comp rx=0 tx=0).stop 2024-05-27T09:21:19.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:19.945+0000 7f1fbec00700 1 -- 172.21.15.119:0/2405565385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fb8101350 msgr2=0x7f1fb81980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:21:19.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:19.945+0000 7f1fbec00700 1 --2- 172.21.15.119:0/2405565385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fb8101350 0x7f1fb81980b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4005b40 tx=0x7f1fa400be20 comp rx=0 tx=0).stop 2024-05-27T09:21:19.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:19.945+0000 7f1fbec00700 1 -- 172.21.15.119:0/2405565385 shutdown_connections 2024-05-27T09:21:19.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:19.945+0000 7f1fbec00700 1 --2- 172.21.15.119:0/2405565385 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1f9c06c610 0x7f1f9c06eac0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:21:19.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:19.945+0000 7f1fbec00700 1 --2- 172.21.15.119:0/2405565385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fb8100150 0x7f1fb8197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:21:19.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:19.945+0000 7f1fbec00700 1 --2- 172.21.15.119:0/2405565385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fb8101350 0x7f1fb81980b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:21:19.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:19.945+0000 7f1fbec00700 1 -- 172.21.15.119:0/2405565385 >> 172.21.15.119:0/2405565385 conn(0x7f1fb80fb6e0 msgr2=0x7f1fb8102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:21:19.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:19.945+0000 7f1fbec00700 1 -- 172.21.15.119:0/2405565385 shutdown_connections 2024-05-27T09:21:19.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:19.945+0000 7f1fbec00700 1 -- 172.21.15.119:0/2405565385 wait complete. 2024-05-27T09:21:20.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:19 smithi138 ceph-mon[34885]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:20.671 DEBUG:teuthology.orchestra.run.smithi119:osd.0> sudo journalctl -f -n 0 -u ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.0.service 2024-05-27T09:21:20.680 INFO:tasks.cephadm:Deploying osd.1 on smithi119 with /dev/vg_nvme/lv_3... 2024-05-27T09:21:20.680 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-27T09:21:20.995 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:20 smithi119 ceph-mon[29082]: from='osd.0 [v2:172.21.15.119:6802/448778921,v1:172.21.15.119:6803/448778921]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-27T09:21:20.995 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:20 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:20.995 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:20 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:20.995 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:20 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:20.995 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:20 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:21.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:20 smithi138 ceph-mon[34885]: from='osd.0 [v2:172.21.15.119:6802/448778921,v1:172.21.15.119:6803/448778921]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-27T09:21:21.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:21.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:21.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:21.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:21.439 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:21:21.941 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:21 smithi119 ceph-mon[29082]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:21.941 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:21 smithi119 ceph-mon[29082]: from='osd.0 [v2:172.21.15.119:6802/448778921,v1:172.21.15.119:6803/448778921]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-27T09:21:21.941 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:21 smithi119 ceph-mon[29082]: osdmap e7: 1 total, 0 up, 1 in 2024-05-27T09:21:21.941 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:21 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T09:21:21.941 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:21 smithi119 ceph-mon[29082]: from='osd.0 [v2:172.21.15.119:6802/448778921,v1:172.21.15.119:6803/448778921]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-05-27T09:21:21.941 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:21 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:21:21.942 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:21:21 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0[43827]: 2024-05-27T09:21:21.700+0000 7fadf7c00700 -1 osd.0 0 waiting for initial osdmap 2024-05-27T09:21:21.942 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:21:21 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0[43827]: 2024-05-27T09:21:21.711+0000 7fadef800700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T09:21:22.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:21 smithi138 ceph-mon[34885]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:22.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:21 smithi138 ceph-mon[34885]: from='osd.0 [v2:172.21.15.119:6802/448778921,v1:172.21.15.119:6803/448778921]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-27T09:21:22.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:21 smithi138 ceph-mon[34885]: osdmap e7: 1 total, 0 up, 1 in 2024-05-27T09:21:22.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:21 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T09:21:22.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:21 smithi138 ceph-mon[34885]: from='osd.0 [v2:172.21.15.119:6802/448778921,v1:172.21.15.119:6803/448778921]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-05-27T09:21:22.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:21 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:21:22.871 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:22 smithi119 ceph-mon[29082]: purged_snaps scrub starts 2024-05-27T09:21:22.871 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:22 smithi119 ceph-mon[29082]: purged_snaps scrub ok 2024-05-27T09:21:22.871 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:22 smithi119 ceph-mon[29082]: from='osd.0 [v2:172.21.15.119:6802/448778921,v1:172.21.15.119:6803/448778921]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-05-27T09:21:22.871 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:22 smithi119 ceph-mon[29082]: osdmap e8: 1 total, 0 up, 1 in 2024-05-27T09:21:22.872 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:22 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T09:21:22.872 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:22 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T09:21:22.872 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:22 smithi119 ceph-mon[29082]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:22.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:22 smithi138 ceph-mon[34885]: purged_snaps scrub starts 2024-05-27T09:21:22.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:22 smithi138 ceph-mon[34885]: purged_snaps scrub ok 2024-05-27T09:21:22.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:22 smithi138 ceph-mon[34885]: from='osd.0 [v2:172.21.15.119:6802/448778921,v1:172.21.15.119:6803/448778921]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-05-27T09:21:22.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:22 smithi138 ceph-mon[34885]: osdmap e8: 1 total, 0 up, 1 in 2024-05-27T09:21:22.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:22 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T09:21:22.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:22 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T09:21:22.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:22 smithi138 ceph-mon[34885]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T09:21:23.967 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:23 smithi119 ceph-mon[29082]: osd.0 [v2:172.21.15.119:6802/448778921,v1:172.21.15.119:6803/448778921] boot 2024-05-27T09:21:23.967 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:23 smithi119 ceph-mon[29082]: osdmap e9: 1 total, 1 up, 1 in 2024-05-27T09:21:23.968 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:23 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T09:21:24.051 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:23 smithi138 ceph-mon[34885]: osd.0 [v2:172.21.15.119:6802/448778921,v1:172.21.15.119:6803/448778921] boot 2024-05-27T09:21:24.051 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:23 smithi138 ceph-mon[34885]: osdmap e9: 1 total, 1 up, 1 in 2024-05-27T09:21:24.051 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:23 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T09:21:24.282 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-27T09:21:24.283 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-27T09:21:24.283 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2024-05-27T09:21:24.283 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2024-05-27T09:21:24.283 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0325014 s, 323 MB/s 2024-05-27T09:21:24.283 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2024-05-27T09:21:24.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:24 smithi119 ceph-mon[29082]: osdmap e10: 1 total, 1 up, 1 in 2024-05-27T09:21:24.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:24 smithi119 ceph-mon[29082]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:25.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:24 smithi138 ceph-mon[34885]: osdmap e10: 1 total, 1 up, 1 in 2024-05-27T09:21:25.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:24 smithi138 ceph-mon[34885]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:25.185 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi119:vg_nvme/lv_3 2024-05-27T09:21:25.883 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:21:27.174 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:26 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:27.174 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:26 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:27.174 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:26 smithi119 ceph-mon[29082]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:27.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:26 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:27.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:26 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:27.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:26 smithi138 ceph-mon[34885]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:27.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.890+0000 7f2340200700 1 -- 172.21.15.119:0/1936982359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2338100150 msgr2=0x7f2338100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:21:27.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.890+0000 7f2340200700 1 --2- 172.21.15.119:0/1936982359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2338100150 0x7f2338100560 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f232c009910 tx=0x7f232c009c20 comp rx=0 tx=0).stop 2024-05-27T09:21:27.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.890+0000 7f2340200700 1 -- 172.21.15.119:0/1936982359 shutdown_connections 2024-05-27T09:21:27.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.890+0000 7f2340200700 1 --2- 172.21.15.119:0/1936982359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2338101350 0x7f23381017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:21:27.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.890+0000 7f2340200700 1 --2- 172.21.15.119:0/1936982359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2338100150 0x7f2338100560 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:21:27.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.890+0000 7f2340200700 1 -- 172.21.15.119:0/1936982359 >> 172.21.15.119:0/1936982359 conn(0x7f23380fb6e0 msgr2=0x7f23380fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:21:27.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.890+0000 7f2340200700 1 -- 172.21.15.119:0/1936982359 shutdown_connections 2024-05-27T09:21:27.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.890+0000 7f2340200700 1 -- 172.21.15.119:0/1936982359 wait complete. 2024-05-27T09:21:27.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.891+0000 7f2340200700 1 Processor -- start 2024-05-27T09:21:27.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.891+0000 7f2340200700 1 -- start start 2024-05-27T09:21:27.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.892+0000 7f2340200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2338100150 0x7f23381959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:21:27.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.892+0000 7f2340200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2338101350 0x7f2338195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:21:27.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.892+0000 7f2340200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2338196500 con 0x7f2338101350 2024-05-27T09:21:27.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.892+0000 7f2340200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2338196640 con 0x7f2338100150 2024-05-27T09:21:27.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.892+0000 7f233ce00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2338101350 0x7f2338195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:21:27.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.892+0000 7f233ce00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2338101350 0x7f2338195ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53364/0 (socket says 172.21.15.119:53364) 2024-05-27T09:21:27.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.892+0000 7f233ce00700 1 -- 172.21.15.119:0/2626241137 learned_addr learned my addr 172.21.15.119:0/2626241137 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:21:27.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.892+0000 7f233d800700 1 --2- 172.21.15.119:0/2626241137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2338100150 0x7f23381959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:21:27.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.892+0000 7f233ce00700 1 -- 172.21.15.119:0/2626241137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2338100150 msgr2=0x7f23381959a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:21:27.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.892+0000 7f233ce00700 1 --2- 172.21.15.119:0/2626241137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2338100150 0x7f23381959a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:21:27.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.892+0000 7f233ce00700 1 -- 172.21.15.119:0/2626241137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232c0095f0 con 0x7f2338101350 2024-05-27T09:21:27.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.893+0000 7f233ce00700 1 --2- 172.21.15.119:0/2626241137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2338101350 0x7f2338195ee0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f232400d8d0 tx=0x7f232400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:21:27.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.893+0000 7f2336000700 1 -- 172.21.15.119:0/2626241137 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2324009940 con 0x7f2338101350 2024-05-27T09:21:27.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.893+0000 7f2336000700 1 -- 172.21.15.119:0/2626241137 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2324010460 con 0x7f2338101350 2024-05-27T09:21:27.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.893+0000 7f2336000700 1 -- 172.21.15.119:0/2626241137 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f232400f5f0 con 0x7f2338101350 2024-05-27T09:21:27.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.893+0000 7f2340200700 1 -- 172.21.15.119:0/2626241137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f233819b0f0 con 0x7f2338101350 2024-05-27T09:21:27.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.893+0000 7f2340200700 1 -- 172.21.15.119:0/2626241137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f233819b640 con 0x7f2338101350 2024-05-27T09:21:27.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.895+0000 7f2340200700 1 -- 172.21.15.119:0/2626241137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2338108ca0 con 0x7f2338101350 2024-05-27T09:21:27.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.896+0000 7f2336000700 1 -- 172.21.15.119:0/2626241137 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 89860+0+0 (secure 0 0 0) 0x7f23240105d0 con 0x7f2338101350 2024-05-27T09:21:27.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.897+0000 7f2336000700 1 --2- 172.21.15.119:0/2626241137 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f232006c610 0x7f232006eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:21:27.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.897+0000 7f2336000700 1 -- 172.21.15.119:0/2626241137 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f232408a9a0 con 0x7f2338101350 2024-05-27T09:21:27.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.897+0000 7f233d800700 1 --2- 172.21.15.119:0/2626241137 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f232006c610 0x7f232006eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:21:27.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.898+0000 7f233d800700 1 --2- 172.21.15.119:0/2626241137 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f232006c610 0x7f232006eac0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f232c006010 tx=0x7f232c00b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:21:27.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:27.902+0000 7f2336000700 1 -- 172.21.15.119:0/2626241137 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2324059970 con 0x7f2338101350 2024-05-27T09:21:28.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:28.139+0000 7f2340200700 1 -- 172.21.15.119:0/2626241137 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f2338105ca0 con 0x7f232006c610 2024-05-27T09:21:28.461 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:28 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T09:21:28.461 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:28 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T09:21:28.461 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:28 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:21:28.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:28 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T09:21:28.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:28 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T09:21:28.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:28 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:21:29.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:29 smithi138 ceph-mon[34885]: from='client.14272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:21:29.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:29 smithi138 ceph-mon[34885]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:29.429 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:29 smithi119 ceph-mon[29082]: from='client.14272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:21:29.429 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:29 smithi119 ceph-mon[29082]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:30.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:30 smithi119 ceph-mon[29082]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:31.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:30 smithi138 ceph-mon[34885]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:33.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:32 smithi138 ceph-mon[34885]: Detected new or changed devices on smithi119 2024-05-27T09:21:33.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:32 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:33.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:32 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:33.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:32 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:21:33.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:32 smithi138 ceph-mon[34885]: Adjusting osd_memory_target on smithi119 to 11841M 2024-05-27T09:21:33.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:32 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:33.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:32 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:21:33.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:32 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:21:33.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:32 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:33.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:32 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:21:33.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:32 smithi138 ceph-mon[34885]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:33.223 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:32 smithi119 ceph-mon[29082]: Detected new or changed devices on smithi119 2024-05-27T09:21:33.223 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:32 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:33.224 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:32 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:33.224 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:32 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:21:33.224 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:32 smithi119 ceph-mon[29082]: Adjusting osd_memory_target on smithi119 to 11841M 2024-05-27T09:21:33.224 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:32 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:33.224 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:32 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:21:33.224 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:32 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:21:33.224 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:32 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:33.224 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:32 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:21:33.224 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:32 smithi119 ceph-mon[29082]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:34.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:33 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/934496757' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "361a94d8-da7f-4672-9de0-7a2789453e64"}]: dispatch 2024-05-27T09:21:34.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:33 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/934496757' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "361a94d8-da7f-4672-9de0-7a2789453e64"}]': finished 2024-05-27T09:21:34.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:33 smithi119 ceph-mon[29082]: osdmap e11: 2 total, 1 up, 2 in 2024-05-27T09:21:34.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:33 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T09:21:34.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:33 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/934496757' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "361a94d8-da7f-4672-9de0-7a2789453e64"}]: dispatch 2024-05-27T09:21:34.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:33 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/934496757' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "361a94d8-da7f-4672-9de0-7a2789453e64"}]': finished 2024-05-27T09:21:34.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:33 smithi138 ceph-mon[34885]: osdmap e11: 2 total, 1 up, 2 in 2024-05-27T09:21:34.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:33 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T09:21:35.245 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:34 smithi119 ceph-mon[29082]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:35.246 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:34 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/191527321' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T09:21:35.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:34 smithi138 ceph-mon[34885]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:35.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:34 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/191527321' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T09:21:36.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:36 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:36.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:36 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:36.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:36 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:36.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:36 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:37.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:37 smithi119 ceph-mon[29082]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:37.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:37 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:21:37.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:37 smithi138 ceph-mon[34885]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:37.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:37 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:21:38.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:38 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:38.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:38 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:38.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:38 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:21:38.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:38 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:21:38.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:38 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:38.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:38 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:38.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:38 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:38.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:38 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:21:38.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:38 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:21:38.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:38 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:39.719 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:39 smithi119 ceph-mon[29082]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:39.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:39 smithi138 ceph-mon[34885]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:41.232 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:40 smithi119 ceph-mon[29082]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:41.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:40 smithi138 ceph-mon[34885]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:43.556 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:43 smithi119 ceph-mon[29082]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:43.556 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:43 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-27T09:21:43.556 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:43 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:21:43.556 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:43 smithi119 ceph-mon[29082]: Deploying daemon osd.1 on smithi119 2024-05-27T09:21:43.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:43 smithi138 ceph-mon[34885]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:43.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:43 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-27T09:21:43.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:43 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:21:43.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:43 smithi138 ceph-mon[34885]: Deploying daemon osd.1 on smithi119 2024-05-27T09:21:45.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:45 smithi119 ceph-mon[29082]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:45.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:45 smithi138 ceph-mon[34885]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:47.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:47 smithi119 ceph-mon[29082]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:47.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:47 smithi138 ceph-mon[34885]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:49.486 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:49 smithi119 ceph-mon[29082]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:49.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:49 smithi138 ceph-mon[34885]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:51.393 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:51 smithi119 ceph-mon[29082]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:51.393 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:51 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:51.394 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:51 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:51.394 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:51 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:21:51.394 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:51 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:21:51.574 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:51 smithi138 ceph-mon[34885]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:51.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:51 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:51.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:51 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:51.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:51 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:21:51.575 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:51 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:21:53.492 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:53 smithi119 ceph-mon[29082]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:53.492 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:53 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:53.492 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:53 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:53.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:53 smithi138 ceph-mon[34885]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:53.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:53 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:53.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:53 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:55.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:55.144+0000 7f2336000700 1 -- 172.21.15.119:0/2626241137 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2338105ca0 con 0x7f232006c610 2024-05-27T09:21:55.147 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 1 on host 'smithi119' 2024-05-27T09:21:55.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:55.148+0000 7f2340200700 1 -- 172.21.15.119:0/2626241137 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f232006c610 msgr2=0x7f232006eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:21:55.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:55.148+0000 7f2340200700 1 --2- 172.21.15.119:0/2626241137 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f232006c610 0x7f232006eac0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f232c006010 tx=0x7f232c00b540 comp rx=0 tx=0).stop 2024-05-27T09:21:55.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:55.148+0000 7f2340200700 1 -- 172.21.15.119:0/2626241137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2338101350 msgr2=0x7f2338195ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:21:55.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:55.148+0000 7f2340200700 1 --2- 172.21.15.119:0/2626241137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2338101350 0x7f2338195ee0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f232400d8d0 tx=0x7f232400dc90 comp rx=0 tx=0).stop 2024-05-27T09:21:55.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:55.149+0000 7f2340200700 1 -- 172.21.15.119:0/2626241137 shutdown_connections 2024-05-27T09:21:55.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:55.149+0000 7f2340200700 1 --2- 172.21.15.119:0/2626241137 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f232006c610 0x7f232006eac0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:21:55.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:55.149+0000 7f2340200700 1 --2- 172.21.15.119:0/2626241137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2338100150 0x7f23381959a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:21:55.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:55.149+0000 7f2340200700 1 --2- 172.21.15.119:0/2626241137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2338101350 0x7f2338195ee0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:21:55.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:55.149+0000 7f2340200700 1 -- 172.21.15.119:0/2626241137 >> 172.21.15.119:0/2626241137 conn(0x7f23380fb6e0 msgr2=0x7f2338104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:21:55.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:55.149+0000 7f2340200700 1 -- 172.21.15.119:0/2626241137 shutdown_connections 2024-05-27T09:21:55.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:21:55.149+0000 7f2340200700 1 -- 172.21.15.119:0/2626241137 wait complete. 2024-05-27T09:21:55.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:55 smithi138 ceph-mon[34885]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:55.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:55 smithi138 ceph-mon[34885]: from='osd.1 [v2:172.21.15.119:6810/2092077577,v1:172.21.15.119:6811/2092077577]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T09:21:55.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:55 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:55.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:55 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:55.636 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:55 smithi119 ceph-mon[29082]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:55.637 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:55 smithi119 ceph-mon[29082]: from='osd.1 [v2:172.21.15.119:6810/2092077577,v1:172.21.15.119:6811/2092077577]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T09:21:55.637 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:55 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:55.637 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:55 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:21:56.512 DEBUG:teuthology.orchestra.run.smithi119:osd.1> sudo journalctl -f -n 0 -u ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.1.service 2024-05-27T09:21:56.518 INFO:tasks.cephadm:Deploying osd.2 on smithi119 with /dev/vg_nvme/lv_2... 2024-05-27T09:21:56.518 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-27T09:21:56.621 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:56 smithi119 ceph-mon[29082]: from='osd.1 [v2:172.21.15.119:6810/2092077577,v1:172.21.15.119:6811/2092077577]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-27T09:21:56.621 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:56 smithi119 ceph-mon[29082]: osdmap e12: 2 total, 1 up, 2 in 2024-05-27T09:21:56.621 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:56 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T09:21:56.621 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:56 smithi119 ceph-mon[29082]: from='osd.1 [v2:172.21.15.119:6810/2092077577,v1:172.21.15.119:6811/2092077577]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-05-27T09:21:56.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:56 smithi138 ceph-mon[34885]: from='osd.1 [v2:172.21.15.119:6810/2092077577,v1:172.21.15.119:6811/2092077577]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-27T09:21:56.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:56 smithi138 ceph-mon[34885]: osdmap e12: 2 total, 1 up, 2 in 2024-05-27T09:21:56.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:56 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T09:21:56.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:56 smithi138 ceph-mon[34885]: from='osd.1 [v2:172.21.15.119:6810/2092077577,v1:172.21.15.119:6811/2092077577]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-05-27T09:21:57.069 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:21:57.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:57 smithi138 ceph-mon[34885]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:57.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:57 smithi138 ceph-mon[34885]: from='osd.1 [v2:172.21.15.119:6810/2092077577,v1:172.21.15.119:6811/2092077577]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-05-27T09:21:57.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:57 smithi138 ceph-mon[34885]: osdmap e13: 2 total, 1 up, 2 in 2024-05-27T09:21:57.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:57 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T09:21:57.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:57 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T09:21:57.710 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:57 smithi119 ceph-mon[29082]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:57.710 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:57 smithi119 ceph-mon[29082]: from='osd.1 [v2:172.21.15.119:6810/2092077577,v1:172.21.15.119:6811/2092077577]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-05-27T09:21:57.711 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:57 smithi119 ceph-mon[29082]: osdmap e13: 2 total, 1 up, 2 in 2024-05-27T09:21:57.711 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:57 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T09:21:57.711 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:57 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T09:21:58.625 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:58 smithi119 ceph-mon[29082]: purged_snaps scrub starts 2024-05-27T09:21:58.625 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:58 smithi119 ceph-mon[29082]: purged_snaps scrub ok 2024-05-27T09:21:58.625 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:58 smithi119 ceph-mon[29082]: osd.1 [v2:172.21.15.119:6810/2092077577,v1:172.21.15.119:6811/2092077577] boot 2024-05-27T09:21:58.625 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:58 smithi119 ceph-mon[29082]: osdmap e14: 2 total, 2 up, 2 in 2024-05-27T09:21:58.625 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:58 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T09:21:58.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:58 smithi138 ceph-mon[34885]: purged_snaps scrub starts 2024-05-27T09:21:58.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:58 smithi138 ceph-mon[34885]: purged_snaps scrub ok 2024-05-27T09:21:58.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:58 smithi138 ceph-mon[34885]: osd.1 [v2:172.21.15.119:6810/2092077577,v1:172.21.15.119:6811/2092077577] boot 2024-05-27T09:21:58.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:58 smithi138 ceph-mon[34885]: osdmap e14: 2 total, 2 up, 2 in 2024-05-27T09:21:58.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:58 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T09:21:59.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:59 smithi138 ceph-mon[34885]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:59.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:21:59 smithi138 ceph-mon[34885]: osdmap e15: 2 total, 2 up, 2 in 2024-05-27T09:21:59.640 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:59 smithi119 ceph-mon[29082]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T09:21:59.640 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:21:59 smithi119 ceph-mon[29082]: osdmap e15: 2 total, 2 up, 2 in 2024-05-27T09:22:00.538 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:00 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:00.538 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:00 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:00.636 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-27T09:22:00.636 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-27T09:22:00.636 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2024-05-27T09:22:00.636 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2024-05-27T09:22:00.636 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0303708 s, 345 MB/s 2024-05-27T09:22:00.636 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2024-05-27T09:22:00.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:00 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:00.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:00 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:01.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:01 smithi119 ceph-mon[29082]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:01.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:01 smithi138 ceph-mon[34885]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:01.891 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi119:vg_nvme/lv_2 2024-05-27T09:22:02.815 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:22:03.632 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:03 smithi119 ceph-mon[29082]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:03.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:03 smithi138 ceph-mon[34885]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:05.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:05 smithi138 ceph-mon[34885]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:05.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:05 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:05.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:05 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:05.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:05 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:22:05.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:05 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:22:05.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:05 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:05.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:05 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:22:05.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:05 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:22:05.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:05 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:05.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:05 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:22:05.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.707+0000 7f8f40e00700 1 -- 172.21.15.119:0/2765462096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f3c103580 msgr2=0x7f8f3c105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:22:05.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.707+0000 7f8f40e00700 1 --2- 172.21.15.119:0/2765462096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f3c103580 0x7f8f3c105960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009910 tx=0x7f8f28009c20 comp rx=0 tx=0).stop 2024-05-27T09:22:05.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.707+0000 7f8f40e00700 1 -- 172.21.15.119:0/2765462096 shutdown_connections 2024-05-27T09:22:05.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.708+0000 7f8f40e00700 1 --2- 172.21.15.119:0/2765462096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f3c103580 0x7f8f3c105960 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:22:05.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.708+0000 7f8f40e00700 1 --2- 172.21.15.119:0/2765462096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f3c100c60 0x7f8f3c103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:22:05.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.708+0000 7f8f40e00700 1 -- 172.21.15.119:0/2765462096 >> 172.21.15.119:0/2765462096 conn(0x7f8f3c0fa600 msgr2=0x7f8f3c0fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:22:05.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.708+0000 7f8f40e00700 1 -- 172.21.15.119:0/2765462096 shutdown_connections 2024-05-27T09:22:05.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.708+0000 7f8f40e00700 1 -- 172.21.15.119:0/2765462096 wait complete. 2024-05-27T09:22:05.710 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:05 smithi119 ceph-mon[29082]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:05.710 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:05 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:05.710 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:05 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:05.710 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:05 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:22:05.710 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:05 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:22:05.710 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:05 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:05.710 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:05 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:22:05.710 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:05 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:22:05.710 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:05 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:05.710 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:05 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:22:05.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.709+0000 7f8f40e00700 1 Processor -- start 2024-05-27T09:22:05.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.709+0000 7f8f40e00700 1 -- start start 2024-05-27T09:22:05.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.710+0000 7f8f40e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f3c100c60 0x7f8f3c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:22:05.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.710+0000 7f8f40e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f3c103580 0x7f8f3c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:22:05.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.710+0000 7f8f40e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c198680 con 0x7f8f3c100c60 2024-05-27T09:22:05.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.710+0000 7f8f40e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c1987c0 con 0x7f8f3c103580 2024-05-27T09:22:05.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.710+0000 7f8f39e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f3c100c60 0x7f8f3c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:22:05.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.710+0000 7f8f39e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f3c100c60 0x7f8f3c197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49310/0 (socket says 172.21.15.119:49310) 2024-05-27T09:22:05.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.710+0000 7f8f39e00700 1 -- 172.21.15.119:0/1137546379 learned_addr learned my addr 172.21.15.119:0/1137546379 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:22:05.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.710+0000 7f8f39400700 1 --2- 172.21.15.119:0/1137546379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f3c103580 0x7f8f3c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:22:05.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.711+0000 7f8f39e00700 1 -- 172.21.15.119:0/1137546379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f3c103580 msgr2=0x7f8f3c198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:22:05.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.711+0000 7f8f39e00700 1 --2- 172.21.15.119:0/1137546379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f3c103580 0x7f8f3c198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:22:05.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.711+0000 7f8f39e00700 1 -- 172.21.15.119:0/1137546379 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f280095f0 con 0x7f8f3c100c60 2024-05-27T09:22:05.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.711+0000 7f8f39400700 1 --2- 172.21.15.119:0/1137546379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f3c103580 0x7f8f3c198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:22:05.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.711+0000 7f8f39e00700 1 --2- 172.21.15.119:0/1137546379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f3c100c60 0x7f8f3c197b20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00d900 tx=0x7f8f2c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:22:05.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.711+0000 7f8f32a00700 1 -- 172.21.15.119:0/1137546379 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f2c0041f0 con 0x7f8f3c100c60 2024-05-27T09:22:05.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.711+0000 7f8f32a00700 1 -- 172.21.15.119:0/1137546379 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8f2c004ce0 con 0x7f8f3c100c60 2024-05-27T09:22:05.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.711+0000 7f8f32a00700 1 -- 172.21.15.119:0/1137546379 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f2c00b750 con 0x7f8f3c100c60 2024-05-27T09:22:05.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.711+0000 7f8f40e00700 1 -- 172.21.15.119:0/1137546379 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f3c19d270 con 0x7f8f3c100c60 2024-05-27T09:22:05.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.712+0000 7f8f40e00700 1 -- 172.21.15.119:0/1137546379 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3c19d770 con 0x7f8f3c100c60 2024-05-27T09:22:05.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.713+0000 7f8f40e00700 1 -- 172.21.15.119:0/1137546379 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f3c0fc1e0 con 0x7f8f3c100c60 2024-05-27T09:22:05.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.715+0000 7f8f32a00700 1 -- 172.21.15.119:0/1137546379 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 89860+0+0 (secure 0 0 0) 0x7f8f2c004350 con 0x7f8f3c100c60 2024-05-27T09:22:05.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.715+0000 7f8f32a00700 1 --2- 172.21.15.119:0/1137546379 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f8f2006c610 0x7f8f2006eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:22:05.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.716+0000 7f8f32a00700 1 -- 172.21.15.119:0/1137546379 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7f8f2c08abe0 con 0x7f8f3c100c60 2024-05-27T09:22:05.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.716+0000 7f8f39400700 1 --2- 172.21.15.119:0/1137546379 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f8f2006c610 0x7f8f2006eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:22:05.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.716+0000 7f8f39400700 1 --2- 172.21.15.119:0/1137546379 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f8f2006c610 0x7f8f2006eac0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8f28006010 tx=0x7f8f2800b560 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:22:05.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.720+0000 7f8f32a00700 1 -- 172.21.15.119:0/1137546379 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8f2c059980 con 0x7f8f3c100c60 2024-05-27T09:22:05.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:05.950+0000 7f8f40e00700 1 -- 172.21.15.119:0/1137546379 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f3c061190 con 0x7f8f2006c610 2024-05-27T09:22:06.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:06 smithi119 ceph-mon[29082]: Detected new or changed devices on smithi119 2024-05-27T09:22:06.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:06 smithi119 ceph-mon[29082]: Adjusting osd_memory_target on smithi119 to 5920M 2024-05-27T09:22:06.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:06 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T09:22:06.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:06 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T09:22:06.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:06 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:22:06.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:06 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:22:06.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:06 smithi138 ceph-mon[34885]: Detected new or changed devices on smithi119 2024-05-27T09:22:06.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:06 smithi138 ceph-mon[34885]: Adjusting osd_memory_target on smithi119 to 5920M 2024-05-27T09:22:06.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:06 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T09:22:06.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:06 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T09:22:06.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:06 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:22:06.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:06 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:22:07.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:07 smithi119 ceph-mon[29082]: from='client.14290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:22:07.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:07 smithi119 ceph-mon[29082]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:07.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:07 smithi138 ceph-mon[34885]: from='client.14290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:22:07.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:07 smithi138 ceph-mon[34885]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:09.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:09 smithi119 ceph-mon[29082]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:09.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:09 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:09.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:09 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:09.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:09 smithi138 ceph-mon[34885]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:09.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:09 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:09.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:09 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:10.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:10 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/3199232895' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c405f37-b811-464f-a5f7-29e9e2a9c6e8"}]: dispatch 2024-05-27T09:22:10.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:10 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/3199232895' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c405f37-b811-464f-a5f7-29e9e2a9c6e8"}]': finished 2024-05-27T09:22:10.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:10 smithi119 ceph-mon[29082]: osdmap e16: 3 total, 2 up, 3 in 2024-05-27T09:22:10.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:10 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T09:22:10.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:10 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/3199232895' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4c405f37-b811-464f-a5f7-29e9e2a9c6e8"}]: dispatch 2024-05-27T09:22:10.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:10 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/3199232895' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4c405f37-b811-464f-a5f7-29e9e2a9c6e8"}]': finished 2024-05-27T09:22:10.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:10 smithi138 ceph-mon[34885]: osdmap e16: 3 total, 2 up, 3 in 2024-05-27T09:22:10.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:10 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T09:22:11.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:11 smithi119 ceph-mon[29082]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:11.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:11 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/2459580014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T09:22:11.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:11 smithi138 ceph-mon[34885]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:11.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:11 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/2459580014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T09:22:13.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:13 smithi119 ceph-mon[29082]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:13.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:13 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:13.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:13 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:13.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:13 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:22:13.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:13 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:22:13.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:13 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:13.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:13 smithi138 ceph-mon[34885]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:13.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:13 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:13.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:13 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:13.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:13 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:22:13.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:13 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:22:13.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:13 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:15.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:15 smithi119 ceph-mon[29082]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:15.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:15 smithi138 ceph-mon[34885]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:17.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:17 smithi119 ceph-mon[29082]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:17.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:17 smithi138 ceph-mon[34885]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:19.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:19 smithi138 ceph-mon[34885]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:19.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:19 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-27T09:22:19.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:19 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:22:19.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:19 smithi119 ceph-mon[29082]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:19.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:19 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-27T09:22:19.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:19 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:22:20.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:20 smithi119 ceph-mon[29082]: Deploying daemon osd.2 on smithi119 2024-05-27T09:22:20.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:20 smithi138 ceph-mon[34885]: Deploying daemon osd.2 on smithi119 2024-05-27T09:22:21.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:21 smithi119 ceph-mon[29082]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:21.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:21 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:22:21.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:21 smithi138 ceph-mon[34885]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:21.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:21 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:22:23.711 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:23 smithi138 ceph-mon[34885]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:23.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:23 smithi119 ceph-mon[29082]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:25.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:25 smithi138 ceph-mon[34885]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:25.740 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:25 smithi119 ceph-mon[29082]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:27.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:27 smithi119 ceph-mon[29082]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:27.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:27 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:27.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:27 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:22:27.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:27 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:27.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:27 smithi138 ceph-mon[34885]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:27.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:27 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:27.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:27 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:22:27.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:27 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:29.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:29 smithi138 ceph-mon[34885]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:29.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:29 smithi119 ceph-mon[29082]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:31.049 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:30 smithi119 ceph-mon[29082]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:31.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:30 smithi138 ceph-mon[34885]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:31.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:31 smithi119 ceph-mon[29082]: from='osd.2 [v2:172.21.15.119:6818/3743656430,v1:172.21.15.119:6819/3743656430]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T09:22:32.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:32.182+0000 7f8f32a00700 1 -- 172.21.15.119:0/1137546379 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8f3c061190 con 0x7f8f2006c610 2024-05-27T09:22:32.185 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 2 on host 'smithi119' 2024-05-27T09:22:32.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:32.185+0000 7f8f40e00700 1 -- 172.21.15.119:0/1137546379 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f8f2006c610 msgr2=0x7f8f2006eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:22:32.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:32.185+0000 7f8f40e00700 1 --2- 172.21.15.119:0/1137546379 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f8f2006c610 0x7f8f2006eac0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8f28006010 tx=0x7f8f2800b560 comp rx=0 tx=0).stop 2024-05-27T09:22:32.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:32.185+0000 7f8f40e00700 1 -- 172.21.15.119:0/1137546379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f3c100c60 msgr2=0x7f8f3c197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:22:32.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:32.185+0000 7f8f40e00700 1 --2- 172.21.15.119:0/1137546379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f3c100c60 0x7f8f3c197b20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00d900 tx=0x7f8f2c00dcc0 comp rx=0 tx=0).stop 2024-05-27T09:22:32.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:32.185+0000 7f8f40e00700 1 -- 172.21.15.119:0/1137546379 shutdown_connections 2024-05-27T09:22:32.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:32.185+0000 7f8f40e00700 1 --2- 172.21.15.119:0/1137546379 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f8f2006c610 0x7f8f2006eac0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:22:32.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:32.185+0000 7f8f40e00700 1 --2- 172.21.15.119:0/1137546379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f3c100c60 0x7f8f3c197b20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:22:32.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:32.185+0000 7f8f40e00700 1 --2- 172.21.15.119:0/1137546379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f3c103580 0x7f8f3c198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:22:32.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:32.185+0000 7f8f40e00700 1 -- 172.21.15.119:0/1137546379 >> 172.21.15.119:0/1137546379 conn(0x7f8f3c0fa600 msgr2=0x7f8f3c0ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:22:32.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:32.186+0000 7f8f40e00700 1 -- 172.21.15.119:0/1137546379 shutdown_connections 2024-05-27T09:22:32.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:22:32.186+0000 7f8f40e00700 1 -- 172.21.15.119:0/1137546379 wait complete. 2024-05-27T09:22:32.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:31 smithi138 ceph-mon[34885]: from='osd.2 [v2:172.21.15.119:6818/3743656430,v1:172.21.15.119:6819/3743656430]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T09:22:33.177 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:32 smithi119 ceph-mon[29082]: from='osd.2 [v2:172.21.15.119:6818/3743656430,v1:172.21.15.119:6819/3743656430]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-27T09:22:33.177 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:32 smithi119 ceph-mon[29082]: osdmap e17: 3 total, 2 up, 3 in 2024-05-27T09:22:33.177 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:32 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T09:22:33.177 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:32 smithi119 ceph-mon[29082]: from='osd.2 [v2:172.21.15.119:6818/3743656430,v1:172.21.15.119:6819/3743656430]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-05-27T09:22:33.177 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:32 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:33.177 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:32 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:33.177 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:32 smithi119 ceph-mon[29082]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:33.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:32 smithi138 ceph-mon[34885]: from='osd.2 [v2:172.21.15.119:6818/3743656430,v1:172.21.15.119:6819/3743656430]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-27T09:22:33.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:32 smithi138 ceph-mon[34885]: osdmap e17: 3 total, 2 up, 3 in 2024-05-27T09:22:33.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:32 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T09:22:33.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:32 smithi138 ceph-mon[34885]: from='osd.2 [v2:172.21.15.119:6818/3743656430,v1:172.21.15.119:6819/3743656430]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-05-27T09:22:33.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:32 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:33.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:32 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:33.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:32 smithi138 ceph-mon[34885]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-27T09:22:33.654 DEBUG:teuthology.orchestra.run.smithi119:osd.2> sudo journalctl -f -n 0 -u ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.2.service 2024-05-27T09:22:33.662 INFO:tasks.cephadm:Deploying osd.3 on smithi138 with /dev/vg_nvme/lv_4... 2024-05-27T09:22:33.662 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-27T09:22:34.065 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi138/config 2024-05-27T09:22:34.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:33 smithi138 ceph-mon[34885]: from='osd.2 [v2:172.21.15.119:6818/3743656430,v1:172.21.15.119:6819/3743656430]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-05-27T09:22:34.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:33 smithi138 ceph-mon[34885]: osdmap e18: 3 total, 2 up, 3 in 2024-05-27T09:22:34.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:33 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T09:22:34.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:33 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T09:22:34.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:33 smithi119 ceph-mon[29082]: from='osd.2 [v2:172.21.15.119:6818/3743656430,v1:172.21.15.119:6819/3743656430]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-05-27T09:22:34.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:33 smithi119 ceph-mon[29082]: osdmap e18: 3 total, 2 up, 3 in 2024-05-27T09:22:34.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:33 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T09:22:34.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:33 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T09:22:35.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:34 smithi119 ceph-mon[29082]: purged_snaps scrub starts 2024-05-27T09:22:35.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:34 smithi119 ceph-mon[29082]: purged_snaps scrub ok 2024-05-27T09:22:35.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:34 smithi119 ceph-mon[29082]: osd.2 [v2:172.21.15.119:6818/3743656430,v1:172.21.15.119:6819/3743656430] boot 2024-05-27T09:22:35.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:34 smithi119 ceph-mon[29082]: osdmap e19: 3 total, 3 up, 3 in 2024-05-27T09:22:35.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:34 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T09:22:35.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:34 smithi119 ceph-mon[29082]: pgmap v96: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:22:35.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:34 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-27T09:22:35.311 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:34 smithi138 ceph-mon[34885]: purged_snaps scrub starts 2024-05-27T09:22:35.312 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:34 smithi138 ceph-mon[34885]: purged_snaps scrub ok 2024-05-27T09:22:35.312 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:34 smithi138 ceph-mon[34885]: osd.2 [v2:172.21.15.119:6818/3743656430,v1:172.21.15.119:6819/3743656430] boot 2024-05-27T09:22:35.312 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:34 smithi138 ceph-mon[34885]: osdmap e19: 3 total, 3 up, 3 in 2024-05-27T09:22:35.312 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:34 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T09:22:35.312 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:34 smithi138 ceph-mon[34885]: pgmap v96: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:22:35.312 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:34 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-27T09:22:36.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:35 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-27T09:22:36.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:35 smithi138 ceph-mon[34885]: osdmap e20: 3 total, 3 up, 3 in 2024-05-27T09:22:36.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:35 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-27T09:22:36.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:35 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:36.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:35 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:36.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:35 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-27T09:22:36.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:35 smithi119 ceph-mon[29082]: osdmap e20: 3 total, 3 up, 3 in 2024-05-27T09:22:36.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:35 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-27T09:22:36.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:35 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:36.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:35 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:36.912 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-27T09:22:36.913 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-27T09:22:36.913 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-05-27T09:22:36.913 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-05-27T09:22:36.913 INFO:teuthology.orchestra.run.smithi138.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0349769 s, 300 MB/s 2024-05-27T09:22:36.913 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-05-27T09:22:37.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:37 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-27T09:22:37.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:37 smithi119 ceph-mon[29082]: osdmap e21: 3 total, 3 up, 3 in 2024-05-27T09:22:37.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:37 smithi119 ceph-mon[29082]: pgmap v99: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:22:37.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:37 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:22:37.248 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:22:36 smithi119 sudo[56688]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T09:22:37.248 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:22:36 smithi119 sudo[56688]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T09:22:37.248 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:22:36 smithi119 sudo[56688]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-05-27T09:22:37.248 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:22:36 smithi119 sudo[56688]: pam_unix(sudo:session): session closed for user root 2024-05-27T09:22:37.346 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:37 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-27T09:22:37.347 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:37 smithi138 ceph-mon[34885]: osdmap e21: 3 total, 3 up, 3 in 2024-05-27T09:22:37.347 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:37 smithi138 ceph-mon[34885]: pgmap v99: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:22:37.347 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:37 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:22:37.747 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:22:37 smithi119 sudo[56790]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T09:22:37.747 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:22:37 smithi119 sudo[56790]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T09:22:37.747 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:22:37 smithi119 sudo[56790]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-05-27T09:22:37.748 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:22:37 smithi119 sudo[56790]: pam_unix(sudo:session): session closed for user root 2024-05-27T09:22:37.824 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_4 2024-05-27T09:22:38.210 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi138/config 2024-05-27T09:22:38.353 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:38 smithi138 ceph-mon[34885]: osdmap e22: 3 total, 3 up, 3 in 2024-05-27T09:22:38.354 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:38 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:38.363 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:38 smithi119 ceph-mon[29082]: osdmap e22: 3 total, 3 up, 3 in 2024-05-27T09:22:38.363 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:38 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:38.363 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:22:38 smithi119 sudo[56824]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T09:22:38.363 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:22:38 smithi119 sudo[56824]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T09:22:38.363 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:22:38 smithi119 sudo[56824]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-05-27T09:22:38.364 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:22:38 smithi119 sudo[56824]: pam_unix(sudo:session): session closed for user root 2024-05-27T09:22:38.998 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:22:38 smithi119 sudo[56831]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T09:22:38.998 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:22:38 smithi119 sudo[56831]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T09:22:38.998 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:22:38 smithi119 sudo[56831]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-05-27T09:22:38.998 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:22:38 smithi119 sudo[56831]: pam_unix(sudo:session): session closed for user root 2024-05-27T09:22:39.310 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:39 smithi119 ceph-mon[29082]: pgmap v101: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:22:39.311 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:22:39 smithi119 sudo[56856]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T09:22:39.311 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:22:39 smithi119 sudo[56856]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T09:22:39.311 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:22:39 smithi119 sudo[56856]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-05-27T09:22:39.311 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:22:39 smithi119 sudo[56856]: pam_unix(sudo:session): session closed for user root 2024-05-27T09:22:39.378 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:39 smithi138 ceph-mon[34885]: pgmap v101: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:22:40.241 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:22:39 smithi119 sudo[56872]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T09:22:40.241 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:22:39 smithi119 sudo[56872]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T09:22:40.241 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:22:39 smithi119 sudo[56872]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-05-27T09:22:40.241 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:22:39 smithi119 sudo[56872]: pam_unix(sudo:session): session closed for user root 2024-05-27T09:22:40.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.445+0000 7ff752400700 1 -- 172.21.15.138:0/3688555850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff74c1079a0 msgr2=0x7ff74c107df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:22:40.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.445+0000 7ff752400700 1 --2- 172.21.15.138:0/3688555850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff74c1079a0 0x7ff74c107df0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff738009910 tx=0x7ff738009c20 comp rx=0 tx=0).stop 2024-05-27T09:22:40.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.445+0000 7ff752400700 1 -- 172.21.15.138:0/3688555850 shutdown_connections 2024-05-27T09:22:40.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.445+0000 7ff752400700 1 --2- 172.21.15.138:0/3688555850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff74c1079a0 0x7ff74c107df0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:22:40.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.445+0000 7ff752400700 1 --2- 172.21.15.138:0/3688555850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff74c1067a0 0x7ff74c106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:22:40.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.445+0000 7ff752400700 1 -- 172.21.15.138:0/3688555850 >> 172.21.15.138:0/3688555850 conn(0x7ff74c075910 msgr2=0x7ff74c077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:22:40.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.445+0000 7ff752400700 1 -- 172.21.15.138:0/3688555850 shutdown_connections 2024-05-27T09:22:40.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.445+0000 7ff752400700 1 -- 172.21.15.138:0/3688555850 wait complete. 2024-05-27T09:22:40.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.447+0000 7ff752400700 1 Processor -- start 2024-05-27T09:22:40.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.447+0000 7ff752400700 1 -- start start 2024-05-27T09:22:40.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.447+0000 7ff752400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff74c1067a0 0x7ff74c19bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:22:40.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.447+0000 7ff752400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff74c1079a0 0x7ff74c19c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:22:40.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.447+0000 7ff752400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c19cb30 con 0x7ff74c1079a0 2024-05-27T09:22:40.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.447+0000 7ff752400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c19cc70 con 0x7ff74c1067a0 2024-05-27T09:22:40.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.447+0000 7ff74b400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff74c1067a0 0x7ff74c19bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:22:40.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.447+0000 7ff74b400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff74c1067a0 0x7ff74c19bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48454/0 (socket says 172.21.15.138:48454) 2024-05-27T09:22:40.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.448+0000 7ff74b400700 1 -- 172.21.15.138:0/4166102653 learned_addr learned my addr 172.21.15.138:0/4166102653 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:22:40.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.448+0000 7ff74aa00700 1 --2- 172.21.15.138:0/4166102653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff74c1079a0 0x7ff74c19c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:22:40.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.448+0000 7ff74b400700 1 -- 172.21.15.138:0/4166102653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff74c1079a0 msgr2=0x7ff74c19c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:22:40.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.448+0000 7ff74b400700 1 --2- 172.21.15.138:0/4166102653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff74c1079a0 0x7ff74c19c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:22:40.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.448+0000 7ff74b400700 1 -- 172.21.15.138:0/4166102653 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7380095f0 con 0x7ff74c1067a0 2024-05-27T09:22:40.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.448+0000 7ff74b400700 1 --2- 172.21.15.138:0/4166102653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff74c1067a0 0x7ff74c19bfd0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00b480 tx=0x7ff73c00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:22:40.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.448+0000 7ff743e00700 1 -- 172.21.15.138:0/4166102653 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff73c0118c0 con 0x7ff74c1067a0 2024-05-27T09:22:40.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.449+0000 7ff743e00700 1 -- 172.21.15.138:0/4166102653 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff73c010380 con 0x7ff74c1067a0 2024-05-27T09:22:40.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.449+0000 7ff752400700 1 -- 172.21.15.138:0/4166102653 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff74c1a1720 con 0x7ff74c1067a0 2024-05-27T09:22:40.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.449+0000 7ff743e00700 1 -- 172.21.15.138:0/4166102653 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff73c00f4f0 con 0x7ff74c1067a0 2024-05-27T09:22:40.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.449+0000 7ff752400700 1 -- 172.21.15.138:0/4166102653 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74c1a1cf0 con 0x7ff74c1067a0 2024-05-27T09:22:40.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.450+0000 7ff752400700 1 -- 172.21.15.138:0/4166102653 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff74c10f390 con 0x7ff74c1067a0 2024-05-27T09:22:40.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.451+0000 7ff743e00700 1 -- 172.21.15.138:0/4166102653 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7ff73c0104f0 con 0x7ff74c1067a0 2024-05-27T09:22:40.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.452+0000 7ff743e00700 1 --2- 172.21.15.138:0/4166102653 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff73006c680 0x7ff73006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:22:40.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.452+0000 7ff743e00700 1 -- 172.21.15.138:0/4166102653 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 3232+0+0 (secure 0 0 0) 0x7ff73c08ac10 con 0x7ff74c1067a0 2024-05-27T09:22:40.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.453+0000 7ff74aa00700 1 --2- 172.21.15.138:0/4166102653 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff73006c680 0x7ff73006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:22:40.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.453+0000 7ff74aa00700 1 --2- 172.21.15.138:0/4166102653 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff73006c680 0x7ff73006eb30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff738000c00 tx=0x7ff738005fb0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:22:40.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.457+0000 7ff743e00700 1 -- 172.21.15.138:0/4166102653 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff73c059640 con 0x7ff74c1067a0 2024-05-27T09:22:40.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:40 smithi119 sudo[57275]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T09:22:40.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:40 smithi119 sudo[57275]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T09:22:40.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:40 smithi119 sudo[57275]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-05-27T09:22:40.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:40 smithi119 sudo[57275]: pam_unix(sudo:session): session closed for user root 2024-05-27T09:22:40.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:22:40.680+0000 7ff752400700 1 -- 172.21.15.138:0/4166102653 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff74c10c2f0 con 0x7ff73006c680 2024-05-27T09:22:40.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:40 smithi119 sudo[57279]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T09:22:40.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:40 smithi119 sudo[57279]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T09:22:40.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:40 smithi119 sudo[57279]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-05-27T09:22:40.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:40 smithi119 sudo[57279]: pam_unix(sudo:session): session closed for user root 2024-05-27T09:22:41.810 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:41 smithi138 sudo[38669]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T09:22:41.810 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:41 smithi138 sudo[38669]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T09:22:41.810 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:41 smithi138 sudo[38669]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-05-27T09:22:41.811 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:41 smithi138 sudo[38669]: pam_unix(sudo:session): session closed for user root 2024-05-27T09:22:42.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:42 smithi138 sudo[38725]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T09:22:42.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:42 smithi138 sudo[38725]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T09:22:42.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:42 smithi138 sudo[38725]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-05-27T09:22:42.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:42 smithi138 sudo[38725]: pam_unix(sudo:session): session closed for user root 2024-05-27T09:22:42.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:42 smithi138 ceph-mon[34885]: mgrmap e19: smithi119.ncwmog(active, since 3m), standbys: smithi138.dxsuqf 2024-05-27T09:22:42.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:42 smithi138 ceph-mon[34885]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T09:22:42.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:42 smithi119 ceph-mon[29082]: mgrmap e19: smithi119.ncwmog(active, since 3m), standbys: smithi138.dxsuqf 2024-05-27T09:22:42.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:42 smithi119 ceph-mon[29082]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T09:22:43.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:22:43.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:22:43.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: Detected new or changed devices on smithi119 2024-05-27T09:22:43.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T09:22:43.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi119"}]: dispatch 2024-05-27T09:22:43.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:22:43.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T09:22:43.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T09:22:43.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi119"}]: dispatch 2024-05-27T09:22:43.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:22:43.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T09:22:43.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:22:43.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T09:22:43.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:43.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:43.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:22:43.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:22:43.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:22:43.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: Adjusting osd_memory_target on smithi119 to 3947M 2024-05-27T09:22:43.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:43.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:22:43.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:43.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:22:43.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:43.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:22:43.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:43 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:22:43.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:22:43.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:22:43.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: Detected new or changed devices on smithi119 2024-05-27T09:22:43.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T09:22:43.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi119"}]: dispatch 2024-05-27T09:22:43.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:22:43.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T09:22:43.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T09:22:43.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi119"}]: dispatch 2024-05-27T09:22:43.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:22:43.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T09:22:43.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:22:43.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T09:22:43.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:43.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:43.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:22:43.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:22:43.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:22:43.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: Adjusting osd_memory_target on smithi119 to 3947M 2024-05-27T09:22:43.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:43.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:22:43.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:43.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:22:43.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:43.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:22:43.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:43 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:22:45.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:45 smithi138 ceph-mon[34885]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-05-27T09:22:45.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:45 smithi138 ceph-mon[34885]: from='client.? 172.21.15.138:0/1627675914' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a332cce6-4548-4c4d-97c1-705e7ffb2790"}]: dispatch 2024-05-27T09:22:45.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:45 smithi138 ceph-mon[34885]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a332cce6-4548-4c4d-97c1-705e7ffb2790"}]: dispatch 2024-05-27T09:22:45.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:45 smithi138 ceph-mon[34885]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a332cce6-4548-4c4d-97c1-705e7ffb2790"}]': finished 2024-05-27T09:22:45.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:45 smithi138 ceph-mon[34885]: osdmap e23: 4 total, 3 up, 4 in 2024-05-27T09:22:45.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:45 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T09:22:45.603 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:45 smithi119 ceph-mon[29082]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-05-27T09:22:45.604 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:45 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/1627675914' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a332cce6-4548-4c4d-97c1-705e7ffb2790"}]: dispatch 2024-05-27T09:22:45.604 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:45 smithi119 ceph-mon[29082]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a332cce6-4548-4c4d-97c1-705e7ffb2790"}]: dispatch 2024-05-27T09:22:45.604 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:45 smithi119 ceph-mon[29082]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a332cce6-4548-4c4d-97c1-705e7ffb2790"}]': finished 2024-05-27T09:22:45.604 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:45 smithi119 ceph-mon[29082]: osdmap e23: 4 total, 3 up, 4 in 2024-05-27T09:22:45.604 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:45 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T09:22:46.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:46 smithi138 ceph-mon[34885]: from='client.? 172.21.15.138:0/1386843904' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T09:22:46.676 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:46 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/1386843904' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T09:22:47.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:47 smithi138 ceph-mon[34885]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-05-27T09:22:47.535 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:47 smithi119 ceph-mon[29082]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-05-27T09:22:49.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:49 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:49.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:49 smithi119 ceph-mon[29082]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-05-27T09:22:49.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:49 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:49.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:49 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:22:49.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:49 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:22:49.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:49 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:49.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:49 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:49.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:49 smithi138 ceph-mon[34885]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-05-27T09:22:49.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:49 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:49.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:49 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:22:49.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:49 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:22:49.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:49 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:22:51.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:51 smithi138 ceph-mon[34885]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-05-27T09:22:51.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:51 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:22:51.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:51 smithi119 ceph-mon[29082]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-05-27T09:22:51.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:51 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:22:53.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:53 smithi138 ceph-mon[34885]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-05-27T09:22:53.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:53 smithi119 ceph-mon[29082]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-05-27T09:22:54.632 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:54 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-27T09:22:54.632 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:54 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:22:54.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:54 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-27T09:22:54.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:54 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:22:55.549 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:55 smithi138 ceph-mon[34885]: Deploying daemon osd.3 on smithi138 2024-05-27T09:22:55.549 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:55 smithi138 ceph-mon[34885]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:22:55.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:55 smithi119 ceph-mon[29082]: Deploying daemon osd.3 on smithi138 2024-05-27T09:22:55.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:55 smithi119 ceph-mon[29082]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:22:57.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:56 smithi119 ceph-mon[29082]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:22:57.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:56 smithi138 ceph-mon[34885]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:22:59.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:22:59 smithi138 ceph-mon[34885]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:22:59.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:22:59 smithi119 ceph-mon[29082]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:23:01.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:01 smithi138 ceph-mon[34885]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:23:01.537 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:01 smithi119 ceph-mon[29082]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:23:02.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:02 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:02.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:02 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:02.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:02 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:23:02.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:02 smithi138 ceph-mon[34885]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:23:03.106 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:02 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:03.106 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:02 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:03.106 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:02 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:23:03.106 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:02 smithi119 ceph-mon[29082]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:23:05.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:05 smithi119 ceph-mon[29082]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:23:05.516 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:05 smithi138 ceph-mon[34885]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:23:06.590 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:06 smithi138 ceph-mon[34885]: from='osd.3 [v2:172.21.15.138:6800/29238828,v1:172.21.15.138:6801/29238828]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T09:23:06.591 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:06 smithi138 ceph-mon[34885]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T09:23:06.591 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:06 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:23:06.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:06 smithi119 ceph-mon[29082]: from='osd.3 [v2:172.21.15.138:6800/29238828,v1:172.21.15.138:6801/29238828]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T09:23:06.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:06 smithi119 ceph-mon[29082]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T09:23:06.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:06 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:23:07.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:07.057+0000 7ff743e00700 1 -- 172.21.15.138:0/4166102653 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff74c10c2f0 con 0x7ff73006c680 2024-05-27T09:23:07.059 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 3 on host 'smithi138' 2024-05-27T09:23:07.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:07.061+0000 7ff752400700 1 -- 172.21.15.138:0/4166102653 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff73006c680 msgr2=0x7ff73006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:23:07.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:07.061+0000 7ff752400700 1 --2- 172.21.15.138:0/4166102653 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff73006c680 0x7ff73006eb30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff738000c00 tx=0x7ff738005fb0 comp rx=0 tx=0).stop 2024-05-27T09:23:07.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:07.061+0000 7ff752400700 1 -- 172.21.15.138:0/4166102653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff74c1067a0 msgr2=0x7ff74c19bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:23:07.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:07.061+0000 7ff752400700 1 --2- 172.21.15.138:0/4166102653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff74c1067a0 0x7ff74c19bfd0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00b480 tx=0x7ff73c00b840 comp rx=0 tx=0).stop 2024-05-27T09:23:07.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:07.061+0000 7ff752400700 1 -- 172.21.15.138:0/4166102653 shutdown_connections 2024-05-27T09:23:07.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:07.061+0000 7ff752400700 1 --2- 172.21.15.138:0/4166102653 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff73006c680 0x7ff73006eb30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:23:07.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:07.061+0000 7ff752400700 1 --2- 172.21.15.138:0/4166102653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff74c1067a0 0x7ff74c19bfd0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:23:07.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:07.061+0000 7ff752400700 1 --2- 172.21.15.138:0/4166102653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff74c1079a0 0x7ff74c19c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:23:07.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:07.061+0000 7ff752400700 1 -- 172.21.15.138:0/4166102653 >> 172.21.15.138:0/4166102653 conn(0x7ff74c075910 msgr2=0x7ff74c10abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:23:07.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:07.061+0000 7ff752400700 1 -- 172.21.15.138:0/4166102653 shutdown_connections 2024-05-27T09:23:07.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:07.061+0000 7ff752400700 1 -- 172.21.15.138:0/4166102653 wait complete. 2024-05-27T09:23:07.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:07 smithi138 ceph-mon[34885]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:23:07.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:07 smithi138 ceph-mon[34885]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-27T09:23:07.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:07 smithi138 ceph-mon[34885]: osdmap e24: 4 total, 3 up, 4 in 2024-05-27T09:23:07.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:07 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T09:23:07.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:07 smithi138 ceph-mon[34885]: from='osd.3 [v2:172.21.15.138:6800/29238828,v1:172.21.15.138:6801/29238828]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:23:07.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:07 smithi138 ceph-mon[34885]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:23:07.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:07 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:07.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:07 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:07.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:07 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:23:07.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:07 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:23:07.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:07 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:07.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:07 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:07.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:07 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:07.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:07 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:23:07.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:07 smithi119 ceph-mon[29082]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:23:07.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:07 smithi119 ceph-mon[29082]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-27T09:23:07.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:07 smithi119 ceph-mon[29082]: osdmap e24: 4 total, 3 up, 4 in 2024-05-27T09:23:07.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:07 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T09:23:07.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:07 smithi119 ceph-mon[29082]: from='osd.3 [v2:172.21.15.138:6800/29238828,v1:172.21.15.138:6801/29238828]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:23:07.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:07 smithi119 ceph-mon[29082]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:23:07.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:07 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:07.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:07 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:07.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:07 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:23:07.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:07 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:23:07.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:07 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:07.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:07 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:07.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:07 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:07.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:07 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:23:07.905 DEBUG:teuthology.orchestra.run.smithi138:osd.3> sudo journalctl -f -n 0 -u ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.3.service 2024-05-27T09:23:07.909 INFO:tasks.cephadm:Deploying osd.4 on smithi138 with /dev/vg_nvme/lv_3... 2024-05-27T09:23:07.910 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-27T09:23:08.453 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:08 smithi138 ceph-mon[34885]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-05-27T09:23:08.453 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:08 smithi138 ceph-mon[34885]: osdmap e25: 4 total, 3 up, 4 in 2024-05-27T09:23:08.453 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:08 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T09:23:08.453 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:08 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T09:23:08.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:08 smithi119 ceph-mon[29082]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-05-27T09:23:08.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:08 smithi119 ceph-mon[29082]: osdmap e25: 4 total, 3 up, 4 in 2024-05-27T09:23:08.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:08 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T09:23:08.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:08 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T09:23:09.358 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi138/config 2024-05-27T09:23:09.597 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:09 smithi138 ceph-mon[34885]: purged_snaps scrub starts 2024-05-27T09:23:09.598 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:09 smithi138 ceph-mon[34885]: purged_snaps scrub ok 2024-05-27T09:23:09.598 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:09 smithi138 ceph-mon[34885]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:23:09.598 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:09 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T09:23:09.598 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:09 smithi138 ceph-mon[34885]: osd.3 [v2:172.21.15.138:6800/29238828,v1:172.21.15.138:6801/29238828] boot 2024-05-27T09:23:09.598 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:09 smithi138 ceph-mon[34885]: osdmap e26: 4 total, 4 up, 4 in 2024-05-27T09:23:09.598 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:09 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T09:23:09.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:09 smithi119 ceph-mon[29082]: purged_snaps scrub starts 2024-05-27T09:23:09.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:09 smithi119 ceph-mon[29082]: purged_snaps scrub ok 2024-05-27T09:23:09.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:09 smithi119 ceph-mon[29082]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-05-27T09:23:09.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:09 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T09:23:09.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:09 smithi119 ceph-mon[29082]: osd.3 [v2:172.21.15.138:6800/29238828,v1:172.21.15.138:6801/29238828] boot 2024-05-27T09:23:09.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:09 smithi119 ceph-mon[29082]: osdmap e26: 4 total, 4 up, 4 in 2024-05-27T09:23:09.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:09 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T09:23:10.564 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:10 smithi138 ceph-mon[34885]: osdmap e27: 4 total, 4 up, 4 in 2024-05-27T09:23:10.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:10 smithi119 ceph-mon[29082]: osdmap e27: 4 total, 4 up, 4 in 2024-05-27T09:23:11.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:11 smithi138 ceph-mon[34885]: pgmap v122: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:11.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:11 smithi138 ceph-mon[34885]: osdmap e28: 4 total, 4 up, 4 in 2024-05-27T09:23:11.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:11 smithi138 ceph-mon[34885]: osdmap e29: 4 total, 4 up, 4 in 2024-05-27T09:23:11.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:11 smithi119 ceph-mon[29082]: pgmap v122: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:11.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:11 smithi119 ceph-mon[29082]: osdmap e28: 4 total, 4 up, 4 in 2024-05-27T09:23:11.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:11 smithi119 ceph-mon[29082]: osdmap e29: 4 total, 4 up, 4 in 2024-05-27T09:23:13.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:12 smithi138 ceph-mon[34885]: osdmap e30: 4 total, 4 up, 4 in 2024-05-27T09:23:13.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:12 smithi138 ceph-mon[34885]: Detected new or changed devices on smithi138 2024-05-27T09:23:13.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:12 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:13.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:12 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:13.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:12 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:23:13.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:12 smithi138 ceph-mon[34885]: Adjusting osd_memory_target on smithi138 to 14913M 2024-05-27T09:23:13.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:12 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:13.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:12 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:23:13.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:12 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:23:13.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:12 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:13.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:12 smithi138 ceph-mon[34885]: pgmap v126: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:13.202 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:12 smithi119 ceph-mon[29082]: osdmap e30: 4 total, 4 up, 4 in 2024-05-27T09:23:13.202 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:12 smithi119 ceph-mon[29082]: Detected new or changed devices on smithi138 2024-05-27T09:23:13.202 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:12 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:13.202 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:12 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:13.202 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:12 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:23:13.202 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:12 smithi119 ceph-mon[29082]: Adjusting osd_memory_target on smithi138 to 14913M 2024-05-27T09:23:13.202 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:12 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:13.202 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:12 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:23:13.202 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:12 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:23:13.203 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:12 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:13.203 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:12 smithi119 ceph-mon[29082]: pgmap v126: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:13.301 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-27T09:23:13.301 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-27T09:23:13.301 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-05-27T09:23:13.301 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-05-27T09:23:13.301 INFO:teuthology.orchestra.run.smithi138.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0364975 s, 287 MB/s 2024-05-27T09:23:13.301 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-05-27T09:23:14.048 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_3 2024-05-27T09:23:14.441 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi138/config 2024-05-27T09:23:15.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:15 smithi138 ceph-mon[34885]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-05-27T09:23:15.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:15 smithi119 ceph-mon[29082]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-05-27T09:23:16.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.621+0000 7f950d200700 1 -- 172.21.15.138:0/2732376525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95080ff320 msgr2=0x7f95080ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:23:16.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.621+0000 7f950d200700 1 --2- 172.21.15.138:0/2732376525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95080ff320 0x7f95080ff790 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009910 tx=0x7f94f0009c20 comp rx=0 tx=0).stop 2024-05-27T09:23:16.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.622+0000 7f950d200700 1 -- 172.21.15.138:0/2732376525 shutdown_connections 2024-05-27T09:23:16.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.622+0000 7f950d200700 1 --2- 172.21.15.138:0/2732376525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95080ff320 0x7f95080ff790 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:23:16.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.622+0000 7f950d200700 1 --2- 172.21.15.138:0/2732376525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95080fe9d0 0x7f95080fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:23:16.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.622+0000 7f950d200700 1 -- 172.21.15.138:0/2732376525 >> 172.21.15.138:0/2732376525 conn(0x7f95080fa5e0 msgr2=0x7f95080fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:23:16.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.622+0000 7f950d200700 1 -- 172.21.15.138:0/2732376525 shutdown_connections 2024-05-27T09:23:16.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.622+0000 7f950d200700 1 -- 172.21.15.138:0/2732376525 wait complete. 2024-05-27T09:23:16.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.623+0000 7f950d200700 1 Processor -- start 2024-05-27T09:23:16.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.623+0000 7f950d200700 1 -- start start 2024-05-27T09:23:16.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.624+0000 7f950d200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95080fe9d0 0x7f9508197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:23:16.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.624+0000 7f950d200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95080ff320 0x7f9508198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:23:16.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.624+0000 7f950d200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95081986a0 con 0x7f95080fe9d0 2024-05-27T09:23:16.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.624+0000 7f950d200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95081987e0 con 0x7f95080ff320 2024-05-27T09:23:16.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.624+0000 7f94fd400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95080ff320 0x7f9508198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:23:16.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.624+0000 7f94fd400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95080ff320 0x7f9508198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46682/0 (socket says 172.21.15.138:46682) 2024-05-27T09:23:16.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.624+0000 7f94fd400700 1 -- 172.21.15.138:0/2141338265 learned_addr learned my addr 172.21.15.138:0/2141338265 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:23:16.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.624+0000 7f9505e00700 1 --2- 172.21.15.138:0/2141338265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95080fe9d0 0x7f9508197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:23:16.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.625+0000 7f94fd400700 1 -- 172.21.15.138:0/2141338265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95080fe9d0 msgr2=0x7f9508197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:23:16.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.625+0000 7f94fd400700 1 --2- 172.21.15.138:0/2141338265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95080fe9d0 0x7f9508197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:23:16.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.625+0000 7f94fd400700 1 -- 172.21.15.138:0/2141338265 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f00095f0 con 0x7f95080ff320 2024-05-27T09:23:16.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.625+0000 7f94fd400700 1 --2- 172.21.15.138:0/2141338265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95080ff320 0x7f9508198080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f94f00092e0 tx=0x7f94f000f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:23:16.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.625+0000 7f94ff400700 1 -- 172.21.15.138:0/2141338265 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94f001d070 con 0x7f95080ff320 2024-05-27T09:23:16.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.625+0000 7f94ff400700 1 -- 172.21.15.138:0/2141338265 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f94f0005dc0 con 0x7f95080ff320 2024-05-27T09:23:16.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.625+0000 7f94ff400700 1 -- 172.21.15.138:0/2141338265 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94f0017610 con 0x7f95080ff320 2024-05-27T09:23:16.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.625+0000 7f950d200700 1 -- 172.21.15.138:0/2141338265 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f950819d230 con 0x7f95080ff320 2024-05-27T09:23:16.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.625+0000 7f950d200700 1 -- 172.21.15.138:0/2141338265 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f950819d720 con 0x7f95080ff320 2024-05-27T09:23:16.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.626+0000 7f950d200700 1 -- 172.21.15.138:0/2141338265 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f950810ae80 con 0x7f95080ff320 2024-05-27T09:23:16.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.627+0000 7f94ff400700 1 -- 172.21.15.138:0/2141338265 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f94f000fba0 con 0x7f95080ff320 2024-05-27T09:23:16.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.629+0000 7f94ff400700 1 --2- 172.21.15.138:0/2141338265 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f94ec06c680 0x7f94ec06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:23:16.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.629+0000 7f94ff400700 1 -- 172.21.15.138:0/2141338265 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3774+0+0 (secure 0 0 0) 0x7f94f008c690 con 0x7f95080ff320 2024-05-27T09:23:16.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.630+0000 7f9505e00700 1 --2- 172.21.15.138:0/2141338265 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f94ec06c680 0x7f94ec06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:23:16.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.631+0000 7f9505e00700 1 --2- 172.21.15.138:0/2141338265 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f94ec06c680 0x7f94ec06eb30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009bf0 tx=0x7f94f8009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:23:16.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.635+0000 7f94ff400700 1 -- 172.21.15.138:0/2141338265 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f94f005af20 con 0x7f95080ff320 2024-05-27T09:23:16.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:16.858+0000 7f950d200700 1 -- 172.21.15.138:0/2141338265 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9508108720 con 0x7f94ec06c680 2024-05-27T09:23:17.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:16 smithi138 ceph-mon[34885]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T09:23:17.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:16 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T09:23:17.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:16 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T09:23:17.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:16 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:23:17.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:16 smithi119 ceph-mon[29082]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T09:23:17.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:16 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T09:23:17.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:16 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T09:23:17.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:16 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:23:18.001 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:17 smithi138 ceph-mon[34885]: from='client.24141 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:23:18.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:17 smithi119 ceph-mon[29082]: from='client.24141 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:23:19.117 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:18 smithi138 ceph-mon[34885]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-27T09:23:19.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:18 smithi119 ceph-mon[29082]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-05-27T09:23:21.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:21 smithi138 ceph-mon[34885]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-27T09:23:21.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:21 smithi138 ceph-mon[34885]: from='client.? 172.21.15.138:0/2647380333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd95d23f-b654-4cb6-819a-27c4c992f695"}]: dispatch 2024-05-27T09:23:21.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:21 smithi138 ceph-mon[34885]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd95d23f-b654-4cb6-819a-27c4c992f695"}]: dispatch 2024-05-27T09:23:21.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:21 smithi138 ceph-mon[34885]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd95d23f-b654-4cb6-819a-27c4c992f695"}]': finished 2024-05-27T09:23:21.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:21 smithi138 ceph-mon[34885]: osdmap e31: 5 total, 4 up, 5 in 2024-05-27T09:23:21.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:21 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T09:23:21.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:21 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:23:21.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:21 smithi119 ceph-mon[29082]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-27T09:23:21.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:21 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/2647380333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd95d23f-b654-4cb6-819a-27c4c992f695"}]: dispatch 2024-05-27T09:23:21.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:21 smithi119 ceph-mon[29082]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd95d23f-b654-4cb6-819a-27c4c992f695"}]: dispatch 2024-05-27T09:23:21.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:21 smithi119 ceph-mon[29082]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd95d23f-b654-4cb6-819a-27c4c992f695"}]': finished 2024-05-27T09:23:21.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:21 smithi119 ceph-mon[29082]: osdmap e31: 5 total, 4 up, 5 in 2024-05-27T09:23:21.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:21 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T09:23:21.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:21 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:23:22.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:22 smithi138 ceph-mon[34885]: from='client.? 172.21.15.138:0/3289197780' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T09:23:22.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:22 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/3289197780' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T09:23:23.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:23 smithi138 ceph-mon[34885]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T09:23:23.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:23 smithi119 ceph-mon[29082]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T09:23:25.591 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:25 smithi138 ceph-mon[34885]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:25.673 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:25 smithi119 ceph-mon[29082]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:27.067 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:26 smithi138 ceph-mon[34885]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:27.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:26 smithi119 ceph-mon[29082]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:29.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:29 smithi138 ceph-mon[34885]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:29.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:29 smithi119 ceph-mon[29082]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:30.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:30 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-27T09:23:30.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:30 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:23:30.649 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:30 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-27T09:23:30.649 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:30 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:23:31.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:31 smithi138 ceph-mon[34885]: Deploying daemon osd.4 on smithi138 2024-05-27T09:23:31.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:31 smithi138 ceph-mon[34885]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:31.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:31 smithi119 ceph-mon[29082]: Deploying daemon osd.4 on smithi138 2024-05-27T09:23:31.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:31 smithi119 ceph-mon[29082]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:33.614 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:33 smithi138 ceph-mon[34885]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:33.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:33 smithi119 ceph-mon[29082]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:35.593 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:35 smithi138 ceph-mon[34885]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:35.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:35 smithi119 ceph-mon[29082]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:37.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:36 smithi119 ceph-mon[29082]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:37.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:36 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:23:37.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:36 smithi138 ceph-mon[34885]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:37.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:36 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:23:39.331 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:39 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:39.331 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:39 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:39.331 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:39 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:23:39.331 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:39 smithi138 ceph-mon[34885]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:39.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:39 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:39.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:39 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:39.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:39 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:23:39.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:39 smithi119 ceph-mon[29082]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:41.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:41 smithi138 ceph-mon[34885]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:41.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:41 smithi119 ceph-mon[29082]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:43.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:43.028+0000 7f94ff400700 1 -- 172.21.15.138:0/2141338265 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9508108720 con 0x7f94ec06c680 2024-05-27T09:23:43.030 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 4 on host 'smithi138' 2024-05-27T09:23:43.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:43.032+0000 7f950d200700 1 -- 172.21.15.138:0/2141338265 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f94ec06c680 msgr2=0x7f94ec06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:23:43.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:43.032+0000 7f950d200700 1 --2- 172.21.15.138:0/2141338265 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f94ec06c680 0x7f94ec06eb30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009bf0 tx=0x7f94f8009260 comp rx=0 tx=0).stop 2024-05-27T09:23:43.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:43.032+0000 7f950d200700 1 -- 172.21.15.138:0/2141338265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95080ff320 msgr2=0x7f9508198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:23:43.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:43.032+0000 7f950d200700 1 --2- 172.21.15.138:0/2141338265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95080ff320 0x7f9508198080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f94f00092e0 tx=0x7f94f000f740 comp rx=0 tx=0).stop 2024-05-27T09:23:43.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:43.032+0000 7f950d200700 1 -- 172.21.15.138:0/2141338265 shutdown_connections 2024-05-27T09:23:43.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:43.032+0000 7f950d200700 1 --2- 172.21.15.138:0/2141338265 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f94ec06c680 0x7f94ec06eb30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:23:43.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:43.032+0000 7f950d200700 1 --2- 172.21.15.138:0/2141338265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95080fe9d0 0x7f9508197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:23:43.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:43.032+0000 7f950d200700 1 --2- 172.21.15.138:0/2141338265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95080ff320 0x7f9508198080 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:23:43.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:43.032+0000 7f950d200700 1 -- 172.21.15.138:0/2141338265 >> 172.21.15.138:0/2141338265 conn(0x7f95080fa5e0 msgr2=0x7f9508107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:23:43.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:43.032+0000 7f950d200700 1 -- 172.21.15.138:0/2141338265 shutdown_connections 2024-05-27T09:23:43.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:43.032+0000 7f950d200700 1 -- 172.21.15.138:0/2141338265 wait complete. 2024-05-27T09:23:43.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:43 smithi138 ceph-mon[34885]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:43.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:43 smithi138 ceph-mon[34885]: from='osd.4 [v2:172.21.15.138:6808/3397624307,v1:172.21.15.138:6809/3397624307]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T09:23:43.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:43 smithi138 ceph-mon[34885]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T09:23:43.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:43 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:43.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:43 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:43.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:43 smithi119 ceph-mon[29082]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:43.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:43 smithi119 ceph-mon[29082]: from='osd.4 [v2:172.21.15.138:6808/3397624307,v1:172.21.15.138:6809/3397624307]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T09:23:43.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:43 smithi119 ceph-mon[29082]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T09:23:43.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:43 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:43.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:43 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:43.878 DEBUG:teuthology.orchestra.run.smithi138:osd.4> sudo journalctl -f -n 0 -u ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.4.service 2024-05-27T09:23:43.882 INFO:tasks.cephadm:Deploying osd.5 on smithi138 with /dev/vg_nvme/lv_2... 2024-05-27T09:23:43.883 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-27T09:23:44.282 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi138/config 2024-05-27T09:23:44.567 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:44 smithi138 ceph-mon[34885]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-27T09:23:44.567 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:44 smithi138 ceph-mon[34885]: osdmap e32: 5 total, 4 up, 5 in 2024-05-27T09:23:44.567 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:44 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T09:23:44.567 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:44 smithi138 ceph-mon[34885]: from='osd.4 [v2:172.21.15.138:6808/3397624307,v1:172.21.15.138:6809/3397624307]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:23:44.568 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:44 smithi138 ceph-mon[34885]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:23:44.568 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:44 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:44.568 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:44 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:44.568 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:23:44 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[45566]: 2024-05-27T09:23:44.315+0000 7fe3a2c00700 -1 osd.4 0 waiting for initial osdmap 2024-05-27T09:23:44.568 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:23:44 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[45566]: 2024-05-27T09:23:44.330+0000 7fe39b200700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T09:23:44.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:44 smithi119 ceph-mon[29082]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-27T09:23:44.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:44 smithi119 ceph-mon[29082]: osdmap e32: 5 total, 4 up, 5 in 2024-05-27T09:23:44.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:44 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T09:23:44.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:44 smithi119 ceph-mon[29082]: from='osd.4 [v2:172.21.15.138:6808/3397624307,v1:172.21.15.138:6809/3397624307]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:23:44.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:44 smithi119 ceph-mon[29082]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:23:44.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:44 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:44.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:44 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:45.562 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:45 smithi138 ceph-mon[34885]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:45.562 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:45 smithi138 ceph-mon[34885]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-05-27T09:23:45.562 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:45 smithi138 ceph-mon[34885]: osdmap e33: 5 total, 4 up, 5 in 2024-05-27T09:23:45.562 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:45 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T09:23:45.562 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:45 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T09:23:45.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:45 smithi119 ceph-mon[29082]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-05-27T09:23:45.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:45 smithi119 ceph-mon[29082]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-05-27T09:23:45.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:45 smithi119 ceph-mon[29082]: osdmap e33: 5 total, 4 up, 5 in 2024-05-27T09:23:45.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:45 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T09:23:45.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:45 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T09:23:46.410 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:46 smithi138 ceph-mon[34885]: purged_snaps scrub starts 2024-05-27T09:23:46.410 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:46 smithi138 ceph-mon[34885]: purged_snaps scrub ok 2024-05-27T09:23:46.410 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:46 smithi138 ceph-mon[34885]: osd.4 [v2:172.21.15.138:6808/3397624307,v1:172.21.15.138:6809/3397624307] boot 2024-05-27T09:23:46.410 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:46 smithi138 ceph-mon[34885]: osdmap e34: 5 total, 5 up, 5 in 2024-05-27T09:23:46.411 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:46 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T09:23:46.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:46 smithi119 ceph-mon[29082]: purged_snaps scrub starts 2024-05-27T09:23:46.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:46 smithi119 ceph-mon[29082]: purged_snaps scrub ok 2024-05-27T09:23:46.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:46 smithi119 ceph-mon[29082]: osd.4 [v2:172.21.15.138:6808/3397624307,v1:172.21.15.138:6809/3397624307] boot 2024-05-27T09:23:46.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:46 smithi119 ceph-mon[29082]: osdmap e34: 5 total, 5 up, 5 in 2024-05-27T09:23:46.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:46 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T09:23:47.210 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-27T09:23:47.211 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-27T09:23:47.211 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-05-27T09:23:47.211 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-05-27T09:23:47.211 INFO:teuthology.orchestra.run.smithi138.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0330545 s, 317 MB/s 2024-05-27T09:23:47.211 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-05-27T09:23:47.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:47 smithi138 ceph-mon[34885]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:23:47.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:47 smithi138 ceph-mon[34885]: osdmap e35: 5 total, 5 up, 5 in 2024-05-27T09:23:47.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:47 smithi119 ceph-mon[29082]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:23:47.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:47 smithi119 ceph-mon[29082]: osdmap e35: 5 total, 5 up, 5 in 2024-05-27T09:23:48.379 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_2 2024-05-27T09:23:49.050 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi138/config 2024-05-27T09:23:49.437 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:49 smithi138 ceph-mon[34885]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:23:49.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:49 smithi119 ceph-mon[29082]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:23:51.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:51 smithi119 ceph-mon[29082]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:23:51.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:51 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:23:51.524 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:51 smithi138 ceph-mon[34885]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:23:51.524 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:51 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:23:51.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.865+0000 7fd4fa000700 1 -- 172.21.15.138:0/4190912854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f41013f0 msgr2=0x7fd4f4101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:23:51.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.865+0000 7fd4fa000700 1 --2- 172.21.15.138:0/4190912854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f41013f0 0x7fd4f4101840 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009910 tx=0x7fd4e0009c20 comp rx=0 tx=0).stop 2024-05-27T09:23:51.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.865+0000 7fd4fa000700 1 -- 172.21.15.138:0/4190912854 shutdown_connections 2024-05-27T09:23:51.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.865+0000 7fd4fa000700 1 --2- 172.21.15.138:0/4190912854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f41013f0 0x7fd4f4101840 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:23:51.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.865+0000 7fd4fa000700 1 --2- 172.21.15.138:0/4190912854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f40fe410 0x7fd4f40fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:23:51.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.865+0000 7fd4fa000700 1 -- 172.21.15.138:0/4190912854 >> 172.21.15.138:0/4190912854 conn(0x7fd4f40f9d80 msgr2=0x7fd4f40fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:23:51.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.865+0000 7fd4fa000700 1 -- 172.21.15.138:0/4190912854 shutdown_connections 2024-05-27T09:23:51.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.865+0000 7fd4fa000700 1 -- 172.21.15.138:0/4190912854 wait complete. 2024-05-27T09:23:51.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.866+0000 7fd4fa000700 1 Processor -- start 2024-05-27T09:23:51.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.866+0000 7fd4fa000700 1 -- start start 2024-05-27T09:23:51.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.867+0000 7fd4fa000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f40fe410 0x7fd4f4100d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:23:51.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.867+0000 7fd4fa000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f41013f0 0x7fd4f40ff3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:23:51.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.867+0000 7fd4fa000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f40ff8e0 con 0x7fd4f41013f0 2024-05-27T09:23:51.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.867+0000 7fd4fa000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f40ffa20 con 0x7fd4f40fe410 2024-05-27T09:23:51.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.867+0000 7fd4f3400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f40fe410 0x7fd4f4100d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:23:51.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.867+0000 7fd4f3400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f40fe410 0x7fd4f4100d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38108/0 (socket says 172.21.15.138:38108) 2024-05-27T09:23:51.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.867+0000 7fd4f3400700 1 -- 172.21.15.138:0/629499770 learned_addr learned my addr 172.21.15.138:0/629499770 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:23:51.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.867+0000 7fd4f2a00700 1 --2- 172.21.15.138:0/629499770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f41013f0 0x7fd4f40ff3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:23:51.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.867+0000 7fd4f3400700 1 -- 172.21.15.138:0/629499770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f41013f0 msgr2=0x7fd4f40ff3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:23:51.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.867+0000 7fd4f3400700 1 --2- 172.21.15.138:0/629499770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f41013f0 0x7fd4f40ff3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:23:51.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.867+0000 7fd4f3400700 1 -- 172.21.15.138:0/629499770 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e00095f0 con 0x7fd4f40fe410 2024-05-27T09:23:51.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.867+0000 7fd4f2a00700 1 --2- 172.21.15.138:0/629499770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f41013f0 0x7fd4f40ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:23:51.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.868+0000 7fd4f3400700 1 --2- 172.21.15.138:0/629499770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f40fe410 0x7fd4f4100d50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400c910 tx=0x7fd4e400ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:23:51.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.868+0000 7fd4ebe00700 1 -- 172.21.15.138:0/629499770 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4e40041f0 con 0x7fd4f40fe410 2024-05-27T09:23:51.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.868+0000 7fd4fa000700 1 -- 172.21.15.138:0/629499770 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f40ffd00 con 0x7fd4f40fe410 2024-05-27T09:23:51.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.868+0000 7fd4ebe00700 1 -- 172.21.15.138:0/629499770 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd4e4007db0 con 0x7fd4f40fe410 2024-05-27T09:23:51.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.868+0000 7fd4ebe00700 1 -- 172.21.15.138:0/629499770 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4e4003d60 con 0x7fd4f40fe410 2024-05-27T09:23:51.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.868+0000 7fd4fa000700 1 -- 172.21.15.138:0/629499770 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f419d9b0 con 0x7fd4f40fe410 2024-05-27T09:23:51.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.869+0000 7fd4fa000700 1 -- 172.21.15.138:0/629499770 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f4108c90 con 0x7fd4f40fe410 2024-05-27T09:23:51.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.870+0000 7fd4ebe00700 1 -- 172.21.15.138:0/629499770 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fd4e4007900 con 0x7fd4f40fe410 2024-05-27T09:23:51.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.871+0000 7fd4ebe00700 1 --2- 172.21.15.138:0/629499770 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fd4d806c680 0x7fd4d806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:23:51.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.871+0000 7fd4ebe00700 1 -- 172.21.15.138:0/629499770 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4206+0+0 (secure 0 0 0) 0x7fd4e408b0c0 con 0x7fd4f40fe410 2024-05-27T09:23:51.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.872+0000 7fd4f2a00700 1 --2- 172.21.15.138:0/629499770 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fd4d806c680 0x7fd4d806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:23:51.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.873+0000 7fd4f2a00700 1 --2- 172.21.15.138:0/629499770 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fd4d806c680 0x7fd4d806eb30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0006010 tx=0x7fd4e000b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:23:51.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:51.876+0000 7fd4ebe00700 1 -- 172.21.15.138:0/629499770 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd4e4059720 con 0x7fd4f40fe410 2024-05-27T09:23:52.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:23:52.108+0000 7fd4fa000700 1 -- 172.21.15.138:0/629499770 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4f419df60 con 0x7fd4d806c680 2024-05-27T09:23:52.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:52 smithi138 ceph-mon[34885]: Detected new or changed devices on smithi138 2024-05-27T09:23:52.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:52 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:52.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:52 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:52.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:52 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:23:52.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:52 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:23:52.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:52 smithi138 ceph-mon[34885]: Adjusting osd_memory_target on smithi138 to 7456M 2024-05-27T09:23:52.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:52 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:52.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:52 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:23:52.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:52 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:23:52.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:52 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:52.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:52 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:23:52.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:52 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:23:52.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:52 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:23:52.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:52 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:52.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:52 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T09:23:52.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:52 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T09:23:52.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:52 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:23:52.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:52 smithi119 ceph-mon[29082]: Detected new or changed devices on smithi138 2024-05-27T09:23:52.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:52 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:52.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:52 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:52.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:52 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:23:52.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:52 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:23:52.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:52 smithi119 ceph-mon[29082]: Adjusting osd_memory_target on smithi138 to 7456M 2024-05-27T09:23:52.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:52 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:52.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:52 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:23:52.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:52 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:23:52.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:52 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:52.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:52 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:23:52.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:52 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:23:52.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:52 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:23:52.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:52 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:23:52.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:52 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T09:23:52.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:52 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T09:23:52.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:52 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:23:53.881 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:53 smithi138 ceph-mon[34885]: from='client.24159 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:23:53.881 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:53 smithi138 ceph-mon[34885]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:23:53.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:53 smithi119 ceph-mon[29082]: from='client.24159 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:23:53.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:53 smithi119 ceph-mon[29082]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:23:55.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:55 smithi119 ceph-mon[29082]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:23:56.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:55 smithi138 ceph-mon[34885]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:23:57.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:56 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/2848031098' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "172a12ad-ca75-486d-a379-2e202fe6b6bd"}]: dispatch 2024-05-27T09:23:57.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:56 smithi119 ceph-mon[29082]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "172a12ad-ca75-486d-a379-2e202fe6b6bd"}]: dispatch 2024-05-27T09:23:57.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:56 smithi119 ceph-mon[29082]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "172a12ad-ca75-486d-a379-2e202fe6b6bd"}]': finished 2024-05-27T09:23:57.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:56 smithi119 ceph-mon[29082]: osdmap e36: 6 total, 5 up, 6 in 2024-05-27T09:23:57.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:56 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T09:23:57.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:56 smithi119 ceph-mon[29082]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:23:57.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:56 smithi138 ceph-mon[34885]: from='client.? 172.21.15.138:0/2848031098' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "172a12ad-ca75-486d-a379-2e202fe6b6bd"}]: dispatch 2024-05-27T09:23:57.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:56 smithi138 ceph-mon[34885]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "172a12ad-ca75-486d-a379-2e202fe6b6bd"}]: dispatch 2024-05-27T09:23:57.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:56 smithi138 ceph-mon[34885]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "172a12ad-ca75-486d-a379-2e202fe6b6bd"}]': finished 2024-05-27T09:23:57.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:56 smithi138 ceph-mon[34885]: osdmap e36: 6 total, 5 up, 6 in 2024-05-27T09:23:57.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:56 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T09:23:57.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:56 smithi138 ceph-mon[34885]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:23:58.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:57 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/1605600801' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T09:23:58.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:57 smithi138 ceph-mon[34885]: from='client.? 172.21.15.138:0/1605600801' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T09:23:59.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:23:58 smithi119 ceph-mon[29082]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:23:59.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:23:58 smithi138 ceph-mon[34885]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:01.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:01 smithi138 ceph-mon[34885]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:01.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:01 smithi119 ceph-mon[29082]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:03.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:03 smithi138 ceph-mon[34885]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:03.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:03 smithi119 ceph-mon[29082]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:05.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:05 smithi138 ceph-mon[34885]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:05.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:05 smithi119 ceph-mon[29082]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:06.578 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:06 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-27T09:24:06.578 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:06 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:24:06.578 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:06 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:24:06.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:06 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-27T09:24:06.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:06 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:24:06.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:06 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:24:07.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:07 smithi138 ceph-mon[34885]: Deploying daemon osd.5 on smithi138 2024-05-27T09:24:07.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:07 smithi138 ceph-mon[34885]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:07.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:07 smithi119 ceph-mon[29082]: Deploying daemon osd.5 on smithi138 2024-05-27T09:24:07.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:07 smithi119 ceph-mon[29082]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:09.555 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:09 smithi138 ceph-mon[34885]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:09.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:09 smithi119 ceph-mon[29082]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:11.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:11 smithi138 ceph-mon[34885]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:11.688 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:11 smithi119 ceph-mon[29082]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:13.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:13 smithi138 ceph-mon[34885]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:13.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:13 smithi119 ceph-mon[29082]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:15.605 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:15 smithi138 ceph-mon[34885]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:15.605 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:15 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:24:15.605 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:15 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:24:15.605 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:15 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:24:15.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:15 smithi119 ceph-mon[29082]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:15.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:15 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:24:15.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:15 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:24:15.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:15 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:24:17.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:16 smithi119 ceph-mon[29082]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:17.258 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:16 smithi138 ceph-mon[34885]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:19.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:19 smithi138 ceph-mon[34885]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:19.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:19 smithi138 ceph-mon[34885]: from='osd.5 [v2:172.21.15.138:6816/886004363,v1:172.21.15.138:6817/886004363]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T09:24:19.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:19 smithi138 ceph-mon[34885]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T09:24:19.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:19 smithi119 ceph-mon[29082]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:19.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:19 smithi119 ceph-mon[29082]: from='osd.5 [v2:172.21.15.138:6816/886004363,v1:172.21.15.138:6817/886004363]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T09:24:19.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:19 smithi119 ceph-mon[29082]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T09:24:19.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:19.768+0000 7fd4ebe00700 1 -- 172.21.15.138:0/629499770 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd4f419df60 con 0x7fd4d806c680 2024-05-27T09:24:19.769 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 5 on host 'smithi138' 2024-05-27T09:24:19.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:19.773+0000 7fd4fa000700 1 -- 172.21.15.138:0/629499770 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fd4d806c680 msgr2=0x7fd4d806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:19.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:19.773+0000 7fd4fa000700 1 --2- 172.21.15.138:0/629499770 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fd4d806c680 0x7fd4d806eb30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0006010 tx=0x7fd4e000b540 comp rx=0 tx=0).stop 2024-05-27T09:24:19.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:19.773+0000 7fd4fa000700 1 -- 172.21.15.138:0/629499770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f40fe410 msgr2=0x7fd4f4100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:19.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:19.773+0000 7fd4fa000700 1 --2- 172.21.15.138:0/629499770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f40fe410 0x7fd4f4100d50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400c910 tx=0x7fd4e400ccd0 comp rx=0 tx=0).stop 2024-05-27T09:24:19.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:19.773+0000 7fd4fa000700 1 -- 172.21.15.138:0/629499770 shutdown_connections 2024-05-27T09:24:19.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:19.773+0000 7fd4fa000700 1 --2- 172.21.15.138:0/629499770 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fd4d806c680 0x7fd4d806eb30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:19.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:19.773+0000 7fd4fa000700 1 --2- 172.21.15.138:0/629499770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f40fe410 0x7fd4f4100d50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:19.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:19.773+0000 7fd4fa000700 1 --2- 172.21.15.138:0/629499770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f41013f0 0x7fd4f40ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:19.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:19.773+0000 7fd4fa000700 1 -- 172.21.15.138:0/629499770 >> 172.21.15.138:0/629499770 conn(0x7fd4f40f9d80 msgr2=0x7fd4f4104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:19.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:19.774+0000 7fd4fa000700 1 -- 172.21.15.138:0/629499770 shutdown_connections 2024-05-27T09:24:19.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:19.774+0000 7fd4fa000700 1 -- 172.21.15.138:0/629499770 wait complete. 2024-05-27T09:24:20.588 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:20 smithi138 ceph-mon[34885]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-27T09:24:20.588 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:20 smithi138 ceph-mon[34885]: osdmap e37: 6 total, 5 up, 6 in 2024-05-27T09:24:20.588 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T09:24:20.588 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:20 smithi138 ceph-mon[34885]: from='osd.5 [v2:172.21.15.138:6816/886004363,v1:172.21.15.138:6817/886004363]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:24:20.588 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:20 smithi138 ceph-mon[34885]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:24:20.588 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:24:20.588 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:24:20.650 DEBUG:teuthology.orchestra.run.smithi138:osd.5> sudo journalctl -f -n 0 -u ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.5.service 2024-05-27T09:24:20.659 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-05-27T09:24:20.659 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-05-27T09:24:20.739 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:20 smithi119 ceph-mon[29082]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-27T09:24:20.739 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:20 smithi119 ceph-mon[29082]: osdmap e37: 6 total, 5 up, 6 in 2024-05-27T09:24:20.739 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:20 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T09:24:20.739 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:20 smithi119 ceph-mon[29082]: from='osd.5 [v2:172.21.15.138:6816/886004363,v1:172.21.15.138:6817/886004363]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:24:20.739 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:20 smithi119 ceph-mon[29082]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:24:20.739 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:20 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:24:20.739 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:20 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:24:21.071 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:24:21.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:21 smithi119 ceph-mon[29082]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:21.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:21 smithi119 ceph-mon[29082]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-05-27T09:24:21.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:21 smithi119 ceph-mon[29082]: osdmap e38: 6 total, 5 up, 6 in 2024-05-27T09:24:21.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:21 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T09:24:21.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:21 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T09:24:21.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:21 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:24:21.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:21 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:24:21.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:21 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:24:21.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:21 smithi138 ceph-mon[34885]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:21.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:21 smithi138 ceph-mon[34885]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-05-27T09:24:21.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:21 smithi138 ceph-mon[34885]: osdmap e38: 6 total, 5 up, 6 in 2024-05-27T09:24:21.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:21 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T09:24:21.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:21 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T09:24:21.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:21 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:24:21.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:21 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:24:21.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:21 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:24:22.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:22 smithi119 ceph-mon[29082]: purged_snaps scrub starts 2024-05-27T09:24:22.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:22 smithi119 ceph-mon[29082]: purged_snaps scrub ok 2024-05-27T09:24:22.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:22 smithi119 ceph-mon[29082]: osd.5 [v2:172.21.15.138:6816/886004363,v1:172.21.15.138:6817/886004363] boot 2024-05-27T09:24:22.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:22 smithi119 ceph-mon[29082]: osdmap e39: 6 total, 6 up, 6 in 2024-05-27T09:24:22.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:22 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T09:24:22.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:22 smithi138 ceph-mon[34885]: purged_snaps scrub starts 2024-05-27T09:24:22.638 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:22 smithi138 ceph-mon[34885]: purged_snaps scrub ok 2024-05-27T09:24:22.638 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:22 smithi138 ceph-mon[34885]: osd.5 [v2:172.21.15.138:6816/886004363,v1:172.21.15.138:6817/886004363] boot 2024-05-27T09:24:22.638 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:22 smithi138 ceph-mon[34885]: osdmap e39: 6 total, 6 up, 6 in 2024-05-27T09:24:22.638 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:22 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T09:24:23.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.188+0000 7f5cc1600700 1 -- 172.21.15.119:0/2198622714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cbc1079a0 msgr2=0x7f5cbc107df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:23.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.188+0000 7f5cc1600700 1 --2- 172.21.15.119:0/2198622714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cbc1079a0 0x7f5cbc107df0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009910 tx=0x7f5ca8009c20 comp rx=0 tx=0).stop 2024-05-27T09:24:23.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.188+0000 7f5cc1600700 1 -- 172.21.15.119:0/2198622714 shutdown_connections 2024-05-27T09:24:23.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.188+0000 7f5cc1600700 1 --2- 172.21.15.119:0/2198622714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cbc1079a0 0x7f5cbc107df0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:23.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.188+0000 7f5cc1600700 1 --2- 172.21.15.119:0/2198622714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc1067a0 0x7f5cbc106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:23.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.188+0000 7f5cc1600700 1 -- 172.21.15.119:0/2198622714 >> 172.21.15.119:0/2198622714 conn(0x7f5cbc075910 msgr2=0x7f5cbc077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:23.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.188+0000 7f5cc1600700 1 -- 172.21.15.119:0/2198622714 shutdown_connections 2024-05-27T09:24:23.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.188+0000 7f5cc1600700 1 -- 172.21.15.119:0/2198622714 wait complete. 2024-05-27T09:24:23.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.189+0000 7f5cc1600700 1 Processor -- start 2024-05-27T09:24:23.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.189+0000 7f5cc1600700 1 -- start start 2024-05-27T09:24:23.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.190+0000 7f5cc1600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc1067a0 0x7f5cbc19bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:23.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.190+0000 7f5cc1600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cbc1079a0 0x7f5cbc19c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:23.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.190+0000 7f5cc1600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc19cb00 con 0x7f5cbc1079a0 2024-05-27T09:24:23.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.190+0000 7f5cc1600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc19cc40 con 0x7f5cbc1067a0 2024-05-27T09:24:23.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.190+0000 7f5cba000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cbc1079a0 0x7f5cbc19c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:23.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.190+0000 7f5cba000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cbc1079a0 0x7f5cbc19c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37940/0 (socket says 172.21.15.119:37940) 2024-05-27T09:24:23.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.190+0000 7f5cba000700 1 -- 172.21.15.119:0/824400405 learned_addr learned my addr 172.21.15.119:0/824400405 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:24:23.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.190+0000 7f5cbaa00700 1 --2- 172.21.15.119:0/824400405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc1067a0 0x7f5cbc19bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:23.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.190+0000 7f5cba000700 1 -- 172.21.15.119:0/824400405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc1067a0 msgr2=0x7f5cbc19bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:23.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.190+0000 7f5cba000700 1 --2- 172.21.15.119:0/824400405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc1067a0 0x7f5cbc19bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:23.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.190+0000 7f5cba000700 1 -- 172.21.15.119:0/824400405 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca80095f0 con 0x7f5cbc1079a0 2024-05-27T09:24:23.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.191+0000 7f5cba000700 1 --2- 172.21.15.119:0/824400405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cbc1079a0 0x7f5cbc19c4e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8006010 tx=0x7f5ca800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:23.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.191+0000 7f5cb3400700 1 -- 172.21.15.119:0/824400405 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ca801d070 con 0x7f5cbc1079a0 2024-05-27T09:24:23.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.191+0000 7f5cc1600700 1 -- 172.21.15.119:0/824400405 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cbc1a1690 con 0x7f5cbc1079a0 2024-05-27T09:24:23.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.191+0000 7f5cb3400700 1 -- 172.21.15.119:0/824400405 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5ca800f460 con 0x7f5cbc1079a0 2024-05-27T09:24:23.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.191+0000 7f5cb3400700 1 -- 172.21.15.119:0/824400405 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ca8021640 con 0x7f5cbc1079a0 2024-05-27T09:24:23.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.191+0000 7f5cc1600700 1 -- 172.21.15.119:0/824400405 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cbc1a1b80 con 0x7f5cbc1079a0 2024-05-27T09:24:23.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.192+0000 7f5cc1600700 1 -- 172.21.15.119:0/824400405 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cbc10f2f0 con 0x7f5cbc1079a0 2024-05-27T09:24:23.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.193+0000 7f5cb3400700 1 -- 172.21.15.119:0/824400405 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f5ca800fab0 con 0x7f5cbc1079a0 2024-05-27T09:24:23.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.194+0000 7f5cb3400700 1 --2- 172.21.15.119:0/824400405 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f5ca006c680 0x7f5ca006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:23.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.194+0000 7f5cb3400700 1 -- 172.21.15.119:0/824400405 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f5ca808c8b0 con 0x7f5cbc1079a0 2024-05-27T09:24:23.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.194+0000 7f5cbaa00700 1 --2- 172.21.15.119:0/824400405 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f5ca006c680 0x7f5ca006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:23.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.195+0000 7f5cbaa00700 1 --2- 172.21.15.119:0/824400405 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f5ca006c680 0x7f5ca006eb30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc107800 tx=0x7f5cac00b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:23.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.199+0000 7f5cb3400700 1 -- 172.21.15.119:0/824400405 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5ca805acc0 con 0x7f5cbc1079a0 2024-05-27T09:24:23.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.425+0000 7f5cc1600700 1 -- 172.21.15.119:0/824400405 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f5cbc1a2280 con 0x7f5cbc1079a0 2024-05-27T09:24:23.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.426+0000 7f5cb3400700 1 -- 172.21.15.119:0/824400405 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7f5ca805a850 con 0x7f5cbc1079a0 2024-05-27T09:24:23.427 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:24:23.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.429+0000 7f5cc1600700 1 -- 172.21.15.119:0/824400405 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f5ca006c680 msgr2=0x7f5ca006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:23.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.429+0000 7f5cc1600700 1 --2- 172.21.15.119:0/824400405 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f5ca006c680 0x7f5ca006eb30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc107800 tx=0x7f5cac00b410 comp rx=0 tx=0).stop 2024-05-27T09:24:23.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.429+0000 7f5cc1600700 1 -- 172.21.15.119:0/824400405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cbc1079a0 msgr2=0x7f5cbc19c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:23.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.429+0000 7f5cc1600700 1 --2- 172.21.15.119:0/824400405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cbc1079a0 0x7f5cbc19c4e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8006010 tx=0x7f5ca800bd70 comp rx=0 tx=0).stop 2024-05-27T09:24:23.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.429+0000 7f5cc1600700 1 -- 172.21.15.119:0/824400405 shutdown_connections 2024-05-27T09:24:23.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.429+0000 7f5cc1600700 1 --2- 172.21.15.119:0/824400405 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f5ca006c680 0x7f5ca006eb30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:23.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.429+0000 7f5cc1600700 1 --2- 172.21.15.119:0/824400405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc1067a0 0x7f5cbc19bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:23.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.429+0000 7f5cc1600700 1 --2- 172.21.15.119:0/824400405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cbc1079a0 0x7f5cbc19c4e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:23.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.429+0000 7f5cc1600700 1 -- 172.21.15.119:0/824400405 >> 172.21.15.119:0/824400405 conn(0x7f5cbc075910 msgr2=0x7f5cbc10abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:23.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.429+0000 7f5cc1600700 1 -- 172.21.15.119:0/824400405 shutdown_connections 2024-05-27T09:24:23.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:23.429+0000 7f5cc1600700 1 -- 172.21.15.119:0/824400405 wait complete. 2024-05-27T09:24:23.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:23 smithi138 ceph-mon[34885]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:23.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:23 smithi138 ceph-mon[34885]: osdmap e40: 6 total, 6 up, 6 in 2024-05-27T09:24:23.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:23 smithi119 ceph-mon[29082]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-05-27T09:24:23.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:23 smithi119 ceph-mon[29082]: osdmap e40: 6 total, 6 up, 6 in 2024-05-27T09:24:24.319 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1716801861,"num_in_osds":6,"osd_in_since":1716801835,"num_remapped_pgs":0} 2024-05-27T09:24:24.319 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T09:24:24.388 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:24 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/824400405' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-27T09:24:24.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:24 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/824400405' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-27T09:24:24.710 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:24:25.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:25 smithi119 ceph-mon[29082]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:25.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:25 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:24:25.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:25 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:24:25.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:25 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:24:25.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:25 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:24:25.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:25 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:24:25.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:25 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:24:25.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:25 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:24:25.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:25 smithi138 ceph-mon[34885]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:25.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:25 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:24:25.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:25 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:24:25.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:25 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:24:25.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:25 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:24:25.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:25 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:24:25.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:25 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:24:25.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:25 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:24:26.615 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:26 smithi119 ceph-mon[29082]: Detected new or changed devices on smithi138 2024-05-27T09:24:26.615 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:26 smithi119 ceph-mon[29082]: Adjusting osd_memory_target on smithi138 to 4971M 2024-05-27T09:24:26.615 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:26 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:24:26.616 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:26 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:24:26.616 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:26 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:24:26.616 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:26 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:24:26.616 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:26 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:24:26.616 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:26 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:24:26.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:26 smithi138 ceph-mon[34885]: Detected new or changed devices on smithi138 2024-05-27T09:24:26.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:26 smithi138 ceph-mon[34885]: Adjusting osd_memory_target on smithi138 to 4971M 2024-05-27T09:24:26.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:26 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:24:26.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:26 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:24:26.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:26 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:24:26.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:26 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:24:26.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:26 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:24:26.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:26 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:24:26.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.781+0000 7f696a400700 1 -- 172.21.15.119:0/906314831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69641003b0 msgr2=0x7f6964100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:26.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.781+0000 7f696a400700 1 --2- 172.21.15.119:0/906314831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69641003b0 0x7f6964100780 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6954009910 tx=0x7f6954009c20 comp rx=0 tx=0).stop 2024-05-27T09:24:26.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.781+0000 7f696a400700 1 -- 172.21.15.119:0/906314831 shutdown_connections 2024-05-27T09:24:26.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.781+0000 7f696a400700 1 --2- 172.21.15.119:0/906314831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6964100d50 0x7f6964104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:26.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.781+0000 7f696a400700 1 --2- 172.21.15.119:0/906314831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69641003b0 0x7f6964100780 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:26.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.781+0000 7f696a400700 1 -- 172.21.15.119:0/906314831 >> 172.21.15.119:0/906314831 conn(0x7f69640fbc30 msgr2=0x7f69640fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:26.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.781+0000 7f696a400700 1 -- 172.21.15.119:0/906314831 shutdown_connections 2024-05-27T09:24:26.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.781+0000 7f696a400700 1 -- 172.21.15.119:0/906314831 wait complete. 2024-05-27T09:24:26.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.782+0000 7f696a400700 1 Processor -- start 2024-05-27T09:24:26.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.782+0000 7f696a400700 1 -- start start 2024-05-27T09:24:26.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.782+0000 7f696a400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69641003b0 0x7f6964073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:26.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.783+0000 7f696a400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6964100d50 0x7f6964073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:26.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.783+0000 7f696a400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6964073ad0 con 0x7f6964100d50 2024-05-27T09:24:26.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.783+0000 7f696a400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6964073c10 con 0x7f69641003b0 2024-05-27T09:24:26.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.783+0000 7f695aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6964100d50 0x7f6964073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:26.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.783+0000 7f695aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6964100d50 0x7f6964073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37950/0 (socket says 172.21.15.119:37950) 2024-05-27T09:24:26.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.783+0000 7f695aa00700 1 -- 172.21.15.119:0/744848772 learned_addr learned my addr 172.21.15.119:0/744848772 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:24:26.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.783+0000 7f6963400700 1 --2- 172.21.15.119:0/744848772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69641003b0 0x7f6964073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:26.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.783+0000 7f695aa00700 1 -- 172.21.15.119:0/744848772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69641003b0 msgr2=0x7f6964073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:26.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.783+0000 7f695aa00700 1 --2- 172.21.15.119:0/744848772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69641003b0 0x7f6964073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:26.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.783+0000 7f695aa00700 1 -- 172.21.15.119:0/744848772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69540095f0 con 0x7f6964100d50 2024-05-27T09:24:26.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.784+0000 7f695aa00700 1 --2- 172.21.15.119:0/744848772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6964100d50 0x7f6964073590 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f694c00d900 tx=0x7f694c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:26.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.784+0000 7f6960c00700 1 -- 172.21.15.119:0/744848772 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f694c009900 con 0x7f6964100d50 2024-05-27T09:24:26.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.784+0000 7f6960c00700 1 -- 172.21.15.119:0/744848772 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f694c010460 con 0x7f6964100d50 2024-05-27T09:24:26.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.784+0000 7f696a400700 1 -- 172.21.15.119:0/744848772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69641a1ee0 con 0x7f6964100d50 2024-05-27T09:24:26.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.784+0000 7f6960c00700 1 -- 172.21.15.119:0/744848772 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f694c00f5f0 con 0x7f6964100d50 2024-05-27T09:24:26.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.784+0000 7f696a400700 1 -- 172.21.15.119:0/744848772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69641a22f0 con 0x7f6964100d50 2024-05-27T09:24:26.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.785+0000 7f696a400700 1 -- 172.21.15.119:0/744848772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69641086c0 con 0x7f6964100d50 2024-05-27T09:24:26.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.786+0000 7f6960c00700 1 -- 172.21.15.119:0/744848772 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f694c0105d0 con 0x7f6964100d50 2024-05-27T09:24:26.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.787+0000 7f6960c00700 1 --2- 172.21.15.119:0/744848772 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f694406c680 0x7f694406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:26.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.787+0000 7f6960c00700 1 -- 172.21.15.119:0/744848772 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f694c08b410 con 0x7f6964100d50 2024-05-27T09:24:26.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.787+0000 7f6963400700 1 --2- 172.21.15.119:0/744848772 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f694406c680 0x7f694406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:26.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.788+0000 7f6963400700 1 --2- 172.21.15.119:0/744848772 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f694406c680 0x7f694406eb30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6954006010 tx=0x7f695401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:26.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:26.792+0000 7f6960c00700 1 -- 172.21.15.119:0/744848772 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f694c055d90 con 0x7f6964100d50 2024-05-27T09:24:27.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:27.006+0000 7f696a400700 1 -- 172.21.15.119:0/744848772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f696404ea50 con 0x7f6964100d50 2024-05-27T09:24:27.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:27.008+0000 7f6960c00700 1 -- 172.21.15.119:0/744848772 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11353 (secure 0 0 0) 0x7f694c0593b0 con 0x7f6964100d50 2024-05-27T09:24:27.009 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:24:27.009 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":40,"fsid":"b966b06e-1c09-11ef-bc9b-c7b262605968","created":"2024-05-27T09:16:58.005971+0000","modified":"2024-05-27T09:24:22.333656+0000","last_up_change":"2024-05-27T09:24:21.320070+0000","last_in_change":"2024-05-27T09:23:55.998448+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T09:22:34.880683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e3f78266-8de9-4e4f-85d6-7df2efa3b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":448778921},{"type":"v1","addr":"172.21.15.119:6803","nonce":448778921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":448778921},{"type":"v1","addr":"172.21.15.119:6805","nonce":448778921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":448778921},{"type":"v1","addr":"172.21.15.119:6809","nonce":448778921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":448778921},{"type":"v1","addr":"172.21.15.119:6807","nonce":448778921}]},"public_addr":"172.21.15.119:6803/448778921","cluster_addr":"172.21.15.119:6805/448778921","heartbeat_back_addr":"172.21.15.119:6809/448778921","heartbeat_front_addr":"172.21.15.119:6807/448778921","state":["exists","up"]},{"osd":1,"uuid":"361a94d8-da7f-4672-9de0-7a2789453e64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2092077577},{"type":"v1","addr":"172.21.15.119:6811","nonce":2092077577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2092077577},{"type":"v1","addr":"172.21.15.119:6813","nonce":2092077577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2092077577},{"type":"v1","addr":"172.21.15.119:6817","nonce":2092077577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2092077577},{"type":"v1","addr":"172.21.15.119:6815","nonce":2092077577}]},"public_addr":"172.21.15.119:6811/2092077577","cluster_addr":"172.21.15.119:6813/2092077577","heartbeat_back_addr":"172.21.15.119:6817/2092077577","heartbeat_front_addr":"172.21.15.119:6815/2092077577","state":["exists","up"]},{"osd":2,"uuid":"4c405f37-b811-464f-a5f7-29e9e2a9c6e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":3743656430},{"type":"v1","addr":"172.21.15.119:6819","nonce":3743656430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":3743656430},{"type":"v1","addr":"172.21.15.119:6821","nonce":3743656430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3743656430},{"type":"v1","addr":"172.21.15.119:6825","nonce":3743656430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":3743656430},{"type":"v1","addr":"172.21.15.119:6823","nonce":3743656430}]},"public_addr":"172.21.15.119:6819/3743656430","cluster_addr":"172.21.15.119:6821/3743656430","heartbeat_back_addr":"172.21.15.119:6825/3743656430","heartbeat_front_addr":"172.21.15.119:6823/3743656430","state":["exists","up"]},{"osd":3,"uuid":"a332cce6-4548-4c4d-97c1-705e7ffb2790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":29238828},{"type":"v1","addr":"172.21.15.138:6801","nonce":29238828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":29238828},{"type":"v1","addr":"172.21.15.138:6803","nonce":29238828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":29238828},{"type":"v1","addr":"172.21.15.138:6807","nonce":29238828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":29238828},{"type":"v1","addr":"172.21.15.138:6805","nonce":29238828}]},"public_addr":"172.21.15.138:6801/29238828","cluster_addr":"172.21.15.138:6803/29238828","heartbeat_back_addr":"172.21.15.138:6807/29238828","heartbeat_front_addr":"172.21.15.138:6805/29238828","state":["exists","up"]},{"osd":4,"uuid":"fd95d23f-b654-4cb6-819a-27c4c992f695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3397624307},{"type":"v1","addr":"172.21.15.138:6809","nonce":3397624307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3397624307},{"type":"v1","addr":"172.21.15.138:6811","nonce":3397624307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3397624307},{"type":"v1","addr":"172.21.15.138:6815","nonce":3397624307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3397624307},{"type":"v1","addr":"172.21.15.138:6813","nonce":3397624307}]},"public_addr":"172.21.15.138:6809/3397624307","cluster_addr":"172.21.15.138:6811/3397624307","heartbeat_back_addr":"172.21.15.138:6815/3397624307","heartbeat_front_addr":"172.21.15.138:6813/3397624307","state":["exists","up"]},{"osd":5,"uuid":"172a12ad-ca75-486d-a379-2e202fe6b6bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":886004363},{"type":"v1","addr":"172.21.15.138:6817","nonce":886004363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":886004363},{"type":"v1","addr":"172.21.15.138:6819","nonce":886004363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":886004363},{"type":"v1","addr":"172.21.15.138:6823","nonce":886004363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":886004363},{"type":"v1","addr":"172.21.15.138:6821","nonce":886004363}]},"public_addr":"172.21.15.138:6817/886004363","cluster_addr":"172.21.15.138:6819/886004363","heartbeat_back_addr":"172.21.15.138:6823/886004363","heartbeat_front_addr":"172.21.15.138:6821/886004363","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:21:20.705339+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:21:55.547936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:22:32.504686+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:23:06.765636+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:23:43.517892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:24:19.579981+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/2308226979":"2024-05-28T09:19:36.161573+0000","172.21.15.119:0/2281796205":"2024-05-28T09:19:36.161573+0000","172.21.15.119:0/987818704":"2024-05-28T09:18:11.825050+0000","172.21.15.119:6800/2118907438":"2024-05-28T09:19:36.161573+0000","172.21.15.119:6800/1553098156":"2024-05-28T09:18:11.825050+0000","172.21.15.119:0/4074206116":"2024-05-28T09:18:11.825050+0000","172.21.15.119:6800/1743331737":"2024-05-28T09:17:30.956826+0000","172.21.15.119:6801/2118907438":"2024-05-28T09:19:36.161573+0000","172.21.15.119:0/2176658945":"2024-05-28T09:17:30.956826+0000","172.21.15.119:0/2970229585":"2024-05-28T09:18:11.825050+0000","172.21.15.119:0/1310642907":"2024-05-28T09:19:36.161573+0000","172.21.15.119:6801/1743331737":"2024-05-28T09:17:30.956826+0000","172.21.15.119:0/3311320241":"2024-05-28T09:17:30.956826+0000","172.21.15.119:6801/1553098156":"2024-05-28T09:18:11.825050+0000","172.21.15.119:0/2367404262":"2024-05-28T09:17:30.956826+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T09:24:27.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:27.011+0000 7f696a400700 1 -- 172.21.15.119:0/744848772 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f694406c680 msgr2=0x7f694406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:27.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:27.011+0000 7f696a400700 1 --2- 172.21.15.119:0/744848772 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f694406c680 0x7f694406eb30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6954006010 tx=0x7f695401a040 comp rx=0 tx=0).stop 2024-05-27T09:24:27.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:27.011+0000 7f696a400700 1 -- 172.21.15.119:0/744848772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6964100d50 msgr2=0x7f6964073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:27.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:27.011+0000 7f696a400700 1 --2- 172.21.15.119:0/744848772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6964100d50 0x7f6964073590 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f694c00d900 tx=0x7f694c00dcc0 comp rx=0 tx=0).stop 2024-05-27T09:24:27.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:27.011+0000 7f696a400700 1 -- 172.21.15.119:0/744848772 shutdown_connections 2024-05-27T09:24:27.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:27.011+0000 7f696a400700 1 --2- 172.21.15.119:0/744848772 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f694406c680 0x7f694406eb30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:27.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:27.011+0000 7f696a400700 1 --2- 172.21.15.119:0/744848772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69641003b0 0x7f6964073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:27.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:27.011+0000 7f696a400700 1 --2- 172.21.15.119:0/744848772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6964100d50 0x7f6964073590 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:27.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:27.011+0000 7f696a400700 1 -- 172.21.15.119:0/744848772 >> 172.21.15.119:0/744848772 conn(0x7f69640fbc30 msgr2=0x7f69640fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:27.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:27.011+0000 7f696a400700 1 -- 172.21.15.119:0/744848772 shutdown_connections 2024-05-27T09:24:27.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:27.012+0000 7f696a400700 1 -- 172.21.15.119:0/744848772 wait complete. 2024-05-27T09:24:27.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:27 smithi138 ceph-mon[34885]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:27.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:27 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/744848772' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T09:24:27.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:27 smithi119 ceph-mon[29082]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:27.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:27 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/744848772' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T09:24:27.828 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T09:22:34.880683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T09:24:27.828 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-27T09:24:28.249 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:24:29.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:29 smithi119 ceph-mon[29082]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:29.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:29 smithi138 ceph-mon[34885]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:30.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.294+0000 7fedf8c00700 1 -- 172.21.15.119:0/3437052893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedf4102fd0 msgr2=0x7fedf4106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:30.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.294+0000 7fedf8c00700 1 --2- 172.21.15.119:0/3437052893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedf4102fd0 0x7fedf4106fa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fede0009910 tx=0x7fede0009c20 comp rx=0 tx=0).stop 2024-05-27T09:24:30.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.294+0000 7fedf8c00700 1 -- 172.21.15.119:0/3437052893 shutdown_connections 2024-05-27T09:24:30.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.294+0000 7fedf8c00700 1 --2- 172.21.15.119:0/3437052893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedf4102fd0 0x7fedf4106fa0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:30.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.294+0000 7fedf8c00700 1 --2- 172.21.15.119:0/3437052893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedf4102630 0x7fedf4102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:30.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.294+0000 7fedf8c00700 1 -- 172.21.15.119:0/3437052893 >> 172.21.15.119:0/3437052893 conn(0x7fedf40fdef0 msgr2=0x7fedf4100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:30.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.294+0000 7fedf8c00700 1 -- 172.21.15.119:0/3437052893 shutdown_connections 2024-05-27T09:24:30.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.294+0000 7fedf8c00700 1 -- 172.21.15.119:0/3437052893 wait complete. 2024-05-27T09:24:30.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.295+0000 7fedf8c00700 1 Processor -- start 2024-05-27T09:24:30.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.295+0000 7fedf8c00700 1 -- start start 2024-05-27T09:24:30.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.296+0000 7fedf8c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedf4102630 0x7fedf4197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:30.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.296+0000 7fedf8c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedf4102fd0 0x7fedf4198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:30.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.296+0000 7fedf8c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf4198c00 con 0x7fedf4102630 2024-05-27T09:24:30.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.296+0000 7fedf8c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf419c990 con 0x7fedf4102fd0 2024-05-27T09:24:30.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.296+0000 7fedf1e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedf4102630 0x7fedf4197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:30.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.296+0000 7fedf1e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedf4102630 0x7fedf4197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45388/0 (socket says 172.21.15.119:45388) 2024-05-27T09:24:30.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.296+0000 7fedf1e00700 1 -- 172.21.15.119:0/3423550174 learned_addr learned my addr 172.21.15.119:0/3423550174 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:24:30.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.296+0000 7fedf1400700 1 --2- 172.21.15.119:0/3423550174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedf4102fd0 0x7fedf4198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:30.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.297+0000 7fedf1e00700 1 -- 172.21.15.119:0/3423550174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedf4102fd0 msgr2=0x7fedf4198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:30.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.297+0000 7fedf1e00700 1 --2- 172.21.15.119:0/3423550174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedf4102fd0 0x7fedf4198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:30.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.297+0000 7fedf1e00700 1 -- 172.21.15.119:0/3423550174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede00095f0 con 0x7fedf4102630 2024-05-27T09:24:30.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.297+0000 7fedf1e00700 1 --2- 172.21.15.119:0/3423550174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedf4102630 0x7fedf4197fe0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fede400c8a0 tx=0x7fede400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:30.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.297+0000 7fedeaa00700 1 -- 172.21.15.119:0/3423550174 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fede40078e0 con 0x7fedf4102630 2024-05-27T09:24:30.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.297+0000 7fedeaa00700 1 -- 172.21.15.119:0/3423550174 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fede400f450 con 0x7fedf4102630 2024-05-27T09:24:30.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.297+0000 7fedeaa00700 1 -- 172.21.15.119:0/3423550174 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fede400e5e0 con 0x7fedf4102630 2024-05-27T09:24:30.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.298+0000 7fedf8c00700 1 -- 172.21.15.119:0/3423550174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf419cc70 con 0x7fedf4102630 2024-05-27T09:24:30.300 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.298+0000 7fedf8c00700 1 -- 172.21.15.119:0/3423550174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf419d170 con 0x7fedf4102630 2024-05-27T09:24:30.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.299+0000 7fedf8c00700 1 -- 172.21.15.119:0/3423550174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf404ea50 con 0x7fedf4102630 2024-05-27T09:24:30.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.300+0000 7fedeaa00700 1 -- 172.21.15.119:0/3423550174 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fede400f5c0 con 0x7fedf4102630 2024-05-27T09:24:30.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.301+0000 7fedeaa00700 1 --2- 172.21.15.119:0/3423550174 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fedd806c680 0x7fedd806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:30.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.301+0000 7fedeaa00700 1 -- 172.21.15.119:0/3423550174 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fede408b5e0 con 0x7fedf4102630 2024-05-27T09:24:30.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.301+0000 7fedf1400700 1 --2- 172.21.15.119:0/3423550174 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fedd806c680 0x7fedd806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:30.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.302+0000 7fedf1400700 1 --2- 172.21.15.119:0/3423550174 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fedd806c680 0x7fedd806eb30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fedf4199600 tx=0x7fede001a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:30.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.305+0000 7fedeaa00700 1 -- 172.21.15.119:0/3423550174 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fede4055f60 con 0x7fedf4102630 2024-05-27T09:24:30.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.518+0000 7fedf8c00700 1 -- 172.21.15.119:0/3423550174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fedf4199340 con 0x7fedf4102630 2024-05-27T09:24:30.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.519+0000 7fedeaa00700 1 -- 172.21.15.119:0/3423550174 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7fedf4199340 con 0x7fedf4102630 2024-05-27T09:24:30.520 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2024-05-27T09:24:30.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.522+0000 7fedf8c00700 1 -- 172.21.15.119:0/3423550174 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fedd806c680 msgr2=0x7fedd806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:30.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.522+0000 7fedf8c00700 1 --2- 172.21.15.119:0/3423550174 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fedd806c680 0x7fedd806eb30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fedf4199600 tx=0x7fede001a040 comp rx=0 tx=0).stop 2024-05-27T09:24:30.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.522+0000 7fedf8c00700 1 -- 172.21.15.119:0/3423550174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedf4102630 msgr2=0x7fedf4197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:30.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.522+0000 7fedf8c00700 1 --2- 172.21.15.119:0/3423550174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedf4102630 0x7fedf4197fe0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fede400c8a0 tx=0x7fede400cbb0 comp rx=0 tx=0).stop 2024-05-27T09:24:30.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.522+0000 7fedf8c00700 1 -- 172.21.15.119:0/3423550174 shutdown_connections 2024-05-27T09:24:30.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.522+0000 7fedf8c00700 1 --2- 172.21.15.119:0/3423550174 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fedd806c680 0x7fedd806eb30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:30.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.522+0000 7fedf8c00700 1 --2- 172.21.15.119:0/3423550174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedf4102630 0x7fedf4197fe0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:30.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.522+0000 7fedf8c00700 1 --2- 172.21.15.119:0/3423550174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedf4102fd0 0x7fedf4198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:30.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.522+0000 7fedf8c00700 1 -- 172.21.15.119:0/3423550174 >> 172.21.15.119:0/3423550174 conn(0x7fedf40fdef0 msgr2=0x7fedf40ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:30.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.522+0000 7fedf8c00700 1 -- 172.21.15.119:0/3423550174 shutdown_connections 2024-05-27T09:24:30.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:30.522+0000 7fedf8c00700 1 -- 172.21.15.119:0/3423550174 wait complete. 2024-05-27T09:24:31.267 INFO:tasks.cephadm:Setting up client nodes... 2024-05-27T09:24:31.267 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-27T09:24:31.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:31 smithi119 ceph-mon[29082]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:31.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:31 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/3423550174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T09:24:31.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:31 smithi138 ceph-mon[34885]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:31.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:31 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/3423550174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T09:24:31.664 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:24:33.466 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:33 smithi119 ceph-mon[29082]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:33.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:33 smithi138 ceph-mon[34885]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:33.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.644+0000 7f3a2ee00700 1 -- 172.21.15.119:0/513737431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a28103540 msgr2=0x7f3a28105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:33.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.644+0000 7f3a2ee00700 1 --2- 172.21.15.119:0/513737431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a28103540 0x7f3a28105920 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009910 tx=0x7f3a10009c20 comp rx=0 tx=0).stop 2024-05-27T09:24:33.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.644+0000 7f3a2ee00700 1 -- 172.21.15.119:0/513737431 shutdown_connections 2024-05-27T09:24:33.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.644+0000 7f3a2ee00700 1 --2- 172.21.15.119:0/513737431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a28103540 0x7f3a28105920 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:33.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.644+0000 7f3a2ee00700 1 --2- 172.21.15.119:0/513737431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a28100c20 0x7f3a28103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:33.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.644+0000 7f3a2ee00700 1 -- 172.21.15.119:0/513737431 >> 172.21.15.119:0/513737431 conn(0x7f3a280fa5e0 msgr2=0x7f3a280fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:33.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.644+0000 7f3a2ee00700 1 -- 172.21.15.119:0/513737431 shutdown_connections 2024-05-27T09:24:33.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.644+0000 7f3a2ee00700 1 -- 172.21.15.119:0/513737431 wait complete. 2024-05-27T09:24:33.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.645+0000 7f3a2ee00700 1 Processor -- start 2024-05-27T09:24:33.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.646+0000 7f3a2ee00700 1 -- start start 2024-05-27T09:24:33.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.646+0000 7f3a2ee00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a28100c20 0x7f3a28197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:33.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.646+0000 7f3a2ee00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a28103540 0x7f3a28198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:33.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.646+0000 7f3a2ee00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a28198620 con 0x7f3a28103540 2024-05-27T09:24:33.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.646+0000 7f3a2ee00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a28198760 con 0x7f3a28100c20 2024-05-27T09:24:33.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.646+0000 7f3a1f400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a28103540 0x7f3a28198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:33.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.646+0000 7f3a1f400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a28103540 0x7f3a28198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45408/0 (socket says 172.21.15.119:45408) 2024-05-27T09:24:33.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.646+0000 7f3a1f400700 1 -- 172.21.15.119:0/581523964 learned_addr learned my addr 172.21.15.119:0/581523964 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:24:33.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.647+0000 7f3a27e00700 1 --2- 172.21.15.119:0/581523964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a28100c20 0x7f3a28197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:33.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.647+0000 7f3a1f400700 1 -- 172.21.15.119:0/581523964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a28100c20 msgr2=0x7f3a28197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:33.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.647+0000 7f3a1f400700 1 --2- 172.21.15.119:0/581523964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a28100c20 0x7f3a28197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:33.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.647+0000 7f3a1f400700 1 -- 172.21.15.119:0/581523964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a100095f0 con 0x7f3a28103540 2024-05-27T09:24:33.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.647+0000 7f3a1f400700 1 --2- 172.21.15.119:0/581523964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a28103540 0x7f3a28198090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3a10005b40 tx=0x7f3a1000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:33.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.647+0000 7f3a25600700 1 -- 172.21.15.119:0/581523964 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a1001d070 con 0x7f3a28103540 2024-05-27T09:24:33.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.647+0000 7f3a25600700 1 -- 172.21.15.119:0/581523964 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3a10022470 con 0x7f3a28103540 2024-05-27T09:24:33.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.647+0000 7f3a2ee00700 1 -- 172.21.15.119:0/581523964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2819d1c0 con 0x7f3a28103540 2024-05-27T09:24:33.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.647+0000 7f3a25600700 1 -- 172.21.15.119:0/581523964 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a1000f690 con 0x7f3a28103540 2024-05-27T09:24:33.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.648+0000 7f3a2ee00700 1 -- 172.21.15.119:0/581523964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2819d680 con 0x7f3a28103540 2024-05-27T09:24:33.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.649+0000 7f3a2ee00700 1 -- 172.21.15.119:0/581523964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a280fc1c0 con 0x7f3a28103540 2024-05-27T09:24:33.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.650+0000 7f3a25600700 1 -- 172.21.15.119:0/581523964 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f3a100225e0 con 0x7f3a28103540 2024-05-27T09:24:33.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.650+0000 7f3a25600700 1 --2- 172.21.15.119:0/581523964 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f3a0c06c680 0x7f3a0c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:33.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.651+0000 7f3a25600700 1 -- 172.21.15.119:0/581523964 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f3a1008cd50 con 0x7f3a28103540 2024-05-27T09:24:33.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.651+0000 7f3a27e00700 1 --2- 172.21.15.119:0/581523964 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f3a0c06c680 0x7f3a0c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:33.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.651+0000 7f3a27e00700 1 --2- 172.21.15.119:0/581523964 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f3a0c06c680 0x7f3a0c06eb30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3a18005ea0 tx=0x7f3a18005e30 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:33.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.656+0000 7f3a25600700 1 -- 172.21.15.119:0/581523964 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3a1005b260 con 0x7f3a28103540 2024-05-27T09:24:33.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.951+0000 7f3a2ee00700 1 -- 172.21.15.119:0/581523964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f3a2802cc30 con 0x7f3a28103540 2024-05-27T09:24:33.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.959+0000 7f3a25600700 1 -- 172.21.15.119:0/581523964 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7f3a100270f0 con 0x7f3a28103540 2024-05-27T09:24:33.961 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2024-05-27T09:24:33.961 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBRUVRmh2vXOBAA/EiHHSKt91KlltO0VitNGg== 2024-05-27T09:24:33.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.962+0000 7f3a2ee00700 1 -- 172.21.15.119:0/581523964 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f3a0c06c680 msgr2=0x7f3a0c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:33.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.962+0000 7f3a2ee00700 1 --2- 172.21.15.119:0/581523964 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f3a0c06c680 0x7f3a0c06eb30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3a18005ea0 tx=0x7f3a18005e30 comp rx=0 tx=0).stop 2024-05-27T09:24:33.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.962+0000 7f3a2ee00700 1 -- 172.21.15.119:0/581523964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a28103540 msgr2=0x7f3a28198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:33.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.962+0000 7f3a2ee00700 1 --2- 172.21.15.119:0/581523964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a28103540 0x7f3a28198090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3a10005b40 tx=0x7f3a1000be20 comp rx=0 tx=0).stop 2024-05-27T09:24:33.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.962+0000 7f3a2ee00700 1 -- 172.21.15.119:0/581523964 shutdown_connections 2024-05-27T09:24:33.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.962+0000 7f3a2ee00700 1 --2- 172.21.15.119:0/581523964 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f3a0c06c680 0x7f3a0c06eb30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:33.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.962+0000 7f3a2ee00700 1 --2- 172.21.15.119:0/581523964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a28100c20 0x7f3a28197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:33.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.962+0000 7f3a2ee00700 1 --2- 172.21.15.119:0/581523964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a28103540 0x7f3a28198090 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:33.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.962+0000 7f3a2ee00700 1 -- 172.21.15.119:0/581523964 >> 172.21.15.119:0/581523964 conn(0x7f3a280fa5e0 msgr2=0x7f3a280ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:33.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.962+0000 7f3a2ee00700 1 -- 172.21.15.119:0/581523964 shutdown_connections 2024-05-27T09:24:33.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:33.962+0000 7f3a2ee00700 1 -- 172.21.15.119:0/581523964 wait complete. 2024-05-27T09:24:34.619 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:34 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/581523964' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-27T09:24:34.620 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:34 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/581523964' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-27T09:24:34.687 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-27T09:24:34.687 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-27T09:24:34.687 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-27T09:24:34.812 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-27T09:24:34.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:34 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/581523964' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-27T09:24:34.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:34 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/581523964' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-27T09:24:35.233 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi138/config 2024-05-27T09:24:35.490 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:35 smithi138 ceph-mon[34885]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:35.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:35 smithi119 ceph-mon[29082]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:37.055 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:36 smithi138 ceph-mon[34885]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:37.055 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:36 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:24:37.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:36 smithi119 ceph-mon[29082]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:37.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:36 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:24:37.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.583+0000 7fdc2f400700 1 -- 172.21.15.138:0/3525417467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc28102380 msgr2=0x7fdc28102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:37.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.583+0000 7fdc2f400700 1 --2- 172.21.15.138:0/3525417467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc28102380 0x7fdc28102790 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009910 tx=0x7fdc1c009c20 comp rx=0 tx=0).stop 2024-05-27T09:24:37.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.583+0000 7fdc2f400700 1 -- 172.21.15.138:0/3525417467 shutdown_connections 2024-05-27T09:24:37.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.583+0000 7fdc2f400700 1 --2- 172.21.15.138:0/3525417467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc28103580 0x7fdc281039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:37.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.583+0000 7fdc2f400700 1 --2- 172.21.15.138:0/3525417467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc28102380 0x7fdc28102790 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:37.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.583+0000 7fdc2f400700 1 -- 172.21.15.138:0/3525417467 >> 172.21.15.138:0/3525417467 conn(0x7fdc280fd930 msgr2=0x7fdc280ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:37.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.583+0000 7fdc2f400700 1 -- 172.21.15.138:0/3525417467 shutdown_connections 2024-05-27T09:24:37.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.583+0000 7fdc2f400700 1 -- 172.21.15.138:0/3525417467 wait complete. 2024-05-27T09:24:37.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.585+0000 7fdc2f400700 1 Processor -- start 2024-05-27T09:24:37.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.585+0000 7fdc2f400700 1 -- start start 2024-05-27T09:24:37.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.585+0000 7fdc2f400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc28102380 0x7fdc28197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:37.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.585+0000 7fdc2f400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc28103580 0x7fdc28198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:37.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.585+0000 7fdc2f400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc281987a0 con 0x7fdc28103580 2024-05-27T09:24:37.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.585+0000 7fdc2f400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2819d1b0 con 0x7fdc28102380 2024-05-27T09:24:37.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.585+0000 7fdc2ca00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc28102380 0x7fdc28197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:37.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.585+0000 7fdc2ca00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc28102380 0x7fdc28197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60846/0 (socket says 172.21.15.138:60846) 2024-05-27T09:24:37.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.585+0000 7fdc2ca00700 1 -- 172.21.15.138:0/3774755784 learned_addr learned my addr 172.21.15.138:0/3774755784 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-27T09:24:37.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.586+0000 7fdc27e00700 1 --2- 172.21.15.138:0/3774755784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc28103580 0x7fdc28198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:37.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.586+0000 7fdc2ca00700 1 -- 172.21.15.138:0/3774755784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc28103580 msgr2=0x7fdc28198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:37.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.586+0000 7fdc2ca00700 1 --2- 172.21.15.138:0/3774755784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc28103580 0x7fdc28198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:37.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.586+0000 7fdc2ca00700 1 -- 172.21.15.138:0/3774755784 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c0095f0 con 0x7fdc28102380 2024-05-27T09:24:37.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.586+0000 7fdc27e00700 1 --2- 172.21.15.138:0/3774755784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc28103580 0x7fdc28198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:24:37.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.586+0000 7fdc2ca00700 1 --2- 172.21.15.138:0/3774755784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc28102380 0x7fdc28197c40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00bda0 tx=0x7fdc1c00be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:37.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.587+0000 7fdc25600700 1 -- 172.21.15.138:0/3774755784 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc1c01d070 con 0x7fdc28102380 2024-05-27T09:24:37.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.587+0000 7fdc2f400700 1 -- 172.21.15.138:0/3774755784 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2819d350 con 0x7fdc28102380 2024-05-27T09:24:37.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.587+0000 7fdc25600700 1 -- 172.21.15.138:0/3774755784 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdc1c004c50 con 0x7fdc28102380 2024-05-27T09:24:37.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.587+0000 7fdc25600700 1 -- 172.21.15.138:0/3774755784 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc1c00f690 con 0x7fdc28102380 2024-05-27T09:24:37.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.587+0000 7fdc2f400700 1 -- 172.21.15.138:0/3774755784 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2819d890 con 0x7fdc28102380 2024-05-27T09:24:37.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.588+0000 7fdc2f400700 1 -- 172.21.15.138:0/3774755784 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc2810af50 con 0x7fdc28102380 2024-05-27T09:24:37.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.589+0000 7fdc25600700 1 -- 172.21.15.138:0/3774755784 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fdc1c00f900 con 0x7fdc28102380 2024-05-27T09:24:37.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.590+0000 7fdc25600700 1 --2- 172.21.15.138:0/3774755784 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fdc1006c680 0x7fdc1006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:37.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.590+0000 7fdc25600700 1 -- 172.21.15.138:0/3774755784 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fdc1c08cd30 con 0x7fdc28102380 2024-05-27T09:24:37.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.591+0000 7fdc27e00700 1 --2- 172.21.15.138:0/3774755784 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fdc1006c680 0x7fdc1006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:37.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.592+0000 7fdc27e00700 1 --2- 172.21.15.138:0/3774755784 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fdc1006c680 0x7fdc1006eb30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdc14007900 tx=0x7fdc14008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:37.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.595+0000 7fdc25600700 1 -- 172.21.15.138:0/3774755784 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fdc1c057750 con 0x7fdc28102380 2024-05-27T09:24:37.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.892+0000 7fdc2f400700 1 -- 172.21.15.138:0/3774755784 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fdc2819df60 con 0x7fdc28102380 2024-05-27T09:24:37.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.903+0000 7fdc25600700 1 -- 172.21.15.138:0/3774755784 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7fdc1c05ad70 con 0x7fdc28102380 2024-05-27T09:24:37.904 INFO:teuthology.orchestra.run.smithi138.stdout:[client.1] 2024-05-27T09:24:37.904 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQBVUVRm8w5VNRAAt6iDY+9+Pa0/TfPxn3ifQg== 2024-05-27T09:24:37.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.906+0000 7fdc2f400700 1 -- 172.21.15.138:0/3774755784 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fdc1006c680 msgr2=0x7fdc1006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:37.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.906+0000 7fdc2f400700 1 --2- 172.21.15.138:0/3774755784 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fdc1006c680 0x7fdc1006eb30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdc14007900 tx=0x7fdc14008040 comp rx=0 tx=0).stop 2024-05-27T09:24:37.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.906+0000 7fdc2f400700 1 -- 172.21.15.138:0/3774755784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc28102380 msgr2=0x7fdc28197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:37.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.906+0000 7fdc2f400700 1 --2- 172.21.15.138:0/3774755784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc28102380 0x7fdc28197c40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00bda0 tx=0x7fdc1c00be80 comp rx=0 tx=0).stop 2024-05-27T09:24:37.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.906+0000 7fdc2f400700 1 -- 172.21.15.138:0/3774755784 shutdown_connections 2024-05-27T09:24:37.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.906+0000 7fdc2f400700 1 --2- 172.21.15.138:0/3774755784 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fdc1006c680 0x7fdc1006eb30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:37.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.906+0000 7fdc2f400700 1 --2- 172.21.15.138:0/3774755784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc28102380 0x7fdc28197c40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:37.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.906+0000 7fdc2f400700 1 --2- 172.21.15.138:0/3774755784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc28103580 0x7fdc28198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:37.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.906+0000 7fdc2f400700 1 -- 172.21.15.138:0/3774755784 >> 172.21.15.138:0/3774755784 conn(0x7fdc280fd930 msgr2=0x7fdc281067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:37.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.907+0000 7fdc2f400700 1 -- 172.21.15.138:0/3774755784 shutdown_connections 2024-05-27T09:24:37.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:24:37.907+0000 7fdc2f400700 1 -- 172.21.15.138:0/3774755784 wait complete. 2024-05-27T09:24:38.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:37 smithi119 ceph-mon[29082]: from='client.? 172.21.15.138:0/3774755784' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-27T09:24:38.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:37 smithi119 ceph-mon[29082]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-27T09:24:38.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:37 smithi119 ceph-mon[29082]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-27T09:24:38.286 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:37 smithi138 ceph-mon[34885]: from='client.? 172.21.15.138:0/3774755784' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-27T09:24:38.286 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:37 smithi138 ceph-mon[34885]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-27T09:24:38.287 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:37 smithi138 ceph-mon[34885]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-27T09:24:38.820 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-27T09:24:38.821 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-05-27T09:24:38.821 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-27T09:24:38.952 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-27T09:24:38.953 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-27T09:24:38.953 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-05-27T09:24:39.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:38 smithi119 ceph-mon[29082]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:39.356 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:24:39.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:38 smithi138 ceph-mon[34885]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:41.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.342+0000 7fbb77800700 1 -- 172.21.15.119:0/472743886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb701003d0 msgr2=0x7fbb701007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:41.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.342+0000 7fbb77800700 1 --2- 172.21.15.119:0/472743886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb701003d0 0x7fbb701007a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbb64009910 tx=0x7fbb64009c20 comp rx=0 tx=0).stop 2024-05-27T09:24:41.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.342+0000 7fbb77800700 1 -- 172.21.15.119:0/472743886 shutdown_connections 2024-05-27T09:24:41.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.342+0000 7fbb77800700 1 --2- 172.21.15.119:0/472743886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb70100d70 0x7fbb70104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:41.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.342+0000 7fbb77800700 1 --2- 172.21.15.119:0/472743886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb701003d0 0x7fbb701007a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:41.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.342+0000 7fbb77800700 1 -- 172.21.15.119:0/472743886 >> 172.21.15.119:0/472743886 conn(0x7fbb700fbc50 msgr2=0x7fbb700fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:41.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.343+0000 7fbb77800700 1 -- 172.21.15.119:0/472743886 shutdown_connections 2024-05-27T09:24:41.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.343+0000 7fbb77800700 1 -- 172.21.15.119:0/472743886 wait complete. 2024-05-27T09:24:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.344+0000 7fbb77800700 1 Processor -- start 2024-05-27T09:24:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.344+0000 7fbb77800700 1 -- start start 2024-05-27T09:24:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.344+0000 7fbb77800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb701003d0 0x7fbb70197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.344+0000 7fbb77800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb70100d70 0x7fbb701984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.344+0000 7fbb77800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb70198bd0 con 0x7fbb70100d70 2024-05-27T09:24:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.344+0000 7fbb77800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7019c960 con 0x7fbb701003d0 2024-05-27T09:24:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.345+0000 7fbb6fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb70100d70 0x7fbb701984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.345+0000 7fbb6fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb70100d70 0x7fbb701984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55850/0 (socket says 172.21.15.119:55850) 2024-05-27T09:24:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.345+0000 7fbb6fe00700 1 -- 172.21.15.119:0/3689994424 learned_addr learned my addr 172.21.15.119:0/3689994424 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:24:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.345+0000 7fbb74e00700 1 --2- 172.21.15.119:0/3689994424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb701003d0 0x7fbb70197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.345+0000 7fbb6fe00700 1 -- 172.21.15.119:0/3689994424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb701003d0 msgr2=0x7fbb70197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.345+0000 7fbb6fe00700 1 --2- 172.21.15.119:0/3689994424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb701003d0 0x7fbb70197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.345+0000 7fbb6fe00700 1 -- 172.21.15.119:0/3689994424 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb640095f0 con 0x7fbb70100d70 2024-05-27T09:24:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.345+0000 7fbb6fe00700 1 --2- 172.21.15.119:0/3689994424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb70100d70 0x7fbb701984f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00e8f0 tx=0x7fbb5c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.346+0000 7fbb6d600700 1 -- 172.21.15.119:0/3689994424 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb5c009790 con 0x7fbb70100d70 2024-05-27T09:24:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.346+0000 7fbb6d600700 1 -- 172.21.15.119:0/3689994424 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbb5c00c810 con 0x7fbb70100d70 2024-05-27T09:24:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.346+0000 7fbb77800700 1 -- 172.21.15.119:0/3689994424 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7019cc40 con 0x7fbb70100d70 2024-05-27T09:24:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.346+0000 7fbb6d600700 1 -- 172.21.15.119:0/3689994424 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb5c0055b0 con 0x7fbb70100d70 2024-05-27T09:24:41.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.346+0000 7fbb77800700 1 -- 172.21.15.119:0/3689994424 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7019d190 con 0x7fbb70100d70 2024-05-27T09:24:41.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.347+0000 7fbb77800700 1 -- 172.21.15.119:0/3689994424 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb701044c0 con 0x7fbb70100d70 2024-05-27T09:24:41.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.348+0000 7fbb6d600700 1 -- 172.21.15.119:0/3689994424 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fbb5c00c980 con 0x7fbb70100d70 2024-05-27T09:24:41.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.349+0000 7fbb6d600700 1 --2- 172.21.15.119:0/3689994424 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbb5806c680 0x7fbb5806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:41.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.349+0000 7fbb6d600700 1 -- 172.21.15.119:0/3689994424 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbb5c014070 con 0x7fbb70100d70 2024-05-27T09:24:41.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.349+0000 7fbb74e00700 1 --2- 172.21.15.119:0/3689994424 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbb5806c680 0x7fbb5806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:41.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.350+0000 7fbb74e00700 1 --2- 172.21.15.119:0/3689994424 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbb5806c680 0x7fbb5806eb30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbb64005b40 tx=0x7fbb64005a90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:41.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.354+0000 7fbb6d600700 1 -- 172.21.15.119:0/3689994424 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbb5c056310 con 0x7fbb70100d70 2024-05-27T09:24:41.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:41 smithi119 ceph-mon[29082]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:41.562 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:41 smithi138 ceph-mon[34885]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:41.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.637+0000 7fbb77800700 1 -- 172.21.15.119:0/3689994424 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fbb7004ea50 con 0x7fbb70100d70 2024-05-27T09:24:41.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.646+0000 7fbb6d600700 1 -- 172.21.15.119:0/3689994424 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+172862 (secure 0 0 0) 0x7fbb5c059930 con 0x7fbb70100d70 2024-05-27T09:24:41.652 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:24:41.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.651+0000 7fbb77800700 1 -- 172.21.15.119:0/3689994424 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbb5806c680 msgr2=0x7fbb5806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:41.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.651+0000 7fbb77800700 1 --2- 172.21.15.119:0/3689994424 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbb5806c680 0x7fbb5806eb30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbb64005b40 tx=0x7fbb64005a90 comp rx=0 tx=0).stop 2024-05-27T09:24:41.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.651+0000 7fbb77800700 1 -- 172.21.15.119:0/3689994424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb70100d70 msgr2=0x7fbb701984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:41.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.651+0000 7fbb77800700 1 --2- 172.21.15.119:0/3689994424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb70100d70 0x7fbb701984f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00e8f0 tx=0x7fbb5c00ec00 comp rx=0 tx=0).stop 2024-05-27T09:24:41.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.651+0000 7fbb77800700 1 -- 172.21.15.119:0/3689994424 shutdown_connections 2024-05-27T09:24:41.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.651+0000 7fbb77800700 1 --2- 172.21.15.119:0/3689994424 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbb5806c680 0x7fbb5806eb30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:41.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.651+0000 7fbb77800700 1 --2- 172.21.15.119:0/3689994424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb701003d0 0x7fbb70197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:41.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.651+0000 7fbb77800700 1 --2- 172.21.15.119:0/3689994424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb70100d70 0x7fbb701984f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:41.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.651+0000 7fbb77800700 1 -- 172.21.15.119:0/3689994424 >> 172.21.15.119:0/3689994424 conn(0x7fbb700fbc50 msgr2=0x7fbb700fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:41.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.651+0000 7fbb77800700 1 -- 172.21.15.119:0/3689994424 shutdown_connections 2024-05-27T09:24:41.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:41.651+0000 7fbb77800700 1 -- 172.21.15.119:0/3689994424 wait complete. 2024-05-27T09:24:42.408 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":19,"active_gid":14213,"active_name":"smithi119.ncwmog","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2191862652},{"type":"v1","addr":"172.21.15.119:6801","nonce":2191862652}]},"active_addr":"172.21.15.119:6801/2191862652","active_change":"2024-05-27T09:19:36.161834+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14238,"name":"smithi138.dxsuqf","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.119:8443/","prometheus":"http://172.21.15.119:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":1141668291}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":893995738}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":2275270800}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":553584352}]}]} 2024-05-27T09:24:42.445 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-27T09:24:42.445 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-27T09:24:42.446 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T09:24:42.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:42 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/3689994424' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-27T09:24:42.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:42 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/3689994424' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-27T09:24:42.850 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:24:43.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:43 smithi138 ceph-mon[34885]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:43.644 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:43 smithi119 ceph-mon[29082]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:44.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.758+0000 7f1c39400700 1 -- 172.21.15.119:0/1839943541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c3410c010 msgr2=0x7f1c34106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:44.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.758+0000 7f1c39400700 1 --2- 172.21.15.119:0/1839943541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c3410c010 0x7f1c34106b80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009910 tx=0x7f1c20009c20 comp rx=0 tx=0).stop 2024-05-27T09:24:44.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.758+0000 7f1c39400700 1 -- 172.21.15.119:0/1839943541 shutdown_connections 2024-05-27T09:24:44.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.758+0000 7f1c39400700 1 --2- 172.21.15.119:0/1839943541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c3410c010 0x7f1c34106b80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:44.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.758+0000 7f1c39400700 1 --2- 172.21.15.119:0/1839943541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c3410b700 0x7f1c3410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:44.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.758+0000 7f1c39400700 1 -- 172.21.15.119:0/1839943541 >> 172.21.15.119:0/1839943541 conn(0x7f1c34076000 msgr2=0x7f1c34078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:44.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.758+0000 7f1c39400700 1 -- 172.21.15.119:0/1839943541 shutdown_connections 2024-05-27T09:24:44.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.758+0000 7f1c39400700 1 -- 172.21.15.119:0/1839943541 wait complete. 2024-05-27T09:24:44.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.759+0000 7f1c39400700 1 Processor -- start 2024-05-27T09:24:44.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.760+0000 7f1c39400700 1 -- start start 2024-05-27T09:24:44.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.760+0000 7f1c39400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c3410b700 0x7f1c3419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:44.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.760+0000 7f1c39400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c3410c010 0x7f1c3419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:44.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.760+0000 7f1c39400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3419ce70 con 0x7f1c3410c010 2024-05-27T09:24:44.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.760+0000 7f1c39400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3419cfb0 con 0x7f1c3410b700 2024-05-27T09:24:44.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.760+0000 7f1c31e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c3410c010 0x7f1c3419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:44.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.760+0000 7f1c31e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c3410c010 0x7f1c3419c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55868/0 (socket says 172.21.15.119:55868) 2024-05-27T09:24:44.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.760+0000 7f1c31e00700 1 -- 172.21.15.119:0/873940436 learned_addr learned my addr 172.21.15.119:0/873940436 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:24:44.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.761+0000 7f1c32800700 1 --2- 172.21.15.119:0/873940436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c3410b700 0x7f1c3419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:44.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.761+0000 7f1c31e00700 1 -- 172.21.15.119:0/873940436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c3410b700 msgr2=0x7f1c3419c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:44.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.761+0000 7f1c31e00700 1 --2- 172.21.15.119:0/873940436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c3410b700 0x7f1c3419c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:44.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.761+0000 7f1c31e00700 1 -- 172.21.15.119:0/873940436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c200095f0 con 0x7f1c3410c010 2024-05-27T09:24:44.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.761+0000 7f1c32800700 1 --2- 172.21.15.119:0/873940436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c3410b700 0x7f1c3419c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:24:44.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.761+0000 7f1c31e00700 1 --2- 172.21.15.119:0/873940436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c3410c010 0x7f1c3419c820 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1c2000bda0 tx=0x7f1c2000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:44.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.761+0000 7f1c2b400700 1 -- 172.21.15.119:0/873940436 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c2001d070 con 0x7f1c3410c010 2024-05-27T09:24:44.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.761+0000 7f1c2b400700 1 -- 172.21.15.119:0/873940436 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1c20022470 con 0x7f1c3410c010 2024-05-27T09:24:44.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.761+0000 7f1c39400700 1 -- 172.21.15.119:0/873940436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c341a0da0 con 0x7f1c3410c010 2024-05-27T09:24:44.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.762+0000 7f1c39400700 1 -- 172.21.15.119:0/873940436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c341a1290 con 0x7f1c3410c010 2024-05-27T09:24:44.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.762+0000 7f1c2b400700 1 -- 172.21.15.119:0/873940436 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c2000f690 con 0x7f1c3410c010 2024-05-27T09:24:44.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.763+0000 7f1c39400700 1 -- 172.21.15.119:0/873940436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c14005320 con 0x7f1c3410c010 2024-05-27T09:24:44.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.764+0000 7f1c2b400700 1 -- 172.21.15.119:0/873940436 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f1c20004c50 con 0x7f1c3410c010 2024-05-27T09:24:44.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.764+0000 7f1c2b400700 1 --2- 172.21.15.119:0/873940436 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1c1806c680 0x7f1c1806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:44.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.765+0000 7f1c2b400700 1 -- 172.21.15.119:0/873940436 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1c2008ca20 con 0x7f1c3410c010 2024-05-27T09:24:44.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.765+0000 7f1c32800700 1 --2- 172.21.15.119:0/873940436 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1c1806c680 0x7f1c1806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:44.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.766+0000 7f1c32800700 1 --2- 172.21.15.119:0/873940436 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1c1806c680 0x7f1c1806eb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1c24005fd0 tx=0x7f1c24005e20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:44.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.769+0000 7f1c2b400700 1 -- 172.21.15.119:0/873940436 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1c200574f0 con 0x7f1c3410c010 2024-05-27T09:24:44.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.983+0000 7f1c39400700 1 -- 172.21.15.119:0/873940436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c14005190 con 0x7f1c3410c010 2024-05-27T09:24:44.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.986+0000 7f1c2b400700 1 -- 172.21.15.119:0/873940436 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11353 (secure 0 0 0) 0x7f1c20027090 con 0x7f1c3410c010 2024-05-27T09:24:44.987 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:24:44.987 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":40,"fsid":"b966b06e-1c09-11ef-bc9b-c7b262605968","created":"2024-05-27T09:16:58.005971+0000","modified":"2024-05-27T09:24:22.333656+0000","last_up_change":"2024-05-27T09:24:21.320070+0000","last_in_change":"2024-05-27T09:23:55.998448+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T09:22:34.880683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e3f78266-8de9-4e4f-85d6-7df2efa3b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":448778921},{"type":"v1","addr":"172.21.15.119:6803","nonce":448778921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":448778921},{"type":"v1","addr":"172.21.15.119:6805","nonce":448778921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":448778921},{"type":"v1","addr":"172.21.15.119:6809","nonce":448778921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":448778921},{"type":"v1","addr":"172.21.15.119:6807","nonce":448778921}]},"public_addr":"172.21.15.119:6803/448778921","cluster_addr":"172.21.15.119:6805/448778921","heartbeat_back_addr":"172.21.15.119:6809/448778921","heartbeat_front_addr":"172.21.15.119:6807/448778921","state":["exists","up"]},{"osd":1,"uuid":"361a94d8-da7f-4672-9de0-7a2789453e64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2092077577},{"type":"v1","addr":"172.21.15.119:6811","nonce":2092077577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2092077577},{"type":"v1","addr":"172.21.15.119:6813","nonce":2092077577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2092077577},{"type":"v1","addr":"172.21.15.119:6817","nonce":2092077577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2092077577},{"type":"v1","addr":"172.21.15.119:6815","nonce":2092077577}]},"public_addr":"172.21.15.119:6811/2092077577","cluster_addr":"172.21.15.119:6813/2092077577","heartbeat_back_addr":"172.21.15.119:6817/2092077577","heartbeat_front_addr":"172.21.15.119:6815/2092077577","state":["exists","up"]},{"osd":2,"uuid":"4c405f37-b811-464f-a5f7-29e9e2a9c6e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":3743656430},{"type":"v1","addr":"172.21.15.119:6819","nonce":3743656430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":3743656430},{"type":"v1","addr":"172.21.15.119:6821","nonce":3743656430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3743656430},{"type":"v1","addr":"172.21.15.119:6825","nonce":3743656430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":3743656430},{"type":"v1","addr":"172.21.15.119:6823","nonce":3743656430}]},"public_addr":"172.21.15.119:6819/3743656430","cluster_addr":"172.21.15.119:6821/3743656430","heartbeat_back_addr":"172.21.15.119:6825/3743656430","heartbeat_front_addr":"172.21.15.119:6823/3743656430","state":["exists","up"]},{"osd":3,"uuid":"a332cce6-4548-4c4d-97c1-705e7ffb2790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":29238828},{"type":"v1","addr":"172.21.15.138:6801","nonce":29238828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":29238828},{"type":"v1","addr":"172.21.15.138:6803","nonce":29238828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":29238828},{"type":"v1","addr":"172.21.15.138:6807","nonce":29238828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":29238828},{"type":"v1","addr":"172.21.15.138:6805","nonce":29238828}]},"public_addr":"172.21.15.138:6801/29238828","cluster_addr":"172.21.15.138:6803/29238828","heartbeat_back_addr":"172.21.15.138:6807/29238828","heartbeat_front_addr":"172.21.15.138:6805/29238828","state":["exists","up"]},{"osd":4,"uuid":"fd95d23f-b654-4cb6-819a-27c4c992f695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3397624307},{"type":"v1","addr":"172.21.15.138:6809","nonce":3397624307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3397624307},{"type":"v1","addr":"172.21.15.138:6811","nonce":3397624307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3397624307},{"type":"v1","addr":"172.21.15.138:6815","nonce":3397624307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3397624307},{"type":"v1","addr":"172.21.15.138:6813","nonce":3397624307}]},"public_addr":"172.21.15.138:6809/3397624307","cluster_addr":"172.21.15.138:6811/3397624307","heartbeat_back_addr":"172.21.15.138:6815/3397624307","heartbeat_front_addr":"172.21.15.138:6813/3397624307","state":["exists","up"]},{"osd":5,"uuid":"172a12ad-ca75-486d-a379-2e202fe6b6bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":886004363},{"type":"v1","addr":"172.21.15.138:6817","nonce":886004363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":886004363},{"type":"v1","addr":"172.21.15.138:6819","nonce":886004363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":886004363},{"type":"v1","addr":"172.21.15.138:6823","nonce":886004363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":886004363},{"type":"v1","addr":"172.21.15.138:6821","nonce":886004363}]},"public_addr":"172.21.15.138:6817/886004363","cluster_addr":"172.21.15.138:6819/886004363","heartbeat_back_addr":"172.21.15.138:6823/886004363","heartbeat_front_addr":"172.21.15.138:6821/886004363","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:21:20.705339+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:21:55.547936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:22:32.504686+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:23:06.765636+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:23:43.517892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:24:19.579981+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/2308226979":"2024-05-28T09:19:36.161573+0000","172.21.15.119:0/2281796205":"2024-05-28T09:19:36.161573+0000","172.21.15.119:0/987818704":"2024-05-28T09:18:11.825050+0000","172.21.15.119:6800/2118907438":"2024-05-28T09:19:36.161573+0000","172.21.15.119:6800/1553098156":"2024-05-28T09:18:11.825050+0000","172.21.15.119:0/4074206116":"2024-05-28T09:18:11.825050+0000","172.21.15.119:6800/1743331737":"2024-05-28T09:17:30.956826+0000","172.21.15.119:6801/2118907438":"2024-05-28T09:19:36.161573+0000","172.21.15.119:0/2176658945":"2024-05-28T09:17:30.956826+0000","172.21.15.119:0/2970229585":"2024-05-28T09:18:11.825050+0000","172.21.15.119:0/1310642907":"2024-05-28T09:19:36.161573+0000","172.21.15.119:6801/1743331737":"2024-05-28T09:17:30.956826+0000","172.21.15.119:0/3311320241":"2024-05-28T09:17:30.956826+0000","172.21.15.119:6801/1553098156":"2024-05-28T09:18:11.825050+0000","172.21.15.119:0/2367404262":"2024-05-28T09:17:30.956826+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T09:24:44.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.989+0000 7f1c39400700 1 -- 172.21.15.119:0/873940436 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1c1806c680 msgr2=0x7f1c1806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:44.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.989+0000 7f1c39400700 1 --2- 172.21.15.119:0/873940436 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1c1806c680 0x7f1c1806eb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1c24005fd0 tx=0x7f1c24005e20 comp rx=0 tx=0).stop 2024-05-27T09:24:44.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.989+0000 7f1c39400700 1 -- 172.21.15.119:0/873940436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c3410c010 msgr2=0x7f1c3419c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:44.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.989+0000 7f1c39400700 1 --2- 172.21.15.119:0/873940436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c3410c010 0x7f1c3419c820 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1c2000bda0 tx=0x7f1c2000be80 comp rx=0 tx=0).stop 2024-05-27T09:24:44.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.989+0000 7f1c39400700 1 -- 172.21.15.119:0/873940436 shutdown_connections 2024-05-27T09:24:44.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.989+0000 7f1c39400700 1 --2- 172.21.15.119:0/873940436 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1c1806c680 0x7f1c1806eb30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:44.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.989+0000 7f1c39400700 1 --2- 172.21.15.119:0/873940436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c3410b700 0x7f1c3419c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:44.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.989+0000 7f1c39400700 1 --2- 172.21.15.119:0/873940436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c3410c010 0x7f1c3419c820 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:44.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.989+0000 7f1c39400700 1 -- 172.21.15.119:0/873940436 >> 172.21.15.119:0/873940436 conn(0x7f1c34076000 msgr2=0x7f1c34076940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:44.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.990+0000 7f1c39400700 1 -- 172.21.15.119:0/873940436 shutdown_connections 2024-05-27T09:24:44.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:44.990+0000 7f1c39400700 1 -- 172.21.15.119:0/873940436 wait complete. 2024-05-27T09:24:45.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:45 smithi138 ceph-mon[34885]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:45.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:45 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/873940436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T09:24:45.654 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:45 smithi119 ceph-mon[29082]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:45.654 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:45 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/873940436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T09:24:45.708 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-27T09:24:45.708 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T09:24:46.119 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:24:47.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:46 smithi119 ceph-mon[29082]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:47.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:46 smithi138 ceph-mon[34885]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:48.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.119+0000 7efbf8200700 1 -- 172.21.15.119:0/219014237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf0068630 msgr2=0x7efbf0068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:48.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.119+0000 7efbf8200700 1 --2- 172.21.15.119:0/219014237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf0068630 0x7efbf0068a00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efbe4009910 tx=0x7efbe4009c20 comp rx=0 tx=0).stop 2024-05-27T09:24:48.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.120+0000 7efbf8200700 1 -- 172.21.15.119:0/219014237 shutdown_connections 2024-05-27T09:24:48.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.120+0000 7efbf8200700 1 --2- 172.21.15.119:0/219014237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf0068fd0 0x7efbf0104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:48.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.120+0000 7efbf8200700 1 --2- 172.21.15.119:0/219014237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf0068630 0x7efbf0068a00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:48.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.120+0000 7efbf8200700 1 -- 172.21.15.119:0/219014237 >> 172.21.15.119:0/219014237 conn(0x7efbf0074f50 msgr2=0x7efbf0075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:48.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.120+0000 7efbf8200700 1 -- 172.21.15.119:0/219014237 shutdown_connections 2024-05-27T09:24:48.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.120+0000 7efbf8200700 1 -- 172.21.15.119:0/219014237 wait complete. 2024-05-27T09:24:48.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.121+0000 7efbf8200700 1 Processor -- start 2024-05-27T09:24:48.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.121+0000 7efbf8200700 1 -- start start 2024-05-27T09:24:48.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.121+0000 7efbf8200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf0068630 0x7efbf0197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:48.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.121+0000 7efbf8200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf0068fd0 0x7efbf0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:48.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.121+0000 7efbf8200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf0198b50 con 0x7efbf0068fd0 2024-05-27T09:24:48.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.121+0000 7efbf8200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf019c8e0 con 0x7efbf0068630 2024-05-27T09:24:48.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.122+0000 7efbf4e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf0068fd0 0x7efbf0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:48.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.122+0000 7efbf4e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf0068fd0 0x7efbf0198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55878/0 (socket says 172.21.15.119:55878) 2024-05-27T09:24:48.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.122+0000 7efbf4e00700 1 -- 172.21.15.119:0/1197503775 learned_addr learned my addr 172.21.15.119:0/1197503775 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:24:48.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.122+0000 7efbf5800700 1 --2- 172.21.15.119:0/1197503775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf0068630 0x7efbf0197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:48.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.122+0000 7efbf4e00700 1 -- 172.21.15.119:0/1197503775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf0068630 msgr2=0x7efbf0197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:48.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.122+0000 7efbf4e00700 1 --2- 172.21.15.119:0/1197503775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf0068630 0x7efbf0197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:48.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.122+0000 7efbf4e00700 1 -- 172.21.15.119:0/1197503775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbe40095f0 con 0x7efbf0068fd0 2024-05-27T09:24:48.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.122+0000 7efbf4e00700 1 --2- 172.21.15.119:0/1197503775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf0068fd0 0x7efbf0198470 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efbdc00c8f0 tx=0x7efbdc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:48.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.123+0000 7efbee000700 1 -- 172.21.15.119:0/1197503775 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbdc007a50 con 0x7efbf0068fd0 2024-05-27T09:24:48.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.123+0000 7efbee000700 1 -- 172.21.15.119:0/1197503775 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efbdc007bb0 con 0x7efbf0068fd0 2024-05-27T09:24:48.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.123+0000 7efbf8200700 1 -- 172.21.15.119:0/1197503775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf019cbc0 con 0x7efbf0068fd0 2024-05-27T09:24:48.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.123+0000 7efbee000700 1 -- 172.21.15.119:0/1197503775 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbdc018630 con 0x7efbf0068fd0 2024-05-27T09:24:48.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.123+0000 7efbf8200700 1 -- 172.21.15.119:0/1197503775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf019d110 con 0x7efbf0068fd0 2024-05-27T09:24:48.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.124+0000 7efbf8200700 1 -- 172.21.15.119:0/1197503775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf0104570 con 0x7efbf0068fd0 2024-05-27T09:24:48.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.125+0000 7efbee000700 1 -- 172.21.15.119:0/1197503775 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7efbdc007d20 con 0x7efbf0068fd0 2024-05-27T09:24:48.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.126+0000 7efbee000700 1 --2- 172.21.15.119:0/1197503775 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7efbd806c680 0x7efbd806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:48.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.126+0000 7efbee000700 1 -- 172.21.15.119:0/1197503775 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7efbdc08b550 con 0x7efbf0068fd0 2024-05-27T09:24:48.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.126+0000 7efbf5800700 1 --2- 172.21.15.119:0/1197503775 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7efbd806c680 0x7efbd806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:48.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.127+0000 7efbf5800700 1 --2- 172.21.15.119:0/1197503775 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7efbd806c680 0x7efbd806eb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efbe4006010 tx=0x7efbe401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:48.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.131+0000 7efbee000700 1 -- 172.21.15.119:0/1197503775 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7efbdc055f50 con 0x7efbf0068fd0 2024-05-27T09:24:48.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.350+0000 7efbf8200700 1 -- 172.21.15.119:0/1197503775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efbf004ea50 con 0x7efbf0068fd0 2024-05-27T09:24:48.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.352+0000 7efbee000700 1 -- 172.21.15.119:0/1197503775 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11353 (secure 0 0 0) 0x7efbdc059570 con 0x7efbf0068fd0 2024-05-27T09:24:48.353 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:24:48.353 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":40,"fsid":"b966b06e-1c09-11ef-bc9b-c7b262605968","created":"2024-05-27T09:16:58.005971+0000","modified":"2024-05-27T09:24:22.333656+0000","last_up_change":"2024-05-27T09:24:21.320070+0000","last_in_change":"2024-05-27T09:23:55.998448+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T09:22:34.880683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e3f78266-8de9-4e4f-85d6-7df2efa3b70e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":448778921},{"type":"v1","addr":"172.21.15.119:6803","nonce":448778921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":448778921},{"type":"v1","addr":"172.21.15.119:6805","nonce":448778921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":448778921},{"type":"v1","addr":"172.21.15.119:6809","nonce":448778921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":448778921},{"type":"v1","addr":"172.21.15.119:6807","nonce":448778921}]},"public_addr":"172.21.15.119:6803/448778921","cluster_addr":"172.21.15.119:6805/448778921","heartbeat_back_addr":"172.21.15.119:6809/448778921","heartbeat_front_addr":"172.21.15.119:6807/448778921","state":["exists","up"]},{"osd":1,"uuid":"361a94d8-da7f-4672-9de0-7a2789453e64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2092077577},{"type":"v1","addr":"172.21.15.119:6811","nonce":2092077577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2092077577},{"type":"v1","addr":"172.21.15.119:6813","nonce":2092077577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2092077577},{"type":"v1","addr":"172.21.15.119:6817","nonce":2092077577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2092077577},{"type":"v1","addr":"172.21.15.119:6815","nonce":2092077577}]},"public_addr":"172.21.15.119:6811/2092077577","cluster_addr":"172.21.15.119:6813/2092077577","heartbeat_back_addr":"172.21.15.119:6817/2092077577","heartbeat_front_addr":"172.21.15.119:6815/2092077577","state":["exists","up"]},{"osd":2,"uuid":"4c405f37-b811-464f-a5f7-29e9e2a9c6e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":3743656430},{"type":"v1","addr":"172.21.15.119:6819","nonce":3743656430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":3743656430},{"type":"v1","addr":"172.21.15.119:6821","nonce":3743656430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":3743656430},{"type":"v1","addr":"172.21.15.119:6825","nonce":3743656430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":3743656430},{"type":"v1","addr":"172.21.15.119:6823","nonce":3743656430}]},"public_addr":"172.21.15.119:6819/3743656430","cluster_addr":"172.21.15.119:6821/3743656430","heartbeat_back_addr":"172.21.15.119:6825/3743656430","heartbeat_front_addr":"172.21.15.119:6823/3743656430","state":["exists","up"]},{"osd":3,"uuid":"a332cce6-4548-4c4d-97c1-705e7ffb2790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":29238828},{"type":"v1","addr":"172.21.15.138:6801","nonce":29238828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":29238828},{"type":"v1","addr":"172.21.15.138:6803","nonce":29238828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":29238828},{"type":"v1","addr":"172.21.15.138:6807","nonce":29238828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":29238828},{"type":"v1","addr":"172.21.15.138:6805","nonce":29238828}]},"public_addr":"172.21.15.138:6801/29238828","cluster_addr":"172.21.15.138:6803/29238828","heartbeat_back_addr":"172.21.15.138:6807/29238828","heartbeat_front_addr":"172.21.15.138:6805/29238828","state":["exists","up"]},{"osd":4,"uuid":"fd95d23f-b654-4cb6-819a-27c4c992f695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3397624307},{"type":"v1","addr":"172.21.15.138:6809","nonce":3397624307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3397624307},{"type":"v1","addr":"172.21.15.138:6811","nonce":3397624307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3397624307},{"type":"v1","addr":"172.21.15.138:6815","nonce":3397624307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3397624307},{"type":"v1","addr":"172.21.15.138:6813","nonce":3397624307}]},"public_addr":"172.21.15.138:6809/3397624307","cluster_addr":"172.21.15.138:6811/3397624307","heartbeat_back_addr":"172.21.15.138:6815/3397624307","heartbeat_front_addr":"172.21.15.138:6813/3397624307","state":["exists","up"]},{"osd":5,"uuid":"172a12ad-ca75-486d-a379-2e202fe6b6bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":886004363},{"type":"v1","addr":"172.21.15.138:6817","nonce":886004363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":886004363},{"type":"v1","addr":"172.21.15.138:6819","nonce":886004363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":886004363},{"type":"v1","addr":"172.21.15.138:6823","nonce":886004363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":886004363},{"type":"v1","addr":"172.21.15.138:6821","nonce":886004363}]},"public_addr":"172.21.15.138:6817/886004363","cluster_addr":"172.21.15.138:6819/886004363","heartbeat_back_addr":"172.21.15.138:6823/886004363","heartbeat_front_addr":"172.21.15.138:6821/886004363","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:21:20.705339+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:21:55.547936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:22:32.504686+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:23:06.765636+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:23:43.517892+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T09:24:19.579981+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/2308226979":"2024-05-28T09:19:36.161573+0000","172.21.15.119:0/2281796205":"2024-05-28T09:19:36.161573+0000","172.21.15.119:0/987818704":"2024-05-28T09:18:11.825050+0000","172.21.15.119:6800/2118907438":"2024-05-28T09:19:36.161573+0000","172.21.15.119:6800/1553098156":"2024-05-28T09:18:11.825050+0000","172.21.15.119:0/4074206116":"2024-05-28T09:18:11.825050+0000","172.21.15.119:6800/1743331737":"2024-05-28T09:17:30.956826+0000","172.21.15.119:6801/2118907438":"2024-05-28T09:19:36.161573+0000","172.21.15.119:0/2176658945":"2024-05-28T09:17:30.956826+0000","172.21.15.119:0/2970229585":"2024-05-28T09:18:11.825050+0000","172.21.15.119:0/1310642907":"2024-05-28T09:19:36.161573+0000","172.21.15.119:6801/1743331737":"2024-05-28T09:17:30.956826+0000","172.21.15.119:0/3311320241":"2024-05-28T09:17:30.956826+0000","172.21.15.119:6801/1553098156":"2024-05-28T09:18:11.825050+0000","172.21.15.119:0/2367404262":"2024-05-28T09:17:30.956826+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T09:24:48.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.355+0000 7efbf8200700 1 -- 172.21.15.119:0/1197503775 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7efbd806c680 msgr2=0x7efbd806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:48.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.355+0000 7efbf8200700 1 --2- 172.21.15.119:0/1197503775 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7efbd806c680 0x7efbd806eb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efbe4006010 tx=0x7efbe401a040 comp rx=0 tx=0).stop 2024-05-27T09:24:48.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.355+0000 7efbf8200700 1 -- 172.21.15.119:0/1197503775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf0068fd0 msgr2=0x7efbf0198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:48.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.355+0000 7efbf8200700 1 --2- 172.21.15.119:0/1197503775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf0068fd0 0x7efbf0198470 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efbdc00c8f0 tx=0x7efbdc00ccb0 comp rx=0 tx=0).stop 2024-05-27T09:24:48.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.355+0000 7efbf8200700 1 -- 172.21.15.119:0/1197503775 shutdown_connections 2024-05-27T09:24:48.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.355+0000 7efbf8200700 1 --2- 172.21.15.119:0/1197503775 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7efbd806c680 0x7efbd806eb30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:48.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.355+0000 7efbf8200700 1 --2- 172.21.15.119:0/1197503775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf0068630 0x7efbf0197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:48.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.355+0000 7efbf8200700 1 --2- 172.21.15.119:0/1197503775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf0068fd0 0x7efbf0198470 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:48.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.355+0000 7efbf8200700 1 -- 172.21.15.119:0/1197503775 >> 172.21.15.119:0/1197503775 conn(0x7efbf0074f50 msgr2=0x7efbf00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:48.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.355+0000 7efbf8200700 1 -- 172.21.15.119:0/1197503775 shutdown_connections 2024-05-27T09:24:48.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:48.355+0000 7efbf8200700 1 -- 172.21.15.119:0/1197503775 wait complete. 2024-05-27T09:24:49.035 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-27T09:24:49.036 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-27T09:24:49.036 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-27T09:24:49.037 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-27T09:24:49.037 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-27T09:24:49.038 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-27T09:24:49.550 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:24:49.554 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:24:49.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:49 smithi138 ceph-mon[34885]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:49.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:49 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/1197503775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T09:24:49.644 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:24:49.712 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:49 smithi119 ceph-mon[29082]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:49.712 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:49 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/1197503775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T09:24:49.783 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:24:49.885 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:24:49.888 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:24:51.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:51 smithi119 ceph-mon[29082]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:51.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:51 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:24:51.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:51 smithi138 ceph-mon[34885]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:51.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:51 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:24:53.606 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:53 smithi119 ceph-mon[29082]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:53.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:53 smithi138 ceph-mon[34885]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:55.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.444+0000 7fa5e5c00700 1 -- 172.21.15.119:0/2561096689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5e0100d50 msgr2=0x7fa5e0104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:55.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.444+0000 7fa5e5c00700 1 --2- 172.21.15.119:0/2561096689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5e0100d50 0x7fa5e0104d20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009910 tx=0x7fa5c8009c20 comp rx=0 tx=0).stop 2024-05-27T09:24:55.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.444+0000 7fa5e5c00700 1 -- 172.21.15.119:0/2561096689 shutdown_connections 2024-05-27T09:24:55.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.444+0000 7fa5e5c00700 1 --2- 172.21.15.119:0/2561096689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5e0100d50 0x7fa5e0104d20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:55.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.444+0000 7fa5e5c00700 1 --2- 172.21.15.119:0/2561096689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5e01003b0 0x7fa5e0100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:55.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.444+0000 7fa5e5c00700 1 -- 172.21.15.119:0/2561096689 >> 172.21.15.119:0/2561096689 conn(0x7fa5e00fbc30 msgr2=0x7fa5e00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:55.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.444+0000 7fa5e5c00700 1 -- 172.21.15.119:0/2561096689 shutdown_connections 2024-05-27T09:24:55.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.444+0000 7fa5e5c00700 1 -- 172.21.15.119:0/2561096689 wait complete. 2024-05-27T09:24:55.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.445+0000 7fa5e5c00700 1 Processor -- start 2024-05-27T09:24:55.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.446+0000 7fa5e5c00700 1 -- start start 2024-05-27T09:24:55.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.446+0000 7fa5e5c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5e01003b0 0x7fa5e0197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:55.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.446+0000 7fa5e5c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5e0100d50 0x7fa5e0198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:55.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.446+0000 7fa5e5c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e0198c00 con 0x7fa5e01003b0 2024-05-27T09:24:55.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.446+0000 7fa5e5c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e019c990 con 0x7fa5e0100d50 2024-05-27T09:24:55.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.446+0000 7fa5dea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5e01003b0 0x7fa5e0197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:55.448 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:55 smithi119 ceph-mon[29082]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:55.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.446+0000 7fa5dea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5e01003b0 0x7fa5e0197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50126/0 (socket says 172.21.15.119:50126) 2024-05-27T09:24:55.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.446+0000 7fa5dea00700 1 -- 172.21.15.119:0/1615927674 learned_addr learned my addr 172.21.15.119:0/1615927674 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:24:55.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.447+0000 7fa5d7e00700 1 --2- 172.21.15.119:0/1615927674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5e0100d50 0x7fa5e0198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:55.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.447+0000 7fa5dea00700 1 -- 172.21.15.119:0/1615927674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5e0100d50 msgr2=0x7fa5e0198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:55.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.447+0000 7fa5dea00700 1 --2- 172.21.15.119:0/1615927674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5e0100d50 0x7fa5e0198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:55.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.447+0000 7fa5dea00700 1 -- 172.21.15.119:0/1615927674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c80095f0 con 0x7fa5e01003b0 2024-05-27T09:24:55.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.447+0000 7fa5dea00700 1 --2- 172.21.15.119:0/1615927674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5e01003b0 0x7fa5e0197fe0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000d8d0 tx=0x7fa5d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:55.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.448+0000 7fa5d7400700 1 -- 172.21.15.119:0/1615927674 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5d0009940 con 0x7fa5e01003b0 2024-05-27T09:24:55.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.448+0000 7fa5d7400700 1 -- 172.21.15.119:0/1615927674 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa5d0010460 con 0x7fa5e01003b0 2024-05-27T09:24:55.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.448+0000 7fa5d7400700 1 -- 172.21.15.119:0/1615927674 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5d000f5f0 con 0x7fa5e01003b0 2024-05-27T09:24:55.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.448+0000 7fa5e5c00700 1 -- 172.21.15.119:0/1615927674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e019cc70 con 0x7fa5e01003b0 2024-05-27T09:24:55.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.448+0000 7fa5e5c00700 1 -- 172.21.15.119:0/1615927674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e019d170 con 0x7fa5e01003b0 2024-05-27T09:24:55.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.449+0000 7fa5e5c00700 1 -- 172.21.15.119:0/1615927674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa5e010d520 con 0x7fa5e01003b0 2024-05-27T09:24:55.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.450+0000 7fa5d7400700 1 -- 172.21.15.119:0/1615927674 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fa5d00105d0 con 0x7fa5e01003b0 2024-05-27T09:24:55.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.451+0000 7fa5d7400700 1 --2- 172.21.15.119:0/1615927674 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fa5c406c680 0x7fa5c406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:55.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.451+0000 7fa5d7400700 1 -- 172.21.15.119:0/1615927674 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa5d008bae0 con 0x7fa5e01003b0 2024-05-27T09:24:55.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.451+0000 7fa5d7e00700 1 --2- 172.21.15.119:0/1615927674 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fa5c406c680 0x7fa5c406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:55.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.451+0000 7fa5d7400700 1 --2- 172.21.15.119:0/1615927674 >> [v2:172.21.15.119:6818/3743656430,v1:172.21.15.119:6819/3743656430] conn(0x7fa5c4072350 0x7fa5c4074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:55.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.451+0000 7fa5d7400700 1 -- 172.21.15.119:0/1615927674 --> [v2:172.21.15.119:6818/3743656430,v1:172.21.15.119:6819/3743656430] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa5c4074e10 con 0x7fa5c4072350 2024-05-27T09:24:55.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.451+0000 7fa5d7400700 1 -- 172.21.15.119:0/1615927674 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5d008bea0 con 0x7fa5e01003b0 2024-05-27T09:24:55.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.452+0000 7fa5df400700 1 --2- 172.21.15.119:0/1615927674 >> [v2:172.21.15.119:6818/3743656430,v1:172.21.15.119:6819/3743656430] conn(0x7fa5c4072350 0x7fa5c4074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:55.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.452+0000 7fa5d7e00700 1 --2- 172.21.15.119:0/1615927674 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fa5c406c680 0x7fa5c406eb30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0199600 tx=0x7fa5c801a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:55.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.452+0000 7fa5df400700 1 --2- 172.21.15.119:0/1615927674 >> [v2:172.21.15.119:6818/3743656430,v1:172.21.15.119:6819/3743656430] conn(0x7fa5c4072350 0x7fa5c4074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:55.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.453+0000 7fa5d7400700 1 -- 172.21.15.119:0/1615927674 <== osd.2 v2:172.21.15.119:6818/3743656430 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa5c4074e10 con 0x7fa5c4072350 2024-05-27T09:24:55.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.470+0000 7fa5e5c00700 1 -- 172.21.15.119:0/1615927674 --> [v2:172.21.15.119:6818/3743656430,v1:172.21.15.119:6819/3743656430] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa5e004ea50 con 0x7fa5c4072350 2024-05-27T09:24:55.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.471+0000 7fa5d7400700 1 -- 172.21.15.119:0/1615927674 <== osd.2 v2:172.21.15.119:6818/3743656430 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa5e004ea50 con 0x7fa5c4072350 2024-05-27T09:24:55.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.471+0000 7fa5e5c00700 1 -- 172.21.15.119:0/1615927674 >> [v2:172.21.15.119:6818/3743656430,v1:172.21.15.119:6819/3743656430] conn(0x7fa5c4072350 msgr2=0x7fa5c4074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:55.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.471+0000 7fa5e5c00700 1 --2- 172.21.15.119:0/1615927674 >> [v2:172.21.15.119:6818/3743656430,v1:172.21.15.119:6819/3743656430] conn(0x7fa5c4072350 0x7fa5c4074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:55.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.471+0000 7fa5e5c00700 1 -- 172.21.15.119:0/1615927674 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fa5c406c680 msgr2=0x7fa5c406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:55.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.471+0000 7fa5e5c00700 1 --2- 172.21.15.119:0/1615927674 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fa5c406c680 0x7fa5c406eb30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0199600 tx=0x7fa5c801a040 comp rx=0 tx=0).stop 2024-05-27T09:24:55.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.471+0000 7fa5e5c00700 1 -- 172.21.15.119:0/1615927674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5e01003b0 msgr2=0x7fa5e0197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:55.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.471+0000 7fa5e5c00700 1 --2- 172.21.15.119:0/1615927674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5e01003b0 0x7fa5e0197fe0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000d8d0 tx=0x7fa5d000dc90 comp rx=0 tx=0).stop 2024-05-27T09:24:55.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.471+0000 7fa5e5c00700 1 -- 172.21.15.119:0/1615927674 shutdown_connections 2024-05-27T09:24:55.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.471+0000 7fa5e5c00700 1 --2- 172.21.15.119:0/1615927674 >> [v2:172.21.15.119:6818/3743656430,v1:172.21.15.119:6819/3743656430] conn(0x7fa5c4072350 0x7fa5c4074760 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:55.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.471+0000 7fa5e5c00700 1 --2- 172.21.15.119:0/1615927674 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fa5c406c680 0x7fa5c406eb30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:55.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.471+0000 7fa5e5c00700 1 --2- 172.21.15.119:0/1615927674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5e01003b0 0x7fa5e0197fe0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:55.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.471+0000 7fa5e5c00700 1 --2- 172.21.15.119:0/1615927674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5e0100d50 0x7fa5e0198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:55.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.471+0000 7fa5e5c00700 1 -- 172.21.15.119:0/1615927674 >> 172.21.15.119:0/1615927674 conn(0x7fa5e00fbc30 msgr2=0x7fa5e00fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:55.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.472+0000 7fa5e5c00700 1 -- 172.21.15.119:0/1615927674 shutdown_connections 2024-05-27T09:24:55.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:55.472+0000 7fa5e5c00700 1 -- 172.21.15.119:0/1615927674 wait complete. 2024-05-27T09:24:55.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:55 smithi138 ceph-mon[34885]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:56.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.168+0000 7fc054e00700 1 -- 172.21.15.119:0/1826117547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0500ffc80 msgr2=0x7fc050103c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:56.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.168+0000 7fc054e00700 1 --2- 172.21.15.119:0/1826117547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0500ffc80 0x7fc050103c50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc038009910 tx=0x7fc038009c20 comp rx=0 tx=0).stop 2024-05-27T09:24:56.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.168+0000 7fc054e00700 1 -- 172.21.15.119:0/1826117547 shutdown_connections 2024-05-27T09:24:56.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.168+0000 7fc054e00700 1 --2- 172.21.15.119:0/1826117547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0500ffc80 0x7fc050103c50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:56.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.168+0000 7fc054e00700 1 --2- 172.21.15.119:0/1826117547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0500ff2e0 0x7fc0500ff6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:56.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.168+0000 7fc054e00700 1 -- 172.21.15.119:0/1826117547 >> 172.21.15.119:0/1826117547 conn(0x7fc0500fab80 msgr2=0x7fc0500fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:56.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.168+0000 7fc054e00700 1 -- 172.21.15.119:0/1826117547 shutdown_connections 2024-05-27T09:24:56.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.168+0000 7fc054e00700 1 -- 172.21.15.119:0/1826117547 wait complete. 2024-05-27T09:24:56.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.169+0000 7fc054e00700 1 Processor -- start 2024-05-27T09:24:56.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.169+0000 7fc054e00700 1 -- start start 2024-05-27T09:24:56.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.170+0000 7fc054e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0500ff2e0 0x7fc050197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:56.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.170+0000 7fc054e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0500ffc80 0x7fc050198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:56.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.170+0000 7fc054e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc050198c00 con 0x7fc0500ffc80 2024-05-27T09:24:56.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.170+0000 7fc054e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05019c990 con 0x7fc0500ff2e0 2024-05-27T09:24:56.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.170+0000 7fc045400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0500ffc80 0x7fc050198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:56.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.170+0000 7fc045400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0500ffc80 0x7fc050198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50144/0 (socket says 172.21.15.119:50144) 2024-05-27T09:24:56.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.170+0000 7fc045400700 1 -- 172.21.15.119:0/3520890947 learned_addr learned my addr 172.21.15.119:0/3520890947 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:24:56.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.170+0000 7fc04de00700 1 --2- 172.21.15.119:0/3520890947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0500ff2e0 0x7fc050197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:56.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.171+0000 7fc045400700 1 -- 172.21.15.119:0/3520890947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0500ff2e0 msgr2=0x7fc050197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:56.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.171+0000 7fc045400700 1 --2- 172.21.15.119:0/3520890947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0500ff2e0 0x7fc050197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:56.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.171+0000 7fc045400700 1 -- 172.21.15.119:0/3520890947 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0380095f0 con 0x7fc0500ffc80 2024-05-27T09:24:56.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.171+0000 7fc045400700 1 --2- 172.21.15.119:0/3520890947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0500ffc80 0x7fc050198520 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc03800bda0 tx=0x7fc03800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:56.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.171+0000 7fc047400700 1 -- 172.21.15.119:0/3520890947 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc03801d070 con 0x7fc0500ffc80 2024-05-27T09:24:56.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.171+0000 7fc054e00700 1 -- 172.21.15.119:0/3520890947 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc05019cc10 con 0x7fc0500ffc80 2024-05-27T09:24:56.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.171+0000 7fc047400700 1 -- 172.21.15.119:0/3520890947 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc038022470 con 0x7fc0500ffc80 2024-05-27T09:24:56.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.171+0000 7fc047400700 1 -- 172.21.15.119:0/3520890947 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc03800f690 con 0x7fc0500ffc80 2024-05-27T09:24:56.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.171+0000 7fc054e00700 1 -- 172.21.15.119:0/3520890947 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc05019d100 con 0x7fc0500ffc80 2024-05-27T09:24:56.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.172+0000 7fc054e00700 1 -- 172.21.15.119:0/3520890947 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc05010c470 con 0x7fc0500ffc80 2024-05-27T09:24:56.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.173+0000 7fc047400700 1 -- 172.21.15.119:0/3520890947 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fc038022ae0 con 0x7fc0500ffc80 2024-05-27T09:24:56.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.174+0000 7fc047400700 1 --2- 172.21.15.119:0/3520890947 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc03406c680 0x7fc03406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:56.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.174+0000 7fc047400700 1 -- 172.21.15.119:0/3520890947 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc03808ca00 con 0x7fc0500ffc80 2024-05-27T09:24:56.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.174+0000 7fc04de00700 1 --2- 172.21.15.119:0/3520890947 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc03406c680 0x7fc03406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:56.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.174+0000 7fc047400700 1 --2- 172.21.15.119:0/3520890947 >> [v2:172.21.15.119:6810/2092077577,v1:172.21.15.119:6811/2092077577] conn(0x7fc034072350 0x7fc034074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:56.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.175+0000 7fc047400700 1 -- 172.21.15.119:0/3520890947 --> [v2:172.21.15.119:6810/2092077577,v1:172.21.15.119:6811/2092077577] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc034074e10 con 0x7fc034072350 2024-05-27T09:24:56.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.175+0000 7fc047400700 1 -- 172.21.15.119:0/3520890947 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0380912a0 con 0x7fc0500ffc80 2024-05-27T09:24:56.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.175+0000 7fc04e800700 1 --2- 172.21.15.119:0/3520890947 >> [v2:172.21.15.119:6810/2092077577,v1:172.21.15.119:6811/2092077577] conn(0x7fc034072350 0x7fc034074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:56.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.175+0000 7fc04de00700 1 --2- 172.21.15.119:0/3520890947 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc03406c680 0x7fc03406eb30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc040005950 tx=0x7fc040005eb0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:56.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.175+0000 7fc04e800700 1 --2- 172.21.15.119:0/3520890947 >> [v2:172.21.15.119:6810/2092077577,v1:172.21.15.119:6811/2092077577] conn(0x7fc034072350 0x7fc034074760 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:56.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.176+0000 7fc047400700 1 -- 172.21.15.119:0/3520890947 <== osd.1 v2:172.21.15.119:6810/2092077577 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc034074e10 con 0x7fc034072350 2024-05-27T09:24:56.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.194+0000 7fc054e00700 1 -- 172.21.15.119:0/3520890947 --> [v2:172.21.15.119:6810/2092077577,v1:172.21.15.119:6811/2092077577] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc05004ea50 con 0x7fc034072350 2024-05-27T09:24:56.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.194+0000 7fc047400700 1 -- 172.21.15.119:0/3520890947 <== osd.1 v2:172.21.15.119:6810/2092077577 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc05004ea50 con 0x7fc034072350 2024-05-27T09:24:56.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.194+0000 7fc054e00700 1 -- 172.21.15.119:0/3520890947 >> [v2:172.21.15.119:6810/2092077577,v1:172.21.15.119:6811/2092077577] conn(0x7fc034072350 msgr2=0x7fc034074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:56.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.195+0000 7fc054e00700 1 --2- 172.21.15.119:0/3520890947 >> [v2:172.21.15.119:6810/2092077577,v1:172.21.15.119:6811/2092077577] conn(0x7fc034072350 0x7fc034074760 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:56.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.195+0000 7fc054e00700 1 -- 172.21.15.119:0/3520890947 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc03406c680 msgr2=0x7fc03406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:56.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.195+0000 7fc054e00700 1 --2- 172.21.15.119:0/3520890947 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc03406c680 0x7fc03406eb30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc040005950 tx=0x7fc040005eb0 comp rx=0 tx=0).stop 2024-05-27T09:24:56.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.195+0000 7fc054e00700 1 -- 172.21.15.119:0/3520890947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0500ffc80 msgr2=0x7fc050198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:56.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.195+0000 7fc054e00700 1 --2- 172.21.15.119:0/3520890947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0500ffc80 0x7fc050198520 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc03800bda0 tx=0x7fc03800be80 comp rx=0 tx=0).stop 2024-05-27T09:24:56.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.195+0000 7fc054e00700 1 -- 172.21.15.119:0/3520890947 shutdown_connections 2024-05-27T09:24:56.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.195+0000 7fc054e00700 1 --2- 172.21.15.119:0/3520890947 >> [v2:172.21.15.119:6810/2092077577,v1:172.21.15.119:6811/2092077577] conn(0x7fc034072350 0x7fc034074760 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:56.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.195+0000 7fc054e00700 1 --2- 172.21.15.119:0/3520890947 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc03406c680 0x7fc03406eb30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:56.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.195+0000 7fc054e00700 1 --2- 172.21.15.119:0/3520890947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0500ff2e0 0x7fc050197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:56.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.195+0000 7fc054e00700 1 --2- 172.21.15.119:0/3520890947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0500ffc80 0x7fc050198520 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:56.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.195+0000 7fc054e00700 1 -- 172.21.15.119:0/3520890947 >> 172.21.15.119:0/3520890947 conn(0x7fc0500fab80 msgr2=0x7fc050068cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:56.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.195+0000 7fc054e00700 1 -- 172.21.15.119:0/3520890947 shutdown_connections 2024-05-27T09:24:56.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:56.195+0000 7fc054e00700 1 -- 172.21.15.119:0/3520890947 wait complete. 2024-05-27T09:24:56.954 INFO:teuthology.orchestra.run.smithi119.stdout:81604378654 2024-05-27T09:24:56.954 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-27T09:24:57.041 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:56 smithi119 ceph-mon[29082]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:57.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:56 smithi138 ceph-mon[34885]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:57.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.551+0000 7f1da3400700 1 -- 172.21.15.119:0/78948760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c102600 msgr2=0x7f1d9c1029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:57.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.551+0000 7f1da3400700 1 --2- 172.21.15.119:0/78948760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c102600 0x7f1d9c1029d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009910 tx=0x7f1d90009c20 comp rx=0 tx=0).stop 2024-05-27T09:24:57.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.551+0000 7f1da3400700 1 -- 172.21.15.119:0/78948760 shutdown_connections 2024-05-27T09:24:57.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.551+0000 7f1da3400700 1 --2- 172.21.15.119:0/78948760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d9c102fa0 0x7f1d9c106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:57.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.551+0000 7f1da3400700 1 --2- 172.21.15.119:0/78948760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c102600 0x7f1d9c1029d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:57.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.551+0000 7f1da3400700 1 -- 172.21.15.119:0/78948760 >> 172.21.15.119:0/78948760 conn(0x7f1d9c0fdec0 msgr2=0x7f1d9c1002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:57.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.551+0000 7f1da3400700 1 -- 172.21.15.119:0/78948760 shutdown_connections 2024-05-27T09:24:57.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.552+0000 7f1da3400700 1 -- 172.21.15.119:0/78948760 wait complete. 2024-05-27T09:24:57.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.552+0000 7f1da3400700 1 Processor -- start 2024-05-27T09:24:57.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.553+0000 7f1da3400700 1 -- start start 2024-05-27T09:24:57.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.553+0000 7f1da3400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c102600 0x7f1d9c197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:57.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.553+0000 7f1da3400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d9c102fa0 0x7f1d9c1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:57.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.553+0000 7f1da3400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c198bd0 con 0x7f1d9c102600 2024-05-27T09:24:57.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.553+0000 7f1da3400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c19c960 con 0x7f1d9c102fa0 2024-05-27T09:24:57.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.553+0000 7f1da0a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c102600 0x7f1d9c197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:57.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.553+0000 7f1da0a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c102600 0x7f1d9c197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50160/0 (socket says 172.21.15.119:50160) 2024-05-27T09:24:57.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.553+0000 7f1da0a00700 1 -- 172.21.15.119:0/3362342951 learned_addr learned my addr 172.21.15.119:0/3362342951 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:24:57.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.554+0000 7f1da0a00700 1 -- 172.21.15.119:0/3362342951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d9c102fa0 msgr2=0x7f1d9c1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:57.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.554+0000 7f1d9be00700 1 --2- 172.21.15.119:0/3362342951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d9c102fa0 0x7f1d9c1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:57.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.554+0000 7f1da0a00700 1 --2- 172.21.15.119:0/3362342951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d9c102fa0 0x7f1d9c1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:57.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.554+0000 7f1da0a00700 1 -- 172.21.15.119:0/3362342951 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d900095f0 con 0x7f1d9c102600 2024-05-27T09:24:57.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.554+0000 7f1d9be00700 1 --2- 172.21.15.119:0/3362342951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d9c102fa0 0x7f1d9c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:24:57.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.554+0000 7f1da0a00700 1 --2- 172.21.15.119:0/3362342951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c102600 0x7f1d9c197fb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000b5c0 tx=0x7f1d9000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:57.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.554+0000 7f1d99600700 1 -- 172.21.15.119:0/3362342951 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d9001d070 con 0x7f1d9c102600 2024-05-27T09:24:57.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.555+0000 7f1d99600700 1 -- 172.21.15.119:0/3362342951 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1d90022470 con 0x7f1d9c102600 2024-05-27T09:24:57.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.555+0000 7f1da3400700 1 -- 172.21.15.119:0/3362342951 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9c19cbe0 con 0x7f1d9c102600 2024-05-27T09:24:57.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.555+0000 7f1da3400700 1 -- 172.21.15.119:0/3362342951 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9c19d0d0 con 0x7f1d9c102600 2024-05-27T09:24:57.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.555+0000 7f1d99600700 1 -- 172.21.15.119:0/3362342951 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d9000f630 con 0x7f1d9c102600 2024-05-27T09:24:57.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.556+0000 7f1da3400700 1 -- 172.21.15.119:0/3362342951 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1d9c10f790 con 0x7f1d9c102600 2024-05-27T09:24:57.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.557+0000 7f1d99600700 1 -- 172.21.15.119:0/3362342951 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f1d90004c30 con 0x7f1d9c102600 2024-05-27T09:24:57.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.558+0000 7f1d99600700 1 --2- 172.21.15.119:0/3362342951 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1d8406c680 0x7f1d8406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:57.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.558+0000 7f1d99600700 1 -- 172.21.15.119:0/3362342951 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1d9008cf70 con 0x7f1d9c102600 2024-05-27T09:24:57.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.558+0000 7f1d99600700 1 --2- 172.21.15.119:0/3362342951 >> [v2:172.21.15.138:6800/29238828,v1:172.21.15.138:6801/29238828] conn(0x7f1d84072350 0x7f1d84074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:57.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.558+0000 7f1d99600700 1 -- 172.21.15.119:0/3362342951 --> [v2:172.21.15.138:6800/29238828,v1:172.21.15.138:6801/29238828] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d84074e10 con 0x7f1d84072350 2024-05-27T09:24:57.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.558+0000 7f1d99600700 1 -- 172.21.15.119:0/3362342951 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d9008d3d0 con 0x7f1d9c102600 2024-05-27T09:24:57.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.558+0000 7f1d9be00700 1 --2- 172.21.15.119:0/3362342951 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1d8406c680 0x7f1d8406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:57.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.559+0000 7f1da1400700 1 --2- 172.21.15.119:0/3362342951 >> [v2:172.21.15.138:6800/29238828,v1:172.21.15.138:6801/29238828] conn(0x7f1d84072350 0x7f1d84074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:57.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.559+0000 7f1d9be00700 1 --2- 172.21.15.119:0/3362342951 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1d8406c680 0x7f1d8406eb30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1d88005fd0 tx=0x7f1d88005dc0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:57.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.560+0000 7f1da1400700 1 --2- 172.21.15.119:0/3362342951 >> [v2:172.21.15.138:6800/29238828,v1:172.21.15.138:6801/29238828] conn(0x7f1d84072350 0x7f1d84074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:57.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.562+0000 7f1d99600700 1 -- 172.21.15.119:0/3362342951 <== osd.3 v2:172.21.15.138:6800/29238828 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1d84074e10 con 0x7f1d84072350 2024-05-27T09:24:57.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.580+0000 7f1da3400700 1 -- 172.21.15.119:0/3362342951 --> [v2:172.21.15.138:6800/29238828,v1:172.21.15.138:6801/29238828] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1d9c04ea50 con 0x7f1d84072350 2024-05-27T09:24:57.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.581+0000 7f1d99600700 1 -- 172.21.15.119:0/3362342951 <== osd.3 v2:172.21.15.138:6800/29238828 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1d9c04ea50 con 0x7f1d84072350 2024-05-27T09:24:57.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.582+0000 7f1da3400700 1 -- 172.21.15.119:0/3362342951 >> [v2:172.21.15.138:6800/29238828,v1:172.21.15.138:6801/29238828] conn(0x7f1d84072350 msgr2=0x7f1d84074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:57.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.582+0000 7f1da3400700 1 --2- 172.21.15.119:0/3362342951 >> [v2:172.21.15.138:6800/29238828,v1:172.21.15.138:6801/29238828] conn(0x7f1d84072350 0x7f1d84074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:57.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.582+0000 7f1da3400700 1 -- 172.21.15.119:0/3362342951 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1d8406c680 msgr2=0x7f1d8406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:57.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.582+0000 7f1da3400700 1 --2- 172.21.15.119:0/3362342951 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1d8406c680 0x7f1d8406eb30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1d88005fd0 tx=0x7f1d88005dc0 comp rx=0 tx=0).stop 2024-05-27T09:24:57.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.582+0000 7f1da3400700 1 -- 172.21.15.119:0/3362342951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c102600 msgr2=0x7f1d9c197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:57.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.582+0000 7f1da3400700 1 --2- 172.21.15.119:0/3362342951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c102600 0x7f1d9c197fb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000b5c0 tx=0x7f1d9000be00 comp rx=0 tx=0).stop 2024-05-27T09:24:57.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.582+0000 7f1da3400700 1 -- 172.21.15.119:0/3362342951 shutdown_connections 2024-05-27T09:24:57.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.582+0000 7f1da3400700 1 --2- 172.21.15.119:0/3362342951 >> [v2:172.21.15.138:6800/29238828,v1:172.21.15.138:6801/29238828] conn(0x7f1d84072350 0x7f1d84074760 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:57.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.582+0000 7f1da3400700 1 --2- 172.21.15.119:0/3362342951 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1d8406c680 0x7f1d8406eb30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:57.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.582+0000 7f1da3400700 1 --2- 172.21.15.119:0/3362342951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d9c102600 0x7f1d9c197fb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:57.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.582+0000 7f1da3400700 1 --2- 172.21.15.119:0/3362342951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d9c102fa0 0x7f1d9c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:57.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.582+0000 7f1da3400700 1 -- 172.21.15.119:0/3362342951 >> 172.21.15.119:0/3362342951 conn(0x7f1d9c0fdec0 msgr2=0x7f1d9c0ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:57.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.582+0000 7f1da3400700 1 -- 172.21.15.119:0/3362342951 shutdown_connections 2024-05-27T09:24:57.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.582+0000 7f1da3400700 1 -- 172.21.15.119:0/3362342951 wait complete. 2024-05-27T09:24:57.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.850+0000 7f3e13e00700 1 -- 172.21.15.119:0/3094261184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e14100250 msgr2=0x7f3e14100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:57.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.850+0000 7f3e13e00700 1 --2- 172.21.15.119:0/3094261184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e14100250 0x7f3e14100620 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009910 tx=0x7f3e04009c20 comp rx=0 tx=0).stop 2024-05-27T09:24:57.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.850+0000 7f3e13e00700 1 -- 172.21.15.119:0/3094261184 shutdown_connections 2024-05-27T09:24:57.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.850+0000 7f3e13e00700 1 --2- 172.21.15.119:0/3094261184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e14100bf0 0x7f3e14104bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:57.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.850+0000 7f3e13e00700 1 --2- 172.21.15.119:0/3094261184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e14100250 0x7f3e14100620 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:57.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.850+0000 7f3e13e00700 1 -- 172.21.15.119:0/3094261184 >> 172.21.15.119:0/3094261184 conn(0x7f3e140fbb10 msgr2=0x7f3e140fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:57.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.850+0000 7f3e13e00700 1 -- 172.21.15.119:0/3094261184 shutdown_connections 2024-05-27T09:24:57.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.850+0000 7f3e13e00700 1 -- 172.21.15.119:0/3094261184 wait complete. 2024-05-27T09:24:57.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.851+0000 7f3e13e00700 1 Processor -- start 2024-05-27T09:24:57.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.851+0000 7f3e13e00700 1 -- start start 2024-05-27T09:24:57.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.852+0000 7f3e13e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e14100250 0x7f3e14073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:57.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.852+0000 7f3e13e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e14100bf0 0x7f3e14073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:57.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.852+0000 7f3e13e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e14073ad0 con 0x7f3e14100bf0 2024-05-27T09:24:57.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.852+0000 7f3e13e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e14073c10 con 0x7f3e14100250 2024-05-27T09:24:57.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.852+0000 7f3e12000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e14100bf0 0x7f3e14073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:57.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.852+0000 7f3e12000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e14100bf0 0x7f3e14073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50178/0 (socket says 172.21.15.119:50178) 2024-05-27T09:24:57.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.852+0000 7f3e12000700 1 -- 172.21.15.119:0/1867202041 learned_addr learned my addr 172.21.15.119:0/1867202041 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:24:57.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.852+0000 7f3e12a00700 1 --2- 172.21.15.119:0/1867202041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e14100250 0x7f3e14073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:57.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.853+0000 7f3e12000700 1 -- 172.21.15.119:0/1867202041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e14100250 msgr2=0x7f3e14073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:57.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.853+0000 7f3e12000700 1 --2- 172.21.15.119:0/1867202041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e14100250 0x7f3e14073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:57.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.853+0000 7f3e12000700 1 -- 172.21.15.119:0/1867202041 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e040095f0 con 0x7f3e14100bf0 2024-05-27T09:24:57.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.853+0000 7f3e12a00700 1 --2- 172.21.15.119:0/1867202041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e14100250 0x7f3e14073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:24:57.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.853+0000 7f3e12000700 1 --2- 172.21.15.119:0/1867202041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e14100bf0 0x7f3e14073590 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3e0000c960 tx=0x7f3e0000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:57.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.853+0000 7f3e0b400700 1 -- 172.21.15.119:0/1867202041 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e00007a70 con 0x7f3e14100bf0 2024-05-27T09:24:57.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.853+0000 7f3e0b400700 1 -- 172.21.15.119:0/1867202041 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3e00007bd0 con 0x7f3e14100bf0 2024-05-27T09:24:57.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.853+0000 7f3e0b400700 1 -- 172.21.15.119:0/1867202041 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e00018630 con 0x7f3e14100bf0 2024-05-27T09:24:57.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.853+0000 7f3e13e00700 1 -- 172.21.15.119:0/1867202041 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e141a1e00 con 0x7f3e14100bf0 2024-05-27T09:24:57.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.854+0000 7f3e13e00700 1 -- 172.21.15.119:0/1867202041 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e141a2160 con 0x7f3e14100bf0 2024-05-27T09:24:57.855 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.854+0000 7f3e13e00700 1 -- 172.21.15.119:0/1867202041 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3e1410d3b0 con 0x7f3e14100bf0 2024-05-27T09:24:57.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.856+0000 7f3e0b400700 1 -- 172.21.15.119:0/1867202041 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f3e0000fa90 con 0x7f3e14100bf0 2024-05-27T09:24:57.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.857+0000 7f3e0b400700 1 --2- 172.21.15.119:0/1867202041 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f3df806c680 0x7f3df806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:57.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.857+0000 7f3e0b400700 1 -- 172.21.15.119:0/1867202041 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f3e0008b570 con 0x7f3e14100bf0 2024-05-27T09:24:57.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.857+0000 7f3e12a00700 1 --2- 172.21.15.119:0/1867202041 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f3df806c680 0x7f3df806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:57.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.857+0000 7f3e0b400700 1 --2- 172.21.15.119:0/1867202041 >> [v2:172.21.15.138:6808/3397624307,v1:172.21.15.138:6809/3397624307] conn(0x7f3df8072350 0x7f3df8074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:24:57.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.857+0000 7f3e0b400700 1 -- 172.21.15.119:0/1867202041 --> [v2:172.21.15.138:6808/3397624307,v1:172.21.15.138:6809/3397624307] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3df8074e10 con 0x7f3df8072350 2024-05-27T09:24:57.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.857+0000 7f3e0b400700 1 -- 172.21.15.119:0/1867202041 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e0008dba0 con 0x7f3e14100bf0 2024-05-27T09:24:57.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.858+0000 7f3e12a00700 1 --2- 172.21.15.119:0/1867202041 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f3df806c680 0x7f3df806eb30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3e04006010 tx=0x7f3e0401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:57.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.858+0000 7f3e13400700 1 --2- 172.21.15.119:0/1867202041 >> [v2:172.21.15.138:6808/3397624307,v1:172.21.15.138:6809/3397624307] conn(0x7f3df8072350 0x7f3df8074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:24:57.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.859+0000 7f3e13400700 1 --2- 172.21.15.119:0/1867202041 >> [v2:172.21.15.138:6808/3397624307,v1:172.21.15.138:6809/3397624307] conn(0x7f3df8072350 0x7f3df8074760 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:24:57.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.861+0000 7f3e0b400700 1 -- 172.21.15.119:0/1867202041 <== osd.4 v2:172.21.15.138:6808/3397624307 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3df8074e10 con 0x7f3df8072350 2024-05-27T09:24:57.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.878+0000 7f3e13e00700 1 -- 172.21.15.119:0/1867202041 --> [v2:172.21.15.138:6808/3397624307,v1:172.21.15.138:6809/3397624307] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3e1404f2a0 con 0x7f3df8072350 2024-05-27T09:24:57.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.879+0000 7f3e0b400700 1 -- 172.21.15.119:0/1867202041 <== osd.4 v2:172.21.15.138:6808/3397624307 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3e1404f2a0 con 0x7f3df8072350 2024-05-27T09:24:57.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.879+0000 7f3e13e00700 1 -- 172.21.15.119:0/1867202041 >> [v2:172.21.15.138:6808/3397624307,v1:172.21.15.138:6809/3397624307] conn(0x7f3df8072350 msgr2=0x7f3df8074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:57.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.879+0000 7f3e13e00700 1 --2- 172.21.15.119:0/1867202041 >> [v2:172.21.15.138:6808/3397624307,v1:172.21.15.138:6809/3397624307] conn(0x7f3df8072350 0x7f3df8074760 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:57.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.880+0000 7f3e13e00700 1 -- 172.21.15.119:0/1867202041 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f3df806c680 msgr2=0x7f3df806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:57.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.880+0000 7f3e13e00700 1 --2- 172.21.15.119:0/1867202041 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f3df806c680 0x7f3df806eb30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3e04006010 tx=0x7f3e0401a040 comp rx=0 tx=0).stop 2024-05-27T09:24:57.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.880+0000 7f3e13e00700 1 -- 172.21.15.119:0/1867202041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e14100bf0 msgr2=0x7f3e14073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:24:57.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.880+0000 7f3e13e00700 1 --2- 172.21.15.119:0/1867202041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e14100bf0 0x7f3e14073590 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3e0000c960 tx=0x7f3e0000cd20 comp rx=0 tx=0).stop 2024-05-27T09:24:57.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.880+0000 7f3e13e00700 1 -- 172.21.15.119:0/1867202041 shutdown_connections 2024-05-27T09:24:57.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.880+0000 7f3e13e00700 1 --2- 172.21.15.119:0/1867202041 >> [v2:172.21.15.138:6808/3397624307,v1:172.21.15.138:6809/3397624307] conn(0x7f3df8072350 0x7f3df8074760 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:57.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.880+0000 7f3e13e00700 1 --2- 172.21.15.119:0/1867202041 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f3df806c680 0x7f3df806eb30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:57.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.880+0000 7f3e13e00700 1 --2- 172.21.15.119:0/1867202041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e14100250 0x7f3e14073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:57.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.880+0000 7f3e13e00700 1 --2- 172.21.15.119:0/1867202041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e14100bf0 0x7f3e14073590 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:24:57.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.880+0000 7f3e13e00700 1 -- 172.21.15.119:0/1867202041 >> 172.21.15.119:0/1867202041 conn(0x7f3e140fbb10 msgr2=0x7f3e140fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:24:57.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.880+0000 7f3e13e00700 1 -- 172.21.15.119:0/1867202041 shutdown_connections 2024-05-27T09:24:57.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:24:57.880+0000 7f3e13e00700 1 -- 172.21.15.119:0/1867202041 wait complete. 2024-05-27T09:24:58.197 INFO:teuthology.orchestra.run.smithi119.stdout:60129542182 2024-05-27T09:24:58.197 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-27T09:24:59.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:24:59 smithi119 ceph-mon[29082]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:59.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:24:59 smithi138 ceph-mon[34885]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:24:59.683 INFO:teuthology.orchestra.run.smithi119.stdout:146028888081 2024-05-27T09:24:59.683 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-27T09:25:00.628 INFO:teuthology.orchestra.run.smithi119.stdout:111669149721 2024-05-27T09:25:00.628 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-27T09:25:01.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.165+0000 7fd507200700 1 -- 172.21.15.119:0/4016480993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd508106b80 msgr2=0x7fd508106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:01.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.165+0000 7fd507200700 1 --2- 172.21.15.119:0/4016480993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd508106b80 0x7fd508106ff0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4009910 tx=0x7fd4f4009c20 comp rx=0 tx=0).stop 2024-05-27T09:25:01.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.165+0000 7fd507200700 1 -- 172.21.15.119:0/4016480993 shutdown_connections 2024-05-27T09:25:01.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.165+0000 7fd507200700 1 --2- 172.21.15.119:0/4016480993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd508106b80 0x7fd508106ff0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:01.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.165+0000 7fd507200700 1 --2- 172.21.15.119:0/4016480993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd50810b610 0x7fd50810b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:01.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.165+0000 7fd507200700 1 -- 172.21.15.119:0/4016480993 >> 172.21.15.119:0/4016480993 conn(0x7fd508075df0 msgr2=0x7fd508078200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:01.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.165+0000 7fd507200700 1 -- 172.21.15.119:0/4016480993 shutdown_connections 2024-05-27T09:25:01.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.166+0000 7fd507200700 1 -- 172.21.15.119:0/4016480993 wait complete. 2024-05-27T09:25:01.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.167+0000 7fd507200700 1 Processor -- start 2024-05-27T09:25:01.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.167+0000 7fd507200700 1 -- start start 2024-05-27T09:25:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.168+0000 7fd507200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd508106b80 0x7fd50819c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.168+0000 7fd507200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd50810b610 0x7fd50819c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.168+0000 7fd507200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50819cc60 con 0x7fd50810b610 2024-05-27T09:25:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.168+0000 7fd507200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50819cda0 con 0x7fd508106b80 2024-05-27T09:25:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.168+0000 7fd505400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd50810b610 0x7fd50819c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.168+0000 7fd505400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd50810b610 0x7fd50819c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50954/0 (socket says 172.21.15.119:50954) 2024-05-27T09:25:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.168+0000 7fd505400700 1 -- 172.21.15.119:0/3830188931 learned_addr learned my addr 172.21.15.119:0/3830188931 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:25:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.168+0000 7fd505e00700 1 --2- 172.21.15.119:0/3830188931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd508106b80 0x7fd50819c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.168+0000 7fd505400700 1 -- 172.21.15.119:0/3830188931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd508106b80 msgr2=0x7fd50819c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.169+0000 7fd505400700 1 --2- 172.21.15.119:0/3830188931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd508106b80 0x7fd50819c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.169+0000 7fd505400700 1 -- 172.21.15.119:0/3830188931 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f40095f0 con 0x7fd50810b610 2024-05-27T09:25:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.169+0000 7fd505400700 1 --2- 172.21.15.119:0/3830188931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd50810b610 0x7fd50819c610 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd4f400b5c0 tx=0x7fd4f400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.169+0000 7fd4fea00700 1 -- 172.21.15.119:0/3830188931 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4f401d070 con 0x7fd50810b610 2024-05-27T09:25:01.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.169+0000 7fd4fea00700 1 -- 172.21.15.119:0/3830188931 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd4f4022470 con 0x7fd50810b610 2024-05-27T09:25:01.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.169+0000 7fd4fea00700 1 -- 172.21.15.119:0/3830188931 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4f400f670 con 0x7fd50810b610 2024-05-27T09:25:01.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.169+0000 7fd507200700 1 -- 172.21.15.119:0/3830188931 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5081a0b90 con 0x7fd50810b610 2024-05-27T09:25:01.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.169+0000 7fd507200700 1 -- 172.21.15.119:0/3830188931 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5081a1030 con 0x7fd50810b610 2024-05-27T09:25:01.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.170+0000 7fd507200700 1 -- 172.21.15.119:0/3830188931 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd508113a20 con 0x7fd50810b610 2024-05-27T09:25:01.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.172+0000 7fd4fea00700 1 -- 172.21.15.119:0/3830188931 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fd4f4004c30 con 0x7fd50810b610 2024-05-27T09:25:01.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.173+0000 7fd4fea00700 1 --2- 172.21.15.119:0/3830188931 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fd4ec06c680 0x7fd4ec06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:01.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.173+0000 7fd4fea00700 1 -- 172.21.15.119:0/3830188931 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd4f408cc90 con 0x7fd50810b610 2024-05-27T09:25:01.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.173+0000 7fd505e00700 1 --2- 172.21.15.119:0/3830188931 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fd4ec06c680 0x7fd4ec06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:01.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.173+0000 7fd4fea00700 1 --2- 172.21.15.119:0/3830188931 >> [v2:172.21.15.119:6802/448778921,v1:172.21.15.119:6803/448778921] conn(0x7fd4ec072350 0x7fd4ec074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:01.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.173+0000 7fd4fea00700 1 -- 172.21.15.119:0/3830188931 --> [v2:172.21.15.119:6802/448778921,v1:172.21.15.119:6803/448778921] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd4ec074e10 con 0x7fd4ec072350 2024-05-27T09:25:01.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.174+0000 7fd4fea00700 1 -- 172.21.15.119:0/3830188931 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4f408f2c0 con 0x7fd50810b610 2024-05-27T09:25:01.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.174+0000 7fd506800700 1 --2- 172.21.15.119:0/3830188931 >> [v2:172.21.15.119:6802/448778921,v1:172.21.15.119:6803/448778921] conn(0x7fd4ec072350 0x7fd4ec074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:01.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.174+0000 7fd506800700 1 --2- 172.21.15.119:0/3830188931 >> [v2:172.21.15.119:6802/448778921,v1:172.21.15.119:6803/448778921] conn(0x7fd4ec072350 0x7fd4ec074760 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:01.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.174+0000 7fd505e00700 1 --2- 172.21.15.119:0/3830188931 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fd4ec06c680 0x7fd4ec06eb30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8005ea0 tx=0x7fd4f8005e30 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:01.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.176+0000 7fd4fea00700 1 -- 172.21.15.119:0/3830188931 <== osd.0 v2:172.21.15.119:6802/448778921 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd4ec074e10 con 0x7fd4ec072350 2024-05-27T09:25:01.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.194+0000 7fd507200700 1 -- 172.21.15.119:0/3830188931 --> [v2:172.21.15.119:6802/448778921,v1:172.21.15.119:6803/448778921] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd50804f2a0 con 0x7fd4ec072350 2024-05-27T09:25:01.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.194+0000 7fd4fea00700 1 -- 172.21.15.119:0/3830188931 <== osd.0 v2:172.21.15.119:6802/448778921 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd50804f2a0 con 0x7fd4ec072350 2024-05-27T09:25:01.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.195+0000 7fd507200700 1 -- 172.21.15.119:0/3830188931 >> [v2:172.21.15.119:6802/448778921,v1:172.21.15.119:6803/448778921] conn(0x7fd4ec072350 msgr2=0x7fd4ec074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:01.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.195+0000 7fd507200700 1 --2- 172.21.15.119:0/3830188931 >> [v2:172.21.15.119:6802/448778921,v1:172.21.15.119:6803/448778921] conn(0x7fd4ec072350 0x7fd4ec074760 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:01.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.195+0000 7fd507200700 1 -- 172.21.15.119:0/3830188931 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fd4ec06c680 msgr2=0x7fd4ec06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:01.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.195+0000 7fd507200700 1 --2- 172.21.15.119:0/3830188931 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fd4ec06c680 0x7fd4ec06eb30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8005ea0 tx=0x7fd4f8005e30 comp rx=0 tx=0).stop 2024-05-27T09:25:01.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.195+0000 7fd507200700 1 -- 172.21.15.119:0/3830188931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd50810b610 msgr2=0x7fd50819c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:01.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.195+0000 7fd507200700 1 --2- 172.21.15.119:0/3830188931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd50810b610 0x7fd50819c610 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd4f400b5c0 tx=0x7fd4f400be00 comp rx=0 tx=0).stop 2024-05-27T09:25:01.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.195+0000 7fd507200700 1 -- 172.21.15.119:0/3830188931 shutdown_connections 2024-05-27T09:25:01.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.195+0000 7fd507200700 1 --2- 172.21.15.119:0/3830188931 >> [v2:172.21.15.119:6802/448778921,v1:172.21.15.119:6803/448778921] conn(0x7fd4ec072350 0x7fd4ec074760 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:01.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.195+0000 7fd507200700 1 --2- 172.21.15.119:0/3830188931 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fd4ec06c680 0x7fd4ec06eb30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:01.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.195+0000 7fd507200700 1 --2- 172.21.15.119:0/3830188931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd508106b80 0x7fd50819c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:01.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.195+0000 7fd507200700 1 --2- 172.21.15.119:0/3830188931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd50810b610 0x7fd50819c610 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:01.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.195+0000 7fd507200700 1 -- 172.21.15.119:0/3830188931 >> 172.21.15.119:0/3830188931 conn(0x7fd508075df0 msgr2=0x7fd5080776c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:01.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.196+0000 7fd507200700 1 -- 172.21.15.119:0/3830188931 shutdown_connections 2024-05-27T09:25:01.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.196+0000 7fd507200700 1 -- 172.21.15.119:0/3830188931 wait complete. 2024-05-27T09:25:01.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:01 smithi119 ceph-mon[29082]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:01.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:01 smithi138 ceph-mon[34885]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:01.745 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:25:01.747 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:25:01.750 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:25:01.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.772+0000 7ffa9f600700 1 -- 172.21.15.119:0/1454378344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa98100d50 msgr2=0x7ffa98104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:01.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.772+0000 7ffa9f600700 1 --2- 172.21.15.119:0/1454378344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa98100d50 0x7ffa98104d20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffa84009910 tx=0x7ffa84009c20 comp rx=0 tx=0).stop 2024-05-27T09:25:01.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.773+0000 7ffa9f600700 1 -- 172.21.15.119:0/1454378344 shutdown_connections 2024-05-27T09:25:01.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.773+0000 7ffa9f600700 1 --2- 172.21.15.119:0/1454378344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa98100d50 0x7ffa98104d20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:01.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.773+0000 7ffa9f600700 1 --2- 172.21.15.119:0/1454378344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa981003b0 0x7ffa98100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:01.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.773+0000 7ffa9f600700 1 -- 172.21.15.119:0/1454378344 >> 172.21.15.119:0/1454378344 conn(0x7ffa980fbc30 msgr2=0x7ffa980fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:01.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.773+0000 7ffa9f600700 1 -- 172.21.15.119:0/1454378344 shutdown_connections 2024-05-27T09:25:01.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.773+0000 7ffa9f600700 1 -- 172.21.15.119:0/1454378344 wait complete. 2024-05-27T09:25:01.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.776+0000 7ffa9f600700 1 Processor -- start 2024-05-27T09:25:01.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.776+0000 7ffa9f600700 1 -- start start 2024-05-27T09:25:01.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.777+0000 7ffa9f600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa981003b0 0x7ffa98197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:01.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.777+0000 7ffa9f600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa98100d50 0x7ffa98198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:01.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.778+0000 7ffa9f600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa98198b40 con 0x7ffa98100d50 2024-05-27T09:25:01.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.778+0000 7ffa9f600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9819c8d0 con 0x7ffa981003b0 2024-05-27T09:25:01.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.778+0000 7ffa9cc00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa981003b0 0x7ffa98197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:01.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.778+0000 7ffa9cc00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa981003b0 0x7ffa98197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.119:55800/0 (socket says 172.21.15.119:55800) 2024-05-27T09:25:01.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.778+0000 7ffa9cc00700 1 -- 172.21.15.119:0/3126722337 learned_addr learned my addr 172.21.15.119:0/3126722337 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:25:01.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.778+0000 7ffa97e00700 1 --2- 172.21.15.119:0/3126722337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa98100d50 0x7ffa98198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:01.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.779+0000 7ffa9cc00700 1 -- 172.21.15.119:0/3126722337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa98100d50 msgr2=0x7ffa98198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:01.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.779+0000 7ffa9cc00700 1 --2- 172.21.15.119:0/3126722337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa98100d50 0x7ffa98198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:01.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.779+0000 7ffa9cc00700 1 -- 172.21.15.119:0/3126722337 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa840095f0 con 0x7ffa981003b0 2024-05-27T09:25:01.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.779+0000 7ffa97e00700 1 --2- 172.21.15.119:0/3126722337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa98100d50 0x7ffa98198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:25:01.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.780+0000 7ffa9cc00700 1 --2- 172.21.15.119:0/3126722337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa981003b0 0x7ffa98197f20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0060b0 tx=0x7ffa8c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:01.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.781+0000 7ffa95600700 1 -- 172.21.15.119:0/3126722337 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa8c00dda0 con 0x7ffa981003b0 2024-05-27T09:25:01.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.781+0000 7ffa95600700 1 -- 172.21.15.119:0/3126722337 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffa8c00f040 con 0x7ffa981003b0 2024-05-27T09:25:01.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.781+0000 7ffa95600700 1 -- 172.21.15.119:0/3126722337 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa8c0146f0 con 0x7ffa981003b0 2024-05-27T09:25:01.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.781+0000 7ffa9f600700 1 -- 172.21.15.119:0/3126722337 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa9819cbb0 con 0x7ffa981003b0 2024-05-27T09:25:01.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.781+0000 7ffa9f600700 1 -- 172.21.15.119:0/3126722337 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa9819d0b0 con 0x7ffa981003b0 2024-05-27T09:25:01.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.782+0000 7ffa9f600700 1 -- 172.21.15.119:0/3126722337 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffa980744f0 con 0x7ffa981003b0 2024-05-27T09:25:01.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.785+0000 7ffa95600700 1 -- 172.21.15.119:0/3126722337 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7ffa8c004ad0 con 0x7ffa981003b0 2024-05-27T09:25:01.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.787+0000 7ffa95600700 1 --2- 172.21.15.119:0/3126722337 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ffa8006c680 0x7ffa8006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:01.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.787+0000 7ffa95600700 1 -- 172.21.15.119:0/3126722337 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ffa8c08b440 con 0x7ffa981003b0 2024-05-27T09:25:01.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.787+0000 7ffa95600700 1 --2- 172.21.15.119:0/3126722337 >> [v2:172.21.15.138:6816/886004363,v1:172.21.15.138:6817/886004363] conn(0x7ffa80072350 0x7ffa80074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:01.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.787+0000 7ffa97e00700 1 --2- 172.21.15.119:0/3126722337 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ffa8006c680 0x7ffa8006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:01.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.787+0000 7ffa95600700 1 -- 172.21.15.119:0/3126722337 --> [v2:172.21.15.138:6816/886004363,v1:172.21.15.138:6817/886004363] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa80074e10 con 0x7ffa80072350 2024-05-27T09:25:01.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.787+0000 7ffa95600700 1 -- 172.21.15.119:0/3126722337 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa8c08b840 con 0x7ffa981003b0 2024-05-27T09:25:01.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.788+0000 7ffa9d600700 1 --2- 172.21.15.119:0/3126722337 >> [v2:172.21.15.138:6816/886004363,v1:172.21.15.138:6817/886004363] conn(0x7ffa80072350 0x7ffa80074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:01.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.789+0000 7ffa97e00700 1 --2- 172.21.15.119:0/3126722337 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ffa8006c680 0x7ffa8006eb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffa98199540 tx=0x7ffa8401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:01.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.789+0000 7ffa9d600700 1 --2- 172.21.15.119:0/3126722337 >> [v2:172.21.15.138:6816/886004363,v1:172.21.15.138:6817/886004363] conn(0x7ffa80072350 0x7ffa80074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:01.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.791+0000 7ffa95600700 1 -- 172.21.15.119:0/3126722337 <== osd.5 v2:172.21.15.138:6816/886004363 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ffa80074e10 con 0x7ffa80072350 2024-05-27T09:25:01.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.819+0000 7ffa9f600700 1 -- 172.21.15.119:0/3126722337 --> [v2:172.21.15.138:6816/886004363,v1:172.21.15.138:6817/886004363] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffa9804ea50 con 0x7ffa80072350 2024-05-27T09:25:01.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.820+0000 7ffa95600700 1 -- 172.21.15.119:0/3126722337 <== osd.5 v2:172.21.15.138:6816/886004363 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ffa9804ea50 con 0x7ffa80072350 2024-05-27T09:25:01.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.821+0000 7ffa9f600700 1 -- 172.21.15.119:0/3126722337 >> [v2:172.21.15.138:6816/886004363,v1:172.21.15.138:6817/886004363] conn(0x7ffa80072350 msgr2=0x7ffa80074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:01.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.821+0000 7ffa9f600700 1 --2- 172.21.15.119:0/3126722337 >> [v2:172.21.15.138:6816/886004363,v1:172.21.15.138:6817/886004363] conn(0x7ffa80072350 0x7ffa80074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:01.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.821+0000 7ffa9f600700 1 -- 172.21.15.119:0/3126722337 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ffa8006c680 msgr2=0x7ffa8006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:01.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.821+0000 7ffa9f600700 1 --2- 172.21.15.119:0/3126722337 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ffa8006c680 0x7ffa8006eb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffa98199540 tx=0x7ffa8401a040 comp rx=0 tx=0).stop 2024-05-27T09:25:01.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.821+0000 7ffa9f600700 1 -- 172.21.15.119:0/3126722337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa981003b0 msgr2=0x7ffa98197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:01.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.821+0000 7ffa9f600700 1 --2- 172.21.15.119:0/3126722337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa981003b0 0x7ffa98197f20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0060b0 tx=0x7ffa8c00d3b0 comp rx=0 tx=0).stop 2024-05-27T09:25:01.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.821+0000 7ffa9f600700 1 -- 172.21.15.119:0/3126722337 shutdown_connections 2024-05-27T09:25:01.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.821+0000 7ffa9f600700 1 --2- 172.21.15.119:0/3126722337 >> [v2:172.21.15.138:6816/886004363,v1:172.21.15.138:6817/886004363] conn(0x7ffa80072350 0x7ffa80074760 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:01.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.821+0000 7ffa9f600700 1 --2- 172.21.15.119:0/3126722337 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ffa8006c680 0x7ffa8006eb30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:01.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.821+0000 7ffa9f600700 1 --2- 172.21.15.119:0/3126722337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa981003b0 0x7ffa98197f20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:01.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.821+0000 7ffa9f600700 1 --2- 172.21.15.119:0/3126722337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa98100d50 0x7ffa98198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:01.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.821+0000 7ffa9f600700 1 -- 172.21.15.119:0/3126722337 >> 172.21.15.119:0/3126722337 conn(0x7ffa980fbc30 msgr2=0x7ffa980fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:01.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.822+0000 7ffa9f600700 1 -- 172.21.15.119:0/3126722337 shutdown_connections 2024-05-27T09:25:01.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:01.822+0000 7ffa9f600700 1 -- 172.21.15.119:0/3126722337 wait complete. 2024-05-27T09:25:02.459 INFO:teuthology.orchestra.run.smithi119.stdout:38654705709 2024-05-27T09:25:02.459 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-27T09:25:03.179 INFO:teuthology.orchestra.run.smithi119.stdout:167503724554 2024-05-27T09:25:03.180 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-27T09:25:03.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:03 smithi138 ceph-mon[34885]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:03.636 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:03 smithi119 ceph-mon[29082]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:03.789 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:25:05.395 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:05 smithi119 ceph-mon[29082]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:05.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:05 smithi138 ceph-mon[34885]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:05.960 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:25:05.966 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:25:07.097 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:06 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:25:07.098 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:06 smithi119 ceph-mon[29082]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:07.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:06 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:25:07.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:06 smithi138 ceph-mon[34885]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:08.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.752+0000 7efdb2400700 1 -- 172.21.15.119:0/234533671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdac1026a0 msgr2=0x7efdac102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:08.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.752+0000 7efdb2400700 1 --2- 172.21.15.119:0/234533671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdac1026a0 0x7efdac102a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009910 tx=0x7efd9c009c20 comp rx=0 tx=0).stop 2024-05-27T09:25:08.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.752+0000 7efdb2400700 1 -- 172.21.15.119:0/234533671 shutdown_connections 2024-05-27T09:25:08.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.752+0000 7efdb2400700 1 --2- 172.21.15.119:0/234533671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdac103040 0x7efdac107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:08.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.752+0000 7efdb2400700 1 --2- 172.21.15.119:0/234533671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdac1026a0 0x7efdac102a70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:08.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.752+0000 7efdb2400700 1 -- 172.21.15.119:0/234533671 >> 172.21.15.119:0/234533671 conn(0x7efdac0fdf00 msgr2=0x7efdac100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:08.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.752+0000 7efdb2400700 1 -- 172.21.15.119:0/234533671 shutdown_connections 2024-05-27T09:25:08.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.752+0000 7efdb2400700 1 -- 172.21.15.119:0/234533671 wait complete. 2024-05-27T09:25:08.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.753+0000 7efdb2400700 1 Processor -- start 2024-05-27T09:25:08.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.753+0000 7efdb2400700 1 -- start start 2024-05-27T09:25:08.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.754+0000 7efdb2400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdac1026a0 0x7efdac198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:08.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.754+0000 7efdb2400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdac103040 0x7efdac198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:08.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.754+0000 7efdb2400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac198c40 con 0x7efdac103040 2024-05-27T09:25:08.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.754+0000 7efdb2400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac19c9d0 con 0x7efdac1026a0 2024-05-27T09:25:08.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.754+0000 7efda3e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdac103040 0x7efdac198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:08.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.754+0000 7efda3e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdac103040 0x7efdac198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50982/0 (socket says 172.21.15.119:50982) 2024-05-27T09:25:08.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.754+0000 7efda3e00700 1 -- 172.21.15.119:0/614037733 learned_addr learned my addr 172.21.15.119:0/614037733 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:25:08.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.754+0000 7efdab400700 1 --2- 172.21.15.119:0/614037733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdac1026a0 0x7efdac198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:08.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.755+0000 7efda3e00700 1 -- 172.21.15.119:0/614037733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdac1026a0 msgr2=0x7efdac198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:08.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.755+0000 7efda3e00700 1 --2- 172.21.15.119:0/614037733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdac1026a0 0x7efdac198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:08.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.755+0000 7efda3e00700 1 -- 172.21.15.119:0/614037733 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c0095f0 con 0x7efdac103040 2024-05-27T09:25:08.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.755+0000 7efda3e00700 1 --2- 172.21.15.119:0/614037733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdac103040 0x7efdac198560 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efd9400c8f0 tx=0x7efd9400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:08.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.755+0000 7efda8c00700 1 -- 172.21.15.119:0/614037733 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd94007a50 con 0x7efdac103040 2024-05-27T09:25:08.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.755+0000 7efda8c00700 1 -- 172.21.15.119:0/614037733 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efd94007bb0 con 0x7efdac103040 2024-05-27T09:25:08.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.755+0000 7efdb2400700 1 -- 172.21.15.119:0/614037733 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdac19ccb0 con 0x7efdac103040 2024-05-27T09:25:08.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.755+0000 7efda8c00700 1 -- 172.21.15.119:0/614037733 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd94018630 con 0x7efdac103040 2024-05-27T09:25:08.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.755+0000 7efdb2400700 1 -- 172.21.15.119:0/614037733 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdac19d200 con 0x7efdac103040 2024-05-27T09:25:08.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.756+0000 7efdb2400700 1 -- 172.21.15.119:0/614037733 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdac10a9b0 con 0x7efdac103040 2024-05-27T09:25:08.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.757+0000 7efda8c00700 1 -- 172.21.15.119:0/614037733 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7efd9400f450 con 0x7efdac103040 2024-05-27T09:25:08.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.758+0000 7efda8c00700 1 --2- 172.21.15.119:0/614037733 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7efd9006c680 0x7efd9006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:08.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.758+0000 7efda8c00700 1 -- 172.21.15.119:0/614037733 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7efd9408b520 con 0x7efdac103040 2024-05-27T09:25:08.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.759+0000 7efdab400700 1 --2- 172.21.15.119:0/614037733 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7efd9006c680 0x7efd9006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:08.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.759+0000 7efdab400700 1 --2- 172.21.15.119:0/614037733 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7efd9006c680 0x7efd9006eb30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efd9c006010 tx=0x7efd9c01a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:08.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.763+0000 7efda8c00700 1 -- 172.21.15.119:0/614037733 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7efd94055fc0 con 0x7efdac103040 2024-05-27T09:25:08.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.992+0000 7efdb2400700 1 -- 172.21.15.119:0/614037733 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7efdac04ea50 con 0x7efdac103040 2024-05-27T09:25:08.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.992+0000 7efda8c00700 1 -- 172.21.15.119:0/614037733 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efd940595e0 con 0x7efdac103040 2024-05-27T09:25:08.993 INFO:teuthology.orchestra.run.smithi119.stdout:146028888083 2024-05-27T09:25:08.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.995+0000 7efdb2400700 1 -- 172.21.15.119:0/614037733 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7efd9006c680 msgr2=0x7efd9006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:08.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.995+0000 7efdb2400700 1 --2- 172.21.15.119:0/614037733 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7efd9006c680 0x7efd9006eb30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efd9c006010 tx=0x7efd9c01a040 comp rx=0 tx=0).stop 2024-05-27T09:25:08.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.995+0000 7efdb2400700 1 -- 172.21.15.119:0/614037733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdac103040 msgr2=0x7efdac198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:08.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.995+0000 7efdb2400700 1 --2- 172.21.15.119:0/614037733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdac103040 0x7efdac198560 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efd9400c8f0 tx=0x7efd9400ccb0 comp rx=0 tx=0).stop 2024-05-27T09:25:08.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.995+0000 7efdb2400700 1 -- 172.21.15.119:0/614037733 shutdown_connections 2024-05-27T09:25:08.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.995+0000 7efdb2400700 1 --2- 172.21.15.119:0/614037733 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7efd9006c680 0x7efd9006eb30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:08.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.995+0000 7efdb2400700 1 --2- 172.21.15.119:0/614037733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdac1026a0 0x7efdac198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:08.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.995+0000 7efdb2400700 1 --2- 172.21.15.119:0/614037733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdac103040 0x7efdac198560 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:08.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.995+0000 7efdb2400700 1 -- 172.21.15.119:0/614037733 >> 172.21.15.119:0/614037733 conn(0x7efdac0fdf00 msgr2=0x7efdac0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:08.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.995+0000 7efdb2400700 1 -- 172.21.15.119:0/614037733 shutdown_connections 2024-05-27T09:25:08.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:08.995+0000 7efdb2400700 1 -- 172.21.15.119:0/614037733 wait complete. 2024-05-27T09:25:09.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.519+0000 7f1ccd800700 1 -- 172.21.15.119:0/2511540264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cc8102db0 msgr2=0x7f1cc8106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:09.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.519+0000 7f1ccd800700 1 --2- 172.21.15.119:0/2511540264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cc8102db0 0x7f1cc8106d80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009910 tx=0x7f1cb4009c20 comp rx=0 tx=0).stop 2024-05-27T09:25:09.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.519+0000 7f1ccd800700 1 -- 172.21.15.119:0/2511540264 shutdown_connections 2024-05-27T09:25:09.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.519+0000 7f1ccd800700 1 --2- 172.21.15.119:0/2511540264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cc8102db0 0x7f1cc8106d80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:09.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.519+0000 7f1ccd800700 1 --2- 172.21.15.119:0/2511540264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cc8102410 0x7f1cc81027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:09.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.519+0000 7f1ccd800700 1 -- 172.21.15.119:0/2511540264 >> 172.21.15.119:0/2511540264 conn(0x7f1cc80fdcf0 msgr2=0x7f1cc8100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:09.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.520+0000 7f1ccd800700 1 -- 172.21.15.119:0/2511540264 shutdown_connections 2024-05-27T09:25:09.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.520+0000 7f1ccd800700 1 -- 172.21.15.119:0/2511540264 wait complete. 2024-05-27T09:25:09.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.521+0000 7f1ccd800700 1 Processor -- start 2024-05-27T09:25:09.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.521+0000 7f1ccd800700 1 -- start start 2024-05-27T09:25:09.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.521+0000 7f1ccd800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cc8102410 0x7f1cc8197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:09.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.521+0000 7f1ccd800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cc8102db0 0x7f1cc8198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:09.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.521+0000 7f1ccd800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8198a20 con 0x7f1cc8102db0 2024-05-27T09:25:09.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.521+0000 7f1ccd800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc819c7b0 con 0x7f1cc8102410 2024-05-27T09:25:09.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.521+0000 7f1cc7400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cc8102db0 0x7f1cc8198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:09.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.521+0000 7f1cc7400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cc8102db0 0x7f1cc8198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36072/0 (socket says 172.21.15.119:36072) 2024-05-27T09:25:09.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.521+0000 7f1cc7400700 1 -- 172.21.15.119:0/1660086020 learned_addr learned my addr 172.21.15.119:0/1660086020 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:25:09.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.522+0000 7f1cc7e00700 1 --2- 172.21.15.119:0/1660086020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cc8102410 0x7f1cc8197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:09.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.522+0000 7f1cc7400700 1 -- 172.21.15.119:0/1660086020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cc8102410 msgr2=0x7f1cc8197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:09.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.522+0000 7f1cc7400700 1 --2- 172.21.15.119:0/1660086020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cc8102410 0x7f1cc8197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:09.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.522+0000 7f1cc7400700 1 -- 172.21.15.119:0/1660086020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb40095f0 con 0x7f1cc8102db0 2024-05-27T09:25:09.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.522+0000 7f1cc7400700 1 --2- 172.21.15.119:0/1660086020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cc8102db0 0x7f1cc8198340 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40092e0 tx=0x7f1cb400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:09.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.523+0000 7f1cc4c00700 1 -- 172.21.15.119:0/1660086020 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cb401d070 con 0x7f1cc8102db0 2024-05-27T09:25:09.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.523+0000 7f1ccd800700 1 -- 172.21.15.119:0/1660086020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc819ca30 con 0x7f1cc8102db0 2024-05-27T09:25:09.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.523+0000 7f1cc4c00700 1 -- 172.21.15.119:0/1660086020 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1cb4022470 con 0x7f1cc8102db0 2024-05-27T09:25:09.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.523+0000 7f1cc4c00700 1 -- 172.21.15.119:0/1660086020 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cb400f690 con 0x7f1cc8102db0 2024-05-27T09:25:09.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.523+0000 7f1ccd800700 1 -- 172.21.15.119:0/1660086020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc819cf20 con 0x7f1cc8102db0 2024-05-27T09:25:09.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.524+0000 7f1ccd800700 1 -- 172.21.15.119:0/1660086020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc810a6d0 con 0x7f1cc8102db0 2024-05-27T09:25:09.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.525+0000 7f1cc4c00700 1 -- 172.21.15.119:0/1660086020 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f1cb40225e0 con 0x7f1cc8102db0 2024-05-27T09:25:09.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.526+0000 7f1cc4c00700 1 --2- 172.21.15.119:0/1660086020 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1cac06c680 0x7f1cac06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:09.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.526+0000 7f1cc4c00700 1 -- 172.21.15.119:0/1660086020 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1cb408cc00 con 0x7f1cc8102db0 2024-05-27T09:25:09.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.526+0000 7f1cc7e00700 1 --2- 172.21.15.119:0/1660086020 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1cac06c680 0x7f1cac06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:09.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.526+0000 7f1cc7e00700 1 --2- 172.21.15.119:0/1660086020 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1cac06c680 0x7f1cac06eb30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8006fd0 tx=0x7f1cb8008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:09.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.531+0000 7f1cc4c00700 1 -- 172.21.15.119:0/1660086020 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1cb4057630 con 0x7f1cc8102db0 2024-05-27T09:25:09.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:09 smithi138 ceph-mon[34885]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:09.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:09 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/614037733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-27T09:25:09.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:09 smithi119 ceph-mon[29082]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:09.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:09 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/614037733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-27T09:25:09.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.753+0000 7f1ccd800700 1 -- 172.21.15.119:0/1660086020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1cc802cc30 con 0x7f1cc8102db0 2024-05-27T09:25:09.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.754+0000 7f1cc4c00700 1 -- 172.21.15.119:0/1660086020 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1cb4027020 con 0x7f1cc8102db0 2024-05-27T09:25:09.755 INFO:teuthology.orchestra.run.smithi119.stdout:81604378656 2024-05-27T09:25:09.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.757+0000 7f1ccd800700 1 -- 172.21.15.119:0/1660086020 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1cac06c680 msgr2=0x7f1cac06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:09.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.757+0000 7f1ccd800700 1 --2- 172.21.15.119:0/1660086020 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1cac06c680 0x7f1cac06eb30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8006fd0 tx=0x7f1cb8008040 comp rx=0 tx=0).stop 2024-05-27T09:25:09.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.757+0000 7f1ccd800700 1 -- 172.21.15.119:0/1660086020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cc8102db0 msgr2=0x7f1cc8198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:09.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.757+0000 7f1ccd800700 1 --2- 172.21.15.119:0/1660086020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cc8102db0 0x7f1cc8198340 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40092e0 tx=0x7f1cb400be20 comp rx=0 tx=0).stop 2024-05-27T09:25:09.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.758+0000 7f1ccd800700 1 -- 172.21.15.119:0/1660086020 shutdown_connections 2024-05-27T09:25:09.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.758+0000 7f1ccd800700 1 --2- 172.21.15.119:0/1660086020 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f1cac06c680 0x7f1cac06eb30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:09.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.758+0000 7f1ccd800700 1 --2- 172.21.15.119:0/1660086020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cc8102410 0x7f1cc8197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:09.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.758+0000 7f1ccd800700 1 --2- 172.21.15.119:0/1660086020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cc8102db0 0x7f1cc8198340 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:09.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.758+0000 7f1ccd800700 1 -- 172.21.15.119:0/1660086020 >> 172.21.15.119:0/1660086020 conn(0x7f1cc80fdcf0 msgr2=0x7f1cc80ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:09.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.758+0000 7f1ccd800700 1 -- 172.21.15.119:0/1660086020 shutdown_connections 2024-05-27T09:25:09.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:09.758+0000 7f1ccd800700 1 -- 172.21.15.119:0/1660086020 wait complete. 2024-05-27T09:25:10.456 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:10 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/1660086020' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-27T09:25:10.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:10 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/1660086020' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-27T09:25:11.261 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888081 got 146028888083 for osd.4 2024-05-27T09:25:11.261 DEBUG:teuthology.parallel:result is None 2024-05-27T09:25:11.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:11 smithi138 ceph-mon[34885]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:11.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.486+0000 7f16fa600700 1 -- 172.21.15.119:0/2495537397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f410c010 msgr2=0x7f16f4106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:11.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.486+0000 7f16fa600700 1 --2- 172.21.15.119:0/2495537397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f410c010 0x7f16f4106b80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009910 tx=0x7f16dc009c20 comp rx=0 tx=0).stop 2024-05-27T09:25:11.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.486+0000 7f16fa600700 1 -- 172.21.15.119:0/2495537397 shutdown_connections 2024-05-27T09:25:11.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.486+0000 7f16fa600700 1 --2- 172.21.15.119:0/2495537397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f410c010 0x7f16f4106b80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:11.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.486+0000 7f16fa600700 1 --2- 172.21.15.119:0/2495537397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16f410b700 0x7f16f410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:11.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.486+0000 7f16fa600700 1 -- 172.21.15.119:0/2495537397 >> 172.21.15.119:0/2495537397 conn(0x7f16f4076000 msgr2=0x7f16f4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:11.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.486+0000 7f16fa600700 1 -- 172.21.15.119:0/2495537397 shutdown_connections 2024-05-27T09:25:11.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.486+0000 7f16fa600700 1 -- 172.21.15.119:0/2495537397 wait complete. 2024-05-27T09:25:11.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.487+0000 7f16fa600700 1 Processor -- start 2024-05-27T09:25:11.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.487+0000 7f16fa600700 1 -- start start 2024-05-27T09:25:11.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.487+0000 7f16fa600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16f410b700 0x7f16f419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:11.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.487+0000 7f16fa600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f410c010 0x7f16f419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:11.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.487+0000 7f16fa600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f419cf00 con 0x7f16f410c010 2024-05-27T09:25:11.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.487+0000 7f16fa600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f41a0c90 con 0x7f16f410b700 2024-05-27T09:25:11.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.488+0000 7f16ebe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f410c010 0x7f16f419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:11.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.488+0000 7f16ebe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f410c010 0x7f16f419c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36090/0 (socket says 172.21.15.119:36090) 2024-05-27T09:25:11.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.488+0000 7f16ebe00700 1 -- 172.21.15.119:0/3480431674 learned_addr learned my addr 172.21.15.119:0/3480431674 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:25:11.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.488+0000 7f16f3400700 1 --2- 172.21.15.119:0/3480431674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16f410b700 0x7f16f419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:11.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.488+0000 7f16ebe00700 1 -- 172.21.15.119:0/3480431674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16f410b700 msgr2=0x7f16f419c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:11.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.488+0000 7f16ebe00700 1 --2- 172.21.15.119:0/3480431674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16f410b700 0x7f16f419c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:11.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.488+0000 7f16ebe00700 1 -- 172.21.15.119:0/3480431674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc0095f0 con 0x7f16f410c010 2024-05-27T09:25:11.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.488+0000 7f16f3400700 1 --2- 172.21.15.119:0/3480431674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16f410b700 0x7f16f419c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:25:11.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.489+0000 7f16ebe00700 1 --2- 172.21.15.119:0/3480431674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f410c010 0x7f16f419c820 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0092e0 tx=0x7f16dc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:11.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.489+0000 7f16f0c00700 1 -- 172.21.15.119:0/3480431674 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16dc01d070 con 0x7f16f410c010 2024-05-27T09:25:11.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.489+0000 7f16f0c00700 1 -- 172.21.15.119:0/3480431674 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f16dc00f460 con 0x7f16f410c010 2024-05-27T09:25:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.489+0000 7f16f0c00700 1 -- 172.21.15.119:0/3480431674 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16dc017630 con 0x7f16f410c010 2024-05-27T09:25:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.489+0000 7f16fa600700 1 -- 172.21.15.119:0/3480431674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f41a0f10 con 0x7f16f410c010 2024-05-27T09:25:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.489+0000 7f16fa600700 1 -- 172.21.15.119:0/3480431674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f41a1400 con 0x7f16f410c010 2024-05-27T09:25:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.490+0000 7f16fa600700 1 -- 172.21.15.119:0/3480431674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f404ea50 con 0x7f16f410c010 2024-05-27T09:25:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.491+0000 7f16f0c00700 1 -- 172.21.15.119:0/3480431674 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f16dc003a60 con 0x7f16f410c010 2024-05-27T09:25:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.492+0000 7f16f0c00700 1 --2- 172.21.15.119:0/3480431674 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f16d806c680 0x7f16d806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.492+0000 7f16f0c00700 1 -- 172.21.15.119:0/3480431674 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f16dc08ca00 con 0x7f16f410c010 2024-05-27T09:25:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.492+0000 7f16f3400700 1 --2- 172.21.15.119:0/3480431674 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f16d806c680 0x7f16d806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.493+0000 7f16f3400700 1 --2- 172.21.15.119:0/3480431674 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f16d806c680 0x7f16d806eb30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f16e4005fd0 tx=0x7f16e4005ee0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:11.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.498+0000 7f16f0c00700 1 -- 172.21.15.119:0/3480431674 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f16dc057400 con 0x7f16f410c010 2024-05-27T09:25:11.663 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:11 smithi119 ceph-mon[29082]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:11.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.718+0000 7f16fa600700 1 -- 172.21.15.119:0/3480431674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f16f419d640 con 0x7f16f410c010 2024-05-27T09:25:11.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.719+0000 7f16f0c00700 1 -- 172.21.15.119:0/3480431674 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f16dc05aa20 con 0x7f16f410c010 2024-05-27T09:25:11.719 INFO:teuthology.orchestra.run.smithi119.stdout:60129542185 2024-05-27T09:25:11.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.721+0000 7f16fa600700 1 -- 172.21.15.119:0/3480431674 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f16d806c680 msgr2=0x7f16d806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:11.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.721+0000 7f16fa600700 1 --2- 172.21.15.119:0/3480431674 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f16d806c680 0x7f16d806eb30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f16e4005fd0 tx=0x7f16e4005ee0 comp rx=0 tx=0).stop 2024-05-27T09:25:11.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.721+0000 7f16fa600700 1 -- 172.21.15.119:0/3480431674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f410c010 msgr2=0x7f16f419c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:11.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.721+0000 7f16fa600700 1 --2- 172.21.15.119:0/3480431674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f410c010 0x7f16f419c820 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0092e0 tx=0x7f16dc00be20 comp rx=0 tx=0).stop 2024-05-27T09:25:11.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.721+0000 7f16fa600700 1 -- 172.21.15.119:0/3480431674 shutdown_connections 2024-05-27T09:25:11.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.722+0000 7f16fa600700 1 --2- 172.21.15.119:0/3480431674 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f16d806c680 0x7f16d806eb30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:11.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.722+0000 7f16fa600700 1 --2- 172.21.15.119:0/3480431674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16f410b700 0x7f16f419c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:11.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.722+0000 7f16fa600700 1 --2- 172.21.15.119:0/3480431674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16f410c010 0x7f16f419c820 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:11.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.722+0000 7f16fa600700 1 -- 172.21.15.119:0/3480431674 >> 172.21.15.119:0/3480431674 conn(0x7f16f4076000 msgr2=0x7f16f4076940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:11.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.722+0000 7f16fa600700 1 -- 172.21.15.119:0/3480431674 shutdown_connections 2024-05-27T09:25:11.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:11.722+0000 7f16fa600700 1 -- 172.21.15.119:0/3480431674 wait complete. 2024-05-27T09:25:12.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:12 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/3480431674' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-27T09:25:12.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:12 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/3480431674' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-27T09:25:12.707 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542182 got 60129542185 for osd.1 2024-05-27T09:25:12.707 DEBUG:teuthology.parallel:result is None 2024-05-27T09:25:13.157 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378654 got 81604378656 for osd.2 2024-05-27T09:25:13.157 DEBUG:teuthology.parallel:result is None 2024-05-27T09:25:13.420 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:13 smithi119 ceph-mon[29082]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:13.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:13 smithi138 ceph-mon[34885]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:13.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.741+0000 7f847fa00700 1 -- 172.21.15.119:0/816792484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84780ffd30 msgr2=0x7f8478103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:13.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.741+0000 7f847fa00700 1 --2- 172.21.15.119:0/816792484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84780ffd30 0x7f8478103ce0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8464009910 tx=0x7f8464009c20 comp rx=0 tx=0).stop 2024-05-27T09:25:13.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.741+0000 7f847fa00700 1 -- 172.21.15.119:0/816792484 shutdown_connections 2024-05-27T09:25:13.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.741+0000 7f847fa00700 1 --2- 172.21.15.119:0/816792484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84780ffd30 0x7f8478103ce0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:13.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.741+0000 7f847fa00700 1 --2- 172.21.15.119:0/816792484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84780ff390 0x7f84780ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:13.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.741+0000 7f847fa00700 1 -- 172.21.15.119:0/816792484 >> 172.21.15.119:0/816792484 conn(0x7f8478074f50 msgr2=0x7f8478075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:13.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.742+0000 7f847fa00700 1 -- 172.21.15.119:0/816792484 shutdown_connections 2024-05-27T09:25:13.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.742+0000 7f847fa00700 1 -- 172.21.15.119:0/816792484 wait complete. 2024-05-27T09:25:13.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.743+0000 7f847fa00700 1 Processor -- start 2024-05-27T09:25:13.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.743+0000 7f847fa00700 1 -- start start 2024-05-27T09:25:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.743+0000 7f847fa00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84780ff390 0x7f8478197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.743+0000 7f847fa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84780ffd30 0x7f8478198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.743+0000 7f847fa00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8478198bf0 con 0x7f84780ffd30 2024-05-27T09:25:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.743+0000 7f847fa00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847819c980 con 0x7f84780ff390 2024-05-27T09:25:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.744+0000 7f8477e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84780ffd30 0x7f8478198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.744+0000 7f8477e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84780ffd30 0x7f8478198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36118/0 (socket says 172.21.15.119:36118) 2024-05-27T09:25:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.744+0000 7f8477e00700 1 -- 172.21.15.119:0/3552773554 learned_addr learned my addr 172.21.15.119:0/3552773554 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:25:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.744+0000 7f847d000700 1 --2- 172.21.15.119:0/3552773554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84780ff390 0x7f8478197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.744+0000 7f8477e00700 1 -- 172.21.15.119:0/3552773554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84780ff390 msgr2=0x7f8478197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.744+0000 7f8477e00700 1 --2- 172.21.15.119:0/3552773554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84780ff390 0x7f8478197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.744+0000 7f8477e00700 1 -- 172.21.15.119:0/3552773554 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84640095f0 con 0x7f84780ffd30 2024-05-27T09:25:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.744+0000 7f8477e00700 1 --2- 172.21.15.119:0/3552773554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84780ffd30 0x7f8478198510 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8464005f50 tx=0x7f846400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.745+0000 7f8475600700 1 -- 172.21.15.119:0/3552773554 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f846401d070 con 0x7f84780ffd30 2024-05-27T09:25:13.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.745+0000 7f8475600700 1 -- 172.21.15.119:0/3552773554 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f846400f460 con 0x7f84780ffd30 2024-05-27T09:25:13.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.745+0000 7f847fa00700 1 -- 172.21.15.119:0/3552773554 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847819cc00 con 0x7f84780ffd30 2024-05-27T09:25:13.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.745+0000 7f8475600700 1 -- 172.21.15.119:0/3552773554 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8464004fd0 con 0x7f84780ffd30 2024-05-27T09:25:13.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.745+0000 7f847fa00700 1 -- 172.21.15.119:0/3552773554 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847819d0f0 con 0x7f84780ffd30 2024-05-27T09:25:13.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.746+0000 7f847fa00700 1 -- 172.21.15.119:0/3552773554 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f847804ea50 con 0x7f84780ffd30 2024-05-27T09:25:13.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.747+0000 7f8475600700 1 -- 172.21.15.119:0/3552773554 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f846400f5d0 con 0x7f84780ffd30 2024-05-27T09:25:13.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.748+0000 7f8475600700 1 --2- 172.21.15.119:0/3552773554 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f846006c680 0x7f846006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:13.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.748+0000 7f8475600700 1 -- 172.21.15.119:0/3552773554 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f846408c970 con 0x7f84780ffd30 2024-05-27T09:25:13.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.748+0000 7f847d000700 1 --2- 172.21.15.119:0/3552773554 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f846006c680 0x7f846006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:13.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.749+0000 7f847d000700 1 --2- 172.21.15.119:0/3552773554 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f846006c680 0x7f846006eb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f846c0060b0 tx=0x7f846c007b80 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:13.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.753+0000 7f8475600700 1 -- 172.21.15.119:0/3552773554 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f84640572f0 con 0x7f84780ffd30 2024-05-27T09:25:13.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.978+0000 7f847fa00700 1 -- 172.21.15.119:0/3552773554 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f847819d410 con 0x7f84780ffd30 2024-05-27T09:25:13.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.979+0000 7f8475600700 1 -- 172.21.15.119:0/3552773554 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8464026090 con 0x7f84780ffd30 2024-05-27T09:25:13.979 INFO:teuthology.orchestra.run.smithi119.stdout:111669149724 2024-05-27T09:25:13.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.982+0000 7f847fa00700 1 -- 172.21.15.119:0/3552773554 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f846006c680 msgr2=0x7f846006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:13.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.982+0000 7f847fa00700 1 --2- 172.21.15.119:0/3552773554 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f846006c680 0x7f846006eb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f846c0060b0 tx=0x7f846c007b80 comp rx=0 tx=0).stop 2024-05-27T09:25:13.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.982+0000 7f847fa00700 1 -- 172.21.15.119:0/3552773554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84780ffd30 msgr2=0x7f8478198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:13.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.982+0000 7f847fa00700 1 --2- 172.21.15.119:0/3552773554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84780ffd30 0x7f8478198510 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8464005f50 tx=0x7f846400be20 comp rx=0 tx=0).stop 2024-05-27T09:25:13.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.982+0000 7f847fa00700 1 -- 172.21.15.119:0/3552773554 shutdown_connections 2024-05-27T09:25:13.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.982+0000 7f847fa00700 1 --2- 172.21.15.119:0/3552773554 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f846006c680 0x7f846006eb30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:13.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.982+0000 7f847fa00700 1 --2- 172.21.15.119:0/3552773554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84780ff390 0x7f8478197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:13.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.982+0000 7f847fa00700 1 --2- 172.21.15.119:0/3552773554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84780ffd30 0x7f8478198510 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:13.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.982+0000 7f847fa00700 1 -- 172.21.15.119:0/3552773554 >> 172.21.15.119:0/3552773554 conn(0x7f8478074f50 msgr2=0x7f8478068cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:13.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.983+0000 7f847fa00700 1 -- 172.21.15.119:0/3552773554 shutdown_connections 2024-05-27T09:25:13.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.983+0000 7f847fa00700 1 -- 172.21.15.119:0/3552773554 wait complete. 2024-05-27T09:25:14.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.846+0000 7f154b000700 1 -- 172.21.15.119:0/1036568584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15440ffd30 msgr2=0x7f1544103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:14.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.846+0000 7f154b000700 1 --2- 172.21.15.119:0/1036568584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15440ffd30 0x7f1544103ce0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1530009910 tx=0x7f1530009c20 comp rx=0 tx=0).stop 2024-05-27T09:25:14.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.847+0000 7f154b000700 1 -- 172.21.15.119:0/1036568584 shutdown_connections 2024-05-27T09:25:14.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.847+0000 7f154b000700 1 --2- 172.21.15.119:0/1036568584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15440ffd30 0x7f1544103ce0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:14.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.847+0000 7f154b000700 1 --2- 172.21.15.119:0/1036568584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15440ff390 0x7f15440ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:14.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.847+0000 7f154b000700 1 -- 172.21.15.119:0/1036568584 >> 172.21.15.119:0/1036568584 conn(0x7f1544074f50 msgr2=0x7f1544075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:14.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.847+0000 7f154b000700 1 -- 172.21.15.119:0/1036568584 shutdown_connections 2024-05-27T09:25:14.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.847+0000 7f154b000700 1 -- 172.21.15.119:0/1036568584 wait complete. 2024-05-27T09:25:14.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.848+0000 7f154b000700 1 Processor -- start 2024-05-27T09:25:14.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.848+0000 7f154b000700 1 -- start start 2024-05-27T09:25:14.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.848+0000 7f154b000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15440ff390 0x7f1544197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:14.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.848+0000 7f154b000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15440ffd30 0x7f1544198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:14.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.848+0000 7f154b000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1544198bf0 con 0x7f15440ff390 2024-05-27T09:25:14.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.848+0000 7f154b000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154419c980 con 0x7f15440ffd30 2024-05-27T09:25:14.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.848+0000 7f1543e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15440ff390 0x7f1544197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:14.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.849+0000 7f1543e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15440ff390 0x7f1544197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36128/0 (socket says 172.21.15.119:36128) 2024-05-27T09:25:14.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.849+0000 7f1543e00700 1 -- 172.21.15.119:0/2914000958 learned_addr learned my addr 172.21.15.119:0/2914000958 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:25:14.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.849+0000 7f1543400700 1 --2- 172.21.15.119:0/2914000958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15440ffd30 0x7f1544198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:14.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.849+0000 7f1543e00700 1 -- 172.21.15.119:0/2914000958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15440ffd30 msgr2=0x7f1544198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:14.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.849+0000 7f1543e00700 1 --2- 172.21.15.119:0/2914000958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15440ffd30 0x7f1544198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:14.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.849+0000 7f1543e00700 1 -- 172.21.15.119:0/2914000958 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15300095f0 con 0x7f15440ff390 2024-05-27T09:25:14.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.849+0000 7f1543e00700 1 --2- 172.21.15.119:0/2914000958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15440ff390 0x7f1544197fd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f153400d8d0 tx=0x7f153400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:14.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.850+0000 7f1540c00700 1 -- 172.21.15.119:0/2914000958 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1534009940 con 0x7f15440ff390 2024-05-27T09:25:14.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.850+0000 7f1540c00700 1 -- 172.21.15.119:0/2914000958 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1534010460 con 0x7f15440ff390 2024-05-27T09:25:14.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.850+0000 7f1540c00700 1 -- 172.21.15.119:0/2914000958 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f153400f5f0 con 0x7f15440ff390 2024-05-27T09:25:14.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.850+0000 7f154b000700 1 -- 172.21.15.119:0/2914000958 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154419cc60 con 0x7f15440ff390 2024-05-27T09:25:14.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.850+0000 7f154b000700 1 -- 172.21.15.119:0/2914000958 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154419d1b0 con 0x7f15440ff390 2024-05-27T09:25:14.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.851+0000 7f154b000700 1 -- 172.21.15.119:0/2914000958 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154404ea50 con 0x7f15440ff390 2024-05-27T09:25:14.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.853+0000 7f1540c00700 1 -- 172.21.15.119:0/2914000958 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f15340105d0 con 0x7f15440ff390 2024-05-27T09:25:14.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.855+0000 7f1540c00700 1 --2- 172.21.15.119:0/2914000958 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f152806c680 0x7f152806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:14.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.855+0000 7f1540c00700 1 -- 172.21.15.119:0/2914000958 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f153408b510 con 0x7f15440ff390 2024-05-27T09:25:14.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.855+0000 7f1543400700 1 --2- 172.21.15.119:0/2914000958 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f152806c680 0x7f152806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:14.366 INFO:teuthology.orchestra.run.smithi119.stdout:38654705711 2024-05-27T09:25:14.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.856+0000 7f1543400700 1 --2- 172.21.15.119:0/2914000958 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f152806c680 0x7f152806eb30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f15441995f0 tx=0x7f153001a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:14.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:13.860+0000 7f1540c00700 1 -- 172.21.15.119:0/2914000958 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1534055f10 con 0x7f15440ff390 2024-05-27T09:25:14.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.085+0000 7f154b000700 1 -- 172.21.15.119:0/2914000958 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1544199330 con 0x7f15440ff390 2024-05-27T09:25:14.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.085+0000 7f1540c00700 1 -- 172.21.15.119:0/2914000958 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f153405de60 con 0x7f15440ff390 2024-05-27T09:25:14.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.088+0000 7f154b000700 1 -- 172.21.15.119:0/2914000958 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f152806c680 msgr2=0x7f152806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:14.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.088+0000 7f154b000700 1 --2- 172.21.15.119:0/2914000958 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f152806c680 0x7f152806eb30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f15441995f0 tx=0x7f153001a040 comp rx=0 tx=0).stop 2024-05-27T09:25:14.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.088+0000 7f154b000700 1 -- 172.21.15.119:0/2914000958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15440ff390 msgr2=0x7f1544197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:14.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.088+0000 7f154b000700 1 --2- 172.21.15.119:0/2914000958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15440ff390 0x7f1544197fd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f153400d8d0 tx=0x7f153400dc90 comp rx=0 tx=0).stop 2024-05-27T09:25:14.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.088+0000 7f154b000700 1 -- 172.21.15.119:0/2914000958 shutdown_connections 2024-05-27T09:25:14.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.088+0000 7f154b000700 1 --2- 172.21.15.119:0/2914000958 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f152806c680 0x7f152806eb30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:14.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.088+0000 7f154b000700 1 --2- 172.21.15.119:0/2914000958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15440ff390 0x7f1544197fd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:14.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.088+0000 7f154b000700 1 --2- 172.21.15.119:0/2914000958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15440ffd30 0x7f1544198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:14.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.088+0000 7f154b000700 1 -- 172.21.15.119:0/2914000958 >> 172.21.15.119:0/2914000958 conn(0x7f1544074f50 msgr2=0x7f1544068cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:14.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.088+0000 7f154b000700 1 -- 172.21.15.119:0/2914000958 shutdown_connections 2024-05-27T09:25:14.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.089+0000 7f154b000700 1 -- 172.21.15.119:0/2914000958 wait complete. 2024-05-27T09:25:14.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:14 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/3552773554' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-27T09:25:14.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:14 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/2914000958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-27T09:25:14.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.176+0000 7fd935200700 1 -- 172.21.15.119:0/3395728025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd930102630 msgr2=0x7fd930102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:14.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.176+0000 7fd935200700 1 --2- 172.21.15.119:0/3395728025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd930102630 0x7fd930102a00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd920009910 tx=0x7fd920009c20 comp rx=0 tx=0).stop 2024-05-27T09:25:14.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.177+0000 7fd935200700 1 -- 172.21.15.119:0/3395728025 shutdown_connections 2024-05-27T09:25:14.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.177+0000 7fd935200700 1 --2- 172.21.15.119:0/3395728025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd930102fd0 0x7fd930106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:14.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.177+0000 7fd935200700 1 --2- 172.21.15.119:0/3395728025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd930102630 0x7fd930102a00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:14.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.177+0000 7fd935200700 1 -- 172.21.15.119:0/3395728025 >> 172.21.15.119:0/3395728025 conn(0x7fd9300fdef0 msgr2=0x7fd930100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:14.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.177+0000 7fd935200700 1 -- 172.21.15.119:0/3395728025 shutdown_connections 2024-05-27T09:25:14.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.177+0000 7fd935200700 1 -- 172.21.15.119:0/3395728025 wait complete. 2024-05-27T09:25:14.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.178+0000 7fd935200700 1 Processor -- start 2024-05-27T09:25:14.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.178+0000 7fd935200700 1 -- start start 2024-05-27T09:25:14.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.178+0000 7fd935200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd930102630 0x7fd930198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:14.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.178+0000 7fd935200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd930102fd0 0x7fd930198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:14.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.178+0000 7fd935200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd930198c40 con 0x7fd930102fd0 2024-05-27T09:25:14.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.178+0000 7fd935200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93019c9d0 con 0x7fd930102630 2024-05-27T09:25:14.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.178+0000 7fd92d400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd930102fd0 0x7fd930198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:14.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.178+0000 7fd92d400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd930102fd0 0x7fd930198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36154/0 (socket says 172.21.15.119:36154) 2024-05-27T09:25:14.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.178+0000 7fd92d400700 1 -- 172.21.15.119:0/47012479 learned_addr learned my addr 172.21.15.119:0/47012479 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:25:14.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.179+0000 7fd92de00700 1 --2- 172.21.15.119:0/47012479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd930102630 0x7fd930198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:14.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.179+0000 7fd92d400700 1 -- 172.21.15.119:0/47012479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd930102630 msgr2=0x7fd930198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:14.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.179+0000 7fd92d400700 1 --2- 172.21.15.119:0/47012479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd930102630 0x7fd930198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:14.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.179+0000 7fd92d400700 1 -- 172.21.15.119:0/47012479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9200095f0 con 0x7fd930102fd0 2024-05-27T09:25:14.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.179+0000 7fd92d400700 1 --2- 172.21.15.119:0/47012479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd930102fd0 0x7fd930198560 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00c8f0 tx=0x7fd91c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:14.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.179+0000 7fd926a00700 1 -- 172.21.15.119:0/47012479 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd91c007a50 con 0x7fd930102fd0 2024-05-27T09:25:14.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.179+0000 7fd926a00700 1 -- 172.21.15.119:0/47012479 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd91c007bb0 con 0x7fd930102fd0 2024-05-27T09:25:14.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.179+0000 7fd935200700 1 -- 172.21.15.119:0/47012479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd93019ccb0 con 0x7fd930102fd0 2024-05-27T09:25:14.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.179+0000 7fd926a00700 1 -- 172.21.15.119:0/47012479 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd91c018630 con 0x7fd930102fd0 2024-05-27T09:25:14.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.180+0000 7fd935200700 1 -- 172.21.15.119:0/47012479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd93019d200 con 0x7fd930102fd0 2024-05-27T09:25:14.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.181+0000 7fd935200700 1 -- 172.21.15.119:0/47012479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd93010a940 con 0x7fd930102fd0 2024-05-27T09:25:14.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.182+0000 7fd926a00700 1 -- 172.21.15.119:0/47012479 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fd91c007d20 con 0x7fd930102fd0 2024-05-27T09:25:14.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.183+0000 7fd926a00700 1 --2- 172.21.15.119:0/47012479 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fd91406c680 0x7fd91406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:14.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.183+0000 7fd926a00700 1 -- 172.21.15.119:0/47012479 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd91c08b550 con 0x7fd930102fd0 2024-05-27T09:25:14.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.183+0000 7fd92de00700 1 --2- 172.21.15.119:0/47012479 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fd91406c680 0x7fd91406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:14.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.183+0000 7fd92de00700 1 --2- 172.21.15.119:0/47012479 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fd91406c680 0x7fd91406eb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd920000c00 tx=0x7fd920005fd0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:14.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.187+0000 7fd926a00700 1 -- 172.21.15.119:0/47012479 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd91c055f50 con 0x7fd930102fd0 2024-05-27T09:25:14.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.413+0000 7fd935200700 1 -- 172.21.15.119:0/47012479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fd93019d4d0 con 0x7fd930102fd0 2024-05-27T09:25:14.660 INFO:teuthology.orchestra.run.smithi119.stdout:167503724557 2024-05-27T09:25:14.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.414+0000 7fd926a00700 1 -- 172.21.15.119:0/47012479 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd91c059570 con 0x7fd930102fd0 2024-05-27T09:25:14.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.417+0000 7fd935200700 1 -- 172.21.15.119:0/47012479 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fd91406c680 msgr2=0x7fd91406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:14.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.417+0000 7fd935200700 1 --2- 172.21.15.119:0/47012479 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fd91406c680 0x7fd91406eb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd920000c00 tx=0x7fd920005fd0 comp rx=0 tx=0).stop 2024-05-27T09:25:14.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.417+0000 7fd935200700 1 -- 172.21.15.119:0/47012479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd930102fd0 msgr2=0x7fd930198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:14.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.417+0000 7fd935200700 1 --2- 172.21.15.119:0/47012479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd930102fd0 0x7fd930198560 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00c8f0 tx=0x7fd91c00ccb0 comp rx=0 tx=0).stop 2024-05-27T09:25:14.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.417+0000 7fd935200700 1 -- 172.21.15.119:0/47012479 shutdown_connections 2024-05-27T09:25:14.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.417+0000 7fd935200700 1 --2- 172.21.15.119:0/47012479 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fd91406c680 0x7fd91406eb30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:14.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.417+0000 7fd935200700 1 --2- 172.21.15.119:0/47012479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd930102630 0x7fd930198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:14.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.417+0000 7fd935200700 1 --2- 172.21.15.119:0/47012479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd930102fd0 0x7fd930198560 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:14.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.417+0000 7fd935200700 1 -- 172.21.15.119:0/47012479 >> 172.21.15.119:0/47012479 conn(0x7fd9300fdef0 msgr2=0x7fd9301002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:14.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.417+0000 7fd935200700 1 -- 172.21.15.119:0/47012479 shutdown_connections 2024-05-27T09:25:14.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:14.417+0000 7fd935200700 1 -- 172.21.15.119:0/47012479 wait complete. 2024-05-27T09:25:14.661 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:14 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/3552773554' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-27T09:25:14.661 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:14 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/2914000958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-27T09:25:15.379 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705709 got 38654705711 for osd.0 2024-05-27T09:25:15.379 DEBUG:teuthology.parallel:result is None 2024-05-27T09:25:15.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:15 smithi138 ceph-mon[34885]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:15.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:15 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/47012479' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-27T09:25:15.680 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:15 smithi119 ceph-mon[29082]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:15.680 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:15 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/47012479' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-27T09:25:15.726 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149721 got 111669149724 for osd.3 2024-05-27T09:25:15.727 DEBUG:teuthology.parallel:result is None 2024-05-27T09:25:16.251 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724554 got 167503724557 for osd.5 2024-05-27T09:25:16.251 DEBUG:teuthology.parallel:result is None 2024-05-27T09:25:16.251 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-27T09:25:16.252 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T09:25:16.656 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:25:17.080 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:16 smithi119 ceph-mon[29082]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:17.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:16 smithi138 ceph-mon[34885]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:18.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.757+0000 7f31ae800700 1 -- 172.21.15.119:0/1923370141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a8102fd0 msgr2=0x7f31a8106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:18.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.757+0000 7f31ae800700 1 --2- 172.21.15.119:0/1923370141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a8102fd0 0x7f31a8106fa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3194009910 tx=0x7f3194009c20 comp rx=0 tx=0).stop 2024-05-27T09:25:18.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.757+0000 7f31ae800700 1 -- 172.21.15.119:0/1923370141 shutdown_connections 2024-05-27T09:25:18.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.757+0000 7f31ae800700 1 --2- 172.21.15.119:0/1923370141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a8102fd0 0x7f31a8106fa0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:18.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.757+0000 7f31ae800700 1 --2- 172.21.15.119:0/1923370141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31a8102630 0x7f31a8102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:18.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.757+0000 7f31ae800700 1 -- 172.21.15.119:0/1923370141 >> 172.21.15.119:0/1923370141 conn(0x7f31a80fdef0 msgr2=0x7f31a8100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:18.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.757+0000 7f31ae800700 1 -- 172.21.15.119:0/1923370141 shutdown_connections 2024-05-27T09:25:18.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.757+0000 7f31ae800700 1 -- 172.21.15.119:0/1923370141 wait complete. 2024-05-27T09:25:18.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.758+0000 7f31ae800700 1 Processor -- start 2024-05-27T09:25:18.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.759+0000 7f31ae800700 1 -- start start 2024-05-27T09:25:18.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.759+0000 7f31ae800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a8102630 0x7f31a8075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:18.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.759+0000 7f31ae800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31a8102fd0 0x7f31a80757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:18.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.759+0000 7f31ae800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80793f0 con 0x7f31a8102630 2024-05-27T09:25:18.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.759+0000 7f31ae800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a8075ce0 con 0x7f31a8102fd0 2024-05-27T09:25:18.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.759+0000 7f31a7400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a8102630 0x7f31a8075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:18.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.759+0000 7f31a7400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a8102630 0x7f31a8075260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36168/0 (socket says 172.21.15.119:36168) 2024-05-27T09:25:18.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.759+0000 7f31a7400700 1 -- 172.21.15.119:0/3868421709 learned_addr learned my addr 172.21.15.119:0/3868421709 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:25:18.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.760+0000 7f31a6a00700 1 --2- 172.21.15.119:0/3868421709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31a8102fd0 0x7f31a80757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:18.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.760+0000 7f31a7400700 1 -- 172.21.15.119:0/3868421709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31a8102fd0 msgr2=0x7f31a80757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:18.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.760+0000 7f31a7400700 1 --2- 172.21.15.119:0/3868421709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31a8102fd0 0x7f31a80757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:18.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.760+0000 7f31a7400700 1 -- 172.21.15.119:0/3868421709 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31940095f0 con 0x7f31a8102630 2024-05-27T09:25:18.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.760+0000 7f31a7400700 1 --2- 172.21.15.119:0/3868421709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a8102630 0x7f31a8075260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f319800d8d0 tx=0x7f319800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:18.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.760+0000 7f319fe00700 1 -- 172.21.15.119:0/3868421709 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3198009940 con 0x7f31a8102630 2024-05-27T09:25:18.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.760+0000 7f319fe00700 1 -- 172.21.15.119:0/3868421709 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3198010460 con 0x7f31a8102630 2024-05-27T09:25:18.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.760+0000 7f31ae800700 1 -- 172.21.15.119:0/3868421709 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a8075fc0 con 0x7f31a8102630 2024-05-27T09:25:18.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.760+0000 7f319fe00700 1 -- 172.21.15.119:0/3868421709 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f319800f5f0 con 0x7f31a8102630 2024-05-27T09:25:18.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.761+0000 7f31ae800700 1 -- 172.21.15.119:0/3868421709 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a81a6810 con 0x7f31a8102630 2024-05-27T09:25:18.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.762+0000 7f31ae800700 1 -- 172.21.15.119:0/3868421709 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a804ea50 con 0x7f31a8102630 2024-05-27T09:25:18.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.763+0000 7f319fe00700 1 -- 172.21.15.119:0/3868421709 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f31980105d0 con 0x7f31a8102630 2024-05-27T09:25:18.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.763+0000 7f319fe00700 1 --2- 172.21.15.119:0/3868421709 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f318c06c680 0x7f318c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:18.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.764+0000 7f319fe00700 1 -- 172.21.15.119:0/3868421709 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f319808b4f0 con 0x7f31a8102630 2024-05-27T09:25:18.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.764+0000 7f31a6a00700 1 --2- 172.21.15.119:0/3868421709 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f318c06c680 0x7f318c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:18.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.764+0000 7f31a6a00700 1 --2- 172.21.15.119:0/3868421709 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f318c06c680 0x7f318c06eb30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3194006010 tx=0x7f319400b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:18.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.768+0000 7f319fe00700 1 -- 172.21.15.119:0/3868421709 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3198055ef0 con 0x7f31a8102630 2024-05-27T09:25:18.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.979+0000 7f31ae800700 1 -- 172.21.15.119:0/3868421709 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f31a8100200 con 0x7f318c06c680 2024-05-27T09:25:18.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.984+0000 7f319fe00700 1 -- 172.21.15.119:0/3868421709 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27218 (secure 0 0 0) 0x7f31a8100200 con 0x7f318c06c680 2024-05-27T09:25:18.986 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:25:18.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.987+0000 7f31ae800700 1 -- 172.21.15.119:0/3868421709 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f318c06c680 msgr2=0x7f318c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:18.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.987+0000 7f31ae800700 1 --2- 172.21.15.119:0/3868421709 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f318c06c680 0x7f318c06eb30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3194006010 tx=0x7f319400b540 comp rx=0 tx=0).stop 2024-05-27T09:25:18.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.987+0000 7f31ae800700 1 -- 172.21.15.119:0/3868421709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a8102630 msgr2=0x7f31a8075260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:18.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.987+0000 7f31ae800700 1 --2- 172.21.15.119:0/3868421709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a8102630 0x7f31a8075260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f319800d8d0 tx=0x7f319800dc90 comp rx=0 tx=0).stop 2024-05-27T09:25:18.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.987+0000 7f31ae800700 1 -- 172.21.15.119:0/3868421709 shutdown_connections 2024-05-27T09:25:18.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.987+0000 7f31ae800700 1 --2- 172.21.15.119:0/3868421709 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f318c06c680 0x7f318c06eb30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:18.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.987+0000 7f31ae800700 1 --2- 172.21.15.119:0/3868421709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a8102630 0x7f31a8075260 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:18.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.987+0000 7f31ae800700 1 --2- 172.21.15.119:0/3868421709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31a8102fd0 0x7f31a80757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:18.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.987+0000 7f31ae800700 1 -- 172.21.15.119:0/3868421709 >> 172.21.15.119:0/3868421709 conn(0x7f31a80fdef0 msgr2=0x7f31a80ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:18.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.988+0000 7f31ae800700 1 -- 172.21.15.119:0/3868421709 shutdown_connections 2024-05-27T09:25:18.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:18.988+0000 7f31ae800700 1 -- 172.21.15.119:0/3868421709 wait complete. 2024-05-27T09:25:18.990 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-05-27T09:25:19.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:19 smithi138 ceph-mon[34885]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:19.704 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:19 smithi119 ceph-mon[29082]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:19.767 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-05-27T09:25:18.258733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165584,"kb_used_data":3872,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305328,"statfs":{"total":575970213888,"available":575800655872,"internally_reserved":0,"allocated":3964928,"data_stored":2608497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003393"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":186,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T09:24:21.330494+0000","last_change":"2024-05-27T09:23:45.322438+0000","last_active":"2024-05-27T09:24:21.330494+0000","last_peered":"2024-05-27T09:24:21.330494+0000","last_clean":"2024-05-27T09:24:21.330494+0000","last_became_active":"2024-05-27T09:23:45.321314+0000","last_became_peered":"2024-05-27T09:23:45.321314+0000","last_unstale":"2024-05-27T09:24:21.330494+0000","last_undegraded":"2024-05-27T09:24:21.330494+0000","last_fullsized":"2024-05-27T09:24:21.330494+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:22:34.972595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:22:34.972595+0000","last_clean_scrub_stamp":"2024-05-27T09:22:34.972595+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:24:15.456898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0369999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]}]},{"osd":4,"up_from":34,"seq":146028888085,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 27 09:24:46 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":1.3460000000000001}]},{"osd":1,"last update":"Mon May 27 09:24:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.0349999999999999}]},{"osd":2,"last update":"Mon May 27 09:24:46 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.28},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":1.0800000000000001}]},{"osd":3,"last update":"Mon May 27 09:24:46 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.155}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149725,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 27 09:25:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.93700000000000006,"15min":0.92100000000000004},"min":{"1min":0.72299999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.61299999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.97299999999999998,"15min":0.96399999999999997},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.93100000000000005}]},{"osd":1,"last update":"Mon May 27 09:25:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.018,"15min":1.0029999999999999},"min":{"1min":0.77600000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.995},{"interface":"front","average":{"1min":1.097,"5min":1.0129999999999999,"15min":0.999},"min":{"1min":0.81000000000000005,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.70199999999999996}]},{"osd":2,"last update":"Mon May 27 09:25:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.0529999999999999,"15min":1.0429999999999999},"min":{"1min":0.76900000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1259999999999999,"15min":1.1200000000000001},"min":{"1min":0.76100000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.78900000000000003}]},{"osd":4,"last update":"Mon May 27 09:24:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.165}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]}]},{"osd":2,"up_from":19,"seq":81604378658,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27304,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717848,"statfs":{"total":95995035648,"available":95967076352,"internally_reserved":0,"allocated":360448,"data_stored":139008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 27 09:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.79700000000000004,"15min":0.79000000000000004},"min":{"1min":0.66200000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.85599999999999998,"15min":0.86099999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.093,"5min":1.169,"15min":1.169},"last":0.86099999999999999}]},{"osd":1,"last update":"Mon May 27 09:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.88600000000000001,"15min":0.87},"min":{"1min":0.755,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.96899999999999997,"15min":0.95499999999999996},"min":{"1min":0.89700000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.2270000000000001}]},{"osd":3,"last update":"Mon May 27 09:25:12 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.1539999999999999,"15min":1.159},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.415,"5min":1.591,"15min":1.591},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.1160000000000001,"15min":1.1200000000000001},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.0529999999999999}]},{"osd":4,"last update":"Mon May 27 09:24:48 2024","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.264,"5min":1.264,"15min":1.264},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.4059999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.369}]}]},{"osd":0,"up_from":9,"seq":38654705712,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon May 27 09:25:00 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.74099999999999999,"15min":0.70099999999999996},"min":{"1min":0.442,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.80300000000000005,"15min":0.77200000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.135,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.69299999999999995}]},{"osd":2,"last update":"Mon May 27 09:24:37 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.86799999999999999,"15min":0.85999999999999999},"min":{"1min":0.51700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.98099999999999998,"15min":0.97899999999999998},"min":{"1min":0.54000000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.325,"5min":1.865,"15min":1.865},"last":1.0580000000000001}]},{"osd":3,"last update":"Mon May 27 09:24:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.3129999999999999}]},{"osd":4,"last update":"Mon May 27 09:24:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":1.8660000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 27 09:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77200000000000002,"15min":0.751},"min":{"1min":0.40999999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.046,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.83599999999999997,"15min":0.82399999999999995},"min":{"1min":0.51700000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.4059999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.94199999999999995}]},{"osd":2,"last update":"Mon May 27 09:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92700000000000005,"15min":0.92800000000000005},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":1.089},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.99399999999999999,"15min":1.0009999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2010000000000001,"5min":1.23,"15min":1.23},"last":0.98999999999999999}]},{"osd":3,"last update":"Mon May 27 09:25:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0389999999999999,"15min":1.0349999999999999},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":1.181},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0660000000000001,"15min":1.0669999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.2250000000000001}]},{"osd":4,"last update":"Mon May 27 09:24:47 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.2869999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T09:25:19.769 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T09:25:20.180 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:25:20.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:20 smithi138 ceph-mon[34885]: from='client.14420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T09:25:20.666 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:20 smithi119 ceph-mon[29082]: from='client.14420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T09:25:21.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:21 smithi138 ceph-mon[34885]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:21.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:21 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:25:21.652 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:21 smithi119 ceph-mon[29082]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:21.652 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:21 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:25:22.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.422+0000 7fbd90800700 1 -- 172.21.15.119:0/2701953222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd881003d0 msgr2=0x7fbd881007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.422+0000 7fbd90800700 1 --2- 172.21.15.119:0/2701953222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd881003d0 0x7fbd881007a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c009910 tx=0x7fbd7c009c20 comp rx=0 tx=0).stop 2024-05-27T09:25:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.422+0000 7fbd90800700 1 -- 172.21.15.119:0/2701953222 shutdown_connections 2024-05-27T09:25:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.422+0000 7fbd90800700 1 --2- 172.21.15.119:0/2701953222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd88100d70 0x7fbd88104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.422+0000 7fbd90800700 1 --2- 172.21.15.119:0/2701953222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd881003d0 0x7fbd881007a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.422+0000 7fbd90800700 1 -- 172.21.15.119:0/2701953222 >> 172.21.15.119:0/2701953222 conn(0x7fbd880fbc90 msgr2=0x7fbd880fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.422+0000 7fbd90800700 1 -- 172.21.15.119:0/2701953222 shutdown_connections 2024-05-27T09:25:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.422+0000 7fbd90800700 1 -- 172.21.15.119:0/2701953222 wait complete. 2024-05-27T09:25:22.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.423+0000 7fbd90800700 1 Processor -- start 2024-05-27T09:25:22.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.424+0000 7fbd90800700 1 -- start start 2024-05-27T09:25:22.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.424+0000 7fbd90800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd881003d0 0x7fbd88198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:22.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.424+0000 7fbd90800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd88100d70 0x7fbd88198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:22.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.424+0000 7fbd90800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd88198c40 con 0x7fbd88100d70 2024-05-27T09:25:22.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.424+0000 7fbd90800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8819c980 con 0x7fbd881003d0 2024-05-27T09:25:22.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.424+0000 7fbd8d400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd88100d70 0x7fbd88198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:22.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.424+0000 7fbd8d400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd88100d70 0x7fbd88198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54360/0 (socket says 172.21.15.119:54360) 2024-05-27T09:25:22.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.424+0000 7fbd8d400700 1 -- 172.21.15.119:0/615777271 learned_addr learned my addr 172.21.15.119:0/615777271 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:25:22.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.425+0000 7fbd8de00700 1 --2- 172.21.15.119:0/615777271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd881003d0 0x7fbd88198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:22.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.425+0000 7fbd8d400700 1 -- 172.21.15.119:0/615777271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd881003d0 msgr2=0x7fbd88198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:22.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.425+0000 7fbd8d400700 1 --2- 172.21.15.119:0/615777271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd881003d0 0x7fbd88198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:22.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.425+0000 7fbd8d400700 1 -- 172.21.15.119:0/615777271 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c0095f0 con 0x7fbd88100d70 2024-05-27T09:25:22.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.425+0000 7fbd8d400700 1 --2- 172.21.15.119:0/615777271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd88100d70 0x7fbd88198560 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800b480 tx=0x7fbd7800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:22.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.425+0000 7fbd82a00700 1 -- 172.21.15.119:0/615777271 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd780118c0 con 0x7fbd88100d70 2024-05-27T09:25:22.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.425+0000 7fbd90800700 1 -- 172.21.15.119:0/615777271 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd8819cc60 con 0x7fbd88100d70 2024-05-27T09:25:22.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.425+0000 7fbd82a00700 1 -- 172.21.15.119:0/615777271 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbd78010380 con 0x7fbd88100d70 2024-05-27T09:25:22.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.425+0000 7fbd82a00700 1 -- 172.21.15.119:0/615777271 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd7800f4f0 con 0x7fbd88100d70 2024-05-27T09:25:22.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.426+0000 7fbd90800700 1 -- 172.21.15.119:0/615777271 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd8819d1b0 con 0x7fbd88100d70 2024-05-27T09:25:22.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.427+0000 7fbd90800700 1 -- 172.21.15.119:0/615777271 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd881044c0 con 0x7fbd88100d70 2024-05-27T09:25:22.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.428+0000 7fbd82a00700 1 -- 172.21.15.119:0/615777271 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fbd780104f0 con 0x7fbd88100d70 2024-05-27T09:25:22.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.429+0000 7fbd82a00700 1 --2- 172.21.15.119:0/615777271 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbd7006c680 0x7fbd7006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:22.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.429+0000 7fbd82a00700 1 -- 172.21.15.119:0/615777271 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbd7808b220 con 0x7fbd88100d70 2024-05-27T09:25:22.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.429+0000 7fbd8de00700 1 --2- 172.21.15.119:0/615777271 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbd7006c680 0x7fbd7006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:22.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.430+0000 7fbd8de00700 1 --2- 172.21.15.119:0/615777271 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbd7006c680 0x7fbd7006eb30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c006010 tx=0x7fbd7c01a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:22.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.434+0000 7fbd82a00700 1 -- 172.21.15.119:0/615777271 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbd78055ba0 con 0x7fbd88100d70 2024-05-27T09:25:22.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.641+0000 7fbd90800700 1 -- 172.21.15.119:0/615777271 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbd8819d500 con 0x7fbd7006c680 2024-05-27T09:25:22.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.646+0000 7fbd82a00700 1 -- 172.21.15.119:0/615777271 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27160 (secure 0 0 0) 0x7fbd8819d500 con 0x7fbd7006c680 2024-05-27T09:25:22.648 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:25:22.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.649+0000 7fbd90800700 1 -- 172.21.15.119:0/615777271 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbd7006c680 msgr2=0x7fbd7006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:22.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.649+0000 7fbd90800700 1 --2- 172.21.15.119:0/615777271 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbd7006c680 0x7fbd7006eb30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c006010 tx=0x7fbd7c01a040 comp rx=0 tx=0).stop 2024-05-27T09:25:22.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.649+0000 7fbd90800700 1 -- 172.21.15.119:0/615777271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd88100d70 msgr2=0x7fbd88198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:22.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.649+0000 7fbd90800700 1 --2- 172.21.15.119:0/615777271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd88100d70 0x7fbd88198560 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800b480 tx=0x7fbd7800b840 comp rx=0 tx=0).stop 2024-05-27T09:25:22.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.649+0000 7fbd90800700 1 -- 172.21.15.119:0/615777271 shutdown_connections 2024-05-27T09:25:22.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.649+0000 7fbd90800700 1 --2- 172.21.15.119:0/615777271 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbd7006c680 0x7fbd7006eb30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:22.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.649+0000 7fbd90800700 1 --2- 172.21.15.119:0/615777271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd881003d0 0x7fbd88198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:22.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.649+0000 7fbd90800700 1 --2- 172.21.15.119:0/615777271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd88100d70 0x7fbd88198560 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:22.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.649+0000 7fbd90800700 1 -- 172.21.15.119:0/615777271 >> 172.21.15.119:0/615777271 conn(0x7fbd880fbc90 msgr2=0x7fbd880fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:22.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.649+0000 7fbd90800700 1 -- 172.21.15.119:0/615777271 shutdown_connections 2024-05-27T09:25:22.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:22.649+0000 7fbd90800700 1 -- 172.21.15.119:0/615777271 wait complete. 2024-05-27T09:25:22.652 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-05-27T09:25:23.386 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-05-27T09:25:22.259694+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165584,"kb_used_data":3872,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305328,"statfs":{"total":575970213888,"available":575800655872,"internally_reserved":0,"allocated":3964928,"data_stored":2608497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003233"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":186,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-27T09:24:21.330494+0000","last_change":"2024-05-27T09:23:45.322438+0000","last_active":"2024-05-27T09:24:21.330494+0000","last_peered":"2024-05-27T09:24:21.330494+0000","last_clean":"2024-05-27T09:24:21.330494+0000","last_became_active":"2024-05-27T09:23:45.321314+0000","last_became_peered":"2024-05-27T09:23:45.321314+0000","last_unstale":"2024-05-27T09:24:21.330494+0000","last_undegraded":"2024-05-27T09:24:21.330494+0000","last_fullsized":"2024-05-27T09:24:21.330494+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T09:22:34.972595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T09:22:34.972595+0000","last_clean_scrub_stamp":"2024-05-27T09:22:34.972595+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:24:15.456898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0369999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]}]},{"osd":4,"up_from":34,"seq":146028888086,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 27 09:24:46 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":1.0840000000000001}]},{"osd":1,"last update":"Mon May 27 09:24:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.0620000000000001}]},{"osd":2,"last update":"Mon May 27 09:24:46 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":1.0409999999999999}]},{"osd":3,"last update":"Mon May 27 09:24:46 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":0.88300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]}]},{"osd":3,"up_from":26,"seq":111669149726,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 27 09:25:10 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.93700000000000006,"15min":0.92100000000000004},"min":{"1min":0.72299999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.61299999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.97299999999999998,"15min":0.96399999999999997},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.93100000000000005}]},{"osd":1,"last update":"Mon May 27 09:25:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.018,"15min":1.0029999999999999},"min":{"1min":0.77600000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.995},{"interface":"front","average":{"1min":1.097,"5min":1.0129999999999999,"15min":0.999},"min":{"1min":0.81000000000000005,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.70199999999999996}]},{"osd":2,"last update":"Mon May 27 09:25:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.0529999999999999,"15min":1.0429999999999999},"min":{"1min":0.76900000000000002,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1259999999999999,"15min":1.1200000000000001},"min":{"1min":0.76100000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.78900000000000003}]},{"osd":4,"last update":"Mon May 27 09:24:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.165}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]}]},{"osd":2,"up_from":19,"seq":81604378659,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27304,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717848,"statfs":{"total":95995035648,"available":95967076352,"internally_reserved":0,"allocated":360448,"data_stored":139008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 27 09:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.79700000000000004,"15min":0.79000000000000004},"min":{"1min":0.66200000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.85599999999999998,"15min":0.86099999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.093,"5min":1.169,"15min":1.169},"last":0.79600000000000004}]},{"osd":1,"last update":"Mon May 27 09:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.88600000000000001,"15min":0.87},"min":{"1min":0.755,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.96899999999999997,"15min":0.95499999999999996},"min":{"1min":0.89700000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.224}]},{"osd":3,"last update":"Mon May 27 09:25:12 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.1539999999999999,"15min":1.159},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.415,"5min":1.591,"15min":1.591},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.1160000000000001,"15min":1.1200000000000001},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.2869999999999999}]},{"osd":4,"last update":"Mon May 27 09:24:48 2024","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.579},{"interface":"front","average":{"1min":1.264,"5min":1.264,"15min":1.264},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.4930000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.508}]}]},{"osd":0,"up_from":9,"seq":38654705713,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon May 27 09:25:00 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.74099999999999999,"15min":0.70099999999999996},"min":{"1min":0.442,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.80300000000000005,"15min":0.77200000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.135,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.54200000000000004}]},{"osd":2,"last update":"Mon May 27 09:24:37 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.86799999999999999,"15min":0.85999999999999999},"min":{"1min":0.51700000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.98099999999999998,"15min":0.97899999999999998},"min":{"1min":0.54000000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.325,"5min":1.865,"15min":1.865},"last":0.66500000000000004}]},{"osd":3,"last update":"Mon May 27 09:25:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0900000000000001,"15min":1.1000000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.2789999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.056},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.1419999999999999,"15min":1.1259999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":3.6509999999999998,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.875}]},{"osd":4,"last update":"Mon May 27 09:24:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.97199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]}]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon May 27 09:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77200000000000002,"15min":0.751},"min":{"1min":0.40999999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.046,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.0840000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.83599999999999997,"15min":0.82399999999999995},"min":{"1min":0.51700000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.4059999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.78000000000000003}]},{"osd":2,"last update":"Mon May 27 09:24:38 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.92700000000000005,"15min":0.92800000000000005},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.99399999999999999,"15min":1.0009999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2010000000000001,"5min":1.23,"15min":1.23},"last":1.343}]},{"osd":3,"last update":"Mon May 27 09:25:12 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0389999999999999,"15min":1.0349999999999999},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":1.151},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0660000000000001,"15min":1.0669999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.2310000000000001}]},{"osd":4,"last update":"Mon May 27 09:24:47 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.49},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.3049999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T09:25:23.387 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-27T09:25:23.387 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-27T09:25:23.387 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-27T09:25:23.387 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-05-27T09:25:23.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:23 smithi138 ceph-mon[34885]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:23.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:23 smithi138 ceph-mon[34885]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T09:25:23.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:23 smithi119 ceph-mon[29082]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:23.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:23 smithi119 ceph-mon[29082]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T09:25:23.790 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:25:25.410 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:25 smithi119 ceph-mon[29082]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:25.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:25 smithi138 ceph-mon[34885]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:25.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.876+0000 7f8278200700 1 -- 172.21.15.119:0/954629197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82700730f0 msgr2=0x7f82700734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:25.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.876+0000 7f8278200700 1 --2- 172.21.15.119:0/954629197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82700730f0 0x7f82700734c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f826400b0a0 tx=0x7f826400b3b0 comp rx=0 tx=0).stop 2024-05-27T09:25:25.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.876+0000 7f8278200700 1 -- 172.21.15.119:0/954629197 shutdown_connections 2024-05-27T09:25:25.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.876+0000 7f8278200700 1 --2- 172.21.15.119:0/954629197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8270073a00 0x7f827010c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:25.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.876+0000 7f8278200700 1 --2- 172.21.15.119:0/954629197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82700730f0 0x7f82700734c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:25.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.877+0000 7f8278200700 1 -- 172.21.15.119:0/954629197 >> 172.21.15.119:0/954629197 conn(0x7f82700fbc30 msgr2=0x7f82700fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:25.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.877+0000 7f8278200700 1 -- 172.21.15.119:0/954629197 shutdown_connections 2024-05-27T09:25:25.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.877+0000 7f8278200700 1 -- 172.21.15.119:0/954629197 wait complete. 2024-05-27T09:25:25.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.878+0000 7f8278200700 1 Processor -- start 2024-05-27T09:25:25.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.878+0000 7f8278200700 1 -- start start 2024-05-27T09:25:25.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.878+0000 7f8278200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82700730f0 0x7f8270197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:25.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.878+0000 7f8278200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8270073a00 0x7f82701984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:25.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.878+0000 7f8278200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827019c810 con 0x7f82700730f0 2024-05-27T09:25:25.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.879+0000 7f8278200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827019c980 con 0x7f8270073a00 2024-05-27T09:25:25.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.879+0000 7f8275800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82700730f0 0x7f8270197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:25.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.879+0000 7f8275800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82700730f0 0x7f8270197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54392/0 (socket says 172.21.15.119:54392) 2024-05-27T09:25:25.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.879+0000 7f8275800700 1 -- 172.21.15.119:0/949915598 learned_addr learned my addr 172.21.15.119:0/949915598 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:25:25.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.879+0000 7f8275800700 1 -- 172.21.15.119:0/949915598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8270073a00 msgr2=0x7f82701984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:25.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.879+0000 7f8274e00700 1 --2- 172.21.15.119:0/949915598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8270073a00 0x7f82701984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:25.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.879+0000 7f8275800700 1 --2- 172.21.15.119:0/949915598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8270073a00 0x7f82701984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:25.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.879+0000 7f8275800700 1 -- 172.21.15.119:0/949915598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8264009d00 con 0x7f82700730f0 2024-05-27T09:25:25.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.880+0000 7f8275800700 1 --2- 172.21.15.119:0/949915598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82700730f0 0x7f8270197e60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f826400b070 tx=0x7f8264003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:25.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.880+0000 7f826e000700 1 -- 172.21.15.119:0/949915598 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f826400e040 con 0x7f82700730f0 2024-05-27T09:25:25.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.880+0000 7f826e000700 1 -- 172.21.15.119:0/949915598 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8264007d60 con 0x7f82700730f0 2024-05-27T09:25:25.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.880+0000 7f826e000700 1 -- 172.21.15.119:0/949915598 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f826401d8b0 con 0x7f82700730f0 2024-05-27T09:25:25.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.880+0000 7f8278200700 1 -- 172.21.15.119:0/949915598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f827019cc00 con 0x7f82700730f0 2024-05-27T09:25:25.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.880+0000 7f8278200700 1 -- 172.21.15.119:0/949915598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f827019d150 con 0x7f82700730f0 2024-05-27T09:25:25.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.881+0000 7f8278200700 1 -- 172.21.15.119:0/949915598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8270109b50 con 0x7f82700730f0 2024-05-27T09:25:25.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.882+0000 7f826e000700 1 -- 172.21.15.119:0/949915598 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f8264004780 con 0x7f82700730f0 2024-05-27T09:25:25.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.883+0000 7f826e000700 1 --2- 172.21.15.119:0/949915598 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f825806c680 0x7f825806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:25.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.883+0000 7f826e000700 1 -- 172.21.15.119:0/949915598 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f826408cda0 con 0x7f82700730f0 2024-05-27T09:25:25.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.883+0000 7f8274e00700 1 --2- 172.21.15.119:0/949915598 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f825806c680 0x7f825806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:25.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.884+0000 7f8274e00700 1 --2- 172.21.15.119:0/949915598 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f825806c680 0x7f825806eb30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f825c005950 tx=0x7f825c005cf0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:25.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:25.888+0000 7f826e000700 1 -- 172.21.15.119:0/949915598 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f82640577a0 con 0x7f82700730f0 2024-05-27T09:25:26.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:26.170+0000 7f8278200700 1 -- 172.21.15.119:0/949915598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8270199310 con 0x7f82700730f0 2024-05-27T09:25:26.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:26.170+0000 7f826e000700 1 -- 172.21.15.119:0/949915598 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f8270199310 con 0x7f82700730f0 2024-05-27T09:25:26.171 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:25:26.171 INFO:teuthology.orchestra.run.smithi119.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-27T09:25:26.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:26.173+0000 7f8278200700 1 -- 172.21.15.119:0/949915598 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f825806c680 msgr2=0x7f825806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:26.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:26.173+0000 7f8278200700 1 --2- 172.21.15.119:0/949915598 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f825806c680 0x7f825806eb30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f825c005950 tx=0x7f825c005cf0 comp rx=0 tx=0).stop 2024-05-27T09:25:26.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:26.173+0000 7f8278200700 1 -- 172.21.15.119:0/949915598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82700730f0 msgr2=0x7f8270197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:26.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:26.173+0000 7f8278200700 1 --2- 172.21.15.119:0/949915598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82700730f0 0x7f8270197e60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f826400b070 tx=0x7f8264003ce0 comp rx=0 tx=0).stop 2024-05-27T09:25:26.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:26.174+0000 7f8278200700 1 -- 172.21.15.119:0/949915598 shutdown_connections 2024-05-27T09:25:26.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:26.174+0000 7f8278200700 1 --2- 172.21.15.119:0/949915598 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f825806c680 0x7f825806eb30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:26.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:26.174+0000 7f8278200700 1 --2- 172.21.15.119:0/949915598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82700730f0 0x7f8270197e60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:26.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:26.174+0000 7f8278200700 1 --2- 172.21.15.119:0/949915598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8270073a00 0x7f82701984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:26.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:26.174+0000 7f8278200700 1 -- 172.21.15.119:0/949915598 >> 172.21.15.119:0/949915598 conn(0x7f82700fbc30 msgr2=0x7f8270106c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:26.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:26.174+0000 7f8278200700 1 -- 172.21.15.119:0/949915598 shutdown_connections 2024-05-27T09:25:26.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:26.174+0000 7f8278200700 1 -- 172.21.15.119:0/949915598 wait complete. 2024-05-27T09:25:26.989 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-27T09:25:26.989 INFO:tasks.cephadm:Setup complete, yielding 2024-05-27T09:25:26.989 INFO:teuthology.run_tasks:Running task print... 2024-05-27T09:25:26.999 INFO:teuthology.task.print:**** done end installing v18.2.0 cephadm ... 2024-05-27T09:25:26.999 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-27T09:25:27.020 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi119.front.sepia.ceph.com 2024-05-27T09:25:27.020 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-05-27T09:25:27.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:26 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:25:27.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:26 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/949915598' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-27T09:25:27.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:26 smithi119 ceph-mon[29082]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:27.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:26 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:25:27.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:26 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/949915598' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-27T09:25:27.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:26 smithi138 ceph-mon[34885]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:27.405 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:25:28.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:27 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:25:28.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:27 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:25:28.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:27 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:25:28.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:27 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:25:28.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:27 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:25:28.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:27 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:25:29.158 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:28 smithi119 ceph-mon[29082]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:29.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:28 smithi138 ceph-mon[34885]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:29.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.625+0000 7f03b9e00700 1 -- 172.21.15.119:0/347863475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b41033c0 msgr2=0x7f03b4103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:29.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.625+0000 7f03b9e00700 1 --2- 172.21.15.119:0/347863475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b41033c0 0x7f03b4103810 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009910 tx=0x7f03a0009c20 comp rx=0 tx=0).stop 2024-05-27T09:25:29.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.625+0000 7f03b9e00700 1 -- 172.21.15.119:0/347863475 shutdown_connections 2024-05-27T09:25:29.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.625+0000 7f03b9e00700 1 --2- 172.21.15.119:0/347863475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b41033c0 0x7f03b4103810 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:29.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.625+0000 7f03b9e00700 1 --2- 172.21.15.119:0/347863475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03b41021c0 0x7f03b41025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:29.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.625+0000 7f03b9e00700 1 -- 172.21.15.119:0/347863475 >> 172.21.15.119:0/347863475 conn(0x7f03b40fd790 msgr2=0x7f03b40ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:29.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.626+0000 7f03b9e00700 1 -- 172.21.15.119:0/347863475 shutdown_connections 2024-05-27T09:25:29.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.626+0000 7f03b9e00700 1 -- 172.21.15.119:0/347863475 wait complete. 2024-05-27T09:25:29.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.627+0000 7f03b9e00700 1 Processor -- start 2024-05-27T09:25:29.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.627+0000 7f03b9e00700 1 -- start start 2024-05-27T09:25:29.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.627+0000 7f03b9e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03b41021c0 0x7f03b4078b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:29.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.627+0000 7f03b9e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b41033c0 0x7f03b4079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:29.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.627+0000 7f03b9e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b40755f0 con 0x7f03b41033c0 2024-05-27T09:25:29.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.627+0000 7f03b9e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b4075760 con 0x7f03b41021c0 2024-05-27T09:25:29.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.627+0000 7f03b3e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b41033c0 0x7f03b4079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:29.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.627+0000 7f03b3e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b41033c0 0x7f03b4079040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60492/0 (socket says 172.21.15.119:60492) 2024-05-27T09:25:29.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.627+0000 7f03b3e00700 1 -- 172.21.15.119:0/3427487506 learned_addr learned my addr 172.21.15.119:0/3427487506 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:25:29.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.628+0000 7f03b3e00700 1 -- 172.21.15.119:0/3427487506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03b41021c0 msgr2=0x7f03b4078b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:29.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.628+0000 7f03b3e00700 1 --2- 172.21.15.119:0/3427487506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03b41021c0 0x7f03b4078b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:29.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.628+0000 7f03b3e00700 1 -- 172.21.15.119:0/3427487506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a00095f0 con 0x7f03b41033c0 2024-05-27T09:25:29.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.628+0000 7f03b3e00700 1 --2- 172.21.15.119:0/3427487506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b41033c0 0x7f03b4079040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f03a0005b40 tx=0x7f03a000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:29.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.628+0000 7f03b1600700 1 -- 172.21.15.119:0/3427487506 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03a001d070 con 0x7f03b41033c0 2024-05-27T09:25:29.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.628+0000 7f03b1600700 1 -- 172.21.15.119:0/3427487506 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f03a0022470 con 0x7f03b41033c0 2024-05-27T09:25:29.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.628+0000 7f03b1600700 1 -- 172.21.15.119:0/3427487506 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03a000f690 con 0x7f03b41033c0 2024-05-27T09:25:29.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.628+0000 7f03b9e00700 1 -- 172.21.15.119:0/3427487506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b40759e0 con 0x7f03b41033c0 2024-05-27T09:25:29.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.629+0000 7f03b9e00700 1 -- 172.21.15.119:0/3427487506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b4075f50 con 0x7f03b41033c0 2024-05-27T09:25:29.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.630+0000 7f03b9e00700 1 -- 172.21.15.119:0/3427487506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b4076170 con 0x7f03b41033c0 2024-05-27T09:25:29.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.631+0000 7f03b1600700 1 -- 172.21.15.119:0/3427487506 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f03a00225e0 con 0x7f03b41033c0 2024-05-27T09:25:29.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.632+0000 7f03b1600700 1 --2- 172.21.15.119:0/3427487506 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f039c06c6d0 0x7f039c06eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:29.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.632+0000 7f03b1600700 1 -- 172.21.15.119:0/3427487506 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f03a008d5b0 con 0x7f03b41033c0 2024-05-27T09:25:29.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.632+0000 7f03b8a00700 1 --2- 172.21.15.119:0/3427487506 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f039c06c6d0 0x7f039c06eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:29.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.633+0000 7f03b8a00700 1 --2- 172.21.15.119:0/3427487506 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f039c06c6d0 0x7f039c06eb80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f03b4103220 tx=0x7f03a8006d20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:29.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.636+0000 7f03b1600700 1 -- 172.21.15.119:0/3427487506 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f03a005bac0 con 0x7f03b41033c0 2024-05-27T09:25:29.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.853+0000 7f03b9e00700 1 -- 172.21.15.119:0/3427487506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f03b41a2320 con 0x7f03b41033c0 2024-05-27T09:25:29.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.859+0000 7f03b1600700 1 -- 172.21.15.119:0/3427487506 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f03a005b650 con 0x7f03b41033c0 2024-05-27T09:25:29.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.862+0000 7f03b9e00700 1 -- 172.21.15.119:0/3427487506 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f039c06c6d0 msgr2=0x7f039c06eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:29.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.862+0000 7f03b9e00700 1 --2- 172.21.15.119:0/3427487506 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f039c06c6d0 0x7f039c06eb80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f03b4103220 tx=0x7f03a8006d20 comp rx=0 tx=0).stop 2024-05-27T09:25:29.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.862+0000 7f03b9e00700 1 -- 172.21.15.119:0/3427487506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b41033c0 msgr2=0x7f03b4079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:29.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.862+0000 7f03b9e00700 1 --2- 172.21.15.119:0/3427487506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b41033c0 0x7f03b4079040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f03a0005b40 tx=0x7f03a000be20 comp rx=0 tx=0).stop 2024-05-27T09:25:29.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.863+0000 7f03b9e00700 1 -- 172.21.15.119:0/3427487506 shutdown_connections 2024-05-27T09:25:29.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.863+0000 7f03b9e00700 1 --2- 172.21.15.119:0/3427487506 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f039c06c6d0 0x7f039c06eb80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:29.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.863+0000 7f03b9e00700 1 --2- 172.21.15.119:0/3427487506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03b41021c0 0x7f03b4078b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:29.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.863+0000 7f03b9e00700 1 --2- 172.21.15.119:0/3427487506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b41033c0 0x7f03b4079040 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:29.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.863+0000 7f03b9e00700 1 -- 172.21.15.119:0/3427487506 >> 172.21.15.119:0/3427487506 conn(0x7f03b40fd790 msgr2=0x7f03b41065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:29.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.863+0000 7f03b9e00700 1 -- 172.21.15.119:0/3427487506 shutdown_connections 2024-05-27T09:25:29.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:29.863+0000 7f03b9e00700 1 -- 172.21.15.119:0/3427487506 wait complete. 2024-05-27T09:25:30.705 INFO:teuthology.run_tasks:Running task print... 2024-05-27T09:25:30.718 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-05-27T09:25:30.718 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-27T09:25:30.726 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi119.front.sepia.ceph.com 2024-05-27T09:25:30.727 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch status' 2024-05-27T09:25:30.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:30 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/3427487506' entity='client.admin' 2024-05-27T09:25:30.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:30 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:25:30.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:30 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:25:30.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:30 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:25:30.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:30 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:25:30.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:30 smithi119 ceph-mon[29082]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:31.132 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:25:31.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:30 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/3427487506' entity='client.admin' 2024-05-27T09:25:31.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:30 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:25:31.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:30 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:25:31.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:30 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:25:31.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:30 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:25:31.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:30 smithi138 ceph-mon[34885]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.333+0000 7fc425a00700 1 -- 172.21.15.119:0/3570584525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc420100be0 msgr2=0x7fc420102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.333+0000 7fc425a00700 1 --2- 172.21.15.119:0/3570584525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc420100be0 0x7fc420102fc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc410009910 tx=0x7fc410009c20 comp rx=0 tx=0).stop 2024-05-27T09:25:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.333+0000 7fc425a00700 1 -- 172.21.15.119:0/3570584525 shutdown_connections 2024-05-27T09:25:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.333+0000 7fc425a00700 1 --2- 172.21.15.119:0/3570584525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc420103500 0x7fc4201058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.333+0000 7fc425a00700 1 --2- 172.21.15.119:0/3570584525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc420100be0 0x7fc420102fc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.333+0000 7fc425a00700 1 -- 172.21.15.119:0/3570584525 >> 172.21.15.119:0/3570584525 conn(0x7fc4200fa5e0 msgr2=0x7fc4200fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.333+0000 7fc425a00700 1 -- 172.21.15.119:0/3570584525 shutdown_connections 2024-05-27T09:25:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.333+0000 7fc425a00700 1 -- 172.21.15.119:0/3570584525 wait complete. 2024-05-27T09:25:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.335+0000 7fc425a00700 1 Processor -- start 2024-05-27T09:25:33.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.335+0000 7fc425a00700 1 -- start start 2024-05-27T09:25:33.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.335+0000 7fc425a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc420100be0 0x7fc420197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:33.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.335+0000 7fc425a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc420103500 0x7fc420198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:33.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.335+0000 7fc425a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc420198660 con 0x7fc420103500 2024-05-27T09:25:33.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.335+0000 7fc425a00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4201987a0 con 0x7fc420100be0 2024-05-27T09:25:33.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.335+0000 7fc417e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc420103500 0x7fc420198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:33.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.335+0000 7fc417e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc420103500 0x7fc420198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60518/0 (socket says 172.21.15.119:60518) 2024-05-27T09:25:33.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.335+0000 7fc417e00700 1 -- 172.21.15.119:0/3117780708 learned_addr learned my addr 172.21.15.119:0/3117780708 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:25:33.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.336+0000 7fc41ea00700 1 --2- 172.21.15.119:0/3117780708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc420100be0 0x7fc420197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:33.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.336+0000 7fc417e00700 1 -- 172.21.15.119:0/3117780708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc420100be0 msgr2=0x7fc420197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:33.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.336+0000 7fc417e00700 1 --2- 172.21.15.119:0/3117780708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc420100be0 0x7fc420197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:33.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.336+0000 7fc417e00700 1 -- 172.21.15.119:0/3117780708 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4100095f0 con 0x7fc420103500 2024-05-27T09:25:33.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.336+0000 7fc417e00700 1 --2- 172.21.15.119:0/3117780708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc420103500 0x7fc420198040 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc40800c8a0 tx=0x7fc40800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:33.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.336+0000 7fc417400700 1 -- 172.21.15.119:0/3117780708 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc40800cea0 con 0x7fc420103500 2024-05-27T09:25:33.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.336+0000 7fc417400700 1 -- 172.21.15.119:0/3117780708 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc408004830 con 0x7fc420103500 2024-05-27T09:25:33.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.337+0000 7fc425a00700 1 -- 172.21.15.119:0/3117780708 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc42019d250 con 0x7fc420103500 2024-05-27T09:25:33.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.337+0000 7fc417400700 1 -- 172.21.15.119:0/3117780708 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4080055b0 con 0x7fc420103500 2024-05-27T09:25:33.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.337+0000 7fc425a00700 1 -- 172.21.15.119:0/3117780708 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc42019d7a0 con 0x7fc420103500 2024-05-27T09:25:33.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.338+0000 7fc425a00700 1 -- 172.21.15.119:0/3117780708 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4200fc1c0 con 0x7fc420103500 2024-05-27T09:25:33.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.339+0000 7fc417400700 1 -- 172.21.15.119:0/3117780708 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fc408007ad0 con 0x7fc420103500 2024-05-27T09:25:33.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.340+0000 7fc417400700 1 --2- 172.21.15.119:0/3117780708 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc40406c680 0x7fc40406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:33.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.340+0000 7fc417400700 1 -- 172.21.15.119:0/3117780708 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc40808b1f0 con 0x7fc420103500 2024-05-27T09:25:33.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.340+0000 7fc41ea00700 1 --2- 172.21.15.119:0/3117780708 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc40406c680 0x7fc40406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:33.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.341+0000 7fc41ea00700 1 --2- 172.21.15.119:0/3117780708 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc40406c680 0x7fc40406eb30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc410006010 tx=0x7fc41000b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:33.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.345+0000 7fc417400700 1 -- 172.21.15.119:0/3117780708 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc408059720 con 0x7fc420103500 2024-05-27T09:25:33.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:33 smithi119 ceph-mon[29082]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:33.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.579+0000 7fc425a00700 1 -- 172.21.15.119:0/3117780708 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc420061190 con 0x7fc40406c680 2024-05-27T09:25:33.582 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.581+0000 7fc417400700 1 -- 172.21.15.119:0/3117780708 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fc420061190 con 0x7fc40406c680 2024-05-27T09:25:33.582 INFO:teuthology.orchestra.run.smithi119.stdout:Backend: cephadm 2024-05-27T09:25:33.582 INFO:teuthology.orchestra.run.smithi119.stdout:Available: Yes 2024-05-27T09:25:33.582 INFO:teuthology.orchestra.run.smithi119.stdout:Paused: No 2024-05-27T09:25:33.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.584+0000 7fc425a00700 1 -- 172.21.15.119:0/3117780708 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc40406c680 msgr2=0x7fc40406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:33.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.584+0000 7fc425a00700 1 --2- 172.21.15.119:0/3117780708 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc40406c680 0x7fc40406eb30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc410006010 tx=0x7fc41000b540 comp rx=0 tx=0).stop 2024-05-27T09:25:33.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.584+0000 7fc425a00700 1 -- 172.21.15.119:0/3117780708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc420103500 msgr2=0x7fc420198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:33.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.584+0000 7fc425a00700 1 --2- 172.21.15.119:0/3117780708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc420103500 0x7fc420198040 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc40800c8a0 tx=0x7fc40800cc60 comp rx=0 tx=0).stop 2024-05-27T09:25:33.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.585+0000 7fc425a00700 1 -- 172.21.15.119:0/3117780708 shutdown_connections 2024-05-27T09:25:33.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.585+0000 7fc425a00700 1 --2- 172.21.15.119:0/3117780708 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc40406c680 0x7fc40406eb30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:33.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.585+0000 7fc425a00700 1 --2- 172.21.15.119:0/3117780708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc420100be0 0x7fc420197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:33.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.585+0000 7fc425a00700 1 --2- 172.21.15.119:0/3117780708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc420103500 0x7fc420198040 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:33.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.585+0000 7fc425a00700 1 -- 172.21.15.119:0/3117780708 >> 172.21.15.119:0/3117780708 conn(0x7fc4200fa5e0 msgr2=0x7fc4200ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:33.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.585+0000 7fc425a00700 1 -- 172.21.15.119:0/3117780708 shutdown_connections 2024-05-27T09:25:33.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:33.585+0000 7fc425a00700 1 -- 172.21.15.119:0/3117780708 wait complete. 2024-05-27T09:25:33.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:33 smithi138 ceph-mon[34885]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:34.432 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-27T09:25:34.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:34 smithi138 ceph-mon[34885]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:25:34.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:34 smithi119 ceph-mon[29082]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:25:34.835 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:25:35.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:35 smithi138 ceph-mon[34885]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:35.712 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:35 smithi119 ceph-mon[29082]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:36.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.909+0000 7f018d400700 1 -- 172.21.15.119:0/1944547168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01881067a0 msgr2=0x7f0188106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:36.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.909+0000 7f018d400700 1 --2- 172.21.15.119:0/1944547168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01881067a0 0x7f0188106bb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0178009910 tx=0x7f0178009c20 comp rx=0 tx=0).stop 2024-05-27T09:25:36.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.909+0000 7f018d400700 1 -- 172.21.15.119:0/1944547168 shutdown_connections 2024-05-27T09:25:36.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.909+0000 7f018d400700 1 --2- 172.21.15.119:0/1944547168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01881079a0 0x7f0188107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:36.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.909+0000 7f018d400700 1 --2- 172.21.15.119:0/1944547168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01881067a0 0x7f0188106bb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:36.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.909+0000 7f018d400700 1 -- 172.21.15.119:0/1944547168 >> 172.21.15.119:0/1944547168 conn(0x7f0188075910 msgr2=0x7f0188077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:36.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.909+0000 7f018d400700 1 -- 172.21.15.119:0/1944547168 shutdown_connections 2024-05-27T09:25:36.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.909+0000 7f018d400700 1 -- 172.21.15.119:0/1944547168 wait complete. 2024-05-27T09:25:36.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.910+0000 7f018d400700 1 Processor -- start 2024-05-27T09:25:36.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.910+0000 7f018d400700 1 -- start start 2024-05-27T09:25:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.910+0000 7f018d400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01881067a0 0x7f018819bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.910+0000 7f018d400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01881079a0 0x7f018819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.910+0000 7f018d400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018819cb10 con 0x7f01881067a0 2024-05-27T09:25:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.910+0000 7f018d400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018819cc50 con 0x7f01881079a0 2024-05-27T09:25:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.911+0000 7f0186800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01881067a0 0x7f018819bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.911+0000 7f0186800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01881067a0 0x7f018819bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60534/0 (socket says 172.21.15.119:60534) 2024-05-27T09:25:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.911+0000 7f0186800700 1 -- 172.21.15.119:0/2433664718 learned_addr learned my addr 172.21.15.119:0/2433664718 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:25:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.911+0000 7f0185e00700 1 --2- 172.21.15.119:0/2433664718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01881079a0 0x7f018819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.911+0000 7f0186800700 1 -- 172.21.15.119:0/2433664718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01881079a0 msgr2=0x7f018819c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.911+0000 7f0186800700 1 --2- 172.21.15.119:0/2433664718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01881079a0 0x7f018819c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.911+0000 7f0186800700 1 -- 172.21.15.119:0/2433664718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01780095f0 con 0x7f01881067a0 2024-05-27T09:25:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.911+0000 7f0186800700 1 --2- 172.21.15.119:0/2433664718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01881067a0 0x7f018819bfb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f017800bd70 tx=0x7f017800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.912+0000 7f017f400700 1 -- 172.21.15.119:0/2433664718 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f017801d070 con 0x7f01881067a0 2024-05-27T09:25:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.912+0000 7f017f400700 1 -- 172.21.15.119:0/2433664718 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f017800f460 con 0x7f01881067a0 2024-05-27T09:25:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.912+0000 7f018d400700 1 -- 172.21.15.119:0/2433664718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01881a16a0 con 0x7f01881067a0 2024-05-27T09:25:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.912+0000 7f017f400700 1 -- 172.21.15.119:0/2433664718 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0178021640 con 0x7f01881067a0 2024-05-27T09:25:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.912+0000 7f018d400700 1 -- 172.21.15.119:0/2433664718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01881a1b90 con 0x7f01881067a0 2024-05-27T09:25:36.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.913+0000 7f018d400700 1 -- 172.21.15.119:0/2433664718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f018810f2f0 con 0x7f01881067a0 2024-05-27T09:25:36.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.914+0000 7f017f400700 1 -- 172.21.15.119:0/2433664718 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f017800fab0 con 0x7f01881067a0 2024-05-27T09:25:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.915+0000 7f017f400700 1 --2- 172.21.15.119:0/2433664718 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f016c06c680 0x7f016c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.916+0000 7f017f400700 1 -- 172.21.15.119:0/2433664718 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f017808c9a0 con 0x7f01881067a0 2024-05-27T09:25:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.916+0000 7f0185e00700 1 --2- 172.21.15.119:0/2433664718 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f016c06c680 0x7f016c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.916+0000 7f0185e00700 1 --2- 172.21.15.119:0/2433664718 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f016c06c680 0x7f016c06eb30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0174005fd0 tx=0x7f017400a560 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:36.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:36.920+0000 7f017f400700 1 -- 172.21.15.119:0/2433664718 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f017805ae30 con 0x7f01881067a0 2024-05-27T09:25:36.987 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:36 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:25:36.988 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:36 smithi119 ceph-mon[29082]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:37.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:36 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:25:37.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:36 smithi138 ceph-mon[34885]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:37.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:37.147+0000 7f018d400700 1 -- 172.21.15.119:0/2433664718 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f01881a2290 con 0x7f016c06c680 2024-05-27T09:25:37.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:37.173+0000 7f017f400700 1 -- 172.21.15.119:0/2433664718 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2860 (secure 0 0 0) 0x7f01881a2290 con 0x7f016c06c680 2024-05-27T09:25:37.175 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:25:37.175 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (4m) 2m ago 6m 13.9M - 0.25.0 c8568f914cd2 bc280230ab28 2024-05-27T09:25:37.175 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (7m) 2m ago 7m 7063k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:25:37.175 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (5m) 76s ago 5m 6979k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:25:37.175 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (7m) 2m ago 7m 6665k - 18.2.0 dc2bc1663786 e5fafde44183 2024-05-27T09:25:37.175 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (5m) 76s ago 5m 6660k - 18.2.0 dc2bc1663786 46956ab99bab 2024-05-27T09:25:37.175 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (4m) 2m ago 6m 75.3M - 9.4.7 954c08fa6188 16eeaded4295 2024-05-27T09:25:37.175 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:9283,8765,8443 running (8m) 2m ago 8m 480M - 18.2.0 dc2bc1663786 abe5e83463f2 2024-05-27T09:25:37.175 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (5m) 76s ago 5m 436M - 18.2.0 dc2bc1663786 5aa669adddf7 2024-05-27T09:25:37.176 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (8m) 2m ago 8m 44.5M 2048M 18.2.0 dc2bc1663786 ba998b789a1c 2024-05-27T09:25:37.176 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (5m) 76s ago 5m 35.5M 2048M 18.2.0 dc2bc1663786 60216fd67078 2024-05-27T09:25:37.176 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (6m) 2m ago 6m 7658k - 1.5.0 0da6a335fe13 79c36d4e5b04 2024-05-27T09:25:37.176 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (5m) 76s ago 5m 7692k - 1.5.0 0da6a335fe13 80ee890e8830 2024-05-27T09:25:37.176 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (4m) 2m ago 4m 38.9M 3947M 18.2.0 dc2bc1663786 825bd1392508 2024-05-27T09:25:37.176 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (3m) 2m ago 3m 39.0M 3947M 18.2.0 dc2bc1663786 fce9819903d9 2024-05-27T09:25:37.176 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (3m) 2m ago 3m 39.6M 3947M 18.2.0 dc2bc1663786 3955a6112c1c 2024-05-27T09:25:37.176 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (2m) 76s ago 2m 40.9M 4971M 18.2.0 dc2bc1663786 693c92220622 2024-05-27T09:25:37.176 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (119s) 76s ago 119s 37.2M 4971M 18.2.0 dc2bc1663786 7559e658e917 2024-05-27T09:25:37.176 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (83s) 76s ago 82s 11.0M 4971M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:25:37.176 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (4m) 2m ago 6m 40.8M - 2.43.0 a07b618ecd1d 7f3d89d1f425 2024-05-27T09:25:37.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:37.177+0000 7f018d400700 1 -- 172.21.15.119:0/2433664718 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f016c06c680 msgr2=0x7f016c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:37.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:37.177+0000 7f018d400700 1 --2- 172.21.15.119:0/2433664718 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f016c06c680 0x7f016c06eb30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0174005fd0 tx=0x7f017400a560 comp rx=0 tx=0).stop 2024-05-27T09:25:37.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:37.177+0000 7f018d400700 1 -- 172.21.15.119:0/2433664718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01881067a0 msgr2=0x7f018819bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:37.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:37.177+0000 7f018d400700 1 --2- 172.21.15.119:0/2433664718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01881067a0 0x7f018819bfb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f017800bd70 tx=0x7f017800be50 comp rx=0 tx=0).stop 2024-05-27T09:25:37.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:37.177+0000 7f018d400700 1 -- 172.21.15.119:0/2433664718 shutdown_connections 2024-05-27T09:25:37.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:37.177+0000 7f018d400700 1 --2- 172.21.15.119:0/2433664718 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f016c06c680 0x7f016c06eb30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:37.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:37.177+0000 7f018d400700 1 --2- 172.21.15.119:0/2433664718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01881067a0 0x7f018819bfb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:37.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:37.177+0000 7f018d400700 1 --2- 172.21.15.119:0/2433664718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01881079a0 0x7f018819c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:37.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:37.177+0000 7f018d400700 1 -- 172.21.15.119:0/2433664718 >> 172.21.15.119:0/2433664718 conn(0x7f0188075910 msgr2=0x7f018810abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:37.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:37.177+0000 7f018d400700 1 -- 172.21.15.119:0/2433664718 shutdown_connections 2024-05-27T09:25:37.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:37.178+0000 7f018d400700 1 -- 172.21.15.119:0/2433664718 wait complete. 2024-05-27T09:25:38.066 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-27T09:25:38.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:37 smithi119 ceph-mon[29082]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:25:38.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:37 smithi138 ceph-mon[34885]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:25:38.471 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:25:39.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:38 smithi119 ceph-mon[29082]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:39.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:38 smithi138 ceph-mon[34885]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:40.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.568+0000 7f0845200700 1 -- 172.21.15.119:0/4279970409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08401067a0 msgr2=0x7f0840106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:40.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.568+0000 7f0845200700 1 --2- 172.21.15.119:0/4279970409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08401067a0 0x7f0840106bb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0830009910 tx=0x7f0830009c20 comp rx=0 tx=0).stop 2024-05-27T09:25:40.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.568+0000 7f0845200700 1 -- 172.21.15.119:0/4279970409 shutdown_connections 2024-05-27T09:25:40.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.568+0000 7f0845200700 1 --2- 172.21.15.119:0/4279970409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08401079a0 0x7f0840107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:40.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.568+0000 7f0845200700 1 --2- 172.21.15.119:0/4279970409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08401067a0 0x7f0840106bb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:40.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.568+0000 7f0845200700 1 -- 172.21.15.119:0/4279970409 >> 172.21.15.119:0/4279970409 conn(0x7f0840075910 msgr2=0x7f0840077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:40.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.568+0000 7f0845200700 1 -- 172.21.15.119:0/4279970409 shutdown_connections 2024-05-27T09:25:40.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.568+0000 7f0845200700 1 -- 172.21.15.119:0/4279970409 wait complete. 2024-05-27T09:25:40.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.569+0000 7f0845200700 1 Processor -- start 2024-05-27T09:25:40.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.569+0000 7f0845200700 1 -- start start 2024-05-27T09:25:40.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.570+0000 7f0845200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08401067a0 0x7f084019bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:40.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.570+0000 7f0845200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08401079a0 0x7f084019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:40.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.570+0000 7f0845200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084019ca80 con 0x7f08401079a0 2024-05-27T09:25:40.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.570+0000 7f0845200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084019cbc0 con 0x7f08401067a0 2024-05-27T09:25:40.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.570+0000 7f083d400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08401079a0 0x7f084019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:40.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.570+0000 7f083d400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08401079a0 0x7f084019c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54846/0 (socket says 172.21.15.119:54846) 2024-05-27T09:25:40.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.570+0000 7f083d400700 1 -- 172.21.15.119:0/3935930716 learned_addr learned my addr 172.21.15.119:0/3935930716 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:25:40.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.570+0000 7f083de00700 1 --2- 172.21.15.119:0/3935930716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08401067a0 0x7f084019bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:40.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.570+0000 7f083d400700 1 -- 172.21.15.119:0/3935930716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08401067a0 msgr2=0x7f084019bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:40.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.571+0000 7f083d400700 1 --2- 172.21.15.119:0/3935930716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08401067a0 0x7f084019bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:40.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.571+0000 7f083d400700 1 -- 172.21.15.119:0/3935930716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08300095f0 con 0x7f08401079a0 2024-05-27T09:25:40.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.571+0000 7f083de00700 1 --2- 172.21.15.119:0/3935930716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08401067a0 0x7f084019bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:25:40.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.571+0000 7f083d400700 1 --2- 172.21.15.119:0/3935930716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08401079a0 0x7f084019c4f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f082c00c930 tx=0x7f082c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:40.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.571+0000 7f0836a00700 1 -- 172.21.15.119:0/3935930716 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f082c007ad0 con 0x7f08401079a0 2024-05-27T09:25:40.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.571+0000 7f0845200700 1 -- 172.21.15.119:0/3935930716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08401a1680 con 0x7f08401079a0 2024-05-27T09:25:40.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.571+0000 7f0836a00700 1 -- 172.21.15.119:0/3935930716 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f082c007c30 con 0x7f08401079a0 2024-05-27T09:25:40.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.571+0000 7f0836a00700 1 -- 172.21.15.119:0/3935930716 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f082c018630 con 0x7f08401079a0 2024-05-27T09:25:40.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.571+0000 7f0845200700 1 -- 172.21.15.119:0/3935930716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08401a1ba0 con 0x7f08401079a0 2024-05-27T09:25:40.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.572+0000 7f0845200700 1 -- 172.21.15.119:0/3935930716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f084010f2f0 con 0x7f08401079a0 2024-05-27T09:25:40.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.573+0000 7f0836a00700 1 -- 172.21.15.119:0/3935930716 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f082c00f450 con 0x7f08401079a0 2024-05-27T09:25:40.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.575+0000 7f0836a00700 1 --2- 172.21.15.119:0/3935930716 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f082406c680 0x7f082406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:40.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.575+0000 7f0836a00700 1 -- 172.21.15.119:0/3935930716 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f082c08b7e0 con 0x7f08401079a0 2024-05-27T09:25:40.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.575+0000 7f083de00700 1 --2- 172.21.15.119:0/3935930716 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f082406c680 0x7f082406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:40.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.576+0000 7f083de00700 1 --2- 172.21.15.119:0/3935930716 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f082406c680 0x7f082406eb30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0830006010 tx=0x7f083000b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:40.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.579+0000 7f0836a00700 1 -- 172.21.15.119:0/3935930716 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f082c059bf0 con 0x7f08401079a0 2024-05-27T09:25:40.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.806+0000 7f0845200700 1 -- 172.21.15.119:0/3935930716 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f084010c2f0 con 0x7f082406c680 2024-05-27T09:25:40.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.812+0000 7f0836a00700 1 -- 172.21.15.119:0/3935930716 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1310 (secure 0 0 0) 0x7f084010c2f0 con 0x7f082406c680 2024-05-27T09:25:40.813 INFO:teuthology.orchestra.run.smithi119.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-27T09:25:40.813 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager ?:9093,9094 1/1 2m ago 7m count:1 2024-05-27T09:25:40.813 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter 2/2 2m ago 7m * 2024-05-27T09:25:40.813 INFO:teuthology.orchestra.run.smithi119.stdout:crash 2/2 2m ago 7m * 2024-05-27T09:25:40.814 INFO:teuthology.orchestra.run.smithi119.stdout:grafana ?:3000 1/1 2m ago 7m count:1 2024-05-27T09:25:40.814 INFO:teuthology.orchestra.run.smithi119.stdout:mgr 2/2 2m ago 7m count:2 2024-05-27T09:25:40.814 INFO:teuthology.orchestra.run.smithi119.stdout:mon 2/2 2m ago 6m smithi119:172.21.15.119=smithi119;smithi138:172.21.15.138=smithi138;count:2 2024-05-27T09:25:40.814 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter ?:9100 2/2 2m ago 7m * 2024-05-27T09:25:40.814 INFO:teuthology.orchestra.run.smithi119.stdout:osd 6 2m ago - 2024-05-27T09:25:40.814 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus ?:9095 1/1 2m ago 7m count:1 2024-05-27T09:25:40.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.815+0000 7f0845200700 1 -- 172.21.15.119:0/3935930716 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f082406c680 msgr2=0x7f082406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:40.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.815+0000 7f0845200700 1 --2- 172.21.15.119:0/3935930716 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f082406c680 0x7f082406eb30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0830006010 tx=0x7f083000b540 comp rx=0 tx=0).stop 2024-05-27T09:25:40.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.815+0000 7f0845200700 1 -- 172.21.15.119:0/3935930716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08401079a0 msgr2=0x7f084019c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:40.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.815+0000 7f0845200700 1 --2- 172.21.15.119:0/3935930716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08401079a0 0x7f084019c4f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f082c00c930 tx=0x7f082c00ccf0 comp rx=0 tx=0).stop 2024-05-27T09:25:40.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.815+0000 7f0845200700 1 -- 172.21.15.119:0/3935930716 shutdown_connections 2024-05-27T09:25:40.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.816+0000 7f0845200700 1 --2- 172.21.15.119:0/3935930716 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f082406c680 0x7f082406eb30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:40.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.816+0000 7f0845200700 1 --2- 172.21.15.119:0/3935930716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08401067a0 0x7f084019bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:40.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.816+0000 7f0845200700 1 --2- 172.21.15.119:0/3935930716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08401079a0 0x7f084019c4f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:40.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.816+0000 7f0845200700 1 -- 172.21.15.119:0/3935930716 >> 172.21.15.119:0/3935930716 conn(0x7f0840075910 msgr2=0x7f084010abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:40.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.816+0000 7f0845200700 1 -- 172.21.15.119:0/3935930716 shutdown_connections 2024-05-27T09:25:40.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:40.816+0000 7f0845200700 1 -- 172.21.15.119:0/3935930716 wait complete. 2024-05-27T09:25:41.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:41 smithi119 ceph-mon[29082]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:41.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:41 smithi138 ceph-mon[34885]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:41.637 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-27T09:25:42.027 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:25:42.607 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:42 smithi119 ceph-mon[29082]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:25:42.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:42 smithi138 ceph-mon[34885]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:25:43.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:43 smithi138 ceph-mon[34885]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:43.661 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:43 smithi119 ceph-mon[29082]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:44.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.152+0000 7f6f07200700 1 -- 172.21.15.119:0/3039530196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f081021e0 msgr2=0x7f6f081025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:44.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.152+0000 7f6f07200700 1 --2- 172.21.15.119:0/3039530196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f081021e0 0x7f6f081025f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009910 tx=0x7f6ef8009c20 comp rx=0 tx=0).stop 2024-05-27T09:25:44.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.152+0000 7f6f07200700 1 -- 172.21.15.119:0/3039530196 shutdown_connections 2024-05-27T09:25:44.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.152+0000 7f6f07200700 1 --2- 172.21.15.119:0/3039530196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f081033e0 0x7f6f08103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:44.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.152+0000 7f6f07200700 1 --2- 172.21.15.119:0/3039530196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f081021e0 0x7f6f081025f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:44.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.152+0000 7f6f07200700 1 -- 172.21.15.119:0/3039530196 >> 172.21.15.119:0/3039530196 conn(0x7f6f080fd8d0 msgr2=0x7f6f080ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:44.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.152+0000 7f6f07200700 1 -- 172.21.15.119:0/3039530196 shutdown_connections 2024-05-27T09:25:44.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.152+0000 7f6f07200700 1 -- 172.21.15.119:0/3039530196 wait complete. 2024-05-27T09:25:44.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.153+0000 7f6f07200700 1 Processor -- start 2024-05-27T09:25:44.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.154+0000 7f6f07200700 1 -- start start 2024-05-27T09:25:44.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.154+0000 7f6f07200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f081021e0 0x7f6f081979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:44.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.154+0000 7f6f07200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f081033e0 0x7f6f08197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:44.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.154+0000 7f6f07200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f08198520 con 0x7f6f081021e0 2024-05-27T09:25:44.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.154+0000 7f6f07200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f08198660 con 0x7f6f081033e0 2024-05-27T09:25:44.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.154+0000 7f6f05e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f081021e0 0x7f6f081979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:44.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.154+0000 7f6f05e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f081021e0 0x7f6f081979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54860/0 (socket says 172.21.15.119:54860) 2024-05-27T09:25:44.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.154+0000 7f6f05e00700 1 -- 172.21.15.119:0/1048519137 learned_addr learned my addr 172.21.15.119:0/1048519137 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:25:44.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.155+0000 7f6f05400700 1 --2- 172.21.15.119:0/1048519137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f081033e0 0x7f6f08197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:44.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.155+0000 7f6f05e00700 1 -- 172.21.15.119:0/1048519137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f081033e0 msgr2=0x7f6f08197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:44.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.155+0000 7f6f05e00700 1 --2- 172.21.15.119:0/1048519137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f081033e0 0x7f6f08197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:44.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.155+0000 7f6f05e00700 1 -- 172.21.15.119:0/1048519137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef80095f0 con 0x7f6f081021e0 2024-05-27T09:25:44.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.155+0000 7f6f05e00700 1 --2- 172.21.15.119:0/1048519137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f081021e0 0x7f6f081979c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800bd70 tx=0x7f6ef800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:44.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.155+0000 7f6efea00700 1 -- 172.21.15.119:0/1048519137 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ef801d070 con 0x7f6f081021e0 2024-05-27T09:25:44.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.155+0000 7f6efea00700 1 -- 172.21.15.119:0/1048519137 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6ef800f460 con 0x7f6f081021e0 2024-05-27T09:25:44.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.155+0000 7f6f07200700 1 -- 172.21.15.119:0/1048519137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f0819d0b0 con 0x7f6f081021e0 2024-05-27T09:25:44.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.155+0000 7f6efea00700 1 -- 172.21.15.119:0/1048519137 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ef8021640 con 0x7f6f081021e0 2024-05-27T09:25:44.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.156+0000 7f6f07200700 1 -- 172.21.15.119:0/1048519137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f0819d5a0 con 0x7f6f081021e0 2024-05-27T09:25:44.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.157+0000 7f6f07200700 1 -- 172.21.15.119:0/1048519137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f0810ad30 con 0x7f6f081021e0 2024-05-27T09:25:44.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.158+0000 7f6efea00700 1 -- 172.21.15.119:0/1048519137 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f6ef800fab0 con 0x7f6f081021e0 2024-05-27T09:25:44.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.159+0000 7f6efea00700 1 --2- 172.21.15.119:0/1048519137 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f6eec06c680 0x7f6eec06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:44.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.159+0000 7f6efea00700 1 -- 172.21.15.119:0/1048519137 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f6ef808c9a0 con 0x7f6f081021e0 2024-05-27T09:25:44.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.159+0000 7f6f05400700 1 --2- 172.21.15.119:0/1048519137 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f6eec06c680 0x7f6eec06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:44.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.160+0000 7f6f05400700 1 --2- 172.21.15.119:0/1048519137 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f6eec06c680 0x7f6eec06eb30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4005fd0 tx=0x7f6ef400a560 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:44.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.164+0000 7f6efea00700 1 -- 172.21.15.119:0/1048519137 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6ef805ae30 con 0x7f6f081021e0 2024-05-27T09:25:44.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.391+0000 7f6f07200700 1 -- 172.21.15.119:0/1048519137 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f08107d30 con 0x7f6eec06c680 2024-05-27T09:25:44.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.394+0000 7f6efea00700 1 -- 172.21.15.119:0/1048519137 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f6f08107d30 con 0x7f6eec06c680 2024-05-27T09:25:44.395 INFO:teuthology.orchestra.run.smithi119.stdout:HOST ADDR LABELS STATUS 2024-05-27T09:25:44.395 INFO:teuthology.orchestra.run.smithi119.stdout:smithi119 172.21.15.119 2024-05-27T09:25:44.395 INFO:teuthology.orchestra.run.smithi119.stdout:smithi138 172.21.15.138 2024-05-27T09:25:44.395 INFO:teuthology.orchestra.run.smithi119.stdout:2 hosts in cluster 2024-05-27T09:25:44.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.397+0000 7f6f07200700 1 -- 172.21.15.119:0/1048519137 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f6eec06c680 msgr2=0x7f6eec06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:44.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.397+0000 7f6f07200700 1 --2- 172.21.15.119:0/1048519137 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f6eec06c680 0x7f6eec06eb30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4005fd0 tx=0x7f6ef400a560 comp rx=0 tx=0).stop 2024-05-27T09:25:44.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.397+0000 7f6f07200700 1 -- 172.21.15.119:0/1048519137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f081021e0 msgr2=0x7f6f081979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:44.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.397+0000 7f6f07200700 1 --2- 172.21.15.119:0/1048519137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f081021e0 0x7f6f081979c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800bd70 tx=0x7f6ef800be50 comp rx=0 tx=0).stop 2024-05-27T09:25:44.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.398+0000 7f6f07200700 1 -- 172.21.15.119:0/1048519137 shutdown_connections 2024-05-27T09:25:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.398+0000 7f6f07200700 1 --2- 172.21.15.119:0/1048519137 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f6eec06c680 0x7f6eec06eb30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.398+0000 7f6f07200700 1 --2- 172.21.15.119:0/1048519137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f081021e0 0x7f6f081979c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.398+0000 7f6f07200700 1 --2- 172.21.15.119:0/1048519137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f081033e0 0x7f6f08197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.398+0000 7f6f07200700 1 -- 172.21.15.119:0/1048519137 >> 172.21.15.119:0/1048519137 conn(0x7f6f080fd8d0 msgr2=0x7f6f08106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.398+0000 7f6f07200700 1 -- 172.21.15.119:0/1048519137 shutdown_connections 2024-05-27T09:25:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:44.398+0000 7f6f07200700 1 -- 172.21.15.119:0/1048519137 wait complete. 2024-05-27T09:25:45.243 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-27T09:25:45.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:45 smithi119 ceph-mon[29082]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:45.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:45 smithi119 ceph-mon[29082]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:25:45.631 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:25:45.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:45 smithi138 ceph-mon[34885]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:45.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:45 smithi138 ceph-mon[34885]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:25:47.241 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:46 smithi119 ceph-mon[29082]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:47.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:46 smithi138 ceph-mon[34885]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:47.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.758+0000 7fbfb7600700 1 -- 172.21.15.119:0/1847520836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfb01023a0 msgr2=0x7fbfb01027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:47.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.758+0000 7fbfb7600700 1 --2- 172.21.15.119:0/1847520836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfb01023a0 0x7fbfb01027b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009910 tx=0x7fbfa4009c20 comp rx=0 tx=0).stop 2024-05-27T09:25:47.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.758+0000 7fbfb7600700 1 -- 172.21.15.119:0/1847520836 shutdown_connections 2024-05-27T09:25:47.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.758+0000 7fbfb7600700 1 --2- 172.21.15.119:0/1847520836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfb01035a0 0x7fbfb01039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:47.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.758+0000 7fbfb7600700 1 --2- 172.21.15.119:0/1847520836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfb01023a0 0x7fbfb01027b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:47.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.758+0000 7fbfb7600700 1 -- 172.21.15.119:0/1847520836 >> 172.21.15.119:0/1847520836 conn(0x7fbfb00fd990 msgr2=0x7fbfb00ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:47.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.758+0000 7fbfb7600700 1 -- 172.21.15.119:0/1847520836 shutdown_connections 2024-05-27T09:25:47.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.759+0000 7fbfb7600700 1 -- 172.21.15.119:0/1847520836 wait complete. 2024-05-27T09:25:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.759+0000 7fbfb7600700 1 Processor -- start 2024-05-27T09:25:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.760+0000 7fbfb7600700 1 -- start start 2024-05-27T09:25:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.760+0000 7fbfb7600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfb01023a0 0x7fbfb0197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.760+0000 7fbfb7600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfb01035a0 0x7fbfb0198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.760+0000 7fbfb7600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb01987a0 con 0x7fbfb01023a0 2024-05-27T09:25:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.760+0000 7fbfb7600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb019d1b0 con 0x7fbfb01035a0 2024-05-27T09:25:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.760+0000 7fbfb4c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfb01023a0 0x7fbfb0197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:47.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.760+0000 7fbfb4c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfb01023a0 0x7fbfb0197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54874/0 (socket says 172.21.15.119:54874) 2024-05-27T09:25:47.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.760+0000 7fbfb4c00700 1 -- 172.21.15.119:0/458997392 learned_addr learned my addr 172.21.15.119:0/458997392 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:25:47.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.761+0000 7fbfafe00700 1 --2- 172.21.15.119:0/458997392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfb01035a0 0x7fbfb0198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:47.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.761+0000 7fbfb4c00700 1 -- 172.21.15.119:0/458997392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfb01035a0 msgr2=0x7fbfb0198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:47.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.761+0000 7fbfb4c00700 1 --2- 172.21.15.119:0/458997392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfb01035a0 0x7fbfb0198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:47.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.761+0000 7fbfb4c00700 1 -- 172.21.15.119:0/458997392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa40095f0 con 0x7fbfb01023a0 2024-05-27T09:25:47.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.761+0000 7fbfafe00700 1 --2- 172.21.15.119:0/458997392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfb01035a0 0x7fbfb0198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:25:47.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.761+0000 7fbfb4c00700 1 --2- 172.21.15.119:0/458997392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfb01023a0 0x7fbfb0197c40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400bd70 tx=0x7fbfa400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:47.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.761+0000 7fbfad600700 1 -- 172.21.15.119:0/458997392 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfa401d070 con 0x7fbfb01023a0 2024-05-27T09:25:47.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.761+0000 7fbfad600700 1 -- 172.21.15.119:0/458997392 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbfa400f460 con 0x7fbfb01023a0 2024-05-27T09:25:47.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.761+0000 7fbfad600700 1 -- 172.21.15.119:0/458997392 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfa4021640 con 0x7fbfb01023a0 2024-05-27T09:25:47.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.761+0000 7fbfb7600700 1 -- 172.21.15.119:0/458997392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb019d350 con 0x7fbfb01023a0 2024-05-27T09:25:47.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.762+0000 7fbfb7600700 1 -- 172.21.15.119:0/458997392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb019d890 con 0x7fbfb01023a0 2024-05-27T09:25:47.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.763+0000 7fbfb7600700 1 -- 172.21.15.119:0/458997392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb010af70 con 0x7fbfb01023a0 2024-05-27T09:25:47.764 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.764+0000 7fbfad600700 1 -- 172.21.15.119:0/458997392 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fbfa400fab0 con 0x7fbfb01023a0 2024-05-27T09:25:47.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.764+0000 7fbfad600700 1 --2- 172.21.15.119:0/458997392 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbf9806c680 0x7fbf9806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:47.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.765+0000 7fbfad600700 1 -- 172.21.15.119:0/458997392 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbfa408c9a0 con 0x7fbfb01023a0 2024-05-27T09:25:47.765 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.765+0000 7fbfafe00700 1 --2- 172.21.15.119:0/458997392 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbf9806c680 0x7fbf9806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:47.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.766+0000 7fbfafe00700 1 --2- 172.21.15.119:0/458997392 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbf9806c680 0x7fbf9806eb30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c007900 tx=0x7fbf9c008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:47.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.770+0000 7fbfad600700 1 -- 172.21.15.119:0/458997392 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbfa40573a0 con 0x7fbfb01023a0 2024-05-27T09:25:47.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.993+0000 7fbfb7600700 1 -- 172.21.15.119:0/458997392 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfb0107ef0 con 0x7fbf9806c680 2024-05-27T09:25:47.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.995+0000 7fbfad600700 1 -- 172.21.15.119:0/458997392 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7fbfb0107ef0 con 0x7fbf9806c680 2024-05-27T09:25:47.996 INFO:teuthology.orchestra.run.smithi119.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-27T09:25:47.996 INFO:teuthology.orchestra.run.smithi119.stdout:smithi119 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400Y1400BGN 372G No 3m ago LVM detected, locked 2024-05-27T09:25:47.996 INFO:teuthology.orchestra.run.smithi119.stdout:smithi138 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6161003C400BGN 372G No 82s ago LVM detected, locked 2024-05-27T09:25:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.998+0000 7fbfb7600700 1 -- 172.21.15.119:0/458997392 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbf9806c680 msgr2=0x7fbf9806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.998+0000 7fbfb7600700 1 --2- 172.21.15.119:0/458997392 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbf9806c680 0x7fbf9806eb30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c007900 tx=0x7fbf9c008040 comp rx=0 tx=0).stop 2024-05-27T09:25:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.998+0000 7fbfb7600700 1 -- 172.21.15.119:0/458997392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfb01023a0 msgr2=0x7fbfb0197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.998+0000 7fbfb7600700 1 --2- 172.21.15.119:0/458997392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfb01023a0 0x7fbfb0197c40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400bd70 tx=0x7fbfa400be50 comp rx=0 tx=0).stop 2024-05-27T09:25:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.998+0000 7fbfb7600700 1 -- 172.21.15.119:0/458997392 shutdown_connections 2024-05-27T09:25:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.998+0000 7fbfb7600700 1 --2- 172.21.15.119:0/458997392 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbf9806c680 0x7fbf9806eb30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.998+0000 7fbfb7600700 1 --2- 172.21.15.119:0/458997392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfb01023a0 0x7fbfb0197c40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:48.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.999+0000 7fbfb7600700 1 --2- 172.21.15.119:0/458997392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfb01035a0 0x7fbfb0198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:48.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.999+0000 7fbfb7600700 1 -- 172.21.15.119:0/458997392 >> 172.21.15.119:0/458997392 conn(0x7fbfb00fd990 msgr2=0x7fbfb01067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:48.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.999+0000 7fbfb7600700 1 -- 172.21.15.119:0/458997392 shutdown_connections 2024-05-27T09:25:48.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:47.999+0000 7fbfb7600700 1 -- 172.21.15.119:0/458997392 wait complete. 2024-05-27T09:25:48.786 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-27T09:25:48.801 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi119.front.sepia.ceph.com 2024-05-27T09:25:48.802 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-05-27T09:25:49.229 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:25:49.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:49 smithi138 ceph-mon[34885]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:25:49.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:49 smithi138 ceph-mon[34885]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:49.677 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:49 smithi119 ceph-mon[29082]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:25:49.677 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:49 smithi119 ceph-mon[29082]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:51.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.266+0000 7f9405400700 1 -- 172.21.15.119:0/2374422891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9400100a20 msgr2=0x7f9400102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:51.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.266+0000 7f9405400700 1 --2- 172.21.15.119:0/2374422891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9400100a20 0x7f9400102e00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009910 tx=0x7f93f0009c20 comp rx=0 tx=0).stop 2024-05-27T09:25:51.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.266+0000 7f9405400700 1 -- 172.21.15.119:0/2374422891 shutdown_connections 2024-05-27T09:25:51.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.266+0000 7f9405400700 1 --2- 172.21.15.119:0/2374422891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9400103340 0x7f9400105720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:51.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.266+0000 7f9405400700 1 --2- 172.21.15.119:0/2374422891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9400100a20 0x7f9400102e00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:51.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.266+0000 7f9405400700 1 -- 172.21.15.119:0/2374422891 >> 172.21.15.119:0/2374422891 conn(0x7f94000fa440 msgr2=0x7f94000fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:51.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.266+0000 7f9405400700 1 -- 172.21.15.119:0/2374422891 shutdown_connections 2024-05-27T09:25:51.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.266+0000 7f9405400700 1 -- 172.21.15.119:0/2374422891 wait complete. 2024-05-27T09:25:51.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.267+0000 7f9405400700 1 Processor -- start 2024-05-27T09:25:51.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.268+0000 7f9405400700 1 -- start start 2024-05-27T09:25:51.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.268+0000 7f9405400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9400100a20 0x7f94001979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:51.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.268+0000 7f9405400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9400103340 0x7f9400197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:51.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.268+0000 7f9405400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9400198550 con 0x7f9400103340 2024-05-27T09:25:51.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.268+0000 7f9405400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9400198690 con 0x7f9400100a20 2024-05-27T09:25:51.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.269+0000 7f93ffe00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9400100a20 0x7f94001979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:51.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.269+0000 7f93ff400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9400103340 0x7f9400197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:51.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.269+0000 7f93ffe00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9400100a20 0x7f94001979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.119:33244/0 (socket says 172.21.15.119:33244) 2024-05-27T09:25:51.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.269+0000 7f93ffe00700 1 -- 172.21.15.119:0/766470281 learned_addr learned my addr 172.21.15.119:0/766470281 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:25:51.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.269+0000 7f93ff400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9400103340 0x7f9400197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42926/0 (socket says 172.21.15.119:42926) 2024-05-27T09:25:51.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.269+0000 7f93ff400700 1 -- 172.21.15.119:0/766470281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9400100a20 msgr2=0x7f94001979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:51.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.269+0000 7f93ff400700 1 --2- 172.21.15.119:0/766470281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9400100a20 0x7f94001979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:51.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.269+0000 7f93ff400700 1 -- 172.21.15.119:0/766470281 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f00095f0 con 0x7f9400103340 2024-05-27T09:25:51.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.270+0000 7f93ff400700 1 --2- 172.21.15.119:0/766470281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9400103340 0x7f9400197f30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00e970 tx=0x7f93ec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:51.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.270+0000 7f93fcc00700 1 -- 172.21.15.119:0/766470281 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93ec00c9d0 con 0x7f9400103340 2024-05-27T09:25:51.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.270+0000 7f93fcc00700 1 -- 172.21.15.119:0/766470281 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f93ec010430 con 0x7f9400103340 2024-05-27T09:25:51.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.270+0000 7f93fcc00700 1 -- 172.21.15.119:0/766470281 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93ec018650 con 0x7f9400103340 2024-05-27T09:25:51.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.270+0000 7f9405400700 1 -- 172.21.15.119:0/766470281 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f940019d140 con 0x7f9400103340 2024-05-27T09:25:51.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.270+0000 7f9405400700 1 -- 172.21.15.119:0/766470281 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f940019d690 con 0x7f9400103340 2024-05-27T09:25:51.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.272+0000 7f9405400700 1 -- 172.21.15.119:0/766470281 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94000fc000 con 0x7f9400103340 2024-05-27T09:25:51.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.272+0000 7f93fcc00700 1 -- 172.21.15.119:0/766470281 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f93ec0105a0 con 0x7f9400103340 2024-05-27T09:25:51.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.273+0000 7f93fcc00700 1 --2- 172.21.15.119:0/766470281 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f93e406c680 0x7f93e406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.273+0000 7f93fcc00700 1 -- 172.21.15.119:0/766470281 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f93ec014070 con 0x7f9400103340 2024-05-27T09:25:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.274+0000 7f93ffe00700 1 --2- 172.21.15.119:0/766470281 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f93e406c680 0x7f93e406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.274+0000 7f93ffe00700 1 --2- 172.21.15.119:0/766470281 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f93e406c680 0x7f93e406eb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f93f0006010 tx=0x7f93f0005a90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.278+0000 7f93fcc00700 1 -- 172.21.15.119:0/766470281 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f93ec051720 con 0x7f9400103340 2024-05-27T09:25:51.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:51 smithi119 ceph-mon[29082]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:51.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:51 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:25:51.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:51.532+0000 7f9405400700 1 -- 172.21.15.119:0/766470281 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f9400061190 con 0x7f93e406c680 2024-05-27T09:25:51.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:51 smithi138 ceph-mon[34885]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:51.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:51 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:25:52.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:52 smithi138 ceph-mon[34885]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:25:52.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:52 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-05-27T09:25:52.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:52 smithi119 ceph-mon[29082]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:25:52.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:52 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-05-27T09:25:53.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:53.416+0000 7f93fcc00700 1 -- 172.21.15.119:0/766470281 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9400061190 con 0x7f93e406c680 2024-05-27T09:25:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:53.419+0000 7f9405400700 1 -- 172.21.15.119:0/766470281 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f93e406c680 msgr2=0x7f93e406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:53.419+0000 7f9405400700 1 --2- 172.21.15.119:0/766470281 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f93e406c680 0x7f93e406eb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f93f0006010 tx=0x7f93f0005a90 comp rx=0 tx=0).stop 2024-05-27T09:25:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:53.419+0000 7f9405400700 1 -- 172.21.15.119:0/766470281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9400103340 msgr2=0x7f9400197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:53.419+0000 7f9405400700 1 --2- 172.21.15.119:0/766470281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9400103340 0x7f9400197f30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00e970 tx=0x7f93ec00ec80 comp rx=0 tx=0).stop 2024-05-27T09:25:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:53.419+0000 7f9405400700 1 -- 172.21.15.119:0/766470281 shutdown_connections 2024-05-27T09:25:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:53.419+0000 7f9405400700 1 --2- 172.21.15.119:0/766470281 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f93e406c680 0x7f93e406eb30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:53.419+0000 7f9405400700 1 --2- 172.21.15.119:0/766470281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9400100a20 0x7f94001979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:53.419+0000 7f9405400700 1 --2- 172.21.15.119:0/766470281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9400103340 0x7f9400197f30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:53.419+0000 7f9405400700 1 -- 172.21.15.119:0/766470281 >> 172.21.15.119:0/766470281 conn(0x7f94000fa440 msgr2=0x7f94000ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:53.419+0000 7f9405400700 1 -- 172.21.15.119:0/766470281 shutdown_connections 2024-05-27T09:25:53.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:53.419+0000 7f9405400700 1 -- 172.21.15.119:0/766470281 wait complete. 2024-05-27T09:25:53.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:53 smithi138 ceph-mon[34885]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:53.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:53 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-05-27T09:25:53.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:53 smithi138 ceph-mon[34885]: osdmap e41: 6 total, 6 up, 6 in 2024-05-27T09:25:53.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:53 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-05-27T09:25:53.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:53 smithi119 ceph-mon[29082]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:53.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:53 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-05-27T09:25:53.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:53 smithi119 ceph-mon[29082]: osdmap e41: 6 total, 6 up, 6 in 2024-05-27T09:25:53.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:53 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-05-27T09:25:53.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:53 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119[29078]: 2024-05-27T09:25:53.386+0000 7f1b47800700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-27T09:25:54.301 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-27T09:25:54.513 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:54 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-05-27T09:25:54.514 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:54 smithi119 ceph-mon[29082]: osdmap e42: 6 total, 6 up, 6 in 2024-05-27T09:25:54.514 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:54 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-05-27T09:25:54.514 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:54 smithi119 ceph-mon[29082]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-27T09:25:54.514 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:54 smithi119 ceph-mon[29082]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-27T09:25:54.514 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:54 smithi119 ceph-mon[29082]: osdmap e43: 6 total, 6 up, 6 in 2024-05-27T09:25:54.514 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:54 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-05-27T09:25:54.514 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:54 smithi119 ceph-mon[29082]: fsmap cephfs:0 2024-05-27T09:25:54.514 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:54 smithi119 ceph-mon[29082]: Saving service mds.cephfs spec with placement count:4 2024-05-27T09:25:54.514 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:54 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:25:54.514 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:54 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:25:54.514 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:54 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:25:54.515 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:54 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:25:54.515 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:54 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:25:54.515 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:54 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi119.nghcey", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:25:54.515 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:54 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi119.nghcey", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T09:25:54.515 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:54 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:25:54.515 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:54 smithi119 ceph-mon[29082]: Deploying daemon mds.cephfs.smithi119.nghcey on smithi119 2024-05-27T09:25:54.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:54 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-05-27T09:25:54.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:54 smithi138 ceph-mon[34885]: osdmap e42: 6 total, 6 up, 6 in 2024-05-27T09:25:54.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:54 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-05-27T09:25:54.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:54 smithi138 ceph-mon[34885]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-27T09:25:54.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:54 smithi138 ceph-mon[34885]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-27T09:25:54.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:54 smithi138 ceph-mon[34885]: osdmap e43: 6 total, 6 up, 6 in 2024-05-27T09:25:54.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:54 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-05-27T09:25:54.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:54 smithi138 ceph-mon[34885]: fsmap cephfs:0 2024-05-27T09:25:54.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:54 smithi138 ceph-mon[34885]: Saving service mds.cephfs spec with placement count:4 2024-05-27T09:25:54.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:54 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:25:54.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:54 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:25:54.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:54 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:25:54.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:54 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:25:54.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:54 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:25:54.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:54 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi119.nghcey", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:25:54.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:54 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi119.nghcey", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T09:25:54.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:54 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:25:54.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:54 smithi138 ceph-mon[34885]: Deploying daemon mds.cephfs.smithi119.nghcey on smithi119 2024-05-27T09:25:55.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:55 smithi119 ceph-mon[29082]: pgmap v220: 65 pgs: 9 creating+peering, 11 creating+activating, 11 active+clean, 34 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:55.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:55 smithi119 ceph-mon[29082]: osdmap e44: 6 total, 6 up, 6 in 2024-05-27T09:25:55.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:55 smithi138 ceph-mon[34885]: pgmap v220: 65 pgs: 9 creating+peering, 11 creating+activating, 11 active+clean, 34 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:55.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:55 smithi138 ceph-mon[34885]: osdmap e44: 6 total, 6 up, 6 in 2024-05-27T09:25:55.646 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:25:56.655 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:56 smithi119 ceph-mon[29082]: osdmap e45: 6 total, 6 up, 6 in 2024-05-27T09:25:56.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:56 smithi138 ceph-mon[34885]: osdmap e45: 6 total, 6 up, 6 in 2024-05-27T09:25:57.598 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:57 smithi119 ceph-mon[29082]: pgmap v223: 65 pgs: 15 creating+peering, 11 creating+activating, 26 active+clean, 13 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:57.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:57 smithi138 ceph-mon[34885]: pgmap v223: 65 pgs: 15 creating+peering, 11 creating+activating, 26 active+clean, 13 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:58.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.369+0000 7faa10800700 1 -- 172.21.15.119:0/3255184385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa08100270 msgr2=0x7faa081006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:58.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.369+0000 7faa10800700 1 --2- 172.21.15.119:0/3255184385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa08100270 0x7faa081006c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009910 tx=0x7fa9f8009c20 comp rx=0 tx=0).stop 2024-05-27T09:25:58.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.369+0000 7faa10800700 1 -- 172.21.15.119:0/3255184385 shutdown_connections 2024-05-27T09:25:58.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.369+0000 7faa10800700 1 --2- 172.21.15.119:0/3255184385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa08100270 0x7faa081006c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:58.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.369+0000 7faa10800700 1 --2- 172.21.15.119:0/3255184385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa080ff070 0x7faa080ff480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:58.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.369+0000 7faa10800700 1 -- 172.21.15.119:0/3255184385 >> 172.21.15.119:0/3255184385 conn(0x7faa080fa600 msgr2=0x7faa080fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:58.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.369+0000 7faa10800700 1 -- 172.21.15.119:0/3255184385 shutdown_connections 2024-05-27T09:25:58.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.369+0000 7faa10800700 1 -- 172.21.15.119:0/3255184385 wait complete. 2024-05-27T09:25:58.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.370+0000 7faa10800700 1 Processor -- start 2024-05-27T09:25:58.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.370+0000 7faa10800700 1 -- start start 2024-05-27T09:25:58.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.371+0000 7faa10800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa080ff070 0x7faa08195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:58.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.371+0000 7faa10800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa08100270 0x7faa08195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:58.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.371+0000 7faa10800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa081965b0 con 0x7faa08100270 2024-05-27T09:25:58.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.371+0000 7faa10800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa081966f0 con 0x7faa080ff070 2024-05-27T09:25:58.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.371+0000 7faa0d400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa08100270 0x7faa08195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:58.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.371+0000 7faa0d400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa08100270 0x7faa08195f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42972/0 (socket says 172.21.15.119:42972) 2024-05-27T09:25:58.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.371+0000 7faa0d400700 1 -- 172.21.15.119:0/2120261094 learned_addr learned my addr 172.21.15.119:0/2120261094 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:25:58.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.371+0000 7faa0de00700 1 --2- 172.21.15.119:0/2120261094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa080ff070 0x7faa08195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:58.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.371+0000 7faa0d400700 1 -- 172.21.15.119:0/2120261094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa080ff070 msgr2=0x7faa08195a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:58.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.371+0000 7faa0d400700 1 --2- 172.21.15.119:0/2120261094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa080ff070 0x7faa08195a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:58.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.371+0000 7faa0d400700 1 -- 172.21.15.119:0/2120261094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f80095f0 con 0x7faa08100270 2024-05-27T09:25:58.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.372+0000 7faa0d400700 1 --2- 172.21.15.119:0/2120261094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa08100270 0x7faa08195f90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800bd70 tx=0x7fa9f800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:58.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.372+0000 7faa02a00700 1 -- 172.21.15.119:0/2120261094 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9f801d070 con 0x7faa08100270 2024-05-27T09:25:58.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.372+0000 7faa02a00700 1 -- 172.21.15.119:0/2120261094 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa9f8022470 con 0x7faa08100270 2024-05-27T09:25:58.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.372+0000 7faa10800700 1 -- 172.21.15.119:0/2120261094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa0819b140 con 0x7faa08100270 2024-05-27T09:25:58.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.372+0000 7faa02a00700 1 -- 172.21.15.119:0/2120261094 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9f800f670 con 0x7faa08100270 2024-05-27T09:25:58.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.372+0000 7faa10800700 1 -- 172.21.15.119:0/2120261094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa0819b6b0 con 0x7faa08100270 2024-05-27T09:25:58.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.373+0000 7faa10800700 1 -- 172.21.15.119:0/2120261094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa08107c60 con 0x7faa08100270 2024-05-27T09:25:58.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.374+0000 7faa02a00700 1 -- 172.21.15.119:0/2120261094 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fa9f8022a50 con 0x7faa08100270 2024-05-27T09:25:58.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.376+0000 7faa02a00700 1 --2- 172.21.15.119:0/2120261094 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fa9f006c680 0x7fa9f006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:25:58.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.376+0000 7faa02a00700 1 -- 172.21.15.119:0/2120261094 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fa9f808ce80 con 0x7faa08100270 2024-05-27T09:25:58.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.376+0000 7faa0de00700 1 --2- 172.21.15.119:0/2120261094 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fa9f006c680 0x7fa9f006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:25:58.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.376+0000 7faa0de00700 1 --2- 172.21.15.119:0/2120261094 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fa9f006c680 0x7fa9f006eb30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc006fd0 tx=0x7fa9fc008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:25:58.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.381+0000 7faa02a00700 1 -- 172.21.15.119:0/2120261094 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa9f805b120 con 0x7faa08100270 2024-05-27T09:25:58.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.659+0000 7faa10800700 1 -- 172.21.15.119:0/2120261094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7faa0819bd40 con 0x7faa08100270 2024-05-27T09:25:58.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.659+0000 7faa02a00700 1 -- 172.21.15.119:0/2120261094 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 4 v4) v1 ==== 75+0+1264 (secure 0 0 0) 0x7fa9f805acb0 con 0x7faa08100270 2024-05-27T09:25:58.661 INFO:teuthology.orchestra.run.smithi119.stdout:e4 2024-05-27T09:25:58.661 INFO:teuthology.orchestra.run.smithi119.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T09:25:58.661 INFO:teuthology.orchestra.run.smithi119.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:25:58.662 INFO:teuthology.orchestra.run.smithi119.stdout:legacy client fscid: 1 2024-05-27T09:25:58.662 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:25:58.662 INFO:teuthology.orchestra.run.smithi119.stdout:Filesystem 'cephfs' (1) 2024-05-27T09:25:58.662 INFO:teuthology.orchestra.run.smithi119.stdout:fs_name cephfs 2024-05-27T09:25:58.662 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 4 2024-05-27T09:25:58.662 INFO:teuthology.orchestra.run.smithi119.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-27T09:25:58.662 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-05-27T09:25:53.387388+0000 2024-05-27T09:25:58.662 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2024-05-27T09:25:58.421672+0000 2024-05-27T09:25:58.662 INFO:teuthology.orchestra.run.smithi119.stdout:tableserver 0 2024-05-27T09:25:58.662 INFO:teuthology.orchestra.run.smithi119.stdout:root 0 2024-05-27T09:25:58.662 INFO:teuthology.orchestra.run.smithi119.stdout:session_timeout 60 2024-05-27T09:25:58.662 INFO:teuthology.orchestra.run.smithi119.stdout:session_autoclose 300 2024-05-27T09:25:58.662 INFO:teuthology.orchestra.run.smithi119.stdout:max_file_size 1099511627776 2024-05-27T09:25:58.662 INFO:teuthology.orchestra.run.smithi119.stdout:required_client_features {} 2024-05-27T09:25:58.663 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure 0 2024-05-27T09:25:58.663 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure_osd_epoch 0 2024-05-27T09:25:58.663 INFO:teuthology.orchestra.run.smithi119.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:25:58.663 INFO:teuthology.orchestra.run.smithi119.stdout:max_mds 1 2024-05-27T09:25:58.663 INFO:teuthology.orchestra.run.smithi119.stdout:in 0 2024-05-27T09:25:58.663 INFO:teuthology.orchestra.run.smithi119.stdout:up {0=14460} 2024-05-27T09:25:58.663 INFO:teuthology.orchestra.run.smithi119.stdout:failed 2024-05-27T09:25:58.663 INFO:teuthology.orchestra.run.smithi119.stdout:damaged 2024-05-27T09:25:58.663 INFO:teuthology.orchestra.run.smithi119.stdout:stopped 2024-05-27T09:25:58.663 INFO:teuthology.orchestra.run.smithi119.stdout:data_pools [3] 2024-05-27T09:25:58.663 INFO:teuthology.orchestra.run.smithi119.stdout:metadata_pool 2 2024-05-27T09:25:58.663 INFO:teuthology.orchestra.run.smithi119.stdout:inline_data disabled 2024-05-27T09:25:58.663 INFO:teuthology.orchestra.run.smithi119.stdout:balancer 2024-05-27T09:25:58.663 INFO:teuthology.orchestra.run.smithi119.stdout:bal_rank_mask -1 2024-05-27T09:25:58.663 INFO:teuthology.orchestra.run.smithi119.stdout:standby_count_wanted 0 2024-05-27T09:25:58.663 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.nghcey{0:14460} state up:creating seq 1 addr [v2:172.21.15.119:6826/303813653,v1:172.21.15.119:6827/303813653] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:25:58.663 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:25:58.664 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:25:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.662+0000 7faa10800700 1 -- 172.21.15.119:0/2120261094 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fa9f006c680 msgr2=0x7fa9f006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.662+0000 7faa10800700 1 --2- 172.21.15.119:0/2120261094 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fa9f006c680 0x7fa9f006eb30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc006fd0 tx=0x7fa9fc008040 comp rx=0 tx=0).stop 2024-05-27T09:25:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.663+0000 7faa10800700 1 -- 172.21.15.119:0/2120261094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa08100270 msgr2=0x7faa08195f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:25:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.663+0000 7faa10800700 1 --2- 172.21.15.119:0/2120261094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa08100270 0x7faa08195f90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800bd70 tx=0x7fa9f800be50 comp rx=0 tx=0).stop 2024-05-27T09:25:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.663+0000 7faa10800700 1 -- 172.21.15.119:0/2120261094 shutdown_connections 2024-05-27T09:25:58.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.663+0000 7faa10800700 1 --2- 172.21.15.119:0/2120261094 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fa9f006c680 0x7fa9f006eb30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:58.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.663+0000 7faa10800700 1 --2- 172.21.15.119:0/2120261094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa080ff070 0x7faa08195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:58.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.663+0000 7faa10800700 1 --2- 172.21.15.119:0/2120261094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa08100270 0x7faa08195f90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:25:58.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.663+0000 7faa10800700 1 -- 172.21.15.119:0/2120261094 >> 172.21.15.119:0/2120261094 conn(0x7faa080fa600 msgr2=0x7faa08068cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:25:58.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.663+0000 7faa10800700 1 -- 172.21.15.119:0/2120261094 shutdown_connections 2024-05-27T09:25:58.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:25:58.663+0000 7faa10800700 1 -- 172.21.15.119:0/2120261094 wait complete. 2024-05-27T09:25:58.666 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 4 2024-05-27T09:25:59.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:58 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:25:59.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:58 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:25:59.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:58 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:25:59.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:58 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.ndfkhd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:25:59.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:58 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.ndfkhd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T09:25:59.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:58 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:25:59.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:58 smithi138 ceph-mon[34885]: Deploying daemon mds.cephfs.smithi138.ndfkhd on smithi138 2024-05-27T09:25:59.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:58 smithi138 ceph-mon[34885]: pgmap v224: 65 pgs: 15 creating+peering, 11 creating+activating, 39 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:59.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:58 smithi138 ceph-mon[34885]: mds.? [v2:172.21.15.119:6826/303813653,v1:172.21.15.119:6827/303813653] up:boot 2024-05-27T09:25:59.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:58 smithi138 ceph-mon[34885]: daemon mds.cephfs.smithi119.nghcey assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-05-27T09:25:59.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:58 smithi138 ceph-mon[34885]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-27T09:25:59.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:58 smithi138 ceph-mon[34885]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-27T09:25:59.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:58 smithi138 ceph-mon[34885]: Cluster is now healthy 2024-05-27T09:25:59.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:58 smithi138 ceph-mon[34885]: fsmap cephfs:0 1 up:standby 2024-05-27T09:25:59.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:58 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.nghcey"}]: dispatch 2024-05-27T09:25:59.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:58 smithi138 ceph-mon[34885]: fsmap cephfs:1 {0=cephfs.smithi119.nghcey=up:creating} 2024-05-27T09:25:59.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:58 smithi138 ceph-mon[34885]: daemon mds.cephfs.smithi119.nghcey is now active in filesystem cephfs as rank 0 2024-05-27T09:25:59.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:25:58 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/2120261094' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:25:59.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:58 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:25:59.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:58 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:25:59.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:58 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:25:59.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:58 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.ndfkhd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:25:59.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:58 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.ndfkhd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T09:25:59.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:58 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:25:59.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:58 smithi119 ceph-mon[29082]: Deploying daemon mds.cephfs.smithi138.ndfkhd on smithi138 2024-05-27T09:25:59.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:58 smithi119 ceph-mon[29082]: pgmap v224: 65 pgs: 15 creating+peering, 11 creating+activating, 39 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:25:59.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:58 smithi119 ceph-mon[29082]: mds.? [v2:172.21.15.119:6826/303813653,v1:172.21.15.119:6827/303813653] up:boot 2024-05-27T09:25:59.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:58 smithi119 ceph-mon[29082]: daemon mds.cephfs.smithi119.nghcey assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-05-27T09:25:59.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:58 smithi119 ceph-mon[29082]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-27T09:25:59.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:58 smithi119 ceph-mon[29082]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-27T09:25:59.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:58 smithi119 ceph-mon[29082]: Cluster is now healthy 2024-05-27T09:25:59.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:58 smithi119 ceph-mon[29082]: fsmap cephfs:0 1 up:standby 2024-05-27T09:25:59.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:58 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.nghcey"}]: dispatch 2024-05-27T09:25:59.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:58 smithi119 ceph-mon[29082]: fsmap cephfs:1 {0=cephfs.smithi119.nghcey=up:creating} 2024-05-27T09:25:59.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:58 smithi119 ceph-mon[29082]: daemon mds.cephfs.smithi119.nghcey is now active in filesystem cephfs as rank 0 2024-05-27T09:25:59.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:25:58 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/2120261094' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:25:59.581 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-27T09:25:59.592 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi119.front.sepia.ceph.com 2024-05-27T09:25:59.592 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-05-27T09:26:00.001 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:26:00.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:00 smithi119 ceph-mon[29082]: mds.? [v2:172.21.15.119:6826/303813653,v1:172.21.15.119:6827/303813653] up:active 2024-05-27T09:26:00.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:00 smithi119 ceph-mon[29082]: fsmap cephfs:1 {0=cephfs.smithi119.nghcey=up:active} 2024-05-27T09:26:00.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:00 smithi138 ceph-mon[34885]: mds.? [v2:172.21.15.119:6826/303813653,v1:172.21.15.119:6827/303813653] up:active 2024-05-27T09:26:00.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:00 smithi138 ceph-mon[34885]: fsmap cephfs:1 {0=cephfs.smithi119.nghcey=up:active} 2024-05-27T09:26:01.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:01 smithi119 ceph-mon[29082]: pgmap v225: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-27T09:26:01.609 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:01 smithi138 ceph-mon[34885]: pgmap v225: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-27T09:26:02.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.180+0000 7f0ef0e00700 1 -- 172.21.15.119:0/3516152069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eec101350 msgr2=0x7f0eec1017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:02.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.180+0000 7f0ef0e00700 1 --2- 172.21.15.119:0/3516152069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eec101350 0x7f0eec1017a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009910 tx=0x7f0ed8009c20 comp rx=0 tx=0).stop 2024-05-27T09:26:02.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.180+0000 7f0ef0e00700 1 -- 172.21.15.119:0/3516152069 shutdown_connections 2024-05-27T09:26:02.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.180+0000 7f0ef0e00700 1 --2- 172.21.15.119:0/3516152069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eec101350 0x7f0eec1017a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:02.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.180+0000 7f0ef0e00700 1 --2- 172.21.15.119:0/3516152069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eec100150 0x7f0eec100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:02.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.180+0000 7f0ef0e00700 1 -- 172.21.15.119:0/3516152069 >> 172.21.15.119:0/3516152069 conn(0x7f0eec0fb6e0 msgr2=0x7f0eec0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:02.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.180+0000 7f0ef0e00700 1 -- 172.21.15.119:0/3516152069 shutdown_connections 2024-05-27T09:26:02.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.180+0000 7f0ef0e00700 1 -- 172.21.15.119:0/3516152069 wait complete. 2024-05-27T09:26:02.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.181+0000 7f0ef0e00700 1 Processor -- start 2024-05-27T09:26:02.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.182+0000 7f0ef0e00700 1 -- start start 2024-05-27T09:26:02.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.182+0000 7f0ef0e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eec100150 0x7f0eec197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:02.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.182+0000 7f0ef0e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eec101350 0x7f0eec1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:02.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.182+0000 7f0ef0e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec198640 con 0x7f0eec101350 2024-05-27T09:26:02.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.182+0000 7f0ef0e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec198780 con 0x7f0eec100150 2024-05-27T09:26:02.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.182+0000 7f0ee9400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eec101350 0x7f0eec1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:02.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.182+0000 7f0ee9400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eec101350 0x7f0eec1980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47568/0 (socket says 172.21.15.119:47568) 2024-05-27T09:26:02.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.182+0000 7f0ee9400700 1 -- 172.21.15.119:0/1539019370 learned_addr learned my addr 172.21.15.119:0/1539019370 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:26:02.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.183+0000 7f0ee9e00700 1 --2- 172.21.15.119:0/1539019370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eec100150 0x7f0eec197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:02.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.183+0000 7f0ee9400700 1 -- 172.21.15.119:0/1539019370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eec100150 msgr2=0x7f0eec197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:02.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.183+0000 7f0ee9400700 1 --2- 172.21.15.119:0/1539019370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eec100150 0x7f0eec197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:02.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.183+0000 7f0ee9400700 1 -- 172.21.15.119:0/1539019370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed80095f0 con 0x7f0eec101350 2024-05-27T09:26:02.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.183+0000 7f0ee9400700 1 --2- 172.21.15.119:0/1539019370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eec101350 0x7f0eec1980b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800b5c0 tx=0x7f0ed800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:02.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.183+0000 7f0ee2a00700 1 -- 172.21.15.119:0/1539019370 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ed801d070 con 0x7f0eec101350 2024-05-27T09:26:02.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.183+0000 7f0ee2a00700 1 -- 172.21.15.119:0/1539019370 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0ed8022470 con 0x7f0eec101350 2024-05-27T09:26:02.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.183+0000 7f0ee2a00700 1 -- 172.21.15.119:0/1539019370 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ed800f690 con 0x7f0eec101350 2024-05-27T09:26:02.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.184+0000 7f0ef0e00700 1 -- 172.21.15.119:0/1539019370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eec19d1e0 con 0x7f0eec101350 2024-05-27T09:26:02.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.184+0000 7f0ef0e00700 1 -- 172.21.15.119:0/1539019370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eec19d650 con 0x7f0eec101350 2024-05-27T09:26:02.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.185+0000 7f0ef0e00700 1 -- 172.21.15.119:0/1539019370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eec105430 con 0x7f0eec101350 2024-05-27T09:26:02.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.186+0000 7f0ee2a00700 1 -- 172.21.15.119:0/1539019370 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f0ed8004b70 con 0x7f0eec101350 2024-05-27T09:26:02.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.187+0000 7f0ee2a00700 1 --2- 172.21.15.119:0/1539019370 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f0ed006c680 0x7f0ed006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:02.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.187+0000 7f0ee2a00700 1 -- 172.21.15.119:0/1539019370 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f0ed808d530 con 0x7f0eec101350 2024-05-27T09:26:02.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.187+0000 7f0ee9e00700 1 --2- 172.21.15.119:0/1539019370 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f0ed006c680 0x7f0ed006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:02.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.188+0000 7f0ee9e00700 1 --2- 172.21.15.119:0/1539019370 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f0ed006c680 0x7f0ed006eb30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0eec1011b0 tx=0x7f0edc005d70 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:02.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.191+0000 7f0ee2a00700 1 -- 172.21.15.119:0/1539019370 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0ed805b730 con 0x7f0eec101350 2024-05-27T09:26:02.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:02.473+0000 7f0ef0e00700 1 -- 172.21.15.119:0/1539019370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f0eec02ce40 con 0x7f0eec101350 2024-05-27T09:26:02.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:02 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:02.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:02 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:02.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:02 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:02.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:02 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi119.ppmzzk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:26:02.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:02 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi119.ppmzzk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T09:26:02.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:02 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:26:02.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:02 smithi119 ceph-mon[29082]: Deploying daemon mds.cephfs.smithi119.ppmzzk on smithi119 2024-05-27T09:26:02.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:02 smithi119 ceph-mon[29082]: pgmap v226: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-27T09:26:02.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:02 smithi119 ceph-mon[29082]: mds.? [v2:172.21.15.138:6824/897058594,v1:172.21.15.138:6825/897058594] up:boot 2024-05-27T09:26:02.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:02 smithi119 ceph-mon[29082]: fsmap cephfs:1 {0=cephfs.smithi119.nghcey=up:active} 1 up:standby 2024-05-27T09:26:02.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:02 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.ndfkhd"}]: dispatch 2024-05-27T09:26:02.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:02 smithi119 ceph-mon[29082]: fsmap cephfs:1 {0=cephfs.smithi119.nghcey=up:active} 1 up:standby 2024-05-27T09:26:02.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:02 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/1539019370' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-05-27T09:26:03.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:02 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:03.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:02 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:03.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:02 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:03.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:02 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi119.ppmzzk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:26:03.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:02 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi119.ppmzzk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T09:26:03.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:02 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:26:03.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:02 smithi138 ceph-mon[34885]: Deploying daemon mds.cephfs.smithi119.ppmzzk on smithi119 2024-05-27T09:26:03.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:02 smithi138 ceph-mon[34885]: pgmap v226: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-27T09:26:03.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:02 smithi138 ceph-mon[34885]: mds.? [v2:172.21.15.138:6824/897058594,v1:172.21.15.138:6825/897058594] up:boot 2024-05-27T09:26:03.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:02 smithi138 ceph-mon[34885]: fsmap cephfs:1 {0=cephfs.smithi119.nghcey=up:active} 1 up:standby 2024-05-27T09:26:03.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:02 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.ndfkhd"}]: dispatch 2024-05-27T09:26:03.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:02 smithi138 ceph-mon[34885]: fsmap cephfs:1 {0=cephfs.smithi119.nghcey=up:active} 1 up:standby 2024-05-27T09:26:03.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:02 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/1539019370' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-05-27T09:26:03.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:03.459+0000 7f0ee2a00700 1 -- 172.21.15.119:0/1539019370 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v8) v1 ==== 105+0+0 (secure 0 0 0) 0x7f0ed805b2c0 con 0x7f0eec101350 2024-05-27T09:26:03.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:03.463+0000 7f0ef0e00700 1 -- 172.21.15.119:0/1539019370 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f0ed006c680 msgr2=0x7f0ed006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:03.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:03.463+0000 7f0ef0e00700 1 --2- 172.21.15.119:0/1539019370 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f0ed006c680 0x7f0ed006eb30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0eec1011b0 tx=0x7f0edc005d70 comp rx=0 tx=0).stop 2024-05-27T09:26:03.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:03.463+0000 7f0ef0e00700 1 -- 172.21.15.119:0/1539019370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eec101350 msgr2=0x7f0eec1980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:03.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:03.463+0000 7f0ef0e00700 1 --2- 172.21.15.119:0/1539019370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eec101350 0x7f0eec1980b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800b5c0 tx=0x7f0ed800be20 comp rx=0 tx=0).stop 2024-05-27T09:26:03.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:03.463+0000 7f0ef0e00700 1 -- 172.21.15.119:0/1539019370 shutdown_connections 2024-05-27T09:26:03.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:03.463+0000 7f0ef0e00700 1 --2- 172.21.15.119:0/1539019370 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f0ed006c680 0x7f0ed006eb30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:03.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:03.463+0000 7f0ef0e00700 1 --2- 172.21.15.119:0/1539019370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eec100150 0x7f0eec197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:03.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:03.463+0000 7f0ef0e00700 1 --2- 172.21.15.119:0/1539019370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eec101350 0x7f0eec1980b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:03.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:03.463+0000 7f0ef0e00700 1 -- 172.21.15.119:0/1539019370 >> 172.21.15.119:0/1539019370 conn(0x7f0eec0fb6e0 msgr2=0x7f0eec102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:03.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:03.463+0000 7f0ef0e00700 1 -- 172.21.15.119:0/1539019370 shutdown_connections 2024-05-27T09:26:03.465 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:03.463+0000 7f0ef0e00700 1 -- 172.21.15.119:0/1539019370 wait complete. 2024-05-27T09:26:03.950 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:03 smithi119 ceph-mon[29082]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-27T09:26:03.950 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:03 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/1539019370' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-05-27T09:26:03.950 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:03 smithi119 ceph-mon[29082]: daemon mds.cephfs.smithi138.ndfkhd assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-27T09:26:03.950 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:03 smithi119 ceph-mon[29082]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-05-27T09:26:03.950 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:03 smithi119 ceph-mon[29082]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-27T09:26:03.951 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:03 smithi119 ceph-mon[29082]: fsmap cephfs:1 {0=cephfs.smithi119.nghcey=up:active} 1 up:standby 2024-05-27T09:26:03.951 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:03 smithi119 ceph-mon[29082]: fsmap cephfs:2 {0=cephfs.smithi119.nghcey=up:active,1=cephfs.smithi138.ndfkhd=up:creating} 2024-05-27T09:26:03.951 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:03 smithi119 ceph-mon[29082]: daemon mds.cephfs.smithi138.ndfkhd is now active in filesystem cephfs as rank 1 2024-05-27T09:26:04.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:03 smithi138 ceph-mon[34885]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-27T09:26:04.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:03 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/1539019370' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-05-27T09:26:04.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:03 smithi138 ceph-mon[34885]: daemon mds.cephfs.smithi138.ndfkhd assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-27T09:26:04.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:03 smithi138 ceph-mon[34885]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-05-27T09:26:04.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:03 smithi138 ceph-mon[34885]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-27T09:26:04.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:03 smithi138 ceph-mon[34885]: fsmap cephfs:1 {0=cephfs.smithi119.nghcey=up:active} 1 up:standby 2024-05-27T09:26:04.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:03 smithi138 ceph-mon[34885]: fsmap cephfs:2 {0=cephfs.smithi119.nghcey=up:active,1=cephfs.smithi138.ndfkhd=up:creating} 2024-05-27T09:26:04.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:03 smithi138 ceph-mon[34885]: daemon mds.cephfs.smithi138.ndfkhd is now active in filesystem cephfs as rank 1 2024-05-27T09:26:04.823 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-27T09:26:04.833 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi119.front.sepia.ceph.com 2024-05-27T09:26:04.834 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-05-27T09:26:05.572 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:26:05.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:05 smithi138 ceph-mon[34885]: pgmap v227: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 5 op/s 2024-05-27T09:26:05.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:05 smithi138 ceph-mon[34885]: mds.? [v2:172.21.15.138:6824/897058594,v1:172.21.15.138:6825/897058594] up:active 2024-05-27T09:26:05.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:05 smithi138 ceph-mon[34885]: fsmap cephfs:2 {0=cephfs.smithi119.nghcey=up:active,1=cephfs.smithi138.ndfkhd=up:active} 2024-05-27T09:26:05.722 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:05 smithi119 ceph-mon[29082]: pgmap v227: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 5 op/s 2024-05-27T09:26:05.722 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:05 smithi119 ceph-mon[29082]: mds.? [v2:172.21.15.138:6824/897058594,v1:172.21.15.138:6825/897058594] up:active 2024-05-27T09:26:05.722 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:05 smithi119 ceph-mon[29082]: fsmap cephfs:2 {0=cephfs.smithi119.nghcey=up:active,1=cephfs.smithi138.ndfkhd=up:active} 2024-05-27T09:26:06.755 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:06 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:06.756 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:06 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:06.756 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:06 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:06.756 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:06 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.alszpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:26:06.756 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:06 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.alszpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T09:26:06.756 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:06 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:26:06.756 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:06 smithi138 ceph-mon[34885]: Deploying daemon mds.cephfs.smithi138.alszpk on smithi138 2024-05-27T09:26:06.756 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:06 smithi138 ceph-mon[34885]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-05-27T09:26:06.756 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:06 smithi138 ceph-mon[34885]: Cluster is now healthy 2024-05-27T09:26:06.756 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:06 smithi138 ceph-mon[34885]: mds.? [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] up:boot 2024-05-27T09:26:06.756 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:06 smithi138 ceph-mon[34885]: fsmap cephfs:2 {0=cephfs.smithi119.nghcey=up:active,1=cephfs.smithi138.ndfkhd=up:active} 1 up:standby 2024-05-27T09:26:06.756 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:06 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.ppmzzk"}]: dispatch 2024-05-27T09:26:06.757 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:06 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:26:06.757 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:06 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:06.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:06 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:06.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:06 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:06.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:06 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:06.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:06 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.alszpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:26:06.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:06 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.alszpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T09:26:06.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:06 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:26:06.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:06 smithi119 ceph-mon[29082]: Deploying daemon mds.cephfs.smithi138.alszpk on smithi138 2024-05-27T09:26:06.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:06 smithi119 ceph-mon[29082]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-05-27T09:26:06.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:06 smithi119 ceph-mon[29082]: Cluster is now healthy 2024-05-27T09:26:07.000 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:06 smithi119 ceph-mon[29082]: mds.? [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] up:boot 2024-05-27T09:26:07.000 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:06 smithi119 ceph-mon[29082]: fsmap cephfs:2 {0=cephfs.smithi119.nghcey=up:active,1=cephfs.smithi138.ndfkhd=up:active} 1 up:standby 2024-05-27T09:26:07.000 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:06 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.ppmzzk"}]: dispatch 2024-05-27T09:26:07.000 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:06 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:26:07.000 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:06 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:07.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.818+0000 7f4be1200700 1 -- 172.21.15.119:0/4014909898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bdc101350 msgr2=0x7f4bdc1017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:07.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.818+0000 7f4be1200700 1 --2- 172.21.15.119:0/4014909898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bdc101350 0x7f4bdc1017a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009910 tx=0x7f4bc8009c20 comp rx=0 tx=0).stop 2024-05-27T09:26:07.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.818+0000 7f4be1200700 1 -- 172.21.15.119:0/4014909898 shutdown_connections 2024-05-27T09:26:07.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.818+0000 7f4be1200700 1 --2- 172.21.15.119:0/4014909898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bdc101350 0x7f4bdc1017a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:07.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.818+0000 7f4be1200700 1 --2- 172.21.15.119:0/4014909898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bdc100150 0x7f4bdc100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:07.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.818+0000 7f4be1200700 1 -- 172.21.15.119:0/4014909898 >> 172.21.15.119:0/4014909898 conn(0x7f4bdc0fb6e0 msgr2=0x7f4bdc0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:07.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.818+0000 7f4be1200700 1 -- 172.21.15.119:0/4014909898 shutdown_connections 2024-05-27T09:26:07.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.818+0000 7f4be1200700 1 -- 172.21.15.119:0/4014909898 wait complete. 2024-05-27T09:26:07.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.819+0000 7f4be1200700 1 Processor -- start 2024-05-27T09:26:07.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.819+0000 7f4be1200700 1 -- start start 2024-05-27T09:26:07.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.820+0000 7f4be1200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bdc100150 0x7f4bdc074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:07.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.820+0000 7f4be1200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bdc101350 0x7f4bdc073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:07.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.820+0000 7f4be1200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc073680 con 0x7f4bdc101350 2024-05-27T09:26:07.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.820+0000 7f4be1200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc0737c0 con 0x7f4bdc100150 2024-05-27T09:26:07.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.820+0000 7f4bd9400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bdc101350 0x7f4bdc073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:07.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.820+0000 7f4bd9400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bdc101350 0x7f4bdc073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47604/0 (socket says 172.21.15.119:47604) 2024-05-27T09:26:07.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.820+0000 7f4bd9400700 1 -- 172.21.15.119:0/3618864637 learned_addr learned my addr 172.21.15.119:0/3618864637 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:26:07.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.820+0000 7f4bd9e00700 1 --2- 172.21.15.119:0/3618864637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bdc100150 0x7f4bdc074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:07.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.820+0000 7f4bd9400700 1 -- 172.21.15.119:0/3618864637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bdc100150 msgr2=0x7f4bdc074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:07.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.820+0000 7f4bd9400700 1 --2- 172.21.15.119:0/3618864637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bdc100150 0x7f4bdc074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:07.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.820+0000 7f4bd9400700 1 -- 172.21.15.119:0/3618864637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc80095f0 con 0x7f4bdc101350 2024-05-27T09:26:07.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.821+0000 7f4bd9400700 1 --2- 172.21.15.119:0/3618864637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bdc101350 0x7f4bdc073140 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8005b40 tx=0x7f4bc800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:07.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.821+0000 7f4bd2a00700 1 -- 172.21.15.119:0/3618864637 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bc801d070 con 0x7f4bdc101350 2024-05-27T09:26:07.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.821+0000 7f4bd2a00700 1 -- 172.21.15.119:0/3618864637 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4bc800f460 con 0x7f4bdc101350 2024-05-27T09:26:07.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.821+0000 7f4be1200700 1 -- 172.21.15.119:0/3618864637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bdc073a40 con 0x7f4bdc101350 2024-05-27T09:26:07.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.821+0000 7f4bd2a00700 1 -- 172.21.15.119:0/3618864637 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bc8021640 con 0x7f4bdc101350 2024-05-27T09:26:07.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.821+0000 7f4be1200700 1 -- 172.21.15.119:0/3618864637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bdc073f30 con 0x7f4bdc101350 2024-05-27T09:26:07.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.822+0000 7f4be1200700 1 -- 172.21.15.119:0/3618864637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bdc108ca0 con 0x7f4bdc101350 2024-05-27T09:26:07.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.823+0000 7f4bd2a00700 1 -- 172.21.15.119:0/3618864637 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f4bc800fa90 con 0x7f4bdc101350 2024-05-27T09:26:07.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.824+0000 7f4bd2a00700 1 --2- 172.21.15.119:0/3618864637 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4bc006c680 0x7f4bc006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:07.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.824+0000 7f4bd2a00700 1 -- 172.21.15.119:0/3618864637 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f4bc808cb40 con 0x7f4bdc101350 2024-05-27T09:26:07.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.825+0000 7f4bd9e00700 1 --2- 172.21.15.119:0/3618864637 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4bc006c680 0x7f4bc006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:07.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.825+0000 7f4bd9e00700 1 --2- 172.21.15.119:0/3618864637 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4bc006c680 0x7f4bc006eb30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc005950 tx=0x7f4bcc0058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:07.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:07.829+0000 7f4bd2a00700 1 -- 172.21.15.119:0/3618864637 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4bc805ad30 con 0x7f4bdc101350 2024-05-27T09:26:07.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:07 smithi138 ceph-mon[34885]: pgmap v228: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 6 op/s 2024-05-27T09:26:07.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:07 smithi119 ceph-mon[29082]: pgmap v228: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 6 op/s 2024-05-27T09:26:08.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:08.095+0000 7f4be1200700 1 -- 172.21.15.119:0/3618864637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4bdc19df30 con 0x7f4bdc101350 2024-05-27T09:26:08.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:08.607+0000 7f4bd2a00700 1 -- 172.21.15.119:0/3618864637 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4bc805a8c0 con 0x7f4bdc101350 2024-05-27T09:26:08.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:08.613+0000 7f4be1200700 1 -- 172.21.15.119:0/3618864637 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4bc006c680 msgr2=0x7f4bc006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:08.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:08.613+0000 7f4be1200700 1 --2- 172.21.15.119:0/3618864637 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4bc006c680 0x7f4bc006eb30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc005950 tx=0x7f4bcc0058e0 comp rx=0 tx=0).stop 2024-05-27T09:26:08.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:08.613+0000 7f4be1200700 1 -- 172.21.15.119:0/3618864637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bdc101350 msgr2=0x7f4bdc073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:08.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:08.613+0000 7f4be1200700 1 --2- 172.21.15.119:0/3618864637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bdc101350 0x7f4bdc073140 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8005b40 tx=0x7f4bc800be20 comp rx=0 tx=0).stop 2024-05-27T09:26:08.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:08.613+0000 7f4be1200700 1 -- 172.21.15.119:0/3618864637 shutdown_connections 2024-05-27T09:26:08.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:08.613+0000 7f4be1200700 1 --2- 172.21.15.119:0/3618864637 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4bc006c680 0x7f4bc006eb30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:08.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:08.613+0000 7f4be1200700 1 --2- 172.21.15.119:0/3618864637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bdc100150 0x7f4bdc074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:08.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:08.613+0000 7f4be1200700 1 --2- 172.21.15.119:0/3618864637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bdc101350 0x7f4bdc073140 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:08.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:08.613+0000 7f4be1200700 1 -- 172.21.15.119:0/3618864637 >> 172.21.15.119:0/3618864637 conn(0x7f4bdc0fb6e0 msgr2=0x7f4bdc104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:08.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:08.613+0000 7f4be1200700 1 -- 172.21.15.119:0/3618864637 shutdown_connections 2024-05-27T09:26:08.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:08.614+0000 7f4be1200700 1 -- 172.21.15.119:0/3618864637 wait complete. 2024-05-27T09:26:08.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:08 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/3618864637' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-05-27T09:26:08.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:08 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/3618864637' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-05-27T09:26:09.456 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-27T09:26:09.466 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi119.front.sepia.ceph.com 2024-05-27T09:26:09.466 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-05-27T09:26:09.687 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:09 smithi119 ceph-mon[29082]: pgmap v229: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s wr, 6 op/s 2024-05-27T09:26:09.688 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:09 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/3618864637' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-05-27T09:26:09.688 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:09 smithi119 ceph-mon[29082]: fsmap cephfs:2 {0=cephfs.smithi119.nghcey=up:active,1=cephfs.smithi138.ndfkhd=up:active} 1 up:standby 2024-05-27T09:26:09.688 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:09 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:09.688 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:09 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:09.688 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:09 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:09.688 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:09 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:09.688 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:09 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:09.688 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:09 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:26:09.877 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:26:09.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:09 smithi138 ceph-mon[34885]: pgmap v229: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s wr, 6 op/s 2024-05-27T09:26:09.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:09 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/3618864637' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-05-27T09:26:09.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:09 smithi138 ceph-mon[34885]: fsmap cephfs:2 {0=cephfs.smithi119.nghcey=up:active,1=cephfs.smithi138.ndfkhd=up:active} 1 up:standby 2024-05-27T09:26:09.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:09 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:09.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:09 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:09.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:09 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:09.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:09 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:09.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:09 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:09.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:09 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:26:10.879 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:10 smithi119 ceph-mon[29082]: mds.? [v2:172.21.15.138:6826/2345607053,v1:172.21.15.138:6827/2345607053] up:boot 2024-05-27T09:26:10.879 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:10 smithi119 ceph-mon[29082]: mds.? [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] up:standby 2024-05-27T09:26:10.879 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:10 smithi119 ceph-mon[29082]: Dropping low affinity active daemon mds.cephfs.smithi119.nghcey in favor of higher affinity standby. 2024-05-27T09:26:10.879 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:10 smithi119 ceph-mon[29082]: Replacing daemon mds.cephfs.smithi119.nghcey as rank 0 with standby daemon mds.cephfs.smithi119.ppmzzk 2024-05-27T09:26:10.879 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:10 smithi119 ceph-mon[29082]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-27T09:26:10.879 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:10 smithi119 ceph-mon[29082]: fsmap cephfs:2 {0=cephfs.smithi119.nghcey=up:active,1=cephfs.smithi138.ndfkhd=up:active} 2 up:standby 2024-05-27T09:26:10.879 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:10 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.alszpk"}]: dispatch 2024-05-27T09:26:10.879 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:10 smithi119 ceph-mon[29082]: osdmap e46: 6 total, 6 up, 6 in 2024-05-27T09:26:10.879 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:10 smithi119 ceph-mon[29082]: fsmap cephfs:2/2 {0=cephfs.smithi119.ppmzzk=up:replay,1=cephfs.smithi138.ndfkhd=up:active} 1 up:standby 2024-05-27T09:26:10.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:10 smithi138 ceph-mon[34885]: mds.? [v2:172.21.15.138:6826/2345607053,v1:172.21.15.138:6827/2345607053] up:boot 2024-05-27T09:26:10.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:10 smithi138 ceph-mon[34885]: mds.? [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] up:standby 2024-05-27T09:26:10.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:10 smithi138 ceph-mon[34885]: Dropping low affinity active daemon mds.cephfs.smithi119.nghcey in favor of higher affinity standby. 2024-05-27T09:26:10.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:10 smithi138 ceph-mon[34885]: Replacing daemon mds.cephfs.smithi119.nghcey as rank 0 with standby daemon mds.cephfs.smithi119.ppmzzk 2024-05-27T09:26:10.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:10 smithi138 ceph-mon[34885]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-27T09:26:10.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:10 smithi138 ceph-mon[34885]: fsmap cephfs:2 {0=cephfs.smithi119.nghcey=up:active,1=cephfs.smithi138.ndfkhd=up:active} 2 up:standby 2024-05-27T09:26:10.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:10 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.alszpk"}]: dispatch 2024-05-27T09:26:10.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:10 smithi138 ceph-mon[34885]: osdmap e46: 6 total, 6 up, 6 in 2024-05-27T09:26:10.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:10 smithi138 ceph-mon[34885]: fsmap cephfs:2/2 {0=cephfs.smithi119.ppmzzk=up:replay,1=cephfs.smithi138.ndfkhd=up:active} 1 up:standby 2024-05-27T09:26:11.923 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:11 smithi138 ceph-mon[34885]: pgmap v231: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-27T09:26:11.923 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:11 smithi138 ceph-mon[34885]: mds.? [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] up:resolve 2024-05-27T09:26:11.923 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:11 smithi138 ceph-mon[34885]: mds.? [v2:172.21.15.119:6826/413736191,v1:172.21.15.119:6827/413736191] up:boot 2024-05-27T09:26:11.923 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:11 smithi138 ceph-mon[34885]: fsmap cephfs:2/2 {0=cephfs.smithi119.ppmzzk=up:resolve,1=cephfs.smithi138.ndfkhd=up:active} 2 up:standby 2024-05-27T09:26:11.923 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:11 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.nghcey"}]: dispatch 2024-05-27T09:26:11.923 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:11 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:11.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:11 smithi119 ceph-mon[29082]: pgmap v231: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-27T09:26:11.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:11 smithi119 ceph-mon[29082]: mds.? [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] up:resolve 2024-05-27T09:26:11.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:11 smithi119 ceph-mon[29082]: mds.? [v2:172.21.15.119:6826/413736191,v1:172.21.15.119:6827/413736191] up:boot 2024-05-27T09:26:11.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:11 smithi119 ceph-mon[29082]: fsmap cephfs:2/2 {0=cephfs.smithi119.ppmzzk=up:resolve,1=cephfs.smithi138.ndfkhd=up:active} 2 up:standby 2024-05-27T09:26:11.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:11 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.nghcey"}]: dispatch 2024-05-27T09:26:11.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:11 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:12.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.654+0000 7fcba5600700 1 -- 172.21.15.119:0/1908899484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcba0103500 msgr2=0x7fcba01058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:12.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.654+0000 7fcba5600700 1 --2- 172.21.15.119:0/1908899484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcba0103500 0x7fcba01058e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c009910 tx=0x7fcb8c009c20 comp rx=0 tx=0).stop 2024-05-27T09:26:12.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.654+0000 7fcba5600700 1 -- 172.21.15.119:0/1908899484 shutdown_connections 2024-05-27T09:26:12.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.654+0000 7fcba5600700 1 --2- 172.21.15.119:0/1908899484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcba0103500 0x7fcba01058e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:12.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.654+0000 7fcba5600700 1 --2- 172.21.15.119:0/1908899484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcba0100be0 0x7fcba0102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:12.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.654+0000 7fcba5600700 1 -- 172.21.15.119:0/1908899484 >> 172.21.15.119:0/1908899484 conn(0x7fcba00fa5e0 msgr2=0x7fcba00fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:12.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.654+0000 7fcba5600700 1 -- 172.21.15.119:0/1908899484 shutdown_connections 2024-05-27T09:26:12.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.654+0000 7fcba5600700 1 -- 172.21.15.119:0/1908899484 wait complete. 2024-05-27T09:26:12.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.656+0000 7fcba5600700 1 Processor -- start 2024-05-27T09:26:12.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.656+0000 7fcba5600700 1 -- start start 2024-05-27T09:26:12.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.656+0000 7fcba5600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcba0100be0 0x7fcba0195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:12.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.656+0000 7fcba5600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcba0103500 0x7fcba0195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:12.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.656+0000 7fcba5600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba0196490 con 0x7fcba0103500 2024-05-27T09:26:12.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.656+0000 7fcba5600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba01965d0 con 0x7fcba0100be0 2024-05-27T09:26:12.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.657+0000 7fcb9ea00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcba0100be0 0x7fcba0195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:12.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.657+0000 7fcb9ea00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcba0100be0 0x7fcba0195930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.119:54818/0 (socket says 172.21.15.119:54818) 2024-05-27T09:26:12.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.657+0000 7fcb9ea00700 1 -- 172.21.15.119:0/2864945335 learned_addr learned my addr 172.21.15.119:0/2864945335 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:26:12.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.657+0000 7fcb9e000700 1 --2- 172.21.15.119:0/2864945335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcba0103500 0x7fcba0195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:12.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.657+0000 7fcb9ea00700 1 -- 172.21.15.119:0/2864945335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcba0103500 msgr2=0x7fcba0195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:12.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.657+0000 7fcb9ea00700 1 --2- 172.21.15.119:0/2864945335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcba0103500 0x7fcba0195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:12.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.657+0000 7fcb9ea00700 1 -- 172.21.15.119:0/2864945335 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c0095f0 con 0x7fcba0100be0 2024-05-27T09:26:12.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.658+0000 7fcb9ea00700 1 --2- 172.21.15.119:0/2864945335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcba0100be0 0x7fcba0195930 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000d8d0 tx=0x7fcb9000dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:12.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.659+0000 7fcb97400700 1 -- 172.21.15.119:0/2864945335 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb900098a0 con 0x7fcba0100be0 2024-05-27T09:26:12.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.659+0000 7fcba5600700 1 -- 172.21.15.119:0/2864945335 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba019b080 con 0x7fcba0100be0 2024-05-27T09:26:12.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.659+0000 7fcb97400700 1 -- 172.21.15.119:0/2864945335 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcb90010460 con 0x7fcba0100be0 2024-05-27T09:26:12.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.659+0000 7fcb97400700 1 -- 172.21.15.119:0/2864945335 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb9000f5f0 con 0x7fcba0100be0 2024-05-27T09:26:12.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.659+0000 7fcba5600700 1 -- 172.21.15.119:0/2864945335 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba019b5d0 con 0x7fcba0100be0 2024-05-27T09:26:12.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.660+0000 7fcba5600700 1 -- 172.21.15.119:0/2864945335 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba018fb70 con 0x7fcba0100be0 2024-05-27T09:26:12.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.661+0000 7fcb97400700 1 -- 172.21.15.119:0/2864945335 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fcb90009a00 con 0x7fcba0100be0 2024-05-27T09:26:12.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.661+0000 7fcb97400700 1 --2- 172.21.15.119:0/2864945335 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fcb8406c680 0x7fcb8406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:12.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.662+0000 7fcb97400700 1 -- 172.21.15.119:0/2864945335 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fcb9008b880 con 0x7fcba0100be0 2024-05-27T09:26:12.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.662+0000 7fcb9e000700 1 --2- 172.21.15.119:0/2864945335 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fcb8406c680 0x7fcb8406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:12.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.662+0000 7fcb9e000700 1 --2- 172.21.15.119:0/2864945335 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fcb8406c680 0x7fcb8406eb30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00b5c0 tx=0x7fcb8c005ee0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:12.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.671+0000 7fcb97400700 1 -- 172.21.15.119:0/2864945335 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fcb900598f0 con 0x7fcba0100be0 2024-05-27T09:26:12.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:12 smithi138 ceph-mon[34885]: mds.? [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] up:reconnect 2024-05-27T09:26:12.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:12 smithi138 ceph-mon[34885]: mds.? [v2:172.21.15.138:6824/897058594,v1:172.21.15.138:6825/897058594] up:active 2024-05-27T09:26:12.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:12 smithi138 ceph-mon[34885]: fsmap cephfs:2/2 {0=cephfs.smithi119.ppmzzk=up:reconnect,1=cephfs.smithi138.ndfkhd=up:active} 2 up:standby 2024-05-27T09:26:12.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:12 smithi138 ceph-mon[34885]: mds.? [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] up:rejoin 2024-05-27T09:26:12.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:12 smithi138 ceph-mon[34885]: fsmap cephfs:2/2 {0=cephfs.smithi119.ppmzzk=up:rejoin,1=cephfs.smithi138.ndfkhd=up:active} 2 up:standby 2024-05-27T09:26:12.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:12 smithi138 ceph-mon[34885]: daemon mds.cephfs.smithi119.ppmzzk is now active in filesystem cephfs as rank 0 2024-05-27T09:26:12.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:12.995+0000 7fcba5600700 1 -- 172.21.15.119:0/2864945335 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcba002cc30 con 0x7fcba0100be0 2024-05-27T09:26:12.996 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:12 smithi119 ceph-mon[29082]: mds.? [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] up:reconnect 2024-05-27T09:26:12.996 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:12 smithi119 ceph-mon[29082]: mds.? [v2:172.21.15.138:6824/897058594,v1:172.21.15.138:6825/897058594] up:active 2024-05-27T09:26:12.996 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:12 smithi119 ceph-mon[29082]: fsmap cephfs:2/2 {0=cephfs.smithi119.ppmzzk=up:reconnect,1=cephfs.smithi138.ndfkhd=up:active} 2 up:standby 2024-05-27T09:26:12.996 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:12 smithi119 ceph-mon[29082]: mds.? [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] up:rejoin 2024-05-27T09:26:12.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:12 smithi119 ceph-mon[29082]: fsmap cephfs:2/2 {0=cephfs.smithi119.ppmzzk=up:rejoin,1=cephfs.smithi138.ndfkhd=up:active} 2 up:standby 2024-05-27T09:26:12.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:12 smithi119 ceph-mon[29082]: daemon mds.cephfs.smithi119.ppmzzk is now active in filesystem cephfs as rank 0 2024-05-27T09:26:13.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:13.651+0000 7fcb97400700 1 -- 172.21.15.119:0/2864945335 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v18) v1 ==== 168+0+0 (secure 0 0 0) 0x7fcb90059480 con 0x7fcba0100be0 2024-05-27T09:26:13.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:13.655+0000 7fcba5600700 1 -- 172.21.15.119:0/2864945335 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fcb8406c680 msgr2=0x7fcb8406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:13.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:13.655+0000 7fcba5600700 1 --2- 172.21.15.119:0/2864945335 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fcb8406c680 0x7fcb8406eb30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00b5c0 tx=0x7fcb8c005ee0 comp rx=0 tx=0).stop 2024-05-27T09:26:13.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:13.655+0000 7fcba5600700 1 -- 172.21.15.119:0/2864945335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcba0100be0 msgr2=0x7fcba0195930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:13.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:13.655+0000 7fcba5600700 1 --2- 172.21.15.119:0/2864945335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcba0100be0 0x7fcba0195930 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000d8d0 tx=0x7fcb9000dbe0 comp rx=0 tx=0).stop 2024-05-27T09:26:13.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:13.656+0000 7fcba5600700 1 -- 172.21.15.119:0/2864945335 shutdown_connections 2024-05-27T09:26:13.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:13.656+0000 7fcba5600700 1 --2- 172.21.15.119:0/2864945335 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fcb8406c680 0x7fcb8406eb30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:13.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:13.656+0000 7fcba5600700 1 --2- 172.21.15.119:0/2864945335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcba0100be0 0x7fcba0195930 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:13.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:13.656+0000 7fcba5600700 1 --2- 172.21.15.119:0/2864945335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcba0103500 0x7fcba0195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:13.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:13.656+0000 7fcba5600700 1 -- 172.21.15.119:0/2864945335 >> 172.21.15.119:0/2864945335 conn(0x7fcba00fa5e0 msgr2=0x7fcba00fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:13.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:13.656+0000 7fcba5600700 1 -- 172.21.15.119:0/2864945335 shutdown_connections 2024-05-27T09:26:13.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:13.656+0000 7fcba5600700 1 -- 172.21.15.119:0/2864945335 wait complete. 2024-05-27T09:26:13.658 INFO:teuthology.orchestra.run.smithi119.stderr:inline data enabled 2024-05-27T09:26:13.979 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:13 smithi119 ceph-mon[29082]: pgmap v232: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 102 B/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-27T09:26:13.979 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:13 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/2864945335' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-27T09:26:13.979 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:13 smithi119 ceph-mon[29082]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-27T09:26:13.980 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:13 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:13.980 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:13 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:13.980 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:13 smithi119 ceph-mon[29082]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-05-27T09:26:13.980 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:13 smithi119 ceph-mon[29082]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-27T09:26:13.980 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:13 smithi119 ceph-mon[29082]: mds.? [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] up:active 2024-05-27T09:26:13.980 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:13 smithi119 ceph-mon[29082]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-05-27T09:26:13.980 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:13 smithi119 ceph-mon[29082]: mds.? [v2:172.21.15.138:6826/2345607053,v1:172.21.15.138:6827/2345607053] up:standby 2024-05-27T09:26:13.980 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:13 smithi119 ceph-mon[29082]: fsmap cephfs:2 {0=cephfs.smithi119.ppmzzk=up:active,1=cephfs.smithi138.ndfkhd=up:active} 2 up:standby 2024-05-27T09:26:14.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:13 smithi138 ceph-mon[34885]: pgmap v232: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 102 B/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-27T09:26:14.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:13 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/2864945335' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-27T09:26:14.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:13 smithi138 ceph-mon[34885]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-27T09:26:14.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:13 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:14.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:13 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:14.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:13 smithi138 ceph-mon[34885]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-05-27T09:26:14.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:13 smithi138 ceph-mon[34885]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-27T09:26:14.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:13 smithi138 ceph-mon[34885]: mds.? [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] up:active 2024-05-27T09:26:14.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:13 smithi138 ceph-mon[34885]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-05-27T09:26:14.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:13 smithi138 ceph-mon[34885]: mds.? [v2:172.21.15.138:6826/2345607053,v1:172.21.15.138:6827/2345607053] up:standby 2024-05-27T09:26:14.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:13 smithi138 ceph-mon[34885]: fsmap cephfs:2 {0=cephfs.smithi119.ppmzzk=up:active,1=cephfs.smithi138.ndfkhd=up:active} 2 up:standby 2024-05-27T09:26:14.609 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-27T09:26:14.621 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi119.front.sepia.ceph.com 2024-05-27T09:26:14.621 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-27T09:26:15.349 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:26:15.853 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:15 smithi119 ceph-mon[29082]: pgmap v233: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-27T09:26:16.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:15 smithi138 ceph-mon[34885]: pgmap v233: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-27T09:26:17.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:17 smithi138 ceph-mon[34885]: pgmap v234: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.6 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-27T09:26:17.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:17 smithi119 ceph-mon[29082]: pgmap v234: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.6 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-27T09:26:18.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.613+0000 7f703cc00700 1 -- 172.21.15.119:0/243171398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7038076950 msgr2=0x7f7038076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:18.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.613+0000 7f703cc00700 1 --2- 172.21.15.119:0/243171398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7038076950 0x7f7038076dc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7024009910 tx=0x7f7024009c20 comp rx=0 tx=0).stop 2024-05-27T09:26:18.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.613+0000 7f703cc00700 1 -- 172.21.15.119:0/243171398 shutdown_connections 2024-05-27T09:26:18.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.613+0000 7f703cc00700 1 --2- 172.21.15.119:0/243171398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7038076950 0x7f7038076dc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:18.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.613+0000 7f703cc00700 1 --2- 172.21.15.119:0/243171398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7038075700 0x7f7038075b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:18.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.613+0000 7f703cc00700 1 -- 172.21.15.119:0/243171398 >> 172.21.15.119:0/243171398 conn(0x7f70380fd930 msgr2=0x7f70380ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:18.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.613+0000 7f703cc00700 1 -- 172.21.15.119:0/243171398 shutdown_connections 2024-05-27T09:26:18.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.613+0000 7f703cc00700 1 -- 172.21.15.119:0/243171398 wait complete. 2024-05-27T09:26:18.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.614+0000 7f703cc00700 1 Processor -- start 2024-05-27T09:26:18.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.615+0000 7f703cc00700 1 -- start start 2024-05-27T09:26:18.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.615+0000 7f703cc00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7038075700 0x7f703819bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:18.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.615+0000 7f703cc00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7038076950 0x7f703819c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:18.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.615+0000 7f703cc00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703819caa0 con 0x7f7038076950 2024-05-27T09:26:18.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.615+0000 7f703cc00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703819cbe0 con 0x7f7038075700 2024-05-27T09:26:18.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.615+0000 7f7035400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7038076950 0x7f703819c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:18.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.615+0000 7f7035400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7038076950 0x7f703819c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53360/0 (socket says 172.21.15.119:53360) 2024-05-27T09:26:18.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.615+0000 7f7035400700 1 -- 172.21.15.119:0/3819946792 learned_addr learned my addr 172.21.15.119:0/3819946792 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:26:18.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.616+0000 7f7035e00700 1 --2- 172.21.15.119:0/3819946792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7038075700 0x7f703819bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:18.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.616+0000 7f7035400700 1 -- 172.21.15.119:0/3819946792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7038075700 msgr2=0x7f703819bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:18.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.616+0000 7f7035400700 1 --2- 172.21.15.119:0/3819946792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7038075700 0x7f703819bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:18.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.616+0000 7f7035400700 1 -- 172.21.15.119:0/3819946792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70240095f0 con 0x7f7038076950 2024-05-27T09:26:18.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.616+0000 7f7035400700 1 --2- 172.21.15.119:0/3819946792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7038076950 0x7f703819c480 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7024005b40 tx=0x7f702400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:18.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.617+0000 7f702ea00700 1 -- 172.21.15.119:0/3819946792 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f702401d070 con 0x7f7038076950 2024-05-27T09:26:18.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.617+0000 7f702ea00700 1 -- 172.21.15.119:0/3819946792 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f702400f460 con 0x7f7038076950 2024-05-27T09:26:18.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.617+0000 7f703cc00700 1 -- 172.21.15.119:0/3819946792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70381a1630 con 0x7f7038076950 2024-05-27T09:26:18.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.617+0000 7f702ea00700 1 -- 172.21.15.119:0/3819946792 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7024021640 con 0x7f7038076950 2024-05-27T09:26:18.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.617+0000 7f703cc00700 1 -- 172.21.15.119:0/3819946792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70381a1b20 con 0x7f7038076950 2024-05-27T09:26:18.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.618+0000 7f703cc00700 1 -- 172.21.15.119:0/3819946792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f703810f2d0 con 0x7f7038076950 2024-05-27T09:26:18.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.620+0000 7f702ea00700 1 -- 172.21.15.119:0/3819946792 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f702400fa90 con 0x7f7038076950 2024-05-27T09:26:18.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.621+0000 7f702ea00700 1 --2- 172.21.15.119:0/3819946792 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f701c06c680 0x7f701c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:18.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.621+0000 7f702ea00700 1 -- 172.21.15.119:0/3819946792 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f702408cff0 con 0x7f7038076950 2024-05-27T09:26:18.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.621+0000 7f7035e00700 1 --2- 172.21.15.119:0/3819946792 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f701c06c680 0x7f701c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:18.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.622+0000 7f7035e00700 1 --2- 172.21.15.119:0/3819946792 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f701c06c680 0x7f701c06eb30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7028005950 tx=0x7f70280058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:18.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.625+0000 7f702ea00700 1 -- 172.21.15.119:0/3819946792 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f702405b1c0 con 0x7f7038076950 2024-05-27T09:26:18.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.908+0000 7f703cc00700 1 -- 172.21.15.119:0/3819946792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f70381a2270 con 0x7f7038076950 2024-05-27T09:26:18.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.909+0000 7f702ea00700 1 -- 172.21.15.119:0/3819946792 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1836 (secure 0 0 0) 0x7f702405ad50 con 0x7f7038076950 2024-05-27T09:26:18.909 INFO:teuthology.orchestra.run.smithi119.stdout:e18 2024-05-27T09:26:18.910 INFO:teuthology.orchestra.run.smithi119.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T09:26:18.910 INFO:teuthology.orchestra.run.smithi119.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:26:18.910 INFO:teuthology.orchestra.run.smithi119.stdout:legacy client fscid: 1 2024-05-27T09:26:18.910 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:26:18.910 INFO:teuthology.orchestra.run.smithi119.stdout:Filesystem 'cephfs' (1) 2024-05-27T09:26:18.910 INFO:teuthology.orchestra.run.smithi119.stdout:fs_name cephfs 2024-05-27T09:26:18.910 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 18 2024-05-27T09:26:18.910 INFO:teuthology.orchestra.run.smithi119.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-27T09:26:18.910 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-05-27T09:25:53.387388+0000 2024-05-27T09:26:18.910 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2024-05-27T09:26:13.642209+0000 2024-05-27T09:26:18.910 INFO:teuthology.orchestra.run.smithi119.stdout:tableserver 0 2024-05-27T09:26:18.910 INFO:teuthology.orchestra.run.smithi119.stdout:root 0 2024-05-27T09:26:18.911 INFO:teuthology.orchestra.run.smithi119.stdout:session_timeout 60 2024-05-27T09:26:18.911 INFO:teuthology.orchestra.run.smithi119.stdout:session_autoclose 300 2024-05-27T09:26:18.911 INFO:teuthology.orchestra.run.smithi119.stdout:max_file_size 1099511627776 2024-05-27T09:26:18.911 INFO:teuthology.orchestra.run.smithi119.stdout:required_client_features {} 2024-05-27T09:26:18.911 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure 0 2024-05-27T09:26:18.911 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure_osd_epoch 46 2024-05-27T09:26:18.911 INFO:teuthology.orchestra.run.smithi119.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:26:18.911 INFO:teuthology.orchestra.run.smithi119.stdout:max_mds 2 2024-05-27T09:26:18.911 INFO:teuthology.orchestra.run.smithi119.stdout:in 0,1 2024-05-27T09:26:18.911 INFO:teuthology.orchestra.run.smithi119.stdout:up {0=14472,1=24241} 2024-05-27T09:26:18.911 INFO:teuthology.orchestra.run.smithi119.stdout:failed 2024-05-27T09:26:18.911 INFO:teuthology.orchestra.run.smithi119.stdout:damaged 2024-05-27T09:26:18.911 INFO:teuthology.orchestra.run.smithi119.stdout:stopped 2024-05-27T09:26:18.911 INFO:teuthology.orchestra.run.smithi119.stdout:data_pools [3] 2024-05-27T09:26:18.911 INFO:teuthology.orchestra.run.smithi119.stdout:metadata_pool 2 2024-05-27T09:26:18.912 INFO:teuthology.orchestra.run.smithi119.stdout:inline_data enabled 2024-05-27T09:26:18.912 INFO:teuthology.orchestra.run.smithi119.stdout:balancer 2024-05-27T09:26:18.912 INFO:teuthology.orchestra.run.smithi119.stdout:bal_rank_mask -1 2024-05-27T09:26:18.912 INFO:teuthology.orchestra.run.smithi119.stdout:standby_count_wanted 1 2024-05-27T09:26:18.912 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.ppmzzk{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:26:18.912 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.ndfkhd{1:24241} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/897058594,v1:172.21.15.138:6825/897058594] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:26:18.912 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:26:18.912 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:26:18.912 INFO:teuthology.orchestra.run.smithi119.stdout:Standby daemons: 2024-05-27T09:26:18.912 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:26:18.912 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.nghcey{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.119:6826/413736191,v1:172.21.15.119:6827/413736191] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:26:18.912 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.alszpk{-1:24253} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/2345607053,v1:172.21.15.138:6827/2345607053] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:26:18.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.912+0000 7f703cc00700 1 -- 172.21.15.119:0/3819946792 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f701c06c680 msgr2=0x7f701c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:18.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.912+0000 7f703cc00700 1 --2- 172.21.15.119:0/3819946792 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f701c06c680 0x7f701c06eb30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7028005950 tx=0x7f70280058e0 comp rx=0 tx=0).stop 2024-05-27T09:26:18.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.913+0000 7f703cc00700 1 -- 172.21.15.119:0/3819946792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7038076950 msgr2=0x7f703819c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:18.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.913+0000 7f703cc00700 1 --2- 172.21.15.119:0/3819946792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7038076950 0x7f703819c480 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7024005b40 tx=0x7f702400be20 comp rx=0 tx=0).stop 2024-05-27T09:26:18.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.913+0000 7f703cc00700 1 -- 172.21.15.119:0/3819946792 shutdown_connections 2024-05-27T09:26:18.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.913+0000 7f703cc00700 1 --2- 172.21.15.119:0/3819946792 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f701c06c680 0x7f701c06eb30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:18.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.913+0000 7f703cc00700 1 --2- 172.21.15.119:0/3819946792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7038075700 0x7f703819bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:18.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.913+0000 7f703cc00700 1 --2- 172.21.15.119:0/3819946792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7038076950 0x7f703819c480 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:18.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.913+0000 7f703cc00700 1 -- 172.21.15.119:0/3819946792 >> 172.21.15.119:0/3819946792 conn(0x7f70380fd930 msgr2=0x7f703810c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:18.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.913+0000 7f703cc00700 1 -- 172.21.15.119:0/3819946792 shutdown_connections 2024-05-27T09:26:18.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:18.913+0000 7f703cc00700 1 -- 172.21.15.119:0/3819946792 wait complete. 2024-05-27T09:26:18.916 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 18 2024-05-27T09:26:19.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:19 smithi138 ceph-mon[34885]: pgmap v235: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.6 KiB/s rd, 409 B/s wr, 8 op/s 2024-05-27T09:26:19.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:19 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/3819946792' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:26:19.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:19 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:19.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:19 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:19.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:19 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:26:19.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:19 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:26:19.712 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:19 smithi119 ceph-mon[29082]: pgmap v235: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.6 KiB/s rd, 409 B/s wr, 8 op/s 2024-05-27T09:26:19.712 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:19 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/3819946792' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:26:19.712 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:19 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:19.712 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:19 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:19.712 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:19 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:26:19.712 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:19 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:26:19.758 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-05-27T09:26:20.150 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:26:20.586 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:20 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:20.586 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:20 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:26:20.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:20.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:20 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:26:21.469 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:21 smithi119 ceph-mon[29082]: pgmap v236: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-27T09:26:21.469 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:21 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:21.469 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:21 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:26:21.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:21 smithi138 ceph-mon[34885]: pgmap v236: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-27T09:26:21.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:21 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:21.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:21 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:26:22.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.727+0000 7f6eb3e00700 1 -- 172.21.15.119:0/3335629622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6eac1013f0 msgr2=0x7f6eac101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:22.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.727+0000 7f6eb3e00700 1 --2- 172.21.15.119:0/3335629622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6eac1013f0 0x7f6eac101840 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009910 tx=0x7f6e98009c20 comp rx=0 tx=0).stop 2024-05-27T09:26:22.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.727+0000 7f6eb3e00700 1 -- 172.21.15.119:0/3335629622 shutdown_connections 2024-05-27T09:26:22.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.727+0000 7f6eb3e00700 1 --2- 172.21.15.119:0/3335629622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6eac1013f0 0x7f6eac101840 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:22.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.727+0000 7f6eb3e00700 1 --2- 172.21.15.119:0/3335629622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eac0fe410 0x7f6eac0fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:22.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.727+0000 7f6eb3e00700 1 -- 172.21.15.119:0/3335629622 >> 172.21.15.119:0/3335629622 conn(0x7f6eac0f9d80 msgr2=0x7f6eac0fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:22.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.727+0000 7f6eb3e00700 1 -- 172.21.15.119:0/3335629622 shutdown_connections 2024-05-27T09:26:22.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.727+0000 7f6eb3e00700 1 -- 172.21.15.119:0/3335629622 wait complete. 2024-05-27T09:26:22.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.728+0000 7f6eb3e00700 1 Processor -- start 2024-05-27T09:26:22.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.729+0000 7f6eb3e00700 1 -- start start 2024-05-27T09:26:22.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.729+0000 7f6eb3e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eac0fe410 0x7f6eac195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:22.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.729+0000 7f6eb3e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6eac1013f0 0x7f6eac195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:22.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.729+0000 7f6eb3e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac1964b0 con 0x7f6eac1013f0 2024-05-27T09:26:22.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.729+0000 7f6eb3e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac1965f0 con 0x7f6eac0fe410 2024-05-27T09:26:22.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.730+0000 7f6eb1400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eac0fe410 0x7f6eac195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:22.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.730+0000 7f6eb0a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6eac1013f0 0x7f6eac195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:22.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.730+0000 7f6eb1400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eac0fe410 0x7f6eac195950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.119:53178/0 (socket says 172.21.15.119:53178) 2024-05-27T09:26:22.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.730+0000 7f6eb1400700 1 -- 172.21.15.119:0/450465446 learned_addr learned my addr 172.21.15.119:0/450465446 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:26:22.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.730+0000 7f6eb0a00700 1 -- 172.21.15.119:0/450465446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eac0fe410 msgr2=0x7f6eac195950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:22.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.730+0000 7f6eb0a00700 1 --2- 172.21.15.119:0/450465446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eac0fe410 0x7f6eac195950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:22.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.730+0000 7f6eb0a00700 1 -- 172.21.15.119:0/450465446 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e980095f0 con 0x7f6eac1013f0 2024-05-27T09:26:22.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.730+0000 7f6eb0a00700 1 --2- 172.21.15.119:0/450465446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6eac1013f0 0x7f6eac195e90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009910 tx=0x7f6e9800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:22.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.731+0000 7f6eaa000700 1 -- 172.21.15.119:0/450465446 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e9801d070 con 0x7f6eac1013f0 2024-05-27T09:26:22.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.731+0000 7f6eaa000700 1 -- 172.21.15.119:0/450465446 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6e98005dc0 con 0x7f6eac1013f0 2024-05-27T09:26:22.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.731+0000 7f6eb3e00700 1 -- 172.21.15.119:0/450465446 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eac19b040 con 0x7f6eac1013f0 2024-05-27T09:26:22.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.731+0000 7f6eaa000700 1 -- 172.21.15.119:0/450465446 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e98017400 con 0x7f6eac1013f0 2024-05-27T09:26:22.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.731+0000 7f6eb3e00700 1 -- 172.21.15.119:0/450465446 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eac19b530 con 0x7f6eac1013f0 2024-05-27T09:26:22.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.732+0000 7f6eb3e00700 1 -- 172.21.15.119:0/450465446 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eac108c90 con 0x7f6eac1013f0 2024-05-27T09:26:22.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.733+0000 7f6eaa000700 1 -- 172.21.15.119:0/450465446 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f6e98003810 con 0x7f6eac1013f0 2024-05-27T09:26:22.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.734+0000 7f6eaa000700 1 --2- 172.21.15.119:0/450465446 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f6e9406c680 0x7f6e9406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:22.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.734+0000 7f6eaa000700 1 -- 172.21.15.119:0/450465446 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f6e9808cc30 con 0x7f6eac1013f0 2024-05-27T09:26:22.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.734+0000 7f6eb1400700 1 --2- 172.21.15.119:0/450465446 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f6e9406c680 0x7f6e9406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:22.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.735+0000 7f6eb1400700 1 --2- 172.21.15.119:0/450465446 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f6e9406c680 0x7f6e9406eb30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0005950 tx=0x7f6ea0009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:22.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:22.739+0000 7f6eaa000700 1 -- 172.21.15.119:0/450465446 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6e9805ad20 con 0x7f6eac1013f0 2024-05-27T09:26:23.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:23.010+0000 7f6eb3e00700 1 -- 172.21.15.119:0/450465446 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6eac19bd50 con 0x7f6eac1013f0 2024-05-27T09:26:23.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:23.011+0000 7f6eaa000700 1 -- 172.21.15.119:0/450465446 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4760 (secure 0 0 0) 0x7f6e9805a8b0 con 0x7f6eac1013f0 2024-05-27T09:26:23.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:23.014+0000 7f6eb3e00700 1 -- 172.21.15.119:0/450465446 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f6e9406c680 msgr2=0x7f6e9406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:23.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:23.014+0000 7f6eb3e00700 1 --2- 172.21.15.119:0/450465446 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f6e9406c680 0x7f6e9406eb30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0005950 tx=0x7f6ea0009260 comp rx=0 tx=0).stop 2024-05-27T09:26:23.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:23.014+0000 7f6eb3e00700 1 -- 172.21.15.119:0/450465446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6eac1013f0 msgr2=0x7f6eac195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:23.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:23.014+0000 7f6eb3e00700 1 --2- 172.21.15.119:0/450465446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6eac1013f0 0x7f6eac195e90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009910 tx=0x7f6e9800f690 comp rx=0 tx=0).stop 2024-05-27T09:26:23.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:23.014+0000 7f6eb3e00700 1 -- 172.21.15.119:0/450465446 shutdown_connections 2024-05-27T09:26:23.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:23.014+0000 7f6eb3e00700 1 --2- 172.21.15.119:0/450465446 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f6e9406c680 0x7f6e9406eb30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:23.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:23.014+0000 7f6eb3e00700 1 --2- 172.21.15.119:0/450465446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eac0fe410 0x7f6eac195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:23.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:23.014+0000 7f6eb3e00700 1 --2- 172.21.15.119:0/450465446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6eac1013f0 0x7f6eac195e90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:23.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:23.014+0000 7f6eb3e00700 1 -- 172.21.15.119:0/450465446 >> 172.21.15.119:0/450465446 conn(0x7f6eac0f9d80 msgr2=0x7f6eac104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:23.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:23.014+0000 7f6eb3e00700 1 -- 172.21.15.119:0/450465446 shutdown_connections 2024-05-27T09:26:23.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:23.014+0000 7f6eb3e00700 1 -- 172.21.15.119:0/450465446 wait complete. 2024-05-27T09:26:23.016 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 18 2024-05-27T09:26:23.033 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:26:23.503 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:23 smithi138 ceph-mon[34885]: pgmap v237: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.8 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-27T09:26:23.503 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:23 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/450465446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T09:26:23.503 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:23 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:23.503 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:23 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:23.619 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:23 smithi119 ceph-mon[29082]: pgmap v237: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.8 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-27T09:26:23.619 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:23 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/450465446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T09:26:23.619 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:23 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:23.619 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:23 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:24.539 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-05-27T09:26:25.524 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:26:25.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:25 smithi138 ceph-mon[34885]: pgmap v238: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-27T09:26:25.675 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:25 smithi119 ceph-mon[29082]: pgmap v238: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-27T09:26:27.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:27 smithi138 ceph-mon[34885]: pgmap v239: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-05-27T09:26:27.472 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:27 smithi119 ceph-mon[29082]: pgmap v239: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-05-27T09:26:28.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.767+0000 7f7d5da00700 1 -- 172.21.15.119:0/2732659125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d58100150 msgr2=0x7f7d58100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:28.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.767+0000 7f7d5da00700 1 --2- 172.21.15.119:0/2732659125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d58100150 0x7f7d58100560 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009910 tx=0x7f7d48009c20 comp rx=0 tx=0).stop 2024-05-27T09:26:28.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.767+0000 7f7d5da00700 1 -- 172.21.15.119:0/2732659125 shutdown_connections 2024-05-27T09:26:28.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.767+0000 7f7d5da00700 1 --2- 172.21.15.119:0/2732659125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d58101350 0x7f7d581017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:28.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.767+0000 7f7d5da00700 1 --2- 172.21.15.119:0/2732659125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d58100150 0x7f7d58100560 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:28.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.767+0000 7f7d5da00700 1 -- 172.21.15.119:0/2732659125 >> 172.21.15.119:0/2732659125 conn(0x7f7d580fb6e0 msgr2=0x7f7d580fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:28.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.767+0000 7f7d5da00700 1 -- 172.21.15.119:0/2732659125 shutdown_connections 2024-05-27T09:26:28.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.767+0000 7f7d5da00700 1 -- 172.21.15.119:0/2732659125 wait complete. 2024-05-27T09:26:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.768+0000 7f7d5da00700 1 Processor -- start 2024-05-27T09:26:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.769+0000 7f7d5da00700 1 -- start start 2024-05-27T09:26:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.769+0000 7f7d5da00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d58100150 0x7f7d58195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.769+0000 7f7d5da00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d58101350 0x7f7d58195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.769+0000 7f7d5da00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d58196590 con 0x7f7d58101350 2024-05-27T09:26:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.769+0000 7f7d5da00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d581966d0 con 0x7f7d58100150 2024-05-27T09:26:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.769+0000 7f7d56000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d58101350 0x7f7d58195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.769+0000 7f7d56000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d58101350 0x7f7d58195f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54900/0 (socket says 172.21.15.119:54900) 2024-05-27T09:26:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.769+0000 7f7d56000700 1 -- 172.21.15.119:0/2707424435 learned_addr learned my addr 172.21.15.119:0/2707424435 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:26:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.770+0000 7f7d56000700 1 -- 172.21.15.119:0/2707424435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d58100150 msgr2=0x7f7d58195a30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T09:26:28.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.770+0000 7f7d56000700 1 --2- 172.21.15.119:0/2707424435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d58100150 0x7f7d58195a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:28.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.770+0000 7f7d56000700 1 -- 172.21.15.119:0/2707424435 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d480095f0 con 0x7f7d58101350 2024-05-27T09:26:28.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.770+0000 7f7d56000700 1 --2- 172.21.15.119:0/2707424435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d58101350 0x7f7d58195f70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400b4d0 tx=0x7f7d4400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:28.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.770+0000 7f7d4f400700 1 -- 172.21.15.119:0/2707424435 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d44011800 con 0x7f7d58101350 2024-05-27T09:26:28.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.771+0000 7f7d5da00700 1 -- 172.21.15.119:0/2707424435 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5819b180 con 0x7f7d58101350 2024-05-27T09:26:28.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.771+0000 7f7d4f400700 1 -- 172.21.15.119:0/2707424435 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7d44011e10 con 0x7f7d58101350 2024-05-27T09:26:28.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.771+0000 7f7d4f400700 1 -- 172.21.15.119:0/2707424435 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d4400f2e0 con 0x7f7d58101350 2024-05-27T09:26:28.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.771+0000 7f7d5da00700 1 -- 172.21.15.119:0/2707424435 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d5819b750 con 0x7f7d58101350 2024-05-27T09:26:28.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.772+0000 7f7d5da00700 1 -- 172.21.15.119:0/2707424435 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d58108d40 con 0x7f7d58101350 2024-05-27T09:26:28.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.773+0000 7f7d4f400700 1 -- 172.21.15.119:0/2707424435 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f7d44011960 con 0x7f7d58101350 2024-05-27T09:26:28.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.774+0000 7f7d4f400700 1 --2- 172.21.15.119:0/2707424435 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f7d3c06c680 0x7f7d3c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:28.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.775+0000 7f7d4f400700 1 -- 172.21.15.119:0/2707424435 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7d4408b530 con 0x7f7d58101350 2024-05-27T09:26:28.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.775+0000 7f7d56a00700 1 --2- 172.21.15.119:0/2707424435 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f7d3c06c680 0x7f7d3c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:28.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.776+0000 7f7d56a00700 1 --2- 172.21.15.119:0/2707424435 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f7d3c06c680 0x7f7d3c06eb30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009fd0 tx=0x7f7d480058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:28.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:28.781+0000 7f7d4f400700 1 -- 172.21.15.119:0/2707424435 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7d44059640 con 0x7f7d58101350 2024-05-27T09:26:29.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:29.096+0000 7f7d5da00700 1 -- 172.21.15.119:0/2707424435 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f7d58108f00 con 0x7f7d58101350 2024-05-27T09:26:29.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:29.097+0000 7f7d4f400700 1 -- 172.21.15.119:0/2707424435 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v18) v1 ==== 78+0+83 (secure 0 0 0) 0x7f7d440591d0 con 0x7f7d58101350 2024-05-27T09:26:29.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:29.099+0000 7f7d5da00700 1 -- 172.21.15.119:0/2707424435 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f7d3c06c680 msgr2=0x7f7d3c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:29.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:29.100+0000 7f7d5da00700 1 --2- 172.21.15.119:0/2707424435 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f7d3c06c680 0x7f7d3c06eb30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009fd0 tx=0x7f7d480058e0 comp rx=0 tx=0).stop 2024-05-27T09:26:29.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:29.100+0000 7f7d5da00700 1 -- 172.21.15.119:0/2707424435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d58101350 msgr2=0x7f7d58195f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:29.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:29.100+0000 7f7d5da00700 1 --2- 172.21.15.119:0/2707424435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d58101350 0x7f7d58195f70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400b4d0 tx=0x7f7d4400b890 comp rx=0 tx=0).stop 2024-05-27T09:26:29.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:29.100+0000 7f7d5da00700 1 -- 172.21.15.119:0/2707424435 shutdown_connections 2024-05-27T09:26:29.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:29.100+0000 7f7d5da00700 1 --2- 172.21.15.119:0/2707424435 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f7d3c06c680 0x7f7d3c06eb30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:29.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:29.100+0000 7f7d5da00700 1 --2- 172.21.15.119:0/2707424435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d58100150 0x7f7d58195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:29.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:29.100+0000 7f7d5da00700 1 --2- 172.21.15.119:0/2707424435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d58101350 0x7f7d58195f70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:29.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:29.100+0000 7f7d5da00700 1 -- 172.21.15.119:0/2707424435 >> 172.21.15.119:0/2707424435 conn(0x7f7d580fb6e0 msgr2=0x7f7d58104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:29.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:29.100+0000 7f7d5da00700 1 -- 172.21.15.119:0/2707424435 shutdown_connections 2024-05-27T09:26:29.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:29.100+0000 7f7d5da00700 1 -- 172.21.15.119:0/2707424435 wait complete. 2024-05-27T09:26:29.115 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:26:29.348 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:29 smithi119 ceph-mon[29082]: pgmap v240: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 0 B/s wr, 0 op/s 2024-05-27T09:26:29.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:29 smithi138 ceph-mon[34885]: pgmap v240: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 0 B/s wr, 0 op/s 2024-05-27T09:26:29.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:29 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/2707424435' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-05-27T09:26:29.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:29 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/2707424435' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-05-27T09:26:29.905 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-05-27T09:26:29.918 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T09:26:30.279 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:26:31.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:31 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:31.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:31 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:31.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:31 smithi119 ceph-mon[29082]: pgmap v241: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 0 B/s wr, 0 op/s 2024-05-27T09:26:31.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:31 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:31.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:31 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:31.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:31 smithi138 ceph-mon[34885]: pgmap v241: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 0 B/s wr, 0 op/s 2024-05-27T09:26:32.292 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:32 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:26:32.292 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:32 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:26:32.292 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:32 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:32.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:32 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:26:32.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:32 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:26:32.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:32 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:26:32.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.593+0000 7fb22ea00700 1 -- 172.21.15.119:0/1480669832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb228101c30 msgr2=0x7fb22810a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:32.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.593+0000 7fb22ea00700 1 --2- 172.21.15.119:0/1480669832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb228101c30 0x7fb22810a130 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb214009910 tx=0x7fb214009c20 comp rx=0 tx=0).stop 2024-05-27T09:26:32.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.593+0000 7fb22ea00700 1 -- 172.21.15.119:0/1480669832 shutdown_connections 2024-05-27T09:26:32.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.593+0000 7fb22ea00700 1 --2- 172.21.15.119:0/1480669832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb228101c30 0x7fb22810a130 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:32.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.593+0000 7fb22ea00700 1 --2- 172.21.15.119:0/1480669832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb228101320 0x7fb2281016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:32.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.593+0000 7fb22ea00700 1 -- 172.21.15.119:0/1480669832 >> 172.21.15.119:0/1480669832 conn(0x7fb2280fab80 msgr2=0x7fb2280fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:32.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.594+0000 7fb22ea00700 1 -- 172.21.15.119:0/1480669832 shutdown_connections 2024-05-27T09:26:32.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.594+0000 7fb22ea00700 1 -- 172.21.15.119:0/1480669832 wait complete. 2024-05-27T09:26:32.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.595+0000 7fb22ea00700 1 Processor -- start 2024-05-27T09:26:32.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.595+0000 7fb22ea00700 1 -- start start 2024-05-27T09:26:32.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.595+0000 7fb22ea00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb228101320 0x7fb2280fef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:32.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.595+0000 7fb22ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb228101c30 0x7fb2280ff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:32.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.595+0000 7fb22ea00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2280ffa00 con 0x7fb228101c30 2024-05-27T09:26:32.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.595+0000 7fb22ea00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2280ffb40 con 0x7fb228101320 2024-05-27T09:26:32.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.596+0000 7fb227400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb228101c30 0x7fb2280ff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:32.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.596+0000 7fb227400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb228101c30 0x7fb2280ff4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37752/0 (socket says 172.21.15.119:37752) 2024-05-27T09:26:32.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.596+0000 7fb227400700 1 -- 172.21.15.119:0/4116237610 learned_addr learned my addr 172.21.15.119:0/4116237610 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:26:32.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.596+0000 7fb227e00700 1 --2- 172.21.15.119:0/4116237610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb228101320 0x7fb2280fef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:32.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.596+0000 7fb227400700 1 -- 172.21.15.119:0/4116237610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb228101320 msgr2=0x7fb2280fef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:32.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.596+0000 7fb227400700 1 --2- 172.21.15.119:0/4116237610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb228101320 0x7fb2280fef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:32.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.596+0000 7fb227400700 1 -- 172.21.15.119:0/4116237610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2140095f0 con 0x7fb228101c30 2024-05-27T09:26:32.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.596+0000 7fb227e00700 1 --2- 172.21.15.119:0/4116237610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb228101320 0x7fb2280fef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:26:32.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.596+0000 7fb227400700 1 --2- 172.21.15.119:0/4116237610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb228101c30 0x7fb2280ff4c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb214006010 tx=0x7fb21400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:32.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.597+0000 7fb224c00700 1 -- 172.21.15.119:0/4116237610 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb21401d070 con 0x7fb228101c30 2024-05-27T09:26:32.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.597+0000 7fb224c00700 1 -- 172.21.15.119:0/4116237610 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb21400f460 con 0x7fb228101c30 2024-05-27T09:26:32.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.597+0000 7fb22ea00700 1 -- 172.21.15.119:0/4116237610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2280ffdc0 con 0x7fb228101c30 2024-05-27T09:26:32.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.597+0000 7fb224c00700 1 -- 172.21.15.119:0/4116237610 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb214021640 con 0x7fb228101c30 2024-05-27T09:26:32.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.597+0000 7fb22ea00700 1 -- 172.21.15.119:0/4116237610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2281a22b0 con 0x7fb228101c30 2024-05-27T09:26:32.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.598+0000 7fb22ea00700 1 -- 172.21.15.119:0/4116237610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2281077a0 con 0x7fb228101c30 2024-05-27T09:26:32.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.599+0000 7fb224c00700 1 -- 172.21.15.119:0/4116237610 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fb21400faf0 con 0x7fb228101c30 2024-05-27T09:26:32.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.600+0000 7fb224c00700 1 --2- 172.21.15.119:0/4116237610 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fb20c06c680 0x7fb20c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:32.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.600+0000 7fb224c00700 1 -- 172.21.15.119:0/4116237610 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb21408cb60 con 0x7fb228101c30 2024-05-27T09:26:32.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.600+0000 7fb227e00700 1 --2- 172.21.15.119:0/4116237610 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fb20c06c680 0x7fb20c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:32.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.601+0000 7fb227e00700 1 --2- 172.21.15.119:0/4116237610 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fb20c06c680 0x7fb20c06eb30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb218007900 tx=0x7fb218008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:32.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.605+0000 7fb224c00700 1 -- 172.21.15.119:0/4116237610 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb214057260 con 0x7fb228101c30 2024-05-27T09:26:32.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.866+0000 7fb22ea00700 1 -- 172.21.15.119:0/4116237610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb22804ea50 con 0x7fb228101c30 2024-05-27T09:26:32.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.867+0000 7fb224c00700 1 -- 172.21.15.119:0/4116237610 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4760 (secure 0 0 0) 0x7fb214026090 con 0x7fb228101c30 2024-05-27T09:26:32.867 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:26:32.868 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14482,"name":"cephfs.smithi119.nghcey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6827/413736191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":413736191},{"type":"v1","addr":"172.21.15.119:6827","nonce":413736191}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24253,"name":"cephfs.smithi138.alszpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.138:6827/2345607053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2345607053},{"type":"v1","addr":"172.21.15.138:6827","nonce":2345607053}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:25:53.387388+0000","modified":"2024-05-27T09:26:13.642209+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14472,"mds_1":24241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi119.ppmzzk","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6829/2100782307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2100782307},{"type":"v1","addr":"172.21.15.119:6829","nonce":2100782307}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24241":{"gid":24241,"name":"cephfs.smithi138.ndfkhd","rank":1,"incarnation":9,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6825/897058594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":897058594},{"type":"v1","addr":"172.21.15.138:6825","nonce":897058594}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-27T09:26:32.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.870+0000 7fb22ea00700 1 -- 172.21.15.119:0/4116237610 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fb20c06c680 msgr2=0x7fb20c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:32.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.870+0000 7fb22ea00700 1 --2- 172.21.15.119:0/4116237610 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fb20c06c680 0x7fb20c06eb30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb218007900 tx=0x7fb218008040 comp rx=0 tx=0).stop 2024-05-27T09:26:32.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.870+0000 7fb22ea00700 1 -- 172.21.15.119:0/4116237610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb228101c30 msgr2=0x7fb2280ff4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:32.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.870+0000 7fb22ea00700 1 --2- 172.21.15.119:0/4116237610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb228101c30 0x7fb2280ff4c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb214006010 tx=0x7fb21400be20 comp rx=0 tx=0).stop 2024-05-27T09:26:32.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.870+0000 7fb22ea00700 1 -- 172.21.15.119:0/4116237610 shutdown_connections 2024-05-27T09:26:32.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.870+0000 7fb22ea00700 1 --2- 172.21.15.119:0/4116237610 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fb20c06c680 0x7fb20c06eb30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:32.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.870+0000 7fb22ea00700 1 --2- 172.21.15.119:0/4116237610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb228101320 0x7fb2280fef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:32.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.870+0000 7fb22ea00700 1 --2- 172.21.15.119:0/4116237610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb228101c30 0x7fb2280ff4c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:32.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.870+0000 7fb22ea00700 1 -- 172.21.15.119:0/4116237610 >> 172.21.15.119:0/4116237610 conn(0x7fb2280fab80 msgr2=0x7fb2281048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:32.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.870+0000 7fb22ea00700 1 -- 172.21.15.119:0/4116237610 shutdown_connections 2024-05-27T09:26:32.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:32.870+0000 7fb22ea00700 1 -- 172.21.15.119:0/4116237610 wait complete. 2024-05-27T09:26:32.872 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 18 2024-05-27T09:26:33.206 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:33 smithi119 ceph-mon[29082]: pgmap v242: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:26:33.206 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:33 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/4116237610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T09:26:33.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:33 smithi138 ceph-mon[34885]: pgmap v242: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:26:33.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:33 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/4116237610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T09:26:33.585 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 18, 'max_mds': 2, 'flags': 18} 2024-05-27T09:26:33.585 INFO:teuthology.run_tasks:Running task kclient... 2024-05-27T09:26:33.613 INFO:tasks.kclient:Mounting kernel clients... 2024-05-27T09:26:33.613 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-05-27T09:26:33.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:26:33.614 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-05-27T09:26:33.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:26:33.672 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2024-05-27T09:26:33.758 INFO:teuthology.orchestra.run.smithi119.stderr:Cannot find device "ceph-brx" 2024-05-27T09:26:33.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:26:33.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:26:33.761 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-05-27T09:26:33.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:26:33.821 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-05-27T09:26:33.912 INFO:teuthology.orchestra.run.smithi138.stderr:Cannot find device "ceph-brx" 2024-05-27T09:26:33.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:26:33.915 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-05-27T09:26:33.916 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-27T09:26:33.916 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-27T09:26:33.916 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-05-27T09:26:34.316 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:26:35.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:35 smithi138 ceph-mon[34885]: pgmap v243: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:26:35.638 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:35 smithi119 ceph-mon[29082]: pgmap v243: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:26:36.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.498+0000 7ff993e00700 1 -- 172.21.15.119:0/855954506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff994073180 msgr2=0x7ff994073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:36.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.498+0000 7ff993e00700 1 --2- 172.21.15.119:0/855954506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff994073180 0x7ff994073550 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff984009910 tx=0x7ff984009c20 comp rx=0 tx=0).stop 2024-05-27T09:26:36.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.498+0000 7ff993e00700 1 -- 172.21.15.119:0/855954506 shutdown_connections 2024-05-27T09:26:36.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.498+0000 7ff993e00700 1 --2- 172.21.15.119:0/855954506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff994073b20 0x7ff99410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:36.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.498+0000 7ff993e00700 1 --2- 172.21.15.119:0/855954506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff994073180 0x7ff994073550 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:36.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.498+0000 7ff993e00700 1 -- 172.21.15.119:0/855954506 >> 172.21.15.119:0/855954506 conn(0x7ff9940fbb90 msgr2=0x7ff9940fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:36.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.498+0000 7ff993e00700 1 -- 172.21.15.119:0/855954506 shutdown_connections 2024-05-27T09:26:36.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.498+0000 7ff993e00700 1 -- 172.21.15.119:0/855954506 wait complete. 2024-05-27T09:26:36.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.499+0000 7ff993e00700 1 Processor -- start 2024-05-27T09:26:36.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.500+0000 7ff993e00700 1 -- start start 2024-05-27T09:26:36.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.500+0000 7ff993e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff994073180 0x7ff994197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:36.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.500+0000 7ff993e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff994073b20 0x7ff9941982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:36.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.500+0000 7ff993e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9941989b0 con 0x7ff994073b20 2024-05-27T09:26:36.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.500+0000 7ff993e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99419c6f0 con 0x7ff994073180 2024-05-27T09:26:36.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.500+0000 7ff992000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff994073b20 0x7ff9941982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:36.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.500+0000 7ff992000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff994073b20 0x7ff9941982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37768/0 (socket says 172.21.15.119:37768) 2024-05-27T09:26:36.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.500+0000 7ff992000700 1 -- 172.21.15.119:0/3151542458 learned_addr learned my addr 172.21.15.119:0/3151542458 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:26:36.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.501+0000 7ff992a00700 1 --2- 172.21.15.119:0/3151542458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff994073180 0x7ff994197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:36.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.501+0000 7ff992000700 1 -- 172.21.15.119:0/3151542458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff994073180 msgr2=0x7ff994197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:36.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.501+0000 7ff992000700 1 --2- 172.21.15.119:0/3151542458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff994073180 0x7ff994197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:36.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.501+0000 7ff992000700 1 -- 172.21.15.119:0/3151542458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9840095f0 con 0x7ff994073b20 2024-05-27T09:26:36.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.501+0000 7ff992000700 1 --2- 172.21.15.119:0/3151542458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff994073b20 0x7ff9941982d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff98000c8a0 tx=0x7ff98000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:36.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.501+0000 7ff98b400700 1 -- 172.21.15.119:0/3151542458 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff98000cea0 con 0x7ff994073b20 2024-05-27T09:26:36.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.501+0000 7ff993e00700 1 -- 172.21.15.119:0/3151542458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99419c9d0 con 0x7ff994073b20 2024-05-27T09:26:36.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.501+0000 7ff98b400700 1 -- 172.21.15.119:0/3151542458 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff980004830 con 0x7ff994073b20 2024-05-27T09:26:36.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.501+0000 7ff98b400700 1 -- 172.21.15.119:0/3151542458 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9800055b0 con 0x7ff994073b20 2024-05-27T09:26:36.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.502+0000 7ff993e00700 1 -- 172.21.15.119:0/3151542458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99419cf20 con 0x7ff994073b20 2024-05-27T09:26:36.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.503+0000 7ff993e00700 1 -- 172.21.15.119:0/3151542458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff994109a30 con 0x7ff994073b20 2024-05-27T09:26:36.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.504+0000 7ff98b400700 1 -- 172.21.15.119:0/3151542458 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7ff980007ad0 con 0x7ff994073b20 2024-05-27T09:26:36.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.504+0000 7ff98b400700 1 --2- 172.21.15.119:0/3151542458 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff97806c680 0x7ff97806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:36.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.505+0000 7ff98b400700 1 -- 172.21.15.119:0/3151542458 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff98008b590 con 0x7ff994073b20 2024-05-27T09:26:36.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.505+0000 7ff992a00700 1 --2- 172.21.15.119:0/3151542458 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff97806c680 0x7ff97806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:36.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.505+0000 7ff992a00700 1 --2- 172.21.15.119:0/3151542458 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff97806c680 0x7ff97806eb30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff984006010 tx=0x7ff98401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:36.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.509+0000 7ff98b400700 1 -- 172.21.15.119:0/3151542458 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff980055c90 con 0x7ff994073b20 2024-05-27T09:26:36.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.770+0000 7ff993e00700 1 -- 172.21.15.119:0/3151542458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff99419d1f0 con 0x7ff994073b20 2024-05-27T09:26:36.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.770+0000 7ff98b400700 1 -- 172.21.15.119:0/3151542458 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+83 (secure 0 0 0) 0x7ff9800592b0 con 0x7ff994073b20 2024-05-27T09:26:36.772 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-05-27T09:26:36.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.774+0000 7ff993e00700 1 -- 172.21.15.119:0/3151542458 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff97806c680 msgr2=0x7ff97806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:36.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.774+0000 7ff993e00700 1 --2- 172.21.15.119:0/3151542458 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff97806c680 0x7ff97806eb30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff984006010 tx=0x7ff98401a040 comp rx=0 tx=0).stop 2024-05-27T09:26:36.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.774+0000 7ff993e00700 1 -- 172.21.15.119:0/3151542458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff994073b20 msgr2=0x7ff9941982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:36.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.774+0000 7ff993e00700 1 --2- 172.21.15.119:0/3151542458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff994073b20 0x7ff9941982d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff98000c8a0 tx=0x7ff98000cc60 comp rx=0 tx=0).stop 2024-05-27T09:26:36.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.775+0000 7ff993e00700 1 -- 172.21.15.119:0/3151542458 shutdown_connections 2024-05-27T09:26:36.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.775+0000 7ff993e00700 1 --2- 172.21.15.119:0/3151542458 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff97806c680 0x7ff97806eb30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:36.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.775+0000 7ff993e00700 1 --2- 172.21.15.119:0/3151542458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff994073180 0x7ff994197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:36.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.775+0000 7ff993e00700 1 --2- 172.21.15.119:0/3151542458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff994073b20 0x7ff9941982d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:36.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.775+0000 7ff993e00700 1 -- 172.21.15.119:0/3151542458 >> 172.21.15.119:0/3151542458 conn(0x7ff9940fbb90 msgr2=0x7ff994106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:36.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.775+0000 7ff993e00700 1 -- 172.21.15.119:0/3151542458 shutdown_connections 2024-05-27T09:26:36.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:36.775+0000 7ff993e00700 1 -- 172.21.15.119:0/3151542458 wait complete. 2024-05-27T09:26:37.174 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:37 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:26:37.174 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:37 smithi119 ceph-mon[29082]: pgmap v244: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:26:37.174 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:37 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/3151542458' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T09:26:37.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:37 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:26:37.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:37 smithi138 ceph-mon[34885]: pgmap v244: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:26:37.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:37 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/3151542458' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T09:26:37.624 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-27T09:26:37.624 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-27T09:26:37.624 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2024-05-27T09:26:37.624 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-27T09:26:37.625 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-27T09:26:37.625 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-27T09:26:37.625 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-27T09:26:37.625 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-27T09:26:37.625 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-27T09:26:37.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:26:37.625 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2024-05-27T09:26:37.679 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T09:26:37.679 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T09:26:37.679 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T09:26:37.680 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-05-27T09:26:37.680 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2024-05-27T09:26:37.680 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-05-27T09:26:37.680 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T09:26:37.680 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2024-05-27T09:26:37.680 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2024-05-27T09:26:37.680 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T09:26:37.680 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2024-05-27T09:26:37.680 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2024-05-27T09:26:37.680 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T09:26:37.680 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2024-05-27T09:26:37.680 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-27T09:26:37.681 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2024-05-27T09:26:37.681 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-27T09:26:37.681 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 41999sec preferred_lft 41999sec 2024-05-27T09:26:37.681 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2024-05-27T09:26:37.681 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2024-05-27T09:26:37.681 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-27T09:26:37.682 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T09:26:37.682 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-05-27T09:26:37.682 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add name ceph-brx type bridge 2024-05-27T09:26:37.682 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip addr flush dev ceph-brx 2024-05-27T09:26:37.682 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set ceph-brx up 2024-05-27T09:26:37.682 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-27T09:26:37.682 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-05-27T09:26:37.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T09:26:38.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:26:38.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:26:38.188 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-27T09:26:38.261 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-05-27T09:26:38.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:26:38.264 DEBUG:teuthology.orchestra.run.smithi119:> route 2024-05-27T09:26:38.318 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2024-05-27T09:26:38.318 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-27T09:26:38.318 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-27T09:26:38.318 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-27T09:26:38.318 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-27T09:26:38.320 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T09:26:38.320 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-05-27T09:26:38.320 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-27T09:26:38.320 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-27T09:26:38.320 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-27T09:26:38.320 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-05-27T09:26:38.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T09:26:39.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:26:39.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:26:39.095 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-05-27T09:26:39.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:26:39.152 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list-id 2024-05-27T09:26:39.204 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T09:26:39.205 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-05-27T09:26:39.205 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T09:26:39.205 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-27T09:26:39.205 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-05-27T09:26:39.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T09:26:39.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:26:39.395 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-27T09:26:39.396 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T09:26:39.396 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-05-27T09:26:39.396 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-27T09:26:39.396 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-27T09:26:39.396 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-27T09:26:39.396 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-27T09:26:39.396 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-27T09:26:39.396 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-05-27T09:26:39.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T09:26:39.589 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:39 smithi119 ceph-mon[29082]: pgmap v245: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:26:39.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:39 smithi138 ceph-mon[34885]: pgmap v245: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:26:39.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:26:39.687 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T09:26:39.687 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2024-05-27T09:26:39.687 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2024-05-27T09:26:39.687 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set dev brx.0 master ceph-brx 2024-05-27T09:26:39.687 DEBUG:teuthology.orchestra.run.smithi119:> ') 2024-05-27T09:26:39.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T09:26:39.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:26:39.889 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T09:26:39.889 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-27T09:26:39.938 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-27T09:26:39.939 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T09:26:39.939 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-27T09:26:39.988 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-27T09:26:39.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:26:39.988 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-27T09:26:40.126 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-27T09:26:40.126 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-27T09:26:40.126 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-05-27T09:26:40.864 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-27T09:26:40.864 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-27T09:26:40.864 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-05-27T09:26:40.864 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.138:3300" 2024-05-27T09:26:40.864 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-27T09:26:40.905 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-27T09:26:40.906 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-27T09:26:40.906 INFO:teuthology.orchestra.run.smithi119.stdout:invalid new device string format 2024-05-27T09:26:40.906 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.138:3300" 2024-05-27T09:26:40.906 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-27T09:26:40.906 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:6789,172.21.15.138:6789" 2024-05-27T09:26:40.906 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.119:6789,172.21.15.138:6789:/ 2024-05-27T09:26:40.906 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=b966b06e-1c09-11ef-bc9b-c7b262605968" will pass to kernel 2024-05-27T09:26:40.909 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-27T09:26:40.909 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2024-05-27T09:26:40.909 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-27T09:26:40.910 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2024-05-27T09:26:40.910 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-27T09:26:40.914 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-27T09:26:40.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:26:40.915 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-27T09:26:40.997 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T09:26:41.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T09:26:41.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:26:41.132 INFO:teuthology.orchestra.run.smithi119.stdout:client24273 2024-05-27T09:26:41.132 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-27T09:26:41.132 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T09:26:41.183 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T09:26:41.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T09:26:41.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:26:41.367 INFO:teuthology.orchestra.run.smithi119.stdout:client24273 2024-05-27T09:26:41.367 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T09:26:41.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T09:26:41.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:26:41.491 INFO:teuthology.orchestra.run.smithi119.stdout:client24273 2024-05-27T09:26:41.492 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968.client24273/status') 2024-05-27T09:26:41.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T09:26:41.629 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:41 smithi119 ceph-mon[29082]: pgmap v246: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:26:41.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:41 smithi138 ceph-mon[34885]: pgmap v246: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:26:41.637 INFO:teuthology.orchestra.run.smithi119.stdout:instance: client.24273 (3)172.21.15.119:0/3745732299 2024-05-27T09:26:41.637 INFO:teuthology.orchestra.run.smithi119.stdout:blocklisted: false 2024-05-27T09:26:41.637 INFO:teuthology.orchestra.run.smithi119.stderr:0+1 records in 2024-05-27T09:26:41.637 INFO:teuthology.orchestra.run.smithi119.stderr:0+1 records out 2024-05-27T09:26:41.637 INFO:teuthology.orchestra.run.smithi119.stderr:72 bytes copied, 8.0151e-05 s, 898 kB/s 2024-05-27T09:26:41.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:26:41.650 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-05-27T09:26:41.650 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-27T09:26:41.651 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-27T09:26:41.651 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-05-27T09:26:42.052 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:26:43.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:43 smithi138 ceph-mon[34885]: pgmap v247: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:26:43.709 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:43 smithi119 ceph-mon[29082]: pgmap v247: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:26:44.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.284+0000 7f4e35400700 1 -- 172.21.15.119:0/3153518602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e300ff390 msgr2=0x7f4e300ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:44.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.284+0000 7f4e35400700 1 --2- 172.21.15.119:0/3153518602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e300ff390 0x7f4e300ff760 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4e20009910 tx=0x7f4e20009c20 comp rx=0 tx=0).stop 2024-05-27T09:26:44.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.284+0000 7f4e35400700 1 -- 172.21.15.119:0/3153518602 shutdown_connections 2024-05-27T09:26:44.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.284+0000 7f4e35400700 1 --2- 172.21.15.119:0/3153518602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e300ffd30 0x7f4e30103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:44.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.284+0000 7f4e35400700 1 --2- 172.21.15.119:0/3153518602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e300ff390 0x7f4e300ff760 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:44.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.284+0000 7f4e35400700 1 -- 172.21.15.119:0/3153518602 >> 172.21.15.119:0/3153518602 conn(0x7f4e30074f50 msgr2=0x7f4e30075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:44.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.285+0000 7f4e35400700 1 -- 172.21.15.119:0/3153518602 shutdown_connections 2024-05-27T09:26:44.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.285+0000 7f4e35400700 1 -- 172.21.15.119:0/3153518602 wait complete. 2024-05-27T09:26:44.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.286+0000 7f4e35400700 1 Processor -- start 2024-05-27T09:26:44.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.286+0000 7f4e35400700 1 -- start start 2024-05-27T09:26:44.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.286+0000 7f4e35400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e300ff390 0x7f4e30197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:44.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.286+0000 7f4e35400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e300ffd30 0x7f4e30198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:44.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.286+0000 7f4e35400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e30198b50 con 0x7f4e300ff390 2024-05-27T09:26:44.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.286+0000 7f4e35400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3019c8e0 con 0x7f4e300ffd30 2024-05-27T09:26:44.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.287+0000 7f4e2e800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e300ff390 0x7f4e30197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:44.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.287+0000 7f4e2e800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e300ff390 0x7f4e30197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46618/0 (socket says 172.21.15.119:46618) 2024-05-27T09:26:44.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.287+0000 7f4e2e800700 1 -- 172.21.15.119:0/2245032637 learned_addr learned my addr 172.21.15.119:0/2245032637 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:26:44.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.287+0000 7f4e2de00700 1 --2- 172.21.15.119:0/2245032637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e300ffd30 0x7f4e30198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:44.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.287+0000 7f4e2e800700 1 -- 172.21.15.119:0/2245032637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e300ffd30 msgr2=0x7f4e30198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:44.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.287+0000 7f4e2e800700 1 --2- 172.21.15.119:0/2245032637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e300ffd30 0x7f4e30198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:44.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.287+0000 7f4e2e800700 1 -- 172.21.15.119:0/2245032637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e200095f0 con 0x7f4e300ff390 2024-05-27T09:26:44.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.287+0000 7f4e2de00700 1 --2- 172.21.15.119:0/2245032637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e300ffd30 0x7f4e30198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:26:44.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.287+0000 7f4e2e800700 1 --2- 172.21.15.119:0/2245032637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e300ff390 0x7f4e30197f30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000b5c0 tx=0x7f4e2000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:44.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.288+0000 7f4e27400700 1 -- 172.21.15.119:0/2245032637 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e2001d070 con 0x7f4e300ff390 2024-05-27T09:26:44.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.288+0000 7f4e27400700 1 -- 172.21.15.119:0/2245032637 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4e20022470 con 0x7f4e300ff390 2024-05-27T09:26:44.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.288+0000 7f4e35400700 1 -- 172.21.15.119:0/2245032637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e3019cb60 con 0x7f4e300ff390 2024-05-27T09:26:44.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.288+0000 7f4e27400700 1 -- 172.21.15.119:0/2245032637 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e2000f670 con 0x7f4e300ff390 2024-05-27T09:26:44.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.288+0000 7f4e35400700 1 -- 172.21.15.119:0/2245032637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e3019d050 con 0x7f4e300ff390 2024-05-27T09:26:44.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.289+0000 7f4e35400700 1 -- 172.21.15.119:0/2245032637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e3004ea50 con 0x7f4e300ff390 2024-05-27T09:26:44.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.290+0000 7f4e27400700 1 -- 172.21.15.119:0/2245032637 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f4e20004c30 con 0x7f4e300ff390 2024-05-27T09:26:44.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.291+0000 7f4e27400700 1 --2- 172.21.15.119:0/2245032637 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4e1406c680 0x7f4e1406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:44.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.291+0000 7f4e27400700 1 -- 172.21.15.119:0/2245032637 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4e2008ce10 con 0x7f4e300ff390 2024-05-27T09:26:44.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.291+0000 7f4e2de00700 1 --2- 172.21.15.119:0/2245032637 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4e1406c680 0x7f4e1406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:44.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.292+0000 7f4e2de00700 1 --2- 172.21.15.119:0/2245032637 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4e1406c680 0x7f4e1406eb30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c005fd0 tx=0x7f4e1c005dc0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:44.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.296+0000 7f4e27400700 1 -- 172.21.15.119:0/2245032637 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4e200575c0 con 0x7f4e300ff390 2024-05-27T09:26:44.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.573+0000 7f4e35400700 1 -- 172.21.15.119:0/2245032637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4e3019d560 con 0x7f4e300ff390 2024-05-27T09:26:44.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.574+0000 7f4e27400700 1 -- 172.21.15.119:0/2245032637 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+83 (secure 0 0 0) 0x7f4e20027070 con 0x7f4e300ff390 2024-05-27T09:26:44.575 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-05-27T09:26:44.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.577+0000 7f4e35400700 1 -- 172.21.15.119:0/2245032637 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4e1406c680 msgr2=0x7f4e1406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:44.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.577+0000 7f4e35400700 1 --2- 172.21.15.119:0/2245032637 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4e1406c680 0x7f4e1406eb30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c005fd0 tx=0x7f4e1c005dc0 comp rx=0 tx=0).stop 2024-05-27T09:26:44.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.577+0000 7f4e35400700 1 -- 172.21.15.119:0/2245032637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e300ff390 msgr2=0x7f4e30197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:44.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.577+0000 7f4e35400700 1 --2- 172.21.15.119:0/2245032637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e300ff390 0x7f4e30197f30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000b5c0 tx=0x7f4e2000be00 comp rx=0 tx=0).stop 2024-05-27T09:26:44.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.577+0000 7f4e35400700 1 -- 172.21.15.119:0/2245032637 shutdown_connections 2024-05-27T09:26:44.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.577+0000 7f4e35400700 1 --2- 172.21.15.119:0/2245032637 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4e1406c680 0x7f4e1406eb30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:44.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.577+0000 7f4e35400700 1 --2- 172.21.15.119:0/2245032637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e300ff390 0x7f4e30197f30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:44.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.577+0000 7f4e35400700 1 --2- 172.21.15.119:0/2245032637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e300ffd30 0x7f4e30198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:44.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.577+0000 7f4e35400700 1 -- 172.21.15.119:0/2245032637 >> 172.21.15.119:0/2245032637 conn(0x7f4e30074f50 msgr2=0x7f4e300fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:44.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.577+0000 7f4e35400700 1 -- 172.21.15.119:0/2245032637 shutdown_connections 2024-05-27T09:26:44.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:44.577+0000 7f4e35400700 1 -- 172.21.15.119:0/2245032637 wait complete. 2024-05-27T09:26:45.449 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-27T09:26:45.449 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-27T09:26:45.449 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-05-27T09:26:45.449 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-27T09:26:45.449 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-27T09:26:45.449 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-27T09:26:45.449 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-27T09:26:45.449 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-27T09:26:45.449 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-27T09:26:45.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:26:45.450 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-05-27T09:26:45.509 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T09:26:45.509 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T09:26:45.509 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T09:26:45.509 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-27T09:26:45.509 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-05-27T09:26:45.509 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-27T09:26:45.509 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T09:26:45.510 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-05-27T09:26:45.510 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-05-27T09:26:45.510 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T09:26:45.510 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-05-27T09:26:45.510 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-05-27T09:26:45.510 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T09:26:45.510 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-05-27T09:26:45.510 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-27T09:26:45.510 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-05-27T09:26:45.510 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-27T09:26:45.510 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42004sec preferred_lft 42004sec 2024-05-27T09:26:45.510 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2024-05-27T09:26:45.510 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-27T09:26:45.511 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-27T09:26:45.512 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T09:26:45.512 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-27T09:26:45.512 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add name ceph-brx type bridge 2024-05-27T09:26:45.512 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr flush dev ceph-brx 2024-05-27T09:26:45.512 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set ceph-brx up 2024-05-27T09:26:45.512 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-27T09:26:45.512 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-27T09:26:45.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:26:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T09:26:45.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:45 smithi138 ceph-mon[34885]: pgmap v248: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:26:45.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:45 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/2245032637' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T09:26:45.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:45 smithi119 ceph-mon[29082]: pgmap v248: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:26:45.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:45 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/2245032637' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T09:26:45.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:26:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:26:46.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:26:46.004 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-27T09:26:46.069 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-05-27T09:26:46.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:26:46.072 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-05-27T09:26:46.127 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-05-27T09:26:46.127 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-27T09:26:46.127 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-27T09:26:46.127 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-27T09:26:46.127 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-27T09:26:46.128 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T09:26:46.128 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-27T09:26:46.128 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-27T09:26:46.128 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-27T09:26:46.128 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-27T09:26:46.129 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-27T09:26:46.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:26:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T09:26:46.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:26:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:26:46.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:26:46.904 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-05-27T09:26:46.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:26:46.948 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list-id 2024-05-27T09:26:47.031 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T09:26:47.031 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-27T09:26:47.031 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-27T09:26:47.031 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-27T09:26:47.031 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-27T09:26:47.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:26:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T09:26:47.193 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:47 smithi138 ceph-mon[34885]: pgmap v249: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-05-27T09:26:47.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:26:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:26:47.247 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-27T09:26:47.247 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T09:26:47.247 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-27T09:26:47.247 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-27T09:26:47.247 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-27T09:26:47.247 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-27T09:26:47.248 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-27T09:26:47.248 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-27T09:26:47.248 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-27T09:26:47.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:26:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T09:26:47.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:47 smithi119 ceph-mon[29082]: pgmap v249: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-05-27T09:26:47.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:26:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:26:47.523 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T09:26:47.523 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-27T09:26:47.523 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2024-05-27T09:26:47.523 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set dev brx.0 master ceph-brx 2024-05-27T09:26:47.523 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-27T09:26:47.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:26:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T09:26:47.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:26:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:26:47.708 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T09:26:47.708 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-27T09:26:47.751 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-27T09:26:47.751 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T09:26:47.751 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-27T09:26:47.843 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-27T09:26:47.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:26:47.843 DEBUG:teuthology.orchestra.run.smithi138:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-27T09:26:47.980 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-27T09:26:47.980 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-27T09:26:47.980 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2024-05-27T09:26:48.725 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-27T09:26:48.725 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-27T09:26:48.725 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2024-05-27T09:26:48.725 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.138:3300" 2024-05-27T09:26:48.725 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-27T09:26:48.763 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-27T09:26:48.763 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-27T09:26:48.763 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2024-05-27T09:26:48.763 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.138:3300" 2024-05-27T09:26:48.763 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-27T09:26:48.763 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.119:6789,172.21.15.138:6789" 2024-05-27T09:26:48.763 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.119:6789,172.21.15.138:6789:/ 2024-05-27T09:26:48.763 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=b966b06e-1c09-11ef-bc9b-c7b262605968" will pass to kernel 2024-05-27T09:26:48.768 INFO:teuthology.orchestra.run.smithi138.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-27T09:26:48.769 INFO:teuthology.orchestra.run.smithi138.stdout: You just mounted a file system that supports labels which does not 2024-05-27T09:26:48.769 INFO:teuthology.orchestra.run.smithi138.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-27T09:26:48.769 INFO:teuthology.orchestra.run.smithi138.stdout: applications will generate AVC messages and not be allowed access to 2024-05-27T09:26:48.769 INFO:teuthology.orchestra.run.smithi138.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-27T09:26:48.773 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-27T09:26:48.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:26:48.773 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-27T09:26:48.854 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T09:26:48.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:26:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T09:26:48.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:26:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:26:48.974 INFO:teuthology.orchestra.run.smithi138.stdout:client24275 2024-05-27T09:26:48.974 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-27T09:26:48.975 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T09:26:49.024 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T09:26:49.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:26:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T09:26:49.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:26:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:26:49.189 INFO:teuthology.orchestra.run.smithi138.stdout:client24275 2024-05-27T09:26:49.189 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T09:26:49.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:26:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T09:26:49.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:26:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:26:49.310 INFO:teuthology.orchestra.run.smithi138.stdout:client24275 2024-05-27T09:26:49.311 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968.client24275/status') 2024-05-27T09:26:49.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:26:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T09:26:49.444 INFO:teuthology.orchestra.run.smithi138.stdout:instance: client.24275 (3)192.168.0.1:0/2208627419 2024-05-27T09:26:49.444 INFO:teuthology.orchestra.run.smithi138.stdout:blocklisted: false 2024-05-27T09:26:49.444 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records in 2024-05-27T09:26:49.445 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records out 2024-05-27T09:26:49.445 INFO:teuthology.orchestra.run.smithi138.stderr:70 bytes copied, 7.5038e-05 s, 933 kB/s 2024-05-27T09:26:49.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-27T09:26:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T09:26:49.457 INFO:teuthology.run_tasks:Running task print... 2024-05-27T09:26:49.468 INFO:teuthology.task.print:**** done client 2024-05-27T09:26:49.468 INFO:teuthology.run_tasks:Running task parallel... 2024-05-27T09:26:49.500 INFO:teuthology.task.parallel:starting parallel... 2024-05-27T09:26:49.500 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-05-27T09:26:49.501 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-27T09:26:49.501 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi119.front.sepia.ceph.com 2024-05-27T09:26:49.502 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-05-27T09:26:49.502 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-05-27T09:26:49.502 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-05-27T09:26:49.507 INFO:tasks.workunit:Pulling workunits from ref a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:26:49.507 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-27T09:26:49.507 INFO:tasks.workunit:timeout=3h 2024-05-27T09:26:49.507 INFO:tasks.workunit:cleanup=True 2024-05-27T09:26:49.508 DEBUG:teuthology.orchestra.run.smithi119:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-27T09:26:49.564 INFO:teuthology.orchestra.run.smithi119.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-27T09:26:49.564 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-27T09:26:49.564 INFO:teuthology.orchestra.run.smithi119.stdout:Device: a4h/164d Inode: 1 Links: 2 2024-05-27T09:26:49.564 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T09:26:49.564 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-27T09:26:49.564 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-05-27T09:26:49.564 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-05-27 09:25:58.430767379 +0000 2024-05-27T09:26:49.564 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-05-27 09:26:40.988886828 +0000 2024-05-27T09:26:49.564 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-05-27 09:25:58.430767379 +0000 2024-05-27T09:26:49.564 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-27T09:26:49.565 DEBUG:teuthology.orchestra.run.smithi119:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-27T09:26:49.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:49 smithi138 ceph-mon[34885]: pgmap v250: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-05-27T09:26:49.663 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:49 smithi119 ceph-mon[29082]: pgmap v250: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-05-27T09:26:49.687 DEBUG:teuthology.orchestra.run.smithi138:> stat -- /home/ubuntu/cephtest/mnt.1 2024-05-27T09:26:49.748 INFO:teuthology.orchestra.run.smithi138.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-05-27T09:26:49.749 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 1 Blocks: 0 IO Block: 65536 directory 2024-05-27T09:26:49.749 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 80h/128d Inode: 1 Links: 3 2024-05-27T09:26:49.749 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T09:26:49.749 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-27T09:26:49.749 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-05-27T09:26:49.749 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-05-27 09:26:49.671674832 +0000 2024-05-27T09:26:49.749 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-05-27 09:26:49.671674832 +0000 2024-05-27T09:26:49.749 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-05-27 09:25:58.430767379 +0000 2024-05-27T09:26:49.749 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-05-27T09:26:49.749 DEBUG:teuthology.orchestra.run.smithi138:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-05-27T09:26:49.845 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:26:49.846 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:26:49.892 INFO:tasks.workunit.client.0.smithi119.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-27T09:26:49.902 INFO:tasks.workunit.client.1.smithi138.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-05-27T09:26:49.907 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:26:51.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:51 smithi138 ceph-mon[34885]: pgmap v251: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-05-27T09:26:51.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:51 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:26:51.736 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:51 smithi119 ceph-mon[29082]: pgmap v251: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-05-27T09:26:51.737 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:51 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:26:52.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.064+0000 7fc77ba00700 1 -- 172.21.15.119:0/1302944328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7741023a0 msgr2=0x7fc7741027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:52.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.064+0000 7fc77ba00700 1 --2- 172.21.15.119:0/1302944328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7741023a0 0x7fc7741027b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc768009910 tx=0x7fc768009c20 comp rx=0 tx=0).stop 2024-05-27T09:26:52.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.064+0000 7fc77ba00700 1 -- 172.21.15.119:0/1302944328 shutdown_connections 2024-05-27T09:26:52.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.064+0000 7fc77ba00700 1 --2- 172.21.15.119:0/1302944328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7741035a0 0x7fc7741039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:52.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.064+0000 7fc77ba00700 1 --2- 172.21.15.119:0/1302944328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7741023a0 0x7fc7741027b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:52.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.064+0000 7fc77ba00700 1 -- 172.21.15.119:0/1302944328 >> 172.21.15.119:0/1302944328 conn(0x7fc7740fd990 msgr2=0x7fc7740ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:52.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.065+0000 7fc77ba00700 1 -- 172.21.15.119:0/1302944328 shutdown_connections 2024-05-27T09:26:52.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.065+0000 7fc77ba00700 1 -- 172.21.15.119:0/1302944328 wait complete. 2024-05-27T09:26:52.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.066+0000 7fc77ba00700 1 Processor -- start 2024-05-27T09:26:52.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.066+0000 7fc77ba00700 1 -- start start 2024-05-27T09:26:52.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.067+0000 7fc77ba00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7741023a0 0x7fc774197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:52.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.067+0000 7fc77ba00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7741035a0 0x7fc774198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:52.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.067+0000 7fc77ba00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7741987a0 con 0x7fc7741035a0 2024-05-27T09:26:52.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.067+0000 7fc77ba00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77419d1b0 con 0x7fc7741023a0 2024-05-27T09:26:52.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.067+0000 7fc773e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7741035a0 0x7fc774198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:52.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.067+0000 7fc773e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7741035a0 0x7fc774198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53926/0 (socket says 172.21.15.119:53926) 2024-05-27T09:26:52.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.067+0000 7fc773e00700 1 -- 172.21.15.119:0/1442408852 learned_addr learned my addr 172.21.15.119:0/1442408852 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:26:52.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.067+0000 7fc779000700 1 --2- 172.21.15.119:0/1442408852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7741023a0 0x7fc774197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:52.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.068+0000 7fc773e00700 1 -- 172.21.15.119:0/1442408852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7741023a0 msgr2=0x7fc774197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:52.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.068+0000 7fc773e00700 1 --2- 172.21.15.119:0/1442408852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7741023a0 0x7fc774197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:52.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.068+0000 7fc773e00700 1 -- 172.21.15.119:0/1442408852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7680095f0 con 0x7fc7741035a0 2024-05-27T09:26:52.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.068+0000 7fc773e00700 1 --2- 172.21.15.119:0/1442408852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7741035a0 0x7fc774198180 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc76000b4b0 tx=0x7fc76000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:52.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.068+0000 7fc771600700 1 -- 172.21.15.119:0/1442408852 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc760011800 con 0x7fc7741035a0 2024-05-27T09:26:52.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.068+0000 7fc771600700 1 -- 172.21.15.119:0/1442408852 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc760011e10 con 0x7fc7741035a0 2024-05-27T09:26:52.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.068+0000 7fc771600700 1 -- 172.21.15.119:0/1442408852 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc76000f2e0 con 0x7fc7741035a0 2024-05-27T09:26:52.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.068+0000 7fc77ba00700 1 -- 172.21.15.119:0/1442408852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77419d3b0 con 0x7fc7741035a0 2024-05-27T09:26:52.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.068+0000 7fc77ba00700 1 -- 172.21.15.119:0/1442408852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77419d950 con 0x7fc7741035a0 2024-05-27T09:26:52.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.070+0000 7fc77ba00700 1 -- 172.21.15.119:0/1442408852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77410af70 con 0x7fc7741035a0 2024-05-27T09:26:52.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.071+0000 7fc771600700 1 -- 172.21.15.119:0/1442408852 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fc760010380 con 0x7fc7741035a0 2024-05-27T09:26:52.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.071+0000 7fc771600700 1 --2- 172.21.15.119:0/1442408852 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc75c06c680 0x7fc75c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:52.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.072+0000 7fc771600700 1 -- 172.21.15.119:0/1442408852 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc76008b410 con 0x7fc7741035a0 2024-05-27T09:26:52.072 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.072+0000 7fc779000700 1 --2- 172.21.15.119:0/1442408852 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc75c06c680 0x7fc75c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:52.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.073+0000 7fc779000700 1 --2- 172.21.15.119:0/1442408852 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc75c06c680 0x7fc75c06eb30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc768000c00 tx=0x7fc768005fd0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:52.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.076+0000 7fc771600700 1 -- 172.21.15.119:0/1442408852 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc760055b10 con 0x7fc7741035a0 2024-05-27T09:26:52.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.290+0000 7fc77ba00700 1 -- 172.21.15.119:0/1442408852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fc77410b130 con 0x7fc7741035a0 2024-05-27T09:26:52.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.291+0000 7fc771600700 1 -- 172.21.15.119:0/1442408852 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7fc760059130 con 0x7fc7741035a0 2024-05-27T09:26:52.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.293+0000 7fc77ba00700 1 -- 172.21.15.119:0/1442408852 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc75c06c680 msgr2=0x7fc75c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:52.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.293+0000 7fc77ba00700 1 --2- 172.21.15.119:0/1442408852 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc75c06c680 0x7fc75c06eb30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc768000c00 tx=0x7fc768005fd0 comp rx=0 tx=0).stop 2024-05-27T09:26:52.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.293+0000 7fc77ba00700 1 -- 172.21.15.119:0/1442408852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7741035a0 msgr2=0x7fc774198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:52.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.293+0000 7fc77ba00700 1 --2- 172.21.15.119:0/1442408852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7741035a0 0x7fc774198180 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc76000b4b0 tx=0x7fc76000b870 comp rx=0 tx=0).stop 2024-05-27T09:26:52.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.294+0000 7fc77ba00700 1 -- 172.21.15.119:0/1442408852 shutdown_connections 2024-05-27T09:26:52.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.294+0000 7fc77ba00700 1 --2- 172.21.15.119:0/1442408852 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc75c06c680 0x7fc75c06eb30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:52.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.294+0000 7fc77ba00700 1 --2- 172.21.15.119:0/1442408852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7741023a0 0x7fc774197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:52.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.294+0000 7fc77ba00700 1 --2- 172.21.15.119:0/1442408852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7741035a0 0x7fc774198180 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:52.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.294+0000 7fc77ba00700 1 -- 172.21.15.119:0/1442408852 >> 172.21.15.119:0/1442408852 conn(0x7fc7740fd990 msgr2=0x7fc7741067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:52.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.294+0000 7fc77ba00700 1 -- 172.21.15.119:0/1442408852 shutdown_connections 2024-05-27T09:26:52.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:52.294+0000 7fc77ba00700 1 -- 172.21.15.119:0/1442408852 wait complete. 2024-05-27T09:26:53.110 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-05-27T09:26:53.575 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:26:53.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:53 smithi138 ceph-mon[34885]: pgmap v252: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-05-27T09:26:53.720 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:53 smithi119 ceph-mon[29082]: pgmap v252: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-05-27T09:26:55.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:55 smithi119 ceph-mon[29082]: pgmap v253: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-05-27T09:26:55.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:55 smithi138 ceph-mon[34885]: pgmap v253: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-05-27T09:26:55.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.714+0000 7f80a1200700 1 -- 172.21.15.119:0/264816877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f809c1023a0 msgr2=0x7f809c1027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:55.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.714+0000 7f80a1200700 1 --2- 172.21.15.119:0/264816877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f809c1023a0 0x7f809c1027b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f808c009910 tx=0x7f808c009c20 comp rx=0 tx=0).stop 2024-05-27T09:26:55.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.714+0000 7f80a1200700 1 -- 172.21.15.119:0/264816877 shutdown_connections 2024-05-27T09:26:55.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.714+0000 7f80a1200700 1 --2- 172.21.15.119:0/264816877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f809c1035a0 0x7f809c1039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:55.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.714+0000 7f80a1200700 1 --2- 172.21.15.119:0/264816877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f809c1023a0 0x7f809c1027b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:55.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.714+0000 7f80a1200700 1 -- 172.21.15.119:0/264816877 >> 172.21.15.119:0/264816877 conn(0x7f809c0fd990 msgr2=0x7f809c0ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:55.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.715+0000 7f80a1200700 1 -- 172.21.15.119:0/264816877 shutdown_connections 2024-05-27T09:26:55.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.715+0000 7f80a1200700 1 -- 172.21.15.119:0/264816877 wait complete. 2024-05-27T09:26:55.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.716+0000 7f80a1200700 1 Processor -- start 2024-05-27T09:26:55.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.716+0000 7f80a1200700 1 -- start start 2024-05-27T09:26:55.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.717+0000 7f80a1200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f809c1023a0 0x7f809c197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:55.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.717+0000 7f80a1200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f809c1035a0 0x7f809c1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:55.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.717+0000 7f80a1200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c198710 con 0x7f809c1035a0 2024-05-27T09:26:55.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.717+0000 7f80a1200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c198850 con 0x7f809c1023a0 2024-05-27T09:26:55.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.717+0000 7f8091400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f809c1035a0 0x7f809c1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:55.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.717+0000 7f8091400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f809c1035a0 0x7f809c1980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53944/0 (socket says 172.21.15.119:53944) 2024-05-27T09:26:55.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.717+0000 7f8091400700 1 -- 172.21.15.119:0/240978303 learned_addr learned my addr 172.21.15.119:0/240978303 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:26:55.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.717+0000 7f8099e00700 1 --2- 172.21.15.119:0/240978303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f809c1023a0 0x7f809c197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:55.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.718+0000 7f8091400700 1 -- 172.21.15.119:0/240978303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f809c1023a0 msgr2=0x7f809c197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:55.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.718+0000 7f8091400700 1 --2- 172.21.15.119:0/240978303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f809c1023a0 0x7f809c197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:55.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.718+0000 7f8091400700 1 -- 172.21.15.119:0/240978303 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808c0095f0 con 0x7f809c1035a0 2024-05-27T09:26:55.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.718+0000 7f8091400700 1 --2- 172.21.15.119:0/240978303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f809c1035a0 0x7f809c1980f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f808400d8d0 tx=0x7f808400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:55.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.718+0000 7f8093400700 1 -- 172.21.15.119:0/240978303 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8084009940 con 0x7f809c1035a0 2024-05-27T09:26:55.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.718+0000 7f8093400700 1 -- 172.21.15.119:0/240978303 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8084010460 con 0x7f809c1035a0 2024-05-27T09:26:55.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.718+0000 7f8093400700 1 -- 172.21.15.119:0/240978303 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f808400f5f0 con 0x7f809c1035a0 2024-05-27T09:26:55.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.718+0000 7f80a1200700 1 -- 172.21.15.119:0/240978303 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f809c19d300 con 0x7f809c1035a0 2024-05-27T09:26:55.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.718+0000 7f80a1200700 1 -- 172.21.15.119:0/240978303 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809c19d850 con 0x7f809c1035a0 2024-05-27T09:26:55.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.720+0000 7f80a1200700 1 -- 172.21.15.119:0/240978303 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f809c10aef0 con 0x7f809c1035a0 2024-05-27T09:26:55.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.721+0000 7f8093400700 1 -- 172.21.15.119:0/240978303 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f80840105d0 con 0x7f809c1035a0 2024-05-27T09:26:55.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.722+0000 7f8093400700 1 --2- 172.21.15.119:0/240978303 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f808006c680 0x7f808006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:55.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.722+0000 7f8093400700 1 -- 172.21.15.119:0/240978303 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f808408b810 con 0x7f809c1035a0 2024-05-27T09:26:55.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.722+0000 7f8099e00700 1 --2- 172.21.15.119:0/240978303 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f808006c680 0x7f808006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:55.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.723+0000 7f8099e00700 1 --2- 172.21.15.119:0/240978303 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f808006c680 0x7f808006eb30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f808c006010 tx=0x7f808c00b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:55.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.726+0000 7f8093400700 1 -- 172.21.15.119:0/240978303 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f80840599a0 con 0x7f809c1035a0 2024-05-27T09:26:55.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.940+0000 7f80a1200700 1 -- 172.21.15.119:0/240978303 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f809c10b0e0 con 0x7f809c1035a0 2024-05-27T09:26:55.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.941+0000 7f8093400700 1 -- 172.21.15.119:0/240978303 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f808405de60 con 0x7f809c1035a0 2024-05-27T09:26:55.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.943+0000 7f80a1200700 1 -- 172.21.15.119:0/240978303 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f808006c680 msgr2=0x7f808006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:55.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.943+0000 7f80a1200700 1 --2- 172.21.15.119:0/240978303 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f808006c680 0x7f808006eb30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f808c006010 tx=0x7f808c00b540 comp rx=0 tx=0).stop 2024-05-27T09:26:55.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.944+0000 7f80a1200700 1 -- 172.21.15.119:0/240978303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f809c1035a0 msgr2=0x7f809c1980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:55.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.944+0000 7f80a1200700 1 --2- 172.21.15.119:0/240978303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f809c1035a0 0x7f809c1980f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f808400d8d0 tx=0x7f808400dc90 comp rx=0 tx=0).stop 2024-05-27T09:26:55.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.944+0000 7f80a1200700 1 -- 172.21.15.119:0/240978303 shutdown_connections 2024-05-27T09:26:55.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.944+0000 7f80a1200700 1 --2- 172.21.15.119:0/240978303 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f808006c680 0x7f808006eb30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:55.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.944+0000 7f80a1200700 1 --2- 172.21.15.119:0/240978303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f809c1023a0 0x7f809c197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:55.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.944+0000 7f80a1200700 1 --2- 172.21.15.119:0/240978303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f809c1035a0 0x7f809c1980f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:55.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.944+0000 7f80a1200700 1 -- 172.21.15.119:0/240978303 >> 172.21.15.119:0/240978303 conn(0x7f809c0fd990 msgr2=0x7f809c1067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:55.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.944+0000 7f80a1200700 1 -- 172.21.15.119:0/240978303 shutdown_connections 2024-05-27T09:26:55.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:55.944+0000 7f80a1200700 1 -- 172.21.15.119:0/240978303 wait complete. 2024-05-27T09:26:56.784 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'ceph config set global log_to_journald false --force' 2024-05-27T09:26:57.192 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:26:57.345 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:57 smithi119 ceph-mon[29082]: pgmap v254: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 0 op/s 2024-05-27T09:26:57.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:57 smithi138 ceph-mon[34885]: pgmap v254: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 0 op/s 2024-05-27T09:26:59.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.488+0000 7fdbd5a00700 1 -- 172.21.15.119:0/1489233658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbd0073750 msgr2=0x7fdbd0073bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:59.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.488+0000 7fdbd5a00700 1 --2- 172.21.15.119:0/1489233658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbd0073750 0x7fdbd0073bc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc009910 tx=0x7fdbbc009c20 comp rx=0 tx=0).stop 2024-05-27T09:26:59.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.488+0000 7fdbd5a00700 1 -- 172.21.15.119:0/1489233658 shutdown_connections 2024-05-27T09:26:59.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.488+0000 7fdbd5a00700 1 --2- 172.21.15.119:0/1489233658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbd0073750 0x7fdbd0073bc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:59.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.488+0000 7fdbd5a00700 1 --2- 172.21.15.119:0/1489233658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbd0074d20 0x7fdbd0073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:59.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.488+0000 7fdbd5a00700 1 -- 172.21.15.119:0/1489233658 >> 172.21.15.119:0/1489233658 conn(0x7fdbd00fb6e0 msgr2=0x7fdbd00fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:59.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.488+0000 7fdbd5a00700 1 -- 172.21.15.119:0/1489233658 shutdown_connections 2024-05-27T09:26:59.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.488+0000 7fdbd5a00700 1 -- 172.21.15.119:0/1489233658 wait complete. 2024-05-27T09:26:59.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.489+0000 7fdbd5a00700 1 Processor -- start 2024-05-27T09:26:59.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.490+0000 7fdbd5a00700 1 -- start start 2024-05-27T09:26:59.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.490+0000 7fdbd5a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbd0073750 0x7fdbd0197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:59.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.490+0000 7fdbd5a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbd0074d20 0x7fdbd0198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:59.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.490+0000 7fdbd5a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd01986b0 con 0x7fdbd0074d20 2024-05-27T09:26:59.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.490+0000 7fdbd5a00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd01987f0 con 0x7fdbd0073750 2024-05-27T09:26:59.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.490+0000 7fdbce000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbd0074d20 0x7fdbd0198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:59.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.491+0000 7fdbce000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbd0074d20 0x7fdbd0198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48482/0 (socket says 172.21.15.119:48482) 2024-05-27T09:26:59.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.491+0000 7fdbce000700 1 -- 172.21.15.119:0/809203007 learned_addr learned my addr 172.21.15.119:0/809203007 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:26:59.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.491+0000 7fdbcea00700 1 --2- 172.21.15.119:0/809203007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbd0073750 0x7fdbd0197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:59.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.491+0000 7fdbce000700 1 -- 172.21.15.119:0/809203007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbd0073750 msgr2=0x7fdbd0197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:59.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.491+0000 7fdbce000700 1 --2- 172.21.15.119:0/809203007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbd0073750 0x7fdbd0197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:59.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.491+0000 7fdbce000700 1 -- 172.21.15.119:0/809203007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbbc0095f0 con 0x7fdbd0074d20 2024-05-27T09:26:59.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.491+0000 7fdbcea00700 1 --2- 172.21.15.119:0/809203007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbd0073750 0x7fdbd0197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:26:59.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.491+0000 7fdbce000700 1 --2- 172.21.15.119:0/809203007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbd0074d20 0x7fdbd0198090 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc005b40 tx=0x7fdbbc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:59.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.492+0000 7fdbc7400700 1 -- 172.21.15.119:0/809203007 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbbc01d070 con 0x7fdbd0074d20 2024-05-27T09:26:59.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.492+0000 7fdbc7400700 1 -- 172.21.15.119:0/809203007 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdbbc022470 con 0x7fdbd0074d20 2024-05-27T09:26:59.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.492+0000 7fdbc7400700 1 -- 172.21.15.119:0/809203007 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbbc00f690 con 0x7fdbd0074d20 2024-05-27T09:26:59.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.492+0000 7fdbd5a00700 1 -- 172.21.15.119:0/809203007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd019d240 con 0x7fdbd0074d20 2024-05-27T09:26:59.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.492+0000 7fdbd5a00700 1 -- 172.21.15.119:0/809203007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd019d730 con 0x7fdbd0074d20 2024-05-27T09:26:59.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.493+0000 7fdbd5a00700 1 -- 172.21.15.119:0/809203007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd010ae90 con 0x7fdbd0074d20 2024-05-27T09:26:59.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.495+0000 7fdbc7400700 1 -- 172.21.15.119:0/809203007 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fdbbc0225e0 con 0x7fdbd0074d20 2024-05-27T09:26:59.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.496+0000 7fdbc7400700 1 --2- 172.21.15.119:0/809203007 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fdbb406c680 0x7fdbb406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:26:59.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.496+0000 7fdbc7400700 1 -- 172.21.15.119:0/809203007 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fdbbc08d030 con 0x7fdbd0074d20 2024-05-27T09:26:59.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.497+0000 7fdbcea00700 1 --2- 172.21.15.119:0/809203007 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fdbb406c680 0x7fdbb406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:26:59.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.497+0000 7fdbcea00700 1 --2- 172.21.15.119:0/809203007 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fdbb406c680 0x7fdbb406eb30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0005fd0 tx=0x7fdbc0005dc0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:26:59.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.502+0000 7fdbc7400700 1 -- 172.21.15.119:0/809203007 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fdbbc05b260 con 0x7fdbd0074d20 2024-05-27T09:26:59.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:26:59 smithi138 ceph-mon[34885]: pgmap v255: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-05-27T09:26:59.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.719+0000 7fdbd5a00700 1 -- 172.21.15.119:0/809203007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fdbd019dfd0 con 0x7fdbd0074d20 2024-05-27T09:26:59.721 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:26:59 smithi119 ceph-mon[29082]: pgmap v255: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-05-27T09:26:59.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.720+0000 7fdbc7400700 1 -- 172.21.15.119:0/809203007 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7fdbbc05adf0 con 0x7fdbd0074d20 2024-05-27T09:26:59.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.724+0000 7fdbd5a00700 1 -- 172.21.15.119:0/809203007 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fdbb406c680 msgr2=0x7fdbb406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:59.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.724+0000 7fdbd5a00700 1 --2- 172.21.15.119:0/809203007 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fdbb406c680 0x7fdbb406eb30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0005fd0 tx=0x7fdbc0005dc0 comp rx=0 tx=0).stop 2024-05-27T09:26:59.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.724+0000 7fdbd5a00700 1 -- 172.21.15.119:0/809203007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbd0074d20 msgr2=0x7fdbd0198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:26:59.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.724+0000 7fdbd5a00700 1 --2- 172.21.15.119:0/809203007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbd0074d20 0x7fdbd0198090 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc005b40 tx=0x7fdbbc00be20 comp rx=0 tx=0).stop 2024-05-27T09:26:59.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.724+0000 7fdbd5a00700 1 -- 172.21.15.119:0/809203007 shutdown_connections 2024-05-27T09:26:59.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.724+0000 7fdbd5a00700 1 --2- 172.21.15.119:0/809203007 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fdbb406c680 0x7fdbb406eb30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:59.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.724+0000 7fdbd5a00700 1 --2- 172.21.15.119:0/809203007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbd0073750 0x7fdbd0197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:59.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.724+0000 7fdbd5a00700 1 --2- 172.21.15.119:0/809203007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbd0074d20 0x7fdbd0198090 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:26:59.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.724+0000 7fdbd5a00700 1 -- 172.21.15.119:0/809203007 >> 172.21.15.119:0/809203007 conn(0x7fdbd00fb6e0 msgr2=0x7fdbd0105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:26:59.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.724+0000 7fdbd5a00700 1 -- 172.21.15.119:0/809203007 shutdown_connections 2024-05-27T09:26:59.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:26:59.725+0000 7fdbd5a00700 1 -- 172.21.15.119:0/809203007 wait complete. 2024-05-27T09:27:00.654 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-05-27T09:27:01.168 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:27:01.611 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:01 smithi119 ceph-mon[29082]: pgmap v256: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-05-27T09:27:01.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:01 smithi138 ceph-mon[34885]: pgmap v256: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-05-27T09:27:03.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.162+0000 7f55d0200700 1 -- 172.21.15.119:0/3967321158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55c81067a0 msgr2=0x7f55c8106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:03.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.162+0000 7f55d0200700 1 --2- 172.21.15.119:0/3967321158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55c81067a0 0x7f55c8106bb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009910 tx=0x7f55bc009c20 comp rx=0 tx=0).stop 2024-05-27T09:27:03.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.162+0000 7f55d0200700 1 -- 172.21.15.119:0/3967321158 shutdown_connections 2024-05-27T09:27:03.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.162+0000 7f55d0200700 1 --2- 172.21.15.119:0/3967321158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55c81079a0 0x7f55c8107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:03.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.162+0000 7f55d0200700 1 --2- 172.21.15.119:0/3967321158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55c81067a0 0x7f55c8106bb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:03.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.162+0000 7f55d0200700 1 -- 172.21.15.119:0/3967321158 >> 172.21.15.119:0/3967321158 conn(0x7f55c8075910 msgr2=0x7f55c8077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:03.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.162+0000 7f55d0200700 1 -- 172.21.15.119:0/3967321158 shutdown_connections 2024-05-27T09:27:03.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.162+0000 7f55d0200700 1 -- 172.21.15.119:0/3967321158 wait complete. 2024-05-27T09:27:03.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.164+0000 7f55d0200700 1 Processor -- start 2024-05-27T09:27:03.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.164+0000 7f55d0200700 1 -- start start 2024-05-27T09:27:03.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.164+0000 7f55d0200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55c81067a0 0x7f55c819bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:03.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.164+0000 7f55d0200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55c81079a0 0x7f55c819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:03.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.164+0000 7f55d0200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c819cb10 con 0x7f55c81067a0 2024-05-27T09:27:03.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.164+0000 7f55d0200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c819cc50 con 0x7f55c81079a0 2024-05-27T09:27:03.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.165+0000 7f55cd800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55c81067a0 0x7f55c819bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:03.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.165+0000 7f55cd800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55c81067a0 0x7f55c819bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48504/0 (socket says 172.21.15.119:48504) 2024-05-27T09:27:03.165 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.165+0000 7f55cd800700 1 -- 172.21.15.119:0/26564838 learned_addr learned my addr 172.21.15.119:0/26564838 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:27:03.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.165+0000 7f55cce00700 1 --2- 172.21.15.119:0/26564838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55c81079a0 0x7f55c819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:03.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.165+0000 7f55cd800700 1 -- 172.21.15.119:0/26564838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55c81079a0 msgr2=0x7f55c819c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:03.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.165+0000 7f55cd800700 1 --2- 172.21.15.119:0/26564838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55c81079a0 0x7f55c819c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:03.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.165+0000 7f55cd800700 1 -- 172.21.15.119:0/26564838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc0095f0 con 0x7f55c81067a0 2024-05-27T09:27:03.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.165+0000 7f55cd800700 1 --2- 172.21.15.119:0/26564838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55c81067a0 0x7f55c819bfb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f55bc005b40 tx=0x7f55bc00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:03.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.166+0000 7f55c6000700 1 -- 172.21.15.119:0/26564838 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55bc01d070 con 0x7f55c81067a0 2024-05-27T09:27:03.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.166+0000 7f55c6000700 1 -- 172.21.15.119:0/26564838 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f55bc005dc0 con 0x7f55c81067a0 2024-05-27T09:27:03.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.166+0000 7f55c6000700 1 -- 172.21.15.119:0/26564838 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55bc0226a0 con 0x7f55c81067a0 2024-05-27T09:27:03.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.166+0000 7f55d0200700 1 -- 172.21.15.119:0/26564838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c81a16a0 con 0x7f55c81067a0 2024-05-27T09:27:03.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.166+0000 7f55d0200700 1 -- 172.21.15.119:0/26564838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c81a1b90 con 0x7f55c81067a0 2024-05-27T09:27:03.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.167+0000 7f55d0200700 1 -- 172.21.15.119:0/26564838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c810f2f0 con 0x7f55c81067a0 2024-05-27T09:27:03.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.168+0000 7f55c6000700 1 -- 172.21.15.119:0/26564838 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f55bc003680 con 0x7f55c81067a0 2024-05-27T09:27:03.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.169+0000 7f55c6000700 1 --2- 172.21.15.119:0/26564838 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f55b006c680 0x7f55b006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:03.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.169+0000 7f55c6000700 1 -- 172.21.15.119:0/26564838 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f55bc08cea0 con 0x7f55c81067a0 2024-05-27T09:27:03.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.169+0000 7f55cce00700 1 --2- 172.21.15.119:0/26564838 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f55b006c680 0x7f55b006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:03.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.170+0000 7f55cce00700 1 --2- 172.21.15.119:0/26564838 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f55b006c680 0x7f55b006eb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009b30 tx=0x7f55b4009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:03.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.174+0000 7f55c6000700 1 -- 172.21.15.119:0/26564838 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f55bc05af90 con 0x7f55c81067a0 2024-05-27T09:27:03.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:03.409+0000 7f55d0200700 1 -- 172.21.15.119:0/26564838 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f55c810c3a0 con 0x7f55b006c680 2024-05-27T09:27:03.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:03 smithi138 ceph-mon[34885]: pgmap v257: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-05-27T09:27:03.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:03 smithi119 ceph-mon[29082]: pgmap v257: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-05-27T09:27:04.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:04 smithi138 ceph-mon[34885]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:27:04.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:04 smithi119 ceph-mon[29082]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:27:05.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:05 smithi138 ceph-mon[34885]: pgmap v258: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-05-27T09:27:05.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:05 smithi119 ceph-mon[29082]: pgmap v258: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-05-27T09:27:07.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:07 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:27:07.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:07 smithi138 ceph-mon[34885]: pgmap v259: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-05-27T09:27:07.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:07 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:27:07.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:07 smithi119 ceph-mon[29082]: pgmap v259: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-05-27T09:27:09.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:09 smithi138 ceph-mon[34885]: pgmap v260: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:09.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:09 smithi119 ceph-mon[29082]: pgmap v260: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:11.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:11 smithi138 ceph-mon[34885]: pgmap v261: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:11.718 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:11 smithi119 ceph-mon[29082]: pgmap v261: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:13.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:13 smithi138 ceph-mon[34885]: pgmap v262: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:13.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:13 smithi119 ceph-mon[29082]: pgmap v262: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:15.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:15 smithi138 ceph-mon[34885]: pgmap v263: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:15.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:15 smithi119 ceph-mon[29082]: pgmap v263: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:17.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:17 smithi138 ceph-mon[34885]: pgmap v264: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:17.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:17 smithi119 ceph-mon[29082]: pgmap v264: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:19.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:19 smithi138 ceph-mon[34885]: pgmap v265: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:19.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:19 smithi119 ceph-mon[29082]: pgmap v265: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:21.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:21 smithi138 ceph-mon[34885]: pgmap v266: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:21.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:21 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:27:21.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:21 smithi119 ceph-mon[29082]: pgmap v266: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:21.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:21 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:27:23.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:23 smithi138 ceph-mon[34885]: pgmap v267: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:23.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:23 smithi119 ceph-mon[29082]: pgmap v267: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:25.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:25 smithi119 ceph-mon[29082]: pgmap v268: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:25.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:25 smithi138 ceph-mon[34885]: pgmap v268: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:27.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:27 smithi138 ceph-mon[34885]: pgmap v269: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-05-27T09:27:27.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:27 smithi119 ceph-mon[29082]: pgmap v269: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-05-27T09:27:29.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:29 smithi138 ceph-mon[34885]: pgmap v270: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-05-27T09:27:29.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:29 smithi119 ceph-mon[29082]: pgmap v270: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-05-27T09:27:31.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:31 smithi138 ceph-mon[34885]: pgmap v271: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:27:31.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:31 smithi119 ceph-mon[29082]: pgmap v271: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:27:32.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:32 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:27:32.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:32 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:27:33.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:33 smithi119 ceph-mon[29082]: pgmap v272: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:27:33.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:33 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:27:33.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:33 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:27:33.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:33 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:27:33.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:33 smithi138 ceph-mon[34885]: pgmap v272: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:27:33.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:33 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:27:33.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:33 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:27:33.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:33 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:27:35.682 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:35 smithi119 ceph-mon[29082]: pgmap v273: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-05-27T09:27:35.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:35 smithi138 ceph-mon[34885]: pgmap v273: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-05-27T09:27:37.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:37 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:27:37.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:37 smithi138 ceph-mon[34885]: pgmap v274: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-05-27T09:27:37.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:37 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:27:37.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:37 smithi119 ceph-mon[29082]: pgmap v274: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-05-27T09:27:39.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:39 smithi138 ceph-mon[34885]: pgmap v275: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-05-27T09:27:39.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:39 smithi119 ceph-mon[29082]: pgmap v275: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-05-27T09:27:41.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:41 smithi138 ceph-mon[34885]: pgmap v276: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-05-27T09:27:41.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:41 smithi119 ceph-mon[29082]: pgmap v276: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-05-27T09:27:43.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:43 smithi138 ceph-mon[34885]: pgmap v277: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-05-27T09:27:43.736 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:43 smithi119 ceph-mon[29082]: pgmap v277: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-05-27T09:27:45.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:45 smithi119 ceph-mon[29082]: pgmap v278: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-05-27T09:27:45.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:45 smithi138 ceph-mon[34885]: pgmap v278: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-05-27T09:27:47.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:47 smithi138 ceph-mon[34885]: pgmap v279: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:47.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:47 smithi119 ceph-mon[29082]: pgmap v279: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:49.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:49 smithi138 ceph-mon[34885]: pgmap v280: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:49.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:49 smithi119 ceph-mon[29082]: pgmap v280: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:51.605 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:51 smithi119 ceph-mon[29082]: pgmap v281: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:51.605 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:51 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:27:51.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:51 smithi138 ceph-mon[34885]: pgmap v281: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:51.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:51 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:27:52.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:52.035+0000 7f55c6000700 1 -- 172.21.15.119:0/26564838 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f55c810c3a0 con 0x7f55b006c680 2024-05-27T09:27:52.036 INFO:teuthology.orchestra.run.smithi119.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:27:52.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:52.038+0000 7f55d0200700 1 -- 172.21.15.119:0/26564838 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f55b006c680 msgr2=0x7f55b006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:52.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:52.038+0000 7f55d0200700 1 --2- 172.21.15.119:0/26564838 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f55b006c680 0x7f55b006eb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009b30 tx=0x7f55b4009260 comp rx=0 tx=0).stop 2024-05-27T09:27:52.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:52.038+0000 7f55d0200700 1 -- 172.21.15.119:0/26564838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55c81067a0 msgr2=0x7f55c819bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:52.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:52.038+0000 7f55d0200700 1 --2- 172.21.15.119:0/26564838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55c81067a0 0x7f55c819bfb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f55bc005b40 tx=0x7f55bc00fab0 comp rx=0 tx=0).stop 2024-05-27T09:27:52.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:52.038+0000 7f55d0200700 1 -- 172.21.15.119:0/26564838 shutdown_connections 2024-05-27T09:27:52.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:52.038+0000 7f55d0200700 1 --2- 172.21.15.119:0/26564838 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f55b006c680 0x7f55b006eb30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:52.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:52.038+0000 7f55d0200700 1 --2- 172.21.15.119:0/26564838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55c81067a0 0x7f55c819bfb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:52.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:52.038+0000 7f55d0200700 1 --2- 172.21.15.119:0/26564838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55c81079a0 0x7f55c819c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:52.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:52.038+0000 7f55d0200700 1 -- 172.21.15.119:0/26564838 >> 172.21.15.119:0/26564838 conn(0x7f55c8075910 msgr2=0x7f55c810abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:52.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:52.039+0000 7f55d0200700 1 -- 172.21.15.119:0/26564838 shutdown_connections 2024-05-27T09:27:52.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:52.039+0000 7f55d0200700 1 -- 172.21.15.119:0/26564838 wait complete. 2024-05-27T09:27:52.784 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-05-27T09:27:53.245 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:27:53.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:53 smithi138 ceph-mon[34885]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:27:53.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:53 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:27:53.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:53 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:27:53.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:53 smithi138 ceph-mon[34885]: pgmap v282: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:53.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:53 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:27:53.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:53 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:27:53.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:53 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:27:53.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:53 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:27:53.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:53 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:27:53.425 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:53 smithi119 ceph-mon[29082]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:27:53.425 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:53 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:27:53.425 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:53 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:27:53.425 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:53 smithi119 ceph-mon[29082]: pgmap v282: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:53.425 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:53 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:27:53.425 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:53 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:27:53.425 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:53 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:27:53.426 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:53 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:27:53.426 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:53 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:27:54.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:54 smithi119 ceph-mon[29082]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:27:54.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:54 smithi138 ceph-mon[34885]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:27:55.116 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:55 smithi138 ceph-mon[34885]: pgmap v283: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:55.231 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:55 smithi119 ceph-mon[29082]: pgmap v283: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:56.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.124+0000 7f2fe9400700 1 -- 172.21.15.119:0/2242646156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe4071910 msgr2=0x7f2fe4071d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:56.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.124+0000 7f2fe9400700 1 --2- 172.21.15.119:0/2242646156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe4071910 0x7f2fe4071d20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4009910 tx=0x7f2fd4009c20 comp rx=0 tx=0).stop 2024-05-27T09:27:56.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.124+0000 7f2fe9400700 1 -- 172.21.15.119:0/2242646156 shutdown_connections 2024-05-27T09:27:56.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.124+0000 7f2fe9400700 1 --2- 172.21.15.119:0/2242646156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fe40722f0 0x7f2fe4109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:56.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.124+0000 7f2fe9400700 1 --2- 172.21.15.119:0/2242646156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe4071910 0x7f2fe4071d20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:56.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.124+0000 7f2fe9400700 1 -- 172.21.15.119:0/2242646156 >> 172.21.15.119:0/2242646156 conn(0x7f2fe406d050 msgr2=0x7f2fe406f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:56.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.127+0000 7f2fe9400700 1 -- 172.21.15.119:0/2242646156 shutdown_connections 2024-05-27T09:27:56.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.127+0000 7f2fe9400700 1 -- 172.21.15.119:0/2242646156 wait complete. 2024-05-27T09:27:56.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.129+0000 7f2fe9400700 1 Processor -- start 2024-05-27T09:27:56.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.129+0000 7f2fe9400700 1 -- start start 2024-05-27T09:27:56.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.129+0000 7f2fe9400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fe4071910 0x7f2fe410e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:56.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.130+0000 7f2fe9400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe40722f0 0x7f2fe410ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:56.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.130+0000 7f2fe9400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe410b4d0 con 0x7f2fe40722f0 2024-05-27T09:27:56.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.130+0000 7f2fe9400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe410b640 con 0x7f2fe4071910 2024-05-27T09:27:56.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.130+0000 7f2fe1e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe40722f0 0x7f2fe410ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:56.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.130+0000 7f2fe1e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe40722f0 0x7f2fe410ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57402/0 (socket says 172.21.15.119:57402) 2024-05-27T09:27:56.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.130+0000 7f2fe1e00700 1 -- 172.21.15.119:0/1637718513 learned_addr learned my addr 172.21.15.119:0/1637718513 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:27:56.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.130+0000 7f2fe2800700 1 --2- 172.21.15.119:0/1637718513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fe4071910 0x7f2fe410e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:56.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.131+0000 7f2fe1e00700 1 -- 172.21.15.119:0/1637718513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fe4071910 msgr2=0x7f2fe410e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:56.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.131+0000 7f2fe1e00700 1 --2- 172.21.15.119:0/1637718513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fe4071910 0x7f2fe410e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:56.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.131+0000 7f2fe1e00700 1 -- 172.21.15.119:0/1637718513 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd40095f0 con 0x7f2fe40722f0 2024-05-27T09:27:56.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.132+0000 7f2fe1e00700 1 --2- 172.21.15.119:0/1637718513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe40722f0 0x7f2fe410ef20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000e920 tx=0x7f2fd000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:56.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.133+0000 7f2fdb400700 1 -- 172.21.15.119:0/1637718513 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fd000c9d0 con 0x7f2fe40722f0 2024-05-27T09:27:56.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.133+0000 7f2fdb400700 1 -- 172.21.15.119:0/1637718513 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2fd0010430 con 0x7f2fe40722f0 2024-05-27T09:27:56.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.133+0000 7f2fdb400700 1 -- 172.21.15.119:0/1637718513 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fd0018650 con 0x7f2fe40722f0 2024-05-27T09:27:56.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.134+0000 7f2fe9400700 1 -- 172.21.15.119:0/1637718513 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe410b920 con 0x7f2fe40722f0 2024-05-27T09:27:56.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.134+0000 7f2fe9400700 1 -- 172.21.15.119:0/1637718513 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe410be70 con 0x7f2fe40722f0 2024-05-27T09:27:56.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.135+0000 7f2fe9400700 1 -- 172.21.15.119:0/1637718513 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe419a620 con 0x7f2fe40722f0 2024-05-27T09:27:56.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.137+0000 7f2fdb400700 1 -- 172.21.15.119:0/1637718513 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f2fd00105a0 con 0x7f2fe40722f0 2024-05-27T09:27:56.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.139+0000 7f2fdb400700 1 --2- 172.21.15.119:0/1637718513 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f2fc806c720 0x7f2fc806ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:56.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.139+0000 7f2fdb400700 1 -- 172.21.15.119:0/1637718513 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f2fd0014070 con 0x7f2fe40722f0 2024-05-27T09:27:56.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.140+0000 7f2fe2800700 1 --2- 172.21.15.119:0/1637718513 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f2fc806c720 0x7f2fc806ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:56.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.141+0000 7f2fe2800700 1 --2- 172.21.15.119:0/1637718513 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f2fc806c720 0x7f2fc806ebd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400b6f0 tx=0x7f2fd4009f70 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:56.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.148+0000 7f2fdb400700 1 -- 172.21.15.119:0/1637718513 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2fd0059f60 con 0x7f2fe40722f0 2024-05-27T09:27:56.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.510+0000 7f2fe9400700 1 -- 172.21.15.119:0/1637718513 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fe4061190 con 0x7f2fc806c720 2024-05-27T09:27:56.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.513+0000 7f2fdb400700 1 -- 172.21.15.119:0/1637718513 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7f2fe4061190 con 0x7f2fc806c720 2024-05-27T09:27:56.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.515+0000 7f2fe9400700 1 -- 172.21.15.119:0/1637718513 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f2fc806c720 msgr2=0x7f2fc806ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:56.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.515+0000 7f2fe9400700 1 --2- 172.21.15.119:0/1637718513 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f2fc806c720 0x7f2fc806ebd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400b6f0 tx=0x7f2fd4009f70 comp rx=0 tx=0).stop 2024-05-27T09:27:56.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.515+0000 7f2fe9400700 1 -- 172.21.15.119:0/1637718513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe40722f0 msgr2=0x7f2fe410ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:56.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.515+0000 7f2fe9400700 1 --2- 172.21.15.119:0/1637718513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe40722f0 0x7f2fe410ef20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000e920 tx=0x7f2fd000ec30 comp rx=0 tx=0).stop 2024-05-27T09:27:56.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.516+0000 7f2fe9400700 1 -- 172.21.15.119:0/1637718513 shutdown_connections 2024-05-27T09:27:56.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.516+0000 7f2fe9400700 1 --2- 172.21.15.119:0/1637718513 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f2fc806c720 0x7f2fc806ebd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:56.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.516+0000 7f2fe9400700 1 --2- 172.21.15.119:0/1637718513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fe4071910 0x7f2fe410e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:56.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.516+0000 7f2fe9400700 1 --2- 172.21.15.119:0/1637718513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe40722f0 0x7f2fe410ef20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:56.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.516+0000 7f2fe9400700 1 -- 172.21.15.119:0/1637718513 >> 172.21.15.119:0/1637718513 conn(0x7f2fe406d050 msgr2=0x7f2fe4108e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:56.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.516+0000 7f2fe9400700 1 -- 172.21.15.119:0/1637718513 shutdown_connections 2024-05-27T09:27:56.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.516+0000 7f2fe9400700 1 -- 172.21.15.119:0/1637718513 wait complete. 2024-05-27T09:27:56.539 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:27:56.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.693+0000 7f9200800700 1 -- 172.21.15.119:0/2259879063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f8100150 msgr2=0x7f91f8100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:56.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.693+0000 7f9200800700 1 --2- 172.21.15.119:0/2259879063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f8100150 0x7f91f8100560 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009910 tx=0x7f91ec009c20 comp rx=0 tx=0).stop 2024-05-27T09:27:56.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.693+0000 7f9200800700 1 -- 172.21.15.119:0/2259879063 shutdown_connections 2024-05-27T09:27:56.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.693+0000 7f9200800700 1 --2- 172.21.15.119:0/2259879063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91f8101350 0x7f91f81017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:56.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.693+0000 7f9200800700 1 --2- 172.21.15.119:0/2259879063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f8100150 0x7f91f8100560 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:56.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.693+0000 7f9200800700 1 -- 172.21.15.119:0/2259879063 >> 172.21.15.119:0/2259879063 conn(0x7f91f80fb6e0 msgr2=0x7f91f80fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:56.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.694+0000 7f9200800700 1 -- 172.21.15.119:0/2259879063 shutdown_connections 2024-05-27T09:27:56.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.694+0000 7f9200800700 1 -- 172.21.15.119:0/2259879063 wait complete. 2024-05-27T09:27:56.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.695+0000 7f9200800700 1 Processor -- start 2024-05-27T09:27:56.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.695+0000 7f9200800700 1 -- start start 2024-05-27T09:27:56.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.695+0000 7f9200800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91f8100150 0x7f91f8197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:56.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.695+0000 7f9200800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f8101350 0x7f91f81980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:56.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.695+0000 7f9200800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f8198700 con 0x7f91f8101350 2024-05-27T09:27:56.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.695+0000 7f9200800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f8198840 con 0x7f91f8100150 2024-05-27T09:27:56.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.696+0000 7f91fde00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91f8100150 0x7f91f8197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:56.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.696+0000 7f91fd400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f8101350 0x7f91f81980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:56.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.696+0000 7f91fde00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91f8100150 0x7f91f8197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.119:42122/0 (socket says 172.21.15.119:42122) 2024-05-27T09:27:56.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.696+0000 7f91fde00700 1 -- 172.21.15.119:0/2546582662 learned_addr learned my addr 172.21.15.119:0/2546582662 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:27:56.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.696+0000 7f91fde00700 1 -- 172.21.15.119:0/2546582662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f8101350 msgr2=0x7f91f81980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:56.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.696+0000 7f91fde00700 1 --2- 172.21.15.119:0/2546582662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f8101350 0x7f91f81980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:56.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.696+0000 7f91fde00700 1 -- 172.21.15.119:0/2546582662 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ec0095f0 con 0x7f91f8100150 2024-05-27T09:27:56.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.697+0000 7f91fde00700 1 --2- 172.21.15.119:0/2546582662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91f8100150 0x7f91f8197ba0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f91ec005b40 tx=0x7f91ec00bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:56.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.697+0000 7f91f2a00700 1 -- 172.21.15.119:0/2546582662 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91ec01d070 con 0x7f91f8100150 2024-05-27T09:27:56.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.697+0000 7f9200800700 1 -- 172.21.15.119:0/2546582662 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f819d290 con 0x7f91f8100150 2024-05-27T09:27:56.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.697+0000 7f91f2a00700 1 -- 172.21.15.119:0/2546582662 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f91ec00f460 con 0x7f91f8100150 2024-05-27T09:27:56.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.697+0000 7f91f2a00700 1 -- 172.21.15.119:0/2546582662 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91ec021640 con 0x7f91f8100150 2024-05-27T09:27:56.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.698+0000 7f9200800700 1 -- 172.21.15.119:0/2546582662 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f819d780 con 0x7f91f8100150 2024-05-27T09:27:56.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.699+0000 7f9200800700 1 -- 172.21.15.119:0/2546582662 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f8105430 con 0x7f91f8100150 2024-05-27T09:27:56.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.700+0000 7f91f2a00700 1 -- 172.21.15.119:0/2546582662 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f91ec00fa60 con 0x7f91f8100150 2024-05-27T09:27:56.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.701+0000 7f91f2a00700 1 --2- 172.21.15.119:0/2546582662 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f91e006c680 0x7f91e006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:56.701 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.701+0000 7f91f2a00700 1 -- 172.21.15.119:0/2546582662 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f91ec08cb90 con 0x7f91f8100150 2024-05-27T09:27:56.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.701+0000 7f91fd400700 1 --2- 172.21.15.119:0/2546582662 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f91e006c680 0x7f91e006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:56.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.702+0000 7f91fd400700 1 --2- 172.21.15.119:0/2546582662 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f91e006c680 0x7f91e006eb30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f91e8005950 tx=0x7f91e800b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:56.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.710+0000 7f91f2a00700 1 -- 172.21.15.119:0/2546582662 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f91ec05ac80 con 0x7f91f8100150 2024-05-27T09:27:56.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.964+0000 7f9200800700 1 -- 172.21.15.119:0/2546582662 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f91f8061190 con 0x7f91e006c680 2024-05-27T09:27:56.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.966+0000 7f91f2a00700 1 -- 172.21.15.119:0/2546582662 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f91f8061190 con 0x7f91e006c680 2024-05-27T09:27:56.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.968+0000 7f9200800700 1 -- 172.21.15.119:0/2546582662 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f91e006c680 msgr2=0x7f91e006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:56.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.968+0000 7f9200800700 1 --2- 172.21.15.119:0/2546582662 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f91e006c680 0x7f91e006eb30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f91e8005950 tx=0x7f91e800b410 comp rx=0 tx=0).stop 2024-05-27T09:27:56.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.968+0000 7f9200800700 1 -- 172.21.15.119:0/2546582662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91f8100150 msgr2=0x7f91f8197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:56.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.968+0000 7f9200800700 1 --2- 172.21.15.119:0/2546582662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91f8100150 0x7f91f8197ba0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f91ec005b40 tx=0x7f91ec00bd70 comp rx=0 tx=0).stop 2024-05-27T09:27:56.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.969+0000 7f9200800700 1 -- 172.21.15.119:0/2546582662 shutdown_connections 2024-05-27T09:27:56.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.969+0000 7f9200800700 1 --2- 172.21.15.119:0/2546582662 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f91e006c680 0x7f91e006eb30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:56.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.969+0000 7f9200800700 1 --2- 172.21.15.119:0/2546582662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91f8100150 0x7f91f8197ba0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:56.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.969+0000 7f9200800700 1 --2- 172.21.15.119:0/2546582662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f8101350 0x7f91f81980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:56.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.969+0000 7f9200800700 1 -- 172.21.15.119:0/2546582662 >> 172.21.15.119:0/2546582662 conn(0x7f91f80fb6e0 msgr2=0x7f91f8102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:56.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.969+0000 7f9200800700 1 -- 172.21.15.119:0/2546582662 shutdown_connections 2024-05-27T09:27:56.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:56.969+0000 7f9200800700 1 -- 172.21.15.119:0/2546582662 wait complete. 2024-05-27T09:27:57.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.116+0000 7f9fa8400700 1 -- 172.21.15.119:0/1669621855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fa0103500 msgr2=0x7f9fa01058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:57.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.116+0000 7f9fa8400700 1 --2- 172.21.15.119:0/1669621855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fa0103500 0x7f9fa01058e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009910 tx=0x7f9f8c009c20 comp rx=0 tx=0).stop 2024-05-27T09:27:57.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.116+0000 7f9fa8400700 1 -- 172.21.15.119:0/1669621855 shutdown_connections 2024-05-27T09:27:57.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.116+0000 7f9fa8400700 1 --2- 172.21.15.119:0/1669621855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fa0103500 0x7f9fa01058e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:57.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.116+0000 7f9fa8400700 1 --2- 172.21.15.119:0/1669621855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa0100be0 0x7f9fa0102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:57.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.116+0000 7f9fa8400700 1 -- 172.21.15.119:0/1669621855 >> 172.21.15.119:0/1669621855 conn(0x7f9fa00fa5e0 msgr2=0x7f9fa00fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:57.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.116+0000 7f9fa8400700 1 -- 172.21.15.119:0/1669621855 shutdown_connections 2024-05-27T09:27:57.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.117+0000 7f9fa8400700 1 -- 172.21.15.119:0/1669621855 wait complete. 2024-05-27T09:27:57.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.118+0000 7f9fa8400700 1 Processor -- start 2024-05-27T09:27:57.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.118+0000 7f9fa8400700 1 -- start start 2024-05-27T09:27:57.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.119+0000 7f9fa8400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fa0100be0 0x7f9fa0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:57.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.119+0000 7f9fa8400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa0103500 0x7f9fa0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:57.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.119+0000 7f9fa8400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa01986a0 con 0x7f9fa0100be0 2024-05-27T09:27:57.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.119+0000 7f9fa8400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa01987e0 con 0x7f9fa0103500 2024-05-27T09:27:57.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.119+0000 7f9fa5a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fa0100be0 0x7f9fa0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:57.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.119+0000 7f9fa5a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fa0100be0 0x7f9fa0197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57446/0 (socket says 172.21.15.119:57446) 2024-05-27T09:27:57.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.120+0000 7f9fa5a00700 1 -- 172.21.15.119:0/2389439644 learned_addr learned my addr 172.21.15.119:0/2389439644 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:27:57.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.120+0000 7f9fa5000700 1 --2- 172.21.15.119:0/2389439644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa0103500 0x7f9fa0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:57.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.120+0000 7f9fa5a00700 1 -- 172.21.15.119:0/2389439644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa0103500 msgr2=0x7f9fa0198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:57.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.120+0000 7f9fa5a00700 1 --2- 172.21.15.119:0/2389439644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa0103500 0x7f9fa0198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:57.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.121+0000 7f9fa5a00700 1 -- 172.21.15.119:0/2389439644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c0095f0 con 0x7f9fa0100be0 2024-05-27T09:27:57.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.121+0000 7f9fa5a00700 1 --2- 172.21.15.119:0/2389439644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fa0100be0 0x7f9fa0197b40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400ece0 tx=0x7f9f94009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:57.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.121+0000 7f9f9e000700 1 -- 172.21.15.119:0/2389439644 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f9400cab0 con 0x7f9fa0100be0 2024-05-27T09:27:57.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.122+0000 7f9f9e000700 1 -- 172.21.15.119:0/2389439644 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9f9400cc10 con 0x7f9fa0100be0 2024-05-27T09:27:57.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.122+0000 7f9fa8400700 1 -- 172.21.15.119:0/2389439644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa019d290 con 0x7f9fa0100be0 2024-05-27T09:27:57.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.122+0000 7f9f9e000700 1 -- 172.21.15.119:0/2389439644 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f94018650 con 0x7f9fa0100be0 2024-05-27T09:27:57.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.122+0000 7f9fa8400700 1 -- 172.21.15.119:0/2389439644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa019d7e0 con 0x7f9fa0100be0 2024-05-27T09:27:57.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.123+0000 7f9fa8400700 1 -- 172.21.15.119:0/2389439644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa00fc1c0 con 0x7f9fa0100be0 2024-05-27T09:27:57.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.126+0000 7f9f9e000700 1 -- 172.21.15.119:0/2389439644 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f9f94010a70 con 0x7f9fa0100be0 2024-05-27T09:27:57.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.127+0000 7f9f9e000700 1 --2- 172.21.15.119:0/2389439644 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f9f8806c680 0x7f9f8806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:57.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.127+0000 7f9f9e000700 1 -- 172.21.15.119:0/2389439644 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9f94014070 con 0x7f9fa0100be0 2024-05-27T09:27:57.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.128+0000 7f9fa5000700 1 --2- 172.21.15.119:0/2389439644 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f9f8806c680 0x7f9f8806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:57.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.129+0000 7f9fa5000700 1 --2- 172.21.15.119:0/2389439644 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f9f8806c680 0x7f9f8806eb30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009fd0 tx=0x7f9f8c005fd0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:57.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.134+0000 7f9f9e000700 1 -- 172.21.15.119:0/2389439644 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9f94059f40 con 0x7f9fa0100be0 2024-05-27T09:27:57.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.358+0000 7f9fa8400700 1 -- 172.21.15.119:0/2389439644 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9fa0061190 con 0x7f9f8806c680 2024-05-27T09:27:57.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.370+0000 7f9f9e000700 1 -- 172.21.15.119:0/2389439644 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3528 (secure 0 0 0) 0x7f9fa0061190 con 0x7f9f8806c680 2024-05-27T09:27:57.371 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:27:57.371 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (7m) 87s ago 9m 13.1M - 0.25.0 c8568f914cd2 bc280230ab28 2024-05-27T09:27:57.371 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (9m) 87s ago 9m 7486k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:27:57.371 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (8m) 94s ago 8m 7394k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:27:57.371 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (9m) 87s ago 9m 6665k - 18.2.0 dc2bc1663786 e5fafde44183 2024-05-27T09:27:57.371 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (7m) 94s ago 7m 6660k - 18.2.0 dc2bc1663786 46956ab99bab 2024-05-27T09:27:57.371 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (6m) 87s ago 8m 74.6M - 9.4.7 954c08fa6188 16eeaded4295 2024-05-27T09:27:57.371 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (2m) 87s ago 119s 11.5M - 18.2.0 dc2bc1663786 93b21713fb1e 2024-05-27T09:27:57.371 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (112s) 87s ago 111s 14.4M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:27:57.372 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (108s) 94s ago 108s 11.5M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:27:57.372 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (116s) 94s ago 115s 14.4M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:27:57.372 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:9283,8765,8443 running (10m) 87s ago 10m 487M - 18.2.0 dc2bc1663786 abe5e83463f2 2024-05-27T09:27:57.372 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (7m) 94s ago 7m 437M - 18.2.0 dc2bc1663786 5aa669adddf7 2024-05-27T09:27:57.372 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (10m) 87s ago 10m 46.3M 2048M 18.2.0 dc2bc1663786 ba998b789a1c 2024-05-27T09:27:57.372 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (7m) 94s ago 7m 36.1M 2048M 18.2.0 dc2bc1663786 60216fd67078 2024-05-27T09:27:57.372 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (9m) 87s ago 9m 7788k - 1.5.0 0da6a335fe13 79c36d4e5b04 2024-05-27T09:27:57.372 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (7m) 94s ago 7m 7931k - 1.5.0 0da6a335fe13 80ee890e8830 2024-05-27T09:27:57.372 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (6m) 87s ago 6m 45.8M 3947M 18.2.0 dc2bc1663786 825bd1392508 2024-05-27T09:27:57.372 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (6m) 87s ago 6m 47.4M 3947M 18.2.0 dc2bc1663786 fce9819903d9 2024-05-27T09:27:57.372 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (5m) 87s ago 5m 46.2M 3947M 18.2.0 dc2bc1663786 3955a6112c1c 2024-05-27T09:27:57.372 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (4m) 94s ago 4m 45.1M 4971M 18.2.0 dc2bc1663786 693c92220622 2024-05-27T09:27:57.372 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (4m) 94s ago 4m 42.4M 4971M 18.2.0 dc2bc1663786 7559e658e917 2024-05-27T09:27:57.372 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (3m) 94s ago 3m 42.9M 4971M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:27:57.373 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (6m) 87s ago 8m 55.8M - 2.43.0 a07b618ecd1d 7f3d89d1f425 2024-05-27T09:27:57.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.373+0000 7f9fa8400700 1 -- 172.21.15.119:0/2389439644 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f9f8806c680 msgr2=0x7f9f8806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:57.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.373+0000 7f9fa8400700 1 --2- 172.21.15.119:0/2389439644 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f9f8806c680 0x7f9f8806eb30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009fd0 tx=0x7f9f8c005fd0 comp rx=0 tx=0).stop 2024-05-27T09:27:57.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.373+0000 7f9fa8400700 1 -- 172.21.15.119:0/2389439644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fa0100be0 msgr2=0x7f9fa0197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:57.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.373+0000 7f9fa8400700 1 --2- 172.21.15.119:0/2389439644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fa0100be0 0x7f9fa0197b40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400ece0 tx=0x7f9f94009ea0 comp rx=0 tx=0).stop 2024-05-27T09:27:57.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.373+0000 7f9fa8400700 1 -- 172.21.15.119:0/2389439644 shutdown_connections 2024-05-27T09:27:57.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.373+0000 7f9fa8400700 1 --2- 172.21.15.119:0/2389439644 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f9f8806c680 0x7f9f8806eb30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:57.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.373+0000 7f9fa8400700 1 --2- 172.21.15.119:0/2389439644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fa0100be0 0x7f9fa0197b40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:57.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.373+0000 7f9fa8400700 1 --2- 172.21.15.119:0/2389439644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa0103500 0x7f9fa0198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:57.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.373+0000 7f9fa8400700 1 -- 172.21.15.119:0/2389439644 >> 172.21.15.119:0/2389439644 conn(0x7f9fa00fa5e0 msgr2=0x7f9fa0104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:57.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.374+0000 7f9fa8400700 1 -- 172.21.15.119:0/2389439644 shutdown_connections 2024-05-27T09:27:57.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.374+0000 7f9fa8400700 1 -- 172.21.15.119:0/2389439644 wait complete. 2024-05-27T09:27:57.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.516+0000 7fe99c600700 1 -- 172.21.15.119:0/2151034288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9940fe9d0 msgr2=0x7fe9940fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:57.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.516+0000 7fe99c600700 1 --2- 172.21.15.119:0/2151034288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9940fe9d0 0x7fe9940fede0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe988009910 tx=0x7fe988009c20 comp rx=0 tx=0).stop 2024-05-27T09:27:57.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.516+0000 7fe99c600700 1 -- 172.21.15.119:0/2151034288 shutdown_connections 2024-05-27T09:27:57.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.516+0000 7fe99c600700 1 --2- 172.21.15.119:0/2151034288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9940ff320 0x7fe9940ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:57.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.516+0000 7fe99c600700 1 --2- 172.21.15.119:0/2151034288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9940fe9d0 0x7fe9940fede0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:57.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.516+0000 7fe99c600700 1 -- 172.21.15.119:0/2151034288 >> 172.21.15.119:0/2151034288 conn(0x7fe9940fa5e0 msgr2=0x7fe9940fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:57.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.517+0000 7fe99c600700 1 -- 172.21.15.119:0/2151034288 shutdown_connections 2024-05-27T09:27:57.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.517+0000 7fe99c600700 1 -- 172.21.15.119:0/2151034288 wait complete. 2024-05-27T09:27:57.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.518+0000 7fe99c600700 1 Processor -- start 2024-05-27T09:27:57.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.518+0000 7fe99c600700 1 -- start start 2024-05-27T09:27:57.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.518+0000 7fe99c600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9940fe9d0 0x7fe994071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:57.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.519+0000 7fe99c600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9940ff320 0x7fe994072220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:57.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.519+0000 7fe99c600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe994072840 con 0x7fe9940fe9d0 2024-05-27T09:27:57.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.519+0000 7fe99c600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9941a58b0 con 0x7fe9940ff320 2024-05-27T09:27:57.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.519+0000 7fe999c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9940fe9d0 0x7fe994071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:57.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.519+0000 7fe999c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9940fe9d0 0x7fe994071ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57450/0 (socket says 172.21.15.119:57450) 2024-05-27T09:27:57.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.519+0000 7fe999c00700 1 -- 172.21.15.119:0/3609384595 learned_addr learned my addr 172.21.15.119:0/3609384595 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:27:57.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.519+0000 7fe999c00700 1 -- 172.21.15.119:0/3609384595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9940ff320 msgr2=0x7fe994072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:57.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.519+0000 7fe999c00700 1 --2- 172.21.15.119:0/3609384595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9940ff320 0x7fe994072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:57.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.519+0000 7fe999c00700 1 -- 172.21.15.119:0/3609384595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9880095f0 con 0x7fe9940fe9d0 2024-05-27T09:27:57.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.520+0000 7fe999c00700 1 --2- 172.21.15.119:0/3609384595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9940fe9d0 0x7fe994071ce0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe988000c00 tx=0x7fe98800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:57.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.520+0000 7fe992000700 1 -- 172.21.15.119:0/3609384595 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe98801d070 con 0x7fe9940fe9d0 2024-05-27T09:27:57.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.520+0000 7fe992000700 1 -- 172.21.15.119:0/3609384595 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe988005dc0 con 0x7fe9940fe9d0 2024-05-27T09:27:57.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.520+0000 7fe99c600700 1 -- 172.21.15.119:0/3609384595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9941a5a50 con 0x7fe9940fe9d0 2024-05-27T09:27:57.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.520+0000 7fe992000700 1 -- 172.21.15.119:0/3609384595 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9880226a0 con 0x7fe9940fe9d0 2024-05-27T09:27:57.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.521+0000 7fe99c600700 1 -- 172.21.15.119:0/3609384595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9941a5ef0 con 0x7fe9940fe9d0 2024-05-27T09:27:57.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.523+0000 7fe99c600700 1 -- 172.21.15.119:0/3609384595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe99410ae80 con 0x7fe9940fe9d0 2024-05-27T09:27:57.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.524+0000 7fe992000700 1 -- 172.21.15.119:0/3609384595 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fe988003680 con 0x7fe9940fe9d0 2024-05-27T09:27:57.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.526+0000 7fe992000700 1 --2- 172.21.15.119:0/3609384595 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fe97c06c6d0 0x7fe97c06eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:57.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.526+0000 7fe992000700 1 -- 172.21.15.119:0/3609384595 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe98808ccd0 con 0x7fe9940fe9d0 2024-05-27T09:27:57.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.527+0000 7fe999200700 1 --2- 172.21.15.119:0/3609384595 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fe97c06c6d0 0x7fe97c06eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:57.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.528+0000 7fe999200700 1 --2- 172.21.15.119:0/3609384595 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fe97c06c6d0 0x7fe97c06eb80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe980005950 tx=0x7fe9800058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:57.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:57.529+0000 7fe992000700 1 -- 172.21.15.119:0/3609384595 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe988092020 con 0x7fe9940fe9d0 2024-05-27T09:27:57.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:57 smithi119 ceph-mon[29082]: pgmap v284: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:57.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:57 smithi119 ceph-mon[29082]: from='client.14530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:27:57.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:57 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:27:57.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:57 smithi119 ceph-mon[29082]: Upgrade: Target is version 19.0.0-3868-ga6507388 (unknown) 2024-05-27T09:27:57.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:57 smithi119 ceph-mon[29082]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, digests ['quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6'] 2024-05-27T09:27:57.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:57 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:27:57.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:57 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:27:57.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:57 smithi119 ceph-mon[29082]: Upgrade: Need to upgrade myself (mgr.smithi119.ncwmog) 2024-05-27T09:27:57.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:57 smithi138 ceph-mon[34885]: pgmap v284: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:57.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:57 smithi138 ceph-mon[34885]: from='client.14530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:27:57.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:57 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:27:57.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:57 smithi138 ceph-mon[34885]: Upgrade: Target is version 19.0.0-3868-ga6507388 (unknown) 2024-05-27T09:27:57.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:57 smithi138 ceph-mon[34885]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, digests ['quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6'] 2024-05-27T09:27:57.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:57 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:27:57.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:57 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:27:57.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:57 smithi138 ceph-mon[34885]: Upgrade: Need to upgrade myself (mgr.smithi119.ncwmog) 2024-05-27T09:27:58.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.052+0000 7fe99c600700 1 -- 172.21.15.119:0/3609384595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe9941a69e0 con 0x7fe9940fe9d0 2024-05-27T09:27:58.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.053+0000 7fe992000700 1 -- 172.21.15.119:0/3609384595 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7fe98805adc0 con 0x7fe9940fe9d0 2024-05-27T09:27:58.054 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:27:58.054 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:27:58.054 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-05-27T09:27:58.054 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:27:58.054 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:27:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-05-27T09:27:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:27:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:27:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-05-27T09:27:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:27:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:27:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-05-27T09:27:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:27:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:27:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 14 2024-05-27T09:27:58.055 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:27:58.055 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:27:58.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.055+0000 7fe99c600700 1 -- 172.21.15.119:0/3609384595 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fe97c06c6d0 msgr2=0x7fe97c06eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:58.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.055+0000 7fe99c600700 1 --2- 172.21.15.119:0/3609384595 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fe97c06c6d0 0x7fe97c06eb80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe980005950 tx=0x7fe9800058e0 comp rx=0 tx=0).stop 2024-05-27T09:27:58.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.056+0000 7fe99c600700 1 -- 172.21.15.119:0/3609384595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9940fe9d0 msgr2=0x7fe994071ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:58.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.056+0000 7fe99c600700 1 --2- 172.21.15.119:0/3609384595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9940fe9d0 0x7fe994071ce0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe988000c00 tx=0x7fe98800f740 comp rx=0 tx=0).stop 2024-05-27T09:27:58.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.056+0000 7fe99c600700 1 -- 172.21.15.119:0/3609384595 shutdown_connections 2024-05-27T09:27:58.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.056+0000 7fe99c600700 1 --2- 172.21.15.119:0/3609384595 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fe97c06c6d0 0x7fe97c06eb80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:58.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.056+0000 7fe99c600700 1 --2- 172.21.15.119:0/3609384595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9940fe9d0 0x7fe994071ce0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:58.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.056+0000 7fe99c600700 1 --2- 172.21.15.119:0/3609384595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9940ff320 0x7fe994072220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:58.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.056+0000 7fe99c600700 1 -- 172.21.15.119:0/3609384595 >> 172.21.15.119:0/3609384595 conn(0x7fe9940fa5e0 msgr2=0x7fe994107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:58.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.057+0000 7fe99c600700 1 -- 172.21.15.119:0/3609384595 shutdown_connections 2024-05-27T09:27:58.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.057+0000 7fe99c600700 1 -- 172.21.15.119:0/3609384595 wait complete. 2024-05-27T09:27:58.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.200+0000 7f5073e00700 1 -- 172.21.15.119:0/4101527788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5074101190 msgr2=0x7f50741015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:58.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.200+0000 7f5073e00700 1 --2- 172.21.15.119:0/4101527788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5074101190 0x7f50741015e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5060009910 tx=0x7f5060009c20 comp rx=0 tx=0).stop 2024-05-27T09:27:58.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.200+0000 7f5073e00700 1 -- 172.21.15.119:0/4101527788 shutdown_connections 2024-05-27T09:27:58.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.200+0000 7f5073e00700 1 --2- 172.21.15.119:0/4101527788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5074101190 0x7f50741015e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:58.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.200+0000 7f5073e00700 1 --2- 172.21.15.119:0/4101527788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50740fff90 0x7f50741003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:58.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.200+0000 7f5073e00700 1 -- 172.21.15.119:0/4101527788 >> 172.21.15.119:0/4101527788 conn(0x7f50740fb500 msgr2=0x7f50740fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:58.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.201+0000 7f5073e00700 1 -- 172.21.15.119:0/4101527788 shutdown_connections 2024-05-27T09:27:58.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.201+0000 7f5073e00700 1 -- 172.21.15.119:0/4101527788 wait complete. 2024-05-27T09:27:58.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.202+0000 7f5073e00700 1 Processor -- start 2024-05-27T09:27:58.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.202+0000 7f5073e00700 1 -- start start 2024-05-27T09:27:58.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.202+0000 7f5073e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50740fff90 0x7f50741979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:58.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.202+0000 7f5073e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5074101190 0x7f5074197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:58.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.202+0000 7f5073e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5074198510 con 0x7f50740fff90 2024-05-27T09:27:58.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.202+0000 7f5073e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5074198650 con 0x7f5074101190 2024-05-27T09:27:58.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.203+0000 7f5072000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5074101190 0x7f5074197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:58.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.203+0000 7f5072000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5074101190 0x7f5074197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.119:42176/0 (socket says 172.21.15.119:42176) 2024-05-27T09:27:58.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.203+0000 7f5072000700 1 -- 172.21.15.119:0/3696250259 learned_addr learned my addr 172.21.15.119:0/3696250259 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:27:58.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.203+0000 7f5072a00700 1 --2- 172.21.15.119:0/3696250259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50740fff90 0x7f50741979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:58.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.204+0000 7f5072000700 1 -- 172.21.15.119:0/3696250259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50740fff90 msgr2=0x7f50741979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:58.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.204+0000 7f5072000700 1 --2- 172.21.15.119:0/3696250259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50740fff90 0x7f50741979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:58.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.204+0000 7f5072000700 1 -- 172.21.15.119:0/3696250259 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50600095f0 con 0x7f5074101190 2024-05-27T09:27:58.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.204+0000 7f5072000700 1 --2- 172.21.15.119:0/3696250259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5074101190 0x7f5074197ef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f50600092e0 tx=0x7f506000f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:58.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.204+0000 7f506b400700 1 -- 172.21.15.119:0/3696250259 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f506001d070 con 0x7f5074101190 2024-05-27T09:27:58.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.204+0000 7f5073e00700 1 -- 172.21.15.119:0/3696250259 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f507419d0a0 con 0x7f5074101190 2024-05-27T09:27:58.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.204+0000 7f506b400700 1 -- 172.21.15.119:0/3696250259 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f50600052a0 con 0x7f5074101190 2024-05-27T09:27:58.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.204+0000 7f506b400700 1 -- 172.21.15.119:0/3696250259 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5060017630 con 0x7f5074101190 2024-05-27T09:27:58.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.205+0000 7f5073e00700 1 -- 172.21.15.119:0/3696250259 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f507419d590 con 0x7f5074101190 2024-05-27T09:27:58.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.206+0000 7f5073e00700 1 -- 172.21.15.119:0/3696250259 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5054005320 con 0x7f5074101190 2024-05-27T09:27:58.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.207+0000 7f506b400700 1 -- 172.21.15.119:0/3696250259 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f5060003680 con 0x7f5074101190 2024-05-27T09:27:58.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.208+0000 7f506b400700 1 --2- 172.21.15.119:0/3696250259 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f505806c680 0x7f505806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:27:58.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.208+0000 7f506b400700 1 -- 172.21.15.119:0/3696250259 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f506008cdd0 con 0x7f5074101190 2024-05-27T09:27:58.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.209+0000 7f5072a00700 1 --2- 172.21.15.119:0/3696250259 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f505806c680 0x7f505806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:27:58.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.210+0000 7f5072a00700 1 --2- 172.21.15.119:0/3696250259 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f505806c680 0x7f505806eb30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5074100ff0 tx=0x7f5064009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:27:58.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.217+0000 7f506b400700 1 -- 172.21.15.119:0/3696250259 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f506005aec0 con 0x7f5074101190 2024-05-27T09:27:58.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.454+0000 7f5073e00700 1 -- 172.21.15.119:0/3696250259 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5054000bf0 con 0x7f505806c680 2024-05-27T09:27:58.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.456+0000 7f506b400700 1 -- 172.21.15.119:0/3696250259 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f5054000bf0 con 0x7f505806c680 2024-05-27T09:27:58.458 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:27:58.458 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:27:58.458 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:27:58.458 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-27T09:27:58.458 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [], 2024-05-27T09:27:58.459 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "0/2 daemons upgraded", 2024-05-27T09:27:58.459 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 image on host smithi138", 2024-05-27T09:27:58.459 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:27:58.459 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:27:58.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.459+0000 7f5073e00700 1 -- 172.21.15.119:0/3696250259 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f505806c680 msgr2=0x7f505806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:58.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.459+0000 7f5073e00700 1 --2- 172.21.15.119:0/3696250259 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f505806c680 0x7f505806eb30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5074100ff0 tx=0x7f5064009260 comp rx=0 tx=0).stop 2024-05-27T09:27:58.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.459+0000 7f5073e00700 1 -- 172.21.15.119:0/3696250259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5074101190 msgr2=0x7f5074197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:27:58.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.459+0000 7f5073e00700 1 --2- 172.21.15.119:0/3696250259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5074101190 0x7f5074197ef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f50600092e0 tx=0x7f506000f740 comp rx=0 tx=0).stop 2024-05-27T09:27:58.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.460+0000 7f5073e00700 1 -- 172.21.15.119:0/3696250259 shutdown_connections 2024-05-27T09:27:58.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.460+0000 7f5073e00700 1 --2- 172.21.15.119:0/3696250259 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f505806c680 0x7f505806eb30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:58.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.460+0000 7f5073e00700 1 --2- 172.21.15.119:0/3696250259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50740fff90 0x7f50741979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:58.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.460+0000 7f5073e00700 1 --2- 172.21.15.119:0/3696250259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5074101190 0x7f5074197ef0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:27:58.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.460+0000 7f5073e00700 1 -- 172.21.15.119:0/3696250259 >> 172.21.15.119:0/3696250259 conn(0x7f50740fb500 msgr2=0x7f50741023b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:27:58.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.460+0000 7f5073e00700 1 -- 172.21.15.119:0/3696250259 shutdown_connections 2024-05-27T09:27:58.462 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:27:58.460+0000 7f5073e00700 1 -- 172.21.15.119:0/3696250259 wait complete. 2024-05-27T09:27:58.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:58 smithi119 ceph-mon[29082]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:27:58.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:58 smithi119 ceph-mon[29082]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 on smithi138 2024-05-27T09:27:58.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:58 smithi119 ceph-mon[29082]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:27:58.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:58 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/3609384595' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:27:58.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:58 smithi138 ceph-mon[34885]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:27:58.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:58 smithi138 ceph-mon[34885]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 on smithi138 2024-05-27T09:27:58.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:58 smithi138 ceph-mon[34885]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:27:58.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:58 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/3609384595' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:27:59.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:59 smithi138 ceph-mon[34885]: pgmap v285: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:59.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:27:59 smithi138 ceph-mon[34885]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:27:59.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:59 smithi119 ceph-mon[29082]: pgmap v285: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:27:59.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:27:59 smithi119 ceph-mon[29082]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:28:01.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:01 smithi138 ceph-mon[34885]: pgmap v286: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:01.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:01 smithi119 ceph-mon[29082]: pgmap v286: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:03.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:03 smithi138 ceph-mon[34885]: pgmap v287: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:03.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:03 smithi119 ceph-mon[29082]: pgmap v287: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:05.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:05 smithi138 ceph-mon[34885]: pgmap v288: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:05.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:05 smithi119 ceph-mon[29082]: pgmap v288: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:07.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:07 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:28:07.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:07 smithi138 ceph-mon[34885]: pgmap v289: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:07.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:07 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:28:07.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:07 smithi119 ceph-mon[29082]: pgmap v289: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:09.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:09 smithi138 ceph-mon[34885]: pgmap v290: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:09.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:09 smithi119 ceph-mon[29082]: pgmap v290: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:11.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:11 smithi138 ceph-mon[34885]: pgmap v291: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:11.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:11 smithi119 ceph-mon[29082]: pgmap v291: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:13.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:13 smithi138 ceph-mon[34885]: pgmap v292: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:13.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:13 smithi119 ceph-mon[29082]: pgmap v292: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:15.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:15 smithi119 ceph-mon[29082]: pgmap v293: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:15.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:15 smithi138 ceph-mon[34885]: pgmap v293: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:17.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:17 smithi138 ceph-mon[34885]: pgmap v294: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:17.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:17 smithi119 ceph-mon[29082]: pgmap v294: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:19.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:19 smithi138 ceph-mon[34885]: pgmap v295: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:19.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:19 smithi119 ceph-mon[29082]: pgmap v295: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:21.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:21 smithi119 ceph-mon[29082]: pgmap v296: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:21.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:21 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:28:21.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:21 smithi138 ceph-mon[34885]: pgmap v296: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:21.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:21 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:28:23.152 INFO:tasks.workunit.client.0.smithi119.stderr:Updating files: 62% (7614/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-27T09:28:23.688 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:23 smithi138 ceph-mon[34885]: pgmap v297: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:23.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:23 smithi119 ceph-mon[29082]: pgmap v297: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:23.816 INFO:tasks.workunit.client.0.smithi119.stderr:Note: switching to 'a6507388dd5057528934822c0163b0c347ef1d5d'. 2024-05-27T09:28:23.816 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-05-27T09:28:23.816 INFO:tasks.workunit.client.0.smithi119.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-27T09:28:23.816 INFO:tasks.workunit.client.0.smithi119.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-27T09:28:23.816 INFO:tasks.workunit.client.0.smithi119.stderr:state without impacting any branches by switching back to a branch. 2024-05-27T09:28:23.817 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-05-27T09:28:23.817 INFO:tasks.workunit.client.0.smithi119.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-27T09:28:23.817 INFO:tasks.workunit.client.0.smithi119.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-27T09:28:23.817 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-05-27T09:28:23.817 INFO:tasks.workunit.client.0.smithi119.stderr: git switch -c 2024-05-27T09:28:23.817 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-05-27T09:28:23.817 INFO:tasks.workunit.client.0.smithi119.stderr:Or undo this operation with: 2024-05-27T09:28:23.817 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-05-27T09:28:23.817 INFO:tasks.workunit.client.0.smithi119.stderr: git switch - 2024-05-27T09:28:23.817 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-05-27T09:28:23.817 INFO:tasks.workunit.client.0.smithi119.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-27T09:28:23.817 INFO:tasks.workunit.client.0.smithi119.stderr: 2024-05-27T09:28:23.817 INFO:tasks.workunit.client.0.smithi119.stderr:HEAD is now at a6507388dd5 Merge PR #53755 into wip-vshankar-testing-20240521.105740-debug 2024-05-27T09:28:23.831 DEBUG:teuthology.orchestra.run.smithi119:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-27T09:28:23.894 INFO:tasks.workunit.client.0.smithi119.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-27T09:28:23.902 INFO:tasks.workunit.client.0.smithi119.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-27T09:28:23.902 INFO:tasks.workunit.client.0.smithi119.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-27T09:28:24.736 INFO:tasks.workunit.client.0.smithi119.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-27T09:28:24.869 INFO:tasks.workunit.client.0.smithi119.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-27T09:28:24.952 INFO:tasks.workunit.client.0.smithi119.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-27T09:28:24.958 INFO:tasks.workunit.client.0.smithi119.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-27T09:28:24.958 INFO:tasks.workunit.client.0.smithi119.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-27T09:28:25.039 INFO:tasks.workunit.client.0.smithi119.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-27T09:28:25.049 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-27T09:28:25.049 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-27T09:28:25.100 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-05-27T09:28:25.101 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-05-27T09:28:25.101 DEBUG:teuthology.orchestra.run.smithi119:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6507388dd5057528934822c0163b0c347ef1d5d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-05-27T09:28:25.164 INFO:tasks.workunit.client.0.smithi119.stderr:+ mkdir -p fsstress 2024-05-27T09:28:25.169 INFO:tasks.workunit.client.0.smithi119.stderr:+ pushd fsstress 2024-05-27T09:28:25.169 INFO:tasks.workunit.client.0.smithi119.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-27T09:28:25.169 INFO:tasks.workunit.client.0.smithi119.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-05-27T09:28:25.711 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:25 smithi138 ceph-mon[34885]: pgmap v298: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:25.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:25 smithi119 ceph-mon[29082]: pgmap v298: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:26.879 INFO:tasks.workunit.client.0.smithi119.stderr:+ tar xzf ltp-full.tgz 2024-05-27T09:28:27.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:27 smithi138 ceph-mon[34885]: pgmap v299: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:27.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:27 smithi119 ceph-mon[29082]: pgmap v299: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:28.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.637+0000 7ff8f8c00700 1 -- 172.21.15.119:0/856400980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f4103500 msgr2=0x7ff8f41058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:28:28.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.637+0000 7ff8f8c00700 1 --2- 172.21.15.119:0/856400980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f4103500 0x7ff8f41058e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0009910 tx=0x7ff8e0009c20 comp rx=0 tx=0).stop 2024-05-27T09:28:28.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.637+0000 7ff8f8c00700 1 -- 172.21.15.119:0/856400980 shutdown_connections 2024-05-27T09:28:28.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.637+0000 7ff8f8c00700 1 --2- 172.21.15.119:0/856400980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f4103500 0x7ff8f41058e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:28.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.637+0000 7ff8f8c00700 1 --2- 172.21.15.119:0/856400980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8f4100be0 0x7ff8f4102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:28.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.637+0000 7ff8f8c00700 1 -- 172.21.15.119:0/856400980 >> 172.21.15.119:0/856400980 conn(0x7ff8f40fa5e0 msgr2=0x7ff8f40fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:28:28.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.638+0000 7ff8f8c00700 1 -- 172.21.15.119:0/856400980 shutdown_connections 2024-05-27T09:28:28.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.638+0000 7ff8f8c00700 1 -- 172.21.15.119:0/856400980 wait complete. 2024-05-27T09:28:28.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.639+0000 7ff8f8c00700 1 Processor -- start 2024-05-27T09:28:28.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.639+0000 7ff8f8c00700 1 -- start start 2024-05-27T09:28:28.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.640+0000 7ff8f8c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8f4100be0 0x7ff8f4197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:28:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.640+0000 7ff8f8c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f4103500 0x7ff8f41980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:28:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.640+0000 7ff8f8c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f41986c0 con 0x7ff8f4103500 2024-05-27T09:28:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.640+0000 7ff8f8c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f4198800 con 0x7ff8f4100be0 2024-05-27T09:28:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.640+0000 7ff8f1400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f4103500 0x7ff8f41980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:28:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.640+0000 7ff8f1400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f4103500 0x7ff8f41980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43104/0 (socket says 172.21.15.119:43104) 2024-05-27T09:28:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.640+0000 7ff8f1400700 1 -- 172.21.15.119:0/2663235831 learned_addr learned my addr 172.21.15.119:0/2663235831 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:28:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.640+0000 7ff8f1e00700 1 --2- 172.21.15.119:0/2663235831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8f4100be0 0x7ff8f4197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:28:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.641+0000 7ff8f1400700 1 -- 172.21.15.119:0/2663235831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8f4100be0 msgr2=0x7ff8f4197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:28:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.641+0000 7ff8f1400700 1 --2- 172.21.15.119:0/2663235831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8f4100be0 0x7ff8f4197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.641+0000 7ff8f1400700 1 -- 172.21.15.119:0/2663235831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e00095f0 con 0x7ff8f4103500 2024-05-27T09:28:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.641+0000 7ff8f1400700 1 --2- 172.21.15.119:0/2663235831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f4103500 0x7ff8f41980a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0005b40 tx=0x7ff8e000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:28:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.642+0000 7ff8eaa00700 1 -- 172.21.15.119:0/2663235831 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8e001d070 con 0x7ff8f4103500 2024-05-27T09:28:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.642+0000 7ff8eaa00700 1 -- 172.21.15.119:0/2663235831 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff8e000f460 con 0x7ff8f4103500 2024-05-27T09:28:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.642+0000 7ff8f8c00700 1 -- 172.21.15.119:0/2663235831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f419d250 con 0x7ff8f4103500 2024-05-27T09:28:28.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.642+0000 7ff8eaa00700 1 -- 172.21.15.119:0/2663235831 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8e0021640 con 0x7ff8f4103500 2024-05-27T09:28:28.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.642+0000 7ff8f8c00700 1 -- 172.21.15.119:0/2663235831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f419d740 con 0x7ff8f4103500 2024-05-27T09:28:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.644+0000 7ff8f8c00700 1 -- 172.21.15.119:0/2663235831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f40fc1c0 con 0x7ff8f4103500 2024-05-27T09:28:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.645+0000 7ff8eaa00700 1 -- 172.21.15.119:0/2663235831 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7ff8e000fa90 con 0x7ff8f4103500 2024-05-27T09:28:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.646+0000 7ff8eaa00700 1 --2- 172.21.15.119:0/2663235831 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff8d806c680 0x7ff8d806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:28:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.647+0000 7ff8eaa00700 1 -- 172.21.15.119:0/2663235831 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff8e008ccc0 con 0x7ff8f4103500 2024-05-27T09:28:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.647+0000 7ff8f1e00700 1 --2- 172.21.15.119:0/2663235831 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff8d806c680 0x7ff8d806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:28:28.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.648+0000 7ff8f1e00700 1 --2- 172.21.15.119:0/2663235831 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff8d806c680 0x7ff8d806eb30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4005950 tx=0x7ff8e40058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:28:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.652+0000 7ff8eaa00700 1 -- 172.21.15.119:0/2663235831 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff8e005add0 con 0x7ff8f4103500 2024-05-27T09:28:28.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.920+0000 7ff8f8c00700 1 -- 172.21.15.119:0/2663235831 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8f4061190 con 0x7ff8d806c680 2024-05-27T09:28:28.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.922+0000 7ff8eaa00700 1 -- 172.21.15.119:0/2663235831 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7ff8f4061190 con 0x7ff8d806c680 2024-05-27T09:28:28.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.925+0000 7ff8f8c00700 1 -- 172.21.15.119:0/2663235831 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff8d806c680 msgr2=0x7ff8d806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:28:28.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.925+0000 7ff8f8c00700 1 --2- 172.21.15.119:0/2663235831 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff8d806c680 0x7ff8d806eb30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4005950 tx=0x7ff8e40058e0 comp rx=0 tx=0).stop 2024-05-27T09:28:28.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.925+0000 7ff8f8c00700 1 -- 172.21.15.119:0/2663235831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f4103500 msgr2=0x7ff8f41980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:28:28.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.925+0000 7ff8f8c00700 1 --2- 172.21.15.119:0/2663235831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f4103500 0x7ff8f41980a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0005b40 tx=0x7ff8e000be20 comp rx=0 tx=0).stop 2024-05-27T09:28:28.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.925+0000 7ff8f8c00700 1 -- 172.21.15.119:0/2663235831 shutdown_connections 2024-05-27T09:28:28.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.925+0000 7ff8f8c00700 1 --2- 172.21.15.119:0/2663235831 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7ff8d806c680 0x7ff8d806eb30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:28.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.925+0000 7ff8f8c00700 1 --2- 172.21.15.119:0/2663235831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8f4100be0 0x7ff8f4197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:28.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.925+0000 7ff8f8c00700 1 --2- 172.21.15.119:0/2663235831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8f4103500 0x7ff8f41980a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:28.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.925+0000 7ff8f8c00700 1 -- 172.21.15.119:0/2663235831 >> 172.21.15.119:0/2663235831 conn(0x7ff8f40fa5e0 msgr2=0x7ff8f40ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:28:28.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.926+0000 7ff8f8c00700 1 -- 172.21.15.119:0/2663235831 shutdown_connections 2024-05-27T09:28:28.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:28.926+0000 7ff8f8c00700 1 -- 172.21.15.119:0/2663235831 wait complete. 2024-05-27T09:28:28.945 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:28:29.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.131+0000 7f604a800700 1 -- 172.21.15.119:0/4065259936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60440fffa0 msgr2=0x7f60441003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:28:29.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.131+0000 7f604a800700 1 --2- 172.21.15.119:0/4065259936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60440fffa0 0x7f60441003b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6038009910 tx=0x7f6038009c20 comp rx=0 tx=0).stop 2024-05-27T09:28:29.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.131+0000 7f604a800700 1 -- 172.21.15.119:0/4065259936 shutdown_connections 2024-05-27T09:28:29.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.131+0000 7f604a800700 1 --2- 172.21.15.119:0/4065259936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60441011a0 0x7f60441015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:29.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.131+0000 7f604a800700 1 --2- 172.21.15.119:0/4065259936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60440fffa0 0x7f60441003b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:29.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.131+0000 7f604a800700 1 -- 172.21.15.119:0/4065259936 >> 172.21.15.119:0/4065259936 conn(0x7f60440fb510 msgr2=0x7f60440fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:28:29.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.131+0000 7f604a800700 1 -- 172.21.15.119:0/4065259936 shutdown_connections 2024-05-27T09:28:29.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.131+0000 7f604a800700 1 -- 172.21.15.119:0/4065259936 wait complete. 2024-05-27T09:28:29.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.132+0000 7f604a800700 1 Processor -- start 2024-05-27T09:28:29.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.132+0000 7f604a800700 1 -- start start 2024-05-27T09:28:29.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.133+0000 7f604a800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60440fffa0 0x7f60441979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:28:29.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.133+0000 7f604a800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60441011a0 0x7f6044197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:28:29.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.133+0000 7f604a800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6044198540 con 0x7f60440fffa0 2024-05-27T09:28:29.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.133+0000 7f604a800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6044198680 con 0x7f60441011a0 2024-05-27T09:28:29.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.133+0000 7f6049400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60440fffa0 0x7f60441979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:28:29.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.133+0000 7f6049400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60440fffa0 0x7f60441979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37022/0 (socket says 172.21.15.119:37022) 2024-05-27T09:28:29.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.133+0000 7f6049400700 1 -- 172.21.15.119:0/363709957 learned_addr learned my addr 172.21.15.119:0/363709957 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:28:29.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.133+0000 7f6048a00700 1 --2- 172.21.15.119:0/363709957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60441011a0 0x7f6044197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:28:29.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.134+0000 7f6049400700 1 -- 172.21.15.119:0/363709957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60441011a0 msgr2=0x7f6044197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:28:29.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.134+0000 7f6049400700 1 --2- 172.21.15.119:0/363709957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60441011a0 0x7f6044197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:29.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.134+0000 7f6049400700 1 -- 172.21.15.119:0/363709957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60380095f0 con 0x7f60440fffa0 2024-05-27T09:28:29.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.134+0000 7f6049400700 1 --2- 172.21.15.119:0/363709957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60440fffa0 0x7f60441979e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f603800bd70 tx=0x7f603800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:28:29.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.134+0000 7f6042000700 1 -- 172.21.15.119:0/363709957 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f603801d070 con 0x7f60440fffa0 2024-05-27T09:28:29.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.134+0000 7f6042000700 1 -- 172.21.15.119:0/363709957 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6038005dc0 con 0x7f60440fffa0 2024-05-27T09:28:29.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.134+0000 7f604a800700 1 -- 172.21.15.119:0/363709957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f604419d0d0 con 0x7f60440fffa0 2024-05-27T09:28:29.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.134+0000 7f6042000700 1 -- 172.21.15.119:0/363709957 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6038021410 con 0x7f60440fffa0 2024-05-27T09:28:29.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.135+0000 7f604a800700 1 -- 172.21.15.119:0/363709957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604419d5c0 con 0x7f60440fffa0 2024-05-27T09:28:29.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.136+0000 7f604a800700 1 -- 172.21.15.119:0/363709957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6044105280 con 0x7f60440fffa0 2024-05-27T09:28:29.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.137+0000 7f6042000700 1 -- 172.21.15.119:0/363709957 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f603800f7f0 con 0x7f60440fffa0 2024-05-27T09:28:29.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.138+0000 7f6042000700 1 --2- 172.21.15.119:0/363709957 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f602c06c680 0x7f602c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:28:29.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.138+0000 7f6042000700 1 -- 172.21.15.119:0/363709957 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f603808cb60 con 0x7f60440fffa0 2024-05-27T09:28:29.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.138+0000 7f6048a00700 1 --2- 172.21.15.119:0/363709957 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f602c06c680 0x7f602c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:28:29.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.139+0000 7f6048a00700 1 --2- 172.21.15.119:0/363709957 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f602c06c680 0x7f602c06eb30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6030004630 tx=0x7f603000b3c0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:28:29.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.144+0000 7f6042000700 1 -- 172.21.15.119:0/363709957 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f603805ac70 con 0x7f60440fffa0 2024-05-27T09:28:29.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.457+0000 7f604a800700 1 -- 172.21.15.119:0/363709957 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6044061190 con 0x7f602c06c680 2024-05-27T09:28:29.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.459+0000 7f6042000700 1 -- 172.21.15.119:0/363709957 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f6044061190 con 0x7f602c06c680 2024-05-27T09:28:29.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.462+0000 7f604a800700 1 -- 172.21.15.119:0/363709957 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f602c06c680 msgr2=0x7f602c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:28:29.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.462+0000 7f604a800700 1 --2- 172.21.15.119:0/363709957 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f602c06c680 0x7f602c06eb30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6030004630 tx=0x7f603000b3c0 comp rx=0 tx=0).stop 2024-05-27T09:28:29.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.462+0000 7f604a800700 1 -- 172.21.15.119:0/363709957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60440fffa0 msgr2=0x7f60441979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:28:29.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.462+0000 7f604a800700 1 --2- 172.21.15.119:0/363709957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60440fffa0 0x7f60441979e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f603800bd70 tx=0x7f603800bda0 comp rx=0 tx=0).stop 2024-05-27T09:28:29.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.462+0000 7f604a800700 1 -- 172.21.15.119:0/363709957 shutdown_connections 2024-05-27T09:28:29.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.462+0000 7f604a800700 1 --2- 172.21.15.119:0/363709957 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f602c06c680 0x7f602c06eb30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:29.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.462+0000 7f604a800700 1 --2- 172.21.15.119:0/363709957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60440fffa0 0x7f60441979e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:29.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.462+0000 7f604a800700 1 --2- 172.21.15.119:0/363709957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60441011a0 0x7f6044197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:29.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.462+0000 7f604a800700 1 -- 172.21.15.119:0/363709957 >> 172.21.15.119:0/363709957 conn(0x7f60440fb510 msgr2=0x7f60441023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:28:29.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.463+0000 7f604a800700 1 -- 172.21.15.119:0/363709957 shutdown_connections 2024-05-27T09:28:29.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.463+0000 7f604a800700 1 -- 172.21.15.119:0/363709957 wait complete. 2024-05-27T09:28:29.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.689+0000 7fb3c6800700 1 -- 172.21.15.119:0/3758274053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3c01021c0 msgr2=0x7fb3c01025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:28:29.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.689+0000 7fb3c6800700 1 --2- 172.21.15.119:0/3758274053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3c01021c0 0x7fb3c01025d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009910 tx=0x7fb3b4009c20 comp rx=0 tx=0).stop 2024-05-27T09:28:29.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.689+0000 7fb3c6800700 1 -- 172.21.15.119:0/3758274053 shutdown_connections 2024-05-27T09:28:29.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.689+0000 7fb3c6800700 1 --2- 172.21.15.119:0/3758274053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3c01033c0 0x7fb3c0103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:29.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.689+0000 7fb3c6800700 1 --2- 172.21.15.119:0/3758274053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3c01021c0 0x7fb3c01025d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:29.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.689+0000 7fb3c6800700 1 -- 172.21.15.119:0/3758274053 >> 172.21.15.119:0/3758274053 conn(0x7fb3c00fd790 msgr2=0x7fb3c00ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:28:29.691 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:29 smithi119 ceph-mon[29082]: pgmap v300: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:29.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.690+0000 7fb3c6800700 1 -- 172.21.15.119:0/3758274053 shutdown_connections 2024-05-27T09:28:29.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.690+0000 7fb3c6800700 1 -- 172.21.15.119:0/3758274053 wait complete. 2024-05-27T09:28:29.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.691+0000 7fb3c6800700 1 Processor -- start 2024-05-27T09:28:29.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.691+0000 7fb3c6800700 1 -- start start 2024-05-27T09:28:29.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.692+0000 7fb3c6800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3c01021c0 0x7fb3c01979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:28:29.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.692+0000 7fb3c6800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3c01033c0 0x7fb3c0197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:28:29.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.692+0000 7fb3c6800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c0198530 con 0x7fb3c01021c0 2024-05-27T09:28:29.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.692+0000 7fb3c6800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c0198670 con 0x7fb3c01033c0 2024-05-27T09:28:29.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.692+0000 7fb3c5400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3c01021c0 0x7fb3c01979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:28:29.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.692+0000 7fb3c5400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3c01021c0 0x7fb3c01979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37052/0 (socket says 172.21.15.119:37052) 2024-05-27T09:28:29.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.692+0000 7fb3c5400700 1 -- 172.21.15.119:0/864662472 learned_addr learned my addr 172.21.15.119:0/864662472 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:28:29.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.692+0000 7fb3c4a00700 1 --2- 172.21.15.119:0/864662472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3c01033c0 0x7fb3c0197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:28:29.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.693+0000 7fb3c5400700 1 -- 172.21.15.119:0/864662472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3c01033c0 msgr2=0x7fb3c0197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:28:29.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.693+0000 7fb3c5400700 1 --2- 172.21.15.119:0/864662472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3c01033c0 0x7fb3c0197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:29.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.693+0000 7fb3c5400700 1 -- 172.21.15.119:0/864662472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b40095f0 con 0x7fb3c01021c0 2024-05-27T09:28:29.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.693+0000 7fb3c5400700 1 --2- 172.21.15.119:0/864662472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3c01021c0 0x7fb3c01979d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400f690 tx=0x7fb3b400f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:28:29.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.693+0000 7fb3be000700 1 -- 172.21.15.119:0/864662472 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3b401d070 con 0x7fb3c01021c0 2024-05-27T09:28:29.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.694+0000 7fb3be000700 1 -- 172.21.15.119:0/864662472 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb3b4005dc0 con 0x7fb3c01021c0 2024-05-27T09:28:29.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.694+0000 7fb3be000700 1 -- 172.21.15.119:0/864662472 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3b4017630 con 0x7fb3c01021c0 2024-05-27T09:28:29.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.694+0000 7fb3c6800700 1 -- 172.21.15.119:0/864662472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c019d0c0 con 0x7fb3c01021c0 2024-05-27T09:28:29.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.694+0000 7fb3c6800700 1 -- 172.21.15.119:0/864662472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c019d5b0 con 0x7fb3c01021c0 2024-05-27T09:28:29.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.695+0000 7fb3c6800700 1 -- 172.21.15.119:0/864662472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c010ad10 con 0x7fb3c01021c0 2024-05-27T09:28:29.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.697+0000 7fb3be000700 1 -- 172.21.15.119:0/864662472 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fb3b4003910 con 0x7fb3c01021c0 2024-05-27T09:28:29.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.698+0000 7fb3be000700 1 --2- 172.21.15.119:0/864662472 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fb3a806c680 0x7fb3a806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:28:29.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.698+0000 7fb3be000700 1 -- 172.21.15.119:0/864662472 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb3b408cea0 con 0x7fb3c01021c0 2024-05-27T09:28:29.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.698+0000 7fb3c4a00700 1 --2- 172.21.15.119:0/864662472 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fb3a806c680 0x7fb3a806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:28:29.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.699+0000 7fb3c4a00700 1 --2- 172.21.15.119:0/864662472 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fb3a806c680 0x7fb3a806eb30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009c60 tx=0x7fb3ac009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:28:29.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.705+0000 7fb3be000700 1 -- 172.21.15.119:0/864662472 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb3b405b010 con 0x7fb3c01021c0 2024-05-27T09:28:29.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:29 smithi138 ceph-mon[34885]: pgmap v300: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-05-27T09:28:29.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.961+0000 7fb3c6800700 1 -- 172.21.15.119:0/864662472 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3c0107d10 con 0x7fb3a806c680 2024-05-27T09:28:29.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.974+0000 7fb3be000700 1 -- 172.21.15.119:0/864662472 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3480 (secure 0 0 0) 0x7fb3c0107d10 con 0x7fb3a806c680 2024-05-27T09:28:29.975 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:28:29.975 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (7m) 119s ago 9m 13.1M - 0.25.0 c8568f914cd2 bc280230ab28 2024-05-27T09:28:29.975 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (9m) 119s ago 9m 7486k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:28:29.975 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (8m) 2m ago 8m 7394k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:28:29.975 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (9m) 119s ago 9m 6665k - 18.2.0 dc2bc1663786 e5fafde44183 2024-05-27T09:28:29.975 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (8m) 2m ago 8m 6660k - 18.2.0 dc2bc1663786 46956ab99bab 2024-05-27T09:28:29.975 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (7m) 119s ago 9m 74.6M - 9.4.7 954c08fa6188 16eeaded4295 2024-05-27T09:28:29.975 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (2m) 119s ago 2m 11.5M - 18.2.0 dc2bc1663786 93b21713fb1e 2024-05-27T09:28:29.975 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (2m) 119s ago 2m 14.4M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:28:29.975 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (2m) 2m ago 2m 11.5M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:28:29.976 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (2m) 2m ago 2m 14.4M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:28:29.976 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:9283,8765,8443 running (11m) 119s ago 11m 487M - 18.2.0 dc2bc1663786 abe5e83463f2 2024-05-27T09:28:29.976 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (8m) 2m ago 8m 437M - 18.2.0 dc2bc1663786 5aa669adddf7 2024-05-27T09:28:29.976 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (11m) 119s ago 11m 46.3M 2048M 18.2.0 dc2bc1663786 ba998b789a1c 2024-05-27T09:28:29.976 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (8m) 2m ago 8m 36.1M 2048M 18.2.0 dc2bc1663786 60216fd67078 2024-05-27T09:28:29.976 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (9m) 119s ago 9m 7788k - 1.5.0 0da6a335fe13 79c36d4e5b04 2024-05-27T09:28:29.976 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (8m) 2m ago 8m 7931k - 1.5.0 0da6a335fe13 80ee890e8830 2024-05-27T09:28:29.976 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (7m) 119s ago 7m 45.8M 3947M 18.2.0 dc2bc1663786 825bd1392508 2024-05-27T09:28:29.976 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (6m) 119s ago 6m 47.4M 3947M 18.2.0 dc2bc1663786 fce9819903d9 2024-05-27T09:28:29.976 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (6m) 119s ago 6m 46.2M 3947M 18.2.0 dc2bc1663786 3955a6112c1c 2024-05-27T09:28:29.976 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (5m) 2m ago 5m 45.1M 4971M 18.2.0 dc2bc1663786 693c92220622 2024-05-27T09:28:29.976 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (4m) 2m ago 4m 42.4M 4971M 18.2.0 dc2bc1663786 7559e658e917 2024-05-27T09:28:29.976 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (4m) 2m ago 4m 42.9M 4971M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:28:29.976 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (7m) 119s ago 9m 55.8M - 2.43.0 a07b618ecd1d 7f3d89d1f425 2024-05-27T09:28:29.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.978+0000 7fb3c6800700 1 -- 172.21.15.119:0/864662472 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fb3a806c680 msgr2=0x7fb3a806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:28:29.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.978+0000 7fb3c6800700 1 --2- 172.21.15.119:0/864662472 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fb3a806c680 0x7fb3a806eb30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009c60 tx=0x7fb3ac009260 comp rx=0 tx=0).stop 2024-05-27T09:28:29.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.978+0000 7fb3c6800700 1 -- 172.21.15.119:0/864662472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3c01021c0 msgr2=0x7fb3c01979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:28:29.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.978+0000 7fb3c6800700 1 --2- 172.21.15.119:0/864662472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3c01021c0 0x7fb3c01979d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400f690 tx=0x7fb3b400f6c0 comp rx=0 tx=0).stop 2024-05-27T09:28:29.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.978+0000 7fb3c6800700 1 -- 172.21.15.119:0/864662472 shutdown_connections 2024-05-27T09:28:29.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.978+0000 7fb3c6800700 1 --2- 172.21.15.119:0/864662472 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fb3a806c680 0x7fb3a806eb30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:29.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.978+0000 7fb3c6800700 1 --2- 172.21.15.119:0/864662472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3c01021c0 0x7fb3c01979d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:29.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.978+0000 7fb3c6800700 1 --2- 172.21.15.119:0/864662472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3c01033c0 0x7fb3c0197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:29.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.978+0000 7fb3c6800700 1 -- 172.21.15.119:0/864662472 >> 172.21.15.119:0/864662472 conn(0x7fb3c00fd790 msgr2=0x7fb3c01065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:28:29.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.979+0000 7fb3c6800700 1 -- 172.21.15.119:0/864662472 shutdown_connections 2024-05-27T09:28:29.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:29.979+0000 7fb3c6800700 1 -- 172.21.15.119:0/864662472 wait complete. 2024-05-27T09:28:30.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.206+0000 7fccfd800700 1 -- 172.21.15.119:0/32089339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccf8073750 msgr2=0x7fccf8073bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:28:30.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.206+0000 7fccfd800700 1 --2- 172.21.15.119:0/32089339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccf8073750 0x7fccf8073bc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009910 tx=0x7fcce4009c20 comp rx=0 tx=0).stop 2024-05-27T09:28:30.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.207+0000 7fccfd800700 1 -- 172.21.15.119:0/32089339 shutdown_connections 2024-05-27T09:28:30.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.207+0000 7fccfd800700 1 --2- 172.21.15.119:0/32089339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccf8073750 0x7fccf8073bc0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:30.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.207+0000 7fccfd800700 1 --2- 172.21.15.119:0/32089339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccf8074d20 0x7fccf8073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:30.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.207+0000 7fccfd800700 1 -- 172.21.15.119:0/32089339 >> 172.21.15.119:0/32089339 conn(0x7fccf80fb6e0 msgr2=0x7fccf80fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:28:30.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.207+0000 7fccfd800700 1 -- 172.21.15.119:0/32089339 shutdown_connections 2024-05-27T09:28:30.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.207+0000 7fccfd800700 1 -- 172.21.15.119:0/32089339 wait complete. 2024-05-27T09:28:30.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.208+0000 7fccfd800700 1 Processor -- start 2024-05-27T09:28:30.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.208+0000 7fccfd800700 1 -- start start 2024-05-27T09:28:30.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.209+0000 7fccfd800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccf8073750 0x7fccf8197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:28:30.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.209+0000 7fccfd800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccf8074d20 0x7fccf8198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:28:30.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.209+0000 7fccfd800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf81986b0 con 0x7fccf8074d20 2024-05-27T09:28:30.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.209+0000 7fccfd800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf81987f0 con 0x7fccf8073750 2024-05-27T09:28:30.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.209+0000 7fccf6a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccf8073750 0x7fccf8197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:28:30.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.209+0000 7fccf6000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccf8074d20 0x7fccf8198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:28:30.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.209+0000 7fccf6a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccf8073750 0x7fccf8197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.119:36940/0 (socket says 172.21.15.119:36940) 2024-05-27T09:28:30.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.209+0000 7fccf6a00700 1 -- 172.21.15.119:0/1242352211 learned_addr learned my addr 172.21.15.119:0/1242352211 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:28:30.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.210+0000 7fccf6000700 1 -- 172.21.15.119:0/1242352211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccf8073750 msgr2=0x7fccf8197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:28:30.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.210+0000 7fccf6000700 1 --2- 172.21.15.119:0/1242352211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccf8073750 0x7fccf8197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:30.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.210+0000 7fccf6000700 1 -- 172.21.15.119:0/1242352211 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce40095f0 con 0x7fccf8074d20 2024-05-27T09:28:30.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.210+0000 7fccf6000700 1 --2- 172.21.15.119:0/1242352211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccf8074d20 0x7fccf8198090 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcce4005b40 tx=0x7fcce400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:28:30.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.211+0000 7fccef400700 1 -- 172.21.15.119:0/1242352211 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcce401d070 con 0x7fccf8074d20 2024-05-27T09:28:30.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.211+0000 7fccef400700 1 -- 172.21.15.119:0/1242352211 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcce4005dc0 con 0x7fccf8074d20 2024-05-27T09:28:30.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.211+0000 7fccef400700 1 -- 172.21.15.119:0/1242352211 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcce4022470 con 0x7fccf8074d20 2024-05-27T09:28:30.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.211+0000 7fccfd800700 1 -- 172.21.15.119:0/1242352211 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf819d240 con 0x7fccf8074d20 2024-05-27T09:28:30.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.211+0000 7fccfd800700 1 -- 172.21.15.119:0/1242352211 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf819d6e0 con 0x7fccf8074d20 2024-05-27T09:28:30.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.213+0000 7fccfd800700 1 -- 172.21.15.119:0/1242352211 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf810ae90 con 0x7fccf8074d20 2024-05-27T09:28:30.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.213+0000 7fccef400700 1 -- 172.21.15.119:0/1242352211 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fcce4005160 con 0x7fccf8074d20 2024-05-27T09:28:30.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.215+0000 7fccef400700 1 --2- 172.21.15.119:0/1242352211 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fccdc06c680 0x7fccdc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:28:30.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.215+0000 7fccef400700 1 -- 172.21.15.119:0/1242352211 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fcce408cd90 con 0x7fccf8074d20 2024-05-27T09:28:30.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.216+0000 7fccf6a00700 1 --2- 172.21.15.119:0/1242352211 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fccdc06c680 0x7fccdc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:28:30.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.217+0000 7fccf6a00700 1 --2- 172.21.15.119:0/1242352211 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fccdc06c680 0x7fccdc06eb30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009d20 tx=0x7fcce8009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:28:30.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.220+0000 7fccef400700 1 -- 172.21.15.119:0/1242352211 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fcce405aea0 con 0x7fccf8074d20 2024-05-27T09:28:30.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:30 smithi119 ceph-mon[29082]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:28:30.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:30 smithi119 ceph-mon[29082]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:28:30.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.648+0000 7fccfd800700 1 -- 172.21.15.119:0/1242352211 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fccf819dd70 con 0x7fccf8074d20 2024-05-27T09:28:30.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.648+0000 7fccef400700 1 -- 172.21.15.119:0/1242352211 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7fcce402a350 con 0x7fccf8074d20 2024-05-27T09:28:30.649 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:28:30.649 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:28:30.649 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-05-27T09:28:30.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:28:30.649 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:28:30.650 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-05-27T09:28:30.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:28:30.650 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:28:30.650 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-05-27T09:28:30.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:28:30.650 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:28:30.650 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-05-27T09:28:30.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:28:30.650 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:28:30.650 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 14 2024-05-27T09:28:30.650 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:28:30.650 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:28:30.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.651+0000 7fccfd800700 1 -- 172.21.15.119:0/1242352211 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fccdc06c680 msgr2=0x7fccdc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:28:30.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.651+0000 7fccfd800700 1 --2- 172.21.15.119:0/1242352211 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fccdc06c680 0x7fccdc06eb30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009d20 tx=0x7fcce8009260 comp rx=0 tx=0).stop 2024-05-27T09:28:30.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.651+0000 7fccfd800700 1 -- 172.21.15.119:0/1242352211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccf8074d20 msgr2=0x7fccf8198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:28:30.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.651+0000 7fccfd800700 1 --2- 172.21.15.119:0/1242352211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccf8074d20 0x7fccf8198090 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcce4005b40 tx=0x7fcce400fa00 comp rx=0 tx=0).stop 2024-05-27T09:28:30.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.652+0000 7fccfd800700 1 -- 172.21.15.119:0/1242352211 shutdown_connections 2024-05-27T09:28:30.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.652+0000 7fccfd800700 1 --2- 172.21.15.119:0/1242352211 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fccdc06c680 0x7fccdc06eb30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:30.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.652+0000 7fccfd800700 1 --2- 172.21.15.119:0/1242352211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccf8073750 0x7fccf8197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:30.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.652+0000 7fccfd800700 1 --2- 172.21.15.119:0/1242352211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccf8074d20 0x7fccf8198090 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:30.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.652+0000 7fccfd800700 1 -- 172.21.15.119:0/1242352211 >> 172.21.15.119:0/1242352211 conn(0x7fccf80fb6e0 msgr2=0x7fccf8105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:28:30.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.652+0000 7fccfd800700 1 -- 172.21.15.119:0/1242352211 shutdown_connections 2024-05-27T09:28:30.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.652+0000 7fccfd800700 1 -- 172.21.15.119:0/1242352211 wait complete. 2024-05-27T09:28:30.816 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:30 smithi138 ceph-mon[34885]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:28:30.816 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:30 smithi138 ceph-mon[34885]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:28:30.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.843+0000 7fc14be00700 1 -- 172.21.15.119:0/2600990556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc14c103340 msgr2=0x7fc14c105720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:28:30.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.843+0000 7fc14be00700 1 --2- 172.21.15.119:0/2600990556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc14c103340 0x7fc14c105720 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc138009910 tx=0x7fc138009c20 comp rx=0 tx=0).stop 2024-05-27T09:28:30.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.843+0000 7fc14be00700 1 -- 172.21.15.119:0/2600990556 shutdown_connections 2024-05-27T09:28:30.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.843+0000 7fc14be00700 1 --2- 172.21.15.119:0/2600990556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc14c103340 0x7fc14c105720 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:30.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.843+0000 7fc14be00700 1 --2- 172.21.15.119:0/2600990556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc14c100a20 0x7fc14c102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:30.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.843+0000 7fc14be00700 1 -- 172.21.15.119:0/2600990556 >> 172.21.15.119:0/2600990556 conn(0x7fc14c0fa440 msgr2=0x7fc14c0fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:28:30.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.844+0000 7fc14be00700 1 -- 172.21.15.119:0/2600990556 shutdown_connections 2024-05-27T09:28:30.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.844+0000 7fc14be00700 1 -- 172.21.15.119:0/2600990556 wait complete. 2024-05-27T09:28:30.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.845+0000 7fc14be00700 1 Processor -- start 2024-05-27T09:28:30.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.845+0000 7fc14be00700 1 -- start start 2024-05-27T09:28:30.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.845+0000 7fc14be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc14c100a20 0x7fc14c197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:28:30.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.845+0000 7fc14be00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc14c103340 0x7fc14c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:28:30.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.845+0000 7fc14be00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c1984a0 con 0x7fc14c100a20 2024-05-27T09:28:30.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.845+0000 7fc14be00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c1985e0 con 0x7fc14c103340 2024-05-27T09:28:30.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.846+0000 7fc14aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc14c100a20 0x7fc14c197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:28:30.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.846+0000 7fc14aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc14c100a20 0x7fc14c197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37074/0 (socket says 172.21.15.119:37074) 2024-05-27T09:28:30.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.846+0000 7fc14aa00700 1 -- 172.21.15.119:0/2298231267 learned_addr learned my addr 172.21.15.119:0/2298231267 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:28:30.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.846+0000 7fc14a000700 1 --2- 172.21.15.119:0/2298231267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc14c103340 0x7fc14c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:28:30.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.846+0000 7fc14aa00700 1 -- 172.21.15.119:0/2298231267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc14c103340 msgr2=0x7fc14c197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:28:30.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.846+0000 7fc14aa00700 1 --2- 172.21.15.119:0/2298231267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc14c103340 0x7fc14c197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:30.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.846+0000 7fc14aa00700 1 -- 172.21.15.119:0/2298231267 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1380095f0 con 0x7fc14c100a20 2024-05-27T09:28:30.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.846+0000 7fc14aa00700 1 --2- 172.21.15.119:0/2298231267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc14c100a20 0x7fc14c197940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00d8d0 tx=0x7fc13c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:28:30.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.847+0000 7fc143400700 1 -- 172.21.15.119:0/2298231267 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc13c0098a0 con 0x7fc14c100a20 2024-05-27T09:28:30.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.847+0000 7fc143400700 1 -- 172.21.15.119:0/2298231267 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc13c010460 con 0x7fc14c100a20 2024-05-27T09:28:30.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.847+0000 7fc14be00700 1 -- 172.21.15.119:0/2298231267 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc14c19d090 con 0x7fc14c100a20 2024-05-27T09:28:30.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.847+0000 7fc14be00700 1 -- 172.21.15.119:0/2298231267 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14c19d5e0 con 0x7fc14c100a20 2024-05-27T09:28:30.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.847+0000 7fc143400700 1 -- 172.21.15.119:0/2298231267 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc13c00f5f0 con 0x7fc14c100a20 2024-05-27T09:28:30.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.848+0000 7fc14be00700 1 -- 172.21.15.119:0/2298231267 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc12c005320 con 0x7fc14c100a20 2024-05-27T09:28:30.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.850+0000 7fc143400700 1 -- 172.21.15.119:0/2298231267 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fc13c009a00 con 0x7fc14c100a20 2024-05-27T09:28:30.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.851+0000 7fc143400700 1 --2- 172.21.15.119:0/2298231267 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc13006c680 0x7fc13006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:28:30.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.851+0000 7fc143400700 1 -- 172.21.15.119:0/2298231267 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc13c08b700 con 0x7fc14c100a20 2024-05-27T09:28:30.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.851+0000 7fc14a000700 1 --2- 172.21.15.119:0/2298231267 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc13006c680 0x7fc13006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:28:30.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.852+0000 7fc14a000700 1 --2- 172.21.15.119:0/2298231267 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc13006c680 0x7fc13006eb30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc138006010 tx=0x7fc13800b560 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:28:30.857 INFO:tasks.workunit.client.1.smithi138.stderr:Updating files: 50% (6214/12271) Updating files: 51% (6259/12271) Updating files: 52% (6381/12271) Updating files: 53% (6504/12271) Updating files: 54% (6627/12271) Updating files: 55% (6750/12271) Updating files: 56% (6872/12271) Updating files: 57% (6995/12271) Updating files: 58% (7118/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 80% (9895/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-27T09:28:30.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:30.859+0000 7fc143400700 1 -- 172.21.15.119:0/2298231267 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc13c059890 con 0x7fc14c100a20 2024-05-27T09:28:31.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:31.179+0000 7fc14be00700 1 -- 172.21.15.119:0/2298231267 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc12c000bf0 con 0x7fc13006c680 2024-05-27T09:28:31.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:31.181+0000 7fc143400700 1 -- 172.21.15.119:0/2298231267 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fc12c000bf0 con 0x7fc13006c680 2024-05-27T09:28:31.181 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:28:31.181 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:28:31.181 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:28:31.181 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-27T09:28:31.181 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [], 2024-05-27T09:28:31.182 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "0/2 daemons upgraded", 2024-05-27T09:28:31.182 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 image on host smithi138", 2024-05-27T09:28:31.182 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:28:31.182 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:28:31.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:31.183+0000 7fc14be00700 1 -- 172.21.15.119:0/2298231267 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc13006c680 msgr2=0x7fc13006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:28:31.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:31.183+0000 7fc14be00700 1 --2- 172.21.15.119:0/2298231267 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc13006c680 0x7fc13006eb30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc138006010 tx=0x7fc13800b560 comp rx=0 tx=0).stop 2024-05-27T09:28:31.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:31.183+0000 7fc14be00700 1 -- 172.21.15.119:0/2298231267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc14c100a20 msgr2=0x7fc14c197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:28:31.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:31.183+0000 7fc14be00700 1 --2- 172.21.15.119:0/2298231267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc14c100a20 0x7fc14c197940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00d8d0 tx=0x7fc13c00dbe0 comp rx=0 tx=0).stop 2024-05-27T09:28:31.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:31.184+0000 7fc14be00700 1 -- 172.21.15.119:0/2298231267 shutdown_connections 2024-05-27T09:28:31.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:31.184+0000 7fc14be00700 1 --2- 172.21.15.119:0/2298231267 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fc13006c680 0x7fc13006eb30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:31.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:31.184+0000 7fc14be00700 1 --2- 172.21.15.119:0/2298231267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc14c100a20 0x7fc14c197940 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:31.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:31.184+0000 7fc14be00700 1 --2- 172.21.15.119:0/2298231267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc14c103340 0x7fc14c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:28:31.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:31.184+0000 7fc14be00700 1 -- 172.21.15.119:0/2298231267 >> 172.21.15.119:0/2298231267 conn(0x7fc14c0fa440 msgr2=0x7fc14c0ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:28:31.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:31.184+0000 7fc14be00700 1 -- 172.21.15.119:0/2298231267 shutdown_connections 2024-05-27T09:28:31.185 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:28:31.185+0000 7fc14be00700 1 -- 172.21.15.119:0/2298231267 wait complete. 2024-05-27T09:28:31.733 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:31 smithi119 ceph-mon[29082]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:28:31.734 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:31 smithi119 ceph-mon[29082]: pgmap v301: 65 pgs: 65 active+clean; 4.6 MiB data, 165 MiB used, 536 GiB / 536 GiB avail; 341 KiB/s wr, 0 op/s 2024-05-27T09:28:31.734 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:31 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/1242352211' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:28:31.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:31 smithi138 ceph-mon[34885]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:28:31.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:31 smithi138 ceph-mon[34885]: pgmap v301: 65 pgs: 65 active+clean; 4.6 MiB data, 165 MiB used, 536 GiB / 536 GiB avail; 341 KiB/s wr, 0 op/s 2024-05-27T09:28:31.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:31 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/1242352211' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:28:31.929 INFO:tasks.workunit.client.1.smithi138.stderr:Note: switching to 'a6507388dd5057528934822c0163b0c347ef1d5d'. 2024-05-27T09:28:31.929 INFO:tasks.workunit.client.1.smithi138.stderr: 2024-05-27T09:28:31.929 INFO:tasks.workunit.client.1.smithi138.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-27T09:28:31.929 INFO:tasks.workunit.client.1.smithi138.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-27T09:28:31.929 INFO:tasks.workunit.client.1.smithi138.stderr:state without impacting any branches by switching back to a branch. 2024-05-27T09:28:31.929 INFO:tasks.workunit.client.1.smithi138.stderr: 2024-05-27T09:28:31.929 INFO:tasks.workunit.client.1.smithi138.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-27T09:28:31.929 INFO:tasks.workunit.client.1.smithi138.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-27T09:28:31.930 INFO:tasks.workunit.client.1.smithi138.stderr: 2024-05-27T09:28:31.930 INFO:tasks.workunit.client.1.smithi138.stderr: git switch -c 2024-05-27T09:28:31.930 INFO:tasks.workunit.client.1.smithi138.stderr: 2024-05-27T09:28:31.930 INFO:tasks.workunit.client.1.smithi138.stderr:Or undo this operation with: 2024-05-27T09:28:31.930 INFO:tasks.workunit.client.1.smithi138.stderr: 2024-05-27T09:28:31.930 INFO:tasks.workunit.client.1.smithi138.stderr: git switch - 2024-05-27T09:28:31.930 INFO:tasks.workunit.client.1.smithi138.stderr: 2024-05-27T09:28:31.930 INFO:tasks.workunit.client.1.smithi138.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-27T09:28:31.930 INFO:tasks.workunit.client.1.smithi138.stderr: 2024-05-27T09:28:31.930 INFO:tasks.workunit.client.1.smithi138.stderr:HEAD is now at a6507388dd5 Merge PR #53755 into wip-vshankar-testing-20240521.105740-debug 2024-05-27T09:28:31.946 DEBUG:teuthology.orchestra.run.smithi138:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-05-27T09:28:32.000 INFO:tasks.workunit.client.1.smithi138.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-27T09:28:32.006 INFO:tasks.workunit.client.1.smithi138.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-05-27T09:28:32.006 INFO:tasks.workunit.client.1.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-27T09:28:32.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:32 smithi138 ceph-mon[34885]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:28:32.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:32 smithi119 ceph-mon[29082]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:28:33.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:33 smithi119 ceph-mon[29082]: pgmap v302: 65 pgs: 65 active+clean; 4.6 MiB data, 169 MiB used, 536 GiB / 536 GiB avail; 341 KiB/s wr, 0 op/s 2024-05-27T09:28:33.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:33 smithi138 ceph-mon[34885]: pgmap v302: 65 pgs: 65 active+clean; 4.6 MiB data, 169 MiB used, 536 GiB / 536 GiB avail; 341 KiB/s wr, 0 op/s 2024-05-27T09:28:33.949 INFO:tasks.workunit.client.1.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-27T09:28:34.072 INFO:tasks.workunit.client.1.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-27T09:28:34.153 INFO:tasks.workunit.client.1.smithi138.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-05-27T09:28:34.158 INFO:tasks.workunit.client.1.smithi138.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-05-27T09:28:34.158 INFO:tasks.workunit.client.1.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-27T09:28:34.232 INFO:tasks.workunit.client.1.smithi138.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-05-27T09:28:34.243 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-27T09:28:34.243 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-05-27T09:28:34.284 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-05-27T09:28:34.285 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-05-27T09:28:34.285 DEBUG:teuthology.orchestra.run.smithi138:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6507388dd5057528934822c0163b0c347ef1d5d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-05-27T09:28:34.391 INFO:tasks.workunit.client.1.smithi138.stderr:+ mkdir -p fsstress 2024-05-27T09:28:34.396 INFO:tasks.workunit.client.1.smithi138.stderr:+ pushd fsstress 2024-05-27T09:28:34.396 INFO:tasks.workunit.client.1.smithi138.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-27T09:28:34.396 INFO:tasks.workunit.client.1.smithi138.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-05-27T09:28:35.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:35 smithi119 ceph-mon[29082]: Upgrade: Updating mgr.smithi138.dxsuqf 2024-05-27T09:28:35.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:35 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:28:35.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:35 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.dxsuqf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T09:28:35.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:35 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T09:28:35.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:35 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:28:35.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:35 smithi119 ceph-mon[29082]: Deploying daemon mgr.smithi138.dxsuqf on smithi138 2024-05-27T09:28:35.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:35 smithi119 ceph-mon[29082]: pgmap v303: 65 pgs: 65 active+clean; 19 MiB data, 214 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s wr, 19 op/s 2024-05-27T09:28:35.605 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:35 smithi138 ceph-mon[34885]: Upgrade: Updating mgr.smithi138.dxsuqf 2024-05-27T09:28:35.606 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:35 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:28:35.606 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:35 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.dxsuqf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T09:28:35.606 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:35 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T09:28:35.606 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:35 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:28:35.606 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:35 smithi138 ceph-mon[34885]: Deploying daemon mgr.smithi138.dxsuqf on smithi138 2024-05-27T09:28:35.606 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:35 smithi138 ceph-mon[34885]: pgmap v303: 65 pgs: 65 active+clean; 19 MiB data, 214 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s wr, 19 op/s 2024-05-27T09:28:36.484 INFO:tasks.workunit.client.1.smithi138.stderr:+ tar xzf ltp-full.tgz 2024-05-27T09:28:37.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:37 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:28:37.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:37 smithi119 ceph-mon[29082]: pgmap v304: 65 pgs: 65 active+clean; 25 MiB data, 237 MiB used, 536 GiB / 536 GiB avail; 2.1 MiB/s wr, 24 op/s 2024-05-27T09:28:37.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:37 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:28:37.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:37 smithi138 ceph-mon[34885]: pgmap v304: 65 pgs: 65 active+clean; 25 MiB data, 237 MiB used, 536 GiB / 536 GiB avail; 2.1 MiB/s wr, 24 op/s 2024-05-27T09:28:39.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:39 smithi119 ceph-mon[29082]: pgmap v305: 65 pgs: 65 active+clean; 26 MiB data, 245 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s wr, 54 op/s 2024-05-27T09:28:39.890 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:39 smithi138 ceph-mon[34885]: pgmap v305: 65 pgs: 65 active+clean; 26 MiB data, 245 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s wr, 54 op/s 2024-05-27T09:28:41.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:41 smithi119 ceph-mon[29082]: pgmap v306: 65 pgs: 65 active+clean; 36 MiB data, 270 MiB used, 536 GiB / 536 GiB avail; 3.1 MiB/s wr, 140 op/s 2024-05-27T09:28:41.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:41 smithi138 ceph-mon[34885]: pgmap v306: 65 pgs: 65 active+clean; 36 MiB data, 270 MiB used, 536 GiB / 536 GiB avail; 3.1 MiB/s wr, 140 op/s 2024-05-27T09:28:42.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:42 smithi119 ceph-mon[29082]: fsmap cephfs:2 {0=cephfs.smithi119.ppmzzk=up:active,1=cephfs.smithi138.ndfkhd=up:active} 2 up:standby 2024-05-27T09:28:42.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:42 smithi138 ceph-mon[34885]: fsmap cephfs:2 {0=cephfs.smithi119.ppmzzk=up:active,1=cephfs.smithi138.ndfkhd=up:active} 2 up:standby 2024-05-27T09:28:43.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:43 smithi119 ceph-mon[29082]: pgmap v307: 65 pgs: 65 active+clean; 37 MiB data, 278 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s wr, 155 op/s 2024-05-27T09:28:43.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:43 smithi138 ceph-mon[34885]: pgmap v307: 65 pgs: 65 active+clean; 37 MiB data, 278 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s wr, 155 op/s 2024-05-27T09:28:45.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:45 smithi119 ceph-mon[29082]: pgmap v308: 65 pgs: 65 active+clean; 53 MiB data, 333 MiB used, 536 GiB / 536 GiB avail; 4.1 MiB/s wr, 190 op/s 2024-05-27T09:28:45.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:45 smithi138 ceph-mon[34885]: pgmap v308: 65 pgs: 65 active+clean; 53 MiB data, 333 MiB used, 536 GiB / 536 GiB avail; 4.1 MiB/s wr, 190 op/s 2024-05-27T09:28:47.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:47 smithi138 ceph-mon[34885]: pgmap v309: 65 pgs: 65 active+clean; 58 MiB data, 349 MiB used, 536 GiB / 536 GiB avail; 3.5 MiB/s wr, 185 op/s 2024-05-27T09:28:47.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:47 smithi119 ceph-mon[29082]: pgmap v309: 65 pgs: 65 active+clean; 58 MiB data, 349 MiB used, 536 GiB / 536 GiB avail; 3.5 MiB/s wr, 185 op/s 2024-05-27T09:28:49.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:49 smithi119 ceph-mon[29082]: pgmap v310: 65 pgs: 65 active+clean; 62 MiB data, 363 MiB used, 536 GiB / 536 GiB avail; 3.2 MiB/s wr, 233 op/s 2024-05-27T09:28:49.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:49 smithi138 ceph-mon[34885]: pgmap v310: 65 pgs: 65 active+clean; 62 MiB data, 363 MiB used, 536 GiB / 536 GiB avail; 3.2 MiB/s wr, 233 op/s 2024-05-27T09:28:51.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:51 smithi119 ceph-mon[29082]: pgmap v311: 65 pgs: 65 active+clean; 81 MiB data, 432 MiB used, 536 GiB / 536 GiB avail; 4.8 MiB/s wr, 358 op/s 2024-05-27T09:28:51.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:51 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:28:51.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:51 smithi138 ceph-mon[34885]: pgmap v311: 65 pgs: 65 active+clean; 81 MiB data, 432 MiB used, 536 GiB / 536 GiB avail; 4.8 MiB/s wr, 358 op/s 2024-05-27T09:28:51.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:51 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:28:53.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:53 smithi138 ceph-mon[34885]: pgmap v312: 65 pgs: 65 active+clean; 83 MiB data, 446 MiB used, 536 GiB / 536 GiB avail; 4.1 MiB/s wr, 294 op/s 2024-05-27T09:28:53.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:53 smithi119 ceph-mon[29082]: pgmap v312: 65 pgs: 65 active+clean; 83 MiB data, 446 MiB used, 536 GiB / 536 GiB avail; 4.1 MiB/s wr, 294 op/s 2024-05-27T09:28:55.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:55 smithi138 ceph-mon[34885]: pgmap v313: 65 pgs: 65 active+clean; 103 MiB data, 525 MiB used, 536 GiB / 536 GiB avail; 5.8 MiB/s wr, 465 op/s 2024-05-27T09:28:55.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:55 smithi119 ceph-mon[29082]: pgmap v313: 65 pgs: 65 active+clean; 103 MiB data, 525 MiB used, 536 GiB / 536 GiB avail; 5.8 MiB/s wr, 465 op/s 2024-05-27T09:28:56.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:56 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:28:56.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:56 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:28:56.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:56 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:28:56.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:56 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:28:56.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:56 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:28:56.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:56 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:28:57.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:57 smithi138 ceph-mon[34885]: pgmap v314: 65 pgs: 65 active+clean; 108 MiB data, 552 MiB used, 536 GiB / 536 GiB avail; 4.9 MiB/s wr, 501 op/s 2024-05-27T09:28:57.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:57 smithi119 ceph-mon[29082]: pgmap v314: 65 pgs: 65 active+clean; 108 MiB data, 552 MiB used, 536 GiB / 536 GiB avail; 4.9 MiB/s wr, 501 op/s 2024-05-27T09:28:59.689 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:28:59 smithi138 ceph-mon[34885]: pgmap v315: 65 pgs: 65 active+clean; 112 MiB data, 561 MiB used, 536 GiB / 536 GiB avail; 4.8 MiB/s wr, 504 op/s 2024-05-27T09:28:59.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:28:59 smithi119 ceph-mon[29082]: pgmap v315: 65 pgs: 65 active+clean; 112 MiB data, 561 MiB used, 536 GiB / 536 GiB avail; 4.8 MiB/s wr, 504 op/s 2024-05-27T09:29:01.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.377+0000 7f4dca800700 1 -- 172.21.15.119:0/2034334050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4dc41065b0 msgr2=0x7f4dc41069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:01.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.377+0000 7f4dca800700 1 --2- 172.21.15.119:0/2034334050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4dc41065b0 0x7f4dc41069c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009910 tx=0x7f4db8009c20 comp rx=0 tx=0).stop 2024-05-27T09:29:01.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.377+0000 7f4dca800700 1 -- 172.21.15.119:0/2034334050 shutdown_connections 2024-05-27T09:29:01.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.377+0000 7f4dca800700 1 --2- 172.21.15.119:0/2034334050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dc41077b0 0x7f4dc4107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:01.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.377+0000 7f4dca800700 1 --2- 172.21.15.119:0/2034334050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4dc41065b0 0x7f4dc41069c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:01.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.377+0000 7f4dca800700 1 -- 172.21.15.119:0/2034334050 >> 172.21.15.119:0/2034334050 conn(0x7f4dc4075740 msgr2=0x7f4dc4077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:01.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.377+0000 7f4dca800700 1 -- 172.21.15.119:0/2034334050 shutdown_connections 2024-05-27T09:29:01.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.378+0000 7f4dca800700 1 -- 172.21.15.119:0/2034334050 wait complete. 2024-05-27T09:29:01.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.379+0000 7f4dca800700 1 Processor -- start 2024-05-27T09:29:01.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.379+0000 7f4dca800700 1 -- start start 2024-05-27T09:29:01.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.379+0000 7f4dca800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dc41065b0 0x7f4dc419bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:01.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.379+0000 7f4dca800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4dc41077b0 0x7f4dc419c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.379+0000 7f4dca800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc419c920 con 0x7f4dc41077b0 2024-05-27T09:29:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.379+0000 7f4dca800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc419ca60 con 0x7f4dc41065b0 2024-05-27T09:29:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.380+0000 7f4dc8a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4dc41077b0 0x7f4dc419c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.380+0000 7f4dc8a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4dc41077b0 0x7f4dc419c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50836/0 (socket says 172.21.15.119:50836) 2024-05-27T09:29:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.380+0000 7f4dc8a00700 1 -- 172.21.15.119:0/28646530 learned_addr learned my addr 172.21.15.119:0/28646530 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:29:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.380+0000 7f4dc9400700 1 --2- 172.21.15.119:0/28646530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dc41065b0 0x7f4dc419bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.380+0000 7f4dc8a00700 1 -- 172.21.15.119:0/28646530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dc41065b0 msgr2=0x7f4dc419bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.380+0000 7f4dc8a00700 1 --2- 172.21.15.119:0/28646530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dc41065b0 0x7f4dc419bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.380+0000 7f4dc8a00700 1 -- 172.21.15.119:0/28646530 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db80095f0 con 0x7f4dc41077b0 2024-05-27T09:29:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.381+0000 7f4dc8a00700 1 --2- 172.21.15.119:0/28646530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4dc41077b0 0x7f4dc419c300 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4db000e920 tx=0x7f4db000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.381+0000 7f4dc2000700 1 -- 172.21.15.119:0/28646530 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4db000c9d0 con 0x7f4dc41077b0 2024-05-27T09:29:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.381+0000 7f4dc2000700 1 -- 172.21.15.119:0/28646530 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4db0010430 con 0x7f4dc41077b0 2024-05-27T09:29:01.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.381+0000 7f4dc2000700 1 -- 172.21.15.119:0/28646530 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4db0018650 con 0x7f4dc41077b0 2024-05-27T09:29:01.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.381+0000 7f4dca800700 1 -- 172.21.15.119:0/28646530 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc41a1510 con 0x7f4dc41077b0 2024-05-27T09:29:01.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.381+0000 7f4dca800700 1 -- 172.21.15.119:0/28646530 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc41a1a60 con 0x7f4dc41077b0 2024-05-27T09:29:01.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.382+0000 7f4dca800700 1 -- 172.21.15.119:0/28646530 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc410f100 con 0x7f4dc41077b0 2024-05-27T09:29:01.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.383+0000 7f4dc2000700 1 -- 172.21.15.119:0/28646530 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f4db00105a0 con 0x7f4dc41077b0 2024-05-27T09:29:01.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.384+0000 7f4dc2000700 1 --2- 172.21.15.119:0/28646530 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4dac06c680 0x7f4dac06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:01.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.384+0000 7f4dc2000700 1 -- 172.21.15.119:0/28646530 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4db0014070 con 0x7f4dc41077b0 2024-05-27T09:29:01.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.385+0000 7f4dc9400700 1 --2- 172.21.15.119:0/28646530 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4dac06c680 0x7f4dac06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:01.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.385+0000 7f4dc9400700 1 --2- 172.21.15.119:0/28646530 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4dac06c680 0x7f4dac06eb30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4db800b6f0 tx=0x7f4db8005940 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:01.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.389+0000 7f4dc2000700 1 -- 172.21.15.119:0/28646530 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4db0051720 con 0x7f4dc41077b0 2024-05-27T09:29:01.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.756+0000 7f4dca800700 1 -- 172.21.15.119:0/28646530 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4dc410c100 con 0x7f4dac06c680 2024-05-27T09:29:01.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.759+0000 7f4dc2000700 1 -- 172.21.15.119:0/28646530 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f4dc410c100 con 0x7f4dac06c680 2024-05-27T09:29:01.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.765+0000 7f4dca800700 1 -- 172.21.15.119:0/28646530 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4dac06c680 msgr2=0x7f4dac06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:01.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.765+0000 7f4dca800700 1 --2- 172.21.15.119:0/28646530 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4dac06c680 0x7f4dac06eb30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4db800b6f0 tx=0x7f4db8005940 comp rx=0 tx=0).stop 2024-05-27T09:29:01.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.765+0000 7f4dca800700 1 -- 172.21.15.119:0/28646530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4dc41077b0 msgr2=0x7f4dc419c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:01.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.765+0000 7f4dca800700 1 --2- 172.21.15.119:0/28646530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4dc41077b0 0x7f4dc419c300 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4db000e920 tx=0x7f4db000ec30 comp rx=0 tx=0).stop 2024-05-27T09:29:01.767 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.766+0000 7f4dca800700 1 -- 172.21.15.119:0/28646530 shutdown_connections 2024-05-27T09:29:01.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.766+0000 7f4dca800700 1 --2- 172.21.15.119:0/28646530 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f4dac06c680 0x7f4dac06eb30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:01.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.766+0000 7f4dca800700 1 --2- 172.21.15.119:0/28646530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dc41065b0 0x7f4dc419bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:01.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.766+0000 7f4dca800700 1 --2- 172.21.15.119:0/28646530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4dc41077b0 0x7f4dc419c300 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:01.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.766+0000 7f4dca800700 1 -- 172.21.15.119:0/28646530 >> 172.21.15.119:0/28646530 conn(0x7f4dc4075740 msgr2=0x7f4dc410a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:01.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.770+0000 7f4dca800700 1 -- 172.21.15.119:0/28646530 shutdown_connections 2024-05-27T09:29:01.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:01.770+0000 7f4dca800700 1 -- 172.21.15.119:0/28646530 wait complete. 2024-05-27T09:29:01.805 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:29:01.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:01 smithi138 ceph-mon[34885]: pgmap v316: 65 pgs: 65 active+clean; 127 MiB data, 618 MiB used, 536 GiB / 536 GiB avail; 5.8 MiB/s wr, 507 op/s 2024-05-27T09:29:01.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:01 smithi119 ceph-mon[29082]: pgmap v316: 65 pgs: 65 active+clean; 127 MiB data, 618 MiB used, 536 GiB / 536 GiB avail; 5.8 MiB/s wr, 507 op/s 2024-05-27T09:29:02.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.052+0000 7fbf50200700 1 -- 172.21.15.119:0/3222448373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf480fe9d0 msgr2=0x7fbf480fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:02.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.052+0000 7fbf50200700 1 --2- 172.21.15.119:0/3222448373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf480fe9d0 0x7fbf480fede0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009870 tx=0x7fbf3c009b80 comp rx=0 tx=0).stop 2024-05-27T09:29:02.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.052+0000 7fbf50200700 1 -- 172.21.15.119:0/3222448373 shutdown_connections 2024-05-27T09:29:02.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.052+0000 7fbf50200700 1 --2- 172.21.15.119:0/3222448373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf480ff320 0x7fbf480ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:02.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.052+0000 7fbf50200700 1 --2- 172.21.15.119:0/3222448373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf480fe9d0 0x7fbf480fede0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:02.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.052+0000 7fbf50200700 1 -- 172.21.15.119:0/3222448373 >> 172.21.15.119:0/3222448373 conn(0x7fbf480fa5e0 msgr2=0x7fbf480fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:02.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.053+0000 7fbf50200700 1 -- 172.21.15.119:0/3222448373 shutdown_connections 2024-05-27T09:29:02.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.053+0000 7fbf50200700 1 -- 172.21.15.119:0/3222448373 wait complete. 2024-05-27T09:29:02.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.056+0000 7fbf50200700 1 Processor -- start 2024-05-27T09:29:02.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.056+0000 7fbf50200700 1 -- start start 2024-05-27T09:29:02.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.057+0000 7fbf50200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf480fe9d0 0x7fbf4819bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:02.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.057+0000 7fbf50200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf480ff320 0x7fbf4819c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:02.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.057+0000 7fbf50200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4819cac0 con 0x7fbf480fe9d0 2024-05-27T09:29:02.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.057+0000 7fbf50200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4819cc00 con 0x7fbf480ff320 2024-05-27T09:29:02.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.058+0000 7fbf4d800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf480fe9d0 0x7fbf4819bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:02.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.058+0000 7fbf4d800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf480fe9d0 0x7fbf4819bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50846/0 (socket says 172.21.15.119:50846) 2024-05-27T09:29:02.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.058+0000 7fbf4d800700 1 -- 172.21.15.119:0/1869834735 learned_addr learned my addr 172.21.15.119:0/1869834735 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:29:02.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.058+0000 7fbf4ce00700 1 --2- 172.21.15.119:0/1869834735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf480ff320 0x7fbf4819c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:02.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.058+0000 7fbf4d800700 1 -- 172.21.15.119:0/1869834735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf480ff320 msgr2=0x7fbf4819c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:02.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.058+0000 7fbf4d800700 1 --2- 172.21.15.119:0/1869834735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf480ff320 0x7fbf4819c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:02.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.058+0000 7fbf4d800700 1 -- 172.21.15.119:0/1869834735 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf340095f0 con 0x7fbf480fe9d0 2024-05-27T09:29:02.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.059+0000 7fbf4d800700 1 --2- 172.21.15.119:0/1869834735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf480fe9d0 0x7fbf4819bf60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009f50 tx=0x7fbf3c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:02.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.060+0000 7fbf46000700 1 -- 172.21.15.119:0/1869834735 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf3c01c070 con 0x7fbf480fe9d0 2024-05-27T09:29:02.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.060+0000 7fbf46000700 1 -- 172.21.15.119:0/1869834735 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbf3c00be80 con 0x7fbf480fe9d0 2024-05-27T09:29:02.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.060+0000 7fbf50200700 1 -- 172.21.15.119:0/1869834735 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3c009520 con 0x7fbf480fe9d0 2024-05-27T09:29:02.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.060+0000 7fbf50200700 1 -- 172.21.15.119:0/1869834735 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf48190ae0 con 0x7fbf480fe9d0 2024-05-27T09:29:02.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.060+0000 7fbf46000700 1 -- 172.21.15.119:0/1869834735 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf3c0173f0 con 0x7fbf480fe9d0 2024-05-27T09:29:02.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.063+0000 7fbf3b400700 1 -- 172.21.15.119:0/1869834735 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf240052f0 con 0x7fbf480fe9d0 2024-05-27T09:29:02.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.064+0000 7fbf46000700 1 -- 172.21.15.119:0/1869834735 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fbf3c020070 con 0x7fbf480fe9d0 2024-05-27T09:29:02.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.066+0000 7fbf46000700 1 --2- 172.21.15.119:0/1869834735 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbf3006c680 0x7fbf3006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:02.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.066+0000 7fbf46000700 1 -- 172.21.15.119:0/1869834735 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fbf3c05ef20 con 0x7fbf480fe9d0 2024-05-27T09:29:02.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.067+0000 7fbf4ce00700 1 --2- 172.21.15.119:0/1869834735 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbf3006c680 0x7fbf3006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:02.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.068+0000 7fbf4ce00700 1 --2- 172.21.15.119:0/1869834735 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbf3006c680 0x7fbf3006eb30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbf34005fd0 tx=0x7fbf34009310 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:02.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.077+0000 7fbf46000700 1 -- 172.21.15.119:0/1869834735 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbf3c0579c0 con 0x7fbf480fe9d0 2024-05-27T09:29:02.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.520+0000 7fbf3b400700 1 -- 172.21.15.119:0/1869834735 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf24000bc0 con 0x7fbf3006c680 2024-05-27T09:29:02.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.523+0000 7fbf46000700 1 -- 172.21.15.119:0/1869834735 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fbf24000bc0 con 0x7fbf3006c680 2024-05-27T09:29:02.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.530+0000 7fbf3b400700 1 -- 172.21.15.119:0/1869834735 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbf3006c680 msgr2=0x7fbf3006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:02.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.530+0000 7fbf3b400700 1 --2- 172.21.15.119:0/1869834735 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbf3006c680 0x7fbf3006eb30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbf34005fd0 tx=0x7fbf34009310 comp rx=0 tx=0).stop 2024-05-27T09:29:02.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.530+0000 7fbf3b400700 1 -- 172.21.15.119:0/1869834735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf480fe9d0 msgr2=0x7fbf4819bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:02.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.530+0000 7fbf3b400700 1 --2- 172.21.15.119:0/1869834735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf480fe9d0 0x7fbf4819bf60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009f50 tx=0x7fbf3c00f740 comp rx=0 tx=0).stop 2024-05-27T09:29:02.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.530+0000 7fbf3b400700 1 -- 172.21.15.119:0/1869834735 shutdown_connections 2024-05-27T09:29:02.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.530+0000 7fbf3b400700 1 --2- 172.21.15.119:0/1869834735 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fbf3006c680 0x7fbf3006eb30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:02.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.530+0000 7fbf3b400700 1 --2- 172.21.15.119:0/1869834735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf480fe9d0 0x7fbf4819bf60 secure :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009f50 tx=0x7fbf3c00f740 comp rx=0 tx=0).stop 2024-05-27T09:29:02.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.530+0000 7fbf3b400700 1 --2- 172.21.15.119:0/1869834735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf480ff320 0x7fbf4819c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:02.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.530+0000 7fbf3b400700 1 -- 172.21.15.119:0/1869834735 >> 172.21.15.119:0/1869834735 conn(0x7fbf480fa5e0 msgr2=0x7fbf48102bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:02.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.532+0000 7fbf3b400700 1 -- 172.21.15.119:0/1869834735 shutdown_connections 2024-05-27T09:29:02.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.533+0000 7fbf3b400700 1 -- 172.21.15.119:0/1869834735 wait complete. 2024-05-27T09:29:02.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:02 smithi119 ceph-mon[29082]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:29:02.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.784+0000 7fb02a000700 1 -- 172.21.15.119:0/2381885794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb02410c1c0 msgr2=0x7fb02410c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:02.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.784+0000 7fb02a000700 1 --2- 172.21.15.119:0/2381885794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb02410c1c0 0x7fb02410c5d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb014009910 tx=0x7fb014009c20 comp rx=0 tx=0).stop 2024-05-27T09:29:02.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.785+0000 7fb02a000700 1 -- 172.21.15.119:0/2381885794 shutdown_connections 2024-05-27T09:29:02.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.785+0000 7fb02a000700 1 --2- 172.21.15.119:0/2381885794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb02410ad70 0x7fb02410b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:02.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.785+0000 7fb02a000700 1 --2- 172.21.15.119:0/2381885794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb02410c1c0 0x7fb02410c5d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:02.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.785+0000 7fb02a000700 1 -- 172.21.15.119:0/2381885794 >> 172.21.15.119:0/2381885794 conn(0x7fb02406a630 msgr2=0x7fb02406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:02.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.785+0000 7fb02a000700 1 -- 172.21.15.119:0/2381885794 shutdown_connections 2024-05-27T09:29:02.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.785+0000 7fb02a000700 1 -- 172.21.15.119:0/2381885794 wait complete. 2024-05-27T09:29:02.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.786+0000 7fb02a000700 1 Processor -- start 2024-05-27T09:29:02.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.786+0000 7fb02a000700 1 -- start start 2024-05-27T09:29:02.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.786+0000 7fb02a000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb02410ad70 0x7fb0241a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:02.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.786+0000 7fb02a000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb02410c1c0 0x7fb0241a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:02.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.786+0000 7fb02a000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0241a0f80 con 0x7fb02410c1c0 2024-05-27T09:29:02.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.786+0000 7fb02a000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0241a10c0 con 0x7fb02410ad70 2024-05-27T09:29:02.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.787+0000 7fb022a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb02410c1c0 0x7fb0241a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:02.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.787+0000 7fb022a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb02410c1c0 0x7fb0241a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50864/0 (socket says 172.21.15.119:50864) 2024-05-27T09:29:02.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.787+0000 7fb022a00700 1 -- 172.21.15.119:0/539210210 learned_addr learned my addr 172.21.15.119:0/539210210 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:29:02.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.787+0000 7fb023400700 1 --2- 172.21.15.119:0/539210210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb02410ad70 0x7fb0241a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:02.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.787+0000 7fb022a00700 1 -- 172.21.15.119:0/539210210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb02410ad70 msgr2=0x7fb0241a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:02.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.787+0000 7fb022a00700 1 --2- 172.21.15.119:0/539210210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb02410ad70 0x7fb0241a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:02.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.787+0000 7fb022a00700 1 -- 172.21.15.119:0/539210210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0140095f0 con 0x7fb02410c1c0 2024-05-27T09:29:02.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.788+0000 7fb022a00700 1 --2- 172.21.15.119:0/539210210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb02410c1c0 0x7fb0241a0960 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb01000d8d0 tx=0x7fb01000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:02.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.789+0000 7fb01be00700 1 -- 172.21.15.119:0/539210210 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb010009940 con 0x7fb02410c1c0 2024-05-27T09:29:02.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.789+0000 7fb01be00700 1 -- 172.21.15.119:0/539210210 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb010010460 con 0x7fb02410c1c0 2024-05-27T09:29:02.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.789+0000 7fb01be00700 1 -- 172.21.15.119:0/539210210 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb01000f5f0 con 0x7fb02410c1c0 2024-05-27T09:29:02.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.789+0000 7fb02a000700 1 -- 172.21.15.119:0/539210210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0241a5b70 con 0x7fb02410c1c0 2024-05-27T09:29:02.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.789+0000 7fb02a000700 1 -- 172.21.15.119:0/539210210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0241a60c0 con 0x7fb02410c1c0 2024-05-27T09:29:02.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.790+0000 7fb02a000700 1 -- 172.21.15.119:0/539210210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0241136b0 con 0x7fb02410c1c0 2024-05-27T09:29:02.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.794+0000 7fb01be00700 1 -- 172.21.15.119:0/539210210 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fb0100105d0 con 0x7fb02410c1c0 2024-05-27T09:29:02.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.796+0000 7fb01be00700 1 --2- 172.21.15.119:0/539210210 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fb00806c7a0 0x7fb00806ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:02.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.796+0000 7fb01be00700 1 -- 172.21.15.119:0/539210210 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb01008b840 con 0x7fb02410c1c0 2024-05-27T09:29:02.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.797+0000 7fb023400700 1 --2- 172.21.15.119:0/539210210 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fb00806c7a0 0x7fb00806ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:02.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.799+0000 7fb023400700 1 --2- 172.21.15.119:0/539210210 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fb00806c7a0 0x7fb00806ec50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb014006010 tx=0x7fb01400b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:02.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:02.801+0000 7fb01be00700 1 -- 172.21.15.119:0/539210210 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb010059a60 con 0x7fb02410c1c0 2024-05-27T09:29:03.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:02 smithi138 ceph-mon[34885]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:29:03.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.174+0000 7fb02a000700 1 -- 172.21.15.119:0/539210210 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0241106d0 con 0x7fb00806c7a0 2024-05-27T09:29:03.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.189+0000 7fb01be00700 1 -- 172.21.15.119:0/539210210 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3528 (secure 0 0 0) 0x7fb0241106d0 con 0x7fb00806c7a0 2024-05-27T09:29:03.190 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:29:03.190 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (8m) 2m ago 10m 13.1M - 0.25.0 c8568f914cd2 bc280230ab28 2024-05-27T09:29:03.190 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (10m) 2m ago 10m 7486k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:29:03.190 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (9m) 2m ago 9m 7394k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:29:03.190 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (10m) 2m ago 10m 6665k - 18.2.0 dc2bc1663786 e5fafde44183 2024-05-27T09:29:03.191 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (9m) 2m ago 9m 6660k - 18.2.0 dc2bc1663786 46956ab99bab 2024-05-27T09:29:03.191 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (8m) 2m ago 9m 74.6M - 9.4.7 954c08fa6188 16eeaded4295 2024-05-27T09:29:03.191 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (3m) 2m ago 3m 11.5M - 18.2.0 dc2bc1663786 93b21713fb1e 2024-05-27T09:29:03.191 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (2m) 2m ago 2m 14.4M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:29:03.191 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (2m) 2m ago 2m 11.5M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:29:03.191 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (3m) 2m ago 3m 14.4M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:29:03.191 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:9283,8765,8443 running (11m) 2m ago 11m 487M - 18.2.0 dc2bc1663786 abe5e83463f2 2024-05-27T09:29:03.191 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 starting - - - - 2024-05-27T09:29:03.191 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (11m) 2m ago 12m 46.3M 2048M 18.2.0 dc2bc1663786 ba998b789a1c 2024-05-27T09:29:03.191 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (8m) 2m ago 8m 36.1M 2048M 18.2.0 dc2bc1663786 60216fd67078 2024-05-27T09:29:03.191 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (10m) 2m ago 10m 7788k - 1.5.0 0da6a335fe13 79c36d4e5b04 2024-05-27T09:29:03.191 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (8m) 2m ago 8m 7931k - 1.5.0 0da6a335fe13 80ee890e8830 2024-05-27T09:29:03.192 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (7m) 2m ago 7m 45.8M 3947M 18.2.0 dc2bc1663786 825bd1392508 2024-05-27T09:29:03.192 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (7m) 2m ago 7m 47.4M 3947M 18.2.0 dc2bc1663786 fce9819903d9 2024-05-27T09:29:03.192 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (6m) 2m ago 6m 46.2M 3947M 18.2.0 dc2bc1663786 3955a6112c1c 2024-05-27T09:29:03.192 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (6m) 2m ago 6m 45.1M 4971M 18.2.0 dc2bc1663786 693c92220622 2024-05-27T09:29:03.192 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (5m) 2m ago 5m 42.4M 4971M 18.2.0 dc2bc1663786 7559e658e917 2024-05-27T09:29:03.192 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (4m) 2m ago 4m 42.9M 4971M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:29:03.192 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (7m) 2m ago 9m 55.8M - 2.43.0 a07b618ecd1d 7f3d89d1f425 2024-05-27T09:29:03.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.193+0000 7fb02a000700 1 -- 172.21.15.119:0/539210210 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fb00806c7a0 msgr2=0x7fb00806ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:03.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.193+0000 7fb02a000700 1 --2- 172.21.15.119:0/539210210 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fb00806c7a0 0x7fb00806ec50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb014006010 tx=0x7fb01400b540 comp rx=0 tx=0).stop 2024-05-27T09:29:03.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.193+0000 7fb02a000700 1 -- 172.21.15.119:0/539210210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb02410c1c0 msgr2=0x7fb0241a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:03.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.193+0000 7fb02a000700 1 --2- 172.21.15.119:0/539210210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb02410c1c0 0x7fb0241a0960 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb01000d8d0 tx=0x7fb01000dc90 comp rx=0 tx=0).stop 2024-05-27T09:29:03.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.194+0000 7fb02a000700 1 -- 172.21.15.119:0/539210210 shutdown_connections 2024-05-27T09:29:03.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.194+0000 7fb02a000700 1 --2- 172.21.15.119:0/539210210 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7fb00806c7a0 0x7fb00806ec50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:03.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.194+0000 7fb02a000700 1 --2- 172.21.15.119:0/539210210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb02410ad70 0x7fb0241a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:03.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.194+0000 7fb02a000700 1 --2- 172.21.15.119:0/539210210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb02410c1c0 0x7fb0241a0960 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:03.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.194+0000 7fb02a000700 1 -- 172.21.15.119:0/539210210 >> 172.21.15.119:0/539210210 conn(0x7fb02406a630 msgr2=0x7fb02410efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:03.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.194+0000 7fb02a000700 1 -- 172.21.15.119:0/539210210 shutdown_connections 2024-05-27T09:29:03.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.194+0000 7fb02a000700 1 -- 172.21.15.119:0/539210210 wait complete. 2024-05-27T09:29:03.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.372+0000 7f7ea2800700 1 -- 172.21.15.119:0/2470783452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c101190 msgr2=0x7f7e9c1015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:03.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.372+0000 7f7ea2800700 1 --2- 172.21.15.119:0/2470783452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c101190 0x7f7e9c1015e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009910 tx=0x7f7e88009c20 comp rx=0 tx=0).stop 2024-05-27T09:29:03.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.372+0000 7f7ea2800700 1 -- 172.21.15.119:0/2470783452 shutdown_connections 2024-05-27T09:29:03.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.372+0000 7f7ea2800700 1 --2- 172.21.15.119:0/2470783452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c101190 0x7f7e9c1015e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:03.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.372+0000 7f7ea2800700 1 --2- 172.21.15.119:0/2470783452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e9c0fff90 0x7f7e9c1003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:03.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.372+0000 7f7ea2800700 1 -- 172.21.15.119:0/2470783452 >> 172.21.15.119:0/2470783452 conn(0x7f7e9c0fb500 msgr2=0x7f7e9c0fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:03.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.372+0000 7f7ea2800700 1 -- 172.21.15.119:0/2470783452 shutdown_connections 2024-05-27T09:29:03.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.373+0000 7f7ea2800700 1 -- 172.21.15.119:0/2470783452 wait complete. 2024-05-27T09:29:03.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.374+0000 7f7ea2800700 1 Processor -- start 2024-05-27T09:29:03.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.374+0000 7f7ea2800700 1 -- start start 2024-05-27T09:29:03.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.375+0000 7f7ea2800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e9c0fff90 0x7f7e9c1979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:03.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.375+0000 7f7ea2800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c101190 0x7f7e9c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:03.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.375+0000 7f7ea2800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c198510 con 0x7f7e9c101190 2024-05-27T09:29:03.375 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.375+0000 7f7ea2800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c198650 con 0x7f7e9c0fff90 2024-05-27T09:29:03.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.375+0000 7f7ea0a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c101190 0x7f7e9c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:03.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.375+0000 7f7ea1400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e9c0fff90 0x7f7e9c1979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:03.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.375+0000 7f7ea0a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c101190 0x7f7e9c197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50882/0 (socket says 172.21.15.119:50882) 2024-05-27T09:29:03.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.375+0000 7f7ea0a00700 1 -- 172.21.15.119:0/2074846108 learned_addr learned my addr 172.21.15.119:0/2074846108 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:29:03.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.376+0000 7f7ea0a00700 1 -- 172.21.15.119:0/2074846108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e9c0fff90 msgr2=0x7f7e9c1979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:03.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.376+0000 7f7ea0a00700 1 --2- 172.21.15.119:0/2074846108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e9c0fff90 0x7f7e9c1979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:03.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.376+0000 7f7ea0a00700 1 -- 172.21.15.119:0/2074846108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e880095f0 con 0x7f7e9c101190 2024-05-27T09:29:03.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.376+0000 7f7ea1400700 1 --2- 172.21.15.119:0/2074846108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e9c0fff90 0x7f7e9c1979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:29:03.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.376+0000 7f7ea0a00700 1 --2- 172.21.15.119:0/2074846108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c101190 0x7f7e9c197ef0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7e880092e0 tx=0x7f7e8800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:03.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.377+0000 7f7e9a000700 1 -- 172.21.15.119:0/2074846108 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e8801d070 con 0x7f7e9c101190 2024-05-27T09:29:03.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.377+0000 7f7ea2800700 1 -- 172.21.15.119:0/2074846108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9c19d0a0 con 0x7f7e9c101190 2024-05-27T09:29:03.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.377+0000 7f7ea2800700 1 -- 172.21.15.119:0/2074846108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c19d590 con 0x7f7e9c101190 2024-05-27T09:29:03.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.377+0000 7f7e9a000700 1 -- 172.21.15.119:0/2074846108 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7e88005dc0 con 0x7f7e9c101190 2024-05-27T09:29:03.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.377+0000 7f7e9a000700 1 -- 172.21.15.119:0/2074846108 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e88017630 con 0x7f7e9c101190 2024-05-27T09:29:03.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.378+0000 7f7ea2800700 1 -- 172.21.15.119:0/2074846108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e7c005320 con 0x7f7e9c101190 2024-05-27T09:29:03.380 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.379+0000 7f7e9a000700 1 -- 172.21.15.119:0/2074846108 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f7e8800faf0 con 0x7f7e9c101190 2024-05-27T09:29:03.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.381+0000 7f7e9a000700 1 --2- 172.21.15.119:0/2074846108 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f7e8406c680 0x7f7e8406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:03.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.381+0000 7f7e9a000700 1 -- 172.21.15.119:0/2074846108 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7e8808cc80 con 0x7f7e9c101190 2024-05-27T09:29:03.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.381+0000 7f7ea1400700 1 --2- 172.21.15.119:0/2074846108 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f7e8406c680 0x7f7e8406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:03.382 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.382+0000 7f7ea1400700 1 --2- 172.21.15.119:0/2074846108 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f7e8406c680 0x7f7e8406eb30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c100ff0 tx=0x7f7e90009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:03.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.387+0000 7f7e9a000700 1 -- 172.21.15.119:0/2074846108 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7e8805ad70 con 0x7f7e9c101190 2024-05-27T09:29:03.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.734+0000 7f7ea2800700 1 -- 172.21.15.119:0/2074846108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7e7c006200 con 0x7f7e9c101190 2024-05-27T09:29:03.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.735+0000 7f7e9a000700 1 -- 172.21.15.119:0/2074846108 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f7e8805a900 con 0x7f7e9c101190 2024-05-27T09:29:03.736 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:29:03.736 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:29:03.736 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-05-27T09:29:03.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:29:03.736 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:29:03.736 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-05-27T09:29:03.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:29:03.736 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:29:03.736 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-05-27T09:29:03.736 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:29:03.736 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:29:03.737 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-05-27T09:29:03.737 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:29:03.737 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:29:03.737 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 14 2024-05-27T09:29:03.737 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:29:03.737 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:29:03.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.737+0000 7f7ea2800700 1 -- 172.21.15.119:0/2074846108 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f7e8406c680 msgr2=0x7f7e8406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:03.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.737+0000 7f7ea2800700 1 --2- 172.21.15.119:0/2074846108 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f7e8406c680 0x7f7e8406eb30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c100ff0 tx=0x7f7e90009260 comp rx=0 tx=0).stop 2024-05-27T09:29:03.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.737+0000 7f7ea2800700 1 -- 172.21.15.119:0/2074846108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c101190 msgr2=0x7f7e9c197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:03.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.737+0000 7f7ea2800700 1 --2- 172.21.15.119:0/2074846108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c101190 0x7f7e9c197ef0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7e880092e0 tx=0x7f7e8800f690 comp rx=0 tx=0).stop 2024-05-27T09:29:03.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.738+0000 7f7ea2800700 1 -- 172.21.15.119:0/2074846108 shutdown_connections 2024-05-27T09:29:03.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.738+0000 7f7ea2800700 1 --2- 172.21.15.119:0/2074846108 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f7e8406c680 0x7f7e8406eb30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:03.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.738+0000 7f7ea2800700 1 --2- 172.21.15.119:0/2074846108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e9c0fff90 0x7f7e9c1979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:03.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.738+0000 7f7ea2800700 1 --2- 172.21.15.119:0/2074846108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e9c101190 0x7f7e9c197ef0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:03.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.738+0000 7f7ea2800700 1 -- 172.21.15.119:0/2074846108 >> 172.21.15.119:0/2074846108 conn(0x7f7e9c0fb500 msgr2=0x7f7e9c1023b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:03.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.738+0000 7f7ea2800700 1 -- 172.21.15.119:0/2074846108 shutdown_connections 2024-05-27T09:29:03.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.738+0000 7f7ea2800700 1 -- 172.21.15.119:0/2074846108 wait complete. 2024-05-27T09:29:03.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:03 smithi138 ceph-mon[34885]: pgmap v317: 65 pgs: 65 active+clean; 128 MiB data, 633 MiB used, 536 GiB / 536 GiB avail; 4.2 MiB/s wr, 359 op/s 2024-05-27T09:29:03.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:03 smithi138 ceph-mon[34885]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:29:03.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:03 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:29:03.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:03 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:29:03.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.901+0000 7f6105200700 1 -- 172.21.15.119:0/3272041649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61000fe480 msgr2=0x7f61000fe890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:03.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.901+0000 7f6105200700 1 --2- 172.21.15.119:0/3272041649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61000fe480 0x7f61000fe890 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f60f0009910 tx=0x7f60f0009c20 comp rx=0 tx=0).stop 2024-05-27T09:29:03.905 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:03 smithi119 ceph-mon[29082]: pgmap v317: 65 pgs: 65 active+clean; 128 MiB data, 633 MiB used, 536 GiB / 536 GiB avail; 4.2 MiB/s wr, 359 op/s 2024-05-27T09:29:03.905 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:03 smithi119 ceph-mon[29082]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:29:03.905 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:03 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:29:03.905 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:03 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:29:03.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.902+0000 7f6105200700 1 -- 172.21.15.119:0/3272041649 shutdown_connections 2024-05-27T09:29:03.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.902+0000 7f6105200700 1 --2- 172.21.15.119:0/3272041649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6100101460 0x7f61001018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:03.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.902+0000 7f6105200700 1 --2- 172.21.15.119:0/3272041649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61000fe480 0x7f61000fe890 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:03.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.902+0000 7f6105200700 1 -- 172.21.15.119:0/3272041649 >> 172.21.15.119:0/3272041649 conn(0x7f61000f9dd0 msgr2=0x7f61000fc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:03.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.902+0000 7f6105200700 1 -- 172.21.15.119:0/3272041649 shutdown_connections 2024-05-27T09:29:03.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.902+0000 7f6105200700 1 -- 172.21.15.119:0/3272041649 wait complete. 2024-05-27T09:29:03.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.903+0000 7f6105200700 1 Processor -- start 2024-05-27T09:29:03.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.903+0000 7f6105200700 1 -- start start 2024-05-27T09:29:03.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.904+0000 7f6105200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61000fe480 0x7f6100197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:03.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.904+0000 7f6105200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6100101460 0x7f6100198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:03.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.904+0000 7f6105200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61001987b0 con 0x7f6100101460 2024-05-27T09:29:03.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.904+0000 7f6105200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f610019d1c0 con 0x7f61000fe480 2024-05-27T09:29:03.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.904+0000 7f60fd400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6100101460 0x7f6100198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:03.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.904+0000 7f60fd400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6100101460 0x7f6100198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50898/0 (socket says 172.21.15.119:50898) 2024-05-27T09:29:03.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.904+0000 7f60fd400700 1 -- 172.21.15.119:0/1872948485 learned_addr learned my addr 172.21.15.119:0/1872948485 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:29:03.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.904+0000 7f60fde00700 1 --2- 172.21.15.119:0/1872948485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61000fe480 0x7f6100197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:03.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.905+0000 7f60fd400700 1 -- 172.21.15.119:0/1872948485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61000fe480 msgr2=0x7f6100197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:03.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.905+0000 7f60fd400700 1 --2- 172.21.15.119:0/1872948485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61000fe480 0x7f6100197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:03.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.905+0000 7f60fd400700 1 -- 172.21.15.119:0/1872948485 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f00095f0 con 0x7f6100101460 2024-05-27T09:29:03.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.905+0000 7f60fd400700 1 --2- 172.21.15.119:0/1872948485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6100101460 0x7f6100198190 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00e920 tx=0x7f60ec00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:03.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.905+0000 7f60f6a00700 1 -- 172.21.15.119:0/1872948485 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60ec00c9d0 con 0x7f6100101460 2024-05-27T09:29:03.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.905+0000 7f60f6a00700 1 -- 172.21.15.119:0/1872948485 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f60ec010430 con 0x7f6100101460 2024-05-27T09:29:03.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.905+0000 7f6105200700 1 -- 172.21.15.119:0/1872948485 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f610019d3c0 con 0x7f6100101460 2024-05-27T09:29:03.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.905+0000 7f60f6a00700 1 -- 172.21.15.119:0/1872948485 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60ec018650 con 0x7f6100101460 2024-05-27T09:29:03.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.906+0000 7f6105200700 1 -- 172.21.15.119:0/1872948485 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f610019d8e0 con 0x7f6100101460 2024-05-27T09:29:03.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.907+0000 7f6105200700 1 -- 172.21.15.119:0/1872948485 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6100105540 con 0x7f6100101460 2024-05-27T09:29:03.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.908+0000 7f60f6a00700 1 -- 172.21.15.119:0/1872948485 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f60ec0105a0 con 0x7f6100101460 2024-05-27T09:29:03.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.909+0000 7f60f6a00700 1 --2- 172.21.15.119:0/1872948485 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f60e406c680 0x7f60e406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:03.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.909+0000 7f60f6a00700 1 -- 172.21.15.119:0/1872948485 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f60ec014070 con 0x7f6100101460 2024-05-27T09:29:03.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.909+0000 7f60fde00700 1 --2- 172.21.15.119:0/1872948485 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f60e406c680 0x7f60e406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:03.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.910+0000 7f60fde00700 1 --2- 172.21.15.119:0/1872948485 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f60e406c680 0x7f60e406eb30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6100199130 tx=0x7f60f0005a90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:03.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:03.917+0000 7f60f6a00700 1 -- 172.21.15.119:0/1872948485 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f60ec051720 con 0x7f6100101460 2024-05-27T09:29:04.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:04.161+0000 7f6105200700 1 -- 172.21.15.119:0/1872948485 --> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6100061190 con 0x7f60e406c680 2024-05-27T09:29:04.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:04.163+0000 7f60f6a00700 1 -- 172.21.15.119:0/1872948485 <== mgr.14213 v2:172.21.15.119:6800/2191862652 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f6100061190 con 0x7f60e406c680 2024-05-27T09:29:04.164 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:29:04.164 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:29:04.164 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:29:04.164 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-27T09:29:04.164 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [], 2024-05-27T09:29:04.164 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "1/2 daemons upgraded", 2024-05-27T09:29:04.164 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "Currently upgrading mgr daemons", 2024-05-27T09:29:04.164 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:29:04.164 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:29:04.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:04.166+0000 7f6105200700 1 -- 172.21.15.119:0/1872948485 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f60e406c680 msgr2=0x7f60e406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:04.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:04.166+0000 7f6105200700 1 --2- 172.21.15.119:0/1872948485 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f60e406c680 0x7f60e406eb30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6100199130 tx=0x7f60f0005a90 comp rx=0 tx=0).stop 2024-05-27T09:29:04.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:04.166+0000 7f6105200700 1 -- 172.21.15.119:0/1872948485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6100101460 msgr2=0x7f6100198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:04.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:04.166+0000 7f6105200700 1 --2- 172.21.15.119:0/1872948485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6100101460 0x7f6100198190 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00e920 tx=0x7f60ec00ec30 comp rx=0 tx=0).stop 2024-05-27T09:29:04.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:04.166+0000 7f6105200700 1 -- 172.21.15.119:0/1872948485 shutdown_connections 2024-05-27T09:29:04.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:04.166+0000 7f6105200700 1 --2- 172.21.15.119:0/1872948485 >> [v2:172.21.15.119:6800/2191862652,v1:172.21.15.119:6801/2191862652] conn(0x7f60e406c680 0x7f60e406eb30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:04.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:04.166+0000 7f6105200700 1 --2- 172.21.15.119:0/1872948485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61000fe480 0x7f6100197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:04.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:04.166+0000 7f6105200700 1 --2- 172.21.15.119:0/1872948485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6100101460 0x7f6100198190 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:04.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:04.166+0000 7f6105200700 1 -- 172.21.15.119:0/1872948485 >> 172.21.15.119:0/1872948485 conn(0x7f61000f9dd0 msgr2=0x7f6100102680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:04.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:04.166+0000 7f6105200700 1 -- 172.21.15.119:0/1872948485 shutdown_connections 2024-05-27T09:29:04.168 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:04.167+0000 7f6105200700 1 -- 172.21.15.119:0/1872948485 wait complete. 2024-05-27T09:29:04.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:04 smithi119 ceph-mon[29082]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:29:04.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:04 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/2074846108' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:29:05.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:04 smithi138 ceph-mon[34885]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:29:05.022 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:04 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/2074846108' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:29:05.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:05 smithi138 ceph-mon[34885]: from='client.14590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:29:05.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:05 smithi138 ceph-mon[34885]: pgmap v318: 65 pgs: 65 active+clean; 152 MiB data, 708 MiB used, 536 GiB / 536 GiB avail; 6.1 MiB/s wr, 565 op/s 2024-05-27T09:29:05.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:05 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:29:05.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:05 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:29:05.980 INFO:tasks.workunit.client.0.smithi119.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-05-27T09:29:05.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:05 smithi119 ceph-mon[29082]: from='client.14590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:29:05.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:05 smithi119 ceph-mon[29082]: pgmap v318: 65 pgs: 65 active+clean; 152 MiB data, 708 MiB used, 536 GiB / 536 GiB avail; 6.1 MiB/s wr, 565 op/s 2024-05-27T09:29:05.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:05 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:29:05.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:05 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:29:06.017 INFO:tasks.workunit.client.0.smithi119.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-27T09:29:06.017 INFO:tasks.workunit.client.0.smithi119.stderr:+ make 2024-05-27T09:29:06.498 INFO:tasks.workunit.client.0.smithi119.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-05-27T09:29:07.100 INFO:tasks.workunit.client.0.smithi119.stderr:++ readlink -f fsstress 2024-05-27T09:29:07.101 INFO:tasks.workunit.client.0.smithi119.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-05-27T09:29:07.102 INFO:tasks.workunit.client.0.smithi119.stderr:+ popd 2024-05-27T09:29:07.106 INFO:tasks.workunit.client.0.smithi119.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-27T09:29:07.106 INFO:tasks.workunit.client.0.smithi119.stderr:+ popd 2024-05-27T09:29:07.107 INFO:tasks.workunit.client.0.smithi119.stdout:~/cephtest/mnt.0/client.0/tmp 2024-05-27T09:29:07.107 INFO:tasks.workunit.client.0.smithi119.stderr:++ mktemp -d -p . 2024-05-27T09:29:07.110 INFO:tasks.workunit.client.0.smithi119.stderr:+ T=./tmp.uwTadhjwLg 2024-05-27T09:29:07.111 INFO:tasks.workunit.client.0.smithi119.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.uwTadhjwLg -l 1 -n 1000 -p 10 -v 2024-05-27T09:29:07.114 INFO:tasks.workunit.client.0.smithi119.stdout:seed = 1716883712 2024-05-27T09:29:07.118 INFO:tasks.workunit.client.0.smithi119.stdout:0/0: chown . 112183 1 2024-05-27T09:29:07.120 INFO:tasks.workunit.client.0.smithi119.stdout:1/0: rmdir - no directory 2024-05-27T09:29:07.120 INFO:tasks.workunit.client.0.smithi119.stdout:1/1: dwrite - no filename 2024-05-27T09:29:07.120 INFO:tasks.workunit.client.0.smithi119.stdout:1/2: dwrite - no filename 2024-05-27T09:29:07.120 INFO:tasks.workunit.client.0.smithi119.stdout:1/3: rename - no filename 2024-05-27T09:29:07.120 INFO:tasks.workunit.client.0.smithi119.stdout:1/4: chown . 9885497 1 2024-05-27T09:29:07.121 INFO:tasks.workunit.client.0.smithi119.stdout:0/1: symlink l0 0 2024-05-27T09:29:07.122 INFO:tasks.workunit.client.0.smithi119.stdout:2/0: dread - no filename 2024-05-27T09:29:07.124 INFO:tasks.workunit.client.0.smithi119.stdout:1/5: mkdir d0 0 2024-05-27T09:29:07.124 INFO:tasks.workunit.client.0.smithi119.stdout:1/6: dwrite - no filename 2024-05-27T09:29:07.124 INFO:tasks.workunit.client.0.smithi119.stdout:1/7: dwrite - no filename 2024-05-27T09:29:07.124 INFO:tasks.workunit.client.0.smithi119.stdout:1/8: readlink - no filename 2024-05-27T09:29:07.124 INFO:tasks.workunit.client.0.smithi119.stdout:1/9: dread - no filename 2024-05-27T09:29:07.124 INFO:tasks.workunit.client.0.smithi119.stdout:1/10: dwrite - no filename 2024-05-27T09:29:07.125 INFO:tasks.workunit.client.0.smithi119.stdout:1/11: truncate - no filename 2024-05-27T09:29:07.125 INFO:tasks.workunit.client.0.smithi119.stdout:1/12: write - no filename 2024-05-27T09:29:07.125 INFO:tasks.workunit.client.0.smithi119.stdout:1/13: dread - no filename 2024-05-27T09:29:07.125 INFO:tasks.workunit.client.0.smithi119.stdout:1/14: dwrite - no filename 2024-05-27T09:29:07.125 INFO:tasks.workunit.client.0.smithi119.stdout:1/15: link - no file 2024-05-27T09:29:07.125 INFO:tasks.workunit.client.0.smithi119.stdout:0/2: symlink l1 0 2024-05-27T09:29:07.125 INFO:tasks.workunit.client.0.smithi119.stdout:0/3: chown l0 29 1 2024-05-27T09:29:07.126 INFO:tasks.workunit.client.0.smithi119.stdout:0/4: rmdir - no directory 2024-05-27T09:29:07.131 INFO:tasks.workunit.client.0.smithi119.stdout:1/16: rmdir d0 0 2024-05-27T09:29:07.131 INFO:tasks.workunit.client.0.smithi119.stdout:1/17: rename - no filename 2024-05-27T09:29:07.131 INFO:tasks.workunit.client.0.smithi119.stdout:1/18: rename - no filename 2024-05-27T09:29:07.131 INFO:tasks.workunit.client.0.smithi119.stdout:1/19: fsync - no filename 2024-05-27T09:29:07.131 INFO:tasks.workunit.client.0.smithi119.stdout:1/20: write - no filename 2024-05-27T09:29:07.131 INFO:tasks.workunit.client.0.smithi119.stdout:1/21: dwrite - no filename 2024-05-27T09:29:07.138 INFO:tasks.workunit.client.0.smithi119.stdout:1/22: mknod c1 0 2024-05-27T09:29:07.138 INFO:tasks.workunit.client.0.smithi119.stdout:3/0: creat f0 x:0 0 0 2024-05-27T09:29:07.138 INFO:tasks.workunit.client.0.smithi119.stdout:4/0: read - no filename 2024-05-27T09:29:07.138 INFO:tasks.workunit.client.0.smithi119.stdout:4/1: dread - no filename 2024-05-27T09:29:07.138 INFO:tasks.workunit.client.0.smithi119.stdout:3/1: creat f1 x:0 0 0 2024-05-27T09:29:07.138 INFO:tasks.workunit.client.0.smithi119.stdout:3/2: chown f1 3 1 2024-05-27T09:29:07.138 INFO:tasks.workunit.client.0.smithi119.stdout:3/3: fdatasync f1 0 2024-05-27T09:29:07.138 INFO:tasks.workunit.client.0.smithi119.stdout:3/4: creat f2 x:0 0 0 2024-05-27T09:29:07.139 INFO:tasks.workunit.client.0.smithi119.stdout:3/5: read - f1 zero size 2024-05-27T09:29:07.139 INFO:tasks.workunit.client.0.smithi119.stdout:5/0: write - no filename 2024-05-27T09:29:07.140 INFO:tasks.workunit.client.0.smithi119.stdout:5/1: write - no filename 2024-05-27T09:29:07.140 INFO:tasks.workunit.client.0.smithi119.stdout:5/2: dread - no filename 2024-05-27T09:29:07.140 INFO:tasks.workunit.client.0.smithi119.stdout:5/3: unlink - no file 2024-05-27T09:29:07.140 INFO:tasks.workunit.client.0.smithi119.stdout:5/4: rename - no filename 2024-05-27T09:29:07.140 INFO:tasks.workunit.client.0.smithi119.stdout:5/5: readlink - no filename 2024-05-27T09:29:07.144 INFO:tasks.workunit.client.0.smithi119.stdout:1/23: creat f2 x:0 0 0 2024-05-27T09:29:07.145 INFO:tasks.workunit.client.0.smithi119.stdout:1/24: creat f3 x:0 0 0 2024-05-27T09:29:07.146 INFO:tasks.workunit.client.0.smithi119.stdout:4/2: mknod c0 0 2024-05-27T09:29:07.148 INFO:tasks.workunit.client.0.smithi119.stdout:3/6: symlink l3 0 2024-05-27T09:29:07.152 INFO:tasks.workunit.client.0.smithi119.stdout:6/0: chown . 1 1 2024-05-27T09:29:07.152 INFO:tasks.workunit.client.0.smithi119.stdout:6/1: chown . 21942 1 2024-05-27T09:29:07.152 INFO:tasks.workunit.client.0.smithi119.stdout:6/2: unlink - no file 2024-05-27T09:29:07.152 INFO:tasks.workunit.client.0.smithi119.stdout:6/3: dwrite - no filename 2024-05-27T09:29:07.154 INFO:tasks.workunit.client.0.smithi119.stdout:5/6: creat f0 x:0 0 0 2024-05-27T09:29:07.154 INFO:tasks.workunit.client.0.smithi119.stdout:5/7: creat f1 x:0 0 0 2024-05-27T09:29:07.157 INFO:tasks.workunit.client.0.smithi119.stdout:1/25: symlink l4 0 2024-05-27T09:29:07.165 INFO:tasks.workunit.client.0.smithi119.stdout:4/3: creat f1 x:0 0 0 2024-05-27T09:29:07.165 INFO:tasks.workunit.client.0.smithi119.stdout:4/4: dread - f1 zero size 2024-05-27T09:29:07.165 INFO:tasks.workunit.client.0.smithi119.stdout:3/7: mknod c4 0 2024-05-27T09:29:07.165 INFO:tasks.workunit.client.0.smithi119.stdout:6/4: mkdir d0 0 2024-05-27T09:29:07.165 INFO:tasks.workunit.client.0.smithi119.stdout:6/5: chown d0 6655 1 2024-05-27T09:29:07.165 INFO:tasks.workunit.client.0.smithi119.stdout:6/6: readlink - no filename 2024-05-27T09:29:07.165 INFO:tasks.workunit.client.0.smithi119.stdout:6/7: readlink - no filename 2024-05-27T09:29:07.165 INFO:tasks.workunit.client.0.smithi119.stdout:6/8: truncate - no filename 2024-05-27T09:29:07.167 INFO:tasks.workunit.client.0.smithi119.stdout:7/0: write - no filename 2024-05-27T09:29:07.167 INFO:tasks.workunit.client.0.smithi119.stdout:7/1: fdatasync - no filename 2024-05-27T09:29:07.167 INFO:tasks.workunit.client.0.smithi119.stdout:7/2: truncate - no filename 2024-05-27T09:29:07.167 INFO:tasks.workunit.client.0.smithi119.stdout:7/3: truncate - no filename 2024-05-27T09:29:07.167 INFO:tasks.workunit.client.0.smithi119.stdout:7/4: stat - no entries 2024-05-27T09:29:07.167 INFO:tasks.workunit.client.0.smithi119.stdout:7/5: write - no filename 2024-05-27T09:29:07.167 INFO:tasks.workunit.client.0.smithi119.stdout:3/8: symlink l5 0 2024-05-27T09:29:07.167 INFO:tasks.workunit.client.0.smithi119.stdout:3/9: chown c4 6387332 1 2024-05-27T09:29:07.167 INFO:tasks.workunit.client.0.smithi119.stdout:7/6: write - no filename 2024-05-27T09:29:07.168 INFO:tasks.workunit.client.0.smithi119.stdout:7/7: chown . 145 1 2024-05-27T09:29:07.168 INFO:tasks.workunit.client.0.smithi119.stdout:7/8: read - no filename 2024-05-27T09:29:07.169 INFO:tasks.workunit.client.0.smithi119.stdout:6/9: symlink d0/l1 0 2024-05-27T09:29:07.169 INFO:tasks.workunit.client.0.smithi119.stdout:6/10: write - no filename 2024-05-27T09:29:07.172 INFO:tasks.workunit.client.0.smithi119.stdout:8/0: getdents . 0 2024-05-27T09:29:07.172 INFO:tasks.workunit.client.0.smithi119.stdout:8/1: rename - no filename 2024-05-27T09:29:07.172 INFO:tasks.workunit.client.0.smithi119.stdout:8/2: write - no filename 2024-05-27T09:29:07.174 INFO:tasks.workunit.client.0.smithi119.stdout:7/9: creat f0 x:0 0 0 2024-05-27T09:29:07.179 INFO:tasks.workunit.client.0.smithi119.stdout:6/11: mkdir d0/d2 0 2024-05-27T09:29:07.179 INFO:tasks.workunit.client.0.smithi119.stdout:8/3: creat f0 x:0 0 0 2024-05-27T09:29:07.179 INFO:tasks.workunit.client.0.smithi119.stdout:8/4: dread - f0 zero size 2024-05-27T09:29:07.180 INFO:tasks.workunit.client.0.smithi119.stdout:6/12: rename d0 to d0/d2/d3 22 2024-05-27T09:29:07.180 INFO:tasks.workunit.client.0.smithi119.stdout:6/13: chown d0/d2 1 1 2024-05-27T09:29:07.180 INFO:tasks.workunit.client.0.smithi119.stdout:6/14: readlink d0/l1 0 2024-05-27T09:29:07.180 INFO:tasks.workunit.client.0.smithi119.stdout:6/15: readlink d0/l1 0 2024-05-27T09:29:07.180 INFO:tasks.workunit.client.0.smithi119.stdout:6/16: chown d0/d2 27056 1 2024-05-27T09:29:07.182 INFO:tasks.workunit.client.0.smithi119.stdout:8/5: link f0 f1 0 2024-05-27T09:29:07.182 INFO:tasks.workunit.client.0.smithi119.stdout:8/6: write f0 [1022206,129151] 0 2024-05-27T09:29:07.183 INFO:tasks.workunit.client.0.smithi119.stdout:6/17: creat d0/d2/f4 x:0 0 0 2024-05-27T09:29:07.183 INFO:tasks.workunit.client.0.smithi119.stdout:6/18: readlink d0/l1 0 2024-05-27T09:29:07.185 INFO:tasks.workunit.client.0.smithi119.stdout:6/19: creat d0/f5 x:0 0 0 2024-05-27T09:29:07.185 INFO:tasks.workunit.client.0.smithi119.stdout:6/20: truncate d0/f5 178866 0 2024-05-27T09:29:07.308 INFO:tasks.workunit.client.0.smithi119.stdout:0/5: getdents . 0 2024-05-27T09:29:07.308 INFO:tasks.workunit.client.0.smithi119.stdout:0/6: dwrite - no filename 2024-05-27T09:29:07.308 INFO:tasks.workunit.client.0.smithi119.stdout:0/7: read - no filename 2024-05-27T09:29:07.308 INFO:tasks.workunit.client.0.smithi119.stdout:0/8: fsync - no filename 2024-05-27T09:29:07.324 INFO:tasks.workunit.client.0.smithi119.stdout:3/10: fsync f0 0 2024-05-27T09:29:07.328 INFO:tasks.workunit.client.0.smithi119.stdout:1/26: dwrite f2 [0,4194304] 0 2024-05-27T09:29:07.329 INFO:tasks.workunit.client.0.smithi119.stdout:1/27: creat f5 x:0 0 0 2024-05-27T09:29:07.329 INFO:tasks.workunit.client.0.smithi119.stdout:1/28: chown f3 1036719 1 2024-05-27T09:29:07.329 INFO:tasks.workunit.client.0.smithi119.stdout:1/29: dread - f5 zero size 2024-05-27T09:29:07.329 INFO:tasks.workunit.client.0.smithi119.stdout:1/30: dread - f3 zero size 2024-05-27T09:29:07.329 INFO:tasks.workunit.client.0.smithi119.stdout:1/31: creat f6 x:0 0 0 2024-05-27T09:29:07.340 INFO:tasks.workunit.client.0.smithi119.stdout:0/9: mkdir d2 0 2024-05-27T09:29:07.340 INFO:tasks.workunit.client.0.smithi119.stdout:0/10: getdents d2 0 2024-05-27T09:29:07.346 INFO:tasks.workunit.client.0.smithi119.stdout:5/8: fsync f1 0 2024-05-27T09:29:07.350 INFO:tasks.workunit.client.0.smithi119.stdout:4/5: dwrite f1 [0,4194304] 0 2024-05-27T09:29:07.350 INFO:tasks.workunit.client.0.smithi119.stdout:9/0: truncate - no filename 2024-05-27T09:29:07.350 INFO:tasks.workunit.client.0.smithi119.stdout:4/6: fsync f1 0 2024-05-27T09:29:07.350 INFO:tasks.workunit.client.0.smithi119.stdout:9/1: chown . 58 1 2024-05-27T09:29:07.350 INFO:tasks.workunit.client.0.smithi119.stdout:9/2: dwrite - no filename 2024-05-27T09:29:07.350 INFO:tasks.workunit.client.0.smithi119.stdout:9/3: chown . 2070660636 1 2024-05-27T09:29:07.351 INFO:tasks.workunit.client.0.smithi119.stdout:7/10: dwrite f0 [0,4194304] 0 2024-05-27T09:29:07.358 INFO:tasks.workunit.client.0.smithi119.stdout:6/21: fsync d0/d2/f4 0 2024-05-27T09:29:07.371 INFO:tasks.workunit.client.0.smithi119.stdout:4/7: write f1 [782039,130843] 0 2024-05-27T09:29:07.371 INFO:tasks.workunit.client.0.smithi119.stdout:4/8: creat f2 x:0 0 0 2024-05-27T09:29:07.371 INFO:tasks.workunit.client.0.smithi119.stdout:4/9: chown c0 5294216 1 2024-05-27T09:29:07.371 INFO:tasks.workunit.client.0.smithi119.stdout:4/10: creat f3 x:0 0 0 2024-05-27T09:29:07.372 INFO:tasks.workunit.client.0.smithi119.stdout:6/22: dread d0/f5 [0,4194304] 0 2024-05-27T09:29:07.372 INFO:tasks.workunit.client.0.smithi119.stdout:4/11: write f2 [1012713,115324] 0 2024-05-27T09:29:07.372 INFO:tasks.workunit.client.0.smithi119.stdout:0/11: link l0 d2/l3 0 2024-05-27T09:29:07.375 INFO:tasks.workunit.client.0.smithi119.stdout:5/9: mknod c2 0 2024-05-27T09:29:07.375 INFO:tasks.workunit.client.0.smithi119.stdout:5/10: readlink - no filename 2024-05-27T09:29:07.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:07 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:29:07.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:07 smithi138 ceph-mon[34885]: pgmap v319: 65 pgs: 65 active+clean; 171 MiB data, 770 MiB used, 536 GiB / 536 GiB avail; 6.0 MiB/s wr, 494 op/s 2024-05-27T09:29:07.387 INFO:tasks.workunit.client.0.smithi119.stdout:8/7: fdatasync f1 0 2024-05-27T09:29:07.387 INFO:tasks.workunit.client.0.smithi119.stdout:8/8: write f1 [2057706,17331] 0 2024-05-27T09:29:07.387 INFO:tasks.workunit.client.0.smithi119.stdout:8/9: creat f2 x:0 0 0 2024-05-27T09:29:07.387 INFO:tasks.workunit.client.0.smithi119.stdout:3/11: dwrite f0 [0,4194304] 0 2024-05-27T09:29:07.387 INFO:tasks.workunit.client.0.smithi119.stdout:8/10: write f0 [2155356,32494] 0 2024-05-27T09:29:07.387 INFO:tasks.workunit.client.0.smithi119.stdout:3/12: creat f6 x:0 0 0 2024-05-27T09:29:07.392 INFO:tasks.workunit.client.0.smithi119.stdout:6/23: link d0/f5 d0/f6 0 2024-05-27T09:29:07.394 INFO:tasks.workunit.client.0.smithi119.stdout:4/12: mkdir d4 0 2024-05-27T09:29:07.395 INFO:tasks.workunit.client.0.smithi119.stdout:0/12: mkdir d2/d4 0 2024-05-27T09:29:07.396 INFO:tasks.workunit.client.0.smithi119.stdout:0/13: chown l0 3 1 2024-05-27T09:29:07.396 INFO:tasks.workunit.client.0.smithi119.stdout:0/14: read - no filename 2024-05-27T09:29:07.396 INFO:tasks.workunit.client.0.smithi119.stdout:0/15: dwrite - no filename 2024-05-27T09:29:07.400 INFO:tasks.workunit.client.0.smithi119.stdout:5/11: mknod c3 0 2024-05-27T09:29:07.402 INFO:tasks.workunit.client.0.smithi119.stdout:7/11: rename f0 to f1 0 2024-05-27T09:29:07.414 INFO:tasks.workunit.client.0.smithi119.stdout:8/11: mkdir d3 0 2024-05-27T09:29:07.414 INFO:tasks.workunit.client.0.smithi119.stdout:8/12: chown d3 342634430 1 2024-05-27T09:29:07.414 INFO:tasks.workunit.client.0.smithi119.stdout:8/13: chown d3 88486 1 2024-05-27T09:29:07.416 INFO:tasks.workunit.client.0.smithi119.stdout:3/13: mknod c7 0 2024-05-27T09:29:07.416 INFO:tasks.workunit.client.0.smithi119.stdout:3/14: write f6 [444982,18367] 0 2024-05-27T09:29:07.419 INFO:tasks.workunit.client.0.smithi119.stdout:6/24: rename d0/f5 to d0/f7 0 2024-05-27T09:29:07.421 INFO:tasks.workunit.client.0.smithi119.stdout:0/16: unlink l1 0 2024-05-27T09:29:07.423 INFO:tasks.workunit.client.0.smithi119.stdout:5/12: rename c2 to c4 0 2024-05-27T09:29:07.423 INFO:tasks.workunit.client.0.smithi119.stdout:5/13: fdatasync f0 0 2024-05-27T09:29:07.424 INFO:tasks.workunit.client.0.smithi119.stdout:5/14: write f0 [860614,70740] 0 2024-05-27T09:29:07.424 INFO:tasks.workunit.client.0.smithi119.stdout:5/15: read - f1 zero size 2024-05-27T09:29:07.425 INFO:tasks.workunit.client.0.smithi119.stdout:7/12: mknod c2 0 2024-05-27T09:29:07.428 INFO:tasks.workunit.client.0.smithi119.stdout:0/17: mknod d2/d4/c5 0 2024-05-27T09:29:07.430 INFO:tasks.workunit.client.0.smithi119.stdout:0/18: mkdir d2/d6 0 2024-05-27T09:29:07.431 INFO:tasks.workunit.client.0.smithi119.stdout:0/19: mknod d2/d4/c7 0 2024-05-27T09:29:07.431 INFO:tasks.workunit.client.0.smithi119.stdout:0/20: dwrite - no filename 2024-05-27T09:29:07.431 INFO:tasks.workunit.client.0.smithi119.stdout:0/21: dwrite - no filename 2024-05-27T09:29:07.431 INFO:tasks.workunit.client.0.smithi119.stdout:0/22: chown d2/l3 13250264 1 2024-05-27T09:29:07.432 INFO:tasks.workunit.client.0.smithi119.stdout:0/23: creat d2/d6/f8 x:0 0 0 2024-05-27T09:29:07.434 INFO:tasks.workunit.client.0.smithi119.stdout:0/24: mkdir d2/d6/d9 0 2024-05-27T09:29:07.434 INFO:tasks.workunit.client.0.smithi119.stdout:0/25: truncate d2/d6/f8 39007 0 2024-05-27T09:29:07.464 INFO:tasks.workunit.client.0.smithi119.stdout:4/13: truncate f1 1188045 0 2024-05-27T09:29:07.465 INFO:tasks.workunit.client.0.smithi119.stdout:4/14: getdents d4 0 2024-05-27T09:29:07.467 INFO:tasks.workunit.client.0.smithi119.stdout:4/15: mkdir d4/d5 0 2024-05-27T09:29:07.470 INFO:tasks.workunit.client.0.smithi119.stdout:6/25: dwrite d0/f6 [0,4194304] 0 2024-05-27T09:29:07.470 INFO:tasks.workunit.client.0.smithi119.stdout:4/16: mkdir d4/d6 0 2024-05-27T09:29:07.476 INFO:tasks.workunit.client.0.smithi119.stdout:5/16: dwrite f1 [0,4194304] 0 2024-05-27T09:29:07.479 INFO:tasks.workunit.client.0.smithi119.stdout:6/26: mknod d0/d2/c8 0 2024-05-27T09:29:07.480 INFO:tasks.workunit.client.0.smithi119.stdout:4/17: mknod d4/d6/c7 0 2024-05-27T09:29:07.480 INFO:tasks.workunit.client.0.smithi119.stdout:4/18: chown f3 466 1 2024-05-27T09:29:07.480 INFO:tasks.workunit.client.0.smithi119.stdout:6/27: creat d0/d2/f9 x:0 0 0 2024-05-27T09:29:07.496 INFO:tasks.workunit.client.0.smithi119.stdout:5/17: write f1 [5108459,110656] 0 2024-05-27T09:29:07.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:07 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:29:07.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:07 smithi119 ceph-mon[29082]: pgmap v319: 65 pgs: 65 active+clean; 171 MiB data, 770 MiB used, 536 GiB / 536 GiB avail; 6.0 MiB/s wr, 494 op/s 2024-05-27T09:29:07.499 INFO:tasks.workunit.client.0.smithi119.stdout:5/18: rename c3 to c5 0 2024-05-27T09:29:07.503 INFO:tasks.workunit.client.0.smithi119.stdout:5/19: write f0 [543763,93357] 0 2024-05-27T09:29:07.505 INFO:tasks.workunit.client.0.smithi119.stdout:5/20: mkdir d6 0 2024-05-27T09:29:07.525 INFO:tasks.workunit.client.0.smithi119.stdout:7/13: dwrite f1 [0,4194304] 0 2024-05-27T09:29:07.527 INFO:tasks.workunit.client.0.smithi119.stdout:5/21: write f1 [2916365,99144] 0 2024-05-27T09:29:07.527 INFO:tasks.workunit.client.0.smithi119.stdout:5/22: write f0 [408128,7376] 0 2024-05-27T09:29:07.529 INFO:tasks.workunit.client.0.smithi119.stdout:7/14: getdents . 0 2024-05-27T09:29:07.531 INFO:tasks.workunit.client.0.smithi119.stdout:5/23: rename c4 to d6/c7 0 2024-05-27T09:29:07.533 INFO:tasks.workunit.client.0.smithi119.stdout:7/15: creat f3 x:0 0 0 2024-05-27T09:29:07.533 INFO:tasks.workunit.client.0.smithi119.stdout:7/16: write f3 [69376,26762] 0 2024-05-27T09:29:07.533 INFO:tasks.workunit.client.0.smithi119.stdout:7/17: fsync f1 0 2024-05-27T09:29:07.533 INFO:tasks.workunit.client.0.smithi119.stdout:7/18: write f3 [852763,98898] 0 2024-05-27T09:29:07.533 INFO:tasks.workunit.client.0.smithi119.stdout:7/19: fsync f1 0 2024-05-27T09:29:07.559 INFO:tasks.workunit.client.0.smithi119.stdout:7/20: dread f1 [0,4194304] 0 2024-05-27T09:29:07.561 INFO:tasks.workunit.client.0.smithi119.stdout:7/21: rename f3 to f4 0 2024-05-27T09:29:07.562 INFO:tasks.workunit.client.0.smithi119.stdout:7/22: stat f1 0 2024-05-27T09:29:07.569 INFO:tasks.workunit.client.0.smithi119.stdout:3/15: dwrite f0 [4194304,4194304] 0 2024-05-27T09:29:07.573 INFO:tasks.workunit.client.0.smithi119.stdout:5/24: dread f1 [0,4194304] 0 2024-05-27T09:29:07.573 INFO:tasks.workunit.client.0.smithi119.stdout:5/25: chown f0 2 1 2024-05-27T09:29:07.582 INFO:tasks.workunit.client.0.smithi119.stdout:7/23: dread f1 [0,4194304] 0 2024-05-27T09:29:07.600 INFO:tasks.workunit.client.0.smithi119.stdout:7/24: write f1 [3567557,14287] 0 2024-05-27T09:29:07.600 INFO:tasks.workunit.client.0.smithi119.stdout:7/25: write f1 [2518520,82690] 0 2024-05-27T09:29:07.665 INFO:tasks.workunit.client.0.smithi119.stdout:3/16: dwrite f1 [0,4194304] 0 2024-05-27T09:29:07.665 INFO:tasks.workunit.client.0.smithi119.stdout:3/17: readlink l5 0 2024-05-27T09:29:07.665 INFO:tasks.workunit.client.0.smithi119.stdout:7/26: dwrite f1 [0,4194304] 0 2024-05-27T09:29:07.667 INFO:tasks.workunit.client.0.smithi119.stdout:3/18: rename f2 to f8 0 2024-05-27T09:29:07.668 INFO:tasks.workunit.client.0.smithi119.stdout:7/27: write f4 [307002,18710] 0 2024-05-27T09:29:07.672 INFO:tasks.workunit.client.0.smithi119.stdout:3/19: symlink l9 0 2024-05-27T09:29:07.690 INFO:tasks.workunit.client.0.smithi119.stdout:3/20: getdents . 0 2024-05-27T09:29:07.690 INFO:tasks.workunit.client.0.smithi119.stdout:3/21: getdents . 0 2024-05-27T09:29:07.738 INFO:tasks.workunit.client.0.smithi119.stdout:3/22: dread f1 [0,4194304] 0 2024-05-27T09:29:07.741 INFO:tasks.workunit.client.0.smithi119.stdout:7/28: dwrite f1 [4194304,4194304] 0 2024-05-27T09:29:07.741 INFO:tasks.workunit.client.0.smithi119.stdout:7/29: fsync f1 0 2024-05-27T09:29:07.755 INFO:tasks.workunit.client.0.smithi119.stdout:3/23: write f0 [302512,25821] 0 2024-05-27T09:29:07.758 INFO:tasks.workunit.client.0.smithi119.stdout:3/24: write f6 [118820,30278] 0 2024-05-27T09:29:07.760 INFO:tasks.workunit.client.0.smithi119.stdout:3/25: creat fa x:0 0 0 2024-05-27T09:29:07.777 INFO:tasks.workunit.client.0.smithi119.stdout:3/26: dread f6 [0,4194304] 0 2024-05-27T09:29:07.779 INFO:tasks.workunit.client.0.smithi119.stdout:3/27: mkdir db 0 2024-05-27T09:29:07.779 INFO:tasks.workunit.client.0.smithi119.stdout:3/28: chown c7 20 1 2024-05-27T09:29:07.779 INFO:tasks.workunit.client.0.smithi119.stdout:3/29: write f8 [859787,71855] 0 2024-05-27T09:29:07.820 INFO:tasks.workunit.client.0.smithi119.stdout:7/30: dwrite f1 [0,4194304] 0 2024-05-27T09:29:07.821 INFO:tasks.workunit.client.0.smithi119.stdout:7/31: chown f4 89 1 2024-05-27T09:29:07.841 INFO:tasks.workunit.client.0.smithi119.stdout:3/30: dwrite f6 [0,4194304] 0 2024-05-27T09:29:07.842 INFO:tasks.workunit.client.0.smithi119.stdout:7/32: dread f4 [0,4194304] 0 2024-05-27T09:29:07.845 INFO:tasks.workunit.client.0.smithi119.stdout:3/31: creat db/fc x:0 0 0 2024-05-27T09:29:07.847 INFO:tasks.workunit.client.0.smithi119.stdout:7/33: symlink l5 0 2024-05-27T09:29:07.847 INFO:tasks.workunit.client.0.smithi119.stdout:7/34: chown c2 474 1 2024-05-27T09:29:07.849 INFO:tasks.workunit.client.0.smithi119.stdout:7/35: read f4 [315407,39402] 0 2024-05-27T09:29:07.851 INFO:tasks.workunit.client.0.smithi119.stdout:3/32: unlink l3 0 2024-05-27T09:29:07.851 INFO:tasks.workunit.client.0.smithi119.stdout:3/33: readlink l9 0 2024-05-27T09:29:07.875 INFO:tasks.workunit.client.0.smithi119.stdout:3/34: dread f0 [0,4194304] 0 2024-05-27T09:29:07.876 INFO:tasks.workunit.client.0.smithi119.stdout:3/35: truncate fa 292048 0 2024-05-27T09:29:07.876 INFO:tasks.workunit.client.0.smithi119.stdout:3/36: readlink l5 0 2024-05-27T09:29:07.876 INFO:tasks.workunit.client.0.smithi119.stdout:3/37: creat db/fd x:0 0 0 2024-05-27T09:29:07.876 INFO:tasks.workunit.client.0.smithi119.stdout:3/38: creat db/fe x:0 0 0 2024-05-27T09:29:07.889 INFO:tasks.workunit.client.0.smithi119.stdout:3/39: rmdir db 39 2024-05-27T09:29:07.896 INFO:tasks.workunit.client.0.smithi119.stdout:3/40: link f8 db/ff 0 2024-05-27T09:29:07.899 INFO:tasks.workunit.client.0.smithi119.stdout:3/41: mkdir db/d10 0 2024-05-27T09:29:07.901 INFO:tasks.workunit.client.0.smithi119.stdout:3/42: unlink db/fe 0 2024-05-27T09:29:07.902 INFO:tasks.workunit.client.0.smithi119.stdout:3/43: readlink l5 0 2024-05-27T09:29:07.903 INFO:tasks.workunit.client.0.smithi119.stdout:3/44: symlink db/l11 0 2024-05-27T09:29:07.905 INFO:tasks.workunit.client.0.smithi119.stdout:3/45: creat db/f12 x:0 0 0 2024-05-27T09:29:07.906 INFO:tasks.workunit.client.0.smithi119.stdout:3/46: mkdir db/d10/d13 0 2024-05-27T09:29:07.908 INFO:tasks.workunit.client.0.smithi119.stdout:3/47: symlink db/d10/l14 0 2024-05-27T09:29:07.915 INFO:tasks.workunit.client.0.smithi119.stdout:7/36: dwrite f4 [0,4194304] 0 2024-05-27T09:29:07.918 INFO:tasks.workunit.client.0.smithi119.stdout:7/37: mknod c6 0 2024-05-27T09:29:07.918 INFO:tasks.workunit.client.0.smithi119.stdout:7/38: stat f1 0 2024-05-27T09:29:07.918 INFO:tasks.workunit.client.0.smithi119.stdout:7/39: fsync f4 0 2024-05-27T09:29:07.924 INFO:tasks.workunit.client.0.smithi119.stdout:7/40: mkdir d7 0 2024-05-27T09:29:07.928 INFO:tasks.workunit.client.0.smithi119.stdout:7/41: rename c6 to d7/c8 0 2024-05-27T09:29:07.943 INFO:tasks.workunit.client.0.smithi119.stdout:7/42: truncate f1 7285564 0 2024-05-27T09:29:07.945 INFO:tasks.workunit.client.0.smithi119.stdout:7/43: rmdir d7 39 2024-05-27T09:29:07.985 INFO:tasks.workunit.client.0.smithi119.stdout:3/48: dwrite db/fc [0,4194304] 0 2024-05-27T09:29:07.991 INFO:tasks.workunit.client.0.smithi119.stdout:3/49: symlink db/d10/d13/l15 0 2024-05-27T09:29:08.003 INFO:tasks.workunit.client.0.smithi119.stdout:7/44: dwrite f1 [4194304,4194304] 0 2024-05-27T09:29:08.013 INFO:tasks.workunit.client.0.smithi119.stdout:3/50: truncate f8 585005 0 2024-05-27T09:29:08.014 INFO:tasks.workunit.client.0.smithi119.stdout:3/51: read - db/fd zero size 2024-05-27T09:29:08.017 INFO:tasks.workunit.client.0.smithi119.stdout:3/52: symlink db/d10/l16 0 2024-05-27T09:29:08.017 INFO:tasks.workunit.client.0.smithi119.stdout:3/53: rename fa to db/d10/f17 0 2024-05-27T09:29:08.018 INFO:tasks.workunit.client.0.smithi119.stdout:7/45: write f4 [4089793,100522] 0 2024-05-27T09:29:08.019 INFO:tasks.workunit.client.0.smithi119.stdout:7/46: creat d7/f9 x:0 0 0 2024-05-27T09:29:08.019 INFO:tasks.workunit.client.0.smithi119.stdout:7/47: dread - d7/f9 zero size 2024-05-27T09:29:08.019 INFO:tasks.workunit.client.0.smithi119.stdout:7/48: chown d7/f9 1158890 1 2024-05-27T09:29:08.019 INFO:tasks.workunit.client.0.smithi119.stdout:7/49: creat d7/fa x:0 0 0 2024-05-27T09:29:08.023 INFO:tasks.workunit.client.0.smithi119.stdout:7/50: mkdir d7/db 0 2024-05-27T09:29:08.023 INFO:tasks.workunit.client.0.smithi119.stdout:3/54: getdents db/d10 0 2024-05-27T09:29:08.025 INFO:tasks.workunit.client.0.smithi119.stdout:7/51: unlink l5 0 2024-05-27T09:29:08.026 INFO:tasks.workunit.client.0.smithi119.stdout:3/55: truncate f0 6528976 0 2024-05-27T09:29:08.027 INFO:tasks.workunit.client.0.smithi119.stdout:7/52: mknod d7/db/cc 0 2024-05-27T09:29:08.027 INFO:tasks.workunit.client.0.smithi119.stdout:7/53: dread - d7/f9 zero size 2024-05-27T09:29:08.028 INFO:tasks.workunit.client.0.smithi119.stdout:7/54: rmdir d7/db 39 2024-05-27T09:29:08.028 INFO:tasks.workunit.client.0.smithi119.stdout:7/55: creat d7/fd x:0 0 0 2024-05-27T09:29:08.031 INFO:tasks.workunit.client.0.smithi119.stdout:7/56: link f4 d7/db/fe 0 2024-05-27T09:29:08.049 INFO:tasks.workunit.client.0.smithi119.stdout:3/56: dread f0 [0,4194304] 0 2024-05-27T09:29:08.051 INFO:tasks.workunit.client.0.smithi119.stdout:3/57: link db/d10/d13/l15 db/l18 0 2024-05-27T09:29:08.053 INFO:tasks.workunit.client.0.smithi119.stdout:3/58: rename db/fd to db/d10/d13/f19 0 2024-05-27T09:29:08.053 INFO:tasks.workunit.client.0.smithi119.stdout:3/59: fsync f8 0 2024-05-27T09:29:08.053 INFO:tasks.workunit.client.0.smithi119.stdout:3/60: chown f6 1879 1 2024-05-27T09:29:08.053 INFO:tasks.workunit.client.0.smithi119.stdout:3/61: fsync f6 0 2024-05-27T09:29:08.056 INFO:tasks.workunit.client.0.smithi119.stdout:3/62: unlink f0 0 2024-05-27T09:29:08.085 INFO:tasks.workunit.client.0.smithi119.stdout:7/57: dwrite f4 [0,4194304] 0 2024-05-27T09:29:08.085 INFO:tasks.workunit.client.0.smithi119.stdout:7/58: write f4 [4609613,130713] 0 2024-05-27T09:29:08.094 INFO:tasks.workunit.client.0.smithi119.stdout:7/59: mkdir d7/df 0 2024-05-27T09:29:08.109 INFO:tasks.workunit.client.0.smithi119.stdout:7/60: write f4 [1184741,18792] 0 2024-05-27T09:29:08.111 INFO:tasks.workunit.client.0.smithi119.stdout:7/61: creat d7/df/f10 x:0 0 0 2024-05-27T09:29:08.138 INFO:tasks.workunit.client.0.smithi119.stdout:3/63: dwrite db/f12 [0,4194304] 0 2024-05-27T09:29:08.138 INFO:tasks.workunit.client.0.smithi119.stdout:3/64: fdatasync db/d10/f17 0 2024-05-27T09:29:08.141 INFO:tasks.workunit.client.0.smithi119.stdout:3/65: creat db/d10/f1a x:0 0 0 2024-05-27T09:29:08.143 INFO:tasks.workunit.client.0.smithi119.stdout:3/66: unlink f1 0 2024-05-27T09:29:08.167 INFO:tasks.workunit.client.0.smithi119.stdout:7/62: dwrite f4 [0,4194304] 0 2024-05-27T09:29:08.167 INFO:tasks.workunit.client.0.smithi119.stdout:7/63: truncate d7/df/f10 197436 0 2024-05-27T09:29:08.168 INFO:tasks.workunit.client.0.smithi119.stdout:7/64: mkdir d7/df/d11 0 2024-05-27T09:29:08.170 INFO:tasks.workunit.client.0.smithi119.stdout:7/65: symlink d7/df/d11/l12 0 2024-05-27T09:29:08.173 INFO:tasks.workunit.client.0.smithi119.stdout:7/66: rename d7/db/fe to d7/df/f13 0 2024-05-27T09:29:08.173 INFO:tasks.workunit.client.0.smithi119.stdout:7/67: readlink d7/df/d11/l12 0 2024-05-27T09:29:08.175 INFO:tasks.workunit.client.0.smithi119.stdout:7/68: creat d7/df/d11/f14 x:0 0 0 2024-05-27T09:29:08.175 INFO:tasks.workunit.client.0.smithi119.stdout:7/69: truncate d7/f9 635423 0 2024-05-27T09:29:08.177 INFO:tasks.workunit.client.0.smithi119.stdout:7/70: mknod d7/c15 0 2024-05-27T09:29:08.177 INFO:tasks.workunit.client.0.smithi119.stdout:7/71: fdatasync d7/df/f13 0 2024-05-27T09:29:08.177 INFO:tasks.workunit.client.0.smithi119.stdout:7/72: truncate d7/f9 1623787 0 2024-05-27T09:29:08.205 INFO:tasks.workunit.client.0.smithi119.stdout:3/67: dwrite db/fc [0,4194304] 0 2024-05-27T09:29:08.205 INFO:tasks.workunit.client.0.smithi119.stdout:3/68: dread - db/d10/d13/f19 zero size 2024-05-27T09:29:08.218 INFO:tasks.workunit.client.0.smithi119.stdout:3/69: truncate db/fc 2665632 0 2024-05-27T09:29:08.221 INFO:tasks.workunit.client.0.smithi119.stdout:3/70: fsync db/fc 0 2024-05-27T09:29:08.224 INFO:tasks.workunit.client.0.smithi119.stdout:3/71: truncate f6 773365 0 2024-05-27T09:29:08.266 INFO:tasks.workunit.client.0.smithi119.stdout:7/73: dwrite d7/f9 [0,4194304] 0 2024-05-27T09:29:08.273 INFO:tasks.workunit.client.0.smithi119.stdout:7/74: rename d7/c8 to d7/df/c16 0 2024-05-27T09:29:08.275 INFO:tasks.workunit.client.0.smithi119.stdout:7/75: symlink d7/df/d11/l17 0 2024-05-27T09:29:08.277 INFO:tasks.workunit.client.0.smithi119.stdout:7/76: mknod d7/c18 0 2024-05-27T09:29:08.277 INFO:tasks.workunit.client.0.smithi119.stdout:7/77: dread - d7/fd zero size 2024-05-27T09:29:08.286 INFO:tasks.workunit.client.0.smithi119.stdout:7/78: rename d7/df/d11 to d7/df/d19 0 2024-05-27T09:29:08.295 INFO:tasks.workunit.client.0.smithi119.stdout:3/72: dwrite db/d10/f17 [0,4194304] 0 2024-05-27T09:29:08.312 INFO:tasks.workunit.client.0.smithi119.stdout:3/73: dread f6 [0,4194304] 0 2024-05-27T09:29:08.312 INFO:tasks.workunit.client.0.smithi119.stdout:3/74: readlink db/d10/l14 0 2024-05-27T09:29:08.314 INFO:tasks.workunit.client.0.smithi119.stdout:7/79: dread f1 [4194304,4194304] 0 2024-05-27T09:29:08.314 INFO:tasks.workunit.client.0.smithi119.stdout:3/75: link db/d10/l14 db/l1b 0 2024-05-27T09:29:08.317 INFO:tasks.workunit.client.0.smithi119.stdout:7/80: mkdir d7/d1a 0 2024-05-27T09:29:08.319 INFO:tasks.workunit.client.0.smithi119.stdout:3/76: link db/d10/l16 db/l1c 0 2024-05-27T09:29:08.319 INFO:tasks.workunit.client.0.smithi119.stdout:3/77: rename db to db/d1d 22 2024-05-27T09:29:08.320 INFO:tasks.workunit.client.0.smithi119.stdout:7/81: creat d7/df/d19/f1b x:0 0 0 2024-05-27T09:29:08.322 INFO:tasks.workunit.client.0.smithi119.stdout:7/82: creat d7/d1a/f1c x:0 0 0 2024-05-27T09:29:08.322 INFO:tasks.workunit.client.0.smithi119.stdout:7/83: dread - d7/fd zero size 2024-05-27T09:29:08.323 INFO:tasks.workunit.client.0.smithi119.stdout:7/84: mkdir d7/df/d19/d1d 0 2024-05-27T09:29:08.324 INFO:tasks.workunit.client.0.smithi119.stdout:7/85: write d7/fa [73290,123676] 0 2024-05-27T09:29:08.324 INFO:tasks.workunit.client.0.smithi119.stdout:7/86: truncate d7/df/d19/f1b 912360 0 2024-05-27T09:29:08.324 INFO:tasks.workunit.client.0.smithi119.stdout:7/87: readlink d7/df/d19/l17 0 2024-05-27T09:29:08.337 INFO:tasks.workunit.client.0.smithi119.stdout:3/78: dread db/fc [0,4194304] 0 2024-05-27T09:29:08.339 INFO:tasks.workunit.client.0.smithi119.stdout:3/79: mknod db/d10/c1e 0 2024-05-27T09:29:08.345 INFO:tasks.workunit.client.0.smithi119.stdout:7/88: dread f4 [0,4194304] 0 2024-05-27T09:29:08.345 INFO:tasks.workunit.client.0.smithi119.stdout:7/89: stat d7/db/cc 0 2024-05-27T09:29:08.349 INFO:tasks.workunit.client.0.smithi119.stdout:7/90: creat d7/df/d19/d1d/f1e x:0 0 0 2024-05-27T09:29:08.358 INFO:tasks.workunit.client.0.smithi119.stdout:7/91: rmdir d7/d1a 39 2024-05-27T09:29:08.358 INFO:tasks.workunit.client.0.smithi119.stdout:7/92: read - d7/d1a/f1c zero size 2024-05-27T09:29:08.358 INFO:tasks.workunit.client.0.smithi119.stdout:7/93: creat d7/df/d19/f1f x:0 0 0 2024-05-27T09:29:08.358 INFO:tasks.workunit.client.0.smithi119.stdout:7/94: chown d7/df/d19/d1d/f1e 5 1 2024-05-27T09:29:08.358 INFO:tasks.workunit.client.0.smithi119.stdout:7/95: chown d7/df/d19/l17 32487 1 2024-05-27T09:29:08.362 INFO:tasks.workunit.client.0.smithi119.stdout:7/96: rename d7/f9 to d7/d1a/f20 0 2024-05-27T09:29:08.364 INFO:tasks.workunit.client.0.smithi119.stdout:7/97: link c2 d7/df/c21 0 2024-05-27T09:29:08.366 INFO:tasks.workunit.client.0.smithi119.stdout:7/98: creat d7/db/f22 x:0 0 0 2024-05-27T09:29:08.366 INFO:tasks.workunit.client.0.smithi119.stdout:7/99: chown d7/df/d19/d1d 748 1 2024-05-27T09:29:08.410 INFO:tasks.workunit.client.0.smithi119.stdout:7/100: dwrite d7/df/d19/f14 [0,4194304] 0 2024-05-27T09:29:08.461 INFO:tasks.workunit.client.0.smithi119.stdout:7/101: dwrite d7/df/f13 [4194304,4194304] 0 2024-05-27T09:29:08.461 INFO:tasks.workunit.client.0.smithi119.stdout:7/102: readlink d7/df/d19/l12 0 2024-05-27T09:29:08.461 INFO:tasks.workunit.client.0.smithi119.stdout:7/103: chown d7/df/d19/l17 401 1 2024-05-27T09:29:08.523 INFO:tasks.workunit.client.0.smithi119.stdout:7/104: dwrite d7/fd [0,4194304] 0 2024-05-27T09:29:08.542 INFO:tasks.workunit.client.0.smithi119.stdout:7/105: write d7/fd [1508214,42833] 0 2024-05-27T09:29:08.542 INFO:tasks.workunit.client.0.smithi119.stdout:7/106: fsync d7/df/d19/f1b 0 2024-05-27T09:29:08.542 INFO:tasks.workunit.client.0.smithi119.stdout:7/107: chown d7/df/d19 230933 1 2024-05-27T09:29:08.542 INFO:tasks.workunit.client.0.smithi119.stdout:7/108: creat d7/df/d19/d1d/f23 x:0 0 0 2024-05-27T09:29:08.546 INFO:tasks.workunit.client.0.smithi119.stdout:7/109: mknod d7/d1a/c24 0 2024-05-27T09:29:08.548 INFO:tasks.workunit.client.0.smithi119.stdout:7/110: rename d7/c15 to d7/df/d19/d1d/c25 0 2024-05-27T09:29:08.552 INFO:tasks.workunit.client.0.smithi119.stdout:7/111: link d7/df/d19/l17 d7/df/l26 0 2024-05-27T09:29:08.572 INFO:tasks.workunit.client.0.smithi119.stdout:7/112: dread d7/df/f13 [0,4194304] 0 2024-05-27T09:29:08.573 INFO:tasks.workunit.client.0.smithi119.stdout:7/113: stat d7/df/c21 0 2024-05-27T09:29:08.573 INFO:tasks.workunit.client.0.smithi119.stdout:7/114: chown d7/d1a/f1c 23 1 2024-05-27T09:29:08.585 INFO:tasks.workunit.client.0.smithi119.stdout:7/115: dread d7/df/f10 [0,4194304] 0 2024-05-27T09:29:08.589 INFO:tasks.workunit.client.0.smithi119.stdout:7/116: symlink d7/l27 0 2024-05-27T09:29:08.592 INFO:tasks.workunit.client.0.smithi119.stdout:7/117: read d7/df/d19/f1b [514942,46681] 0 2024-05-27T09:29:08.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:08 smithi138 ceph-mon[34885]: Standby manager daemon smithi138.dxsuqf restarted 2024-05-27T09:29:08.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:08 smithi138 ceph-mon[34885]: Standby manager daemon smithi138.dxsuqf started 2024-05-27T09:29:08.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:08 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.dxsuqf/crt"}]: dispatch 2024-05-27T09:29:08.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:08 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T09:29:08.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:08 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.dxsuqf/key"}]: dispatch 2024-05-27T09:29:08.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:08 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T09:29:08.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:08 smithi119 ceph-mon[29082]: Standby manager daemon smithi138.dxsuqf restarted 2024-05-27T09:29:08.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:08 smithi119 ceph-mon[29082]: Standby manager daemon smithi138.dxsuqf started 2024-05-27T09:29:08.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:08 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.dxsuqf/crt"}]: dispatch 2024-05-27T09:29:08.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:08 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T09:29:08.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:08 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.dxsuqf/key"}]: dispatch 2024-05-27T09:29:08.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:08 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T09:29:09.030 INFO:tasks.workunit.client.0.smithi119.stdout:7/118: mknod d7/df/d19/d1d/c28 0 2024-05-27T09:29:09.030 INFO:tasks.workunit.client.0.smithi119.stdout:7/119: stat d7/df/f13 0 2024-05-27T09:29:09.030 INFO:tasks.workunit.client.0.smithi119.stdout:7/120: creat d7/df/f29 x:0 0 0 2024-05-27T09:29:09.033 INFO:tasks.workunit.client.0.smithi119.stdout:7/121: getdents d7/d1a 0 2024-05-27T09:29:09.033 INFO:tasks.workunit.client.0.smithi119.stdout:7/122: stat d7/fd 0 2024-05-27T09:29:09.034 INFO:tasks.workunit.client.0.smithi119.stdout:7/123: creat d7/df/d19/d1d/f2a x:0 0 0 2024-05-27T09:29:09.034 INFO:tasks.workunit.client.0.smithi119.stdout:7/124: readlink d7/df/d19/l17 0 2024-05-27T09:29:09.037 INFO:tasks.workunit.client.0.smithi119.stdout:7/125: getdents d7/db 0 2024-05-27T09:29:09.146 INFO:tasks.workunit.client.0.smithi119.stdout:7/126: dwrite d7/fa [0,4194304] 0 2024-05-27T09:29:09.150 INFO:tasks.workunit.client.0.smithi119.stdout:7/127: creat d7/d1a/f2b x:0 0 0 2024-05-27T09:29:09.152 INFO:tasks.workunit.client.0.smithi119.stdout:7/128: rename d7/df/c21 to d7/df/d19/d1d/c2c 0 2024-05-27T09:29:09.166 INFO:tasks.workunit.client.0.smithi119.stdout:7/129: write d7/df/f13 [2596904,38669] 0 2024-05-27T09:29:09.188 INFO:tasks.workunit.client.0.smithi119.stdout:7/130: write f1 [1870202,6576] 0 2024-05-27T09:29:09.213 INFO:tasks.workunit.client.0.smithi119.stdout:7/131: write d7/df/f13 [7979268,80624] 0 2024-05-27T09:29:09.249 INFO:tasks.workunit.client.0.smithi119.stdout:7/132: rmdir d7/df 39 2024-05-27T09:29:09.328 INFO:tasks.workunit.client.0.smithi119.stdout:7/133: dwrite d7/df/f13 [4194304,4194304] 0 2024-05-27T09:29:09.328 INFO:tasks.workunit.client.0.smithi119.stdout:7/134: creat d7/d1a/f2d x:0 0 0 2024-05-27T09:29:09.331 INFO:tasks.workunit.client.0.smithi119.stdout:7/135: rmdir d7/db 39 2024-05-27T09:29:09.332 INFO:tasks.workunit.client.0.smithi119.stdout:7/136: symlink d7/df/d19/d1d/l2e 0 2024-05-27T09:29:09.334 INFO:tasks.workunit.client.0.smithi119.stdout:7/137: link d7/df/f10 d7/db/f2f 0 2024-05-27T09:29:09.334 INFO:tasks.workunit.client.0.smithi119.stdout:7/138: dread - d7/d1a/f1c zero size 2024-05-27T09:29:09.336 INFO:tasks.workunit.client.0.smithi119.stdout:7/139: mknod d7/df/d19/d1d/c30 0 2024-05-27T09:29:09.336 INFO:tasks.workunit.client.0.smithi119.stdout:7/140: fdatasync d7/df/f29 0 2024-05-27T09:29:09.338 INFO:tasks.workunit.client.0.smithi119.stdout:7/141: rmdir d7/df 39 2024-05-27T09:29:09.341 INFO:tasks.workunit.client.0.smithi119.stdout:7/142: creat d7/db/f31 x:0 0 0 2024-05-27T09:29:09.343 INFO:tasks.workunit.client.0.smithi119.stdout:7/143: mknod d7/db/c32 0 2024-05-27T09:29:09.343 INFO:tasks.workunit.client.0.smithi119.stdout:7/144: chown d7/df/f13 124 1 2024-05-27T09:29:09.356 INFO:tasks.workunit.client.0.smithi119.stdout:1/32: sync 2024-05-27T09:29:09.356 INFO:tasks.workunit.client.0.smithi119.stdout:1/33: chown c1 1 1 2024-05-27T09:29:09.356 INFO:tasks.workunit.client.0.smithi119.stdout:2/1: sync 2024-05-27T09:29:09.356 INFO:tasks.workunit.client.0.smithi119.stdout:1/34: dread - f6 zero size 2024-05-27T09:29:09.356 INFO:tasks.workunit.client.0.smithi119.stdout:2/2: fdatasync - no filename 2024-05-27T09:29:09.356 INFO:tasks.workunit.client.0.smithi119.stdout:2/3: truncate - no filename 2024-05-27T09:29:09.356 INFO:tasks.workunit.client.0.smithi119.stdout:2/4: rename - no filename 2024-05-27T09:29:09.356 INFO:tasks.workunit.client.0.smithi119.stdout:9/4: sync 2024-05-27T09:29:09.356 INFO:tasks.workunit.client.0.smithi119.stdout:9/5: dread - no filename 2024-05-27T09:29:09.356 INFO:tasks.workunit.client.0.smithi119.stdout:9/6: link - no file 2024-05-27T09:29:09.356 INFO:tasks.workunit.client.0.smithi119.stdout:8/14: sync 2024-05-27T09:29:09.357 INFO:tasks.workunit.client.0.smithi119.stdout:0/26: sync 2024-05-27T09:29:09.357 INFO:tasks.workunit.client.0.smithi119.stdout:3/80: sync 2024-05-27T09:29:09.357 INFO:tasks.workunit.client.0.smithi119.stdout:6/28: sync 2024-05-27T09:29:09.357 INFO:tasks.workunit.client.0.smithi119.stdout:5/26: sync 2024-05-27T09:29:09.357 INFO:tasks.workunit.client.0.smithi119.stdout:0/27: write d2/d6/f8 [599015,91877] 0 2024-05-27T09:29:09.357 INFO:tasks.workunit.client.0.smithi119.stdout:0/28: chown d2/d6 49183935 1 2024-05-27T09:29:09.357 INFO:tasks.workunit.client.0.smithi119.stdout:4/19: sync 2024-05-27T09:29:09.362 INFO:tasks.workunit.client.0.smithi119.stdout:8/15: write f1 [1956675,15946] 0 2024-05-27T09:29:09.367 INFO:tasks.workunit.client.0.smithi119.stdout:7/145: dread d7/fd [0,4194304] 0 2024-05-27T09:29:09.367 INFO:tasks.workunit.client.0.smithi119.stdout:2/5: symlink l0 0 2024-05-27T09:29:09.367 INFO:tasks.workunit.client.0.smithi119.stdout:9/7: creat f0 x:0 0 0 2024-05-27T09:29:09.367 INFO:tasks.workunit.client.0.smithi119.stdout:9/8: write f0 [418977,11226] 0 2024-05-27T09:29:09.367 INFO:tasks.workunit.client.0.smithi119.stdout:7/146: creat d7/f33 x:0 0 0 2024-05-27T09:29:09.367 INFO:tasks.workunit.client.0.smithi119.stdout:1/35: mkdir d7 0 2024-05-27T09:29:09.371 INFO:tasks.workunit.client.0.smithi119.stdout:6/29: unlink d0/d2/c8 0 2024-05-27T09:29:09.377 INFO:tasks.workunit.client.0.smithi119.stdout:3/81: link db/d10/d13/l15 db/d10/d13/l1f 0 2024-05-27T09:29:09.381 INFO:tasks.workunit.client.0.smithi119.stdout:4/20: rename f2 to d4/f8 0 2024-05-27T09:29:09.381 INFO:tasks.workunit.client.0.smithi119.stdout:4/21: write f1 [1533375,93960] 0 2024-05-27T09:29:09.385 INFO:tasks.workunit.client.0.smithi119.stdout:8/16: creat d3/f4 x:0 0 0 2024-05-27T09:29:09.388 INFO:tasks.workunit.client.0.smithi119.stdout:2/6: mknod c1 0 2024-05-27T09:29:09.399 INFO:tasks.workunit.client.0.smithi119.stdout:7/147: symlink d7/df/d19/l34 0 2024-05-27T09:29:09.399 INFO:tasks.workunit.client.0.smithi119.stdout:1/36: creat d7/f8 x:0 0 0 2024-05-27T09:29:09.400 INFO:tasks.workunit.client.0.smithi119.stdout:2/7: mkdir d2 0 2024-05-27T09:29:09.400 INFO:tasks.workunit.client.0.smithi119.stdout:2/8: truncate - no filename 2024-05-27T09:29:09.400 INFO:tasks.workunit.client.0.smithi119.stdout:2/9: dread - no filename 2024-05-27T09:29:09.404 INFO:tasks.workunit.client.0.smithi119.stdout:2/10: link l0 d2/l3 0 2024-05-27T09:29:09.404 INFO:tasks.workunit.client.0.smithi119.stdout:2/11: read - no filename 2024-05-27T09:29:09.405 INFO:tasks.workunit.client.0.smithi119.stdout:2/12: chown c1 0 1 2024-05-27T09:29:09.405 INFO:tasks.workunit.client.0.smithi119.stdout:2/13: dread - no filename 2024-05-27T09:29:09.405 INFO:tasks.workunit.client.0.smithi119.stdout:2/14: dwrite - no filename 2024-05-27T09:29:09.408 INFO:tasks.workunit.client.0.smithi119.stdout:2/15: mknod d2/c4 0 2024-05-27T09:29:09.408 INFO:tasks.workunit.client.0.smithi119.stdout:2/16: truncate - no filename 2024-05-27T09:29:09.408 INFO:tasks.workunit.client.0.smithi119.stdout:2/17: dwrite - no filename 2024-05-27T09:29:09.412 INFO:tasks.workunit.client.0.smithi119.stdout:2/18: mkdir d2/d5 0 2024-05-27T09:29:09.413 INFO:tasks.workunit.client.0.smithi119.stdout:6/30: write d0/f7 [4130349,18041] 0 2024-05-27T09:29:09.421 INFO:tasks.workunit.client.0.smithi119.stdout:2/19: creat d2/f6 x:0 0 0 2024-05-27T09:29:09.435 INFO:tasks.workunit.client.0.smithi119.stdout:1/37: dread f2 [0,4194304] 0 2024-05-27T09:29:09.438 INFO:tasks.workunit.client.0.smithi119.stdout:1/38: write f5 [74081,49201] 0 2024-05-27T09:29:09.438 INFO:tasks.workunit.client.0.smithi119.stdout:8/17: dread f0 [0,4194304] 0 2024-05-27T09:29:09.438 INFO:tasks.workunit.client.0.smithi119.stdout:8/18: chown f2 12784 1 2024-05-27T09:29:09.448 INFO:tasks.workunit.client.0.smithi119.stdout:1/39: mknod d7/c9 0 2024-05-27T09:29:09.448 INFO:tasks.workunit.client.0.smithi119.stdout:8/19: rename d3/f4 to d3/f5 0 2024-05-27T09:29:09.449 INFO:tasks.workunit.client.0.smithi119.stdout:8/20: truncate d3/f5 298360 0 2024-05-27T09:29:09.451 INFO:tasks.workunit.client.0.smithi119.stdout:9/9: dwrite f0 [0,4194304] 0 2024-05-27T09:29:09.459 INFO:tasks.workunit.client.0.smithi119.stdout:8/21: dread f1 [0,4194304] 0 2024-05-27T09:29:09.459 INFO:tasks.workunit.client.0.smithi119.stdout:8/22: stat d3 0 2024-05-27T09:29:09.459 INFO:tasks.workunit.client.0.smithi119.stdout:8/23: truncate f2 766179 0 2024-05-27T09:29:09.464 INFO:tasks.workunit.client.0.smithi119.stdout:8/24: mknod d3/c6 0 2024-05-27T09:29:09.472 INFO:tasks.workunit.client.0.smithi119.stdout:8/25: mknod d3/c7 0 2024-05-27T09:29:09.483 INFO:tasks.workunit.client.0.smithi119.stdout:7/148: dread f1 [0,4194304] 0 2024-05-27T09:29:09.485 INFO:tasks.workunit.client.0.smithi119.stdout:7/149: symlink d7/d1a/l35 0 2024-05-27T09:29:09.490 INFO:tasks.workunit.client.0.smithi119.stdout:3/82: dwrite db/d10/f17 [0,4194304] 0 2024-05-27T09:29:09.492 INFO:tasks.workunit.client.0.smithi119.stdout:3/83: rmdir db/d10/d13 39 2024-05-27T09:29:09.492 INFO:tasks.workunit.client.0.smithi119.stdout:3/84: truncate db/d10/d13/f19 697679 0 2024-05-27T09:29:09.493 INFO:tasks.workunit.client.0.smithi119.stdout:3/85: stat c4 0 2024-05-27T09:29:09.493 INFO:tasks.workunit.client.0.smithi119.stdout:3/86: dread - db/d10/f1a zero size 2024-05-27T09:29:09.496 INFO:tasks.workunit.client.0.smithi119.stdout:3/87: write f8 [67014,60603] 0 2024-05-27T09:29:09.496 INFO:tasks.workunit.client.0.smithi119.stdout:3/88: read - db/d10/f1a zero size 2024-05-27T09:29:09.498 INFO:tasks.workunit.client.0.smithi119.stdout:3/89: mknod db/c20 0 2024-05-27T09:29:09.498 INFO:tasks.workunit.client.0.smithi119.stdout:3/90: readlink db/d10/l14 0 2024-05-27T09:29:09.498 INFO:tasks.workunit.client.0.smithi119.stdout:3/91: write db/fc [3576980,32927] 0 2024-05-27T09:29:09.505 INFO:tasks.workunit.client.0.smithi119.stdout:3/92: write f6 [215965,34627] 0 2024-05-27T09:29:09.505 INFO:tasks.workunit.client.0.smithi119.stdout:3/93: dread - db/d10/f1a zero size 2024-05-27T09:29:09.508 INFO:tasks.workunit.client.0.smithi119.stdout:3/94: creat db/d10/d13/f21 x:0 0 0 2024-05-27T09:29:09.508 INFO:tasks.workunit.client.0.smithi119.stdout:3/95: truncate db/f12 4982701 0 2024-05-27T09:29:09.508 INFO:tasks.workunit.client.0.smithi119.stdout:3/96: creat db/f22 x:0 0 0 2024-05-27T09:29:09.508 INFO:tasks.workunit.client.0.smithi119.stdout:3/97: write f6 [382296,5301] 0 2024-05-27T09:29:09.508 INFO:tasks.workunit.client.0.smithi119.stdout:3/98: truncate db/d10/d13/f21 342341 0 2024-05-27T09:29:09.509 INFO:tasks.workunit.client.0.smithi119.stdout:4/22: dwrite f1 [0,4194304] 0 2024-05-27T09:29:09.509 INFO:tasks.workunit.client.0.smithi119.stdout:4/23: fsync f1 0 2024-05-27T09:29:09.509 INFO:tasks.workunit.client.0.smithi119.stdout:4/24: dread - f3 zero size 2024-05-27T09:29:09.514 INFO:tasks.workunit.client.0.smithi119.stdout:5/27: dwrite f0 [0,4194304] 0 2024-05-27T09:29:09.515 INFO:tasks.workunit.client.0.smithi119.stdout:4/25: symlink d4/d6/l9 0 2024-05-27T09:29:09.519 INFO:tasks.workunit.client.0.smithi119.stdout:5/28: creat d6/f8 x:0 0 0 2024-05-27T09:29:09.519 INFO:tasks.workunit.client.0.smithi119.stdout:5/29: fsync f0 0 2024-05-27T09:29:09.521 INFO:tasks.workunit.client.0.smithi119.stdout:6/31: dwrite d0/f7 [0,4194304] 0 2024-05-27T09:29:09.521 INFO:tasks.workunit.client.0.smithi119.stdout:4/26: creat d4/d6/fa x:0 0 0 2024-05-27T09:29:09.521 INFO:tasks.workunit.client.0.smithi119.stdout:1/40: dwrite f3 [0,4194304] 0 2024-05-27T09:29:09.523 INFO:tasks.workunit.client.0.smithi119.stdout:9/10: getdents . 0 2024-05-27T09:29:09.528 INFO:tasks.workunit.client.0.smithi119.stdout:0/29: dwrite d2/d6/f8 [0,4194304] 0 2024-05-27T09:29:09.532 INFO:tasks.workunit.client.0.smithi119.stdout:8/26: dwrite f0 [0,4194304] 0 2024-05-27T09:29:09.535 INFO:tasks.workunit.client.0.smithi119.stdout:4/27: mknod d4/d6/cb 0 2024-05-27T09:29:09.535 INFO:tasks.workunit.client.0.smithi119.stdout:2/20: dwrite d2/f6 [0,4194304] 0 2024-05-27T09:29:09.536 INFO:tasks.workunit.client.0.smithi119.stdout:1/41: rename f5 to d7/fa 0 2024-05-27T09:29:09.549 INFO:tasks.workunit.client.0.smithi119.stdout:8/27: mkdir d3/d8 0 2024-05-27T09:29:09.549 INFO:tasks.workunit.client.0.smithi119.stdout:8/28: fdatasync f0 0 2024-05-27T09:29:09.549 INFO:tasks.workunit.client.0.smithi119.stdout:8/29: chown f1 6759729 1 2024-05-27T09:29:09.550 INFO:tasks.workunit.client.0.smithi119.stdout:9/11: dread f0 [0,4194304] 0 2024-05-27T09:29:09.550 INFO:tasks.workunit.client.0.smithi119.stdout:5/30: getdents d6 0 2024-05-27T09:29:09.552 INFO:tasks.workunit.client.0.smithi119.stdout:2/21: rmdir d2/d5 0 2024-05-27T09:29:09.553 INFO:tasks.workunit.client.0.smithi119.stdout:1/42: mkdir d7/db 0 2024-05-27T09:29:09.553 INFO:tasks.workunit.client.0.smithi119.stdout:1/43: fsync f6 0 2024-05-27T09:29:09.555 INFO:tasks.workunit.client.0.smithi119.stdout:5/31: creat d6/f9 x:0 0 0 2024-05-27T09:29:09.557 INFO:tasks.workunit.client.0.smithi119.stdout:1/44: symlink d7/db/lc 0 2024-05-27T09:29:09.557 INFO:tasks.workunit.client.0.smithi119.stdout:1/45: chown d7/f8 160902 1 2024-05-27T09:29:09.558 INFO:tasks.workunit.client.0.smithi119.stdout:9/12: creat f1 x:0 0 0 2024-05-27T09:29:09.562 INFO:tasks.workunit.client.0.smithi119.stdout:5/32: rename c5 to d6/ca 0 2024-05-27T09:29:09.565 INFO:tasks.workunit.client.0.smithi119.stdout:1/46: mkdir d7/dd 0 2024-05-27T09:29:09.565 INFO:tasks.workunit.client.0.smithi119.stdout:8/30: write f1 [2514599,114472] 0 2024-05-27T09:29:09.569 INFO:tasks.workunit.client.0.smithi119.stdout:5/33: mkdir d6/db 0 2024-05-27T09:29:09.573 INFO:tasks.workunit.client.0.smithi119.stdout:8/31: symlink d3/l9 0 2024-05-27T09:29:09.573 INFO:tasks.workunit.client.0.smithi119.stdout:1/47: truncate f2 2872777 0 2024-05-27T09:29:09.574 INFO:tasks.workunit.client.0.smithi119.stdout:5/34: mkdir d6/db/dc 0 2024-05-27T09:29:09.577 INFO:tasks.workunit.client.0.smithi119.stdout:8/32: link f0 d3/fa 0 2024-05-27T09:29:09.577 INFO:tasks.workunit.client.0.smithi119.stdout:1/48: symlink d7/dd/le 0 2024-05-27T09:29:09.578 INFO:tasks.workunit.client.0.smithi119.stdout:5/35: mkdir d6/dd 0 2024-05-27T09:29:09.585 INFO:tasks.workunit.client.0.smithi119.stdout:5/36: mkdir d6/db/de 0 2024-05-27T09:29:09.587 INFO:tasks.workunit.client.0.smithi119.stdout:9/13: truncate f0 3224357 0 2024-05-27T09:29:09.587 INFO:tasks.workunit.client.0.smithi119.stdout:9/14: dread - f1 zero size 2024-05-27T09:29:09.588 INFO:tasks.workunit.client.0.smithi119.stdout:9/15: write f1 [487674,6602] 0 2024-05-27T09:29:09.589 INFO:tasks.workunit.client.0.smithi119.stdout:5/37: mkdir d6/dd/df 0 2024-05-27T09:29:09.590 INFO:tasks.workunit.client.0.smithi119.stdout:9/16: read f1 [454832,18015] 0 2024-05-27T09:29:09.593 INFO:tasks.workunit.client.0.smithi119.stdout:5/38: truncate f0 135997 0 2024-05-27T09:29:09.593 INFO:tasks.workunit.client.0.smithi119.stdout:5/39: dread - d6/f9 zero size 2024-05-27T09:29:09.593 INFO:tasks.workunit.client.0.smithi119.stdout:5/40: stat d6/db/dc 0 2024-05-27T09:29:09.593 INFO:tasks.workunit.client.0.smithi119.stdout:5/41: fsync d6/f8 0 2024-05-27T09:29:09.596 INFO:tasks.workunit.client.0.smithi119.stdout:5/42: stat d6/ca 0 2024-05-27T09:29:09.602 INFO:tasks.workunit.client.0.smithi119.stdout:1/49: rmdir d7 39 2024-05-27T09:29:09.610 INFO:tasks.workunit.client.0.smithi119.stdout:0/30: dwrite d2/d6/f8 [0,4194304] 0 2024-05-27T09:29:09.612 INFO:tasks.workunit.client.0.smithi119.stdout:9/17: unlink f0 0 2024-05-27T09:29:09.613 INFO:tasks.workunit.client.0.smithi119.stdout:7/150: dwrite d7/df/d19/d1d/f23 [0,4194304] 0 2024-05-27T09:29:09.613 INFO:tasks.workunit.client.0.smithi119.stdout:7/151: dread - d7/d1a/f2b zero size 2024-05-27T09:29:09.614 INFO:tasks.workunit.client.0.smithi119.stdout:0/31: mknod d2/d4/ca 0 2024-05-27T09:29:09.614 INFO:tasks.workunit.client.0.smithi119.stdout:0/32: chown d2/d4/ca 20459974 1 2024-05-27T09:29:09.616 INFO:tasks.workunit.client.0.smithi119.stdout:3/99: dwrite f8 [0,4194304] 0 2024-05-27T09:29:09.619 INFO:tasks.workunit.client.0.smithi119.stdout:4/28: dwrite f1 [0,4194304] 0 2024-05-27T09:29:09.619 INFO:tasks.workunit.client.0.smithi119.stdout:4/29: truncate d4/d6/fa 988056 0 2024-05-27T09:29:09.625 INFO:tasks.workunit.client.0.smithi119.stdout:4/30: readlink d4/d6/l9 0 2024-05-27T09:29:09.625 INFO:tasks.workunit.client.0.smithi119.stdout:1/50: write f2 [126237,31604] 0 2024-05-27T09:29:09.625 INFO:tasks.workunit.client.0.smithi119.stdout:1/51: truncate d7/f8 319586 0 2024-05-27T09:29:09.625 INFO:tasks.workunit.client.0.smithi119.stdout:1/52: stat c1 0 2024-05-27T09:29:09.625 INFO:tasks.workunit.client.0.smithi119.stdout:1/53: truncate f6 341081 0 2024-05-27T09:29:09.630 INFO:tasks.workunit.client.0.smithi119.stdout:4/31: dread d4/d6/fa [0,4194304] 0 2024-05-27T09:29:09.631 INFO:tasks.workunit.client.0.smithi119.stdout:4/32: dread - f3 zero size 2024-05-27T09:29:09.670 INFO:tasks.workunit.client.0.smithi119.stdout:0/33: dwrite d2/d6/f8 [4194304,4194304] 0 2024-05-27T09:29:09.671 INFO:tasks.workunit.client.0.smithi119.stdout:0/34: chown d2/d6/f8 27885347 1 2024-05-27T09:29:09.683 INFO:tasks.workunit.client.0.smithi119.stdout:5/43: dwrite d6/f9 [0,4194304] 0 2024-05-27T09:29:09.683 INFO:tasks.workunit.client.0.smithi119.stdout:5/44: fdatasync f1 0 2024-05-27T09:29:09.683 INFO:tasks.workunit.client.0.smithi119.stdout:5/45: fsync f0 0 2024-05-27T09:29:09.684 INFO:tasks.workunit.client.0.smithi119.stdout:2/22: dwrite d2/f6 [0,4194304] 0 2024-05-27T09:29:09.684 INFO:tasks.workunit.client.0.smithi119.stdout:2/23: readlink l0 0 2024-05-27T09:29:09.685 INFO:tasks.workunit.client.0.smithi119.stdout:2/24: stat d2/f6 0 2024-05-27T09:29:09.685 INFO:tasks.workunit.client.0.smithi119.stdout:2/25: creat d2/f7 x:0 0 0 2024-05-27T09:29:09.685 INFO:tasks.workunit.client.0.smithi119.stdout:2/26: fsync d2/f6 0 2024-05-27T09:29:09.689 INFO:tasks.workunit.client.0.smithi119.stdout:9/18: getdents . 0 2024-05-27T09:29:09.693 INFO:tasks.workunit.client.0.smithi119.stdout:5/46: mknod d6/c10 0 2024-05-27T09:29:09.693 INFO:tasks.workunit.client.0.smithi119.stdout:5/47: rename d6/dd/df to d6/dd/df/d11 22 2024-05-27T09:29:09.693 INFO:tasks.workunit.client.0.smithi119.stdout:5/48: creat d6/f12 x:0 0 0 2024-05-27T09:29:09.698 INFO:tasks.workunit.client.0.smithi119.stdout:9/19: unlink f1 0 2024-05-27T09:29:09.700 INFO:tasks.workunit.client.0.smithi119.stdout:3/100: dwrite db/f22 [0,4194304] 0 2024-05-27T09:29:09.700 INFO:tasks.workunit.client.0.smithi119.stdout:3/101: creat db/f23 x:0 0 0 2024-05-27T09:29:09.700 INFO:tasks.workunit.client.0.smithi119.stdout:3/102: creat db/f24 x:0 0 0 2024-05-27T09:29:09.700 INFO:tasks.workunit.client.0.smithi119.stdout:3/103: fdatasync db/d10/d13/f19 0 2024-05-27T09:29:09.700 INFO:tasks.workunit.client.0.smithi119.stdout:3/104: readlink db/d10/d13/l15 0 2024-05-27T09:29:09.701 INFO:tasks.workunit.client.0.smithi119.stdout:9/20: creat f2 x:0 0 0 2024-05-27T09:29:09.702 INFO:tasks.workunit.client.0.smithi119.stdout:5/49: dread f0 [0,4194304] 0 2024-05-27T09:29:09.702 INFO:tasks.workunit.client.0.smithi119.stdout:5/50: write f1 [5422883,36234] 0 2024-05-27T09:29:09.705 INFO:tasks.workunit.client.0.smithi119.stdout:2/27: rmdir d2 39 2024-05-27T09:29:09.709 INFO:tasks.workunit.client.0.smithi119.stdout:3/105: unlink db/d10/d13/f19 0 2024-05-27T09:29:09.709 INFO:tasks.workunit.client.0.smithi119.stdout:3/106: fdatasync f8 0 2024-05-27T09:29:09.716 INFO:tasks.workunit.client.0.smithi119.stdout:9/21: link f2 f3 0 2024-05-27T09:29:09.716 INFO:tasks.workunit.client.0.smithi119.stdout:9/22: read - f2 zero size 2024-05-27T09:29:09.717 INFO:tasks.workunit.client.0.smithi119.stdout:9/23: creat f4 x:0 0 0 2024-05-27T09:29:09.717 INFO:tasks.workunit.client.0.smithi119.stdout:9/24: write f2 [740850,80195] 0 2024-05-27T09:29:09.717 INFO:tasks.workunit.client.0.smithi119.stdout:7/152: dwrite d7/df/d19/d1d/f2a [0,4194304] 0 2024-05-27T09:29:09.717 INFO:tasks.workunit.client.0.smithi119.stdout:7/153: read - d7/df/d19/f1f zero size 2024-05-27T09:29:09.717 INFO:tasks.workunit.client.0.smithi119.stdout:1/54: dwrite f6 [0,4194304] 0 2024-05-27T09:29:09.717 INFO:tasks.workunit.client.0.smithi119.stdout:2/28: mknod d2/c8 0 2024-05-27T09:29:09.718 INFO:tasks.workunit.client.0.smithi119.stdout:5/51: write d6/f9 [2504635,30401] 0 2024-05-27T09:29:09.728 INFO:tasks.workunit.client.0.smithi119.stdout:5/52: getdents d6/db 0 2024-05-27T09:29:09.728 INFO:tasks.workunit.client.0.smithi119.stdout:5/53: write d6/f8 [96627,34871] 0 2024-05-27T09:29:09.728 INFO:tasks.workunit.client.0.smithi119.stdout:2/29: creat d2/f9 x:0 0 0 2024-05-27T09:29:09.730 INFO:tasks.workunit.client.0.smithi119.stdout:2/30: mkdir d2/da 0 2024-05-27T09:29:09.732 INFO:tasks.workunit.client.0.smithi119.stdout:2/31: link c1 d2/da/cb 0 2024-05-27T09:29:09.733 INFO:tasks.workunit.client.0.smithi119.stdout:2/32: truncate d2/f6 594396 0 2024-05-27T09:29:09.734 INFO:tasks.workunit.client.0.smithi119.stdout:2/33: chown d2/da/cb 3 1 2024-05-27T09:29:09.736 INFO:tasks.workunit.client.0.smithi119.stdout:2/34: mkdir d2/da/dc 0 2024-05-27T09:29:09.741 INFO:tasks.workunit.client.0.smithi119.stdout:1/55: write f3 [1245446,67592] 0 2024-05-27T09:29:09.741 INFO:tasks.workunit.client.0.smithi119.stdout:5/54: fsync f1 0 2024-05-27T09:29:09.744 INFO:tasks.workunit.client.0.smithi119.stdout:5/55: mkdir d6/dd/df/d13 0 2024-05-27T09:29:09.746 INFO:tasks.workunit.client.0.smithi119.stdout:5/56: creat d6/db/de/f14 x:0 0 0 2024-05-27T09:29:09.746 INFO:tasks.workunit.client.0.smithi119.stdout:5/57: read f1 [659253,77938] 0 2024-05-27T09:29:09.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:09 smithi119 ceph-mon[29082]: pgmap v320: 65 pgs: 65 active+clean; 189 MiB data, 800 MiB used, 536 GiB / 536 GiB avail; 682 KiB/s rd, 7.2 MiB/s wr, 497 op/s 2024-05-27T09:29:09.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:09 smithi119 ceph-mon[29082]: mgrmap e20: smithi119.ncwmog(active, since 9m), standbys: smithi138.dxsuqf 2024-05-27T09:29:09.767 INFO:tasks.workunit.client.0.smithi119.stdout:4/33: dwrite d4/f8 [0,4194304] 0 2024-05-27T09:29:09.779 INFO:tasks.workunit.client.0.smithi119.stdout:4/34: write d4/f8 [4626379,52452] 0 2024-05-27T09:29:09.779 INFO:tasks.workunit.client.0.smithi119.stdout:4/35: dread - f3 zero size 2024-05-27T09:29:09.779 INFO:tasks.workunit.client.0.smithi119.stdout:4/36: write f1 [4932961,117093] 0 2024-05-27T09:29:09.779 INFO:tasks.workunit.client.0.smithi119.stdout:4/37: creat d4/d6/fc x:0 0 0 2024-05-27T09:29:09.782 INFO:tasks.workunit.client.0.smithi119.stdout:4/38: rmdir d4/d5 0 2024-05-27T09:29:09.793 INFO:tasks.workunit.client.0.smithi119.stdout:7/154: write d7/fa [1190760,7980] 0 2024-05-27T09:29:09.796 INFO:tasks.workunit.client.0.smithi119.stdout:7/155: mkdir d7/df/d36 0 2024-05-27T09:29:09.797 INFO:tasks.workunit.client.0.smithi119.stdout:7/156: mkdir d7/db/d37 0 2024-05-27T09:29:09.803 INFO:tasks.workunit.client.0.smithi119.stdout:7/157: link c2 d7/df/d19/d1d/c38 0 2024-05-27T09:29:09.805 INFO:tasks.workunit.client.0.smithi119.stdout:7/158: write d7/db/f2f [98818,8814] 0 2024-05-27T09:29:09.805 INFO:tasks.workunit.client.0.smithi119.stdout:7/159: stat d7/db/d37 0 2024-05-27T09:29:09.807 INFO:tasks.workunit.client.0.smithi119.stdout:7/160: mknod d7/c39 0 2024-05-27T09:29:09.810 INFO:tasks.workunit.client.0.smithi119.stdout:7/161: symlink d7/df/d19/l3a 0 2024-05-27T09:29:09.810 INFO:tasks.workunit.client.0.smithi119.stdout:7/162: truncate d7/db/f31 958101 0 2024-05-27T09:29:09.810 INFO:tasks.workunit.client.0.smithi119.stdout:7/163: chown d7/c39 449134 1 2024-05-27T09:29:09.822 INFO:tasks.workunit.client.0.smithi119.stdout:0/35: dwrite d2/d6/f8 [0,4194304] 0 2024-05-27T09:29:09.833 INFO:tasks.workunit.client.0.smithi119.stdout:9/25: dwrite f2 [0,4194304] 0 2024-05-27T09:29:09.835 INFO:tasks.workunit.client.0.smithi119.stdout:2/35: dwrite d2/f6 [0,4194304] 0 2024-05-27T09:29:09.838 INFO:tasks.workunit.client.0.smithi119.stdout:2/36: rmdir d2/da/dc 0 2024-05-27T09:29:09.843 INFO:tasks.workunit.client.0.smithi119.stdout:2/37: creat d2/da/fd x:0 0 0 2024-05-27T09:29:09.855 INFO:tasks.workunit.client.0.smithi119.stdout:9/26: write f2 [3120619,11426] 0 2024-05-27T09:29:09.856 INFO:tasks.workunit.client.0.smithi119.stdout:4/39: dwrite d4/d6/fc [0,4194304] 0 2024-05-27T09:29:09.856 INFO:tasks.workunit.client.0.smithi119.stdout:4/40: readlink d4/d6/l9 0 2024-05-27T09:29:09.856 INFO:tasks.workunit.client.0.smithi119.stdout:4/41: creat d4/d6/fd x:0 0 0 2024-05-27T09:29:09.856 INFO:tasks.workunit.client.0.smithi119.stdout:4/42: dread - d4/d6/fd zero size 2024-05-27T09:29:09.858 INFO:tasks.workunit.client.0.smithi119.stdout:9/27: mknod c5 0 2024-05-27T09:29:09.858 INFO:tasks.workunit.client.0.smithi119.stdout:9/28: write f3 [4261100,22985] 0 2024-05-27T09:29:09.858 INFO:tasks.workunit.client.0.smithi119.stdout:9/29: write f4 [855138,38528] 0 2024-05-27T09:29:09.858 INFO:tasks.workunit.client.0.smithi119.stdout:9/30: chown f4 359962 1 2024-05-27T09:29:09.858 INFO:tasks.workunit.client.0.smithi119.stdout:9/31: creat f6 x:0 0 0 2024-05-27T09:29:09.858 INFO:tasks.workunit.client.0.smithi119.stdout:9/32: creat f7 x:0 0 0 2024-05-27T09:29:09.858 INFO:tasks.workunit.client.0.smithi119.stdout:9/33: stat f3 0 2024-05-27T09:29:09.858 INFO:tasks.workunit.client.0.smithi119.stdout:9/34: creat f8 x:0 0 0 2024-05-27T09:29:09.859 INFO:tasks.workunit.client.0.smithi119.stdout:2/38: rmdir d2 39 2024-05-27T09:29:09.863 INFO:tasks.workunit.client.0.smithi119.stdout:9/35: fsync f3 0 2024-05-27T09:29:09.863 INFO:tasks.workunit.client.0.smithi119.stdout:4/43: mkdir d4/de 0 2024-05-27T09:29:09.863 INFO:tasks.workunit.client.0.smithi119.stdout:4/44: chown f3 1686168 1 2024-05-27T09:29:09.871 INFO:tasks.workunit.client.0.smithi119.stdout:1/56: write f6 [1644411,112384] 0 2024-05-27T09:29:09.871 INFO:tasks.workunit.client.0.smithi119.stdout:3/107: dwrite db/d10/f17 [0,4194304] 0 2024-05-27T09:29:09.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:09 smithi138 ceph-mon[34885]: pgmap v320: 65 pgs: 65 active+clean; 189 MiB data, 800 MiB used, 536 GiB / 536 GiB avail; 682 KiB/s rd, 7.2 MiB/s wr, 497 op/s 2024-05-27T09:29:09.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:09 smithi138 ceph-mon[34885]: mgrmap e20: smithi119.ncwmog(active, since 9m), standbys: smithi138.dxsuqf 2024-05-27T09:29:09.890 INFO:tasks.workunit.client.0.smithi119.stdout:3/108: read db/fc [916963,51865] 0 2024-05-27T09:29:09.890 INFO:tasks.workunit.client.0.smithi119.stdout:3/109: chown db/f24 16624701 1 2024-05-27T09:29:09.891 INFO:tasks.workunit.client.0.smithi119.stdout:4/45: unlink d4/d6/fc 0 2024-05-27T09:29:09.891 INFO:tasks.workunit.client.0.smithi119.stdout:1/57: mknod d7/dd/cf 0 2024-05-27T09:29:09.893 INFO:tasks.workunit.client.0.smithi119.stdout:3/110: link db/d10/l14 db/d10/l25 0 2024-05-27T09:29:09.894 INFO:tasks.workunit.client.0.smithi119.stdout:0/36: dread d2/d6/f8 [0,4194304] 0 2024-05-27T09:29:09.895 INFO:tasks.workunit.client.0.smithi119.stdout:5/58: dwrite d6/f12 [0,4194304] 0 2024-05-27T09:29:09.895 INFO:tasks.workunit.client.0.smithi119.stdout:1/58: mknod d7/db/c10 0 2024-05-27T09:29:09.897 INFO:tasks.workunit.client.0.smithi119.stdout:3/111: symlink db/l26 0 2024-05-27T09:29:09.897 INFO:tasks.workunit.client.0.smithi119.stdout:3/112: chown db/ff 19281 1 2024-05-27T09:29:09.900 INFO:tasks.workunit.client.0.smithi119.stdout:3/113: write db/f12 [4544302,80584] 0 2024-05-27T09:29:09.902 INFO:tasks.workunit.client.0.smithi119.stdout:5/59: rmdir d6/db/de 39 2024-05-27T09:29:09.902 INFO:tasks.workunit.client.0.smithi119.stdout:5/60: chown f0 316795600 1 2024-05-27T09:29:09.902 INFO:tasks.workunit.client.0.smithi119.stdout:5/61: rename d6 to d6/db/de/d15 22 2024-05-27T09:29:09.903 INFO:tasks.workunit.client.0.smithi119.stdout:3/114: symlink db/d10/l27 0 2024-05-27T09:29:09.906 INFO:tasks.workunit.client.0.smithi119.stdout:9/36: dwrite f4 [0,4194304] 0 2024-05-27T09:29:09.909 INFO:tasks.workunit.client.0.smithi119.stdout:4/46: rmdir d4/d6 39 2024-05-27T09:29:09.948 INFO:tasks.workunit.client.0.smithi119.stdout:1/59: dwrite d7/f8 [0,4194304] 0 2024-05-27T09:29:09.948 INFO:tasks.workunit.client.0.smithi119.stdout:1/60: chown l4 2537806 1 2024-05-27T09:29:09.952 INFO:tasks.workunit.client.0.smithi119.stdout:1/61: truncate f3 2607749 0 2024-05-27T09:29:09.952 INFO:tasks.workunit.client.0.smithi119.stdout:1/62: readlink d7/dd/le 0 2024-05-27T09:29:09.952 INFO:tasks.workunit.client.0.smithi119.stdout:1/63: chown d7/db/lc 0 1 2024-05-27T09:29:09.954 INFO:tasks.workunit.client.0.smithi119.stdout:1/64: unlink d7/fa 0 2024-05-27T09:29:09.956 INFO:tasks.workunit.client.0.smithi119.stdout:1/65: creat d7/f11 x:0 0 0 2024-05-27T09:29:09.957 INFO:tasks.workunit.client.0.smithi119.stdout:1/66: mkdir d7/db/d12 0 2024-05-27T09:29:09.966 INFO:tasks.workunit.client.0.smithi119.stdout:9/37: dwrite f7 [0,4194304] 0 2024-05-27T09:29:09.971 INFO:tasks.workunit.client.0.smithi119.stdout:9/38: unlink f3 0 2024-05-27T09:29:09.974 INFO:tasks.workunit.client.0.smithi119.stdout:9/39: symlink l9 0 2024-05-27T09:29:09.974 INFO:tasks.workunit.client.0.smithi119.stdout:9/40: chown f6 18295097 1 2024-05-27T09:29:09.974 INFO:tasks.workunit.client.0.smithi119.stdout:9/41: creat fa x:0 0 0 2024-05-27T09:29:09.978 INFO:tasks.workunit.client.0.smithi119.stdout:3/115: dwrite db/f24 [0,4194304] 0 2024-05-27T09:29:09.979 INFO:tasks.workunit.client.0.smithi119.stdout:3/116: write db/d10/d13/f21 [993834,98762] 0 2024-05-27T09:29:09.986 INFO:tasks.workunit.client.0.smithi119.stdout:3/117: getdents db/d10/d13 0 2024-05-27T09:29:09.989 INFO:tasks.workunit.client.0.smithi119.stdout:3/118: rename db/d10/f1a to db/d10/d13/f28 0 2024-05-27T09:29:09.994 INFO:tasks.workunit.client.0.smithi119.stdout:3/119: mknod db/d10/d13/c29 0 2024-05-27T09:29:09.995 INFO:tasks.workunit.client.0.smithi119.stdout:3/120: readlink db/l1c 0 2024-05-27T09:29:09.998 INFO:tasks.workunit.client.0.smithi119.stdout:3/121: creat db/d10/d13/f2a x:0 0 0 2024-05-27T09:29:10.001 INFO:tasks.workunit.client.0.smithi119.stdout:1/67: dwrite f2 [0,4194304] 0 2024-05-27T09:29:10.011 INFO:tasks.workunit.client.0.smithi119.stdout:9/42: dwrite f8 [0,4194304] 0 2024-05-27T09:29:10.011 INFO:tasks.workunit.client.0.smithi119.stdout:9/43: chown f7 40951635 1 2024-05-27T09:29:10.040 INFO:tasks.workunit.client.0.smithi119.stdout:3/122: dwrite db/d10/f17 [0,4194304] 0 2024-05-27T09:29:10.044 INFO:tasks.workunit.client.0.smithi119.stdout:6/32: sync 2024-05-27T09:29:10.049 INFO:tasks.workunit.client.0.smithi119.stdout:6/33: symlink d0/la 0 2024-05-27T09:29:10.049 INFO:tasks.workunit.client.0.smithi119.stdout:6/34: fsync d0/d2/f9 0 2024-05-27T09:29:10.049 INFO:tasks.workunit.client.0.smithi119.stdout:6/35: read - d0/d2/f4 zero size 2024-05-27T09:29:10.050 INFO:tasks.workunit.client.0.smithi119.stdout:3/123: dread f6 [0,4194304] 0 2024-05-27T09:29:10.056 INFO:tasks.workunit.client.0.smithi119.stdout:3/124: rename db/l1b to db/l2b 0 2024-05-27T09:29:10.068 INFO:tasks.workunit.client.0.smithi119.stdout:9/44: dwrite f7 [0,4194304] 0 2024-05-27T09:29:10.069 INFO:tasks.workunit.client.0.smithi119.stdout:3/125: read db/f22 [321117,128449] 0 2024-05-27T09:29:10.076 INFO:tasks.workunit.client.0.smithi119.stdout:9/45: truncate f7 202735 0 2024-05-27T09:29:10.076 INFO:tasks.workunit.client.0.smithi119.stdout:9/46: creat fb x:0 0 0 2024-05-27T09:29:10.085 INFO:tasks.workunit.client.0.smithi119.stdout:1/68: dwrite f2 [0,4194304] 0 2024-05-27T09:29:10.100 INFO:tasks.workunit.client.0.smithi119.stdout:3/126: write db/ff [200810,50090] 0 2024-05-27T09:29:10.101 INFO:tasks.workunit.client.0.smithi119.stdout:3/127: truncate db/d10/d13/f21 1148250 0 2024-05-27T09:29:10.102 INFO:tasks.workunit.client.0.smithi119.stdout:3/128: creat db/d10/d13/f2c x:0 0 0 2024-05-27T09:29:10.102 INFO:tasks.workunit.client.0.smithi119.stdout:3/129: chown db/l2b 192214039 1 2024-05-27T09:29:10.102 INFO:tasks.workunit.client.0.smithi119.stdout:6/36: dwrite d0/d2/f9 [0,4194304] 0 2024-05-27T09:29:10.106 INFO:tasks.workunit.client.0.smithi119.stdout:3/130: unlink db/d10/l16 0 2024-05-27T09:29:10.109 INFO:tasks.workunit.client.0.smithi119.stdout:1/69: write d7/f8 [1537021,48695] 0 2024-05-27T09:29:10.111 INFO:tasks.workunit.client.0.smithi119.stdout:8/33: sync 2024-05-27T09:29:10.111 INFO:tasks.workunit.client.0.smithi119.stdout:7/164: sync 2024-05-27T09:29:10.111 INFO:tasks.workunit.client.0.smithi119.stdout:8/34: rename d3 to d3/d8/db 22 2024-05-27T09:29:10.111 INFO:tasks.workunit.client.0.smithi119.stdout:0/37: sync 2024-05-27T09:29:10.112 INFO:tasks.workunit.client.0.smithi119.stdout:2/39: sync 2024-05-27T09:29:10.112 INFO:tasks.workunit.client.0.smithi119.stdout:1/70: creat d7/dd/f13 x:0 0 0 2024-05-27T09:29:10.112 INFO:tasks.workunit.client.0.smithi119.stdout:5/62: sync 2024-05-27T09:29:10.113 INFO:tasks.workunit.client.0.smithi119.stdout:4/47: sync 2024-05-27T09:29:10.113 INFO:tasks.workunit.client.0.smithi119.stdout:6/37: truncate d0/d2/f9 1866222 0 2024-05-27T09:29:10.113 INFO:tasks.workunit.client.0.smithi119.stdout:6/38: fdatasync d0/f6 0 2024-05-27T09:29:10.117 INFO:tasks.workunit.client.0.smithi119.stdout:0/38: mkdir d2/d6/db 0 2024-05-27T09:29:10.117 INFO:tasks.workunit.client.0.smithi119.stdout:2/40: symlink d2/le 0 2024-05-27T09:29:10.120 INFO:tasks.workunit.client.0.smithi119.stdout:0/39: creat d2/d6/fc x:0 0 0 2024-05-27T09:29:10.120 INFO:tasks.workunit.client.0.smithi119.stdout:8/35: dread f2 [0,4194304] 0 2024-05-27T09:29:10.120 INFO:tasks.workunit.client.0.smithi119.stdout:2/41: write d2/f7 [343207,57553] 0 2024-05-27T09:29:10.121 INFO:tasks.workunit.client.0.smithi119.stdout:7/165: symlink d7/db/d37/l3b 0 2024-05-27T09:29:10.123 INFO:tasks.workunit.client.0.smithi119.stdout:5/63: link d6/ca d6/dd/df/c16 0 2024-05-27T09:29:10.123 INFO:tasks.workunit.client.0.smithi119.stdout:5/64: fsync d6/db/de/f14 0 2024-05-27T09:29:10.123 INFO:tasks.workunit.client.0.smithi119.stdout:7/166: chown d7/d1a/c24 26 1 2024-05-27T09:29:10.123 INFO:tasks.workunit.client.0.smithi119.stdout:7/167: chown d7/df/d19/d1d/f2a 0 1 2024-05-27T09:29:10.132 INFO:tasks.workunit.client.0.smithi119.stdout:8/36: truncate f1 2991374 0 2024-05-27T09:29:10.132 INFO:tasks.workunit.client.0.smithi119.stdout:2/42: rename d2/f7 to d2/da/ff 0 2024-05-27T09:29:10.132 INFO:tasks.workunit.client.0.smithi119.stdout:5/65: truncate d6/f12 949310 0 2024-05-27T09:29:10.132 INFO:tasks.workunit.client.0.smithi119.stdout:5/66: write d6/f8 [228559,126800] 0 2024-05-27T09:29:10.132 INFO:tasks.workunit.client.0.smithi119.stdout:5/67: write d6/db/de/f14 [591954,91159] 0 2024-05-27T09:29:10.134 INFO:tasks.workunit.client.0.smithi119.stdout:8/37: truncate f2 158450 0 2024-05-27T09:29:10.136 INFO:tasks.workunit.client.0.smithi119.stdout:7/168: symlink d7/df/d36/l3c 0 2024-05-27T09:29:10.136 INFO:tasks.workunit.client.0.smithi119.stdout:7/169: dread - d7/d1a/f2d zero size 2024-05-27T09:29:10.136 INFO:tasks.workunit.client.0.smithi119.stdout:2/43: creat d2/da/f10 x:0 0 0 2024-05-27T09:29:10.138 INFO:tasks.workunit.client.0.smithi119.stdout:7/170: unlink d7/df/c16 0 2024-05-27T09:29:10.210 INFO:tasks.workunit.client.0.smithi119.stdout:7/171: read d7/d1a/f20 [3268923,34554] 0 2024-05-27T09:29:10.215 INFO:tasks.workunit.client.0.smithi119.stdout:6/39: write d0/f6 [2971272,111324] 0 2024-05-27T09:29:10.215 INFO:tasks.workunit.client.0.smithi119.stdout:6/40: write d0/d2/f4 [80535,119355] 0 2024-05-27T09:29:10.215 INFO:tasks.workunit.client.0.smithi119.stdout:6/41: truncate d0/d2/f4 931160 0 2024-05-27T09:29:10.215 INFO:tasks.workunit.client.0.smithi119.stdout:6/42: chown d0/l1 141332610 1 2024-05-27T09:29:10.215 INFO:tasks.workunit.client.0.smithi119.stdout:3/131: dwrite f6 [0,4194304] 0 2024-05-27T09:29:10.216 INFO:tasks.workunit.client.0.smithi119.stdout:6/43: creat d0/fb x:0 0 0 2024-05-27T09:29:10.217 INFO:tasks.workunit.client.0.smithi119.stdout:1/71: dwrite d7/dd/f13 [0,4194304] 0 2024-05-27T09:29:10.218 INFO:tasks.workunit.client.0.smithi119.stdout:1/72: stat d7 0 2024-05-27T09:29:10.218 INFO:tasks.workunit.client.0.smithi119.stdout:1/73: creat d7/f14 x:0 0 0 2024-05-27T09:29:10.218 INFO:tasks.workunit.client.0.smithi119.stdout:1/74: fdatasync d7/f11 0 2024-05-27T09:29:10.220 INFO:tasks.workunit.client.0.smithi119.stdout:6/44: symlink d0/lc 0 2024-05-27T09:29:10.220 INFO:tasks.workunit.client.0.smithi119.stdout:6/45: readlink d0/la 0 2024-05-27T09:29:10.222 INFO:tasks.workunit.client.0.smithi119.stdout:1/75: truncate f2 2488082 0 2024-05-27T09:29:10.225 INFO:tasks.workunit.client.0.smithi119.stdout:3/132: unlink db/f22 0 2024-05-27T09:29:10.226 INFO:tasks.workunit.client.0.smithi119.stdout:4/48: dwrite f3 [0,4194304] 0 2024-05-27T09:29:10.226 INFO:tasks.workunit.client.0.smithi119.stdout:4/49: fsync d4/f8 0 2024-05-27T09:29:10.228 INFO:tasks.workunit.client.0.smithi119.stdout:3/133: write db/f23 [886338,85668] 0 2024-05-27T09:29:10.228 INFO:tasks.workunit.client.0.smithi119.stdout:4/50: symlink d4/de/lf 0 2024-05-27T09:29:10.228 INFO:tasks.workunit.client.0.smithi119.stdout:4/51: readlink d4/de/lf 0 2024-05-27T09:29:10.229 INFO:tasks.workunit.client.0.smithi119.stdout:3/134: truncate db/d10/d13/f28 765720 0 2024-05-27T09:29:10.232 INFO:tasks.workunit.client.0.smithi119.stdout:4/52: creat d4/d6/f10 x:0 0 0 2024-05-27T09:29:10.249 INFO:tasks.workunit.client.0.smithi119.stdout:0/40: dwrite d2/d6/f8 [0,4194304] 0 2024-05-27T09:29:10.250 INFO:tasks.workunit.client.0.smithi119.stdout:8/38: dwrite f0 [0,4194304] 0 2024-05-27T09:29:10.255 INFO:tasks.workunit.client.0.smithi119.stdout:8/39: symlink d3/d8/lc 0 2024-05-27T09:29:10.256 INFO:tasks.workunit.client.0.smithi119.stdout:8/40: stat d3/c7 0 2024-05-27T09:29:10.267 INFO:tasks.workunit.client.0.smithi119.stdout:4/53: dread d4/f8 [0,4194304] 0 2024-05-27T09:29:10.272 INFO:tasks.workunit.client.0.smithi119.stdout:8/41: creat d3/d8/fd x:0 0 0 2024-05-27T09:29:10.274 INFO:tasks.workunit.client.0.smithi119.stdout:8/42: mkdir d3/de 0 2024-05-27T09:29:10.274 INFO:tasks.workunit.client.0.smithi119.stdout:8/43: truncate d3/d8/fd 407748 0 2024-05-27T09:29:10.276 INFO:tasks.workunit.client.0.smithi119.stdout:8/44: creat d3/de/ff x:0 0 0 2024-05-27T09:29:10.276 INFO:tasks.workunit.client.0.smithi119.stdout:8/45: creat d3/de/f10 x:0 0 0 2024-05-27T09:29:10.281 INFO:tasks.workunit.client.0.smithi119.stdout:8/46: mknod d3/de/c11 0 2024-05-27T09:29:10.285 INFO:tasks.workunit.client.0.smithi119.stdout:4/54: write d4/f8 [3629195,114607] 0 2024-05-27T09:29:10.287 INFO:tasks.workunit.client.0.smithi119.stdout:4/55: creat d4/d6/f11 x:0 0 0 2024-05-27T09:29:10.287 INFO:tasks.workunit.client.0.smithi119.stdout:5/68: dwrite d6/f8 [0,4194304] 0 2024-05-27T09:29:10.288 INFO:tasks.workunit.client.0.smithi119.stdout:5/69: creat d6/f17 x:0 0 0 2024-05-27T09:29:10.288 INFO:tasks.workunit.client.0.smithi119.stdout:5/70: getdents d6/db/dc 0 2024-05-27T09:29:10.289 INFO:tasks.workunit.client.0.smithi119.stdout:6/46: dwrite d0/f7 [0,4194304] 0 2024-05-27T09:29:10.289 INFO:tasks.workunit.client.0.smithi119.stdout:5/71: creat d6/db/de/f18 x:0 0 0 2024-05-27T09:29:10.290 INFO:tasks.workunit.client.0.smithi119.stdout:5/72: chown d6/ca 15 1 2024-05-27T09:29:10.290 INFO:tasks.workunit.client.0.smithi119.stdout:6/47: creat d0/fd x:0 0 0 2024-05-27T09:29:10.291 INFO:tasks.workunit.client.0.smithi119.stdout:5/73: mkdir d6/dd/df/d13/d19 0 2024-05-27T09:29:10.299 INFO:tasks.workunit.client.0.smithi119.stdout:8/47: fsync d3/de/ff 0 2024-05-27T09:29:10.299 INFO:tasks.workunit.client.0.smithi119.stdout:7/172: dwrite f4 [0,4194304] 0 2024-05-27T09:29:10.299 INFO:tasks.workunit.client.0.smithi119.stdout:1/76: dread d7/f8 [0,4194304] 0 2024-05-27T09:29:10.299 INFO:tasks.workunit.client.0.smithi119.stdout:6/48: getdents d0/d2 0 2024-05-27T09:29:10.299 INFO:tasks.workunit.client.0.smithi119.stdout:7/173: write d7/d1a/f1c [816521,92673] 0 2024-05-27T09:29:10.299 INFO:tasks.workunit.client.0.smithi119.stdout:7/174: truncate d7/db/f2f 442498 0 2024-05-27T09:29:10.299 INFO:tasks.workunit.client.0.smithi119.stdout:1/77: mkdir d7/db/d12/d15 0 2024-05-27T09:29:10.299 INFO:tasks.workunit.client.0.smithi119.stdout:6/49: mkdir d0/d2/de 0 2024-05-27T09:29:10.300 INFO:tasks.workunit.client.0.smithi119.stdout:6/50: stat d0/d2 0 2024-05-27T09:29:10.301 INFO:tasks.workunit.client.0.smithi119.stdout:7/175: rename d7/d1a/c24 to d7/db/c3d 0 2024-05-27T09:29:10.303 INFO:tasks.workunit.client.0.smithi119.stdout:6/51: mknod d0/cf 0 2024-05-27T09:29:10.307 INFO:tasks.workunit.client.0.smithi119.stdout:6/52: unlink d0/lc 0 2024-05-27T09:29:10.307 INFO:tasks.workunit.client.0.smithi119.stdout:6/53: write d0/d2/f4 [950428,49175] 0 2024-05-27T09:29:10.309 INFO:tasks.workunit.client.0.smithi119.stdout:6/54: creat d0/d2/f10 x:0 0 0 2024-05-27T09:29:10.311 INFO:tasks.workunit.client.0.smithi119.stdout:6/55: mkdir d0/d11 0 2024-05-27T09:29:10.311 INFO:tasks.workunit.client.0.smithi119.stdout:6/56: dread - d0/fd zero size 2024-05-27T09:29:10.313 INFO:tasks.workunit.client.0.smithi119.stdout:7/176: truncate d7/df/f13 8303909 0 2024-05-27T09:29:10.313 INFO:tasks.workunit.client.0.smithi119.stdout:7/177: fdatasync d7/d1a/f20 0 2024-05-27T09:29:10.315 INFO:tasks.workunit.client.0.smithi119.stdout:7/178: symlink d7/df/l3e 0 2024-05-27T09:29:10.316 INFO:tasks.workunit.client.0.smithi119.stdout:7/179: creat d7/df/d36/f3f x:0 0 0 2024-05-27T09:29:10.318 INFO:tasks.workunit.client.0.smithi119.stdout:1/78: dread f2 [0,4194304] 0 2024-05-27T09:29:10.318 INFO:tasks.workunit.client.0.smithi119.stdout:1/79: write d7/f14 [348482,99159] 0 2024-05-27T09:29:10.327 INFO:tasks.workunit.client.0.smithi119.stdout:0/41: dwrite d2/d6/f8 [4194304,4194304] 0 2024-05-27T09:29:10.332 INFO:tasks.workunit.client.0.smithi119.stdout:8/48: dread f2 [0,4194304] 0 2024-05-27T09:29:10.332 INFO:tasks.workunit.client.0.smithi119.stdout:3/135: dwrite db/d10/f17 [4194304,4194304] 0 2024-05-27T09:29:10.333 INFO:tasks.workunit.client.0.smithi119.stdout:5/74: dread f0 [0,4194304] 0 2024-05-27T09:29:10.333 INFO:tasks.workunit.client.0.smithi119.stdout:8/49: truncate d3/de/ff 971828 0 2024-05-27T09:29:10.337 INFO:tasks.workunit.client.0.smithi119.stdout:0/42: link l0 d2/d6/d9/ld 0 2024-05-27T09:29:10.337 INFO:tasks.workunit.client.0.smithi119.stdout:0/43: write d2/d6/fc [666044,109607] 0 2024-05-27T09:29:10.339 INFO:tasks.workunit.client.0.smithi119.stdout:3/136: fsync db/f12 0 2024-05-27T09:29:10.346 INFO:tasks.workunit.client.0.smithi119.stdout:5/75: rmdir d6/db/de 39 2024-05-27T09:29:10.346 INFO:tasks.workunit.client.0.smithi119.stdout:2/44: sync 2024-05-27T09:29:10.350 INFO:tasks.workunit.client.0.smithi119.stdout:9/47: sync 2024-05-27T09:29:10.350 INFO:tasks.workunit.client.0.smithi119.stdout:2/45: link d2/c4 d2/c11 0 2024-05-27T09:29:10.350 INFO:tasks.workunit.client.0.smithi119.stdout:2/46: mknod d2/da/c12 0 2024-05-27T09:29:10.351 INFO:tasks.workunit.client.0.smithi119.stdout:2/47: creat d2/da/f13 x:0 0 0 2024-05-27T09:29:10.352 INFO:tasks.workunit.client.0.smithi119.stdout:2/48: truncate d2/f6 1736419 0 2024-05-27T09:29:10.352 INFO:tasks.workunit.client.0.smithi119.stdout:2/49: rename d2 to d2/da/d14 22 2024-05-27T09:29:10.355 INFO:tasks.workunit.client.0.smithi119.stdout:5/76: dread d6/db/de/f14 [0,4194304] 0 2024-05-27T09:29:10.355 INFO:tasks.workunit.client.0.smithi119.stdout:2/50: mkdir d2/da/d15 0 2024-05-27T09:29:10.355 INFO:tasks.workunit.client.0.smithi119.stdout:2/51: read - d2/da/f13 zero size 2024-05-27T09:29:10.355 INFO:tasks.workunit.client.0.smithi119.stdout:2/52: truncate d2/da/f10 500266 0 2024-05-27T09:29:10.357 INFO:tasks.workunit.client.0.smithi119.stdout:5/77: creat d6/dd/df/d13/f1a x:0 0 0 2024-05-27T09:29:10.357 INFO:tasks.workunit.client.0.smithi119.stdout:5/78: stat d6/db/dc 0 2024-05-27T09:29:10.357 INFO:tasks.workunit.client.0.smithi119.stdout:5/79: read - d6/dd/df/d13/f1a zero size 2024-05-27T09:29:10.357 INFO:tasks.workunit.client.0.smithi119.stdout:0/44: dread d2/d6/f8 [4194304,4194304] 0 2024-05-27T09:29:10.360 INFO:tasks.workunit.client.0.smithi119.stdout:7/180: dread d7/df/d19/d1d/f2a [0,4194304] 0 2024-05-27T09:29:10.361 INFO:tasks.workunit.client.0.smithi119.stdout:5/80: creat d6/dd/f1b x:0 0 0 2024-05-27T09:29:10.361 INFO:tasks.workunit.client.0.smithi119.stdout:4/56: dwrite d4/d6/f11 [0,4194304] 0 2024-05-27T09:29:10.362 INFO:tasks.workunit.client.0.smithi119.stdout:8/50: dread d3/fa [0,4194304] 0 2024-05-27T09:29:10.362 INFO:tasks.workunit.client.0.smithi119.stdout:8/51: creat d3/f12 x:0 0 0 2024-05-27T09:29:10.370 INFO:tasks.workunit.client.0.smithi119.stdout:9/48: dread f8 [0,4194304] 0 2024-05-27T09:29:10.371 INFO:tasks.workunit.client.0.smithi119.stdout:4/57: unlink d4/d6/f10 0 2024-05-27T09:29:10.371 INFO:tasks.workunit.client.0.smithi119.stdout:7/181: mkdir d7/d1a/d40 0 2024-05-27T09:29:10.371 INFO:tasks.workunit.client.0.smithi119.stdout:9/49: creat fc x:0 0 0 2024-05-27T09:29:10.382 INFO:tasks.workunit.client.0.smithi119.stdout:8/52: link d3/f5 d3/d8/f13 0 2024-05-27T09:29:10.382 INFO:tasks.workunit.client.0.smithi119.stdout:7/182: mkdir d7/d1a/d41 0 2024-05-27T09:29:10.382 INFO:tasks.workunit.client.0.smithi119.stdout:7/183: chown d7/db/c32 924555255 1 2024-05-27T09:29:10.382 INFO:tasks.workunit.client.0.smithi119.stdout:7/184: chown d7/df/d19/l12 1143397 1 2024-05-27T09:29:10.382 INFO:tasks.workunit.client.0.smithi119.stdout:3/137: dread db/f23 [0,4194304] 0 2024-05-27T09:29:10.382 INFO:tasks.workunit.client.0.smithi119.stdout:7/185: mkdir d7/df/d36/d42 0 2024-05-27T09:29:10.382 INFO:tasks.workunit.client.0.smithi119.stdout:7/186: read - d7/df/d19/f1f zero size 2024-05-27T09:29:10.382 INFO:tasks.workunit.client.0.smithi119.stdout:1/80: dwrite f6 [0,4194304] 0 2024-05-27T09:29:10.389 INFO:tasks.workunit.client.0.smithi119.stdout:1/81: creat d7/db/f16 x:0 0 0 2024-05-27T09:29:10.394 INFO:tasks.workunit.client.0.smithi119.stdout:1/82: mknod d7/dd/c17 0 2024-05-27T09:29:10.400 INFO:tasks.workunit.client.0.smithi119.stdout:0/45: dread d2/d6/fc [0,4194304] 0 2024-05-27T09:29:10.400 INFO:tasks.workunit.client.0.smithi119.stdout:2/53: read d2/f6 [1544829,18770] 0 2024-05-27T09:29:10.401 INFO:tasks.workunit.client.0.smithi119.stdout:1/83: symlink d7/dd/l18 0 2024-05-27T09:29:10.401 INFO:tasks.workunit.client.0.smithi119.stdout:1/84: stat d7/c9 0 2024-05-27T09:29:10.411 INFO:tasks.workunit.client.0.smithi119.stdout:2/54: creat d2/da/d15/f16 x:0 0 0 2024-05-27T09:29:10.411 INFO:tasks.workunit.client.0.smithi119.stdout:2/55: dread - d2/da/d15/f16 zero size 2024-05-27T09:29:10.412 INFO:tasks.workunit.client.0.smithi119.stdout:1/85: symlink d7/dd/l19 0 2024-05-27T09:29:10.415 INFO:tasks.workunit.client.0.smithi119.stdout:1/86: mknod d7/db/c1a 0 2024-05-27T09:29:10.415 INFO:tasks.workunit.client.0.smithi119.stdout:1/87: chown d7/db/lc 14882 1 2024-05-27T09:29:10.416 INFO:tasks.workunit.client.0.smithi119.stdout:1/88: chown d7/dd/cf 39291 1 2024-05-27T09:29:10.416 INFO:tasks.workunit.client.0.smithi119.stdout:1/89: rename d7/db to d7/db/d12/d15/d1b 22 2024-05-27T09:29:10.416 INFO:tasks.workunit.client.0.smithi119.stdout:1/90: write d7/f14 [1406671,108995] 0 2024-05-27T09:29:10.418 INFO:tasks.workunit.client.0.smithi119.stdout:1/91: symlink d7/l1c 0 2024-05-27T09:29:10.421 INFO:tasks.workunit.client.0.smithi119.stdout:5/81: dwrite d6/dd/df/d13/f1a [0,4194304] 0 2024-05-27T09:29:10.422 INFO:tasks.workunit.client.0.smithi119.stdout:5/82: chown d6/db 3196022 1 2024-05-27T09:29:10.428 INFO:tasks.workunit.client.0.smithi119.stdout:5/83: symlink d6/dd/df/d13/d19/l1c 0 2024-05-27T09:29:10.434 INFO:tasks.workunit.client.0.smithi119.stdout:5/84: write f1 [5047793,78978] 0 2024-05-27T09:29:10.435 INFO:tasks.workunit.client.0.smithi119.stdout:5/85: creat d6/f1d x:0 0 0 2024-05-27T09:29:10.439 INFO:tasks.workunit.client.0.smithi119.stdout:5/86: rename d6/dd/df/d13/f1a to d6/f1e 0 2024-05-27T09:29:10.450 INFO:tasks.workunit.client.0.smithi119.stdout:8/53: dwrite d3/de/ff [0,4194304] 0 2024-05-27T09:29:10.467 INFO:tasks.workunit.client.0.smithi119.stdout:8/54: rmdir d3/d8 39 2024-05-27T09:29:10.480 INFO:tasks.workunit.client.0.smithi119.stdout:8/55: dread d3/f5 [0,4194304] 0 2024-05-27T09:29:10.483 INFO:tasks.workunit.client.0.smithi119.stdout:2/56: dwrite d2/da/fd [0,4194304] 0 2024-05-27T09:29:10.486 INFO:tasks.workunit.client.0.smithi119.stdout:8/56: creat d3/d8/f14 x:0 0 0 2024-05-27T09:29:10.486 INFO:tasks.workunit.client.0.smithi119.stdout:8/57: creat d3/f15 x:0 0 0 2024-05-27T09:29:10.496 INFO:tasks.workunit.client.0.smithi119.stdout:2/57: getdents d2/da/d15 0 2024-05-27T09:29:10.499 INFO:tasks.workunit.client.0.smithi119.stdout:2/58: link d2/l3 d2/l17 0 2024-05-27T09:29:10.500 INFO:tasks.workunit.client.0.smithi119.stdout:7/187: dwrite d7/fa [0,4194304] 0 2024-05-27T09:29:10.501 INFO:tasks.workunit.client.0.smithi119.stdout:2/59: creat d2/da/d15/f18 x:0 0 0 2024-05-27T09:29:10.501 INFO:tasks.workunit.client.0.smithi119.stdout:2/60: readlink d2/l3 0 2024-05-27T09:29:10.503 INFO:tasks.workunit.client.0.smithi119.stdout:7/188: mkdir d7/df/d43 0 2024-05-27T09:29:10.504 INFO:tasks.workunit.client.0.smithi119.stdout:2/61: creat d2/f19 x:0 0 0 2024-05-27T09:29:10.505 INFO:tasks.workunit.client.0.smithi119.stdout:2/62: mkdir d2/d1a 0 2024-05-27T09:29:10.509 INFO:tasks.workunit.client.0.smithi119.stdout:9/50: dwrite fb [0,4194304] 0 2024-05-27T09:29:10.510 INFO:tasks.workunit.client.0.smithi119.stdout:9/51: dread - f6 zero size 2024-05-27T09:29:10.513 INFO:tasks.workunit.client.0.smithi119.stdout:7/189: dread d7/df/d19/f1b [0,4194304] 0 2024-05-27T09:29:10.515 INFO:tasks.workunit.client.0.smithi119.stdout:5/87: dwrite d6/f1d [0,4194304] 0 2024-05-27T09:29:10.516 INFO:tasks.workunit.client.0.smithi119.stdout:3/138: dwrite db/d10/d13/f21 [0,4194304] 0 2024-05-27T09:29:10.516 INFO:tasks.workunit.client.0.smithi119.stdout:3/139: stat db 0 2024-05-27T09:29:10.516 INFO:tasks.workunit.client.0.smithi119.stdout:0/46: dwrite d2/d6/f8 [0,4194304] 0 2024-05-27T09:29:10.516 INFO:tasks.workunit.client.0.smithi119.stdout:0/47: chown d2/d6/fc 14653085 1 2024-05-27T09:29:10.523 INFO:tasks.workunit.client.0.smithi119.stdout:5/88: rename d6/c10 to d6/db/dc/c1f 0 2024-05-27T09:29:10.527 INFO:tasks.workunit.client.0.smithi119.stdout:1/92: dwrite d7/f8 [0,4194304] 0 2024-05-27T09:29:10.527 INFO:tasks.workunit.client.0.smithi119.stdout:1/93: dread - d7/f11 zero size 2024-05-27T09:29:10.527 INFO:tasks.workunit.client.0.smithi119.stdout:5/89: creat d6/db/f20 x:0 0 0 2024-05-27T09:29:10.527 INFO:tasks.workunit.client.0.smithi119.stdout:5/90: truncate d6/f12 1908491 0 2024-05-27T09:29:10.530 INFO:tasks.workunit.client.0.smithi119.stdout:3/140: link db/f24 db/d10/f2d 0 2024-05-27T09:29:10.538 INFO:tasks.workunit.client.0.smithi119.stdout:9/52: read f4 [3626248,107335] 0 2024-05-27T09:29:10.538 INFO:tasks.workunit.client.0.smithi119.stdout:5/91: creat d6/db/dc/f21 x:0 0 0 2024-05-27T09:29:10.538 INFO:tasks.workunit.client.0.smithi119.stdout:5/92: chown d6/dd/df/c16 811 1 2024-05-27T09:29:10.538 INFO:tasks.workunit.client.0.smithi119.stdout:5/93: write d6/db/dc/f21 [200621,117225] 0 2024-05-27T09:29:10.538 INFO:tasks.workunit.client.0.smithi119.stdout:0/48: write d2/d6/f8 [1180147,19109] 0 2024-05-27T09:29:10.539 INFO:tasks.workunit.client.0.smithi119.stdout:9/53: mknod cd 0 2024-05-27T09:29:10.539 INFO:tasks.workunit.client.0.smithi119.stdout:9/54: fdatasync fa 0 2024-05-27T09:29:10.539 INFO:tasks.workunit.client.0.smithi119.stdout:3/141: creat db/d10/d13/f2e x:0 0 0 2024-05-27T09:29:10.539 INFO:tasks.workunit.client.0.smithi119.stdout:9/55: stat f8 0 2024-05-27T09:29:10.539 INFO:tasks.workunit.client.0.smithi119.stdout:9/56: readlink l9 0 2024-05-27T09:29:10.540 INFO:tasks.workunit.client.0.smithi119.stdout:9/57: rename f7 to fe 0 2024-05-27T09:29:10.553 INFO:tasks.workunit.client.0.smithi119.stdout:3/142: getdents db/d10 0 2024-05-27T09:29:10.555 INFO:tasks.workunit.client.0.smithi119.stdout:3/143: creat db/f2f x:0 0 0 2024-05-27T09:29:10.562 INFO:tasks.workunit.client.0.smithi119.stdout:5/94: write d6/f8 [4135583,6638] 0 2024-05-27T09:29:10.563 INFO:tasks.workunit.client.0.smithi119.stdout:5/95: chown d6/db/de/f14 41 1 2024-05-27T09:29:10.563 INFO:tasks.workunit.client.0.smithi119.stdout:8/58: dwrite f1 [0,4194304] 0 2024-05-27T09:29:10.563 INFO:tasks.workunit.client.0.smithi119.stdout:8/59: chown d3/d8/f14 510730 1 2024-05-27T09:29:10.563 INFO:tasks.workunit.client.0.smithi119.stdout:5/96: creat d6/f22 x:0 0 0 2024-05-27T09:29:10.563 INFO:tasks.workunit.client.0.smithi119.stdout:0/49: dread d2/d6/fc [0,4194304] 0 2024-05-27T09:29:10.578 INFO:tasks.workunit.client.0.smithi119.stdout:5/97: symlink d6/db/l23 0 2024-05-27T09:29:10.579 INFO:tasks.workunit.client.0.smithi119.stdout:8/60: unlink f2 0 2024-05-27T09:29:10.581 INFO:tasks.workunit.client.0.smithi119.stdout:0/50: truncate d2/d6/fc 34472 0 2024-05-27T09:29:10.581 INFO:tasks.workunit.client.0.smithi119.stdout:0/51: chown d2 7107033 1 2024-05-27T09:29:10.582 INFO:tasks.workunit.client.0.smithi119.stdout:9/58: dread f8 [0,4194304] 0 2024-05-27T09:29:10.584 INFO:tasks.workunit.client.0.smithi119.stdout:9/59: creat ff x:0 0 0 2024-05-27T09:29:10.584 INFO:tasks.workunit.client.0.smithi119.stdout:9/60: dread - f6 zero size 2024-05-27T09:29:10.585 INFO:tasks.workunit.client.0.smithi119.stdout:9/61: creat f10 x:0 0 0 2024-05-27T09:29:10.590 INFO:tasks.workunit.client.0.smithi119.stdout:5/98: unlink d6/db/f20 0 2024-05-27T09:29:10.590 INFO:tasks.workunit.client.0.smithi119.stdout:8/61: dread d3/d8/f13 [0,4194304] 0 2024-05-27T09:29:10.590 INFO:tasks.workunit.client.0.smithi119.stdout:5/99: creat d6/db/dc/f24 x:0 0 0 2024-05-27T09:29:10.590 INFO:tasks.workunit.client.0.smithi119.stdout:5/100: readlink d6/db/l23 0 2024-05-27T09:29:10.590 INFO:tasks.workunit.client.0.smithi119.stdout:5/101: truncate d6/db/de/f18 65237 0 2024-05-27T09:29:10.593 INFO:tasks.workunit.client.0.smithi119.stdout:9/62: mknod c11 0 2024-05-27T09:29:10.594 INFO:tasks.workunit.client.0.smithi119.stdout:5/102: creat d6/db/de/f25 x:0 0 0 2024-05-27T09:29:10.594 INFO:tasks.workunit.client.0.smithi119.stdout:5/103: creat d6/db/dc/f26 x:0 0 0 2024-05-27T09:29:10.600 INFO:tasks.workunit.client.0.smithi119.stdout:5/104: mkdir d6/db/de/d27 0 2024-05-27T09:29:10.600 INFO:tasks.workunit.client.0.smithi119.stdout:1/94: dread d7/f8 [0,4194304] 0 2024-05-27T09:29:10.602 INFO:tasks.workunit.client.0.smithi119.stdout:2/63: dwrite d2/da/f10 [0,4194304] 0 2024-05-27T09:29:10.602 INFO:tasks.workunit.client.0.smithi119.stdout:2/64: chown d2 4 1 2024-05-27T09:29:10.605 INFO:tasks.workunit.client.0.smithi119.stdout:2/65: creat d2/da/d15/f1b x:0 0 0 2024-05-27T09:29:10.608 INFO:tasks.workunit.client.0.smithi119.stdout:2/66: rename d2/f9 to d2/da/f1c 0 2024-05-27T09:29:10.613 INFO:tasks.workunit.client.0.smithi119.stdout:7/190: dwrite d7/d1a/f20 [0,4194304] 0 2024-05-27T09:29:10.613 INFO:tasks.workunit.client.0.smithi119.stdout:2/67: mknod d2/da/d15/c1d 0 2024-05-27T09:29:10.613 INFO:tasks.workunit.client.0.smithi119.stdout:2/68: fdatasync d2/da/f13 0 2024-05-27T09:29:10.613 INFO:tasks.workunit.client.0.smithi119.stdout:2/69: truncate d2/da/d15/f1b 454004 0 2024-05-27T09:29:10.616 INFO:tasks.workunit.client.0.smithi119.stdout:8/62: write f1 [3989013,11252] 0 2024-05-27T09:29:10.616 INFO:tasks.workunit.client.0.smithi119.stdout:8/63: write f0 [3780291,54527] 0 2024-05-27T09:29:10.616 INFO:tasks.workunit.client.0.smithi119.stdout:8/64: write f0 [4364272,121375] 0 2024-05-27T09:29:10.616 INFO:tasks.workunit.client.0.smithi119.stdout:8/65: truncate d3/f12 682330 0 2024-05-27T09:29:10.617 INFO:tasks.workunit.client.0.smithi119.stdout:2/70: creat d2/d1a/f1e x:0 0 0 2024-05-27T09:29:10.626 INFO:tasks.workunit.client.0.smithi119.stdout:8/66: getdents d3/de 0 2024-05-27T09:29:10.628 INFO:tasks.workunit.client.0.smithi119.stdout:2/71: dread d2/da/d15/f1b [0,4194304] 0 2024-05-27T09:29:10.628 INFO:tasks.workunit.client.0.smithi119.stdout:2/72: readlink l0 0 2024-05-27T09:29:10.631 INFO:tasks.workunit.client.0.smithi119.stdout:6/57: sync 2024-05-27T09:29:10.631 INFO:tasks.workunit.client.0.smithi119.stdout:6/58: dread - d0/d2/f10 zero size 2024-05-27T09:29:10.633 INFO:tasks.workunit.client.0.smithi119.stdout:4/58: sync 2024-05-27T09:29:10.633 INFO:tasks.workunit.client.0.smithi119.stdout:6/59: mkdir d0/d11/d12 0 2024-05-27T09:29:10.638 INFO:tasks.workunit.client.0.smithi119.stdout:6/60: symlink d0/l13 0 2024-05-27T09:29:10.643 INFO:tasks.workunit.client.0.smithi119.stdout:6/61: creat d0/d11/f14 x:0 0 0 2024-05-27T09:29:10.665 INFO:tasks.workunit.client.0.smithi119.stdout:3/144: dwrite db/ff [4194304,4194304] 0 2024-05-27T09:29:10.670 INFO:tasks.workunit.client.0.smithi119.stdout:3/145: creat db/f30 x:0 0 0 2024-05-27T09:29:10.682 INFO:tasks.workunit.client.0.smithi119.stdout:3/146: link db/d10/f17 db/d10/d13/f31 0 2024-05-27T09:29:10.693 INFO:tasks.workunit.client.0.smithi119.stdout:4/59: dread d4/f8 [0,4194304] 0 2024-05-27T09:29:10.701 INFO:tasks.workunit.client.0.smithi119.stdout:9/63: dwrite fb [4194304,4194304] 0 2024-05-27T09:29:10.707 INFO:tasks.workunit.client.0.smithi119.stdout:6/62: dwrite d0/f6 [0,4194304] 0 2024-05-27T09:29:10.716 INFO:tasks.workunit.client.0.smithi119.stdout:5/105: dwrite d6/f1e [4194304,4194304] 0 2024-05-27T09:29:10.729 INFO:tasks.workunit.client.0.smithi119.stdout:2/73: dwrite d2/da/d15/f18 [0,4194304] 0 2024-05-27T09:29:10.739 INFO:tasks.workunit.client.0.smithi119.stdout:7/191: dwrite d7/df/f29 [0,4194304] 0 2024-05-27T09:29:10.739 INFO:tasks.workunit.client.0.smithi119.stdout:7/192: write d7/df/d36/f3f [924178,71526] 0 2024-05-27T09:29:10.739 INFO:tasks.workunit.client.0.smithi119.stdout:7/193: truncate d7/d1a/f2b 683540 0 2024-05-27T09:29:10.739 INFO:tasks.workunit.client.0.smithi119.stdout:7/194: chown d7/d1a/d41 204791320 1 2024-05-27T09:29:10.739 INFO:tasks.workunit.client.0.smithi119.stdout:7/195: getdents d7/df/d36/d42 0 2024-05-27T09:29:10.761 INFO:tasks.workunit.client.0.smithi119.stdout:5/106: write d6/f1e [1142766,82755] 0 2024-05-27T09:29:10.766 INFO:tasks.workunit.client.0.smithi119.stdout:0/52: dwrite d2/d6/f8 [0,4194304] 0 2024-05-27T09:29:10.766 INFO:tasks.workunit.client.0.smithi119.stdout:0/53: fdatasync d2/d6/f8 0 2024-05-27T09:29:10.766 INFO:tasks.workunit.client.0.smithi119.stdout:0/54: fsync d2/d6/f8 0 2024-05-27T09:29:10.766 INFO:tasks.workunit.client.0.smithi119.stdout:8/67: dwrite f1 [0,4194304] 0 2024-05-27T09:29:10.770 INFO:tasks.workunit.client.0.smithi119.stdout:5/107: unlink d6/db/dc/f24 0 2024-05-27T09:29:10.770 INFO:tasks.workunit.client.0.smithi119.stdout:0/55: symlink d2/le 0 2024-05-27T09:29:10.798 INFO:tasks.workunit.client.0.smithi119.stdout:3/147: dwrite db/d10/d13/f2c [0,4194304] 0 2024-05-27T09:29:10.800 INFO:tasks.workunit.client.0.smithi119.stdout:3/148: mknod db/d10/c32 0 2024-05-27T09:29:10.804 INFO:tasks.workunit.client.0.smithi119.stdout:3/149: write db/fc [3282164,21275] 0 2024-05-27T09:29:10.805 INFO:tasks.workunit.client.0.smithi119.stdout:2/74: dwrite d2/da/d15/f16 [0,4194304] 0 2024-05-27T09:29:10.805 INFO:tasks.workunit.client.0.smithi119.stdout:2/75: creat d2/d1a/f1f x:0 0 0 2024-05-27T09:29:10.808 INFO:tasks.workunit.client.0.smithi119.stdout:5/108: write d6/f1d [55806,37149] 0 2024-05-27T09:29:10.809 INFO:tasks.workunit.client.0.smithi119.stdout:0/56: truncate d2/d6/f8 7741511 0 2024-05-27T09:29:10.809 INFO:tasks.workunit.client.0.smithi119.stdout:4/60: dwrite d4/d6/fa [0,4194304] 0 2024-05-27T09:29:10.811 INFO:tasks.workunit.client.0.smithi119.stdout:2/76: link d2/le d2/da/l20 0 2024-05-27T09:29:10.811 INFO:tasks.workunit.client.0.smithi119.stdout:2/77: write d2/d1a/f1f [448016,96370] 0 2024-05-27T09:29:10.813 INFO:tasks.workunit.client.0.smithi119.stdout:6/63: dwrite d0/f6 [0,4194304] 0 2024-05-27T09:29:10.813 INFO:tasks.workunit.client.0.smithi119.stdout:6/64: write d0/d2/f9 [2604718,87495] 0 2024-05-27T09:29:10.813 INFO:tasks.workunit.client.0.smithi119.stdout:6/65: creat d0/d11/f15 x:0 0 0 2024-05-27T09:29:10.813 INFO:tasks.workunit.client.0.smithi119.stdout:3/150: mkdir db/d10/d13/d33 0 2024-05-27T09:29:10.816 INFO:tasks.workunit.client.0.smithi119.stdout:9/64: dwrite f6 [0,4194304] 0 2024-05-27T09:29:10.816 INFO:tasks.workunit.client.0.smithi119.stdout:0/57: creat d2/d6/d9/ff x:0 0 0 2024-05-27T09:29:10.820 INFO:tasks.workunit.client.0.smithi119.stdout:8/68: dwrite d3/d8/fd [0,4194304] 0 2024-05-27T09:29:10.822 INFO:tasks.workunit.client.0.smithi119.stdout:6/66: creat d0/d2/de/f16 x:0 0 0 2024-05-27T09:29:10.823 INFO:tasks.workunit.client.0.smithi119.stdout:3/151: rename db/d10/l14 to db/d10/d13/d33/l34 0 2024-05-27T09:29:10.834 INFO:tasks.workunit.client.0.smithi119.stdout:4/61: rmdir d4/d6 39 2024-05-27T09:29:10.835 INFO:tasks.workunit.client.0.smithi119.stdout:9/65: dread f4 [0,4194304] 0 2024-05-27T09:29:10.836 INFO:tasks.workunit.client.0.smithi119.stdout:2/78: rmdir d2/da 39 2024-05-27T09:29:10.841 INFO:tasks.workunit.client.0.smithi119.stdout:6/67: link d0/d11/f15 d0/d2/de/f17 0 2024-05-27T09:29:10.842 INFO:tasks.workunit.client.0.smithi119.stdout:3/152: creat db/f35 x:0 0 0 2024-05-27T09:29:10.847 INFO:tasks.workunit.client.0.smithi119.stdout:4/62: link d4/d6/c7 d4/c12 0 2024-05-27T09:29:10.847 INFO:tasks.workunit.client.0.smithi119.stdout:8/69: read d3/fa [471888,28822] 0 2024-05-27T09:29:10.847 INFO:tasks.workunit.client.0.smithi119.stdout:8/70: getdents d3/de 0 2024-05-27T09:29:10.847 INFO:tasks.workunit.client.0.smithi119.stdout:8/71: creat d3/f16 x:0 0 0 2024-05-27T09:29:10.849 INFO:tasks.workunit.client.0.smithi119.stdout:8/72: symlink d3/de/l17 0 2024-05-27T09:29:10.850 INFO:tasks.workunit.client.0.smithi119.stdout:8/73: mknod d3/d8/c18 0 2024-05-27T09:29:10.850 INFO:tasks.workunit.client.0.smithi119.stdout:8/74: fsync d3/d8/fd 0 2024-05-27T09:29:10.854 INFO:tasks.workunit.client.0.smithi119.stdout:3/153: rmdir db/d10/d13 39 2024-05-27T09:29:10.854 INFO:tasks.workunit.client.0.smithi119.stdout:8/75: mknod d3/c19 0 2024-05-27T09:29:10.855 INFO:tasks.workunit.client.0.smithi119.stdout:9/66: truncate f6 268265 0 2024-05-27T09:29:10.864 INFO:tasks.workunit.client.0.smithi119.stdout:4/63: dread d4/d6/fa [0,4194304] 0 2024-05-27T09:29:10.864 INFO:tasks.workunit.client.0.smithi119.stdout:4/64: read d4/f8 [1992097,88215] 0 2024-05-27T09:29:10.864 INFO:tasks.workunit.client.0.smithi119.stdout:4/65: chown c0 0 1 2024-05-27T09:29:10.871 INFO:tasks.workunit.client.0.smithi119.stdout:0/58: write d2/d6/f8 [1353418,113920] 0 2024-05-27T09:29:10.871 INFO:tasks.workunit.client.0.smithi119.stdout:8/76: link d3/d8/f13 d3/f1a 0 2024-05-27T09:29:10.871 INFO:tasks.workunit.client.0.smithi119.stdout:9/67: rename cd to c12 0 2024-05-27T09:29:10.872 INFO:tasks.workunit.client.0.smithi119.stdout:6/68: getdents d0/d2/de 0 2024-05-27T09:29:10.872 INFO:tasks.workunit.client.0.smithi119.stdout:3/154: mkdir db/d36 0 2024-05-27T09:29:10.872 INFO:tasks.workunit.client.0.smithi119.stdout:3/155: write db/d10/d13/f2e [337471,12974] 0 2024-05-27T09:29:10.874 INFO:tasks.workunit.client.0.smithi119.stdout:4/66: creat d4/d6/f13 x:0 0 0 2024-05-27T09:29:10.879 INFO:tasks.workunit.client.0.smithi119.stdout:8/77: mkdir d3/d8/d1b 0 2024-05-27T09:29:10.879 INFO:tasks.workunit.client.0.smithi119.stdout:9/68: rename f4 to f13 0 2024-05-27T09:29:10.880 INFO:tasks.workunit.client.0.smithi119.stdout:4/67: mknod d4/de/c14 0 2024-05-27T09:29:10.887 INFO:tasks.workunit.client.0.smithi119.stdout:5/109: dwrite d6/db/dc/f21 [0,4194304] 0 2024-05-27T09:29:10.887 INFO:tasks.workunit.client.0.smithi119.stdout:5/110: dread - d6/db/de/f25 zero size 2024-05-27T09:29:10.901 INFO:tasks.workunit.client.0.smithi119.stdout:2/79: dwrite d2/da/fd [0,4194304] 0 2024-05-27T09:29:10.901 INFO:tasks.workunit.client.0.smithi119.stdout:8/78: dread d3/fa [0,4194304] 0 2024-05-27T09:29:10.908 INFO:tasks.workunit.client.0.smithi119.stdout:9/69: unlink f13 0 2024-05-27T09:29:10.908 INFO:tasks.workunit.client.0.smithi119.stdout:9/70: read - fa zero size 2024-05-27T09:29:10.908 INFO:tasks.workunit.client.0.smithi119.stdout:0/59: dread d2/d6/f8 [4194304,4194304] 0 2024-05-27T09:29:10.908 INFO:tasks.workunit.client.0.smithi119.stdout:0/60: stat d2/d6/fc 0 2024-05-27T09:29:10.909 INFO:tasks.workunit.client.0.smithi119.stdout:3/156: dread db/d10/f2d [0,4194304] 0 2024-05-27T09:29:10.909 INFO:tasks.workunit.client.0.smithi119.stdout:4/68: rmdir d4/d6 39 2024-05-27T09:29:10.911 INFO:tasks.workunit.client.0.smithi119.stdout:2/80: getdents d2/d1a 0 2024-05-27T09:29:10.915 INFO:tasks.workunit.client.0.smithi119.stdout:3/157: creat db/f37 x:0 0 0 2024-05-27T09:29:10.915 INFO:tasks.workunit.client.0.smithi119.stdout:4/69: symlink d4/d6/l15 0 2024-05-27T09:29:10.922 INFO:tasks.workunit.client.0.smithi119.stdout:1/95: sync 2024-05-27T09:29:10.929 INFO:tasks.workunit.client.0.smithi119.stdout:9/71: getdents . 0 2024-05-27T09:29:10.929 INFO:tasks.workunit.client.0.smithi119.stdout:7/196: sync 2024-05-27T09:29:10.929 INFO:tasks.workunit.client.0.smithi119.stdout:1/96: truncate f2 1233846 0 2024-05-27T09:29:10.930 INFO:tasks.workunit.client.0.smithi119.stdout:1/97: creat d7/db/d12/d15/f1d x:0 0 0 2024-05-27T09:29:10.942 INFO:tasks.workunit.client.0.smithi119.stdout:7/197: dread d7/d1a/f2b [0,4194304] 0 2024-05-27T09:29:10.948 INFO:tasks.workunit.client.0.smithi119.stdout:7/198: rename d7/df/d43 to d7/d1a/d40/d44 0 2024-05-27T09:29:10.949 INFO:tasks.workunit.client.0.smithi119.stdout:7/199: fsync d7/d1a/f2d 0 2024-05-27T09:29:10.949 INFO:tasks.workunit.client.0.smithi119.stdout:7/200: dread - d7/df/d19/f1f zero size 2024-05-27T09:29:10.970 INFO:tasks.workunit.client.0.smithi119.stdout:8/79: dwrite d3/de/ff [4194304,4194304] 0 2024-05-27T09:29:10.974 INFO:tasks.workunit.client.0.smithi119.stdout:8/80: read d3/d8/f13 [3072,26088] 0 2024-05-27T09:29:10.974 INFO:tasks.workunit.client.0.smithi119.stdout:8/81: readlink d3/d8/lc 0 2024-05-27T09:29:10.975 INFO:tasks.workunit.client.0.smithi119.stdout:8/82: dread - d3/f15 zero size 2024-05-27T09:29:10.975 INFO:tasks.workunit.client.0.smithi119.stdout:1/98: dread f2 [0,4194304] 0 2024-05-27T09:29:10.975 INFO:tasks.workunit.client.0.smithi119.stdout:1/99: creat d7/f1e x:0 0 0 2024-05-27T09:29:10.977 INFO:tasks.workunit.client.0.smithi119.stdout:8/83: link d3/de/ff d3/de/f1c 0 2024-05-27T09:29:10.980 INFO:tasks.workunit.client.0.smithi119.stdout:2/81: dwrite d2/da/fd [0,4194304] 0 2024-05-27T09:29:10.980 INFO:tasks.workunit.client.0.smithi119.stdout:8/84: unlink d3/c19 0 2024-05-27T09:29:10.982 INFO:tasks.workunit.client.0.smithi119.stdout:8/85: creat d3/de/f1d x:0 0 0 2024-05-27T09:29:10.983 INFO:tasks.workunit.client.0.smithi119.stdout:8/86: mknod d3/d8/c1e 0 2024-05-27T09:29:10.983 INFO:tasks.workunit.client.0.smithi119.stdout:8/87: readlink d3/d8/lc 0 2024-05-27T09:29:10.994 INFO:tasks.workunit.client.0.smithi119.stdout:8/88: getdents d3/d8 0 2024-05-27T09:29:10.994 INFO:tasks.workunit.client.0.smithi119.stdout:8/89: write d3/f15 [306902,61819] 0 2024-05-27T09:29:10.995 INFO:tasks.workunit.client.0.smithi119.stdout:8/90: write d3/d8/f14 [8045,64382] 0 2024-05-27T09:29:11.000 INFO:tasks.workunit.client.0.smithi119.stdout:0/61: dwrite d2/d6/f8 [4194304,4194304] 0 2024-05-27T09:29:11.000 INFO:tasks.workunit.client.0.smithi119.stdout:0/62: truncate d2/d6/d9/ff 361319 0 2024-05-27T09:29:11.007 INFO:tasks.workunit.client.0.smithi119.stdout:4/70: dread f1 [0,4194304] 0 2024-05-27T09:29:11.008 INFO:tasks.workunit.client.0.smithi119.stdout:4/71: symlink d4/de/l16 0 2024-05-27T09:29:11.011 INFO:tasks.workunit.client.0.smithi119.stdout:4/72: write d4/d6/f13 [118060,65296] 0 2024-05-27T09:29:11.013 INFO:tasks.workunit.client.0.smithi119.stdout:5/111: dwrite d6/db/de/f25 [0,4194304] 0 2024-05-27T09:29:11.014 INFO:tasks.workunit.client.0.smithi119.stdout:1/100: dread f6 [0,4194304] 0 2024-05-27T09:29:11.016 INFO:tasks.workunit.client.0.smithi119.stdout:3/158: dread f6 [0,4194304] 0 2024-05-27T09:29:11.018 INFO:tasks.workunit.client.0.smithi119.stdout:1/101: unlink d7/dd/c17 0 2024-05-27T09:29:11.018 INFO:tasks.workunit.client.0.smithi119.stdout:1/102: readlink d7/dd/l19 0 2024-05-27T09:29:11.021 INFO:tasks.workunit.client.0.smithi119.stdout:5/112: rmdir d6/db 39 2024-05-27T09:29:11.022 INFO:tasks.workunit.client.0.smithi119.stdout:1/103: mkdir d7/db/d12/d15/d1f 0 2024-05-27T09:29:11.032 INFO:tasks.workunit.client.0.smithi119.stdout:5/113: dread d6/db/de/f14 [0,4194304] 0 2024-05-27T09:29:11.034 INFO:tasks.workunit.client.0.smithi119.stdout:5/114: rmdir d6/dd/df 39 2024-05-27T09:29:11.035 INFO:tasks.workunit.client.0.smithi119.stdout:5/115: write d6/dd/f1b [749703,98793] 0 2024-05-27T09:29:11.038 INFO:tasks.workunit.client.0.smithi119.stdout:9/72: dwrite f8 [0,4194304] 0 2024-05-27T09:29:11.042 INFO:tasks.workunit.client.0.smithi119.stdout:9/73: symlink l14 0 2024-05-27T09:29:11.042 INFO:tasks.workunit.client.0.smithi119.stdout:9/74: chown c12 9981 1 2024-05-27T09:29:11.053 INFO:tasks.workunit.client.0.smithi119.stdout:7/201: dwrite d7/df/d19/f1f [0,4194304] 0 2024-05-27T09:29:11.058 INFO:tasks.workunit.client.0.smithi119.stdout:0/63: dread d2/d6/f8 [4194304,4194304] 0 2024-05-27T09:29:11.070 INFO:tasks.workunit.client.0.smithi119.stdout:2/82: dwrite d2/da/d15/f16 [0,4194304] 0 2024-05-27T09:29:11.070 INFO:tasks.workunit.client.0.smithi119.stdout:2/83: write d2/f6 [2096984,5855] 0 2024-05-27T09:29:11.070 INFO:tasks.workunit.client.0.smithi119.stdout:2/84: unlink d2/c8 0 2024-05-27T09:29:11.072 INFO:tasks.workunit.client.0.smithi119.stdout:2/85: unlink d2/l3 0 2024-05-27T09:29:11.072 INFO:tasks.workunit.client.0.smithi119.stdout:2/86: truncate d2/d1a/f1e 107122 0 2024-05-27T09:29:11.072 INFO:tasks.workunit.client.0.smithi119.stdout:0/64: dread d2/d6/f8 [4194304,4194304] 0 2024-05-27T09:29:11.074 INFO:tasks.workunit.client.0.smithi119.stdout:2/87: creat d2/d1a/f21 x:0 0 0 2024-05-27T09:29:11.076 INFO:tasks.workunit.client.0.smithi119.stdout:0/65: rename l0 to d2/l10 0 2024-05-27T09:29:11.077 INFO:tasks.workunit.client.0.smithi119.stdout:0/66: read d2/d6/d9/ff [232075,95070] 0 2024-05-27T09:29:11.079 INFO:tasks.workunit.client.0.smithi119.stdout:2/88: creat d2/da/d15/f22 x:0 0 0 2024-05-27T09:29:11.085 INFO:tasks.workunit.client.0.smithi119.stdout:0/67: rename d2/d4/c7 to d2/d6/db/c11 0 2024-05-27T09:29:11.085 INFO:tasks.workunit.client.0.smithi119.stdout:0/68: stat d2/d6 0 2024-05-27T09:29:11.093 INFO:tasks.workunit.client.0.smithi119.stdout:0/69: mkdir d2/d6/d9/d12 0 2024-05-27T09:29:11.095 INFO:tasks.workunit.client.0.smithi119.stdout:1/104: dwrite d7/f8 [0,4194304] 0 2024-05-27T09:29:11.099 INFO:tasks.workunit.client.0.smithi119.stdout:0/70: mknod d2/d6/db/c13 0 2024-05-27T09:29:11.125 INFO:tasks.workunit.client.0.smithi119.stdout:1/105: write f6 [868686,21358] 0 2024-05-27T09:29:11.125 INFO:tasks.workunit.client.0.smithi119.stdout:1/106: stat d7/dd/cf 0 2024-05-27T09:29:11.126 INFO:tasks.workunit.client.0.smithi119.stdout:1/107: symlink d7/l20 0 2024-05-27T09:29:11.126 INFO:tasks.workunit.client.0.smithi119.stdout:1/108: creat d7/db/d12/d15/f21 x:0 0 0 2024-05-27T09:29:11.126 INFO:tasks.workunit.client.0.smithi119.stdout:1/109: rename d7 to d7/d22 22 2024-05-27T09:29:11.128 INFO:tasks.workunit.client.0.smithi119.stdout:1/110: read d7/f14 [556482,108532] 0 2024-05-27T09:29:11.139 INFO:tasks.workunit.client.0.smithi119.stdout:5/116: dwrite d6/db/dc/f26 [0,4194304] 0 2024-05-27T09:29:11.144 INFO:tasks.workunit.client.0.smithi119.stdout:5/117: mknod d6/c28 0 2024-05-27T09:29:11.150 INFO:tasks.workunit.client.0.smithi119.stdout:4/73: dwrite d4/d6/f13 [0,4194304] 0 2024-05-27T09:29:11.214 INFO:tasks.workunit.client.0.smithi119.stdout:4/74: dread d4/d6/f13 [0,4194304] 0 2024-05-27T09:29:11.215 INFO:tasks.workunit.client.0.smithi119.stdout:4/75: readlink d4/d6/l9 0 2024-05-27T09:29:11.216 INFO:tasks.workunit.client.0.smithi119.stdout:4/76: truncate d4/f8 4394469 0 2024-05-27T09:29:11.223 INFO:tasks.workunit.client.0.smithi119.stdout:4/77: link f3 d4/d6/f17 0 2024-05-27T09:29:11.223 INFO:tasks.workunit.client.0.smithi119.stdout:4/78: chown d4 1 1 2024-05-27T09:29:11.226 INFO:tasks.workunit.client.0.smithi119.stdout:4/79: mknod d4/d6/c18 0 2024-05-27T09:29:11.246 INFO:tasks.workunit.client.0.smithi119.stdout:2/89: rmdir d2 39 2024-05-27T09:29:11.250 INFO:tasks.workunit.client.0.smithi119.stdout:2/90: unlink d2/da/d15/f22 0 2024-05-27T09:29:11.250 INFO:tasks.workunit.client.0.smithi119.stdout:2/91: fdatasync d2/da/f13 0 2024-05-27T09:29:11.253 INFO:tasks.workunit.client.0.smithi119.stdout:2/92: symlink d2/l23 0 2024-05-27T09:29:11.384 INFO:tasks.workunit.client.0.smithi119.stdout:9/75: dwrite fb [4194304,4194304] 0 2024-05-27T09:29:11.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:29:11.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:29:11.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:29:11.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:29:11.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:29:11.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:29:11.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr fail", "who": "smithi119.ncwmog"}]: dispatch 2024-05-27T09:29:11.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: osdmap e47: 6 total, 6 up, 6 in 2024-05-27T09:29:11.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "mgr fail", "who": "smithi119.ncwmog"}]': finished 2024-05-27T09:29:11.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: mgrmap e21: smithi138.dxsuqf(active, starting, since 0.022446s) 2024-05-27T09:29:11.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mon metadata", "id": "smithi119"}]: dispatch 2024-05-27T09:29:11.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:29:11.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.ppmzzk"}]: dispatch 2024-05-27T09:29:11.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.nghcey"}]: dispatch 2024-05-27T09:29:11.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.ndfkhd"}]: dispatch 2024-05-27T09:29:11.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.alszpk"}]: dispatch 2024-05-27T09:29:11.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mgr metadata", "who": "smithi138.dxsuqf", "id": "smithi138.dxsuqf"}]: dispatch 2024-05-27T09:29:11.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T09:29:11.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T09:29:11.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T09:29:11.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T09:29:11.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T09:29:11.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T09:29:11.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T09:29:11.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T09:29:11.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T09:29:11.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: Manager daemon smithi138.dxsuqf is now available 2024-05-27T09:29:11.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:29:11.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:29:11.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.dxsuqf/mirror_snapshot_schedule"}]: dispatch 2024-05-27T09:29:11.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.dxsuqf/mirror_snapshot_schedule"}]: dispatch 2024-05-27T09:29:11.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.dxsuqf/trash_purge_schedule"}]: dispatch 2024-05-27T09:29:11.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:11 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.dxsuqf/trash_purge_schedule"}]: dispatch 2024-05-27T09:29:11.391 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:29:11.391 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:29:11.391 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:29:11.391 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:29:11.391 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' 2024-05-27T09:29:11.391 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:29:11.391 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr fail", "who": "smithi119.ncwmog"}]: dispatch 2024-05-27T09:29:11.391 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: osdmap e47: 6 total, 6 up, 6 in 2024-05-27T09:29:11.391 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14213 172.21.15.119:0/3613874854' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "mgr fail", "who": "smithi119.ncwmog"}]': finished 2024-05-27T09:29:11.391 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: mgrmap e21: smithi138.dxsuqf(active, starting, since 0.022446s) 2024-05-27T09:29:11.391 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mon metadata", "id": "smithi119"}]: dispatch 2024-05-27T09:29:11.391 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:29:11.392 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.ppmzzk"}]: dispatch 2024-05-27T09:29:11.392 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.nghcey"}]: dispatch 2024-05-27T09:29:11.392 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.ndfkhd"}]: dispatch 2024-05-27T09:29:11.392 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.alszpk"}]: dispatch 2024-05-27T09:29:11.392 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mgr metadata", "who": "smithi138.dxsuqf", "id": "smithi138.dxsuqf"}]: dispatch 2024-05-27T09:29:11.392 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T09:29:11.392 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T09:29:11.392 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T09:29:11.392 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T09:29:11.392 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T09:29:11.392 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T09:29:11.392 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T09:29:11.392 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T09:29:11.392 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T09:29:11.392 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: Manager daemon smithi138.dxsuqf is now available 2024-05-27T09:29:11.393 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:29:11.393 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:29:11.393 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.dxsuqf/mirror_snapshot_schedule"}]: dispatch 2024-05-27T09:29:11.393 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.dxsuqf/mirror_snapshot_schedule"}]: dispatch 2024-05-27T09:29:11.393 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.dxsuqf/trash_purge_schedule"}]: dispatch 2024-05-27T09:29:11.393 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:11 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.dxsuqf/trash_purge_schedule"}]: dispatch 2024-05-27T09:29:11.393 INFO:tasks.workunit.client.0.smithi119.stdout:9/76: creat f15 x:0 0 0 2024-05-27T09:29:11.398 INFO:tasks.workunit.client.0.smithi119.stdout:9/77: truncate f2 2161704 0 2024-05-27T09:29:11.398 INFO:tasks.workunit.client.0.smithi119.stdout:9/78: readlink l14 0 2024-05-27T09:29:11.399 INFO:tasks.workunit.client.0.smithi119.stdout:9/79: creat f16 x:0 0 0 2024-05-27T09:29:11.624 INFO:tasks.workunit.client.0.smithi119.stdout:9/80: write f8 [3722202,7608] 0 2024-05-27T09:29:11.624 INFO:tasks.workunit.client.0.smithi119.stdout:9/81: creat f17 x:0 0 0 2024-05-27T09:29:11.627 INFO:tasks.workunit.client.0.smithi119.stdout:9/82: mkdir d18 0 2024-05-27T09:29:11.627 INFO:tasks.workunit.client.0.smithi119.stdout:9/83: getdents d18 0 2024-05-27T09:29:11.628 INFO:tasks.workunit.client.0.smithi119.stdout:9/84: mkdir d18/d19 0 2024-05-27T09:29:11.628 INFO:tasks.workunit.client.0.smithi119.stdout:9/85: write f10 [448471,88250] 0 2024-05-27T09:29:11.628 INFO:tasks.workunit.client.0.smithi119.stdout:9/86: write ff [260836,8371] 0 2024-05-27T09:29:11.630 INFO:tasks.workunit.client.0.smithi119.stdout:9/87: creat d18/f1a x:0 0 0 2024-05-27T09:29:11.632 INFO:tasks.workunit.client.0.smithi119.stdout:9/88: mkdir d18/d19/d1b 0 2024-05-27T09:29:11.634 INFO:tasks.workunit.client.0.smithi119.stdout:0/71: dwrite d2/d6/fc [0,4194304] 0 2024-05-27T09:29:11.634 INFO:tasks.workunit.client.0.smithi119.stdout:0/72: chown d2/le 28 1 2024-05-27T09:29:11.634 INFO:tasks.workunit.client.0.smithi119.stdout:9/89: creat d18/d19/f1c x:0 0 0 2024-05-27T09:29:11.642 INFO:tasks.workunit.client.0.smithi119.stdout:0/73: symlink d2/l14 0 2024-05-27T09:29:11.645 INFO:tasks.workunit.client.0.smithi119.stdout:9/90: link fc d18/d19/d1b/f1d 0 2024-05-27T09:29:11.645 INFO:tasks.workunit.client.0.smithi119.stdout:9/91: dread - d18/f1a zero size 2024-05-27T09:29:11.648 INFO:tasks.workunit.client.0.smithi119.stdout:9/92: rmdir d18/d19/d1b 39 2024-05-27T09:29:11.654 INFO:tasks.workunit.client.0.smithi119.stdout:9/93: creat d18/d19/d1b/f1e x:0 0 0 2024-05-27T09:29:11.776 INFO:tasks.workunit.client.0.smithi119.stdout:0/74: dread d2/d6/f8 [4194304,4194304] 0 2024-05-27T09:29:11.776 INFO:tasks.workunit.client.0.smithi119.stdout:0/75: chown d2/d6/db 230191777 1 2024-05-27T09:29:11.783 INFO:tasks.workunit.client.0.smithi119.stdout:0/76: link d2/d6/d9/ld d2/l15 0 2024-05-27T09:29:11.787 INFO:tasks.workunit.client.0.smithi119.stdout:4/80: dwrite d4/d6/fa [0,4194304] 0 2024-05-27T09:29:11.819 INFO:tasks.workunit.client.0.smithi119.stdout:1/111: getdents d7 0 2024-05-27T09:29:11.837 INFO:tasks.workunit.client.0.smithi119.stdout:5/118: dwrite d6/db/de/f14 [0,4194304] 0 2024-05-27T09:29:11.837 INFO:tasks.workunit.client.0.smithi119.stdout:5/119: dread - d6/f17 zero size 2024-05-27T09:29:11.855 INFO:tasks.workunit.client.0.smithi119.stdout:5/120: symlink d6/dd/df/d13/d19/l29 0 2024-05-27T09:29:11.859 INFO:tasks.workunit.client.0.smithi119.stdout:2/93: truncate d2/da/d15/f16 351102 0 2024-05-27T09:29:11.859 INFO:tasks.workunit.client.0.smithi119.stdout:2/94: write d2/f19 [677519,50252] 0 2024-05-27T09:29:11.860 INFO:tasks.workunit.client.0.smithi119.stdout:2/95: stat d2/da/fd 0 2024-05-27T09:29:11.860 INFO:tasks.workunit.client.0.smithi119.stdout:5/121: creat d6/db/de/d27/f2a x:0 0 0 2024-05-27T09:29:11.889 INFO:tasks.workunit.client.0.smithi119.stdout:5/122: dread d6/dd/f1b [0,4194304] 0 2024-05-27T09:29:11.891 INFO:tasks.workunit.client.0.smithi119.stdout:5/123: mknod d6/db/de/c2b 0 2024-05-27T09:29:11.891 INFO:tasks.workunit.client.0.smithi119.stdout:5/124: readlink d6/dd/df/d13/d19/l29 0 2024-05-27T09:29:11.894 INFO:tasks.workunit.client.0.smithi119.stdout:5/125: symlink d6/db/l2c 0 2024-05-27T09:29:11.906 INFO:tasks.workunit.client.0.smithi119.stdout:4/81: dread d4/d6/fa [0,4194304] 0 2024-05-27T09:29:11.907 INFO:tasks.workunit.client.0.smithi119.stdout:4/82: mknod d4/de/c19 0 2024-05-27T09:29:11.908 INFO:tasks.workunit.client.0.smithi119.stdout:4/83: fdatasync d4/d6/fa 0 2024-05-27T09:29:11.908 INFO:tasks.workunit.client.0.smithi119.stdout:4/84: chown d4/d6/c7 380712 1 2024-05-27T09:29:11.908 INFO:tasks.workunit.client.0.smithi119.stdout:4/85: stat d4/d6/f17 0 2024-05-27T09:29:11.981 INFO:tasks.workunit.client.0.smithi119.stdout:4/86: dread d4/d6/f17 [0,4194304] 0 2024-05-27T09:29:11.982 INFO:tasks.workunit.client.0.smithi119.stdout:4/87: write f1 [6059892,41373] 0 2024-05-27T09:29:12.037 INFO:tasks.workunit.client.0.smithi119.stdout:2/96: dread d2/d1a/f1e [0,4194304] 0 2024-05-27T09:29:12.104 INFO:tasks.workunit.client.0.smithi119.stdout:1/112: dwrite d7/f8 [0,4194304] 0 2024-05-27T09:29:12.104 INFO:tasks.workunit.client.0.smithi119.stdout:1/113: write d7/f11 [260353,56662] 0 2024-05-27T09:29:12.109 INFO:tasks.workunit.client.0.smithi119.stdout:3/159: sync 2024-05-27T09:29:12.109 INFO:tasks.workunit.client.0.smithi119.stdout:7/202: sync 2024-05-27T09:29:12.109 INFO:tasks.workunit.client.0.smithi119.stdout:6/69: sync 2024-05-27T09:29:12.109 INFO:tasks.workunit.client.0.smithi119.stdout:8/91: sync 2024-05-27T09:29:12.123 INFO:tasks.workunit.client.0.smithi119.stdout:9/94: dwrite d18/d19/d1b/f1d [0,4194304] 0 2024-05-27T09:29:12.123 INFO:tasks.workunit.client.0.smithi119.stdout:9/95: chown fe 5 1 2024-05-27T09:29:12.123 INFO:tasks.workunit.client.0.smithi119.stdout:9/96: creat d18/d19/d1b/f1f x:0 0 0 2024-05-27T09:29:12.131 INFO:tasks.workunit.client.0.smithi119.stdout:7/203: mkdir d7/df/d36/d45 0 2024-05-27T09:29:12.133 INFO:tasks.workunit.client.0.smithi119.stdout:8/92: truncate d3/d8/f13 19450 0 2024-05-27T09:29:12.133 INFO:tasks.workunit.client.0.smithi119.stdout:8/93: chown d3/d8/f14 1776 1 2024-05-27T09:29:12.137 INFO:tasks.workunit.client.0.smithi119.stdout:9/97: mknod d18/c20 0 2024-05-27T09:29:12.139 INFO:tasks.workunit.client.0.smithi119.stdout:7/204: mkdir d7/df/d19/d46 0 2024-05-27T09:29:12.140 INFO:tasks.workunit.client.0.smithi119.stdout:8/94: creat d3/d8/f1f x:0 0 0 2024-05-27T09:29:12.142 INFO:tasks.workunit.client.0.smithi119.stdout:9/98: link d18/d19/f1c d18/f21 0 2024-05-27T09:29:12.142 INFO:tasks.workunit.client.0.smithi119.stdout:9/99: read f2 [1735827,33083] 0 2024-05-27T09:29:12.153 INFO:tasks.workunit.client.0.smithi119.stdout:7/205: mknod d7/d1a/d40/c47 0 2024-05-27T09:29:12.157 INFO:tasks.workunit.client.0.smithi119.stdout:7/206: creat d7/d1a/d40/f48 x:0 0 0 2024-05-27T09:29:12.178 INFO:tasks.workunit.client.0.smithi119.stdout:1/114: dread d7/f8 [0,4194304] 0 2024-05-27T09:29:12.179 INFO:tasks.workunit.client.0.smithi119.stdout:1/115: write f2 [1519121,87415] 0 2024-05-27T09:29:12.179 INFO:tasks.workunit.client.0.smithi119.stdout:1/116: chown d7/f8 3922 1 2024-05-27T09:29:12.188 INFO:tasks.workunit.client.0.smithi119.stdout:9/100: dread f6 [0,4194304] 0 2024-05-27T09:29:12.189 INFO:tasks.workunit.client.0.smithi119.stdout:9/101: chown fe 4 1 2024-05-27T09:29:12.189 INFO:tasks.workunit.client.0.smithi119.stdout:9/102: creat d18/f22 x:0 0 0 2024-05-27T09:29:12.189 INFO:tasks.workunit.client.0.smithi119.stdout:9/103: creat d18/f23 x:0 0 0 2024-05-27T09:29:12.189 INFO:tasks.workunit.client.0.smithi119.stdout:9/104: chown f6 7797692 1 2024-05-27T09:29:12.190 INFO:tasks.workunit.client.0.smithi119.stdout:9/105: write d18/f1a [364810,67471] 0 2024-05-27T09:29:12.190 INFO:tasks.workunit.client.0.smithi119.stdout:9/106: truncate d18/f21 908833 0 2024-05-27T09:29:12.190 INFO:tasks.workunit.client.0.smithi119.stdout:9/107: write d18/d19/d1b/f1e [492109,108970] 0 2024-05-27T09:29:12.191 INFO:tasks.workunit.client.0.smithi119.stdout:1/117: creat d7/db/d12/f23 x:0 0 0 2024-05-27T09:29:12.197 INFO:tasks.workunit.client.0.smithi119.stdout:4/88: dwrite d4/f8 [0,4194304] 0 2024-05-27T09:29:12.204 INFO:tasks.workunit.client.0.smithi119.stdout:1/118: creat d7/db/d12/d15/d1f/f24 x:0 0 0 2024-05-27T09:29:12.204 INFO:tasks.workunit.client.0.smithi119.stdout:4/89: creat d4/d6/f1a x:0 0 0 2024-05-27T09:29:12.206 INFO:tasks.workunit.client.0.smithi119.stdout:4/90: symlink d4/de/l1b 0 2024-05-27T09:29:12.220 INFO:tasks.workunit.client.0.smithi119.stdout:7/207: write d7/df/d19/f14 [1122186,16804] 0 2024-05-27T09:29:12.220 INFO:tasks.workunit.client.0.smithi119.stdout:7/208: fdatasync d7/d1a/f2d 0 2024-05-27T09:29:12.220 INFO:tasks.workunit.client.0.smithi119.stdout:7/209: creat d7/df/d36/f49 x:0 0 0 2024-05-27T09:29:12.220 INFO:tasks.workunit.client.0.smithi119.stdout:7/210: creat d7/d1a/f4a x:0 0 0 2024-05-27T09:29:12.240 INFO:tasks.workunit.client.0.smithi119.stdout:7/211: symlink d7/df/d19/l4b 0 2024-05-27T09:29:12.242 INFO:tasks.workunit.client.0.smithi119.stdout:4/91: dread d4/f8 [0,4194304] 0 2024-05-27T09:29:12.244 INFO:tasks.workunit.client.0.smithi119.stdout:4/92: mkdir d4/d1c 0 2024-05-27T09:29:12.244 INFO:tasks.workunit.client.0.smithi119.stdout:4/93: fsync d4/f8 0 2024-05-27T09:29:12.253 INFO:tasks.workunit.client.0.smithi119.stdout:3/160: write f8 [6716025,15836] 0 2024-05-27T09:29:12.262 INFO:tasks.workunit.client.0.smithi119.stdout:3/161: truncate db/d10/f2d 7131 0 2024-05-27T09:29:12.269 INFO:tasks.workunit.client.0.smithi119.stdout:4/94: write d4/f8 [1905603,89013] 0 2024-05-27T09:29:12.272 INFO:tasks.workunit.client.0.smithi119.stdout:4/95: mkdir d4/d6/d1d 0 2024-05-27T09:29:12.273 INFO:tasks.workunit.client.0.smithi119.stdout:0/77: dread d2/d6/fc [0,4194304] 0 2024-05-27T09:29:12.275 INFO:tasks.workunit.client.0.smithi119.stdout:4/96: truncate d4/d6/f17 2949060 0 2024-05-27T09:29:12.276 INFO:tasks.workunit.client.0.smithi119.stdout:0/78: creat d2/d6/d9/d12/f16 x:0 0 0 2024-05-27T09:29:12.276 INFO:tasks.workunit.client.0.smithi119.stdout:0/79: truncate d2/d6/d9/ff 1165367 0 2024-05-27T09:29:12.276 INFO:tasks.workunit.client.0.smithi119.stdout:0/80: chown d2/d6/d9/d12 21 1 2024-05-27T09:29:12.277 INFO:tasks.workunit.client.0.smithi119.stdout:4/97: creat d4/d1c/f1e x:0 0 0 2024-05-27T09:29:12.285 INFO:tasks.workunit.client.0.smithi119.stdout:0/81: mknod d2/d6/db/c17 0 2024-05-27T09:29:12.285 INFO:tasks.workunit.client.0.smithi119.stdout:0/82: fdatasync d2/d6/fc 0 2024-05-27T09:29:12.303 INFO:tasks.workunit.client.0.smithi119.stdout:7/212: write d7/fa [2702647,119477] 0 2024-05-27T09:29:12.310 INFO:tasks.workunit.client.0.smithi119.stdout:3/162: dread db/d10/d13/f2c [0,4194304] 0 2024-05-27T09:29:12.389 INFO:tasks.workunit.client.0.smithi119.stdout:7/213: dread d7/d1a/f20 [0,4194304] 0 2024-05-27T09:29:12.394 INFO:tasks.workunit.client.0.smithi119.stdout:7/214: mknod d7/db/d37/c4c 0 2024-05-27T09:29:12.394 INFO:tasks.workunit.client.0.smithi119.stdout:7/215: write d7/df/f10 [747053,34493] 0 2024-05-27T09:29:12.436 INFO:tasks.workunit.client.0.smithi119.stdout:6/70: dwrite d0/fb [0,4194304] 0 2024-05-27T09:29:12.436 INFO:tasks.workunit.client.0.smithi119.stdout:6/71: fdatasync d0/d2/f9 0 2024-05-27T09:29:12.436 INFO:tasks.workunit.client.0.smithi119.stdout:6/72: truncate d0/d11/f15 1002342 0 2024-05-27T09:29:12.437 INFO:tasks.workunit.client.0.smithi119.stdout:6/73: fdatasync d0/d11/f14 0 2024-05-27T09:29:12.437 INFO:tasks.workunit.client.0.smithi119.stdout:6/74: stat d0/la 0 2024-05-27T09:29:12.437 INFO:tasks.workunit.client.0.smithi119.stdout:6/75: chown d0/fb 0 1 2024-05-27T09:29:12.437 INFO:tasks.workunit.client.0.smithi119.stdout:6/76: truncate d0/d11/f14 858178 0 2024-05-27T09:29:12.468 INFO:tasks.workunit.client.0.smithi119.stdout:6/77: dread d0/f6 [0,4194304] 0 2024-05-27T09:29:12.469 INFO:tasks.workunit.client.0.smithi119.stdout:6/78: creat d0/f18 x:0 0 0 2024-05-27T09:29:12.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:12 smithi119 ceph-mon[29082]: mgrmap e22: smithi138.dxsuqf(active, since 1.03723s) 2024-05-27T09:29:12.500 INFO:tasks.workunit.client.0.smithi119.stdout:6/79: dread d0/f6 [0,4194304] 0 2024-05-27T09:29:12.501 INFO:tasks.workunit.client.0.smithi119.stdout:6/80: mkdir d0/d19 0 2024-05-27T09:29:12.502 INFO:tasks.workunit.client.0.smithi119.stdout:6/81: write d0/d2/de/f17 [1904738,67754] 0 2024-05-27T09:29:12.502 INFO:tasks.workunit.client.0.smithi119.stdout:6/82: fdatasync d0/d2/de/f16 0 2024-05-27T09:29:12.502 INFO:tasks.workunit.client.0.smithi119.stdout:6/83: stat d0/f6 0 2024-05-27T09:29:12.506 INFO:tasks.workunit.client.0.smithi119.stdout:6/84: symlink d0/d2/l1a 0 2024-05-27T09:29:12.514 INFO:tasks.workunit.client.0.smithi119.stdout:3/163: read db/d10/d13/f2e [313913,125552] 0 2024-05-27T09:29:12.514 INFO:tasks.workunit.client.0.smithi119.stdout:3/164: write db/f30 [300596,120737] 0 2024-05-27T09:29:12.518 INFO:tasks.workunit.client.0.smithi119.stdout:3/165: creat db/d10/f38 x:0 0 0 2024-05-27T09:29:12.525 INFO:tasks.workunit.client.0.smithi119.stdout:9/108: dwrite d18/f23 [0,4194304] 0 2024-05-27T09:29:12.527 INFO:tasks.workunit.client.0.smithi119.stdout:3/166: mkdir db/d10/d13/d39 0 2024-05-27T09:29:12.542 INFO:tasks.workunit.client.0.smithi119.stdout:0/83: dwrite d2/d6/d9/ff [0,4194304] 0 2024-05-27T09:29:12.558 INFO:tasks.workunit.client.0.smithi119.stdout:8/95: rename d3/d8/f13 to d3/d8/d1b/f20 0 2024-05-27T09:29:12.558 INFO:tasks.workunit.client.0.smithi119.stdout:8/96: write d3/de/f10 [892220,84816] 0 2024-05-27T09:29:12.561 INFO:tasks.workunit.client.0.smithi119.stdout:8/97: rename d3/d8/f1f to d3/de/f21 0 2024-05-27T09:29:12.569 INFO:tasks.workunit.client.0.smithi119.stdout:7/216: rename d7/d1a/d41 to d7/df/d19/d4d 0 2024-05-27T09:29:12.570 INFO:tasks.workunit.client.0.smithi119.stdout:7/217: chown d7/df/d19/l34 16181444 1 2024-05-27T09:29:12.570 INFO:tasks.workunit.client.0.smithi119.stdout:7/218: truncate d7/db/f2f 1007190 0 2024-05-27T09:29:12.571 INFO:tasks.workunit.client.0.smithi119.stdout:8/98: creat d3/d8/d1b/f22 x:0 0 0 2024-05-27T09:29:12.577 INFO:tasks.workunit.client.0.smithi119.stdout:3/167: rename db/ff to db/d10/f3a 0 2024-05-27T09:29:12.580 INFO:tasks.workunit.client.0.smithi119.stdout:1/119: fsync d7/db/d12/d15/d1f/f24 0 2024-05-27T09:29:12.581 INFO:tasks.workunit.client.0.smithi119.stdout:1/120: readlink d7/dd/l18 0 2024-05-27T09:29:12.581 INFO:tasks.workunit.client.0.smithi119.stdout:8/99: rename d3/fa to d3/de/f23 0 2024-05-27T09:29:12.590 INFO:tasks.workunit.client.0.smithi119.stdout:1/121: mkdir d7/dd/d25 0 2024-05-27T09:29:12.603 INFO:tasks.workunit.client.0.smithi119.stdout:1/122: mkdir d7/db/d12/d26 0 2024-05-27T09:29:12.608 INFO:tasks.workunit.client.0.smithi119.stdout:6/85: truncate d0/d2/de/f17 378675 0 2024-05-27T09:29:12.611 INFO:tasks.workunit.client.0.smithi119.stdout:1/123: creat d7/f27 x:0 0 0 2024-05-27T09:29:12.613 INFO:tasks.workunit.client.0.smithi119.stdout:6/86: rename d0/l1 to d0/d11/d12/l1b 0 2024-05-27T09:29:12.620 INFO:tasks.workunit.client.0.smithi119.stdout:1/124: mknod d7/db/d12/c28 0 2024-05-27T09:29:12.620 INFO:tasks.workunit.client.0.smithi119.stdout:9/109: dwrite f8 [0,4194304] 0 2024-05-27T09:29:12.620 INFO:tasks.workunit.client.0.smithi119.stdout:9/110: creat d18/d19/d1b/f24 x:0 0 0 2024-05-27T09:29:12.623 INFO:tasks.workunit.client.0.smithi119.stdout:1/125: mkdir d7/dd/d25/d29 0 2024-05-27T09:29:12.623 INFO:tasks.workunit.client.0.smithi119.stdout:1/126: dread - d7/f27 zero size 2024-05-27T09:29:12.623 INFO:tasks.workunit.client.0.smithi119.stdout:1/127: write d7/db/d12/d15/f1d [242972,88010] 0 2024-05-27T09:29:12.623 INFO:tasks.workunit.client.0.smithi119.stdout:1/128: truncate d7/db/d12/d15/d1f/f24 727176 0 2024-05-27T09:29:12.631 INFO:tasks.workunit.client.0.smithi119.stdout:1/129: mknod d7/db/d12/d15/c2a 0 2024-05-27T09:29:12.639 INFO:tasks.workunit.client.0.smithi119.stdout:1/130: mknod d7/db/c2b 0 2024-05-27T09:29:12.644 INFO:tasks.workunit.client.0.smithi119.stdout:1/131: getdents d7/dd 0 2024-05-27T09:29:12.644 INFO:tasks.workunit.client.0.smithi119.stdout:1/132: fsync f2 0 2024-05-27T09:29:12.645 INFO:tasks.workunit.client.0.smithi119.stdout:1/133: mknod d7/db/d12/d15/d1f/c2c 0 2024-05-27T09:29:12.647 INFO:tasks.workunit.client.0.smithi119.stdout:8/100: dwrite d3/de/f1c [4194304,4194304] 0 2024-05-27T09:29:12.647 INFO:tasks.workunit.client.0.smithi119.stdout:4/98: dwrite f3 [0,4194304] 0 2024-05-27T09:29:12.647 INFO:tasks.workunit.client.0.smithi119.stdout:4/99: readlink d4/de/lf 0 2024-05-27T09:29:12.649 INFO:tasks.workunit.client.0.smithi119.stdout:1/134: write d7/f14 [142127,11237] 0 2024-05-27T09:29:12.653 INFO:tasks.workunit.client.0.smithi119.stdout:8/101: mkdir d3/d24 0 2024-05-27T09:29:12.654 INFO:tasks.workunit.client.0.smithi119.stdout:8/102: chown d3/f16 194407883 1 2024-05-27T09:29:12.654 INFO:tasks.workunit.client.0.smithi119.stdout:8/103: dread - d3/f16 zero size 2024-05-27T09:29:12.654 INFO:tasks.workunit.client.0.smithi119.stdout:8/104: creat d3/f25 x:0 0 0 2024-05-27T09:29:12.654 INFO:tasks.workunit.client.0.smithi119.stdout:4/100: mknod d4/de/c1f 0 2024-05-27T09:29:12.654 INFO:tasks.workunit.client.0.smithi119.stdout:4/101: readlink d4/de/lf 0 2024-05-27T09:29:12.656 INFO:tasks.workunit.client.0.smithi119.stdout:3/168: dwrite db/d10/d13/f21 [4194304,4194304] 0 2024-05-27T09:29:12.656 INFO:tasks.workunit.client.0.smithi119.stdout:3/169: fdatasync f6 0 2024-05-27T09:29:12.656 INFO:tasks.workunit.client.0.smithi119.stdout:3/170: readlink db/d10/l25 0 2024-05-27T09:29:12.657 INFO:tasks.workunit.client.0.smithi119.stdout:8/105: creat d3/d24/f26 x:0 0 0 2024-05-27T09:29:12.657 INFO:tasks.workunit.client.0.smithi119.stdout:3/171: stat db/d10/l25 0 2024-05-27T09:29:12.662 INFO:tasks.workunit.client.0.smithi119.stdout:3/172: mknod db/d10/d13/c3b 0 2024-05-27T09:29:12.672 INFO:tasks.workunit.client.0.smithi119.stdout:1/135: dread d7/dd/f13 [0,4194304] 0 2024-05-27T09:29:12.675 INFO:tasks.workunit.client.0.smithi119.stdout:1/136: symlink d7/dd/l2d 0 2024-05-27T09:29:12.677 INFO:tasks.workunit.client.0.smithi119.stdout:1/137: unlink d7/l20 0 2024-05-27T09:29:12.689 INFO:tasks.workunit.client.0.smithi119.stdout:3/173: write db/d10/f17 [4541758,54220] 0 2024-05-27T09:29:12.690 INFO:tasks.workunit.client.0.smithi119.stdout:3/174: chown f8 17 1 2024-05-27T09:29:12.690 INFO:tasks.workunit.client.0.smithi119.stdout:3/175: mknod db/d10/d13/d33/c3c 0 2024-05-27T09:29:12.694 INFO:tasks.workunit.client.0.smithi119.stdout:3/176: creat db/d10/d13/f3d x:0 0 0 2024-05-27T09:29:12.697 INFO:tasks.workunit.client.0.smithi119.stdout:3/177: read db/d10/d13/f28 [4547,115076] 0 2024-05-27T09:29:12.697 INFO:tasks.workunit.client.0.smithi119.stdout:3/178: truncate db/f35 538646 0 2024-05-27T09:29:12.698 INFO:tasks.workunit.client.0.smithi119.stdout:3/179: creat db/d10/d13/d39/f3e x:0 0 0 2024-05-27T09:29:12.698 INFO:tasks.workunit.client.0.smithi119.stdout:3/180: chown db/d10/c1e 3 1 2024-05-27T09:29:12.702 INFO:tasks.workunit.client.0.smithi119.stdout:3/181: link db/d10/d13/f3d db/d10/f3f 0 2024-05-27T09:29:12.705 INFO:tasks.workunit.client.0.smithi119.stdout:3/182: creat db/d10/d13/d33/f40 x:0 0 0 2024-05-27T09:29:12.721 INFO:tasks.workunit.client.0.smithi119.stdout:3/183: dread db/d10/d13/f2c [0,4194304] 0 2024-05-27T09:29:12.722 INFO:tasks.workunit.client.0.smithi119.stdout:6/87: dwrite d0/d2/f10 [0,4194304] 0 2024-05-27T09:29:12.722 INFO:tasks.workunit.client.0.smithi119.stdout:8/106: dwrite d3/d8/f14 [0,4194304] 0 2024-05-27T09:29:12.722 INFO:tasks.workunit.client.0.smithi119.stdout:8/107: chown d3/f16 6940 1 2024-05-27T09:29:12.722 INFO:tasks.workunit.client.0.smithi119.stdout:3/184: mknod db/d36/c41 0 2024-05-27T09:29:12.727 INFO:tasks.workunit.client.0.smithi119.stdout:6/88: mkdir d0/d2/d1c 0 2024-05-27T09:29:12.727 INFO:tasks.workunit.client.0.smithi119.stdout:6/89: write d0/d2/de/f16 [790202,35608] 0 2024-05-27T09:29:12.736 INFO:tasks.workunit.client.0.smithi119.stdout:8/108: rename d3/d8/c1e to d3/d8/c27 0 2024-05-27T09:29:12.736 INFO:tasks.workunit.client.0.smithi119.stdout:6/90: symlink d0/d11/d12/l1d 0 2024-05-27T09:29:12.736 INFO:tasks.workunit.client.0.smithi119.stdout:6/91: creat d0/f1e x:0 0 0 2024-05-27T09:29:12.739 INFO:tasks.workunit.client.0.smithi119.stdout:1/138: dread f6 [0,4194304] 0 2024-05-27T09:29:12.739 INFO:tasks.workunit.client.0.smithi119.stdout:1/139: stat d7/db/d12/d26 0 2024-05-27T09:29:12.739 INFO:tasks.workunit.client.0.smithi119.stdout:1/140: getdents d7/db/d12/d26 0 2024-05-27T09:29:12.739 INFO:tasks.workunit.client.0.smithi119.stdout:1/141: dread - d7/db/d12/f23 zero size 2024-05-27T09:29:12.739 INFO:tasks.workunit.client.0.smithi119.stdout:1/142: write d7/f1e [539647,125332] 0 2024-05-27T09:29:12.739 INFO:tasks.workunit.client.0.smithi119.stdout:1/143: chown d7/db/d12/d15/d1f/f24 682 1 2024-05-27T09:29:12.739 INFO:tasks.workunit.client.0.smithi119.stdout:8/109: getdents d3/de 0 2024-05-27T09:29:12.744 INFO:tasks.workunit.client.0.smithi119.stdout:1/144: symlink d7/dd/d25/d29/l2e 0 2024-05-27T09:29:12.744 INFO:tasks.workunit.client.0.smithi119.stdout:8/110: unlink d3/d8/f14 0 2024-05-27T09:29:12.745 INFO:tasks.workunit.client.0.smithi119.stdout:8/111: getdents d3 0 2024-05-27T09:29:12.746 INFO:tasks.workunit.client.0.smithi119.stdout:8/112: creat d3/d8/d1b/f28 x:0 0 0 2024-05-27T09:29:12.748 INFO:tasks.workunit.client.0.smithi119.stdout:4/102: dwrite d4/d1c/f1e [0,4194304] 0 2024-05-27T09:29:12.751 INFO:tasks.workunit.client.0.smithi119.stdout:4/103: truncate d4/f8 4350885 0 2024-05-27T09:29:12.756 INFO:tasks.workunit.client.0.smithi119.stdout:8/113: dread d3/f15 [0,4194304] 0 2024-05-27T09:29:12.756 INFO:tasks.workunit.client.0.smithi119.stdout:8/114: write d3/d8/d1b/f20 [415922,52453] 0 2024-05-27T09:29:12.756 INFO:tasks.workunit.client.0.smithi119.stdout:8/115: dread - d3/f25 zero size 2024-05-27T09:29:12.758 INFO:tasks.workunit.client.0.smithi119.stdout:8/116: getdents d3/d8 0 2024-05-27T09:29:12.758 INFO:tasks.workunit.client.0.smithi119.stdout:8/117: chown f1 30993 1 2024-05-27T09:29:12.758 INFO:tasks.workunit.client.0.smithi119.stdout:8/118: truncate d3/d8/d1b/f20 849016 0 2024-05-27T09:29:12.758 INFO:tasks.workunit.client.0.smithi119.stdout:8/119: chown d3/d8 5116 1 2024-05-27T09:29:12.758 INFO:tasks.workunit.client.0.smithi119.stdout:8/120: readlink d3/de/l17 0 2024-05-27T09:29:12.758 INFO:tasks.workunit.client.0.smithi119.stdout:8/121: chown d3/d8/c27 24805625 1 2024-05-27T09:29:12.761 INFO:tasks.workunit.client.0.smithi119.stdout:8/122: rmdir d3/d8/d1b 39 2024-05-27T09:29:12.763 INFO:tasks.workunit.client.0.smithi119.stdout:8/123: rename d3/d8/d1b/f22 to d3/d24/f29 0 2024-05-27T09:29:12.763 INFO:tasks.workunit.client.0.smithi119.stdout:8/124: truncate d3/d24/f26 399908 0 2024-05-27T09:29:12.769 INFO:tasks.workunit.client.0.smithi119.stdout:8/125: creat d3/d8/d1b/f2a x:0 0 0 2024-05-27T09:29:12.779 INFO:tasks.workunit.client.0.smithi119.stdout:4/104: read d4/d1c/f1e [1031113,61349] 0 2024-05-27T09:29:12.779 INFO:tasks.workunit.client.0.smithi119.stdout:4/105: write d4/d6/f17 [4352022,11227] 0 2024-05-27T09:29:12.784 INFO:tasks.workunit.client.0.smithi119.stdout:3/185: dwrite db/f12 [0,4194304] 0 2024-05-27T09:29:12.787 INFO:tasks.workunit.client.0.smithi119.stdout:3/186: mknod db/d10/d13/c42 0 2024-05-27T09:29:12.787 INFO:tasks.workunit.client.0.smithi119.stdout:3/187: write db/d10/f38 [588518,54427] 0 2024-05-27T09:29:12.801 INFO:tasks.workunit.client.0.smithi119.stdout:4/106: write f1 [1428245,51984] 0 2024-05-27T09:29:12.803 INFO:tasks.workunit.client.0.smithi119.stdout:3/188: mkdir db/d10/d13/d33/d43 0 2024-05-27T09:29:12.804 INFO:tasks.workunit.client.0.smithi119.stdout:3/189: read - db/d10/d13/f3d zero size 2024-05-27T09:29:12.804 INFO:tasks.workunit.client.0.smithi119.stdout:6/92: dwrite d0/fd [0,4194304] 0 2024-05-27T09:29:12.811 INFO:tasks.workunit.client.0.smithi119.stdout:1/145: dwrite d7/dd/f13 [0,4194304] 0 2024-05-27T09:29:12.811 INFO:tasks.workunit.client.0.smithi119.stdout:1/146: fsync d7/db/d12/d15/f21 0 2024-05-27T09:29:12.811 INFO:tasks.workunit.client.0.smithi119.stdout:1/147: write d7/f11 [949857,70564] 0 2024-05-27T09:29:12.813 INFO:tasks.workunit.client.0.smithi119.stdout:1/148: creat d7/db/d12/d26/f2f x:0 0 0 2024-05-27T09:29:12.817 INFO:tasks.workunit.client.0.smithi119.stdout:1/149: link d7/db/d12/d15/f1d d7/dd/d25/f30 0 2024-05-27T09:29:12.819 INFO:tasks.workunit.client.0.smithi119.stdout:1/150: rename d7/db/d12/d15/d1f/f24 to d7/db/d12/d15/d1f/f31 0 2024-05-27T09:29:12.823 INFO:tasks.workunit.client.0.smithi119.stdout:1/151: write d7/db/d12/d15/f21 [556374,12908] 0 2024-05-27T09:29:12.825 INFO:tasks.workunit.client.0.smithi119.stdout:8/126: dwrite d3/d24/f26 [0,4194304] 0 2024-05-27T09:29:12.826 INFO:tasks.workunit.client.0.smithi119.stdout:8/127: creat d3/d24/f2b x:0 0 0 2024-05-27T09:29:12.843 INFO:tasks.workunit.client.0.smithi119.stdout:8/128: write d3/d8/fd [2429963,97210] 0 2024-05-27T09:29:12.847 INFO:tasks.workunit.client.0.smithi119.stdout:8/129: creat d3/de/f2c x:0 0 0 2024-05-27T09:29:12.847 INFO:tasks.workunit.client.0.smithi119.stdout:8/130: write d3/de/f1d [492748,94858] 0 2024-05-27T09:29:12.847 INFO:tasks.workunit.client.0.smithi119.stdout:8/131: dread - d3/de/f2c zero size 2024-05-27T09:29:12.847 INFO:tasks.workunit.client.0.smithi119.stdout:8/132: chown d3/de/f1d 1242 1 2024-05-27T09:29:12.849 INFO:tasks.workunit.client.0.smithi119.stdout:8/133: creat d3/f2d x:0 0 0 2024-05-27T09:29:12.852 INFO:tasks.workunit.client.0.smithi119.stdout:8/134: mknod d3/d8/c2e 0 2024-05-27T09:29:12.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:12 smithi138 ceph-mon[34885]: mgrmap e22: smithi138.dxsuqf(active, since 1.03723s) 2024-05-27T09:29:12.900 INFO:tasks.workunit.client.0.smithi119.stdout:4/107: dwrite d4/d6/f1a [0,4194304] 0 2024-05-27T09:29:12.904 INFO:tasks.workunit.client.0.smithi119.stdout:4/108: rename d4/de/c14 to d4/c20 0 2024-05-27T09:29:12.904 INFO:tasks.workunit.client.0.smithi119.stdout:4/109: fdatasync f3 0 2024-05-27T09:29:12.908 INFO:tasks.workunit.client.0.smithi119.stdout:4/110: rename d4/d6/d1d to d4/de/d21 0 2024-05-27T09:29:12.918 INFO:tasks.workunit.client.0.smithi119.stdout:3/190: dwrite db/d10/f3a [0,4194304] 0 2024-05-27T09:29:12.918 INFO:tasks.workunit.client.0.smithi119.stdout:3/191: fdatasync db/d10/d13/f3d 0 2024-05-27T09:29:12.921 INFO:tasks.workunit.client.0.smithi119.stdout:3/192: mknod db/d10/c44 0 2024-05-27T09:29:12.922 INFO:tasks.workunit.client.0.smithi119.stdout:3/193: write db/f24 [359932,118692] 0 2024-05-27T09:29:12.925 INFO:tasks.workunit.client.0.smithi119.stdout:6/93: dwrite d0/d11/f15 [0,4194304] 0 2024-05-27T09:29:12.926 INFO:tasks.workunit.client.0.smithi119.stdout:4/111: dread f1 [0,4194304] 0 2024-05-27T09:29:12.926 INFO:tasks.workunit.client.0.smithi119.stdout:6/94: write d0/f18 [997093,26730] 0 2024-05-27T09:29:12.935 INFO:tasks.workunit.client.0.smithi119.stdout:1/152: dwrite f2 [0,4194304] 0 2024-05-27T09:29:12.938 INFO:tasks.workunit.client.0.smithi119.stdout:4/112: creat d4/f22 x:0 0 0 2024-05-27T09:29:12.943 INFO:tasks.workunit.client.0.smithi119.stdout:1/153: mkdir d7/db/d12/d26/d32 0 2024-05-27T09:29:12.943 INFO:tasks.workunit.client.0.smithi119.stdout:1/154: dread - d7/db/d12/d26/f2f zero size 2024-05-27T09:29:12.943 INFO:tasks.workunit.client.0.smithi119.stdout:1/155: write d7/db/f16 [331646,114667] 0 2024-05-27T09:29:12.943 INFO:tasks.workunit.client.0.smithi119.stdout:1/156: dread - d7/db/d12/d26/f2f zero size 2024-05-27T09:29:12.946 INFO:tasks.workunit.client.0.smithi119.stdout:1/157: rename d7/dd/le to d7/dd/d25/d29/l33 0 2024-05-27T09:29:12.946 INFO:tasks.workunit.client.0.smithi119.stdout:1/158: fsync f3 0 2024-05-27T09:29:12.949 INFO:tasks.workunit.client.0.smithi119.stdout:1/159: rename d7/db/d12/d26/f2f to d7/db/d12/d15/d1f/f34 0 2024-05-27T09:29:12.951 INFO:tasks.workunit.client.0.smithi119.stdout:1/160: mkdir d7/db/d35 0 2024-05-27T09:29:12.951 INFO:tasks.workunit.client.0.smithi119.stdout:1/161: chown d7/db/c2b 12858796 1 2024-05-27T09:29:12.953 INFO:tasks.workunit.client.0.smithi119.stdout:1/162: mkdir d7/dd/d36 0 2024-05-27T09:29:12.953 INFO:tasks.workunit.client.0.smithi119.stdout:1/163: mknod d7/db/d35/c37 0 2024-05-27T09:29:12.997 INFO:tasks.workunit.client.0.smithi119.stdout:3/194: dwrite db/d10/f17 [0,4194304] 0 2024-05-27T09:29:12.998 INFO:tasks.workunit.client.0.smithi119.stdout:3/195: stat db/d10/d13/f2e 0 2024-05-27T09:29:12.998 INFO:tasks.workunit.client.0.smithi119.stdout:1/164: dwrite d7/dd/d25/f30 [0,4194304] 0 2024-05-27T09:29:13.004 INFO:tasks.workunit.client.0.smithi119.stdout:3/196: mkdir db/d36/d45 0 2024-05-27T09:29:13.008 INFO:tasks.workunit.client.0.smithi119.stdout:1/165: rename d7/db/d12/d26/d32 to d7/dd/d25/d29/d38 0 2024-05-27T09:29:13.010 INFO:tasks.workunit.client.0.smithi119.stdout:3/197: truncate db/fc 134472 0 2024-05-27T09:29:13.012 INFO:tasks.workunit.client.0.smithi119.stdout:1/166: mkdir d7/db/d12/d26/d39 0 2024-05-27T09:29:13.012 INFO:tasks.workunit.client.0.smithi119.stdout:1/167: creat d7/db/d12/d15/d1f/f3a x:0 0 0 2024-05-27T09:29:13.013 INFO:tasks.workunit.client.0.smithi119.stdout:3/198: chown db/l18 51 1 2024-05-27T09:29:13.018 INFO:tasks.workunit.client.0.smithi119.stdout:1/168: write d7/f11 [837757,54678] 0 2024-05-27T09:29:13.019 INFO:tasks.workunit.client.0.smithi119.stdout:1/169: getdents d7/db/d35 0 2024-05-27T09:29:13.021 INFO:tasks.workunit.client.0.smithi119.stdout:1/170: symlink d7/db/d12/d26/l3b 0 2024-05-27T09:29:13.024 INFO:tasks.workunit.client.0.smithi119.stdout:1/171: mkdir d7/dd/d25/d29/d38/d3c 0 2024-05-27T09:29:13.024 INFO:tasks.workunit.client.0.smithi119.stdout:1/172: chown d7/dd/f13 2863971 1 2024-05-27T09:29:13.085 INFO:tasks.workunit.client.0.smithi119.stdout:3/199: dwrite db/f2f [0,4194304] 0 2024-05-27T09:29:13.085 INFO:tasks.workunit.client.0.smithi119.stdout:3/200: creat db/d10/f46 x:0 0 0 2024-05-27T09:29:13.091 INFO:tasks.workunit.client.0.smithi119.stdout:3/201: creat db/d10/d13/d33/d43/f47 x:0 0 0 2024-05-27T09:29:13.091 INFO:tasks.workunit.client.0.smithi119.stdout:3/202: write db/d10/d13/f2a [276725,119297] 0 2024-05-27T09:29:13.091 INFO:tasks.workunit.client.0.smithi119.stdout:3/203: write db/d10/f38 [1441893,41151] 0 2024-05-27T09:29:13.093 INFO:tasks.workunit.client.0.smithi119.stdout:3/204: mknod db/d10/d13/d39/c48 0 2024-05-27T09:29:13.096 INFO:tasks.workunit.client.0.smithi119.stdout:3/205: mkdir db/d36/d45/d49 0 2024-05-27T09:29:13.098 INFO:tasks.workunit.client.0.smithi119.stdout:3/206: symlink db/d36/l4a 0 2024-05-27T09:29:13.098 INFO:tasks.workunit.client.0.smithi119.stdout:3/207: write db/fc [364294,94330] 0 2024-05-27T09:29:13.099 INFO:tasks.workunit.client.0.smithi119.stdout:1/173: dwrite d7/f11 [0,4194304] 0 2024-05-27T09:29:13.099 INFO:tasks.workunit.client.0.smithi119.stdout:1/174: fsync f2 0 2024-05-27T09:29:13.099 INFO:tasks.workunit.client.0.smithi119.stdout:1/175: fsync d7/db/d12/d15/d1f/f3a 0 2024-05-27T09:29:13.099 INFO:tasks.workunit.client.0.smithi119.stdout:1/176: creat d7/f3d x:0 0 0 2024-05-27T09:29:13.108 INFO:tasks.workunit.client.0.smithi119.stdout:3/208: symlink db/l4b 0 2024-05-27T09:29:13.108 INFO:tasks.workunit.client.0.smithi119.stdout:3/209: readlink db/d10/d13/d33/l34 0 2024-05-27T09:29:13.108 INFO:tasks.workunit.client.0.smithi119.stdout:3/210: dread - db/d10/d13/d39/f3e zero size 2024-05-27T09:29:13.111 INFO:tasks.workunit.client.0.smithi119.stdout:3/211: mkdir db/d36/d45/d4c 0 2024-05-27T09:29:13.114 INFO:tasks.workunit.client.0.smithi119.stdout:3/212: getdents db/d36 0 2024-05-27T09:29:13.114 INFO:tasks.workunit.client.0.smithi119.stdout:3/213: read db/d10/f17 [6245228,118715] 0 2024-05-27T09:29:13.179 INFO:tasks.workunit.client.0.smithi119.stdout:1/177: dwrite d7/dd/d25/f30 [0,4194304] 0 2024-05-27T09:29:13.180 INFO:tasks.workunit.client.0.smithi119.stdout:3/214: dwrite db/d10/d13/f31 [4194304,4194304] 0 2024-05-27T09:29:13.181 INFO:tasks.workunit.client.0.smithi119.stdout:3/215: readlink db/l1c 0 2024-05-27T09:29:13.200 INFO:tasks.workunit.client.0.smithi119.stdout:1/178: dread d7/f11 [0,4194304] 0 2024-05-27T09:29:13.200 INFO:tasks.workunit.client.0.smithi119.stdout:1/179: chown d7/dd/d25/f30 1338997 1 2024-05-27T09:29:13.206 INFO:tasks.workunit.client.0.smithi119.stdout:3/216: dread f6 [0,4194304] 0 2024-05-27T09:29:13.206 INFO:tasks.workunit.client.0.smithi119.stdout:3/217: fsync db/d10/f17 0 2024-05-27T09:29:13.206 INFO:tasks.workunit.client.0.smithi119.stdout:3/218: fsync db/d10/d13/f2c 0 2024-05-27T09:29:13.206 INFO:tasks.workunit.client.0.smithi119.stdout:3/219: fsync db/d10/d13/f28 0 2024-05-27T09:29:13.206 INFO:tasks.workunit.client.0.smithi119.stdout:3/220: dread - db/d10/d13/f3d zero size 2024-05-27T09:29:13.212 INFO:tasks.workunit.client.0.smithi119.stdout:1/180: dread f3 [0,4194304] 0 2024-05-27T09:29:13.214 INFO:tasks.workunit.client.0.smithi119.stdout:3/221: unlink db/l2b 0 2024-05-27T09:29:13.229 INFO:tasks.workunit.client.0.smithi119.stdout:1/181: dread d7/f8 [0,4194304] 0 2024-05-27T09:29:13.235 INFO:tasks.workunit.client.0.smithi119.stdout:1/182: unlink d7/db/c1a 0 2024-05-27T09:29:13.235 INFO:tasks.workunit.client.0.smithi119.stdout:1/183: fsync d7/db/d12/d15/f1d 0 2024-05-27T09:29:13.235 INFO:tasks.workunit.client.0.smithi119.stdout:1/184: write d7/f14 [132708,89482] 0 2024-05-27T09:29:13.259 INFO:tasks.workunit.client.0.smithi119.stdout:1/185: read d7/dd/d25/f30 [2983871,48097] 0 2024-05-27T09:29:13.270 INFO:tasks.workunit.client.0.smithi119.stdout:1/186: dread d7/f11 [0,4194304] 0 2024-05-27T09:29:13.270 INFO:tasks.workunit.client.0.smithi119.stdout:1/187: stat d7/db/d12/d26/l3b 0 2024-05-27T09:29:13.292 INFO:tasks.workunit.client.0.smithi119.stdout:3/222: dwrite db/d10/d13/d33/d43/f47 [0,4194304] 0 2024-05-27T09:29:13.295 INFO:tasks.workunit.client.0.smithi119.stdout:5/126: sync 2024-05-27T09:29:13.295 INFO:tasks.workunit.client.0.smithi119.stdout:2/97: sync 2024-05-27T09:29:13.295 INFO:tasks.workunit.client.0.smithi119.stdout:5/127: truncate d6/db/de/d27/f2a 1047502 0 2024-05-27T09:29:13.295 INFO:tasks.workunit.client.0.smithi119.stdout:5/128: dread - d6/f22 zero size 2024-05-27T09:29:13.296 INFO:tasks.workunit.client.0.smithi119.stdout:3/223: mkdir db/d36/d45/d4d 0 2024-05-27T09:29:13.296 INFO:tasks.workunit.client.0.smithi119.stdout:3/224: write db/d10/d13/d33/f40 [578629,2810] 0 2024-05-27T09:29:13.296 INFO:tasks.workunit.client.0.smithi119.stdout:3/225: fdatasync db/d10/f3a 0 2024-05-27T09:29:13.298 INFO:tasks.workunit.client.0.smithi119.stdout:2/98: symlink d2/l24 0 2024-05-27T09:29:13.298 INFO:tasks.workunit.client.0.smithi119.stdout:2/99: dread - d2/da/f13 zero size 2024-05-27T09:29:13.299 INFO:tasks.workunit.client.0.smithi119.stdout:2/100: write d2/da/f13 [270318,35681] 0 2024-05-27T09:29:13.303 INFO:tasks.workunit.client.0.smithi119.stdout:5/129: mknod d6/db/de/c2d 0 2024-05-27T09:29:13.305 INFO:tasks.workunit.client.0.smithi119.stdout:2/101: symlink d2/d1a/l25 0 2024-05-27T09:29:13.306 INFO:tasks.workunit.client.0.smithi119.stdout:2/102: read d2/da/d15/f1b [194406,130612] 0 2024-05-27T09:29:13.310 INFO:tasks.workunit.client.0.smithi119.stdout:2/103: creat d2/f26 x:0 0 0 2024-05-27T09:29:13.310 INFO:tasks.workunit.client.0.smithi119.stdout:2/104: chown d2/d1a/f1e 59681 1 2024-05-27T09:29:13.319 INFO:tasks.workunit.client.0.smithi119.stdout:2/105: creat d2/d1a/f27 x:0 0 0 2024-05-27T09:29:13.341 INFO:tasks.workunit.client.0.smithi119.stdout:1/188: dwrite d7/db/d12/d15/f1d [0,4194304] 0 2024-05-27T09:29:13.341 INFO:tasks.workunit.client.0.smithi119.stdout:1/189: chown d7/db/d12/d15/d1f/f31 182387653 1 2024-05-27T09:29:13.341 INFO:tasks.workunit.client.0.smithi119.stdout:2/106: dread d2/f6 [0,4194304] 0 2024-05-27T09:29:13.342 INFO:tasks.workunit.client.0.smithi119.stdout:2/107: write d2/da/d15/f16 [380568,39342] 0 2024-05-27T09:29:13.342 INFO:tasks.workunit.client.0.smithi119.stdout:2/108: write d2/da/ff [1091642,79268] 0 2024-05-27T09:29:13.347 INFO:tasks.workunit.client.0.smithi119.stdout:3/226: dwrite db/d10/d13/f2a [0,4194304] 0 2024-05-27T09:29:13.350 INFO:tasks.workunit.client.0.smithi119.stdout:2/109: creat d2/da/f28 x:0 0 0 2024-05-27T09:29:13.350 INFO:tasks.workunit.client.0.smithi119.stdout:2/110: chown d2/da/f1c 392512339 1 2024-05-27T09:29:13.359 INFO:tasks.workunit.client.0.smithi119.stdout:2/111: rmdir d2/da/d15 39 2024-05-27T09:29:13.360 INFO:tasks.workunit.client.0.smithi119.stdout:2/112: fsync d2/da/fd 0 2024-05-27T09:29:13.362 INFO:tasks.workunit.client.0.smithi119.stdout:2/113: creat d2/da/d15/f29 x:0 0 0 2024-05-27T09:29:13.362 INFO:tasks.workunit.client.0.smithi119.stdout:5/130: dwrite d6/db/de/f25 [4194304,4194304] 0 2024-05-27T09:29:13.363 INFO:tasks.workunit.client.0.smithi119.stdout:5/131: write f1 [5727678,52519] 0 2024-05-27T09:29:13.367 INFO:tasks.workunit.client.0.smithi119.stdout:2/114: rename d2/da/d15/c1d to d2/c2a 0 2024-05-27T09:29:13.368 INFO:tasks.workunit.client.0.smithi119.stdout:2/115: dread - d2/d1a/f21 zero size 2024-05-27T09:29:13.371 INFO:tasks.workunit.client.0.smithi119.stdout:5/132: symlink d6/db/de/d27/l2e 0 2024-05-27T09:29:13.378 INFO:tasks.workunit.client.0.smithi119.stdout:5/133: getdents d6/dd/df/d13 0 2024-05-27T09:29:13.380 INFO:tasks.workunit.client.0.smithi119.stdout:2/116: dread d2/d1a/f1f [0,4194304] 0 2024-05-27T09:29:13.380 INFO:tasks.workunit.client.0.smithi119.stdout:2/117: creat d2/da/d15/f2b x:0 0 0 2024-05-27T09:29:13.380 INFO:tasks.workunit.client.0.smithi119.stdout:2/118: write d2/d1a/f21 [676594,13131] 0 2024-05-27T09:29:13.392 INFO:tasks.workunit.client.0.smithi119.stdout:1/190: dwrite d7/f27 [0,4194304] 0 2024-05-27T09:29:13.412 INFO:tasks.workunit.client.0.smithi119.stdout:1/191: write d7/dd/f13 [1017034,34709] 0 2024-05-27T09:29:13.414 INFO:tasks.workunit.client.0.smithi119.stdout:3/227: dwrite db/d10/d13/f2e [0,4194304] 0 2024-05-27T09:29:13.421 INFO:tasks.workunit.client.0.smithi119.stdout:5/134: write d6/db/de/f25 [4685704,66258] 0 2024-05-27T09:29:13.433 INFO:tasks.workunit.client.0.smithi119.stdout:1/192: read d7/f8 [1878530,122916] 0 2024-05-27T09:29:13.441 INFO:tasks.workunit.client.0.smithi119.stdout:2/119: dwrite d2/da/d15/f1b [0,4194304] 0 2024-05-27T09:29:13.443 INFO:tasks.workunit.client.0.smithi119.stdout:2/120: mkdir d2/da/d2c 0 2024-05-27T09:29:13.444 INFO:tasks.workunit.client.0.smithi119.stdout:2/121: fdatasync d2/da/f28 0 2024-05-27T09:29:13.444 INFO:tasks.workunit.client.0.smithi119.stdout:2/122: creat d2/f2d x:0 0 0 2024-05-27T09:29:13.444 INFO:tasks.workunit.client.0.smithi119.stdout:2/123: creat d2/da/f2e x:0 0 0 2024-05-27T09:29:13.445 INFO:tasks.workunit.client.0.smithi119.stdout:2/124: chown l0 267587552 1 2024-05-27T09:29:13.446 INFO:tasks.workunit.client.0.smithi119.stdout:3/228: write db/d10/d13/f21 [1516438,70947] 0 2024-05-27T09:29:13.446 INFO:tasks.workunit.client.0.smithi119.stdout:5/135: dread d6/db/de/d27/f2a [0,4194304] 0 2024-05-27T09:29:13.448 INFO:tasks.workunit.client.0.smithi119.stdout:7/219: sync 2024-05-27T09:29:13.448 INFO:tasks.workunit.client.0.smithi119.stdout:0/84: sync 2024-05-27T09:29:13.448 INFO:tasks.workunit.client.0.smithi119.stdout:5/136: mkdir d6/d2f 0 2024-05-27T09:29:13.450 INFO:tasks.workunit.client.0.smithi119.stdout:0/85: rename d2/d6/db/c11 to d2/d6/db/c18 0 2024-05-27T09:29:13.451 INFO:tasks.workunit.client.0.smithi119.stdout:5/137: mknod d6/dd/df/d13/c30 0 2024-05-27T09:29:13.451 INFO:tasks.workunit.client.0.smithi119.stdout:5/138: fdatasync d6/db/dc/f26 0 2024-05-27T09:29:13.451 INFO:tasks.workunit.client.0.smithi119.stdout:5/139: chown d6/f1e 108 1 2024-05-27T09:29:13.453 INFO:tasks.workunit.client.0.smithi119.stdout:0/86: mknod d2/d4/c19 0 2024-05-27T09:29:13.453 INFO:tasks.workunit.client.0.smithi119.stdout:5/140: creat d6/db/f31 x:0 0 0 2024-05-27T09:29:13.480 INFO:tasks.workunit.client.0.smithi119.stdout:1/193: dread f3 [0,4194304] 0 2024-05-27T09:29:13.480 INFO:tasks.workunit.client.0.smithi119.stdout:7/220: write f1 [7238905,59144] 0 2024-05-27T09:29:13.484 INFO:tasks.workunit.client.0.smithi119.stdout:1/194: link d7/db/d12/f23 d7/db/d35/f3e 0 2024-05-27T09:29:13.484 INFO:tasks.workunit.client.0.smithi119.stdout:1/195: creat d7/db/d12/d15/f3f x:0 0 0 2024-05-27T09:29:13.490 INFO:tasks.workunit.client.0.smithi119.stdout:8/135: sync 2024-05-27T09:29:13.490 INFO:tasks.workunit.client.0.smithi119.stdout:6/95: sync 2024-05-27T09:29:13.490 INFO:tasks.workunit.client.0.smithi119.stdout:9/111: sync 2024-05-27T09:29:13.490 INFO:tasks.workunit.client.0.smithi119.stdout:9/112: chown fe 865 1 2024-05-27T09:29:13.490 INFO:tasks.workunit.client.0.smithi119.stdout:4/113: sync 2024-05-27T09:29:13.497 INFO:tasks.workunit.client.0.smithi119.stdout:3/229: write db/d10/d13/f31 [7943278,73728] 0 2024-05-27T09:29:13.504 INFO:tasks.workunit.client.0.smithi119.stdout:6/96: creat d0/d2/de/f1f x:0 0 0 2024-05-27T09:29:13.506 INFO:tasks.workunit.client.0.smithi119.stdout:4/114: mkdir d4/d23 0 2024-05-27T09:29:13.506 INFO:tasks.workunit.client.0.smithi119.stdout:9/113: mknod d18/d19/d1b/c25 0 2024-05-27T09:29:13.506 INFO:tasks.workunit.client.0.smithi119.stdout:6/97: creat d0/d2/f20 x:0 0 0 2024-05-27T09:29:13.506 INFO:tasks.workunit.client.0.smithi119.stdout:8/136: link d3/f12 d3/d24/f2f 0 2024-05-27T09:29:13.506 INFO:tasks.workunit.client.0.smithi119.stdout:5/141: getdents d6/db 0 2024-05-27T09:29:13.506 INFO:tasks.workunit.client.0.smithi119.stdout:5/142: fdatasync d6/f22 0 2024-05-27T09:29:13.506 INFO:tasks.workunit.client.0.smithi119.stdout:3/230: symlink db/d36/l4e 0 2024-05-27T09:29:13.506 INFO:tasks.workunit.client.0.smithi119.stdout:3/231: write db/d10/f17 [5958949,15606] 0 2024-05-27T09:29:13.506 INFO:tasks.workunit.client.0.smithi119.stdout:3/232: write db/f37 [109255,62440] 0 2024-05-27T09:29:13.515 INFO:tasks.workunit.client.0.smithi119.stdout:9/114: unlink f17 0 2024-05-27T09:29:13.515 INFO:tasks.workunit.client.0.smithi119.stdout:9/115: chown d18/d19/d1b/f1f 9893199 1 2024-05-27T09:29:13.515 INFO:tasks.workunit.client.0.smithi119.stdout:9/116: creat d18/f26 x:0 0 0 2024-05-27T09:29:13.515 INFO:tasks.workunit.client.0.smithi119.stdout:9/117: fsync f6 0 2024-05-27T09:29:13.520 INFO:tasks.workunit.client.0.smithi119.stdout:8/137: mkdir d3/d24/d30 0 2024-05-27T09:29:13.520 INFO:tasks.workunit.client.0.smithi119.stdout:8/138: dread - d3/de/f21 zero size 2024-05-27T09:29:13.520 INFO:tasks.workunit.client.0.smithi119.stdout:8/139: write d3/d8/d1b/f20 [1002098,62907] 0 2024-05-27T09:29:13.520 INFO:tasks.workunit.client.0.smithi119.stdout:7/221: dread d7/df/f13 [4194304,4194304] 0 2024-05-27T09:29:13.521 INFO:tasks.workunit.client.0.smithi119.stdout:2/125: dwrite d2/da/f28 [0,4194304] 0 2024-05-27T09:29:13.521 INFO:tasks.workunit.client.0.smithi119.stdout:2/126: chown d2/d1a/f1e 2 1 2024-05-27T09:29:13.521 INFO:tasks.workunit.client.0.smithi119.stdout:7/222: rename d7/df/d19/d1d/f1e to d7/df/d19/d4d/f4e 0 2024-05-27T09:29:13.521 INFO:tasks.workunit.client.0.smithi119.stdout:4/115: mkdir d4/d23/d24 0 2024-05-27T09:29:13.529 INFO:tasks.workunit.client.0.smithi119.stdout:3/233: dread db/fc [0,4194304] 0 2024-05-27T09:29:13.533 INFO:tasks.workunit.client.0.smithi119.stdout:2/127: symlink d2/d1a/l2f 0 2024-05-27T09:29:13.533 INFO:tasks.workunit.client.0.smithi119.stdout:2/128: truncate d2/d1a/f1e 948163 0 2024-05-27T09:29:13.533 INFO:tasks.workunit.client.0.smithi119.stdout:2/129: fsync d2/f6 0 2024-05-27T09:29:13.533 INFO:tasks.workunit.client.0.smithi119.stdout:4/116: mknod d4/de/d21/c25 0 2024-05-27T09:29:13.533 INFO:tasks.workunit.client.0.smithi119.stdout:9/118: dread d18/d19/d1b/f1e [0,4194304] 0 2024-05-27T09:29:13.537 INFO:tasks.workunit.client.0.smithi119.stdout:4/117: rename d4/d6/f13 to d4/d23/f26 0 2024-05-27T09:29:13.537 INFO:tasks.workunit.client.0.smithi119.stdout:4/118: rename d4 to d4/de/d21/d27 22 2024-05-27T09:29:13.537 INFO:tasks.workunit.client.0.smithi119.stdout:4/119: dread - d4/f22 zero size 2024-05-27T09:29:13.538 INFO:tasks.workunit.client.0.smithi119.stdout:9/119: rename d18/d19 to d18/d19/d1b/d27 22 2024-05-27T09:29:13.538 INFO:tasks.workunit.client.0.smithi119.stdout:3/234: getdents db/d10/d13/d39 0 2024-05-27T09:29:13.538 INFO:tasks.workunit.client.0.smithi119.stdout:6/98: write d0/fb [3969652,62335] 0 2024-05-27T09:29:13.538 INFO:tasks.workunit.client.0.smithi119.stdout:6/99: stat d0/f6 0 2024-05-27T09:29:13.540 INFO:tasks.workunit.client.0.smithi119.stdout:4/120: mkdir d4/d6/d28 0 2024-05-27T09:29:13.541 INFO:tasks.workunit.client.0.smithi119.stdout:9/120: unlink d18/c20 0 2024-05-27T09:29:13.541 INFO:tasks.workunit.client.0.smithi119.stdout:9/121: truncate f15 375576 0 2024-05-27T09:29:13.541 INFO:tasks.workunit.client.0.smithi119.stdout:4/121: write d4/d1c/f1e [2268193,119017] 0 2024-05-27T09:29:13.541 INFO:tasks.workunit.client.0.smithi119.stdout:2/130: fdatasync d2/da/ff 0 2024-05-27T09:29:13.541 INFO:tasks.workunit.client.0.smithi119.stdout:2/131: fsync d2/da/f10 0 2024-05-27T09:29:13.543 INFO:tasks.workunit.client.0.smithi119.stdout:0/87: dwrite d2/d6/d9/d12/f16 [0,4194304] 0 2024-05-27T09:29:13.543 INFO:tasks.workunit.client.0.smithi119.stdout:0/88: chown d2/d6/db 811 1 2024-05-27T09:29:13.547 INFO:tasks.workunit.client.0.smithi119.stdout:9/122: mknod d18/d19/c28 0 2024-05-27T09:29:13.551 INFO:tasks.workunit.client.0.smithi119.stdout:4/122: unlink d4/d6/f17 0 2024-05-27T09:29:13.551 INFO:tasks.workunit.client.0.smithi119.stdout:4/123: truncate d4/f22 714232 0 2024-05-27T09:29:13.551 INFO:tasks.workunit.client.0.smithi119.stdout:4/124: stat d4/d6/c18 0 2024-05-27T09:29:13.551 INFO:tasks.workunit.client.0.smithi119.stdout:0/89: mkdir d2/d1a 0 2024-05-27T09:29:13.567 INFO:tasks.workunit.client.0.smithi119.stdout:3/235: dread db/f35 [0,4194304] 0 2024-05-27T09:29:13.567 INFO:tasks.workunit.client.0.smithi119.stdout:3/236: stat db/d10/c32 0 2024-05-27T09:29:13.568 INFO:tasks.workunit.client.0.smithi119.stdout:3/237: mkdir db/d36/d45/d49/d4f 0 2024-05-27T09:29:13.578 INFO:tasks.workunit.client.0.smithi119.stdout:1/196: dwrite d7/dd/f13 [0,4194304] 0 2024-05-27T09:29:13.601 INFO:tasks.workunit.client.0.smithi119.stdout:4/125: dread d4/f8 [0,4194304] 0 2024-05-27T09:29:13.612 INFO:tasks.workunit.client.0.smithi119.stdout:5/143: dwrite d6/db/dc/f26 [0,4194304] 0 2024-05-27T09:29:13.615 INFO:tasks.workunit.client.0.smithi119.stdout:3/238: dread db/d10/d13/f2a [0,4194304] 0 2024-05-27T09:29:13.615 INFO:tasks.workunit.client.0.smithi119.stdout:3/239: write db/f24 [459842,127047] 0 2024-05-27T09:29:13.617 INFO:tasks.workunit.client.0.smithi119.stdout:9/123: dread d18/d19/d1b/f1d [0,4194304] 0 2024-05-27T09:29:13.618 INFO:tasks.workunit.client.0.smithi119.stdout:2/132: dwrite d2/f26 [0,4194304] 0 2024-05-27T09:29:13.618 INFO:tasks.workunit.client.0.smithi119.stdout:2/133: chown d2/da/d15/f2b 810 1 2024-05-27T09:29:13.618 INFO:tasks.workunit.client.0.smithi119.stdout:2/134: write d2/da/f13 [293718,11672] 0 2024-05-27T09:29:13.619 INFO:tasks.workunit.client.0.smithi119.stdout:2/135: creat d2/d1a/f30 x:0 0 0 2024-05-27T09:29:13.619 INFO:tasks.workunit.client.0.smithi119.stdout:2/136: creat d2/da/d15/f31 x:0 0 0 2024-05-27T09:29:13.631 INFO:tasks.workunit.client.0.smithi119.stdout:5/144: rename d6/db/dc/f21 to d6/dd/f32 0 2024-05-27T09:29:13.631 INFO:tasks.workunit.client.0.smithi119.stdout:5/145: creat d6/dd/f33 x:0 0 0 2024-05-27T09:29:13.637 INFO:tasks.workunit.client.0.smithi119.stdout:2/137: rename d2/c4 to d2/da/d2c/c32 0 2024-05-27T09:29:13.637 INFO:tasks.workunit.client.0.smithi119.stdout:6/100: dread d0/f7 [0,4194304] 0 2024-05-27T09:29:13.641 INFO:tasks.workunit.client.0.smithi119.stdout:2/138: mkdir d2/da/d15/d33 0 2024-05-27T09:29:13.641 INFO:tasks.workunit.client.0.smithi119.stdout:2/139: dread - d2/da/d15/f29 zero size 2024-05-27T09:29:13.651 INFO:tasks.workunit.client.0.smithi119.stdout:3/240: dread db/d10/f17 [4194304,4194304] 0 2024-05-27T09:29:13.651 INFO:tasks.workunit.client.0.smithi119.stdout:3/241: readlink db/l1c 0 2024-05-27T09:29:13.653 INFO:tasks.workunit.client.0.smithi119.stdout:3/242: chown db/l26 2736590 1 2024-05-27T09:29:13.656 INFO:tasks.workunit.client.0.smithi119.stdout:3/243: unlink db/d10/d13/f28 0 2024-05-27T09:29:13.657 INFO:tasks.workunit.client.0.smithi119.stdout:3/244: mknod db/d36/d45/d49/d4f/c50 0 2024-05-27T09:29:13.657 INFO:tasks.workunit.client.0.smithi119.stdout:3/245: write db/d10/f46 [502478,115993] 0 2024-05-27T09:29:13.658 INFO:tasks.workunit.client.0.smithi119.stdout:2/140: dread d2/f6 [0,4194304] 0 2024-05-27T09:29:13.658 INFO:tasks.workunit.client.0.smithi119.stdout:2/141: chown d2/le 6639 1 2024-05-27T09:29:13.659 INFO:tasks.workunit.client.0.smithi119.stdout:3/246: truncate db/f23 915118 0 2024-05-27T09:29:13.660 INFO:tasks.workunit.client.0.smithi119.stdout:3/247: chown c7 85 1 2024-05-27T09:29:13.660 INFO:tasks.workunit.client.0.smithi119.stdout:3/248: readlink l9 0 2024-05-27T09:29:13.664 INFO:tasks.workunit.client.0.smithi119.stdout:3/249: creat db/d10/d13/d39/f51 x:0 0 0 2024-05-27T09:29:13.666 INFO:tasks.workunit.client.0.smithi119.stdout:7/223: dwrite d7/d1a/f4a [0,4194304] 0 2024-05-27T09:29:13.666 INFO:tasks.workunit.client.0.smithi119.stdout:0/90: dwrite d2/d6/d9/d12/f16 [0,4194304] 0 2024-05-27T09:29:13.666 INFO:tasks.workunit.client.0.smithi119.stdout:0/91: creat d2/d6/d9/d12/f1b x:0 0 0 2024-05-27T09:29:13.666 INFO:tasks.workunit.client.0.smithi119.stdout:0/92: creat d2/d6/d9/f1c x:0 0 0 2024-05-27T09:29:13.680 INFO:tasks.workunit.client.0.smithi119.stdout:0/93: link d2/l15 d2/d6/d9/d12/l1d 0 2024-05-27T09:29:13.682 INFO:tasks.workunit.client.0.smithi119.stdout:2/142: dread d2/da/f28 [0,4194304] 0 2024-05-27T09:29:13.683 INFO:tasks.workunit.client.0.smithi119.stdout:1/197: dwrite d7/f3d [0,4194304] 0 2024-05-27T09:29:13.690 INFO:tasks.workunit.client.0.smithi119.stdout:0/94: truncate d2/d6/fc 2618598 0 2024-05-27T09:29:13.692 INFO:tasks.workunit.client.0.smithi119.stdout:3/250: dread db/d10/f3a [4194304,4194304] 0 2024-05-27T09:29:13.692 INFO:tasks.workunit.client.0.smithi119.stdout:0/95: creat d2/d1a/f1e x:0 0 0 2024-05-27T09:29:13.695 INFO:tasks.workunit.client.0.smithi119.stdout:3/251: truncate db/d10/d13/f2c 3961536 0 2024-05-27T09:29:13.695 INFO:tasks.workunit.client.0.smithi119.stdout:0/96: link d2/d6/d9/f1c d2/d1a/f1f 0 2024-05-27T09:29:13.695 INFO:tasks.workunit.client.0.smithi119.stdout:0/97: chown d2/d6/f8 20876091 1 2024-05-27T09:29:13.700 INFO:tasks.workunit.client.0.smithi119.stdout:3/252: creat db/d36/d45/f52 x:0 0 0 2024-05-27T09:29:13.700 INFO:tasks.workunit.client.0.smithi119.stdout:3/253: getdents db/d36/d45/d4d 0 2024-05-27T09:29:13.700 INFO:tasks.workunit.client.0.smithi119.stdout:6/101: dwrite d0/fd [0,4194304] 0 2024-05-27T09:29:13.701 INFO:tasks.workunit.client.0.smithi119.stdout:6/102: chown d0/d11/f15 1036522 1 2024-05-27T09:29:13.701 INFO:tasks.workunit.client.0.smithi119.stdout:0/98: mkdir d2/d1a/d20 0 2024-05-27T09:29:13.702 INFO:tasks.workunit.client.0.smithi119.stdout:6/103: rename d0/d11/d12 to d0/d2/d1c/d21 0 2024-05-27T09:29:13.706 INFO:tasks.workunit.client.0.smithi119.stdout:6/104: rename d0/la to d0/d11/l22 0 2024-05-27T09:29:13.707 INFO:tasks.workunit.client.0.smithi119.stdout:6/105: mkdir d0/d19/d23 0 2024-05-27T09:29:13.708 INFO:tasks.workunit.client.0.smithi119.stdout:3/254: dread db/f23 [0,4194304] 0 2024-05-27T09:29:13.712 INFO:tasks.workunit.client.0.smithi119.stdout:4/126: read f3 [982343,35735] 0 2024-05-27T09:29:13.713 INFO:tasks.workunit.client.0.smithi119.stdout:4/127: creat d4/d1c/f29 x:0 0 0 2024-05-27T09:29:13.713 INFO:tasks.workunit.client.0.smithi119.stdout:9/124: dwrite d18/d19/d1b/f24 [0,4194304] 0 2024-05-27T09:29:13.713 INFO:tasks.workunit.client.0.smithi119.stdout:9/125: readlink l14 0 2024-05-27T09:29:13.716 INFO:tasks.workunit.client.0.smithi119.stdout:0/99: unlink d2/d6/fc 0 2024-05-27T09:29:13.716 INFO:tasks.workunit.client.0.smithi119.stdout:0/100: creat d2/d1a/f21 x:0 0 0 2024-05-27T09:29:13.718 INFO:tasks.workunit.client.0.smithi119.stdout:3/255: truncate db/d10/f38 761041 0 2024-05-27T09:29:13.725 INFO:tasks.workunit.client.0.smithi119.stdout:3/256: rename db/d10/f46 to db/d10/d13/d39/f53 0 2024-05-27T09:29:13.725 INFO:tasks.workunit.client.0.smithi119.stdout:3/257: rename db/d10 to db/d10/d13/d33/d54 22 2024-05-27T09:29:13.725 INFO:tasks.workunit.client.0.smithi119.stdout:3/258: chown db/d36/d45/f52 4 1 2024-05-27T09:29:13.725 INFO:tasks.workunit.client.0.smithi119.stdout:3/259: write db/f37 [1139557,93722] 0 2024-05-27T09:29:13.736 INFO:tasks.workunit.client.0.smithi119.stdout:3/260: mknod db/d10/c55 0 2024-05-27T09:29:13.738 INFO:tasks.workunit.client.0.smithi119.stdout:3/261: creat db/f56 x:0 0 0 2024-05-27T09:29:13.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:13 smithi119 ceph-mon[29082]: [27/May/2024:09:29:12] ENGINE Bus STARTING 2024-05-27T09:29:13.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:13 smithi119 ceph-mon[29082]: Deploying cephadm binary to smithi138 2024-05-27T09:29:13.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:13 smithi119 ceph-mon[29082]: Deploying cephadm binary to smithi119 2024-05-27T09:29:13.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:13 smithi119 ceph-mon[29082]: [27/May/2024:09:29:12] ENGINE Serving on http://172.21.15.138:8765 2024-05-27T09:29:13.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:13 smithi119 ceph-mon[29082]: mgrmap e23: smithi138.dxsuqf(active, since 2s) 2024-05-27T09:29:13.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:13 smithi119 ceph-mon[29082]: pgmap v4: 65 pgs: 65 active+clean; 473 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:29:13.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:13 smithi119 ceph-mon[29082]: [27/May/2024:09:29:12] ENGINE Serving on https://172.21.15.138:7150 2024-05-27T09:29:13.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:13 smithi119 ceph-mon[29082]: [27/May/2024:09:29:12] ENGINE Bus STARTED 2024-05-27T09:29:13.758 INFO:tasks.workunit.client.0.smithi119.stdout:7/224: dwrite d7/fa [4194304,4194304] 0 2024-05-27T09:29:13.758 INFO:tasks.workunit.client.0.smithi119.stdout:7/225: readlink d7/l27 0 2024-05-27T09:29:13.761 INFO:tasks.workunit.client.0.smithi119.stdout:7/226: creat d7/df/d19/d1d/f4f x:0 0 0 2024-05-27T09:29:13.761 INFO:tasks.workunit.client.0.smithi119.stdout:7/227: readlink d7/df/l26 0 2024-05-27T09:29:13.767 INFO:tasks.workunit.client.0.smithi119.stdout:5/146: dwrite d6/f12 [0,4194304] 0 2024-05-27T09:29:13.806 INFO:tasks.workunit.client.0.smithi119.stdout:4/128: dread d4/f22 [0,4194304] 0 2024-05-27T09:29:13.806 INFO:tasks.workunit.client.0.smithi119.stdout:4/129: getdents d4/d6/d28 0 2024-05-27T09:29:13.807 INFO:tasks.workunit.client.0.smithi119.stdout:4/130: chown d4/de/lf 793848712 1 2024-05-27T09:29:13.807 INFO:tasks.workunit.client.0.smithi119.stdout:4/131: readlink d4/de/lf 0 2024-05-27T09:29:13.807 INFO:tasks.workunit.client.0.smithi119.stdout:4/132: truncate d4/d6/f11 4905412 0 2024-05-27T09:29:13.807 INFO:tasks.workunit.client.0.smithi119.stdout:3/262: dread f8 [4194304,4194304] 0 2024-05-27T09:29:13.807 INFO:tasks.workunit.client.0.smithi119.stdout:7/228: dread d7/df/d19/d1d/f23 [0,4194304] 0 2024-05-27T09:29:13.807 INFO:tasks.workunit.client.0.smithi119.stdout:3/263: rename db/d10/c1e to db/d36/d45/d49/c57 0 2024-05-27T09:29:13.807 INFO:tasks.workunit.client.0.smithi119.stdout:4/133: write f3 [4302138,101647] 0 2024-05-27T09:29:13.807 INFO:tasks.workunit.client.0.smithi119.stdout:7/229: link d7/d1a/f1c d7/df/d19/f50 0 2024-05-27T09:29:13.807 INFO:tasks.workunit.client.0.smithi119.stdout:7/230: chown d7/df/d19/d1d/l2e 45888 1 2024-05-27T09:29:13.807 INFO:tasks.workunit.client.0.smithi119.stdout:7/231: read d7/df/d19/f50 [73890,67553] 0 2024-05-27T09:29:13.807 INFO:tasks.workunit.client.0.smithi119.stdout:7/232: truncate d7/d1a/f1c 1887670 0 2024-05-27T09:29:13.807 INFO:tasks.workunit.client.0.smithi119.stdout:7/233: chown d7/d1a/f2b 6 1 2024-05-27T09:29:13.807 INFO:tasks.workunit.client.0.smithi119.stdout:2/143: dwrite d2/d1a/f21 [0,4194304] 0 2024-05-27T09:29:13.808 INFO:tasks.workunit.client.0.smithi119.stdout:2/144: getdents d2/da/d15/d33 0 2024-05-27T09:29:13.808 INFO:tasks.workunit.client.0.smithi119.stdout:2/145: rmdir d2/da 39 2024-05-27T09:29:13.808 INFO:tasks.workunit.client.0.smithi119.stdout:0/101: dwrite d2/d6/d9/ff [4194304,4194304] 0 2024-05-27T09:29:13.808 INFO:tasks.workunit.client.0.smithi119.stdout:6/106: dread d0/d11/f15 [0,4194304] 0 2024-05-27T09:29:13.808 INFO:tasks.workunit.client.0.smithi119.stdout:0/102: mknod d2/d1a/d20/c22 0 2024-05-27T09:29:13.808 INFO:tasks.workunit.client.0.smithi119.stdout:0/103: dread - d2/d1a/f21 zero size 2024-05-27T09:29:13.808 INFO:tasks.workunit.client.0.smithi119.stdout:9/126: write d18/d19/d1b/f24 [3810973,39924] 0 2024-05-27T09:29:13.809 INFO:tasks.workunit.client.0.smithi119.stdout:3/264: dread db/f30 [0,4194304] 0 2024-05-27T09:29:13.810 INFO:tasks.workunit.client.0.smithi119.stdout:3/265: read db/f35 [484602,105577] 0 2024-05-27T09:29:13.810 INFO:tasks.workunit.client.0.smithi119.stdout:3/266: fdatasync db/d10/d13/f3d 0 2024-05-27T09:29:13.810 INFO:tasks.workunit.client.0.smithi119.stdout:2/146: dread d2/d1a/f1e [0,4194304] 0 2024-05-27T09:29:13.815 INFO:tasks.workunit.client.0.smithi119.stdout:5/147: dread d6/f1d [0,4194304] 0 2024-05-27T09:29:13.815 INFO:tasks.workunit.client.0.smithi119.stdout:1/198: dwrite d7/db/d12/d15/f3f [0,4194304] 0 2024-05-27T09:29:13.815 INFO:tasks.workunit.client.0.smithi119.stdout:9/127: mkdir d18/d19/d1b/d29 0 2024-05-27T09:29:13.815 INFO:tasks.workunit.client.0.smithi119.stdout:2/147: symlink d2/da/d15/d33/l34 0 2024-05-27T09:29:13.816 INFO:tasks.workunit.client.0.smithi119.stdout:2/148: rename d2/da to d2/da/d15/d35 22 2024-05-27T09:29:13.816 INFO:tasks.workunit.client.0.smithi119.stdout:2/149: dread - d2/da/d15/f31 zero size 2024-05-27T09:29:13.816 INFO:tasks.workunit.client.0.smithi119.stdout:2/150: truncate d2/da/f1c 576951 0 2024-05-27T09:29:13.816 INFO:tasks.workunit.client.0.smithi119.stdout:2/151: chown d2/le 289400 1 2024-05-27T09:29:13.816 INFO:tasks.workunit.client.0.smithi119.stdout:3/267: creat db/d36/d45/d4d/f58 x:0 0 0 2024-05-27T09:29:13.816 INFO:tasks.workunit.client.0.smithi119.stdout:3/268: write db/d10/d13/d39/f3e [271255,54183] 0 2024-05-27T09:29:13.824 INFO:tasks.workunit.client.0.smithi119.stdout:9/128: mkdir d18/d19/d1b/d2a 0 2024-05-27T09:29:13.824 INFO:tasks.workunit.client.0.smithi119.stdout:5/148: symlink d6/db/de/l34 0 2024-05-27T09:29:13.824 INFO:tasks.workunit.client.0.smithi119.stdout:9/129: readlink l14 0 2024-05-27T09:29:13.825 INFO:tasks.workunit.client.0.smithi119.stdout:8/140: sync 2024-05-27T09:29:13.826 INFO:tasks.workunit.client.0.smithi119.stdout:2/152: rename d2/d1a/f1e to d2/d1a/f36 0 2024-05-27T09:29:13.833 INFO:tasks.workunit.client.0.smithi119.stdout:8/141: mknod d3/c31 0 2024-05-27T09:29:13.833 INFO:tasks.workunit.client.0.smithi119.stdout:9/130: symlink d18/d19/d1b/l2b 0 2024-05-27T09:29:13.833 INFO:tasks.workunit.client.0.smithi119.stdout:5/149: truncate d6/f9 1263654 0 2024-05-27T09:29:13.838 INFO:tasks.workunit.client.0.smithi119.stdout:2/153: dread d2/da/ff [0,4194304] 0 2024-05-27T09:29:13.838 INFO:tasks.workunit.client.0.smithi119.stdout:2/154: read - d2/da/d15/f2b zero size 2024-05-27T09:29:13.838 INFO:tasks.workunit.client.0.smithi119.stdout:8/142: rename d3/de/ff to d3/d24/d30/f32 0 2024-05-27T09:29:13.838 INFO:tasks.workunit.client.0.smithi119.stdout:8/143: read d3/d8/fd [2463832,35361] 0 2024-05-27T09:29:13.839 INFO:tasks.workunit.client.0.smithi119.stdout:2/155: creat d2/da/f37 x:0 0 0 2024-05-27T09:29:13.841 INFO:tasks.workunit.client.0.smithi119.stdout:0/104: write d2/d6/d9/ff [6066040,16775] 0 2024-05-27T09:29:13.843 INFO:tasks.workunit.client.0.smithi119.stdout:2/156: rename d2/da/d15/f31 to d2/da/d2c/f38 0 2024-05-27T09:29:13.843 INFO:tasks.workunit.client.0.smithi119.stdout:2/157: creat d2/d1a/f39 x:0 0 0 2024-05-27T09:29:13.847 INFO:tasks.workunit.client.0.smithi119.stdout:0/105: rename d2/d6/db to d2/d6/d23 0 2024-05-27T09:29:13.851 INFO:tasks.workunit.client.0.smithi119.stdout:0/106: write d2/d6/d9/f1c [324552,129648] 0 2024-05-27T09:29:13.851 INFO:tasks.workunit.client.0.smithi119.stdout:2/158: truncate d2/da/d15/f18 3362012 0 2024-05-27T09:29:13.861 INFO:tasks.workunit.client.0.smithi119.stdout:2/159: rmdir d2/da 39 2024-05-27T09:29:13.868 INFO:tasks.workunit.client.0.smithi119.stdout:2/160: mknod d2/d1a/c3a 0 2024-05-27T09:29:13.868 INFO:tasks.workunit.client.0.smithi119.stdout:2/161: creat d2/d1a/f3b x:0 0 0 2024-05-27T09:29:13.868 INFO:tasks.workunit.client.0.smithi119.stdout:2/162: link d2/d1a/f30 d2/da/d2c/f3c 0 2024-05-27T09:29:13.868 INFO:tasks.workunit.client.0.smithi119.stdout:2/163: rmdir d2/da/d2c 39 2024-05-27T09:29:13.868 INFO:tasks.workunit.client.0.smithi119.stdout:2/164: creat d2/d1a/f3d x:0 0 0 2024-05-27T09:29:13.870 INFO:tasks.workunit.client.0.smithi119.stdout:0/107: dread d2/d6/d9/d12/f16 [0,4194304] 0 2024-05-27T09:29:13.870 INFO:tasks.workunit.client.0.smithi119.stdout:0/108: chown d2/d6/d9/d12/f1b 2747479 1 2024-05-27T09:29:13.875 INFO:tasks.workunit.client.0.smithi119.stdout:2/165: mknod d2/d1a/c3e 0 2024-05-27T09:29:13.875 INFO:tasks.workunit.client.0.smithi119.stdout:2/166: chown d2/d1a/f1f 688972126 1 2024-05-27T09:29:13.875 INFO:tasks.workunit.client.0.smithi119.stdout:4/134: write d4/d6/f1a [1358184,96930] 0 2024-05-27T09:29:13.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:13 smithi138 ceph-mon[34885]: [27/May/2024:09:29:12] ENGINE Bus STARTING 2024-05-27T09:29:13.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:13 smithi138 ceph-mon[34885]: Deploying cephadm binary to smithi138 2024-05-27T09:29:13.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:13 smithi138 ceph-mon[34885]: Deploying cephadm binary to smithi119 2024-05-27T09:29:13.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:13 smithi138 ceph-mon[34885]: [27/May/2024:09:29:12] ENGINE Serving on http://172.21.15.138:8765 2024-05-27T09:29:13.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:13 smithi138 ceph-mon[34885]: mgrmap e23: smithi138.dxsuqf(active, since 2s) 2024-05-27T09:29:13.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:13 smithi138 ceph-mon[34885]: pgmap v4: 65 pgs: 65 active+clean; 473 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:29:13.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:13 smithi138 ceph-mon[34885]: [27/May/2024:09:29:12] ENGINE Serving on https://172.21.15.138:7150 2024-05-27T09:29:13.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:13 smithi138 ceph-mon[34885]: [27/May/2024:09:29:12] ENGINE Bus STARTED 2024-05-27T09:29:13.891 INFO:tasks.workunit.client.0.smithi119.stdout:0/109: truncate d2/d6/d9/ff 4563882 0 2024-05-27T09:29:13.891 INFO:tasks.workunit.client.0.smithi119.stdout:2/167: mknod d2/da/c3f 0 2024-05-27T09:29:13.891 INFO:tasks.workunit.client.0.smithi119.stdout:2/168: write d2/f2d [311684,65294] 0 2024-05-27T09:29:13.895 INFO:tasks.workunit.client.0.smithi119.stdout:0/110: mkdir d2/d6/d9/d24 0 2024-05-27T09:29:13.895 INFO:tasks.workunit.client.0.smithi119.stdout:0/111: fdatasync d2/d6/d9/ff 0 2024-05-27T09:29:13.895 INFO:tasks.workunit.client.0.smithi119.stdout:0/112: creat d2/d6/d9/d12/f25 x:0 0 0 2024-05-27T09:29:13.897 INFO:tasks.workunit.client.0.smithi119.stdout:1/199: dwrite d7/dd/d25/f30 [0,4194304] 0 2024-05-27T09:29:13.899 INFO:tasks.workunit.client.0.smithi119.stdout:1/200: mkdir d7/d40 0 2024-05-27T09:29:13.904 INFO:tasks.workunit.client.0.smithi119.stdout:0/113: dread d2/d6/d9/ff [4194304,4194304] 0 2024-05-27T09:29:13.905 INFO:tasks.workunit.client.0.smithi119.stdout:0/114: readlink d2/d6/d9/ld 0 2024-05-27T09:29:13.905 INFO:tasks.workunit.client.0.smithi119.stdout:8/144: dwrite d3/d8/d1b/f20 [0,4194304] 0 2024-05-27T09:29:13.906 INFO:tasks.workunit.client.0.smithi119.stdout:8/145: write d3/de/f21 [39561,72994] 0 2024-05-27T09:29:13.924 INFO:tasks.workunit.client.0.smithi119.stdout:1/201: write d7/f27 [1118625,104322] 0 2024-05-27T09:29:13.926 INFO:tasks.workunit.client.0.smithi119.stdout:8/146: write d3/de/f1c [4864288,30645] 0 2024-05-27T09:29:13.927 INFO:tasks.workunit.client.0.smithi119.stdout:8/147: fdatasync d3/de/f10 0 2024-05-27T09:29:13.945 INFO:tasks.workunit.client.0.smithi119.stdout:8/148: dread d3/d8/fd [0,4194304] 0 2024-05-27T09:29:13.969 INFO:tasks.workunit.client.0.smithi119.stdout:6/107: dwrite d0/d2/f10 [4194304,4194304] 0 2024-05-27T09:29:13.969 INFO:tasks.workunit.client.0.smithi119.stdout:6/108: stat d0/cf 0 2024-05-27T09:29:13.970 INFO:tasks.workunit.client.0.smithi119.stdout:3/269: dwrite db/d10/d13/d33/f40 [0,4194304] 0 2024-05-27T09:29:13.970 INFO:tasks.workunit.client.0.smithi119.stdout:3/270: fsync db/fc 0 2024-05-27T09:29:13.970 INFO:tasks.workunit.client.0.smithi119.stdout:3/271: dread - db/d10/d13/d39/f51 zero size 2024-05-27T09:29:13.982 INFO:tasks.workunit.client.0.smithi119.stdout:6/109: mkdir d0/d2/d24 0 2024-05-27T09:29:13.985 INFO:tasks.workunit.client.0.smithi119.stdout:6/110: symlink d0/d2/d1c/l25 0 2024-05-27T09:29:13.986 INFO:tasks.workunit.client.0.smithi119.stdout:6/111: getdents d0/d19 0 2024-05-27T09:29:13.991 INFO:tasks.workunit.client.0.smithi119.stdout:6/112: rename d0/d2/de to d0/d11/d26 0 2024-05-27T09:29:14.039 INFO:tasks.workunit.client.0.smithi119.stdout:8/149: dwrite d3/d24/f2b [0,4194304] 0 2024-05-27T09:29:14.040 INFO:tasks.workunit.client.0.smithi119.stdout:8/150: fdatasync d3/de/f23 0 2024-05-27T09:29:14.042 INFO:tasks.workunit.client.0.smithi119.stdout:8/151: mkdir d3/d24/d33 0 2024-05-27T09:29:14.045 INFO:tasks.workunit.client.0.smithi119.stdout:8/152: rename d3/de/f10 to d3/d8/f34 0 2024-05-27T09:29:14.055 INFO:tasks.workunit.client.0.smithi119.stdout:6/113: dwrite d0/d2/f9 [0,4194304] 0 2024-05-27T09:29:14.055 INFO:tasks.workunit.client.0.smithi119.stdout:3/272: dwrite db/d10/d13/f3d [0,4194304] 0 2024-05-27T09:29:14.061 INFO:tasks.workunit.client.0.smithi119.stdout:6/114: mknod d0/d11/d26/c27 0 2024-05-27T09:29:14.061 INFO:tasks.workunit.client.0.smithi119.stdout:6/115: write d0/d11/d26/f1f [820053,46212] 0 2024-05-27T09:29:14.061 INFO:tasks.workunit.client.0.smithi119.stdout:6/116: chown d0/cf 3 1 2024-05-27T09:29:14.061 INFO:tasks.workunit.client.0.smithi119.stdout:8/153: dread d3/f1a [0,4194304] 0 2024-05-27T09:29:14.063 INFO:tasks.workunit.client.0.smithi119.stdout:3/273: unlink db/d10/c44 0 2024-05-27T09:29:14.067 INFO:tasks.workunit.client.0.smithi119.stdout:3/274: link c4 db/d36/c59 0 2024-05-27T09:29:14.068 INFO:tasks.workunit.client.0.smithi119.stdout:3/275: creat db/d10/d13/d39/f5a x:0 0 0 2024-05-27T09:29:14.068 INFO:tasks.workunit.client.0.smithi119.stdout:3/276: fsync db/d10/f17 0 2024-05-27T09:29:14.068 INFO:tasks.workunit.client.0.smithi119.stdout:3/277: chown db/d10/d13/d33/d43 496566890 1 2024-05-27T09:29:14.073 INFO:tasks.workunit.client.0.smithi119.stdout:5/150: dwrite d6/db/dc/f26 [0,4194304] 0 2024-05-27T09:29:14.081 INFO:tasks.workunit.client.0.smithi119.stdout:2/169: dread d2/da/f1c [0,4194304] 0 2024-05-27T09:29:14.081 INFO:tasks.workunit.client.0.smithi119.stdout:3/278: mkdir db/d5b 0 2024-05-27T09:29:14.081 INFO:tasks.workunit.client.0.smithi119.stdout:4/135: dwrite d4/f22 [0,4194304] 0 2024-05-27T09:29:14.084 INFO:tasks.workunit.client.0.smithi119.stdout:9/131: dread d18/d19/d1b/f24 [0,4194304] 0 2024-05-27T09:29:14.084 INFO:tasks.workunit.client.0.smithi119.stdout:9/132: fdatasync ff 0 2024-05-27T09:29:14.085 INFO:tasks.workunit.client.0.smithi119.stdout:6/117: read d0/fd [800625,55139] 0 2024-05-27T09:29:14.088 INFO:tasks.workunit.client.0.smithi119.stdout:5/151: mknod d6/db/c35 0 2024-05-27T09:29:14.089 INFO:tasks.workunit.client.0.smithi119.stdout:8/154: write d3/d24/d30/f32 [1942832,95130] 0 2024-05-27T09:29:14.091 INFO:tasks.workunit.client.0.smithi119.stdout:4/136: fdatasync d4/d6/f1a 0 2024-05-27T09:29:14.091 INFO:tasks.workunit.client.0.smithi119.stdout:4/137: creat d4/d6/f2a x:0 0 0 2024-05-27T09:29:14.092 INFO:tasks.workunit.client.0.smithi119.stdout:1/202: dwrite d7/db/d12/d15/d1f/f34 [0,4194304] 0 2024-05-27T09:29:14.092 INFO:tasks.workunit.client.0.smithi119.stdout:3/279: rename db/d10/d13/d39/f53 to db/d10/d13/d33/d43/f5c 0 2024-05-27T09:29:14.096 INFO:tasks.workunit.client.0.smithi119.stdout:9/133: link fe d18/d19/d1b/d29/f2c 0 2024-05-27T09:29:14.099 INFO:tasks.workunit.client.0.smithi119.stdout:6/118: rmdir d0/d19/d23 0 2024-05-27T09:29:14.102 INFO:tasks.workunit.client.0.smithi119.stdout:5/152: creat d6/db/f36 x:0 0 0 2024-05-27T09:29:14.102 INFO:tasks.workunit.client.0.smithi119.stdout:5/153: creat d6/dd/f37 x:0 0 0 2024-05-27T09:29:14.106 INFO:tasks.workunit.client.0.smithi119.stdout:8/155: symlink d3/d8/l35 0 2024-05-27T09:29:14.106 INFO:tasks.workunit.client.0.smithi119.stdout:8/156: dread - d3/d8/d1b/f2a zero size 2024-05-27T09:29:14.108 INFO:tasks.workunit.client.0.smithi119.stdout:9/134: mknod d18/d19/c2d 0 2024-05-27T09:29:14.108 INFO:tasks.workunit.client.0.smithi119.stdout:9/135: dread - d18/f26 zero size 2024-05-27T09:29:14.109 INFO:tasks.workunit.client.0.smithi119.stdout:6/119: creat d0/d2/d1c/d21/f28 x:0 0 0 2024-05-27T09:29:14.110 INFO:tasks.workunit.client.0.smithi119.stdout:2/170: write d2/da/d15/f1b [3862657,64382] 0 2024-05-27T09:29:14.112 INFO:tasks.workunit.client.0.smithi119.stdout:8/157: creat d3/d24/d30/f36 x:0 0 0 2024-05-27T09:29:14.115 INFO:tasks.workunit.client.0.smithi119.stdout:9/136: link fb d18/f2e 0 2024-05-27T09:29:14.115 INFO:tasks.workunit.client.0.smithi119.stdout:9/137: fdatasync d18/d19/d1b/f1f 0 2024-05-27T09:29:14.115 INFO:tasks.workunit.client.0.smithi119.stdout:9/138: write f2 [134055,21055] 0 2024-05-27T09:29:14.115 INFO:tasks.workunit.client.0.smithi119.stdout:9/139: chown f6 24157958 1 2024-05-27T09:29:14.115 INFO:tasks.workunit.client.0.smithi119.stdout:3/280: dread db/d10/d13/f2e [0,4194304] 0 2024-05-27T09:29:14.115 INFO:tasks.workunit.client.0.smithi119.stdout:6/120: symlink d0/d2/l29 0 2024-05-27T09:29:14.115 INFO:tasks.workunit.client.0.smithi119.stdout:6/121: getdents d0/d2/d24 0 2024-05-27T09:29:14.115 INFO:tasks.workunit.client.0.smithi119.stdout:0/115: dwrite d2/d6/d9/f1c [0,4194304] 0 2024-05-27T09:29:14.116 INFO:tasks.workunit.client.0.smithi119.stdout:0/116: readlink d2/d6/d9/ld 0 2024-05-27T09:29:14.116 INFO:tasks.workunit.client.0.smithi119.stdout:0/117: readlink d2/d6/d9/d12/l1d 0 2024-05-27T09:29:14.116 INFO:tasks.workunit.client.0.smithi119.stdout:2/171: mknod d2/da/d2c/c40 0 2024-05-27T09:29:14.116 INFO:tasks.workunit.client.0.smithi119.stdout:2/172: readlink d2/l24 0 2024-05-27T09:29:14.119 INFO:tasks.workunit.client.0.smithi119.stdout:6/122: read d0/f18 [809462,47370] 0 2024-05-27T09:29:14.121 INFO:tasks.workunit.client.0.smithi119.stdout:9/140: creat d18/d19/d1b/d29/f2f x:0 0 0 2024-05-27T09:29:14.121 INFO:tasks.workunit.client.0.smithi119.stdout:9/141: fsync d18/d19/d1b/f1e 0 2024-05-27T09:29:14.121 INFO:tasks.workunit.client.0.smithi119.stdout:9/142: stat d18/d19/f1c 0 2024-05-27T09:29:14.122 INFO:tasks.workunit.client.0.smithi119.stdout:5/154: getdents d6/db 0 2024-05-27T09:29:14.122 INFO:tasks.workunit.client.0.smithi119.stdout:5/155: chown d6/d2f 9 1 2024-05-27T09:29:14.124 INFO:tasks.workunit.client.0.smithi119.stdout:0/118: stat d2/l15 0 2024-05-27T09:29:14.126 INFO:tasks.workunit.client.0.smithi119.stdout:2/173: creat d2/da/d15/d33/f41 x:0 0 0 2024-05-27T09:29:14.127 INFO:tasks.workunit.client.0.smithi119.stdout:8/158: getdents d3/d24 0 2024-05-27T09:29:14.127 INFO:tasks.workunit.client.0.smithi119.stdout:8/159: truncate d3/d8/d1b/f28 298098 0 2024-05-27T09:29:14.127 INFO:tasks.workunit.client.0.smithi119.stdout:8/160: stat d3/de/l17 0 2024-05-27T09:29:14.129 INFO:tasks.workunit.client.0.smithi119.stdout:3/281: mknod db/d10/d13/c5d 0 2024-05-27T09:29:14.134 INFO:tasks.workunit.client.0.smithi119.stdout:8/161: rename f1 to d3/d24/f37 0 2024-05-27T09:29:14.134 INFO:tasks.workunit.client.0.smithi119.stdout:8/162: chown d3/l9 633926 1 2024-05-27T09:29:14.134 INFO:tasks.workunit.client.0.smithi119.stdout:8/163: chown d3/f5 22362755 1 2024-05-27T09:29:14.136 INFO:tasks.workunit.client.0.smithi119.stdout:3/282: unlink db/f30 0 2024-05-27T09:29:14.139 INFO:tasks.workunit.client.0.smithi119.stdout:0/119: truncate d2/d6/d9/f1c 350376 0 2024-05-27T09:29:14.139 INFO:tasks.workunit.client.0.smithi119.stdout:0/120: creat d2/d6/f26 x:0 0 0 2024-05-27T09:29:14.139 INFO:tasks.workunit.client.0.smithi119.stdout:5/156: dread d6/f12 [0,4194304] 0 2024-05-27T09:29:14.141 INFO:tasks.workunit.client.0.smithi119.stdout:3/283: rename c7 to db/d36/d45/c5e 0 2024-05-27T09:29:14.141 INFO:tasks.workunit.client.0.smithi119.stdout:3/284: creat db/d10/d13/d33/d43/f5f x:0 0 0 2024-05-27T09:29:14.141 INFO:tasks.workunit.client.0.smithi119.stdout:3/285: stat db/f12 0 2024-05-27T09:29:14.141 INFO:tasks.workunit.client.0.smithi119.stdout:3/286: chown db/d36/l4a 1688280650 1 2024-05-27T09:29:14.142 INFO:tasks.workunit.client.0.smithi119.stdout:8/164: getdents d3/de 0 2024-05-27T09:29:14.142 INFO:tasks.workunit.client.0.smithi119.stdout:8/165: read d3/d24/d30/f32 [4336209,90472] 0 2024-05-27T09:29:14.149 INFO:tasks.workunit.client.0.smithi119.stdout:6/123: write d0/d2/f9 [577379,28179] 0 2024-05-27T09:29:14.149 INFO:tasks.workunit.client.0.smithi119.stdout:5/157: link d6/db/l23 d6/db/de/d27/l38 0 2024-05-27T09:29:14.158 INFO:tasks.workunit.client.0.smithi119.stdout:3/287: getdents db/d10/d13 0 2024-05-27T09:29:14.160 INFO:tasks.workunit.client.0.smithi119.stdout:3/288: truncate db/d10/d13/d33/d43/f47 5110799 0 2024-05-27T09:29:14.169 INFO:tasks.workunit.client.0.smithi119.stdout:2/174: dwrite d2/d1a/f30 [0,4194304] 0 2024-05-27T09:29:14.169 INFO:tasks.workunit.client.0.smithi119.stdout:2/175: truncate d2/d1a/f3b 397652 0 2024-05-27T09:29:14.176 INFO:tasks.workunit.client.0.smithi119.stdout:2/176: link d2/da/d2c/c40 d2/c42 0 2024-05-27T09:29:14.176 INFO:tasks.workunit.client.0.smithi119.stdout:2/177: fsync d2/da/d2c/f38 0 2024-05-27T09:29:14.176 INFO:tasks.workunit.client.0.smithi119.stdout:2/178: mknod d2/d1a/c43 0 2024-05-27T09:29:14.177 INFO:tasks.workunit.client.0.smithi119.stdout:2/179: symlink d2/da/d15/l44 0 2024-05-27T09:29:14.180 INFO:tasks.workunit.client.0.smithi119.stdout:2/180: link d2/da/c3f d2/da/d2c/c45 0 2024-05-27T09:29:14.180 INFO:tasks.workunit.client.0.smithi119.stdout:2/181: creat d2/d1a/f46 x:0 0 0 2024-05-27T09:29:14.184 INFO:tasks.workunit.client.0.smithi119.stdout:2/182: readlink d2/d1a/l2f 0 2024-05-27T09:29:14.205 INFO:tasks.workunit.client.0.smithi119.stdout:7/234: sync 2024-05-27T09:29:14.210 INFO:tasks.workunit.client.0.smithi119.stdout:7/235: mkdir d7/d1a/d40/d44/d51 0 2024-05-27T09:29:14.211 INFO:tasks.workunit.client.0.smithi119.stdout:7/236: creat d7/df/d36/f52 x:0 0 0 2024-05-27T09:29:14.213 INFO:tasks.workunit.client.0.smithi119.stdout:6/124: dwrite d0/d11/f15 [4194304,4194304] 0 2024-05-27T09:29:14.219 INFO:tasks.workunit.client.0.smithi119.stdout:6/125: rename d0/d11/f15 to d0/d2/d1c/f2a 0 2024-05-27T09:29:14.221 INFO:tasks.workunit.client.0.smithi119.stdout:6/126: rename d0/d2/f9 to d0/d2/d1c/d21/f2b 0 2024-05-27T09:29:14.226 INFO:tasks.workunit.client.0.smithi119.stdout:6/127: symlink d0/d11/l2c 0 2024-05-27T09:29:14.226 INFO:tasks.workunit.client.0.smithi119.stdout:0/121: dwrite d2/d6/f8 [0,4194304] 0 2024-05-27T09:29:14.226 INFO:tasks.workunit.client.0.smithi119.stdout:3/289: write db/f2f [1816490,42347] 0 2024-05-27T09:29:14.227 INFO:tasks.workunit.client.0.smithi119.stdout:3/290: creat db/d36/d45/d49/d4f/f60 x:0 0 0 2024-05-27T09:29:14.228 INFO:tasks.workunit.client.0.smithi119.stdout:0/122: rmdir d2/d6 39 2024-05-27T09:29:14.230 INFO:tasks.workunit.client.0.smithi119.stdout:0/123: rmdir d2/d1a 39 2024-05-27T09:29:14.232 INFO:tasks.workunit.client.0.smithi119.stdout:0/124: rename d2/d1a/f1f to d2/d6/d9/f27 0 2024-05-27T09:29:14.244 INFO:tasks.workunit.client.0.smithi119.stdout:6/128: write d0/f6 [1259536,12527] 0 2024-05-27T09:29:14.244 INFO:tasks.workunit.client.0.smithi119.stdout:6/129: chown d0/d2/d1c/d21/f28 158558 1 2024-05-27T09:29:14.245 INFO:tasks.workunit.client.0.smithi119.stdout:5/158: dwrite d6/db/dc/f26 [0,4194304] 0 2024-05-27T09:29:14.253 INFO:tasks.workunit.client.0.smithi119.stdout:6/130: dread d0/f18 [0,4194304] 0 2024-05-27T09:29:14.253 INFO:tasks.workunit.client.0.smithi119.stdout:8/166: dwrite d3/d24/d30/f36 [0,4194304] 0 2024-05-27T09:29:14.253 INFO:tasks.workunit.client.0.smithi119.stdout:6/131: write d0/d2/d1c/d21/f28 [20232,9585] 0 2024-05-27T09:29:14.253 INFO:tasks.workunit.client.0.smithi119.stdout:6/132: write d0/d2/f20 [530696,49304] 0 2024-05-27T09:29:14.257 INFO:tasks.workunit.client.0.smithi119.stdout:8/167: creat d3/d24/f38 x:0 0 0 2024-05-27T09:29:14.257 INFO:tasks.workunit.client.0.smithi119.stdout:6/133: rename d0/f7 to d0/d2/d1c/f2d 0 2024-05-27T09:29:14.259 INFO:tasks.workunit.client.0.smithi119.stdout:8/168: rename d3/d24/d33 to d3/d39 0 2024-05-27T09:29:14.260 INFO:tasks.workunit.client.0.smithi119.stdout:8/169: chown d3/d24 2 1 2024-05-27T09:29:14.269 INFO:tasks.workunit.client.0.smithi119.stdout:2/183: dwrite d2/f6 [0,4194304] 0 2024-05-27T09:29:14.269 INFO:tasks.workunit.client.0.smithi119.stdout:2/184: write d2/f26 [5081586,76933] 0 2024-05-27T09:29:14.269 INFO:tasks.workunit.client.0.smithi119.stdout:2/185: creat d2/f47 x:0 0 0 2024-05-27T09:29:14.269 INFO:tasks.workunit.client.0.smithi119.stdout:2/186: fsync d2/d1a/f30 0 2024-05-27T09:29:14.272 INFO:tasks.workunit.client.0.smithi119.stdout:8/170: dread d3/f5 [0,4194304] 0 2024-05-27T09:29:14.278 INFO:tasks.workunit.client.0.smithi119.stdout:2/187: write d2/d1a/f3b [256005,2538] 0 2024-05-27T09:29:14.281 INFO:tasks.workunit.client.0.smithi119.stdout:7/237: dwrite d7/df/f29 [0,4194304] 0 2024-05-27T09:29:14.281 INFO:tasks.workunit.client.0.smithi119.stdout:8/171: dread d3/f12 [0,4194304] 0 2024-05-27T09:29:14.281 INFO:tasks.workunit.client.0.smithi119.stdout:8/172: chown d3/d8/c27 434788 1 2024-05-27T09:29:14.282 INFO:tasks.workunit.client.0.smithi119.stdout:7/238: getdents d7/df/d19/d4d 0 2024-05-27T09:29:14.282 INFO:tasks.workunit.client.0.smithi119.stdout:7/239: write d7/df/d19/d4d/f4e [83571,76885] 0 2024-05-27T09:29:14.284 INFO:tasks.workunit.client.0.smithi119.stdout:8/173: rename d3/c7 to d3/d39/c3a 0 2024-05-27T09:29:14.284 INFO:tasks.workunit.client.0.smithi119.stdout:8/174: chown d3/d8/d1b 4 1 2024-05-27T09:29:14.284 INFO:tasks.workunit.client.0.smithi119.stdout:7/240: symlink d7/df/d19/d46/l53 0 2024-05-27T09:29:14.284 INFO:tasks.workunit.client.0.smithi119.stdout:7/241: fsync d7/fd 0 2024-05-27T09:29:14.285 INFO:tasks.workunit.client.0.smithi119.stdout:8/175: read d3/d24/f2f [490465,26817] 0 2024-05-27T09:29:14.285 INFO:tasks.workunit.client.0.smithi119.stdout:6/134: write d0/d2/f10 [6924577,102542] 0 2024-05-27T09:29:14.286 INFO:tasks.workunit.client.0.smithi119.stdout:7/242: mknod d7/d1a/d40/d44/c54 0 2024-05-27T09:29:14.287 INFO:tasks.workunit.client.0.smithi119.stdout:8/176: link d3/d8/c2e d3/d8/c3b 0 2024-05-27T09:29:14.288 INFO:tasks.workunit.client.0.smithi119.stdout:8/177: fdatasync d3/f5 0 2024-05-27T09:29:14.288 INFO:tasks.workunit.client.0.smithi119.stdout:8/178: getdents d3/de 0 2024-05-27T09:29:14.288 INFO:tasks.workunit.client.0.smithi119.stdout:2/188: dread d2/f2d [0,4194304] 0 2024-05-27T09:29:14.288 INFO:tasks.workunit.client.0.smithi119.stdout:2/189: creat d2/f48 x:0 0 0 2024-05-27T09:29:14.293 INFO:tasks.workunit.client.0.smithi119.stdout:8/179: mknod d3/d24/c3c 0 2024-05-27T09:29:14.293 INFO:tasks.workunit.client.0.smithi119.stdout:7/243: truncate d7/df/d36/f3f 973430 0 2024-05-27T09:29:14.293 INFO:tasks.workunit.client.0.smithi119.stdout:0/125: dread d2/d6/d9/ff [0,4194304] 0 2024-05-27T09:29:14.293 INFO:tasks.workunit.client.0.smithi119.stdout:0/126: write d2/d6/d9/d12/f25 [418566,91235] 0 2024-05-27T09:29:14.295 INFO:tasks.workunit.client.0.smithi119.stdout:8/180: mkdir d3/d8/d3d 0 2024-05-27T09:29:14.296 INFO:tasks.workunit.client.0.smithi119.stdout:3/291: dwrite db/d10/d13/d33/d43/f5c [0,4194304] 0 2024-05-27T09:29:14.299 INFO:tasks.workunit.client.0.smithi119.stdout:8/181: unlink d3/f5 0 2024-05-27T09:29:14.303 INFO:tasks.workunit.client.0.smithi119.stdout:6/135: truncate d0/d2/d1c/f2d 4036702 0 2024-05-27T09:29:14.316 INFO:tasks.workunit.client.0.smithi119.stdout:6/136: creat d0/d11/d26/f2e x:0 0 0 2024-05-27T09:29:14.316 INFO:tasks.workunit.client.0.smithi119.stdout:7/244: rmdir d7/d1a/d40 39 2024-05-27T09:29:14.319 INFO:tasks.workunit.client.0.smithi119.stdout:7/245: creat d7/db/d37/f55 x:0 0 0 2024-05-27T09:29:14.331 INFO:tasks.workunit.client.0.smithi119.stdout:3/292: dread db/d10/d13/d33/d43/f5c [0,4194304] 0 2024-05-27T09:29:14.337 INFO:tasks.workunit.client.0.smithi119.stdout:6/137: dread d0/fb [0,4194304] 0 2024-05-27T09:29:14.337 INFO:tasks.workunit.client.0.smithi119.stdout:6/138: truncate d0/d11/d26/f1f 1624088 0 2024-05-27T09:29:14.338 INFO:tasks.workunit.client.0.smithi119.stdout:6/139: chown d0/d2/f4 354037057 1 2024-05-27T09:29:14.339 INFO:tasks.workunit.client.0.smithi119.stdout:6/140: symlink d0/d11/d26/l2f 0 2024-05-27T09:29:14.341 INFO:tasks.workunit.client.0.smithi119.stdout:3/293: dread db/fc [0,4194304] 0 2024-05-27T09:29:14.342 INFO:tasks.workunit.client.0.smithi119.stdout:3/294: truncate db/d10/d13/f21 3868600 0 2024-05-27T09:29:14.342 INFO:tasks.workunit.client.0.smithi119.stdout:3/295: chown db/d10/d13/d39/c48 22 1 2024-05-27T09:29:14.345 INFO:tasks.workunit.client.0.smithi119.stdout:3/296: unlink db/d36/d45/d49/c57 0 2024-05-27T09:29:14.345 INFO:tasks.workunit.client.0.smithi119.stdout:3/297: getdents db/d10/d13 0 2024-05-27T09:29:14.346 INFO:tasks.workunit.client.0.smithi119.stdout:3/298: stat db/d36/d45/d49 0 2024-05-27T09:29:14.347 INFO:tasks.workunit.client.0.smithi119.stdout:3/299: mknod db/d10/d13/c61 0 2024-05-27T09:29:14.351 INFO:tasks.workunit.client.0.smithi119.stdout:3/300: truncate db/f37 97357 0 2024-05-27T09:29:14.355 INFO:tasks.workunit.client.0.smithi119.stdout:8/182: dwrite d3/f25 [0,4194304] 0 2024-05-27T09:29:14.355 INFO:tasks.workunit.client.0.smithi119.stdout:3/301: readlink db/l11 0 2024-05-27T09:29:14.356 INFO:tasks.workunit.client.0.smithi119.stdout:8/183: write d3/d8/d1b/f2a [612300,8110] 0 2024-05-27T09:29:14.357 INFO:tasks.workunit.client.0.smithi119.stdout:3/302: truncate db/f23 830724 0 2024-05-27T09:29:14.364 INFO:tasks.workunit.client.0.smithi119.stdout:3/303: unlink db/d10/d13/c61 0 2024-05-27T09:29:14.364 INFO:tasks.workunit.client.0.smithi119.stdout:3/304: write db/f56 [521851,120118] 0 2024-05-27T09:29:14.380 INFO:tasks.workunit.client.0.smithi119.stdout:0/127: dwrite d2/d1a/f1e [0,4194304] 0 2024-05-27T09:29:14.419 INFO:tasks.workunit.client.0.smithi119.stdout:3/305: dwrite db/f37 [0,4194304] 0 2024-05-27T09:29:14.426 INFO:tasks.workunit.client.0.smithi119.stdout:3/306: rename db/d10 to db/d36/d45/d4c/d62 0 2024-05-27T09:29:14.427 INFO:tasks.workunit.client.0.smithi119.stdout:3/307: creat db/d5b/f63 x:0 0 0 2024-05-27T09:29:14.429 INFO:tasks.workunit.client.0.smithi119.stdout:6/141: dwrite d0/f6 [0,4194304] 0 2024-05-27T09:29:14.429 INFO:tasks.workunit.client.0.smithi119.stdout:6/142: creat d0/d2/f30 x:0 0 0 2024-05-27T09:29:14.440 INFO:tasks.workunit.client.0.smithi119.stdout:7/246: dwrite d7/df/d19/d1d/f2a [0,4194304] 0 2024-05-27T09:29:14.443 INFO:tasks.workunit.client.0.smithi119.stdout:0/128: dwrite d2/d1a/f1e [0,4194304] 0 2024-05-27T09:29:14.445 INFO:tasks.workunit.client.0.smithi119.stdout:7/247: unlink d7/db/f22 0 2024-05-27T09:29:14.445 INFO:tasks.workunit.client.0.smithi119.stdout:7/248: fdatasync d7/df/d36/f49 0 2024-05-27T09:29:14.446 INFO:tasks.workunit.client.0.smithi119.stdout:0/129: mkdir d2/d6/d28 0 2024-05-27T09:29:14.451 INFO:tasks.workunit.client.0.smithi119.stdout:7/249: write d7/db/f31 [541423,91410] 0 2024-05-27T09:29:14.452 INFO:tasks.workunit.client.0.smithi119.stdout:0/130: mknod d2/d6/c29 0 2024-05-27T09:29:14.452 INFO:tasks.workunit.client.0.smithi119.stdout:3/308: write db/d36/d45/d4c/d62/f17 [1855959,118999] 0 2024-05-27T09:29:14.454 INFO:tasks.workunit.client.0.smithi119.stdout:4/138: sync 2024-05-27T09:29:14.454 INFO:tasks.workunit.client.0.smithi119.stdout:1/203: sync 2024-05-27T09:29:14.454 INFO:tasks.workunit.client.0.smithi119.stdout:9/143: sync 2024-05-27T09:29:14.455 INFO:tasks.workunit.client.0.smithi119.stdout:1/204: fsync d7/dd/d25/f30 0 2024-05-27T09:29:14.455 INFO:tasks.workunit.client.0.smithi119.stdout:1/205: chown d7/db/d12/c28 7340488 1 2024-05-27T09:29:14.455 INFO:tasks.workunit.client.0.smithi119.stdout:3/309: write db/f23 [1506427,80213] 0 2024-05-27T09:29:14.457 INFO:tasks.workunit.client.0.smithi119.stdout:0/131: rmdir d2/d6/d28 0 2024-05-27T09:29:14.457 INFO:tasks.workunit.client.0.smithi119.stdout:0/132: fdatasync d2/d6/d9/d12/f16 0 2024-05-27T09:29:14.457 INFO:tasks.workunit.client.0.smithi119.stdout:0/133: write d2/d6/d9/f1c [1196939,18852] 0 2024-05-27T09:29:14.457 INFO:tasks.workunit.client.0.smithi119.stdout:0/134: readlink d2/l10 0 2024-05-27T09:29:14.459 INFO:tasks.workunit.client.0.smithi119.stdout:9/144: rename d18/f2e to d18/f30 0 2024-05-27T09:29:14.459 INFO:tasks.workunit.client.0.smithi119.stdout:9/145: read f10 [449765,39943] 0 2024-05-27T09:29:14.459 INFO:tasks.workunit.client.0.smithi119.stdout:9/146: stat d18/d19/d1b/f24 0 2024-05-27T09:29:14.459 INFO:tasks.workunit.client.0.smithi119.stdout:9/147: write f15 [952625,98012] 0 2024-05-27T09:29:14.459 INFO:tasks.workunit.client.0.smithi119.stdout:9/148: fdatasync d18/d19/d1b/f1d 0 2024-05-27T09:29:14.459 INFO:tasks.workunit.client.0.smithi119.stdout:9/149: chown d18/f26 939143 1 2024-05-27T09:29:14.472 INFO:tasks.workunit.client.0.smithi119.stdout:4/139: dread d4/d6/fa [0,4194304] 0 2024-05-27T09:29:14.472 INFO:tasks.workunit.client.0.smithi119.stdout:1/206: unlink d7/db/d12/d15/d1f/f31 0 2024-05-27T09:29:14.472 INFO:tasks.workunit.client.0.smithi119.stdout:1/207: chown d7/db/d12/f23 381560 1 2024-05-27T09:29:14.472 INFO:tasks.workunit.client.0.smithi119.stdout:1/208: write d7/db/d12/d15/d1f/f3a [145396,20388] 0 2024-05-27T09:29:14.472 INFO:tasks.workunit.client.0.smithi119.stdout:3/310: rename db/d36/d45/d4c/d62/d13/d39 to db/d36/d45/d4d/d64 0 2024-05-27T09:29:14.472 INFO:tasks.workunit.client.0.smithi119.stdout:3/311: truncate db/d36/d45/d4d/d64/f5a 743618 0 2024-05-27T09:29:14.472 INFO:tasks.workunit.client.0.smithi119.stdout:3/312: write db/f2f [1541206,112615] 0 2024-05-27T09:29:14.472 INFO:tasks.workunit.client.0.smithi119.stdout:7/250: dread d7/fd [0,4194304] 0 2024-05-27T09:29:14.473 INFO:tasks.workunit.client.0.smithi119.stdout:7/251: readlink d7/df/d19/l4b 0 2024-05-27T09:29:14.473 INFO:tasks.workunit.client.0.smithi119.stdout:7/252: dread - d7/db/d37/f55 zero size 2024-05-27T09:29:14.473 INFO:tasks.workunit.client.0.smithi119.stdout:8/184: dwrite d3/f2d [0,4194304] 0 2024-05-27T09:29:14.473 INFO:tasks.workunit.client.0.smithi119.stdout:0/135: link d2/d1a/f1e d2/d1a/f2a 0 2024-05-27T09:29:14.473 INFO:tasks.workunit.client.0.smithi119.stdout:9/150: unlink d18/d19/d1b/f24 0 2024-05-27T09:29:14.473 INFO:tasks.workunit.client.0.smithi119.stdout:9/151: chown l14 263 1 2024-05-27T09:29:14.477 INFO:tasks.workunit.client.0.smithi119.stdout:3/313: truncate db/f35 366197 0 2024-05-27T09:29:14.477 INFO:tasks.workunit.client.0.smithi119.stdout:3/314: write db/d36/d45/d4c/d62/d13/f21 [3973631,34343] 0 2024-05-27T09:29:14.478 INFO:tasks.workunit.client.0.smithi119.stdout:3/315: write db/fc [1033410,31100] 0 2024-05-27T09:29:14.483 INFO:tasks.workunit.client.0.smithi119.stdout:0/136: creat d2/d6/f2b x:0 0 0 2024-05-27T09:29:14.484 INFO:tasks.workunit.client.0.smithi119.stdout:7/253: getdents d7/db/d37 0 2024-05-27T09:29:14.486 INFO:tasks.workunit.client.0.smithi119.stdout:0/137: creat d2/d6/d9/f2c x:0 0 0 2024-05-27T09:29:14.497 INFO:tasks.workunit.client.0.smithi119.stdout:4/140: read d4/d23/f26 [3314834,33072] 0 2024-05-27T09:29:14.501 INFO:tasks.workunit.client.0.smithi119.stdout:4/141: creat d4/de/d21/f2b x:0 0 0 2024-05-27T09:29:14.503 INFO:tasks.workunit.client.0.smithi119.stdout:5/159: sync 2024-05-27T09:29:14.506 INFO:tasks.workunit.client.0.smithi119.stdout:6/143: dwrite d0/d11/d26/f2e [0,4194304] 0 2024-05-27T09:29:14.507 INFO:tasks.workunit.client.0.smithi119.stdout:5/160: getdents d6/dd 0 2024-05-27T09:29:14.507 INFO:tasks.workunit.client.0.smithi119.stdout:5/161: truncate d6/dd/f32 5084769 0 2024-05-27T09:29:14.512 INFO:tasks.workunit.client.0.smithi119.stdout:5/162: rename d6/dd/df/d13 to d6/dd/d39 0 2024-05-27T09:29:14.518 INFO:tasks.workunit.client.0.smithi119.stdout:6/144: rmdir d0/d2 39 2024-05-27T09:29:14.518 INFO:tasks.workunit.client.0.smithi119.stdout:5/163: creat d6/d2f/f3a x:0 0 0 2024-05-27T09:29:14.518 INFO:tasks.workunit.client.0.smithi119.stdout:5/164: creat d6/db/dc/f3b x:0 0 0 2024-05-27T09:29:14.526 INFO:tasks.workunit.client.0.smithi119.stdout:3/316: dread f8 [4194304,4194304] 0 2024-05-27T09:29:14.528 INFO:tasks.workunit.client.0.smithi119.stdout:3/317: symlink db/d5b/l65 0 2024-05-27T09:29:14.536 INFO:tasks.workunit.client.0.smithi119.stdout:0/138: write d2/d6/d9/ff [1329545,18731] 0 2024-05-27T09:29:14.537 INFO:tasks.workunit.client.0.smithi119.stdout:7/254: dread d7/d1a/f1c [0,4194304] 0 2024-05-27T09:29:14.538 INFO:tasks.workunit.client.0.smithi119.stdout:7/255: mknod d7/db/d37/c56 0 2024-05-27T09:29:14.540 INFO:tasks.workunit.client.0.smithi119.stdout:1/209: dwrite f2 [0,4194304] 0 2024-05-27T09:29:14.540 INFO:tasks.workunit.client.0.smithi119.stdout:1/210: chown f3 60022 1 2024-05-27T09:29:14.540 INFO:tasks.workunit.client.0.smithi119.stdout:1/211: chown d7/db/d35/c37 4 1 2024-05-27T09:29:14.540 INFO:tasks.workunit.client.0.smithi119.stdout:1/212: write d7/db/d12/f23 [9068,125634] 0 2024-05-27T09:29:14.547 INFO:tasks.workunit.client.0.smithi119.stdout:4/142: dread d4/d6/f11 [0,4194304] 0 2024-05-27T09:29:14.547 INFO:tasks.workunit.client.0.smithi119.stdout:0/139: rename d2/d6/c29 to d2/d6/d9/d24/c2d 0 2024-05-27T09:29:14.548 INFO:tasks.workunit.client.0.smithi119.stdout:5/165: dread d6/f1d [0,4194304] 0 2024-05-27T09:29:14.548 INFO:tasks.workunit.client.0.smithi119.stdout:5/166: fdatasync d6/db/de/f25 0 2024-05-27T09:29:14.548 INFO:tasks.workunit.client.0.smithi119.stdout:8/185: dwrite d3/f25 [0,4194304] 0 2024-05-27T09:29:14.551 INFO:tasks.workunit.client.0.smithi119.stdout:4/143: symlink d4/d1c/l2c 0 2024-05-27T09:29:14.555 INFO:tasks.workunit.client.0.smithi119.stdout:1/213: rename d7/db/d12/d26/l3b to d7/dd/d25/d29/d38/d3c/l41 0 2024-05-27T09:29:14.557 INFO:tasks.workunit.client.0.smithi119.stdout:1/214: mknod d7/dd/d25/d29/c42 0 2024-05-27T09:29:14.559 INFO:tasks.workunit.client.0.smithi119.stdout:8/186: getdents d3/d24 0 2024-05-27T09:29:14.559 INFO:tasks.workunit.client.0.smithi119.stdout:8/187: dread - d3/d24/f38 zero size 2024-05-27T09:29:14.559 INFO:tasks.workunit.client.0.smithi119.stdout:8/188: chown d3/d24/f26 0 1 2024-05-27T09:29:14.570 INFO:tasks.workunit.client.0.smithi119.stdout:1/215: truncate d7/db/d12/f23 132177 0 2024-05-27T09:29:14.571 INFO:tasks.workunit.client.0.smithi119.stdout:1/216: creat d7/db/d12/d26/f43 x:0 0 0 2024-05-27T09:29:14.571 INFO:tasks.workunit.client.0.smithi119.stdout:1/217: write d7/db/d12/d15/f3f [5155778,25920] 0 2024-05-27T09:29:14.573 INFO:tasks.workunit.client.0.smithi119.stdout:6/145: dwrite d0/d2/f10 [4194304,4194304] 0 2024-05-27T09:29:14.574 INFO:tasks.workunit.client.0.smithi119.stdout:0/140: dread d2/d1a/f2a [0,4194304] 0 2024-05-27T09:29:14.574 INFO:tasks.workunit.client.0.smithi119.stdout:1/218: mkdir d7/dd/d44 0 2024-05-27T09:29:14.574 INFO:tasks.workunit.client.0.smithi119.stdout:8/189: write d3/d8/f34 [649444,33572] 0 2024-05-27T09:29:14.576 INFO:tasks.workunit.client.0.smithi119.stdout:6/146: creat d0/d2/f31 x:0 0 0 2024-05-27T09:29:14.576 INFO:tasks.workunit.client.0.smithi119.stdout:6/147: fdatasync d0/d2/f20 0 2024-05-27T09:29:14.579 INFO:tasks.workunit.client.0.smithi119.stdout:8/190: unlink d3/de/f21 0 2024-05-27T09:29:14.581 INFO:tasks.workunit.client.0.smithi119.stdout:5/167: write d6/f1e [6226222,61983] 0 2024-05-27T09:29:14.582 INFO:tasks.workunit.client.0.smithi119.stdout:5/168: truncate d6/d2f/f3a 56847 0 2024-05-27T09:29:14.582 INFO:tasks.workunit.client.0.smithi119.stdout:8/191: unlink d3/de/f23 0 2024-05-27T09:29:14.582 INFO:tasks.workunit.client.0.smithi119.stdout:5/169: write d6/f1e [371753,122] 0 2024-05-27T09:29:14.582 INFO:tasks.workunit.client.0.smithi119.stdout:5/170: write d6/f17 [891932,121539] 0 2024-05-27T09:29:14.587 INFO:tasks.workunit.client.0.smithi119.stdout:5/171: mkdir d6/dd/d39/d19/d3c 0 2024-05-27T09:29:14.587 INFO:tasks.workunit.client.0.smithi119.stdout:5/172: fsync d6/db/dc/f26 0 2024-05-27T09:29:14.592 INFO:tasks.workunit.client.0.smithi119.stdout:3/318: dwrite db/d36/d45/d4c/d62/d13/d33/d43/f5f [0,4194304] 0 2024-05-27T09:29:14.593 INFO:tasks.workunit.client.0.smithi119.stdout:3/319: write db/d36/d45/d4c/d62/f17 [6511874,5592] 0 2024-05-27T09:29:14.598 INFO:tasks.workunit.client.0.smithi119.stdout:6/148: dread d0/fd [0,4194304] 0 2024-05-27T09:29:14.618 INFO:tasks.workunit.client.0.smithi119.stdout:5/173: dread d6/db/de/f14 [0,4194304] 0 2024-05-27T09:29:14.620 INFO:tasks.workunit.client.0.smithi119.stdout:5/174: symlink d6/db/de/d27/l3d 0 2024-05-27T09:29:14.621 INFO:tasks.workunit.client.0.smithi119.stdout:5/175: fsync d6/db/de/f14 0 2024-05-27T09:29:14.621 INFO:tasks.workunit.client.0.smithi119.stdout:5/176: creat d6/db/de/d27/f3e x:0 0 0 2024-05-27T09:29:14.621 INFO:tasks.workunit.client.0.smithi119.stdout:5/177: chown d6/db/dc/c1f 1018590 1 2024-05-27T09:29:14.621 INFO:tasks.workunit.client.0.smithi119.stdout:5/178: write d6/db/de/f18 [409722,31885] 0 2024-05-27T09:29:14.624 INFO:tasks.workunit.client.0.smithi119.stdout:5/179: rename d6/f17 to d6/db/de/d27/f3f 0 2024-05-27T09:29:14.654 INFO:tasks.workunit.client.0.smithi119.stdout:0/141: dwrite d2/d1a/f21 [0,4194304] 0 2024-05-27T09:29:14.654 INFO:tasks.workunit.client.0.smithi119.stdout:0/142: dread - d2/d6/f2b zero size 2024-05-27T09:29:14.654 INFO:tasks.workunit.client.0.smithi119.stdout:0/143: truncate d2/d6/f2b 435664 0 2024-05-27T09:29:14.658 INFO:tasks.workunit.client.0.smithi119.stdout:3/320: dread f6 [0,4194304] 0 2024-05-27T09:29:14.662 INFO:tasks.workunit.client.0.smithi119.stdout:5/180: dread d6/dd/f1b [0,4194304] 0 2024-05-27T09:29:14.663 INFO:tasks.workunit.client.0.smithi119.stdout:0/144: link d2/le d2/d6/d23/l2e 0 2024-05-27T09:29:14.663 INFO:tasks.workunit.client.0.smithi119.stdout:0/145: readlink d2/le 0 2024-05-27T09:29:14.663 INFO:tasks.workunit.client.0.smithi119.stdout:0/146: write d2/d6/d9/f1c [2130987,3786] 0 2024-05-27T09:29:14.665 INFO:tasks.workunit.client.0.smithi119.stdout:0/147: mkdir d2/d6/d9/d2f 0 2024-05-27T09:29:14.666 INFO:tasks.workunit.client.0.smithi119.stdout:0/148: mkdir d2/d6/d9/d24/d30 0 2024-05-27T09:29:14.667 INFO:tasks.workunit.client.0.smithi119.stdout:0/149: symlink d2/d6/d9/d24/d30/l31 0 2024-05-27T09:29:14.668 INFO:tasks.workunit.client.0.smithi119.stdout:0/150: rename d2/d6/f2b to d2/d6/d9/d12/f32 0 2024-05-27T09:29:14.669 INFO:tasks.workunit.client.0.smithi119.stdout:0/151: chown d2/d6/d9/d12/f1b 88970552 1 2024-05-27T09:29:14.669 INFO:tasks.workunit.client.0.smithi119.stdout:0/152: stat d2/d6/d9 0 2024-05-27T09:29:14.669 INFO:tasks.workunit.client.0.smithi119.stdout:1/219: dwrite d7/db/d12/d15/f1d [4194304,4194304] 0 2024-05-27T09:29:14.672 INFO:tasks.workunit.client.0.smithi119.stdout:6/149: dwrite d0/f6 [4194304,4194304] 0 2024-05-27T09:29:14.672 INFO:tasks.workunit.client.0.smithi119.stdout:0/153: creat d2/d6/d23/f33 x:0 0 0 2024-05-27T09:29:14.674 INFO:tasks.workunit.client.0.smithi119.stdout:0/154: rmdir d2 39 2024-05-27T09:29:14.676 INFO:tasks.workunit.client.0.smithi119.stdout:6/150: rename d0/d2/d1c/d21/l1b to d0/d2/d1c/d21/l32 0 2024-05-27T09:29:14.677 INFO:tasks.workunit.client.0.smithi119.stdout:0/155: mknod d2/d6/d9/d24/d30/c34 0 2024-05-27T09:29:14.678 INFO:tasks.workunit.client.0.smithi119.stdout:6/151: truncate d0/d2/f20 287504 0 2024-05-27T09:29:14.678 INFO:tasks.workunit.client.0.smithi119.stdout:6/152: chown d0/d2/d1c/d21/l1d 462578517 1 2024-05-27T09:29:14.680 INFO:tasks.workunit.client.0.smithi119.stdout:6/153: mknod d0/d2/c33 0 2024-05-27T09:29:14.683 INFO:tasks.workunit.client.0.smithi119.stdout:2/190: sync 2024-05-27T09:29:14.684 INFO:tasks.workunit.client.0.smithi119.stdout:3/321: dread db/d36/d45/d4c/d62/f3f [0,4194304] 0 2024-05-27T09:29:14.691 INFO:tasks.workunit.client.0.smithi119.stdout:3/322: getdents db/d36/d45/d4c 0 2024-05-27T09:29:14.692 INFO:tasks.workunit.client.0.smithi119.stdout:3/323: stat db/f56 0 2024-05-27T09:29:14.694 INFO:tasks.workunit.client.0.smithi119.stdout:3/324: creat db/d36/d45/d4c/d62/f66 x:0 0 0 2024-05-27T09:29:14.694 INFO:tasks.workunit.client.0.smithi119.stdout:3/325: creat db/d36/d45/d4d/f67 x:0 0 0 2024-05-27T09:29:14.706 INFO:tasks.workunit.client.0.smithi119.stdout:1/220: dread d7/db/f16 [0,4194304] 0 2024-05-27T09:29:14.706 INFO:tasks.workunit.client.0.smithi119.stdout:1/221: readlink d7/dd/l2d 0 2024-05-27T09:29:14.707 INFO:tasks.workunit.client.0.smithi119.stdout:1/222: mknod d7/dd/d44/c45 0 2024-05-27T09:29:14.707 INFO:tasks.workunit.client.0.smithi119.stdout:1/223: chown d7/dd/l19 2565 1 2024-05-27T09:29:14.730 INFO:tasks.workunit.client.0.smithi119.stdout:5/181: dwrite d6/f9 [0,4194304] 0 2024-05-27T09:29:14.731 INFO:tasks.workunit.client.0.smithi119.stdout:5/182: chown d6/db/dc 3353206 1 2024-05-27T09:29:14.731 INFO:tasks.workunit.client.0.smithi119.stdout:5/183: dread - d6/db/dc/f3b zero size 2024-05-27T09:29:14.732 INFO:tasks.workunit.client.0.smithi119.stdout:5/184: rmdir d6/dd/d39/d19 39 2024-05-27T09:29:14.734 INFO:tasks.workunit.client.0.smithi119.stdout:5/185: symlink d6/db/dc/l40 0 2024-05-27T09:29:14.734 INFO:tasks.workunit.client.0.smithi119.stdout:5/186: chown d6/db/c35 32514858 1 2024-05-27T09:29:14.736 INFO:tasks.workunit.client.0.smithi119.stdout:5/187: mkdir d6/dd/df/d41 0 2024-05-27T09:29:14.736 INFO:tasks.workunit.client.0.smithi119.stdout:5/188: dread - d6/f22 zero size 2024-05-27T09:29:14.759 INFO:tasks.workunit.client.0.smithi119.stdout:5/189: dread d6/dd/f32 [0,4194304] 0 2024-05-27T09:29:14.761 INFO:tasks.workunit.client.0.smithi119.stdout:5/190: unlink d6/db/de/d27/l2e 0 2024-05-27T09:29:14.763 INFO:tasks.workunit.client.0.smithi119.stdout:5/191: rename d6/db/f31 to d6/d2f/f42 0 2024-05-27T09:29:14.765 INFO:tasks.workunit.client.0.smithi119.stdout:5/192: creat d6/dd/d39/d19/d3c/f43 x:0 0 0 2024-05-27T09:29:14.778 INFO:tasks.workunit.client.0.smithi119.stdout:6/154: dwrite d0/d11/f14 [0,4194304] 0 2024-05-27T09:29:14.783 INFO:tasks.workunit.client.0.smithi119.stdout:6/155: rename d0/d2/d1c/d21/l1d to d0/l34 0 2024-05-27T09:29:14.789 INFO:tasks.workunit.client.0.smithi119.stdout:6/156: rename d0/d2/l29 to d0/d2/d24/l35 0 2024-05-27T09:29:14.789 INFO:tasks.workunit.client.0.smithi119.stdout:2/191: dwrite d2/da/d15/f16 [0,4194304] 0 2024-05-27T09:29:14.800 INFO:tasks.workunit.client.0.smithi119.stdout:0/156: dwrite d2/d6/d9/f2c [0,4194304] 0 2024-05-27T09:29:14.804 INFO:tasks.workunit.client.0.smithi119.stdout:1/224: dwrite d7/db/d12/d15/d1f/f34 [0,4194304] 0 2024-05-27T09:29:14.805 INFO:tasks.workunit.client.0.smithi119.stdout:6/157: dread d0/fd [0,4194304] 0 2024-05-27T09:29:14.809 INFO:tasks.workunit.client.0.smithi119.stdout:2/192: dread d2/da/d15/f16 [0,4194304] 0 2024-05-27T09:29:14.818 INFO:tasks.workunit.client.0.smithi119.stdout:3/326: dwrite db/d36/d45/d4c/d62/d13/d33/d43/f5c [0,4194304] 0 2024-05-27T09:29:14.820 INFO:tasks.workunit.client.0.smithi119.stdout:2/193: rename d2/da/cb to d2/da/d2c/c49 0 2024-05-27T09:29:14.821 INFO:tasks.workunit.client.0.smithi119.stdout:2/194: symlink d2/da/l4a 0 2024-05-27T09:29:14.823 INFO:tasks.workunit.client.0.smithi119.stdout:2/195: rename d2/le to d2/da/d15/d33/l4b 0 2024-05-27T09:29:14.825 INFO:tasks.workunit.client.0.smithi119.stdout:5/193: dwrite d6/db/de/f25 [0,4194304] 0 2024-05-27T09:29:14.825 INFO:tasks.workunit.client.0.smithi119.stdout:5/194: chown d6/ca 949 1 2024-05-27T09:29:14.826 INFO:tasks.workunit.client.0.smithi119.stdout:1/225: dread f6 [0,4194304] 0 2024-05-27T09:29:14.826 INFO:tasks.workunit.client.0.smithi119.stdout:1/226: chown d7/db/d35/f3e 3729 1 2024-05-27T09:29:14.827 INFO:tasks.workunit.client.0.smithi119.stdout:5/195: symlink d6/db/dc/l44 0 2024-05-27T09:29:14.827 INFO:tasks.workunit.client.0.smithi119.stdout:5/196: chown d6/f8 444 1 2024-05-27T09:29:14.831 INFO:tasks.workunit.client.0.smithi119.stdout:1/227: rename d7/dd/l2d to d7/l46 0 2024-05-27T09:29:14.831 INFO:tasks.workunit.client.0.smithi119.stdout:5/197: mknod d6/dd/d39/d19/d3c/c45 0 2024-05-27T09:29:14.837 INFO:tasks.workunit.client.0.smithi119.stdout:5/198: mkdir d6/dd/d46 0 2024-05-27T09:29:14.857 INFO:tasks.workunit.client.0.smithi119.stdout:0/157: dwrite d2/d6/d9/d12/f25 [0,4194304] 0 2024-05-27T09:29:14.858 INFO:tasks.workunit.client.0.smithi119.stdout:0/158: read - d2/d6/d9/d12/f1b zero size 2024-05-27T09:29:14.862 INFO:tasks.workunit.client.0.smithi119.stdout:0/159: mknod d2/c35 0 2024-05-27T09:29:14.862 INFO:tasks.workunit.client.0.smithi119.stdout:0/160: dread - d2/d6/d23/f33 zero size 2024-05-27T09:29:14.872 INFO:tasks.workunit.client.0.smithi119.stdout:0/161: symlink d2/d6/d9/l36 0 2024-05-27T09:29:14.874 INFO:tasks.workunit.client.0.smithi119.stdout:0/162: mkdir d2/d6/d9/d12/d37 0 2024-05-27T09:29:14.876 INFO:tasks.workunit.client.0.smithi119.stdout:0/163: mknod d2/d6/d9/d24/c38 0 2024-05-27T09:29:14.878 INFO:tasks.workunit.client.0.smithi119.stdout:0/164: creat d2/d1a/d20/f39 x:0 0 0 2024-05-27T09:29:14.879 INFO:tasks.workunit.client.0.smithi119.stdout:0/165: truncate d2/d1a/f21 1448971 0 2024-05-27T09:29:14.879 INFO:tasks.workunit.client.0.smithi119.stdout:0/166: readlink d2/l14 0 2024-05-27T09:29:14.879 INFO:tasks.workunit.client.0.smithi119.stdout:0/167: creat d2/d6/d9/f3a x:0 0 0 2024-05-27T09:29:14.886 INFO:tasks.workunit.client.0.smithi119.stdout:6/158: dwrite d0/d11/d26/f17 [4194304,4194304] 0 2024-05-27T09:29:14.889 INFO:tasks.workunit.client.0.smithi119.stdout:6/159: symlink d0/d2/d24/l36 0 2024-05-27T09:29:14.890 INFO:tasks.workunit.client.0.smithi119.stdout:1/228: dwrite d7/db/d12/d26/f43 [0,4194304] 0 2024-05-27T09:29:14.895 INFO:tasks.workunit.client.0.smithi119.stdout:1/229: mkdir d7/dd/d44/d47 0 2024-05-27T09:29:14.895 INFO:tasks.workunit.client.0.smithi119.stdout:1/230: fsync d7/f11 0 2024-05-27T09:29:14.904 INFO:tasks.workunit.client.0.smithi119.stdout:8/192: sync 2024-05-27T09:29:14.904 INFO:tasks.workunit.client.0.smithi119.stdout:7/256: sync 2024-05-27T09:29:14.905 INFO:tasks.workunit.client.0.smithi119.stdout:3/327: dwrite db/fc [0,4194304] 0 2024-05-27T09:29:14.907 INFO:tasks.workunit.client.0.smithi119.stdout:9/152: sync 2024-05-27T09:29:14.907 INFO:tasks.workunit.client.0.smithi119.stdout:4/144: sync 2024-05-27T09:29:14.907 INFO:tasks.workunit.client.0.smithi119.stdout:4/145: stat d4/d6/f2a 0 2024-05-27T09:29:14.907 INFO:tasks.workunit.client.0.smithi119.stdout:9/153: truncate d18/d19/d1b/d29/f2f 72581 0 2024-05-27T09:29:14.907 INFO:tasks.workunit.client.0.smithi119.stdout:9/154: truncate f6 446438 0 2024-05-27T09:29:14.907 INFO:tasks.workunit.client.0.smithi119.stdout:2/196: dwrite d2/da/d15/f29 [0,4194304] 0 2024-05-27T09:29:14.907 INFO:tasks.workunit.client.0.smithi119.stdout:9/155: write d18/f26 [734797,43910] 0 2024-05-27T09:29:14.907 INFO:tasks.workunit.client.0.smithi119.stdout:4/146: write d4/d6/fd [848910,118315] 0 2024-05-27T09:29:14.909 INFO:tasks.workunit.client.0.smithi119.stdout:8/193: link d3/d8/d1b/f28 d3/d39/f3e 0 2024-05-27T09:29:14.911 INFO:tasks.workunit.client.0.smithi119.stdout:9/156: symlink d18/d19/d1b/l31 0 2024-05-27T09:29:14.911 INFO:tasks.workunit.client.0.smithi119.stdout:9/157: creat d18/f32 x:0 0 0 2024-05-27T09:29:14.911 INFO:tasks.workunit.client.0.smithi119.stdout:8/194: creat d3/d24/d30/f3f x:0 0 0 2024-05-27T09:29:14.912 INFO:tasks.workunit.client.0.smithi119.stdout:8/195: write d3/de/f2c [262633,69013] 0 2024-05-27T09:29:14.912 INFO:tasks.workunit.client.0.smithi119.stdout:4/147: creat d4/de/d21/f2d x:0 0 0 2024-05-27T09:29:14.912 INFO:tasks.workunit.client.0.smithi119.stdout:4/148: readlink d4/de/l1b 0 2024-05-27T09:29:14.914 INFO:tasks.workunit.client.0.smithi119.stdout:9/158: truncate d18/d19/d1b/f1e 58149 0 2024-05-27T09:29:14.914 INFO:tasks.workunit.client.0.smithi119.stdout:9/159: chown d18/d19/d1b/d2a 5 1 2024-05-27T09:29:14.914 INFO:tasks.workunit.client.0.smithi119.stdout:4/149: creat d4/de/f2e x:0 0 0 2024-05-27T09:29:14.921 INFO:tasks.workunit.client.0.smithi119.stdout:9/160: link d18/d19/d1b/l2b d18/d19/d1b/d29/l33 0 2024-05-27T09:29:14.921 INFO:tasks.workunit.client.0.smithi119.stdout:4/150: mkdir d4/d6/d28/d2f 0 2024-05-27T09:29:14.921 INFO:tasks.workunit.client.0.smithi119.stdout:4/151: chown d4/d6/f11 1 1 2024-05-27T09:29:14.922 INFO:tasks.workunit.client.0.smithi119.stdout:1/231: dread d7/db/d12/d26/f43 [0,4194304] 0 2024-05-27T09:29:14.922 INFO:tasks.workunit.client.0.smithi119.stdout:1/232: readlink l4 0 2024-05-27T09:29:14.926 INFO:tasks.workunit.client.0.smithi119.stdout:1/233: symlink d7/db/d35/l48 0 2024-05-27T09:29:14.927 INFO:tasks.workunit.client.0.smithi119.stdout:4/152: getdents d4 0 2024-05-27T09:29:14.928 INFO:tasks.workunit.client.0.smithi119.stdout:4/153: write d4/f8 [3723987,118714] 0 2024-05-27T09:29:14.928 INFO:tasks.workunit.client.0.smithi119.stdout:2/197: read d2/da/d2c/f3c [2757992,130543] 0 2024-05-27T09:29:14.928 INFO:tasks.workunit.client.0.smithi119.stdout:2/198: dread - d2/d1a/f27 zero size 2024-05-27T09:29:14.933 INFO:tasks.workunit.client.0.smithi119.stdout:4/154: link d4/d1c/l2c d4/d23/d24/l30 0 2024-05-27T09:29:14.988 INFO:tasks.workunit.client.0.smithi119.stdout:3/328: dwrite db/fc [0,4194304] 0 2024-05-27T09:29:14.988 INFO:tasks.workunit.client.0.smithi119.stdout:3/329: write db/d5b/f63 [351895,34448] 0 2024-05-27T09:29:14.996 INFO:tasks.workunit.client.0.smithi119.stdout:1/234: write d7/db/f16 [131332,48041] 0 2024-05-27T09:29:14.997 INFO:tasks.workunit.client.0.smithi119.stdout:7/257: dwrite d7/df/d36/f49 [0,4194304] 0 2024-05-27T09:29:14.997 INFO:tasks.workunit.client.0.smithi119.stdout:3/330: read db/d36/d45/d4d/d64/f5a [715257,11300] 0 2024-05-27T09:29:15.004 INFO:tasks.workunit.client.0.smithi119.stdout:7/258: rename d7/df/d19/d1d/f2a to d7/db/f57 0 2024-05-27T09:29:15.004 INFO:tasks.workunit.client.0.smithi119.stdout:3/331: truncate db/d36/d45/d4d/d64/f3e 239830 0 2024-05-27T09:29:15.004 INFO:tasks.workunit.client.0.smithi119.stdout:7/259: getdents d7/df/d36/d42 0 2024-05-27T09:29:15.008 INFO:tasks.workunit.client.0.smithi119.stdout:3/332: mkdir db/d36/d45/d4c/d62/d13/d68 0 2024-05-27T09:29:15.020 INFO:tasks.workunit.client.0.smithi119.stdout:9/161: dwrite f2 [0,4194304] 0 2024-05-27T09:29:15.023 INFO:tasks.workunit.client.0.smithi119.stdout:9/162: mknod d18/d19/d1b/d2a/c34 0 2024-05-27T09:29:15.023 INFO:tasks.workunit.client.0.smithi119.stdout:9/163: stat d18/d19/d1b/d29/f2c 0 2024-05-27T09:29:15.024 INFO:tasks.workunit.client.0.smithi119.stdout:9/164: link d18/f26 d18/d19/f35 0 2024-05-27T09:29:15.044 INFO:tasks.workunit.client.0.smithi119.stdout:7/260: write f4 [910971,42004] 0 2024-05-27T09:29:15.044 INFO:tasks.workunit.client.0.smithi119.stdout:4/155: dwrite d4/d6/f1a [4194304,4194304] 0 2024-05-27T09:29:15.072 INFO:tasks.workunit.client.0.smithi119.stdout:4/156: dread d4/d1c/f1e [0,4194304] 0 2024-05-27T09:29:15.075 INFO:tasks.workunit.client.0.smithi119.stdout:7/261: dread d7/df/d36/f3f [0,4194304] 0 2024-05-27T09:29:15.075 INFO:tasks.workunit.client.0.smithi119.stdout:1/235: dwrite d7/db/f16 [0,4194304] 0 2024-05-27T09:29:15.075 INFO:tasks.workunit.client.0.smithi119.stdout:1/236: chown d7/db/d12 217 1 2024-05-27T09:29:15.075 INFO:tasks.workunit.client.0.smithi119.stdout:1/237: stat d7/dd/d44/d47 0 2024-05-27T09:29:15.077 INFO:tasks.workunit.client.0.smithi119.stdout:4/157: read d4/d6/fd [499754,107241] 0 2024-05-27T09:29:15.077 INFO:tasks.workunit.client.0.smithi119.stdout:4/158: read d4/d6/fd [896147,96628] 0 2024-05-27T09:29:15.077 INFO:tasks.workunit.client.0.smithi119.stdout:4/159: readlink d4/d6/l15 0 2024-05-27T09:29:15.077 INFO:tasks.workunit.client.0.smithi119.stdout:4/160: chown d4/de/lf 74465 1 2024-05-27T09:29:15.081 INFO:tasks.workunit.client.0.smithi119.stdout:7/262: symlink d7/l58 0 2024-05-27T09:29:15.081 INFO:tasks.workunit.client.0.smithi119.stdout:7/263: dread - d7/d1a/d40/f48 zero size 2024-05-27T09:29:15.081 INFO:tasks.workunit.client.0.smithi119.stdout:7/264: creat d7/d1a/f59 x:0 0 0 2024-05-27T09:29:15.082 INFO:tasks.workunit.client.0.smithi119.stdout:4/161: getdents d4/d23/d24 0 2024-05-27T09:29:15.082 INFO:tasks.workunit.client.0.smithi119.stdout:9/165: dwrite f6 [0,4194304] 0 2024-05-27T09:29:15.084 INFO:tasks.workunit.client.0.smithi119.stdout:7/265: truncate d7/d1a/f4a 1094777 0 2024-05-27T09:29:15.084 INFO:tasks.workunit.client.0.smithi119.stdout:3/333: dwrite db/d36/d45/d4d/d64/f5a [0,4194304] 0 2024-05-27T09:29:15.084 INFO:tasks.workunit.client.0.smithi119.stdout:7/266: fsync d7/d1a/f2b 0 2024-05-27T09:29:15.084 INFO:tasks.workunit.client.0.smithi119.stdout:7/267: readlink d7/l58 0 2024-05-27T09:29:15.085 INFO:tasks.workunit.client.0.smithi119.stdout:7/268: fsync d7/df/f29 0 2024-05-27T09:29:15.085 INFO:tasks.workunit.client.0.smithi119.stdout:4/162: mkdir d4/d23/d31 0 2024-05-27T09:29:15.088 INFO:tasks.workunit.client.0.smithi119.stdout:3/334: creat db/f69 x:0 0 0 2024-05-27T09:29:15.088 INFO:tasks.workunit.client.0.smithi119.stdout:3/335: creat db/d36/d45/d4c/d62/f6a x:0 0 0 2024-05-27T09:29:15.088 INFO:tasks.workunit.client.0.smithi119.stdout:7/269: creat d7/df/d36/d45/f5a x:0 0 0 2024-05-27T09:29:15.089 INFO:tasks.workunit.client.0.smithi119.stdout:7/270: stat d7/d1a/f1c 0 2024-05-27T09:29:15.092 INFO:tasks.workunit.client.0.smithi119.stdout:7/271: write d7/d1a/f2d [108605,66851] 0 2024-05-27T09:29:15.092 INFO:tasks.workunit.client.0.smithi119.stdout:4/163: creat d4/f32 x:0 0 0 2024-05-27T09:29:15.092 INFO:tasks.workunit.client.0.smithi119.stdout:1/238: write d7/dd/f13 [4046302,75294] 0 2024-05-27T09:29:15.093 INFO:tasks.workunit.client.0.smithi119.stdout:3/336: rename db/d36/d45/d4c/d62/d13/d33/d43/f47 to db/d36/d45/d4d/f6b 0 2024-05-27T09:29:15.095 INFO:tasks.workunit.client.0.smithi119.stdout:3/337: creat db/d36/d45/d4c/f6c x:0 0 0 2024-05-27T09:29:15.097 INFO:tasks.workunit.client.0.smithi119.stdout:3/338: mknod db/d36/d45/d4c/d62/c6d 0 2024-05-27T09:29:15.097 INFO:tasks.workunit.client.0.smithi119.stdout:9/166: dread f10 [0,4194304] 0 2024-05-27T09:29:15.097 INFO:tasks.workunit.client.0.smithi119.stdout:9/167: write d18/f26 [1630393,126848] 0 2024-05-27T09:29:15.099 INFO:tasks.workunit.client.0.smithi119.stdout:6/160: sync 2024-05-27T09:29:15.100 INFO:tasks.workunit.client.0.smithi119.stdout:8/196: sync 2024-05-27T09:29:15.100 INFO:tasks.workunit.client.0.smithi119.stdout:5/199: sync 2024-05-27T09:29:15.101 INFO:tasks.workunit.client.0.smithi119.stdout:7/272: dread d7/df/d19/d4d/f4e [0,4194304] 0 2024-05-27T09:29:15.101 INFO:tasks.workunit.client.0.smithi119.stdout:2/199: sync 2024-05-27T09:29:15.101 INFO:tasks.workunit.client.0.smithi119.stdout:0/168: sync 2024-05-27T09:29:15.101 INFO:tasks.workunit.client.0.smithi119.stdout:0/169: fdatasync d2/d6/d9/ff 0 2024-05-27T09:29:15.102 INFO:tasks.workunit.client.0.smithi119.stdout:2/200: write d2/f2d [955264,24293] 0 2024-05-27T09:29:15.105 INFO:tasks.workunit.client.0.smithi119.stdout:8/197: symlink d3/de/l40 0 2024-05-27T09:29:15.107 INFO:tasks.workunit.client.0.smithi119.stdout:0/170: mknod d2/d6/d9/d12/c3b 0 2024-05-27T09:29:15.107 INFO:tasks.workunit.client.0.smithi119.stdout:0/171: stat d2/l14 0 2024-05-27T09:29:15.107 INFO:tasks.workunit.client.0.smithi119.stdout:0/172: fsync d2/d6/d9/d12/f25 0 2024-05-27T09:29:15.109 INFO:tasks.workunit.client.0.smithi119.stdout:7/273: mknod d7/c5b 0 2024-05-27T09:29:15.109 INFO:tasks.workunit.client.0.smithi119.stdout:7/274: write d7/fa [2635448,78276] 0 2024-05-27T09:29:15.109 INFO:tasks.workunit.client.0.smithi119.stdout:7/275: creat d7/db/f5c x:0 0 0 2024-05-27T09:29:15.111 INFO:tasks.workunit.client.0.smithi119.stdout:8/198: rename d3/d24/d30/f3f to d3/d8/d1b/f41 0 2024-05-27T09:29:15.112 INFO:tasks.workunit.client.0.smithi119.stdout:8/199: write d3/d24/f29 [1037847,49547] 0 2024-05-27T09:29:15.115 INFO:tasks.workunit.client.0.smithi119.stdout:7/276: creat d7/d1a/d40/d44/f5d x:0 0 0 2024-05-27T09:29:15.115 INFO:tasks.workunit.client.0.smithi119.stdout:7/277: dread - d7/df/d19/d1d/f4f zero size 2024-05-27T09:29:15.120 INFO:tasks.workunit.client.0.smithi119.stdout:0/173: rename d2/d1a/f2a to d2/d6/f3c 0 2024-05-27T09:29:15.120 INFO:tasks.workunit.client.0.smithi119.stdout:8/200: symlink d3/de/l42 0 2024-05-27T09:29:15.129 INFO:tasks.workunit.client.0.smithi119.stdout:7/278: creat d7/db/d37/f5e x:0 0 0 2024-05-27T09:29:15.129 INFO:tasks.workunit.client.0.smithi119.stdout:8/201: creat d3/d24/f43 x:0 0 0 2024-05-27T09:29:15.138 INFO:tasks.workunit.client.0.smithi119.stdout:8/202: mknod d3/c44 0 2024-05-27T09:29:15.141 INFO:tasks.workunit.client.0.smithi119.stdout:8/203: rename d3/d24/c3c to d3/d24/c45 0 2024-05-27T09:29:15.141 INFO:tasks.workunit.client.0.smithi119.stdout:8/204: chown d3/d8/d1b 6388706 1 2024-05-27T09:29:15.163 INFO:tasks.workunit.client.0.smithi119.stdout:8/205: dread d3/de/f1c [4194304,4194304] 0 2024-05-27T09:29:15.177 INFO:tasks.workunit.client.0.smithi119.stdout:9/168: dread d18/f23 [0,4194304] 0 2024-05-27T09:29:15.177 INFO:tasks.workunit.client.0.smithi119.stdout:9/169: creat d18/d19/f36 x:0 0 0 2024-05-27T09:29:15.177 INFO:tasks.workunit.client.0.smithi119.stdout:3/339: dwrite db/d36/d45/d4d/f6b [0,4194304] 0 2024-05-27T09:29:15.177 INFO:tasks.workunit.client.0.smithi119.stdout:9/170: creat d18/d19/d1b/d29/f37 x:0 0 0 2024-05-27T09:29:15.177 INFO:tasks.workunit.client.0.smithi119.stdout:3/340: fsync db/f2f 0 2024-05-27T09:29:15.178 INFO:tasks.workunit.client.0.smithi119.stdout:3/341: readlink db/l4b 0 2024-05-27T09:29:15.178 INFO:tasks.workunit.client.0.smithi119.stdout:3/342: dread - db/d36/d45/d49/d4f/f60 zero size 2024-05-27T09:29:15.179 INFO:tasks.workunit.client.0.smithi119.stdout:5/200: write d6/db/de/f25 [1676507,672] 0 2024-05-27T09:29:15.183 INFO:tasks.workunit.client.0.smithi119.stdout:7/279: rmdir d7 39 2024-05-27T09:29:15.188 INFO:tasks.workunit.client.0.smithi119.stdout:9/171: rename d18/f26 to d18/d19/d1b/f38 0 2024-05-27T09:29:15.188 INFO:tasks.workunit.client.0.smithi119.stdout:3/343: mknod db/d36/d45/d4c/d62/c6e 0 2024-05-27T09:29:15.189 INFO:tasks.workunit.client.0.smithi119.stdout:5/201: mkdir d6/dd/d46/d47 0 2024-05-27T09:29:15.191 INFO:tasks.workunit.client.0.smithi119.stdout:3/344: mkdir db/d6f 0 2024-05-27T09:29:15.194 INFO:tasks.workunit.client.0.smithi119.stdout:2/201: dwrite d2/d1a/f46 [0,4194304] 0 2024-05-27T09:29:15.203 INFO:tasks.workunit.client.0.smithi119.stdout:3/345: rename db/d36/d45/d4c/d62/f6a to db/d36/d45/d4d/f70 0 2024-05-27T09:29:15.222 INFO:tasks.workunit.client.0.smithi119.stdout:3/346: creat db/d36/f71 x:0 0 0 2024-05-27T09:29:15.222 INFO:tasks.workunit.client.0.smithi119.stdout:9/172: dread fc [0,4194304] 0 2024-05-27T09:29:15.227 INFO:tasks.workunit.client.0.smithi119.stdout:9/173: write f10 [703700,128917] 0 2024-05-27T09:29:15.227 INFO:tasks.workunit.client.0.smithi119.stdout:9/174: truncate d18/d19/f35 2527589 0 2024-05-27T09:29:15.227 INFO:tasks.workunit.client.0.smithi119.stdout:6/161: dwrite d0/d2/f31 [0,4194304] 0 2024-05-27T09:29:15.227 INFO:tasks.workunit.client.0.smithi119.stdout:4/164: dwrite d4/f22 [4194304,4194304] 0 2024-05-27T09:29:15.227 INFO:tasks.workunit.client.0.smithi119.stdout:2/202: stat d2/da/c3f 0 2024-05-27T09:29:15.227 INFO:tasks.workunit.client.0.smithi119.stdout:3/347: creat db/d36/d45/d4c/d62/d13/d68/f72 x:0 0 0 2024-05-27T09:29:15.227 INFO:tasks.workunit.client.0.smithi119.stdout:1/239: dwrite d7/db/d35/f3e [0,4194304] 0 2024-05-27T09:29:15.227 INFO:tasks.workunit.client.0.smithi119.stdout:1/240: stat d7/l1c 0 2024-05-27T09:29:15.228 INFO:tasks.workunit.client.0.smithi119.stdout:1/241: write d7/f14 [1603701,58118] 0 2024-05-27T09:29:15.229 INFO:tasks.workunit.client.0.smithi119.stdout:6/162: mkdir d0/d2/d24/d37 0 2024-05-27T09:29:15.230 INFO:tasks.workunit.client.0.smithi119.stdout:3/348: creat db/d6f/f73 x:0 0 0 2024-05-27T09:29:15.233 INFO:tasks.workunit.client.0.smithi119.stdout:8/206: dwrite d3/d24/d30/f36 [0,4194304] 0 2024-05-27T09:29:15.239 INFO:tasks.workunit.client.0.smithi119.stdout:1/242: mkdir d7/dd/d25/d29/d38/d49 0 2024-05-27T09:29:15.241 INFO:tasks.workunit.client.0.smithi119.stdout:3/349: creat db/d36/d45/d4d/d64/f74 x:0 0 0 2024-05-27T09:29:15.248 INFO:tasks.workunit.client.0.smithi119.stdout:1/243: mkdir d7/db/d12/d26/d4a 0 2024-05-27T09:29:15.248 INFO:tasks.workunit.client.0.smithi119.stdout:8/207: chown d3/f12 11 1 2024-05-27T09:29:15.260 INFO:tasks.workunit.client.0.smithi119.stdout:1/244: mknod d7/dd/d25/d29/d38/d49/c4b 0 2024-05-27T09:29:15.260 INFO:tasks.workunit.client.0.smithi119.stdout:1/245: creat d7/db/d12/f4c x:0 0 0 2024-05-27T09:29:15.267 INFO:tasks.workunit.client.0.smithi119.stdout:3/350: rename db/d36/d45/d4c/d62/d13/d33/c3c to db/d36/d45/d4d/d64/c75 0 2024-05-27T09:29:15.267 INFO:tasks.workunit.client.0.smithi119.stdout:7/280: dwrite d7/df/d36/f3f [0,4194304] 0 2024-05-27T09:29:15.273 INFO:tasks.workunit.client.0.smithi119.stdout:2/203: dread d2/d1a/f46 [0,4194304] 0 2024-05-27T09:29:15.279 INFO:tasks.workunit.client.0.smithi119.stdout:9/175: dread f6 [0,4194304] 0 2024-05-27T09:29:15.279 INFO:tasks.workunit.client.0.smithi119.stdout:9/176: write fa [286246,35148] 0 2024-05-27T09:29:15.279 INFO:tasks.workunit.client.0.smithi119.stdout:9/177: chown c11 13 1 2024-05-27T09:29:15.281 INFO:tasks.workunit.client.0.smithi119.stdout:9/178: symlink d18/d19/l39 0 2024-05-27T09:29:15.281 INFO:tasks.workunit.client.0.smithi119.stdout:9/179: chown l14 1043373994 1 2024-05-27T09:29:15.288 INFO:tasks.workunit.client.0.smithi119.stdout:1/246: write d7/f3d [3805878,72850] 0 2024-05-27T09:29:15.288 INFO:tasks.workunit.client.0.smithi119.stdout:1/247: write d7/db/d12/d15/f21 [1297667,40027] 0 2024-05-27T09:29:15.295 INFO:tasks.workunit.client.0.smithi119.stdout:3/351: write db/d36/d45/d4c/d62/d13/f2a [1708644,111748] 0 2024-05-27T09:29:15.297 INFO:tasks.workunit.client.0.smithi119.stdout:3/352: mkdir db/d36/d45/d4c/d62/d76 0 2024-05-27T09:29:15.297 INFO:tasks.workunit.client.0.smithi119.stdout:3/353: truncate db/d36/f71 428215 0 2024-05-27T09:29:15.301 INFO:tasks.workunit.client.0.smithi119.stdout:3/354: symlink db/l77 0 2024-05-27T09:29:15.301 INFO:tasks.workunit.client.0.smithi119.stdout:3/355: chown db/f24 293497 1 2024-05-27T09:29:15.301 INFO:tasks.workunit.client.0.smithi119.stdout:3/356: truncate db/d36/d45/d4c/d62/f3a 1376247 0 2024-05-27T09:29:15.309 INFO:tasks.workunit.client.0.smithi119.stdout:5/202: dwrite d6/db/de/f25 [4194304,4194304] 0 2024-05-27T09:29:15.309 INFO:tasks.workunit.client.0.smithi119.stdout:5/203: write f0 [445045,15359] 0 2024-05-27T09:29:15.309 INFO:tasks.workunit.client.0.smithi119.stdout:5/204: dread - d6/db/f36 zero size 2024-05-27T09:29:15.309 INFO:tasks.workunit.client.0.smithi119.stdout:5/205: dread - d6/db/dc/f3b zero size 2024-05-27T09:29:15.312 INFO:tasks.workunit.client.0.smithi119.stdout:5/206: link d6/db/dc/l40 d6/dd/d46/d47/l48 0 2024-05-27T09:29:15.313 INFO:tasks.workunit.client.0.smithi119.stdout:5/207: symlink d6/dd/df/d41/l49 0 2024-05-27T09:29:15.317 INFO:tasks.workunit.client.0.smithi119.stdout:2/204: dread d2/da/f10 [0,4194304] 0 2024-05-27T09:29:15.317 INFO:tasks.workunit.client.0.smithi119.stdout:2/205: readlink d2/da/d15/d33/l34 0 2024-05-27T09:29:15.318 INFO:tasks.workunit.client.0.smithi119.stdout:2/206: write d2/f19 [1432786,86268] 0 2024-05-27T09:29:15.318 INFO:tasks.workunit.client.0.smithi119.stdout:2/207: write d2/f47 [658143,17798] 0 2024-05-27T09:29:15.325 INFO:tasks.workunit.client.0.smithi119.stdout:3/357: write db/d36/d45/d4c/d62/d13/f2c [1472411,108482] 0 2024-05-27T09:29:15.327 INFO:tasks.workunit.client.0.smithi119.stdout:6/163: dwrite d0/fd [0,4194304] 0 2024-05-27T09:29:15.334 INFO:tasks.workunit.client.0.smithi119.stdout:9/180: rmdir d18/d19 39 2024-05-27T09:29:15.335 INFO:tasks.workunit.client.0.smithi119.stdout:8/208: dwrite d3/d24/f2b [0,4194304] 0 2024-05-27T09:29:15.336 INFO:tasks.workunit.client.0.smithi119.stdout:4/165: dwrite d4/d6/fa [0,4194304] 0 2024-05-27T09:29:15.340 INFO:tasks.workunit.client.0.smithi119.stdout:6/164: rename d0/d2/d24/l36 to d0/d2/d24/d37/l38 0 2024-05-27T09:29:15.340 INFO:tasks.workunit.client.0.smithi119.stdout:3/358: unlink db/d5b/l65 0 2024-05-27T09:29:15.340 INFO:tasks.workunit.client.0.smithi119.stdout:3/359: creat db/d36/d45/d49/d4f/f78 x:0 0 0 2024-05-27T09:29:15.341 INFO:tasks.workunit.client.0.smithi119.stdout:6/165: write d0/d2/f20 [38105,54178] 0 2024-05-27T09:29:15.345 INFO:tasks.workunit.client.0.smithi119.stdout:1/248: dread d7/db/d12/d15/f21 [0,4194304] 0 2024-05-27T09:29:15.351 INFO:tasks.workunit.client.0.smithi119.stdout:7/281: dwrite d7/fa [0,4194304] 0 2024-05-27T09:29:15.351 INFO:tasks.workunit.client.0.smithi119.stdout:2/208: dread d2/d1a/f21 [0,4194304] 0 2024-05-27T09:29:15.351 INFO:tasks.workunit.client.0.smithi119.stdout:4/166: creat d4/d23/d31/f33 x:0 0 0 2024-05-27T09:29:15.351 INFO:tasks.workunit.client.0.smithi119.stdout:4/167: dread - d4/d23/d31/f33 zero size 2024-05-27T09:29:15.351 INFO:tasks.workunit.client.0.smithi119.stdout:4/168: write d4/d23/f26 [594999,76398] 0 2024-05-27T09:29:15.351 INFO:tasks.workunit.client.0.smithi119.stdout:6/166: mknod d0/d19/c39 0 2024-05-27T09:29:15.351 INFO:tasks.workunit.client.0.smithi119.stdout:6/167: chown d0/d11/d26/f17 604 1 2024-05-27T09:29:15.351 INFO:tasks.workunit.client.0.smithi119.stdout:1/249: mknod d7/db/d12/d26/d4a/c4d 0 2024-05-27T09:29:15.351 INFO:tasks.workunit.client.0.smithi119.stdout:9/181: rename f8 to d18/d19/d1b/d29/f3a 0 2024-05-27T09:29:15.357 INFO:tasks.workunit.client.0.smithi119.stdout:2/209: truncate d2/f26 4374322 0 2024-05-27T09:29:15.362 INFO:tasks.workunit.client.0.smithi119.stdout:7/282: mkdir d7/d5f 0 2024-05-27T09:29:15.362 INFO:tasks.workunit.client.0.smithi119.stdout:4/169: truncate d4/f8 3146548 0 2024-05-27T09:29:15.362 INFO:tasks.workunit.client.0.smithi119.stdout:4/170: stat d4/d6 0 2024-05-27T09:29:15.375 INFO:tasks.workunit.client.0.smithi119.stdout:1/250: dread f6 [0,4194304] 0 2024-05-27T09:29:15.375 INFO:tasks.workunit.client.0.smithi119.stdout:1/251: truncate d7/f8 4221817 0 2024-05-27T09:29:15.377 INFO:tasks.workunit.client.0.smithi119.stdout:1/252: truncate d7/dd/d25/f30 2841110 0 2024-05-27T09:29:15.377 INFO:tasks.workunit.client.0.smithi119.stdout:1/253: chown d7/dd/d25/d29/d38/d3c/l41 3 1 2024-05-27T09:29:15.379 INFO:tasks.workunit.client.0.smithi119.stdout:1/254: unlink d7/c9 0 2024-05-27T09:29:15.381 INFO:tasks.workunit.client.0.smithi119.stdout:1/255: creat d7/dd/d44/f4e x:0 0 0 2024-05-27T09:29:15.395 INFO:tasks.workunit.client.0.smithi119.stdout:6/168: dread d0/f6 [4194304,4194304] 0 2024-05-27T09:29:15.411 INFO:tasks.workunit.client.0.smithi119.stdout:5/208: dwrite d6/db/de/d27/f3f [0,4194304] 0 2024-05-27T09:29:15.411 INFO:tasks.workunit.client.0.smithi119.stdout:5/209: stat d6/db/dc/f3b 0 2024-05-27T09:29:15.411 INFO:tasks.workunit.client.0.smithi119.stdout:5/210: truncate d6/db/dc/f3b 784967 0 2024-05-27T09:29:15.411 INFO:tasks.workunit.client.0.smithi119.stdout:5/211: chown d6/db/dc/c1f 445 1 2024-05-27T09:29:15.411 INFO:tasks.workunit.client.0.smithi119.stdout:5/212: truncate d6/f22 576488 0 2024-05-27T09:29:15.420 INFO:tasks.workunit.client.0.smithi119.stdout:5/213: mknod d6/dd/df/d41/c4a 0 2024-05-27T09:29:15.421 INFO:tasks.workunit.client.0.smithi119.stdout:1/256: dread d7/f1e [0,4194304] 0 2024-05-27T09:29:15.425 INFO:tasks.workunit.client.0.smithi119.stdout:5/214: mkdir d6/dd/d46/d47/d4b 0 2024-05-27T09:29:15.427 INFO:tasks.workunit.client.0.smithi119.stdout:5/215: creat d6/db/f4c x:0 0 0 2024-05-27T09:29:15.437 INFO:tasks.workunit.client.0.smithi119.stdout:0/174: sync 2024-05-27T09:29:15.438 INFO:tasks.workunit.client.0.smithi119.stdout:7/283: dread d7/df/d36/f3f [0,4194304] 0 2024-05-27T09:29:15.441 INFO:tasks.workunit.client.0.smithi119.stdout:2/210: dwrite d2/da/d15/f18 [0,4194304] 0 2024-05-27T09:29:15.441 INFO:tasks.workunit.client.0.smithi119.stdout:2/211: chown d2/d1a/f46 404521 1 2024-05-27T09:29:15.444 INFO:tasks.workunit.client.0.smithi119.stdout:9/182: dwrite d18/f30 [4194304,4194304] 0 2024-05-27T09:29:15.444 INFO:tasks.workunit.client.0.smithi119.stdout:0/175: dread d2/d6/d9/d12/f32 [0,4194304] 0 2024-05-27T09:29:15.444 INFO:tasks.workunit.client.0.smithi119.stdout:0/176: fsync d2/d1a/f21 0 2024-05-27T09:29:15.444 INFO:tasks.workunit.client.0.smithi119.stdout:0/177: chown d2/d1a/d20/f39 32695 1 2024-05-27T09:29:15.445 INFO:tasks.workunit.client.0.smithi119.stdout:8/209: dwrite d3/f16 [0,4194304] 0 2024-05-27T09:29:15.446 INFO:tasks.workunit.client.0.smithi119.stdout:9/183: mknod d18/d19/d1b/d2a/c3b 0 2024-05-27T09:29:15.449 INFO:tasks.workunit.client.0.smithi119.stdout:1/257: getdents d7 0 2024-05-27T09:29:15.449 INFO:tasks.workunit.client.0.smithi119.stdout:3/360: dwrite db/d36/d45/d49/d4f/f60 [0,4194304] 0 2024-05-27T09:29:15.449 INFO:tasks.workunit.client.0.smithi119.stdout:3/361: readlink db/d36/d45/d4c/d62/d13/l1f 0 2024-05-27T09:29:15.455 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:15 smithi119 ceph-mon[29082]: pgmap v5: 65 pgs: 65 active+clean; 473 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:29:15.455 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:15 smithi119 ceph-mon[29082]: mgrmap e24: smithi138.dxsuqf(active, since 4s) 2024-05-27T09:29:15.456 INFO:tasks.workunit.client.0.smithi119.stdout:9/184: mknod d18/d19/d1b/c3c 0 2024-05-27T09:29:15.456 INFO:tasks.workunit.client.0.smithi119.stdout:9/185: readlink d18/d19/d1b/l2b 0 2024-05-27T09:29:15.456 INFO:tasks.workunit.client.0.smithi119.stdout:9/186: write d18/d19/d1b/f1f [215067,42471] 0 2024-05-27T09:29:15.456 INFO:tasks.workunit.client.0.smithi119.stdout:9/187: chown d18/d19 349928555 1 2024-05-27T09:29:15.457 INFO:tasks.workunit.client.0.smithi119.stdout:1/258: mknod d7/db/c4f 0 2024-05-27T09:29:15.457 INFO:tasks.workunit.client.0.smithi119.stdout:1/259: fsync f2 0 2024-05-27T09:29:15.459 INFO:tasks.workunit.client.0.smithi119.stdout:4/171: dwrite d4/d23/d31/f33 [0,4194304] 0 2024-05-27T09:29:15.459 INFO:tasks.workunit.client.0.smithi119.stdout:4/172: getdents d4/d6/d28/d2f 0 2024-05-27T09:29:15.461 INFO:tasks.workunit.client.0.smithi119.stdout:5/216: getdents d6/db 0 2024-05-27T09:29:15.461 INFO:tasks.workunit.client.0.smithi119.stdout:5/217: readlink d6/dd/d46/d47/l48 0 2024-05-27T09:29:15.461 INFO:tasks.workunit.client.0.smithi119.stdout:5/218: chown d6/db/dc/c1f 75 1 2024-05-27T09:29:15.462 INFO:tasks.workunit.client.0.smithi119.stdout:6/169: dread d0/d2/f20 [0,4194304] 0 2024-05-27T09:29:15.469 INFO:tasks.workunit.client.0.smithi119.stdout:3/362: creat db/d36/d45/d4c/d62/d76/f79 x:0 0 0 2024-05-27T09:29:15.469 INFO:tasks.workunit.client.0.smithi119.stdout:3/363: readlink db/d36/d45/d4c/d62/d13/d33/l34 0 2024-05-27T09:29:15.469 INFO:tasks.workunit.client.0.smithi119.stdout:9/188: unlink d18/f22 0 2024-05-27T09:29:15.472 INFO:tasks.workunit.client.0.smithi119.stdout:2/212: truncate d2/da/d15/f18 3536174 0 2024-05-27T09:29:15.472 INFO:tasks.workunit.client.0.smithi119.stdout:1/260: mknod d7/dd/d25/d29/d38/d3c/c50 0 2024-05-27T09:29:15.472 INFO:tasks.workunit.client.0.smithi119.stdout:1/261: readlink d7/l46 0 2024-05-27T09:29:15.472 INFO:tasks.workunit.client.0.smithi119.stdout:5/219: unlink d6/dd/df/c16 0 2024-05-27T09:29:15.476 INFO:tasks.workunit.client.0.smithi119.stdout:1/262: rmdir d7/dd/d25/d29/d38/d49 39 2024-05-27T09:29:15.476 INFO:tasks.workunit.client.0.smithi119.stdout:8/210: rename d3/d24 to d3/d8/d1b/d46 0 2024-05-27T09:29:15.477 INFO:tasks.workunit.client.0.smithi119.stdout:8/211: fsync d3/d39/f3e 0 2024-05-27T09:29:15.478 INFO:tasks.workunit.client.0.smithi119.stdout:1/263: mknod d7/dd/d44/c51 0 2024-05-27T09:29:15.478 INFO:tasks.workunit.client.0.smithi119.stdout:1/264: truncate d7/dd/d44/f4e 966133 0 2024-05-27T09:29:15.480 INFO:tasks.workunit.client.0.smithi119.stdout:8/212: rename d3/d8/d1b/f2a to d3/d8/f47 0 2024-05-27T09:29:15.480 INFO:tasks.workunit.client.0.smithi119.stdout:8/213: chown d3/d8/d1b/d46/d30 5 1 2024-05-27T09:29:15.480 INFO:tasks.workunit.client.0.smithi119.stdout:8/214: write d3/d8/d1b/d46/f29 [1597534,113107] 0 2024-05-27T09:29:15.487 INFO:tasks.workunit.client.0.smithi119.stdout:1/265: unlink d7/dd/d25/d29/c42 0 2024-05-27T09:29:15.490 INFO:tasks.workunit.client.0.smithi119.stdout:8/215: mknod d3/d39/c48 0 2024-05-27T09:29:15.491 INFO:tasks.workunit.client.0.smithi119.stdout:0/178: write d2/d6/f3c [2879492,37703] 0 2024-05-27T09:29:15.491 INFO:tasks.workunit.client.0.smithi119.stdout:0/179: write d2/d6/f3c [4412372,113083] 0 2024-05-27T09:29:15.493 INFO:tasks.workunit.client.0.smithi119.stdout:4/173: rename d4/f8 to d4/d6/d28/f34 0 2024-05-27T09:29:15.493 INFO:tasks.workunit.client.0.smithi119.stdout:4/174: dread - d4/de/d21/f2d zero size 2024-05-27T09:29:15.496 INFO:tasks.workunit.client.0.smithi119.stdout:6/170: dread d0/d11/f14 [0,4194304] 0 2024-05-27T09:29:15.497 INFO:tasks.workunit.client.0.smithi119.stdout:1/266: mkdir d7/db/d52 0 2024-05-27T09:29:15.500 INFO:tasks.workunit.client.0.smithi119.stdout:0/180: getdents d2/d6/d9/d24/d30 0 2024-05-27T09:29:15.508 INFO:tasks.workunit.client.0.smithi119.stdout:6/171: unlink d0/cf 0 2024-05-27T09:29:15.511 INFO:tasks.workunit.client.0.smithi119.stdout:1/267: creat d7/dd/d25/d29/d38/d3c/f53 x:0 0 0 2024-05-27T09:29:15.516 INFO:tasks.workunit.client.0.smithi119.stdout:9/189: dread f6 [0,4194304] 0 2024-05-27T09:29:15.516 INFO:tasks.workunit.client.0.smithi119.stdout:0/181: creat d2/f3d x:0 0 0 2024-05-27T09:29:15.516 INFO:tasks.workunit.client.0.smithi119.stdout:3/364: dread db/d36/d45/d4c/d62/d13/f21 [0,4194304] 0 2024-05-27T09:29:15.516 INFO:tasks.workunit.client.0.smithi119.stdout:3/365: fsync db/f23 0 2024-05-27T09:29:15.518 INFO:tasks.workunit.client.0.smithi119.stdout:8/216: rename d3/d8/d3d to d3/d49 0 2024-05-27T09:29:15.521 INFO:tasks.workunit.client.0.smithi119.stdout:1/268: dread d7/f1e [0,4194304] 0 2024-05-27T09:29:15.525 INFO:tasks.workunit.client.0.smithi119.stdout:4/175: getdents d4/d23/d31 0 2024-05-27T09:29:15.525 INFO:tasks.workunit.client.0.smithi119.stdout:4/176: dread - d4/de/d21/f2d zero size 2024-05-27T09:29:15.529 INFO:tasks.workunit.client.0.smithi119.stdout:6/172: creat d0/d2/d24/f3a x:0 0 0 2024-05-27T09:29:15.529 INFO:tasks.workunit.client.0.smithi119.stdout:6/173: write d0/d2/f20 [687453,22389] 0 2024-05-27T09:29:15.529 INFO:tasks.workunit.client.0.smithi119.stdout:6/174: write d0/d2/f20 [1073182,126043] 0 2024-05-27T09:29:15.531 INFO:tasks.workunit.client.0.smithi119.stdout:0/182: rename d2/d4/c5 to d2/d6/d9/d12/c3e 0 2024-05-27T09:29:15.531 INFO:tasks.workunit.client.0.smithi119.stdout:9/190: write f6 [3738770,85579] 0 2024-05-27T09:29:15.533 INFO:tasks.workunit.client.0.smithi119.stdout:7/284: dwrite d7/d1a/d40/f48 [0,4194304] 0 2024-05-27T09:29:15.533 INFO:tasks.workunit.client.0.smithi119.stdout:8/217: creat d3/d8/d1b/d46/f4a x:0 0 0 2024-05-27T09:29:15.534 INFO:tasks.workunit.client.0.smithi119.stdout:4/177: creat d4/d23/f35 x:0 0 0 2024-05-27T09:29:15.535 INFO:tasks.workunit.client.0.smithi119.stdout:5/220: dwrite d6/dd/f32 [4194304,4194304] 0 2024-05-27T09:29:15.535 INFO:tasks.workunit.client.0.smithi119.stdout:5/221: fsync d6/dd/d39/d19/d3c/f43 0 2024-05-27T09:29:15.538 INFO:tasks.workunit.client.0.smithi119.stdout:0/183: rmdir d2/d6/d9/d12/d37 0 2024-05-27T09:29:15.539 INFO:tasks.workunit.client.0.smithi119.stdout:6/175: creat d0/d2/d1c/d21/f3b x:0 0 0 2024-05-27T09:29:15.539 INFO:tasks.workunit.client.0.smithi119.stdout:3/366: rename db/d36/d45/d4c/d62/d13/d33/d43 to db/d6f/d7a 0 2024-05-27T09:29:15.539 INFO:tasks.workunit.client.0.smithi119.stdout:2/213: dwrite d2/f48 [0,4194304] 0 2024-05-27T09:29:15.541 INFO:tasks.workunit.client.0.smithi119.stdout:7/285: rmdir d7/df/d19/d4d 39 2024-05-27T09:29:15.547 INFO:tasks.workunit.client.0.smithi119.stdout:4/178: mkdir d4/d6/d28/d2f/d36 0 2024-05-27T09:29:15.548 INFO:tasks.workunit.client.0.smithi119.stdout:7/286: creat d7/df/d36/d42/f60 x:0 0 0 2024-05-27T09:29:15.548 INFO:tasks.workunit.client.0.smithi119.stdout:6/176: rmdir d0/d2/d1c 39 2024-05-27T09:29:15.549 INFO:tasks.workunit.client.0.smithi119.stdout:2/214: mknod d2/da/d15/c4c 0 2024-05-27T09:29:15.549 INFO:tasks.workunit.client.0.smithi119.stdout:9/191: dread fc [0,4194304] 0 2024-05-27T09:29:15.549 INFO:tasks.workunit.client.0.smithi119.stdout:9/192: chown d18/d19/f1c 385910544 1 2024-05-27T09:29:15.553 INFO:tasks.workunit.client.0.smithi119.stdout:9/193: symlink d18/l3d 0 2024-05-27T09:29:15.553 INFO:tasks.workunit.client.0.smithi119.stdout:7/287: rename d7/df/d36/d45/f5a to d7/df/d36/f61 0 2024-05-27T09:29:15.553 INFO:tasks.workunit.client.0.smithi119.stdout:2/215: creat d2/da/f4d x:0 0 0 2024-05-27T09:29:15.553 INFO:tasks.workunit.client.0.smithi119.stdout:7/288: fdatasync d7/df/d19/f14 0 2024-05-27T09:29:15.554 INFO:tasks.workunit.client.0.smithi119.stdout:6/177: rename d0/d2/f20 to d0/d2/d1c/d21/f3c 0 2024-05-27T09:29:15.558 INFO:tasks.workunit.client.0.smithi119.stdout:9/194: symlink d18/d19/d1b/d2a/l3e 0 2024-05-27T09:29:15.560 INFO:tasks.workunit.client.0.smithi119.stdout:6/178: rename d0/d11/d26/c27 to d0/d2/d24/d37/c3d 0 2024-05-27T09:29:15.563 INFO:tasks.workunit.client.0.smithi119.stdout:6/179: creat d0/d2/d1c/f3e x:0 0 0 2024-05-27T09:29:15.565 INFO:tasks.workunit.client.0.smithi119.stdout:6/180: rename d0/d11/d26/f16 to d0/d11/f3f 0 2024-05-27T09:29:15.566 INFO:tasks.workunit.client.0.smithi119.stdout:2/216: rmdir d2/da/d15/d33 39 2024-05-27T09:29:15.571 INFO:tasks.workunit.client.0.smithi119.stdout:5/222: dread d6/dd/f32 [0,4194304] 0 2024-05-27T09:29:15.573 INFO:tasks.workunit.client.0.smithi119.stdout:6/181: symlink d0/d2/d24/l40 0 2024-05-27T09:29:15.576 INFO:tasks.workunit.client.0.smithi119.stdout:4/179: dread d4/d6/d28/f34 [0,4194304] 0 2024-05-27T09:29:15.576 INFO:tasks.workunit.client.0.smithi119.stdout:3/367: dread db/d36/d45/d4c/d62/d13/f2e [0,4194304] 0 2024-05-27T09:29:15.576 INFO:tasks.workunit.client.0.smithi119.stdout:4/180: truncate d4/d23/d31/f33 4583325 0 2024-05-27T09:29:15.582 INFO:tasks.workunit.client.0.smithi119.stdout:6/182: dread d0/f18 [0,4194304] 0 2024-05-27T09:29:15.588 INFO:tasks.workunit.client.0.smithi119.stdout:6/183: mknod d0/d11/d26/c41 0 2024-05-27T09:29:15.588 INFO:tasks.workunit.client.0.smithi119.stdout:6/184: chown d0/d2/d1c/d21 752 1 2024-05-27T09:29:15.611 INFO:tasks.workunit.client.0.smithi119.stdout:8/218: dwrite d3/d8/fd [0,4194304] 0 2024-05-27T09:29:15.611 INFO:tasks.workunit.client.0.smithi119.stdout:8/219: creat d3/d8/d1b/d46/f4b x:0 0 0 2024-05-27T09:29:15.637 INFO:tasks.workunit.client.0.smithi119.stdout:0/184: dwrite d2/d6/d9/d12/f25 [0,4194304] 0 2024-05-27T09:29:15.655 INFO:tasks.workunit.client.0.smithi119.stdout:0/185: dread d2/d1a/f1e [0,4194304] 0 2024-05-27T09:29:15.665 INFO:tasks.workunit.client.0.smithi119.stdout:4/181: dwrite d4/de/f2e [0,4194304] 0 2024-05-27T09:29:15.671 INFO:tasks.workunit.client.0.smithi119.stdout:1/269: dwrite d7/db/d12/d15/f3f [4194304,4194304] 0 2024-05-27T09:29:15.675 INFO:tasks.workunit.client.0.smithi119.stdout:4/182: link d4/de/lf d4/de/l37 0 2024-05-27T09:29:15.676 INFO:tasks.workunit.client.0.smithi119.stdout:4/183: dread - d4/d6/f2a zero size 2024-05-27T09:29:15.678 INFO:tasks.workunit.client.0.smithi119.stdout:1/270: symlink d7/dd/d44/d47/l54 0 2024-05-27T09:29:15.680 INFO:tasks.workunit.client.0.smithi119.stdout:1/271: creat d7/dd/d25/d29/f55 x:0 0 0 2024-05-27T09:29:15.680 INFO:tasks.workunit.client.0.smithi119.stdout:1/272: stat d7/db/d12/d15/f3f 0 2024-05-27T09:29:15.683 INFO:tasks.workunit.client.0.smithi119.stdout:1/273: rmdir d7/db/d12/d26 39 2024-05-27T09:29:15.683 INFO:tasks.workunit.client.0.smithi119.stdout:1/274: creat d7/dd/d44/f56 x:0 0 0 2024-05-27T09:29:15.695 INFO:tasks.workunit.client.0.smithi119.stdout:2/217: dwrite d2/f19 [0,4194304] 0 2024-05-27T09:29:15.704 INFO:tasks.workunit.client.0.smithi119.stdout:1/275: dread d7/f3d [0,4194304] 0 2024-05-27T09:29:15.704 INFO:tasks.workunit.client.0.smithi119.stdout:1/276: fdatasync d7/f11 0 2024-05-27T09:29:15.707 INFO:tasks.workunit.client.0.smithi119.stdout:1/277: rmdir d7/dd/d44/d47 39 2024-05-27T09:29:15.714 INFO:tasks.workunit.client.0.smithi119.stdout:1/278: rename d7/db/f16 to d7/db/d52/f57 0 2024-05-27T09:29:15.728 INFO:tasks.workunit.client.0.smithi119.stdout:7/289: dwrite d7/df/f10 [0,4194304] 0 2024-05-27T09:29:15.728 INFO:tasks.workunit.client.0.smithi119.stdout:5/223: dwrite d6/d2f/f42 [0,4194304] 0 2024-05-27T09:29:15.735 INFO:tasks.workunit.client.0.smithi119.stdout:7/290: truncate d7/df/f13 7326348 0 2024-05-27T09:29:15.739 INFO:tasks.workunit.client.0.smithi119.stdout:7/291: unlink d7/df/d19/d4d/f4e 0 2024-05-27T09:29:15.739 INFO:tasks.workunit.client.0.smithi119.stdout:7/292: chown d7/df/d19/d1d/l2e 107 1 2024-05-27T09:29:15.755 INFO:tasks.workunit.client.0.smithi119.stdout:4/184: dwrite d4/f32 [0,4194304] 0 2024-05-27T09:29:15.759 INFO:tasks.workunit.client.0.smithi119.stdout:4/185: mknod d4/de/d21/c38 0 2024-05-27T09:29:15.759 INFO:tasks.workunit.client.0.smithi119.stdout:4/186: chown d4/d6/l9 0 1 2024-05-27T09:29:15.760 INFO:tasks.workunit.client.0.smithi119.stdout:4/187: mkdir d4/d1c/d39 0 2024-05-27T09:29:15.762 INFO:tasks.workunit.client.0.smithi119.stdout:0/186: dwrite d2/f3d [0,4194304] 0 2024-05-27T09:29:15.763 INFO:tasks.workunit.client.0.smithi119.stdout:4/188: rename d4/c20 to d4/d6/d28/c3a 0 2024-05-27T09:29:15.763 INFO:tasks.workunit.client.0.smithi119.stdout:4/189: creat d4/d6/f3b x:0 0 0 2024-05-27T09:29:15.763 INFO:tasks.workunit.client.0.smithi119.stdout:4/190: dread - d4/d6/f2a zero size 2024-05-27T09:29:15.764 INFO:tasks.workunit.client.0.smithi119.stdout:1/279: write d7/f11 [3925797,41848] 0 2024-05-27T09:29:15.764 INFO:tasks.workunit.client.0.smithi119.stdout:0/187: rmdir d2/d1a 39 2024-05-27T09:29:15.767 INFO:tasks.workunit.client.0.smithi119.stdout:1/280: symlink d7/l58 0 2024-05-27T09:29:15.767 INFO:tasks.workunit.client.0.smithi119.stdout:0/188: mknod d2/d6/d9/d12/c3f 0 2024-05-27T09:29:15.767 INFO:tasks.workunit.client.0.smithi119.stdout:0/189: dread - d2/d6/f26 zero size 2024-05-27T09:29:15.767 INFO:tasks.workunit.client.0.smithi119.stdout:0/190: chown d2/d4/c19 423 1 2024-05-27T09:29:15.772 INFO:tasks.workunit.client.0.smithi119.stdout:4/191: truncate d4/f22 5695488 0 2024-05-27T09:29:15.773 INFO:tasks.workunit.client.0.smithi119.stdout:3/368: dwrite db/f12 [0,4194304] 0 2024-05-27T09:29:15.773 INFO:tasks.workunit.client.0.smithi119.stdout:6/185: dread d0/d11/d26/f17 [0,4194304] 0 2024-05-27T09:29:15.773 INFO:tasks.workunit.client.0.smithi119.stdout:3/369: chown db/d6f/d7a/f5f 5 1 2024-05-27T09:29:15.773 INFO:tasks.workunit.client.0.smithi119.stdout:1/281: mknod d7/dd/d44/d47/c59 0 2024-05-27T09:29:15.775 INFO:tasks.workunit.client.0.smithi119.stdout:6/186: symlink d0/d2/d24/d37/l42 0 2024-05-27T09:29:15.776 INFO:tasks.workunit.client.0.smithi119.stdout:1/282: mknod d7/d40/c5a 0 2024-05-27T09:29:15.777 INFO:tasks.workunit.client.0.smithi119.stdout:6/187: creat d0/d2/d24/d37/f43 x:0 0 0 2024-05-27T09:29:15.781 INFO:tasks.workunit.client.0.smithi119.stdout:1/283: truncate d7/db/d12/d15/d1f/f34 2788543 0 2024-05-27T09:29:15.781 INFO:tasks.workunit.client.0.smithi119.stdout:6/188: mknod d0/d2/d1c/d21/c44 0 2024-05-27T09:29:15.782 INFO:tasks.workunit.client.0.smithi119.stdout:6/189: readlink d0/d11/d26/l2f 0 2024-05-27T09:29:15.782 INFO:tasks.workunit.client.0.smithi119.stdout:1/284: fdatasync d7/db/d12/d15/f1d 0 2024-05-27T09:29:15.782 INFO:tasks.workunit.client.0.smithi119.stdout:1/285: stat d7/db/d12/c28 0 2024-05-27T09:29:15.783 INFO:tasks.workunit.client.0.smithi119.stdout:6/190: truncate d0/f18 52792 0 2024-05-27T09:29:15.783 INFO:tasks.workunit.client.0.smithi119.stdout:6/191: creat d0/d2/d1c/d21/f45 x:0 0 0 2024-05-27T09:29:15.784 INFO:tasks.workunit.client.0.smithi119.stdout:1/286: creat d7/f5b x:0 0 0 2024-05-27T09:29:15.785 INFO:tasks.workunit.client.0.smithi119.stdout:6/192: truncate d0/f6 4205058 0 2024-05-27T09:29:15.789 INFO:tasks.workunit.client.0.smithi119.stdout:1/287: link c1 d7/dd/d44/c5c 0 2024-05-27T09:29:15.791 INFO:tasks.workunit.client.0.smithi119.stdout:1/288: truncate d7/dd/f13 1188281 0 2024-05-27T09:29:15.793 INFO:tasks.workunit.client.0.smithi119.stdout:1/289: creat d7/dd/f5d x:0 0 0 2024-05-27T09:29:15.794 INFO:tasks.workunit.client.0.smithi119.stdout:1/290: creat d7/d40/f5e x:0 0 0 2024-05-27T09:29:15.794 INFO:tasks.workunit.client.0.smithi119.stdout:1/291: dread - d7/dd/d25/d29/d38/d3c/f53 zero size 2024-05-27T09:29:15.794 INFO:tasks.workunit.client.0.smithi119.stdout:0/191: write d2/d6/f8 [4984017,26036] 0 2024-05-27T09:29:15.794 INFO:tasks.workunit.client.0.smithi119.stdout:0/192: chown d2/d6/d9/d12/l1d 577170 1 2024-05-27T09:29:15.795 INFO:tasks.workunit.client.0.smithi119.stdout:1/292: truncate d7/db/d12/d15/f21 2100596 0 2024-05-27T09:29:15.795 INFO:tasks.workunit.client.0.smithi119.stdout:1/293: dread - d7/dd/f5d zero size 2024-05-27T09:29:15.795 INFO:tasks.workunit.client.0.smithi119.stdout:2/218: dwrite d2/da/fd [4194304,4194304] 0 2024-05-27T09:29:15.795 INFO:tasks.workunit.client.0.smithi119.stdout:5/224: dwrite d6/db/de/f18 [0,4194304] 0 2024-05-27T09:29:15.796 INFO:tasks.workunit.client.0.smithi119.stdout:5/225: readlink d6/dd/d46/d47/l48 0 2024-05-27T09:29:15.796 INFO:tasks.workunit.client.0.smithi119.stdout:5/226: chown d6/f9 575 1 2024-05-27T09:29:15.796 INFO:tasks.workunit.client.0.smithi119.stdout:0/193: creat d2/d6/d9/d24/f40 x:0 0 0 2024-05-27T09:29:15.797 INFO:tasks.workunit.client.0.smithi119.stdout:1/294: mknod d7/dd/d25/d29/d38/d3c/c5f 0 2024-05-27T09:29:15.802 INFO:tasks.workunit.client.0.smithi119.stdout:0/194: symlink d2/d6/d9/d24/l41 0 2024-05-27T09:29:15.802 INFO:tasks.workunit.client.0.smithi119.stdout:0/195: chown d2/d6/d9/d24/c38 1 1 2024-05-27T09:29:15.803 INFO:tasks.workunit.client.0.smithi119.stdout:1/295: mkdir d7/dd/d25/d60 0 2024-05-27T09:29:15.805 INFO:tasks.workunit.client.0.smithi119.stdout:0/196: creat d2/d4/f42 x:0 0 0 2024-05-27T09:29:15.807 INFO:tasks.workunit.client.0.smithi119.stdout:1/296: mkdir d7/dd/d25/d60/d61 0 2024-05-27T09:29:15.807 INFO:tasks.workunit.client.0.smithi119.stdout:1/297: readlink d7/dd/l18 0 2024-05-27T09:29:15.817 INFO:tasks.workunit.client.0.smithi119.stdout:2/219: dread d2/da/d15/f18 [0,4194304] 0 2024-05-27T09:29:15.817 INFO:tasks.workunit.client.0.smithi119.stdout:2/220: stat d2/da/d15/l44 0 2024-05-27T09:29:15.818 INFO:tasks.workunit.client.0.smithi119.stdout:2/221: readlink l0 0 2024-05-27T09:29:15.821 INFO:tasks.workunit.client.0.smithi119.stdout:1/298: link d7/db/d12/d15/c2a d7/db/c62 0 2024-05-27T09:29:15.821 INFO:tasks.workunit.client.0.smithi119.stdout:6/193: write d0/f6 [4412098,69875] 0 2024-05-27T09:29:15.821 INFO:tasks.workunit.client.0.smithi119.stdout:6/194: write d0/d2/f30 [347211,7045] 0 2024-05-27T09:29:15.821 INFO:tasks.workunit.client.0.smithi119.stdout:6/195: creat d0/d2/d1c/d21/f46 x:0 0 0 2024-05-27T09:29:15.826 INFO:tasks.workunit.client.0.smithi119.stdout:0/197: symlink d2/d6/d9/l43 0 2024-05-27T09:29:15.826 INFO:tasks.workunit.client.0.smithi119.stdout:0/198: chown d2/d6/d9/d12/f25 50202 1 2024-05-27T09:29:15.826 INFO:tasks.workunit.client.0.smithi119.stdout:0/199: stat d2/d6/d9/f1c 0 2024-05-27T09:29:15.830 INFO:tasks.workunit.client.0.smithi119.stdout:5/227: truncate d6/db/de/f18 1497194 0 2024-05-27T09:29:15.836 INFO:tasks.workunit.client.0.smithi119.stdout:2/222: write d2/d1a/f46 [3785785,69621] 0 2024-05-27T09:29:15.839 INFO:tasks.workunit.client.0.smithi119.stdout:7/293: dwrite d7/df/d19/d1d/f23 [0,4194304] 0 2024-05-27T09:29:15.839 INFO:tasks.workunit.client.0.smithi119.stdout:7/294: dread - d7/db/d37/f5e zero size 2024-05-27T09:29:15.840 INFO:tasks.workunit.client.0.smithi119.stdout:7/295: fdatasync d7/d1a/d40/f48 0 2024-05-27T09:29:15.852 INFO:tasks.workunit.client.0.smithi119.stdout:2/223: stat d2/da/d2c/c49 0 2024-05-27T09:29:15.852 INFO:tasks.workunit.client.0.smithi119.stdout:0/200: write d2/d6/d9/d12/f16 [1808085,7136] 0 2024-05-27T09:29:15.853 INFO:tasks.workunit.client.0.smithi119.stdout:0/201: link d2/d6/d9/d12/f1b d2/d4/f44 0 2024-05-27T09:29:15.855 INFO:tasks.workunit.client.0.smithi119.stdout:0/202: mkdir d2/d6/d9/d24/d30/d45 0 2024-05-27T09:29:15.855 INFO:tasks.workunit.client.0.smithi119.stdout:0/203: creat d2/d4/f46 x:0 0 0 2024-05-27T09:29:15.873 INFO:tasks.workunit.client.0.smithi119.stdout:0/204: dread d2/d6/d9/d12/f25 [0,4194304] 0 2024-05-27T09:29:15.885 INFO:tasks.workunit.client.0.smithi119.stdout:3/370: dwrite db/f24 [0,4194304] 0 2024-05-27T09:29:15.885 INFO:tasks.workunit.client.0.smithi119.stdout:3/371: truncate db/d36/d45/d4d/f70 1030892 0 2024-05-27T09:29:15.885 INFO:tasks.workunit.client.0.smithi119.stdout:3/372: creat db/d36/d45/d49/d4f/f7b x:0 0 0 2024-05-27T09:29:15.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:15 smithi138 ceph-mon[34885]: pgmap v5: 65 pgs: 65 active+clean; 473 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:29:15.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:15 smithi138 ceph-mon[34885]: mgrmap e24: smithi138.dxsuqf(active, since 4s) 2024-05-27T09:29:15.890 INFO:tasks.workunit.client.0.smithi119.stdout:2/224: dread d2/da/f13 [0,4194304] 0 2024-05-27T09:29:15.905 INFO:tasks.workunit.client.0.smithi119.stdout:3/373: getdents db/d36/d45/d4c 0 2024-05-27T09:29:15.905 INFO:tasks.workunit.client.0.smithi119.stdout:7/296: dwrite d7/d1a/d40/f48 [4194304,4194304] 0 2024-05-27T09:29:15.906 INFO:tasks.workunit.client.0.smithi119.stdout:7/297: fsync d7/db/f2f 0 2024-05-27T09:29:15.908 INFO:tasks.workunit.client.0.smithi119.stdout:7/298: creat d7/f62 x:0 0 0 2024-05-27T09:29:15.908 INFO:tasks.workunit.client.0.smithi119.stdout:7/299: fdatasync d7/f33 0 2024-05-27T09:29:15.908 INFO:tasks.workunit.client.0.smithi119.stdout:7/300: chown d7/df/l26 242 1 2024-05-27T09:29:15.908 INFO:tasks.workunit.client.0.smithi119.stdout:8/220: sync 2024-05-27T09:29:15.913 INFO:tasks.workunit.client.0.smithi119.stdout:9/195: sync 2024-05-27T09:29:15.952 INFO:tasks.workunit.client.0.smithi119.stdout:0/205: dread d2/d1a/f21 [0,4194304] 0 2024-05-27T09:29:15.953 INFO:tasks.workunit.client.0.smithi119.stdout:1/299: dwrite d7/f3d [4194304,4194304] 0 2024-05-27T09:29:15.953 INFO:tasks.workunit.client.0.smithi119.stdout:1/300: chown d7/db/d12/d15/d1f/f34 2 1 2024-05-27T09:29:15.953 INFO:tasks.workunit.client.0.smithi119.stdout:1/301: chown d7/db/c4f 437676 1 2024-05-27T09:29:15.963 INFO:tasks.workunit.client.0.smithi119.stdout:1/302: truncate d7/dd/d44/f4e 769548 0 2024-05-27T09:29:15.965 INFO:tasks.workunit.client.0.smithi119.stdout:1/303: creat d7/dd/d44/d47/f63 x:0 0 0 2024-05-27T09:29:15.965 INFO:tasks.workunit.client.0.smithi119.stdout:1/304: write d7/f3d [4014301,39461] 0 2024-05-27T09:29:15.983 INFO:tasks.workunit.client.0.smithi119.stdout:8/221: dwrite d3/de/f2c [0,4194304] 0 2024-05-27T09:29:15.991 INFO:tasks.workunit.client.0.smithi119.stdout:7/301: dwrite d7/db/f57 [0,4194304] 0 2024-05-27T09:29:15.991 INFO:tasks.workunit.client.0.smithi119.stdout:7/302: dread - d7/f33 zero size 2024-05-27T09:29:15.997 INFO:tasks.workunit.client.0.smithi119.stdout:9/196: dwrite d18/d19/d1b/f38 [0,4194304] 0 2024-05-27T09:29:15.997 INFO:tasks.workunit.client.0.smithi119.stdout:9/197: stat d18/d19/d1b/d29 0 2024-05-27T09:29:15.997 INFO:tasks.workunit.client.0.smithi119.stdout:9/198: fdatasync d18/d19/d1b/d29/f2c 0 2024-05-27T09:29:15.999 INFO:tasks.workunit.client.0.smithi119.stdout:0/206: dwrite d2/d4/f42 [0,4194304] 0 2024-05-27T09:29:16.000 INFO:tasks.workunit.client.0.smithi119.stdout:0/207: truncate d2/d6/d9/d24/f40 465809 0 2024-05-27T09:29:16.007 INFO:tasks.workunit.client.0.smithi119.stdout:6/196: dwrite d0/d2/d1c/f2d [0,4194304] 0 2024-05-27T09:29:16.007 INFO:tasks.workunit.client.0.smithi119.stdout:6/197: write d0/d2/d24/d37/f43 [574480,61165] 0 2024-05-27T09:29:16.021 INFO:tasks.workunit.client.0.smithi119.stdout:8/222: dread d3/d8/d1b/d46/f26 [0,4194304] 0 2024-05-27T09:29:16.026 INFO:tasks.workunit.client.0.smithi119.stdout:0/208: dread d2/d4/f42 [0,4194304] 0 2024-05-27T09:29:16.026 INFO:tasks.workunit.client.0.smithi119.stdout:0/209: stat d2/d6/d23 0 2024-05-27T09:29:16.027 INFO:tasks.workunit.client.0.smithi119.stdout:6/198: write d0/d2/f4 [446476,15720] 0 2024-05-27T09:29:16.027 INFO:tasks.workunit.client.0.smithi119.stdout:6/199: fsync d0/d2/d1c/d21/f45 0 2024-05-27T09:29:16.027 INFO:tasks.workunit.client.0.smithi119.stdout:6/200: write d0/d2/f4 [1033628,86831] 0 2024-05-27T09:29:16.027 INFO:tasks.workunit.client.0.smithi119.stdout:7/303: dread d7/df/f29 [0,4194304] 0 2024-05-27T09:29:16.027 INFO:tasks.workunit.client.0.smithi119.stdout:2/225: dread d2/d1a/f46 [0,4194304] 0 2024-05-27T09:29:16.027 INFO:tasks.workunit.client.0.smithi119.stdout:8/223: dread d3/d8/d1b/d46/f2f [0,4194304] 0 2024-05-27T09:29:16.028 INFO:tasks.workunit.client.0.smithi119.stdout:7/304: write d7/df/d36/f52 [381998,62783] 0 2024-05-27T09:29:16.028 INFO:tasks.workunit.client.0.smithi119.stdout:7/305: truncate d7/f33 884662 0 2024-05-27T09:29:16.028 INFO:tasks.workunit.client.0.smithi119.stdout:3/374: dwrite db/d36/d45/d4c/f6c [0,4194304] 0 2024-05-27T09:29:16.028 INFO:tasks.workunit.client.0.smithi119.stdout:0/210: rename d2/d1a/d20/f39 to d2/d6/d9/d12/f47 0 2024-05-27T09:29:16.029 INFO:tasks.workunit.client.0.smithi119.stdout:1/305: dwrite f3 [0,4194304] 0 2024-05-27T09:29:16.033 INFO:tasks.workunit.client.0.smithi119.stdout:6/201: mknod d0/c47 0 2024-05-27T09:29:16.033 INFO:tasks.workunit.client.0.smithi119.stdout:6/202: read d0/f18 [20618,127886] 0 2024-05-27T09:29:16.034 INFO:tasks.workunit.client.0.smithi119.stdout:2/226: mkdir d2/da/d4e 0 2024-05-27T09:29:16.036 INFO:tasks.workunit.client.0.smithi119.stdout:8/224: unlink d3/d8/d1b/d46/f38 0 2024-05-27T09:29:16.037 INFO:tasks.workunit.client.0.smithi119.stdout:7/306: symlink d7/d1a/l63 0 2024-05-27T09:29:16.037 INFO:tasks.workunit.client.0.smithi119.stdout:7/307: chown d7/df/d19/l4b 8199473 1 2024-05-27T09:29:16.039 INFO:tasks.workunit.client.0.smithi119.stdout:3/375: link db/f24 db/d36/d45/d4c/d62/f7c 0 2024-05-27T09:29:16.041 INFO:tasks.workunit.client.0.smithi119.stdout:0/211: truncate d2/d6/d9/ff 3951249 0 2024-05-27T09:29:16.043 INFO:tasks.workunit.client.0.smithi119.stdout:1/306: rename d7/dd/d25/d29/l33 to d7/dd/d44/d47/l64 0 2024-05-27T09:29:16.044 INFO:tasks.workunit.client.0.smithi119.stdout:6/203: rename d0 to d0/d11/d48 22 2024-05-27T09:29:16.045 INFO:tasks.workunit.client.0.smithi119.stdout:5/228: sync 2024-05-27T09:29:16.047 INFO:tasks.workunit.client.0.smithi119.stdout:2/227: link d2/da/f37 d2/da/d4e/f4f 0 2024-05-27T09:29:16.047 INFO:tasks.workunit.client.0.smithi119.stdout:2/228: fsync d2/d1a/f1f 0 2024-05-27T09:29:16.049 INFO:tasks.workunit.client.0.smithi119.stdout:3/376: truncate db/d36/d45/d4c/d62/d13/f2c 2419379 0 2024-05-27T09:29:16.050 INFO:tasks.workunit.client.0.smithi119.stdout:3/377: write db/d36/d45/d4d/f67 [378458,28731] 0 2024-05-27T09:29:16.050 INFO:tasks.workunit.client.0.smithi119.stdout:3/378: chown db/d36/d45/d4c/d62/d13/d33/l34 1 1 2024-05-27T09:29:16.050 INFO:tasks.workunit.client.0.smithi119.stdout:3/379: fsync db/d36/d45/d4c/d62/d13/f2a 0 2024-05-27T09:29:16.050 INFO:tasks.workunit.client.0.smithi119.stdout:3/380: chown db/d36/d45/d4d/d64/f3e 469 1 2024-05-27T09:29:16.051 INFO:tasks.workunit.client.0.smithi119.stdout:8/225: getdents d3/d8/d1b/d46/d30 0 2024-05-27T09:29:16.052 INFO:tasks.workunit.client.0.smithi119.stdout:1/307: mkdir d7/db/d52/d65 0 2024-05-27T09:29:16.052 INFO:tasks.workunit.client.0.smithi119.stdout:8/226: truncate d3/d8/d1b/f41 14620 0 2024-05-27T09:29:16.052 INFO:tasks.workunit.client.0.smithi119.stdout:4/192: sync 2024-05-27T09:29:16.052 INFO:tasks.workunit.client.0.smithi119.stdout:6/204: truncate d0/d2/f10 2330937 0 2024-05-27T09:29:16.060 INFO:tasks.workunit.client.0.smithi119.stdout:2/229: symlink d2/da/d15/d33/l50 0 2024-05-27T09:29:16.061 INFO:tasks.workunit.client.0.smithi119.stdout:8/227: rmdir d3/d8/d1b/d46/d30 39 2024-05-27T09:29:16.061 INFO:tasks.workunit.client.0.smithi119.stdout:1/308: truncate d7/db/d52/f57 2251430 0 2024-05-27T09:29:16.061 INFO:tasks.workunit.client.0.smithi119.stdout:4/193: symlink d4/d6/l3c 0 2024-05-27T09:29:16.061 INFO:tasks.workunit.client.0.smithi119.stdout:4/194: stat c0 0 2024-05-27T09:29:16.064 INFO:tasks.workunit.client.0.smithi119.stdout:6/205: rename d0/d2/d24/l35 to d0/d2/d1c/d21/l49 0 2024-05-27T09:29:16.074 INFO:tasks.workunit.client.0.smithi119.stdout:3/381: symlink db/l7d 0 2024-05-27T09:29:16.074 INFO:tasks.workunit.client.0.smithi119.stdout:2/230: mkdir d2/d1a/d51 0 2024-05-27T09:29:16.074 INFO:tasks.workunit.client.0.smithi119.stdout:4/195: symlink d4/d6/d28/l3d 0 2024-05-27T09:29:16.074 INFO:tasks.workunit.client.0.smithi119.stdout:4/196: chown d4/d1c/l2c 1 1 2024-05-27T09:29:16.074 INFO:tasks.workunit.client.0.smithi119.stdout:2/231: symlink d2/l52 0 2024-05-27T09:29:16.074 INFO:tasks.workunit.client.0.smithi119.stdout:4/197: mkdir d4/d6/d3e 0 2024-05-27T09:29:16.074 INFO:tasks.workunit.client.0.smithi119.stdout:2/232: mkdir d2/da/d15/d53 0 2024-05-27T09:29:16.074 INFO:tasks.workunit.client.0.smithi119.stdout:4/198: symlink d4/d1c/l3f 0 2024-05-27T09:29:16.074 INFO:tasks.workunit.client.0.smithi119.stdout:2/233: read d2/da/ff [344869,120801] 0 2024-05-27T09:29:16.079 INFO:tasks.workunit.client.0.smithi119.stdout:4/199: mkdir d4/de/d21/d40 0 2024-05-27T09:29:16.079 INFO:tasks.workunit.client.0.smithi119.stdout:4/200: write d4/d1c/f1e [3401183,106969] 0 2024-05-27T09:29:16.079 INFO:tasks.workunit.client.0.smithi119.stdout:4/201: creat d4/f41 x:0 0 0 2024-05-27T09:29:16.081 INFO:tasks.workunit.client.0.smithi119.stdout:6/206: dread d0/d11/d26/f1f [0,4194304] 0 2024-05-27T09:29:16.081 INFO:tasks.workunit.client.0.smithi119.stdout:6/207: creat d0/d2/d24/f4a x:0 0 0 2024-05-27T09:29:16.081 INFO:tasks.workunit.client.0.smithi119.stdout:6/208: write d0/d2/d1c/f3e [493852,88163] 0 2024-05-27T09:29:16.082 INFO:tasks.workunit.client.0.smithi119.stdout:4/202: creat d4/de/d21/d40/f42 x:0 0 0 2024-05-27T09:29:16.098 INFO:tasks.workunit.client.0.smithi119.stdout:4/203: unlink d4/d1c/l2c 0 2024-05-27T09:29:16.098 INFO:tasks.workunit.client.0.smithi119.stdout:6/209: getdents d0 0 2024-05-27T09:29:16.098 INFO:tasks.workunit.client.0.smithi119.stdout:4/204: symlink d4/d23/d24/l43 0 2024-05-27T09:29:16.114 INFO:tasks.workunit.client.0.smithi119.stdout:8/228: dread d3/d8/fd [0,4194304] 0 2024-05-27T09:29:16.115 INFO:tasks.workunit.client.0.smithi119.stdout:7/308: dwrite d7/df/d19/f14 [0,4194304] 0 2024-05-27T09:29:16.115 INFO:tasks.workunit.client.0.smithi119.stdout:8/229: chown d3/de 17421 1 2024-05-27T09:29:16.118 INFO:tasks.workunit.client.0.smithi119.stdout:8/230: truncate d3/f25 3431933 0 2024-05-27T09:29:16.118 INFO:tasks.workunit.client.0.smithi119.stdout:7/309: rmdir d7/df/d36/d45 0 2024-05-27T09:29:16.119 INFO:tasks.workunit.client.0.smithi119.stdout:6/210: write d0/d11/d26/f17 [7996843,51932] 0 2024-05-27T09:29:16.125 INFO:tasks.workunit.client.0.smithi119.stdout:0/212: dwrite d2/d4/f42 [0,4194304] 0 2024-05-27T09:29:16.125 INFO:tasks.workunit.client.0.smithi119.stdout:0/213: write d2/d6/d9/ff [1587461,64411] 0 2024-05-27T09:29:16.126 INFO:tasks.workunit.client.0.smithi119.stdout:0/214: truncate d2/d6/f26 777334 0 2024-05-27T09:29:16.126 INFO:tasks.workunit.client.0.smithi119.stdout:0/215: dread - d2/d6/d9/d12/f47 zero size 2024-05-27T09:29:16.132 INFO:tasks.workunit.client.0.smithi119.stdout:4/205: dread d4/de/f2e [0,4194304] 0 2024-05-27T09:29:16.136 INFO:tasks.workunit.client.0.smithi119.stdout:0/216: symlink d2/l48 0 2024-05-27T09:29:16.138 INFO:tasks.workunit.client.0.smithi119.stdout:4/206: symlink d4/l44 0 2024-05-27T09:29:16.138 INFO:tasks.workunit.client.0.smithi119.stdout:4/207: write d4/d1c/f29 [345863,119265] 0 2024-05-27T09:29:16.138 INFO:tasks.workunit.client.0.smithi119.stdout:4/208: truncate d4/d1c/f29 740978 0 2024-05-27T09:29:16.138 INFO:tasks.workunit.client.0.smithi119.stdout:4/209: truncate d4/de/d21/f2d 283669 0 2024-05-27T09:29:16.138 INFO:tasks.workunit.client.0.smithi119.stdout:4/210: chown d4/d6/d28/d2f 3 1 2024-05-27T09:29:16.138 INFO:tasks.workunit.client.0.smithi119.stdout:4/211: creat d4/d23/f45 x:0 0 0 2024-05-27T09:29:16.140 INFO:tasks.workunit.client.0.smithi119.stdout:4/212: getdents d4/d6 0 2024-05-27T09:29:16.146 INFO:tasks.workunit.client.0.smithi119.stdout:2/234: dwrite d2/d1a/f36 [0,4194304] 0 2024-05-27T09:29:16.150 INFO:tasks.workunit.client.0.smithi119.stdout:2/235: link d2/da/d2c/c40 d2/da/d15/c54 0 2024-05-27T09:29:16.180 INFO:tasks.workunit.client.0.smithi119.stdout:8/231: dwrite d3/f16 [0,4194304] 0 2024-05-27T09:29:16.182 INFO:tasks.workunit.client.0.smithi119.stdout:8/232: mkdir d3/d8/d1b/d4c 0 2024-05-27T09:29:16.184 INFO:tasks.workunit.client.0.smithi119.stdout:8/233: unlink f0 0 2024-05-27T09:29:16.186 INFO:tasks.workunit.client.0.smithi119.stdout:3/382: dwrite db/d36/d45/d4c/d62/d13/f2e [0,4194304] 0 2024-05-27T09:29:16.192 INFO:tasks.workunit.client.0.smithi119.stdout:8/234: dread d3/d8/f47 [0,4194304] 0 2024-05-27T09:29:16.194 INFO:tasks.workunit.client.0.smithi119.stdout:3/383: rename db/d36/d45/d4c/d62/d13/d68 to db/d36/d45/d49/d4f/d7e 0 2024-05-27T09:29:16.195 INFO:tasks.workunit.client.0.smithi119.stdout:8/235: getdents d3/de 0 2024-05-27T09:29:16.195 INFO:tasks.workunit.client.0.smithi119.stdout:8/236: chown d3/d8/c27 775924604 1 2024-05-27T09:29:16.195 INFO:tasks.workunit.client.0.smithi119.stdout:8/237: fsync d3/de/f1d 0 2024-05-27T09:29:16.198 INFO:tasks.workunit.client.0.smithi119.stdout:7/310: dwrite d7/df/d19/f1b [0,4194304] 0 2024-05-27T09:29:16.198 INFO:tasks.workunit.client.0.smithi119.stdout:8/238: rename d3/d8/d1b/d46/d30 to d3/d8/d4d 0 2024-05-27T09:29:16.198 INFO:tasks.workunit.client.0.smithi119.stdout:3/384: getdents db/d36/d45/d4d/d64 0 2024-05-27T09:29:16.201 INFO:tasks.workunit.client.0.smithi119.stdout:8/239: rename d3/de/l40 to d3/d49/l4e 0 2024-05-27T09:29:16.201 INFO:tasks.workunit.client.0.smithi119.stdout:8/240: fdatasync d3/d8/d1b/d46/f43 0 2024-05-27T09:29:16.203 INFO:tasks.workunit.client.0.smithi119.stdout:0/217: dwrite d2/d6/d9/d24/f40 [0,4194304] 0 2024-05-27T09:29:16.203 INFO:tasks.workunit.client.0.smithi119.stdout:0/218: creat d2/f49 x:0 0 0 2024-05-27T09:29:16.205 INFO:tasks.workunit.client.0.smithi119.stdout:8/241: symlink d3/d8/d1b/d46/l4f 0 2024-05-27T09:29:16.205 INFO:tasks.workunit.client.0.smithi119.stdout:3/385: rename db/d36/d45/d4d/d64 to db/d7f 0 2024-05-27T09:29:16.205 INFO:tasks.workunit.client.0.smithi119.stdout:8/242: write d3/d8/d1b/d46/f43 [1034855,24908] 0 2024-05-27T09:29:16.212 INFO:tasks.workunit.client.0.smithi119.stdout:8/243: rmdir d3 39 2024-05-27T09:29:16.215 INFO:tasks.workunit.client.0.smithi119.stdout:8/244: mknod d3/d39/c50 0 2024-05-27T09:29:16.233 INFO:tasks.workunit.client.0.smithi119.stdout:8/245: dread d3/f1a [0,4194304] 0 2024-05-27T09:29:16.234 INFO:tasks.workunit.client.0.smithi119.stdout:8/246: fdatasync d3/f25 0 2024-05-27T09:29:16.236 INFO:tasks.workunit.client.0.smithi119.stdout:8/247: rename d3/d39/c50 to d3/de/c51 0 2024-05-27T09:29:16.238 INFO:tasks.workunit.client.0.smithi119.stdout:8/248: creat d3/d39/f52 x:0 0 0 2024-05-27T09:29:16.239 INFO:tasks.workunit.client.0.smithi119.stdout:8/249: stat d3/d39/c3a 0 2024-05-27T09:29:16.240 INFO:tasks.workunit.client.0.smithi119.stdout:8/250: rename d3/d39/c48 to d3/d39/c53 0 2024-05-27T09:29:16.246 INFO:tasks.workunit.client.0.smithi119.stdout:3/386: dwrite db/d36/d45/d4c/d62/d13/f3d [0,4194304] 0 2024-05-27T09:29:16.255 INFO:tasks.workunit.client.0.smithi119.stdout:0/219: dwrite d2/f49 [0,4194304] 0 2024-05-27T09:29:16.256 INFO:tasks.workunit.client.0.smithi119.stdout:9/199: sync 2024-05-27T09:29:16.273 INFO:tasks.workunit.client.0.smithi119.stdout:3/387: dread db/d36/d45/d4c/d62/f3f [0,4194304] 0 2024-05-27T09:29:16.276 INFO:tasks.workunit.client.0.smithi119.stdout:3/388: rename db/d36/d45/d49/d4f/d7e/f72 to db/d6f/f80 0 2024-05-27T09:29:16.279 INFO:tasks.workunit.client.0.smithi119.stdout:3/389: rename db/d36/d45/d4d/f6b to db/d36/d45/d4c/d62/d13/f81 0 2024-05-27T09:29:16.279 INFO:tasks.workunit.client.0.smithi119.stdout:3/390: creat db/d5b/f82 x:0 0 0 2024-05-27T09:29:16.279 INFO:tasks.workunit.client.0.smithi119.stdout:3/391: write db/f2f [5109738,34230] 0 2024-05-27T09:29:16.285 INFO:tasks.workunit.client.0.smithi119.stdout:3/392: rename db/d36/d45/f52 to db/f83 0 2024-05-27T09:29:16.288 INFO:tasks.workunit.client.0.smithi119.stdout:3/393: creat db/d6f/d7a/f84 x:0 0 0 2024-05-27T09:29:16.293 INFO:tasks.workunit.client.0.smithi119.stdout:3/394: creat db/d36/d45/d49/f85 x:0 0 0 2024-05-27T09:29:16.293 INFO:tasks.workunit.client.0.smithi119.stdout:3/395: write db/d6f/d7a/f5c [4589303,24708] 0 2024-05-27T09:29:16.293 INFO:tasks.workunit.client.0.smithi119.stdout:3/396: chown db/d5b/f63 0 1 2024-05-27T09:29:16.299 INFO:tasks.workunit.client.0.smithi119.stdout:3/397: rmdir db/d36/d45/d49/d4f/d7e 0 2024-05-27T09:29:16.299 INFO:tasks.workunit.client.0.smithi119.stdout:3/398: write db/d6f/f73 [371848,94415] 0 2024-05-27T09:29:16.299 INFO:tasks.workunit.client.0.smithi119.stdout:3/399: fdatasync db/d36/d45/d4c/d62/f38 0 2024-05-27T09:29:16.299 INFO:tasks.workunit.client.0.smithi119.stdout:3/400: fsync db/d36/d45/d4c/d62/d13/f2e 0 2024-05-27T09:29:16.300 INFO:tasks.workunit.client.0.smithi119.stdout:3/401: symlink db/d7f/l86 0 2024-05-27T09:29:16.306 INFO:tasks.workunit.client.0.smithi119.stdout:3/402: rename db/f35 to db/d36/d45/d4c/f87 0 2024-05-27T09:29:16.306 INFO:tasks.workunit.client.0.smithi119.stdout:3/403: creat db/d36/d45/d4c/d62/f88 x:0 0 0 2024-05-27T09:29:16.316 INFO:tasks.workunit.client.0.smithi119.stdout:0/220: dwrite d2/d6/d9/d12/f25 [0,4194304] 0 2024-05-27T09:29:16.316 INFO:tasks.workunit.client.0.smithi119.stdout:0/221: chown d2/d6/d23/l2e 64 1 2024-05-27T09:29:16.316 INFO:tasks.workunit.client.0.smithi119.stdout:0/222: chown d2/d6/f3c 0 1 2024-05-27T09:29:16.319 INFO:tasks.workunit.client.0.smithi119.stdout:0/223: mkdir d2/d6/d9/d24/d30/d4a 0 2024-05-27T09:29:16.336 INFO:tasks.workunit.client.0.smithi119.stdout:0/224: dread d2/d1a/f1e [0,4194304] 0 2024-05-27T09:29:16.347 INFO:tasks.workunit.client.0.smithi119.stdout:0/225: dread d2/d6/d9/f1c [0,4194304] 0 2024-05-27T09:29:16.350 INFO:tasks.workunit.client.0.smithi119.stdout:0/226: getdents d2 0 2024-05-27T09:29:16.350 INFO:tasks.workunit.client.0.smithi119.stdout:0/227: truncate d2/d6/d9/d12/f32 559452 0 2024-05-27T09:29:16.351 INFO:tasks.workunit.client.0.smithi119.stdout:0/228: mkdir d2/d6/d9/d12/d4b 0 2024-05-27T09:29:16.353 INFO:tasks.workunit.client.0.smithi119.stdout:9/200: dwrite d18/d19/f1c [0,4194304] 0 2024-05-27T09:29:16.356 INFO:tasks.workunit.client.0.smithi119.stdout:0/229: creat d2/d6/d9/d24/d30/d4a/f4c x:0 0 0 2024-05-27T09:29:16.362 INFO:tasks.workunit.client.0.smithi119.stdout:9/201: creat d18/d19/f3f x:0 0 0 2024-05-27T09:29:16.364 INFO:tasks.workunit.client.0.smithi119.stdout:9/202: mkdir d18/d19/d1b/d40 0 2024-05-27T09:29:16.368 INFO:tasks.workunit.client.0.smithi119.stdout:3/404: dwrite f8 [0,4194304] 0 2024-05-27T09:29:16.368 INFO:tasks.workunit.client.0.smithi119.stdout:3/405: creat db/d36/d45/d4c/d62/f89 x:0 0 0 2024-05-27T09:29:16.368 INFO:tasks.workunit.client.0.smithi119.stdout:3/406: write db/f69 [518850,33614] 0 2024-05-27T09:29:16.376 INFO:tasks.workunit.client.0.smithi119.stdout:3/407: symlink db/d36/d45/d49/l8a 0 2024-05-27T09:29:16.377 INFO:tasks.workunit.client.0.smithi119.stdout:0/230: write d2/d6/d9/f2c [2420390,49905] 0 2024-05-27T09:29:16.381 INFO:tasks.workunit.client.0.smithi119.stdout:3/408: rename db/d36/d45/d4c/d62/d13/c3b to db/d36/d45/d4c/d62/d13/c8b 0 2024-05-27T09:29:16.381 INFO:tasks.workunit.client.0.smithi119.stdout:3/409: fdatasync db/d6f/d7a/f5c 0 2024-05-27T09:29:16.381 INFO:tasks.workunit.client.0.smithi119.stdout:3/410: dread - db/d36/d45/d4d/f58 zero size 2024-05-27T09:29:16.383 INFO:tasks.workunit.client.0.smithi119.stdout:0/231: unlink d2/d6/d9/d12/f25 0 2024-05-27T09:29:16.383 INFO:tasks.workunit.client.0.smithi119.stdout:0/232: chown d2/d6/d9/ff 12420 1 2024-05-27T09:29:16.383 INFO:tasks.workunit.client.0.smithi119.stdout:3/411: mknod db/d5b/c8c 0 2024-05-27T09:29:16.383 INFO:tasks.workunit.client.0.smithi119.stdout:3/412: write db/f56 [1401736,16484] 0 2024-05-27T09:29:16.384 INFO:tasks.workunit.client.0.smithi119.stdout:0/233: symlink d2/d1a/l4d 0 2024-05-27T09:29:16.384 INFO:tasks.workunit.client.0.smithi119.stdout:0/234: stat d2/d6/f26 0 2024-05-27T09:29:16.385 INFO:tasks.workunit.client.0.smithi119.stdout:0/235: readlink d2/l10 0 2024-05-27T09:29:16.385 INFO:tasks.workunit.client.0.smithi119.stdout:0/236: chown d2/f3d 155640 1 2024-05-27T09:29:16.385 INFO:tasks.workunit.client.0.smithi119.stdout:0/237: write d2/d6/f26 [1654084,81492] 0 2024-05-27T09:29:16.385 INFO:tasks.workunit.client.0.smithi119.stdout:0/238: readlink d2/l14 0 2024-05-27T09:29:16.385 INFO:tasks.workunit.client.0.smithi119.stdout:3/413: getdents db/d36/d45/d4c/d62/d13/d33 0 2024-05-27T09:29:16.385 INFO:tasks.workunit.client.0.smithi119.stdout:3/414: chown db/d7f/c48 459 1 2024-05-27T09:29:16.386 INFO:tasks.workunit.client.0.smithi119.stdout:0/239: rename d2/d6/d9/d12/c3b to d2/d6/d9/d24/d30/d4a/c4e 0 2024-05-27T09:29:16.387 INFO:tasks.workunit.client.0.smithi119.stdout:3/415: symlink db/d36/d45/d4c/d62/d13/l8d 0 2024-05-27T09:29:16.388 INFO:tasks.workunit.client.0.smithi119.stdout:0/240: mkdir d2/d6/d9/d12/d4f 0 2024-05-27T09:29:16.389 INFO:tasks.workunit.client.0.smithi119.stdout:3/416: rename db/d7f/f3e to db/d36/d45/d4c/d62/f8e 0 2024-05-27T09:29:16.390 INFO:tasks.workunit.client.0.smithi119.stdout:0/241: symlink d2/d6/d9/l50 0 2024-05-27T09:29:16.394 INFO:tasks.workunit.client.0.smithi119.stdout:3/417: mknod db/d6f/c8f 0 2024-05-27T09:29:16.394 INFO:tasks.workunit.client.0.smithi119.stdout:2/236: sync 2024-05-27T09:29:16.394 INFO:tasks.workunit.client.0.smithi119.stdout:7/311: sync 2024-05-27T09:29:16.394 INFO:tasks.workunit.client.0.smithi119.stdout:1/309: sync 2024-05-27T09:29:16.395 INFO:tasks.workunit.client.0.smithi119.stdout:4/213: sync 2024-05-27T09:29:16.395 INFO:tasks.workunit.client.0.smithi119.stdout:5/229: sync 2024-05-27T09:29:16.395 INFO:tasks.workunit.client.0.smithi119.stdout:7/312: fsync d7/df/d19/f50 0 2024-05-27T09:29:16.395 INFO:tasks.workunit.client.0.smithi119.stdout:7/313: chown d7/df/l26 60984 1 2024-05-27T09:29:16.395 INFO:tasks.workunit.client.0.smithi119.stdout:6/211: sync 2024-05-27T09:29:16.395 INFO:tasks.workunit.client.0.smithi119.stdout:3/418: mknod db/d36/d45/d4c/d62/c90 0 2024-05-27T09:29:16.395 INFO:tasks.workunit.client.0.smithi119.stdout:3/419: write db/d36/f71 [1402531,29905] 0 2024-05-27T09:29:16.396 INFO:tasks.workunit.client.0.smithi119.stdout:4/214: symlink d4/d1c/d39/l46 0 2024-05-27T09:29:16.402 INFO:tasks.workunit.client.0.smithi119.stdout:2/237: link d2/da/f2e d2/da/d2c/f55 0 2024-05-27T09:29:16.403 INFO:tasks.workunit.client.0.smithi119.stdout:2/238: chown d2/da/f10 5 1 2024-05-27T09:29:16.403 INFO:tasks.workunit.client.0.smithi119.stdout:7/314: truncate d7/df/d19/f1f 3870757 0 2024-05-27T09:29:16.403 INFO:tasks.workunit.client.0.smithi119.stdout:5/230: symlink d6/dd/df/d41/l4d 0 2024-05-27T09:29:16.403 INFO:tasks.workunit.client.0.smithi119.stdout:3/420: fsync db/f56 0 2024-05-27T09:29:16.403 INFO:tasks.workunit.client.0.smithi119.stdout:1/310: rename d7/db/d12/d15/c2a to d7/dd/d25/d60/d61/c66 0 2024-05-27T09:29:16.403 INFO:tasks.workunit.client.0.smithi119.stdout:3/421: truncate db/d7f/f51 462731 0 2024-05-27T09:29:16.403 INFO:tasks.workunit.client.0.smithi119.stdout:6/212: dread d0/d2/d1c/d21/f3c [0,4194304] 0 2024-05-27T09:29:16.404 INFO:tasks.workunit.client.0.smithi119.stdout:2/239: creat d2/d1a/d51/f56 x:0 0 0 2024-05-27T09:29:16.404 INFO:tasks.workunit.client.0.smithi119.stdout:2/240: chown d2/f26 0 1 2024-05-27T09:29:16.404 INFO:tasks.workunit.client.0.smithi119.stdout:2/241: fsync d2/d1a/f3b 0 2024-05-27T09:29:16.405 INFO:tasks.workunit.client.0.smithi119.stdout:4/215: getdents d4/d1c 0 2024-05-27T09:29:16.408 INFO:tasks.workunit.client.0.smithi119.stdout:7/315: mkdir d7/df/d36/d42/d64 0 2024-05-27T09:29:16.408 INFO:tasks.workunit.client.0.smithi119.stdout:7/316: chown d7/d1a/d40/d44/f5d 991 1 2024-05-27T09:29:16.415 INFO:tasks.workunit.client.0.smithi119.stdout:1/311: mknod d7/dd/d25/d29/d38/d49/c67 0 2024-05-27T09:29:16.415 INFO:tasks.workunit.client.0.smithi119.stdout:1/312: write d7/dd/d44/f56 [900136,50696] 0 2024-05-27T09:29:16.415 INFO:tasks.workunit.client.0.smithi119.stdout:6/213: getdents d0/d19 0 2024-05-27T09:29:16.416 INFO:tasks.workunit.client.0.smithi119.stdout:2/242: rename d2/d1a/f21 to d2/da/d15/d33/f57 0 2024-05-27T09:29:16.416 INFO:tasks.workunit.client.0.smithi119.stdout:2/243: fdatasync d2/da/d15/d33/f57 0 2024-05-27T09:29:16.417 INFO:tasks.workunit.client.0.smithi119.stdout:7/317: rmdir d7/db/d37 39 2024-05-27T09:29:16.418 INFO:tasks.workunit.client.0.smithi119.stdout:7/318: write d7/f62 [983684,36209] 0 2024-05-27T09:29:16.418 INFO:tasks.workunit.client.0.smithi119.stdout:7/319: chown d7/d1a/d40 12 1 2024-05-27T09:29:16.419 INFO:tasks.workunit.client.0.smithi119.stdout:2/244: creat d2/da/d2c/f58 x:0 0 0 2024-05-27T09:29:16.420 INFO:tasks.workunit.client.0.smithi119.stdout:2/245: readlink d2/l24 0 2024-05-27T09:29:16.420 INFO:tasks.workunit.client.0.smithi119.stdout:7/320: mknod d7/df/d36/d42/c65 0 2024-05-27T09:29:16.423 INFO:tasks.workunit.client.0.smithi119.stdout:5/231: getdents d6/dd/df/d41 0 2024-05-27T09:29:16.423 INFO:tasks.workunit.client.0.smithi119.stdout:5/232: chown d6/dd/d39/d19 5 1 2024-05-27T09:29:16.424 INFO:tasks.workunit.client.0.smithi119.stdout:2/246: write d2/f26 [4930377,115663] 0 2024-05-27T09:29:16.426 INFO:tasks.workunit.client.0.smithi119.stdout:5/233: link d6/dd/df/d41/l4d d6/dd/d39/l4e 0 2024-05-27T09:29:16.428 INFO:tasks.workunit.client.0.smithi119.stdout:2/247: truncate d2/da/fd 6785566 0 2024-05-27T09:29:16.428 INFO:tasks.workunit.client.0.smithi119.stdout:2/248: fdatasync d2/d1a/f30 0 2024-05-27T09:29:16.428 INFO:tasks.workunit.client.0.smithi119.stdout:3/422: dread db/f2f [0,4194304] 0 2024-05-27T09:29:16.433 INFO:tasks.workunit.client.0.smithi119.stdout:5/234: symlink d6/dd/d46/d47/d4b/l4f 0 2024-05-27T09:29:16.433 INFO:tasks.workunit.client.0.smithi119.stdout:5/235: chown d6/db/l2c 16 1 2024-05-27T09:29:16.433 INFO:tasks.workunit.client.0.smithi119.stdout:2/249: rename d2/f6 to d2/da/d15/d53/f59 0 2024-05-27T09:29:16.433 INFO:tasks.workunit.client.0.smithi119.stdout:2/250: mknod d2/c5a 0 2024-05-27T09:29:16.433 INFO:tasks.workunit.client.0.smithi119.stdout:2/251: write d2/da/d2c/f3c [1552305,97091] 0 2024-05-27T09:29:16.435 INFO:tasks.workunit.client.0.smithi119.stdout:2/252: mknod d2/da/d2c/c5b 0 2024-05-27T09:29:16.435 INFO:tasks.workunit.client.0.smithi119.stdout:2/253: chown d2/da/fd 13 1 2024-05-27T09:29:16.436 INFO:tasks.workunit.client.0.smithi119.stdout:2/254: mknod d2/da/d15/d53/c5c 0 2024-05-27T09:29:16.441 INFO:tasks.workunit.client.0.smithi119.stdout:3/423: dread db/d36/d45/d4d/f67 [0,4194304] 0 2024-05-27T09:29:16.446 INFO:tasks.workunit.client.0.smithi119.stdout:3/424: mknod db/d36/d45/d4c/d62/c91 0 2024-05-27T09:29:16.446 INFO:tasks.workunit.client.0.smithi119.stdout:3/425: fsync db/d36/d45/d4c/d62/f2d 0 2024-05-27T09:29:16.450 INFO:tasks.workunit.client.0.smithi119.stdout:7/321: write d7/db/f2f [2656706,34389] 0 2024-05-27T09:29:16.451 INFO:tasks.workunit.client.0.smithi119.stdout:7/322: truncate d7/df/f29 1217897 0 2024-05-27T09:29:16.465 INFO:tasks.workunit.client.0.smithi119.stdout:2/255: unlink d2/da/d15/d53/f59 0 2024-05-27T09:29:16.467 INFO:tasks.workunit.client.0.smithi119.stdout:3/426: dread db/d36/d45/d4c/f6c [0,4194304] 0 2024-05-27T09:29:16.467 INFO:tasks.workunit.client.0.smithi119.stdout:3/427: readlink db/d36/d45/d4c/d62/d13/l1f 0 2024-05-27T09:29:16.469 INFO:tasks.workunit.client.0.smithi119.stdout:2/256: rename d2/c5a to d2/da/d15/c5d 0 2024-05-27T09:29:16.469 INFO:tasks.workunit.client.0.smithi119.stdout:2/257: truncate d2/da/d15/d33/f41 263551 0 2024-05-27T09:29:16.471 INFO:tasks.workunit.client.0.smithi119.stdout:3/428: link db/d36/d45/d4c/d62/d13/l8d db/d36/d45/d4c/l92 0 2024-05-27T09:29:16.471 INFO:tasks.workunit.client.0.smithi119.stdout:2/258: rename d2/da/c3f to d2/da/d15/d33/c5e 0 2024-05-27T09:29:16.471 INFO:tasks.workunit.client.0.smithi119.stdout:2/259: chown d2/da 949971 1 2024-05-27T09:29:16.474 INFO:tasks.workunit.client.0.smithi119.stdout:3/429: mkdir db/d36/d45/d93 0 2024-05-27T09:29:16.476 INFO:tasks.workunit.client.0.smithi119.stdout:0/242: dwrite d2/d4/f42 [4194304,4194304] 0 2024-05-27T09:29:16.477 INFO:tasks.workunit.client.0.smithi119.stdout:3/430: unlink db/d36/d45/d4c/d62/f66 0 2024-05-27T09:29:16.483 INFO:tasks.workunit.client.0.smithi119.stdout:0/243: link d2/d4/f46 d2/d1a/f51 0 2024-05-27T09:29:16.483 INFO:tasks.workunit.client.0.smithi119.stdout:7/323: read d7/df/f29 [1040426,20666] 0 2024-05-27T09:29:16.484 INFO:tasks.workunit.client.0.smithi119.stdout:5/236: dread d6/f9 [0,4194304] 0 2024-05-27T09:29:16.486 INFO:tasks.workunit.client.0.smithi119.stdout:6/214: dwrite d0/d2/d24/d37/f43 [0,4194304] 0 2024-05-27T09:29:16.486 INFO:tasks.workunit.client.0.smithi119.stdout:1/313: dwrite d7/dd/d44/f4e [0,4194304] 0 2024-05-27T09:29:16.486 INFO:tasks.workunit.client.0.smithi119.stdout:6/215: creat d0/d11/d26/f4b x:0 0 0 2024-05-27T09:29:16.486 INFO:tasks.workunit.client.0.smithi119.stdout:1/314: stat d7/db/d35 0 2024-05-27T09:29:16.486 INFO:tasks.workunit.client.0.smithi119.stdout:6/216: fdatasync d0/d11/d26/f4b 0 2024-05-27T09:29:16.486 INFO:tasks.workunit.client.0.smithi119.stdout:7/324: unlink d7/db/f31 0 2024-05-27T09:29:16.498 INFO:tasks.workunit.client.0.smithi119.stdout:6/217: rename d0/d2/d24/f3a to d0/d2/d1c/f4c 0 2024-05-27T09:29:16.499 INFO:tasks.workunit.client.0.smithi119.stdout:0/244: write d2/d6/f8 [3136251,100856] 0 2024-05-27T09:29:16.499 INFO:tasks.workunit.client.0.smithi119.stdout:1/315: unlink d7/dd/d44/d47/f63 0 2024-05-27T09:29:16.499 INFO:tasks.workunit.client.0.smithi119.stdout:1/316: truncate d7/db/d35/f3e 4908769 0 2024-05-27T09:29:16.499 INFO:tasks.workunit.client.0.smithi119.stdout:1/317: stat d7/dd/d25/d29/d38/d3c/c50 0 2024-05-27T09:29:16.499 INFO:tasks.workunit.client.0.smithi119.stdout:1/318: truncate d7/dd/f5d 965931 0 2024-05-27T09:29:16.502 INFO:tasks.workunit.client.0.smithi119.stdout:0/245: rename d2/l48 to d2/d6/d9/l52 0 2024-05-27T09:29:16.502 INFO:tasks.workunit.client.0.smithi119.stdout:5/237: rmdir d6/dd/d39/d19/d3c 39 2024-05-27T09:29:16.502 INFO:tasks.workunit.client.0.smithi119.stdout:5/238: read d6/f1d [3764387,83232] 0 2024-05-27T09:29:16.508 INFO:tasks.workunit.client.0.smithi119.stdout:1/319: truncate d7/f27 422655 0 2024-05-27T09:29:16.508 INFO:tasks.workunit.client.0.smithi119.stdout:5/239: rename d6/db/de/f14 to d6/db/de/d27/f50 0 2024-05-27T09:29:16.508 INFO:tasks.workunit.client.0.smithi119.stdout:5/240: creat d6/d2f/f51 x:0 0 0 2024-05-27T09:29:16.508 INFO:tasks.workunit.client.0.smithi119.stdout:5/241: fsync d6/dd/f33 0 2024-05-27T09:29:16.508 INFO:tasks.workunit.client.0.smithi119.stdout:5/242: readlink d6/db/de/l34 0 2024-05-27T09:29:16.529 INFO:tasks.workunit.client.0.smithi119.stdout:7/325: dread d7/f62 [0,4194304] 0 2024-05-27T09:29:16.529 INFO:tasks.workunit.client.0.smithi119.stdout:7/326: truncate d7/d1a/d40/d44/f5d 177449 0 2024-05-27T09:29:16.529 INFO:tasks.workunit.client.0.smithi119.stdout:7/327: truncate d7/db/d37/f5e 761504 0 2024-05-27T09:29:16.530 INFO:tasks.workunit.client.0.smithi119.stdout:7/328: creat d7/f66 x:0 0 0 2024-05-27T09:29:16.535 INFO:tasks.workunit.client.0.smithi119.stdout:5/243: dread d6/f12 [0,4194304] 0 2024-05-27T09:29:16.535 INFO:tasks.workunit.client.0.smithi119.stdout:5/244: chown d6/c7 10020344 1 2024-05-27T09:29:16.536 INFO:tasks.workunit.client.0.smithi119.stdout:5/245: fdatasync d6/db/de/f25 0 2024-05-27T09:29:16.536 INFO:tasks.workunit.client.0.smithi119.stdout:5/246: readlink d6/db/l23 0 2024-05-27T09:29:16.549 INFO:tasks.workunit.client.0.smithi119.stdout:3/431: dwrite db/d36/d45/d4c/d62/f2d [0,4194304] 0 2024-05-27T09:29:16.550 INFO:tasks.workunit.client.0.smithi119.stdout:7/329: rmdir d7 39 2024-05-27T09:29:16.553 INFO:tasks.workunit.client.0.smithi119.stdout:7/330: mkdir d7/d1a/d40/d44/d67 0 2024-05-27T09:29:16.554 INFO:tasks.workunit.client.0.smithi119.stdout:7/331: creat d7/df/d19/d46/f68 x:0 0 0 2024-05-27T09:29:16.555 INFO:tasks.workunit.client.0.smithi119.stdout:3/432: read db/f69 [137459,11276] 0 2024-05-27T09:29:16.557 INFO:tasks.workunit.client.0.smithi119.stdout:3/433: link db/l18 db/d36/d45/d4c/d62/d13/d33/l94 0 2024-05-27T09:29:16.557 INFO:tasks.workunit.client.0.smithi119.stdout:3/434: chown db/d36/d45/d4c/d62/d13/f2e 2039980 1 2024-05-27T09:29:16.567 INFO:tasks.workunit.client.0.smithi119.stdout:0/246: dwrite d2/d6/f3c [0,4194304] 0 2024-05-27T09:29:16.569 INFO:tasks.workunit.client.0.smithi119.stdout:0/247: mkdir d2/d4/d53 0 2024-05-27T09:29:16.570 INFO:tasks.workunit.client.0.smithi119.stdout:0/248: creat d2/d4/d53/f54 x:0 0 0 2024-05-27T09:29:16.570 INFO:tasks.workunit.client.0.smithi119.stdout:0/249: fsync d2/d6/d9/d12/f32 0 2024-05-27T09:29:16.570 INFO:tasks.workunit.client.0.smithi119.stdout:0/250: dread - d2/d6/d9/d12/f47 zero size 2024-05-27T09:29:16.574 INFO:tasks.workunit.client.0.smithi119.stdout:3/435: dread db/d7f/f5a [0,4194304] 0 2024-05-27T09:29:16.575 INFO:tasks.workunit.client.0.smithi119.stdout:0/251: link d2/f3d d2/d6/f55 0 2024-05-27T09:29:16.575 INFO:tasks.workunit.client.0.smithi119.stdout:0/252: chown d2/d6/d23 1 1 2024-05-27T09:29:16.577 INFO:tasks.workunit.client.0.smithi119.stdout:0/253: rename d2/l15 to d2/d6/d9/d24/l56 0 2024-05-27T09:29:16.577 INFO:tasks.workunit.client.0.smithi119.stdout:0/254: read - d2/d6/d23/f33 zero size 2024-05-27T09:29:16.623 INFO:tasks.workunit.client.0.smithi119.stdout:2/260: dwrite d2/d1a/d51/f56 [0,4194304] 0 2024-05-27T09:29:16.624 INFO:tasks.workunit.client.0.smithi119.stdout:6/218: dwrite d0/d11/d26/f4b [0,4194304] 0 2024-05-27T09:29:16.624 INFO:tasks.workunit.client.0.smithi119.stdout:6/219: fsync d0/fb 0 2024-05-27T09:29:16.640 INFO:tasks.workunit.client.0.smithi119.stdout:7/332: dwrite d7/d1a/d40/f48 [4194304,4194304] 0 2024-05-27T09:29:16.641 INFO:tasks.workunit.client.0.smithi119.stdout:8/251: sync 2024-05-27T09:29:16.642 INFO:tasks.workunit.client.0.smithi119.stdout:7/333: mkdir d7/df/d36/d69 0 2024-05-27T09:29:16.646 INFO:tasks.workunit.client.0.smithi119.stdout:8/252: stat d3/d8/d1b/d46/f4a 0 2024-05-27T09:29:16.660 INFO:tasks.workunit.client.0.smithi119.stdout:8/253: write d3/d8/d1b/f20 [191436,65858] 0 2024-05-27T09:29:16.661 INFO:tasks.workunit.client.0.smithi119.stdout:8/254: write d3/d8/d1b/d46/f4b [240925,57122] 0 2024-05-27T09:29:16.662 INFO:tasks.workunit.client.0.smithi119.stdout:8/255: chown d3/d8/d1b 5392 1 2024-05-27T09:29:16.666 INFO:tasks.workunit.client.0.smithi119.stdout:8/256: mknod d3/c54 0 2024-05-27T09:29:16.666 INFO:tasks.workunit.client.0.smithi119.stdout:8/257: creat d3/d8/d1b/f55 x:0 0 0 2024-05-27T09:29:16.671 INFO:tasks.workunit.client.0.smithi119.stdout:1/320: dwrite d7/f11 [0,4194304] 0 2024-05-27T09:29:16.701 INFO:tasks.workunit.client.0.smithi119.stdout:6/220: dwrite d0/d11/d26/f4b [4194304,4194304] 0 2024-05-27T09:29:16.706 INFO:tasks.workunit.client.0.smithi119.stdout:6/221: mknod d0/d11/d26/c4d 0 2024-05-27T09:29:16.710 INFO:tasks.workunit.client.0.smithi119.stdout:5/247: dwrite d6/d2f/f51 [0,4194304] 0 2024-05-27T09:29:16.718 INFO:tasks.workunit.client.0.smithi119.stdout:5/248: mkdir d6/dd/d39/d19/d3c/d52 0 2024-05-27T09:29:16.718 INFO:tasks.workunit.client.0.smithi119.stdout:5/249: chown d6/dd/f1b 2 1 2024-05-27T09:29:16.721 INFO:tasks.workunit.client.0.smithi119.stdout:5/250: mknod d6/db/de/c53 0 2024-05-27T09:29:16.727 INFO:tasks.workunit.client.0.smithi119.stdout:9/203: sync 2024-05-27T09:29:16.727 INFO:tasks.workunit.client.0.smithi119.stdout:4/216: sync 2024-05-27T09:29:16.728 INFO:tasks.workunit.client.0.smithi119.stdout:9/204: write d18/d19/f1c [4416455,112060] 0 2024-05-27T09:29:16.728 INFO:tasks.workunit.client.0.smithi119.stdout:9/205: getdents d18/d19/d1b/d40 0 2024-05-27T09:29:16.728 INFO:tasks.workunit.client.0.smithi119.stdout:6/222: write d0/f6 [3600493,82708] 0 2024-05-27T09:29:16.728 INFO:tasks.workunit.client.0.smithi119.stdout:6/223: write d0/d2/d1c/f4c [645216,66957] 0 2024-05-27T09:29:16.729 INFO:tasks.workunit.client.0.smithi119.stdout:8/258: dwrite d3/d8/d1b/f41 [0,4194304] 0 2024-05-27T09:29:16.729 INFO:tasks.workunit.client.0.smithi119.stdout:4/217: unlink d4/d6/cb 0 2024-05-27T09:29:16.732 INFO:tasks.workunit.client.0.smithi119.stdout:9/206: truncate d18/d19/f35 342153 0 2024-05-27T09:29:16.740 INFO:tasks.workunit.client.0.smithi119.stdout:4/218: mkdir d4/de/d21/d40/d47 0 2024-05-27T09:29:16.741 INFO:tasks.workunit.client.0.smithi119.stdout:1/321: dwrite d7/dd/d44/f56 [0,4194304] 0 2024-05-27T09:29:16.742 INFO:tasks.workunit.client.0.smithi119.stdout:9/207: getdents d18/d19 0 2024-05-27T09:29:16.742 INFO:tasks.workunit.client.0.smithi119.stdout:9/208: creat d18/f41 x:0 0 0 2024-05-27T09:29:16.746 INFO:tasks.workunit.client.0.smithi119.stdout:6/224: rename d0/d11/d26 to d0/d2/d4e 0 2024-05-27T09:29:16.747 INFO:tasks.workunit.client.0.smithi119.stdout:6/225: fsync d0/d2/d1c/d21/f2b 0 2024-05-27T09:29:16.747 INFO:tasks.workunit.client.0.smithi119.stdout:6/226: chown d0/d2/d4e 3109 1 2024-05-27T09:29:16.747 INFO:tasks.workunit.client.0.smithi119.stdout:6/227: dread - d0/d2/d24/f4a zero size 2024-05-27T09:29:16.747 INFO:tasks.workunit.client.0.smithi119.stdout:6/228: creat d0/d2/f4f x:0 0 0 2024-05-27T09:29:16.747 INFO:tasks.workunit.client.0.smithi119.stdout:4/219: mkdir d4/d48 0 2024-05-27T09:29:16.747 INFO:tasks.workunit.client.0.smithi119.stdout:5/251: write d6/d2f/f51 [3189475,22494] 0 2024-05-27T09:29:16.750 INFO:tasks.workunit.client.0.smithi119.stdout:1/322: unlink d7/db/d12/f23 0 2024-05-27T09:29:16.765 INFO:tasks.workunit.client.0.smithi119.stdout:4/220: truncate d4/d6/f11 1895834 0 2024-05-27T09:29:16.766 INFO:tasks.workunit.client.0.smithi119.stdout:8/259: read d3/d8/d1b/d46/f2b [1589978,116689] 0 2024-05-27T09:29:16.766 INFO:tasks.workunit.client.0.smithi119.stdout:4/221: write d4/d1c/f29 [1064086,127182] 0 2024-05-27T09:29:16.768 INFO:tasks.workunit.client.0.smithi119.stdout:8/260: symlink d3/d39/l56 0 2024-05-27T09:29:16.769 INFO:tasks.workunit.client.0.smithi119.stdout:4/222: truncate d4/d23/d31/f33 3795422 0 2024-05-27T09:29:16.774 INFO:tasks.workunit.client.0.smithi119.stdout:8/261: link d3/d8/lc d3/d8/d4d/l57 0 2024-05-27T09:29:16.789 INFO:tasks.workunit.client.0.smithi119.stdout:8/262: dread d3/d8/f34 [0,4194304] 0 2024-05-27T09:29:16.790 INFO:tasks.workunit.client.0.smithi119.stdout:8/263: rmdir d3/d8/d1b/d4c 0 2024-05-27T09:29:16.792 INFO:tasks.workunit.client.0.smithi119.stdout:8/264: mkdir d3/d8/d4d/d58 0 2024-05-27T09:29:16.794 INFO:tasks.workunit.client.0.smithi119.stdout:8/265: creat d3/d8/d4d/f59 x:0 0 0 2024-05-27T09:29:16.796 INFO:tasks.workunit.client.0.smithi119.stdout:8/266: mkdir d3/d39/d5a 0 2024-05-27T09:29:16.797 INFO:tasks.workunit.client.0.smithi119.stdout:8/267: chown d3/c6 48702471 1 2024-05-27T09:29:16.797 INFO:tasks.workunit.client.0.smithi119.stdout:8/268: truncate d3/d8/d1b/d46/f4a 1039157 0 2024-05-27T09:29:16.805 INFO:tasks.workunit.client.0.smithi119.stdout:6/229: dwrite d0/d2/d1c/d21/f45 [0,4194304] 0 2024-05-27T09:29:16.808 INFO:tasks.workunit.client.0.smithi119.stdout:3/436: sync 2024-05-27T09:29:16.809 INFO:tasks.workunit.client.0.smithi119.stdout:3/437: symlink db/d36/d45/d93/l95 0 2024-05-27T09:29:16.816 INFO:tasks.workunit.client.0.smithi119.stdout:4/223: read d4/d1c/f29 [836449,76277] 0 2024-05-27T09:29:16.816 INFO:tasks.workunit.client.0.smithi119.stdout:4/224: chown d4/de/l1b 5 1 2024-05-27T09:29:16.820 INFO:tasks.workunit.client.0.smithi119.stdout:1/323: dwrite d7/d40/f5e [0,4194304] 0 2024-05-27T09:29:16.826 INFO:tasks.workunit.client.0.smithi119.stdout:4/225: rmdir d4/d23 39 2024-05-27T09:29:16.826 INFO:tasks.workunit.client.0.smithi119.stdout:4/226: chown d4 3 1 2024-05-27T09:29:16.826 INFO:tasks.workunit.client.0.smithi119.stdout:4/227: chown d4/d6/fd 73939 1 2024-05-27T09:29:16.829 INFO:tasks.workunit.client.0.smithi119.stdout:8/269: dread d3/d8/d4d/f32 [0,4194304] 0 2024-05-27T09:29:16.832 INFO:tasks.workunit.client.0.smithi119.stdout:3/438: dread db/d7f/f51 [0,4194304] 0 2024-05-27T09:29:16.832 INFO:tasks.workunit.client.0.smithi119.stdout:1/324: dread d7/db/d12/d26/f43 [0,4194304] 0 2024-05-27T09:29:16.832 INFO:tasks.workunit.client.0.smithi119.stdout:1/325: dread - d7/dd/d25/d29/d38/d3c/f53 zero size 2024-05-27T09:29:16.837 INFO:tasks.workunit.client.0.smithi119.stdout:8/270: fsync d3/f15 0 2024-05-27T09:29:16.837 INFO:tasks.workunit.client.0.smithi119.stdout:8/271: chown d3/d39 29 1 2024-05-27T09:29:16.839 INFO:tasks.workunit.client.0.smithi119.stdout:5/252: dwrite d6/f22 [0,4194304] 0 2024-05-27T09:29:16.839 INFO:tasks.workunit.client.0.smithi119.stdout:5/253: write d6/dd/f1b [1221893,124701] 0 2024-05-27T09:29:16.839 INFO:tasks.workunit.client.0.smithi119.stdout:5/254: chown d6/db/dc/l44 3685819 1 2024-05-27T09:29:16.845 INFO:tasks.workunit.client.0.smithi119.stdout:8/272: symlink d3/d8/l5b 0 2024-05-27T09:29:16.848 INFO:tasks.workunit.client.0.smithi119.stdout:1/326: write f2 [4151600,15073] 0 2024-05-27T09:29:16.848 INFO:tasks.workunit.client.0.smithi119.stdout:5/255: creat d6/dd/df/f54 x:0 0 0 2024-05-27T09:29:16.852 INFO:tasks.workunit.client.0.smithi119.stdout:5/256: rename d6/dd/d39/d19/l29 to d6/dd/d39/d19/d3c/l55 0 2024-05-27T09:29:16.859 INFO:tasks.workunit.client.0.smithi119.stdout:9/209: dwrite fe [0,4194304] 0 2024-05-27T09:29:16.859 INFO:tasks.workunit.client.0.smithi119.stdout:3/439: write db/d36/d45/d4c/d62/d13/f3d [3118008,119906] 0 2024-05-27T09:29:16.860 INFO:tasks.workunit.client.0.smithi119.stdout:3/440: chown db/d36/d45/d49/d4f/f78 223620 1 2024-05-27T09:29:16.868 INFO:tasks.workunit.client.0.smithi119.stdout:9/210: link d18/d19/d1b/d2a/l3e d18/l42 0 2024-05-27T09:29:16.868 INFO:tasks.workunit.client.0.smithi119.stdout:9/211: stat ff 0 2024-05-27T09:29:16.868 INFO:tasks.workunit.client.0.smithi119.stdout:3/441: mknod db/d7f/c96 0 2024-05-27T09:29:16.870 INFO:tasks.workunit.client.0.smithi119.stdout:9/212: creat d18/d19/d1b/d29/f43 x:0 0 0 2024-05-27T09:29:16.870 INFO:tasks.workunit.client.0.smithi119.stdout:9/213: read - d18/f41 zero size 2024-05-27T09:29:16.874 INFO:tasks.workunit.client.0.smithi119.stdout:3/442: rename db/d36/d45/d4c/d62/d13/d33/f40 to db/d36/d45/d4c/d62/d13/f97 0 2024-05-27T09:29:16.874 INFO:tasks.workunit.client.0.smithi119.stdout:3/443: stat db/d36/d45/d4c/d62/d13/c42 0 2024-05-27T09:29:16.874 INFO:tasks.workunit.client.0.smithi119.stdout:3/444: write db/d36/d45/d4c/d62/f8e [1014642,50047] 0 2024-05-27T09:29:16.874 INFO:tasks.workunit.client.0.smithi119.stdout:3/445: fdatasync db/d6f/f80 0 2024-05-27T09:29:16.875 INFO:tasks.workunit.client.0.smithi119.stdout:9/214: mkdir d18/d19/d1b/d40/d44 0 2024-05-27T09:29:16.875 INFO:tasks.workunit.client.0.smithi119.stdout:0/255: sync 2024-05-27T09:29:16.877 INFO:tasks.workunit.client.0.smithi119.stdout:9/215: rename d18/d19/d1b/d29/f37 to d18/d19/d1b/d40/d44/f45 0 2024-05-27T09:29:16.880 INFO:tasks.workunit.client.0.smithi119.stdout:0/256: rename d2/d6/d23/l2e to d2/d6/d9/d24/l57 0 2024-05-27T09:29:16.882 INFO:tasks.workunit.client.0.smithi119.stdout:0/257: getdents d2/d6/d9 0 2024-05-27T09:29:16.882 INFO:tasks.workunit.client.0.smithi119.stdout:0/258: chown d2/d6/d9/d24/f40 1 1 2024-05-27T09:29:16.889 INFO:tasks.workunit.client.0.smithi119.stdout:4/228: dwrite d4/d6/f11 [0,4194304] 0 2024-05-27T09:29:16.891 INFO:tasks.workunit.client.0.smithi119.stdout:4/229: truncate d4/d1c/f1e 2443569 0 2024-05-27T09:29:16.891 INFO:tasks.workunit.client.0.smithi119.stdout:4/230: fdatasync d4/d23/d31/f33 0 2024-05-27T09:29:16.896 INFO:tasks.workunit.client.0.smithi119.stdout:9/216: read f10 [619670,45573] 0 2024-05-27T09:29:16.932 INFO:tasks.workunit.client.0.smithi119.stdout:1/327: dwrite d7/f1e [0,4194304] 0 2024-05-27T09:29:16.933 INFO:tasks.workunit.client.0.smithi119.stdout:8/273: dwrite d3/d8/d1b/f41 [0,4194304] 0 2024-05-27T09:29:16.937 INFO:tasks.workunit.client.0.smithi119.stdout:8/274: mknod d3/d49/c5c 0 2024-05-27T09:29:16.951 INFO:tasks.workunit.client.0.smithi119.stdout:9/217: dwrite fe [4194304,4194304] 0 2024-05-27T09:29:16.951 INFO:tasks.workunit.client.0.smithi119.stdout:9/218: creat d18/f46 x:0 0 0 2024-05-27T09:29:16.951 INFO:tasks.workunit.client.0.smithi119.stdout:9/219: readlink d18/d19/d1b/d29/l33 0 2024-05-27T09:29:16.952 INFO:tasks.workunit.client.0.smithi119.stdout:8/275: rename d3/d49 to d3/d8/d4d/d5d 0 2024-05-27T09:29:16.956 INFO:tasks.workunit.client.0.smithi119.stdout:8/276: symlink d3/de/l5e 0 2024-05-27T09:29:16.956 INFO:tasks.workunit.client.0.smithi119.stdout:8/277: readlink d3/d8/d4d/d5d/l4e 0 2024-05-27T09:29:16.956 INFO:tasks.workunit.client.0.smithi119.stdout:1/328: dread d7/dd/d25/f30 [0,4194304] 0 2024-05-27T09:29:16.957 INFO:tasks.workunit.client.0.smithi119.stdout:8/278: getdents d3/d8/d4d/d5d 0 2024-05-27T09:29:16.965 INFO:tasks.workunit.client.0.smithi119.stdout:4/231: dwrite d4/d6/fd [0,4194304] 0 2024-05-27T09:29:16.979 INFO:tasks.workunit.client.0.smithi119.stdout:8/279: write d3/d8/d1b/d46/f29 [2185083,35579] 0 2024-05-27T09:29:16.979 INFO:tasks.workunit.client.0.smithi119.stdout:8/280: readlink d3/d39/l56 0 2024-05-27T09:29:16.992 INFO:tasks.workunit.client.0.smithi119.stdout:8/281: write d3/f16 [1900550,46855] 0 2024-05-27T09:29:16.992 INFO:tasks.workunit.client.0.smithi119.stdout:8/282: chown d3/d8/c3b 274141761 1 2024-05-27T09:29:16.992 INFO:tasks.workunit.client.0.smithi119.stdout:4/232: creat d4/de/d21/d40/d47/f49 x:0 0 0 2024-05-27T09:29:16.998 INFO:tasks.workunit.client.0.smithi119.stdout:4/233: dread d4/de/d21/f2d [0,4194304] 0 2024-05-27T09:29:17.042 INFO:tasks.workunit.client.0.smithi119.stdout:8/283: symlink d3/d8/d4d/d5d/l5f 0 2024-05-27T09:29:17.042 INFO:tasks.workunit.client.0.smithi119.stdout:8/284: chown d3/d39 18863 1 2024-05-27T09:29:17.043 INFO:tasks.workunit.client.0.smithi119.stdout:4/234: mkdir d4/d6/d28/d4a 0 2024-05-27T09:29:17.049 INFO:tasks.workunit.client.0.smithi119.stdout:4/235: truncate d4/f32 3220547 0 2024-05-27T09:29:17.049 INFO:tasks.workunit.client.0.smithi119.stdout:4/236: dread - d4/f41 zero size 2024-05-27T09:29:17.052 INFO:tasks.workunit.client.0.smithi119.stdout:4/237: creat d4/d1c/d39/f4b x:0 0 0 2024-05-27T09:29:17.052 INFO:tasks.workunit.client.0.smithi119.stdout:4/238: chown d4/d6/d28/d2f 65311446 1 2024-05-27T09:29:17.058 INFO:tasks.workunit.client.0.smithi119.stdout:8/285: dread d3/d8/d1b/d46/f29 [0,4194304] 0 2024-05-27T09:29:17.062 INFO:tasks.workunit.client.0.smithi119.stdout:8/286: dread d3/d8/d1b/f28 [0,4194304] 0 2024-05-27T09:29:17.065 INFO:tasks.workunit.client.0.smithi119.stdout:8/287: truncate d3/f1a 190855 0 2024-05-27T09:29:17.065 INFO:tasks.workunit.client.0.smithi119.stdout:8/288: write d3/d8/f34 [188681,46466] 0 2024-05-27T09:29:17.079 INFO:tasks.workunit.client.0.smithi119.stdout:8/289: dread d3/d8/d1b/d46/f37 [0,4194304] 0 2024-05-27T09:29:17.080 INFO:tasks.workunit.client.0.smithi119.stdout:8/290: mkdir d3/d8/d4d/d5d/d60 0 2024-05-27T09:29:17.099 INFO:tasks.workunit.client.0.smithi119.stdout:1/329: rename l4 to d7/dd/l68 0 2024-05-27T09:29:17.102 INFO:tasks.workunit.client.0.smithi119.stdout:4/239: rename d4/d23/f45 to d4/f4c 0 2024-05-27T09:29:17.104 INFO:tasks.workunit.client.0.smithi119.stdout:8/291: rename d3/d8/d1b/d46/f4a to d3/d39/f61 0 2024-05-27T09:29:17.104 INFO:tasks.workunit.client.0.smithi119.stdout:8/292: stat d3/d8/f47 0 2024-05-27T09:29:17.105 INFO:tasks.workunit.client.0.smithi119.stdout:4/240: symlink d4/d23/d31/l4d 0 2024-05-27T09:29:17.107 INFO:tasks.workunit.client.0.smithi119.stdout:8/293: creat d3/de/f62 x:0 0 0 2024-05-27T09:29:17.108 INFO:tasks.workunit.client.0.smithi119.stdout:4/241: mkdir d4/d4e 0 2024-05-27T09:29:17.109 INFO:tasks.workunit.client.0.smithi119.stdout:8/294: write d3/f1a [1106346,59077] 0 2024-05-27T09:29:17.109 INFO:tasks.workunit.client.0.smithi119.stdout:8/295: readlink d3/d39/l56 0 2024-05-27T09:29:17.110 INFO:tasks.workunit.client.0.smithi119.stdout:1/330: rename d7/db/d12/d15/f21 to d7/db/d35/f69 0 2024-05-27T09:29:17.110 INFO:tasks.workunit.client.0.smithi119.stdout:1/331: chown d7/f5b 53 1 2024-05-27T09:29:17.112 INFO:tasks.workunit.client.0.smithi119.stdout:8/296: creat d3/d8/d4d/d58/f63 x:0 0 0 2024-05-27T09:29:17.113 INFO:tasks.workunit.client.0.smithi119.stdout:1/332: symlink d7/d40/l6a 0 2024-05-27T09:29:17.113 INFO:tasks.workunit.client.0.smithi119.stdout:1/333: fsync d7/db/d12/d15/f3f 0 2024-05-27T09:29:17.115 INFO:tasks.workunit.client.0.smithi119.stdout:8/297: truncate d3/d8/d1b/d46/f2b 2055011 0 2024-05-27T09:29:17.116 INFO:tasks.workunit.client.0.smithi119.stdout:1/334: creat d7/db/d12/f6b x:0 0 0 2024-05-27T09:29:17.116 INFO:tasks.workunit.client.0.smithi119.stdout:1/335: truncate d7/dd/d25/d29/f55 825869 0 2024-05-27T09:29:17.116 INFO:tasks.workunit.client.0.smithi119.stdout:1/336: stat d7/dd/d25/d29/d38/d3c 0 2024-05-27T09:29:17.117 INFO:tasks.workunit.client.0.smithi119.stdout:1/337: chown c1 0 1 2024-05-27T09:29:17.120 INFO:tasks.workunit.client.0.smithi119.stdout:1/338: unlink d7/d40/f5e 0 2024-05-27T09:29:17.120 INFO:tasks.workunit.client.0.smithi119.stdout:1/339: fdatasync f2 0 2024-05-27T09:29:17.122 INFO:tasks.workunit.client.0.smithi119.stdout:8/298: unlink d3/l9 0 2024-05-27T09:29:17.166 INFO:tasks.workunit.client.0.smithi119.stdout:2/261: sync 2024-05-27T09:29:17.170 INFO:tasks.workunit.client.0.smithi119.stdout:2/262: creat d2/da/d15/f5f x:0 0 0 2024-05-27T09:29:17.176 INFO:tasks.workunit.client.0.smithi119.stdout:2/263: rename d2/da/d15/f29 to d2/da/d15/d33/f60 0 2024-05-27T09:29:17.189 INFO:tasks.workunit.client.0.smithi119.stdout:4/242: dwrite f3 [0,4194304] 0 2024-05-27T09:29:17.192 INFO:tasks.workunit.client.0.smithi119.stdout:4/243: rmdir d4 39 2024-05-27T09:29:17.192 INFO:tasks.workunit.client.0.smithi119.stdout:4/244: write d4/d6/d28/f34 [3223318,114500] 0 2024-05-27T09:29:17.193 INFO:tasks.workunit.client.0.smithi119.stdout:4/245: chown d4/d23/d24/l43 58246 1 2024-05-27T09:29:17.193 INFO:tasks.workunit.client.0.smithi119.stdout:4/246: dread - d4/f4c zero size 2024-05-27T09:29:17.193 INFO:tasks.workunit.client.0.smithi119.stdout:4/247: chown d4/d6/d3e 241 1 2024-05-27T09:29:17.195 INFO:tasks.workunit.client.0.smithi119.stdout:4/248: mknod d4/d6/c4f 0 2024-05-27T09:29:17.198 INFO:tasks.workunit.client.0.smithi119.stdout:4/249: getdents d4/de/d21/d40/d47 0 2024-05-27T09:29:17.208 INFO:tasks.workunit.client.0.smithi119.stdout:2/264: dread d2/da/d15/f18 [0,4194304] 0 2024-05-27T09:29:17.208 INFO:tasks.workunit.client.0.smithi119.stdout:2/265: chown d2/da/f28 1473830 1 2024-05-27T09:29:17.208 INFO:tasks.workunit.client.0.smithi119.stdout:2/266: creat d2/d1a/f61 x:0 0 0 2024-05-27T09:29:17.210 INFO:tasks.workunit.client.0.smithi119.stdout:1/340: dwrite d7/dd/d25/d29/f55 [0,4194304] 0 2024-05-27T09:29:17.215 INFO:tasks.workunit.client.0.smithi119.stdout:1/341: creat d7/db/f6c x:0 0 0 2024-05-27T09:29:17.252 INFO:tasks.workunit.client.0.smithi119.stdout:4/250: dwrite d4/d23/f35 [0,4194304] 0 2024-05-27T09:29:17.252 INFO:tasks.workunit.client.0.smithi119.stdout:7/334: sync 2024-05-27T09:29:17.255 INFO:tasks.workunit.client.0.smithi119.stdout:4/251: creat d4/d6/d28/d2f/f50 x:0 0 0 2024-05-27T09:29:17.256 INFO:tasks.workunit.client.0.smithi119.stdout:7/335: mkdir d7/d6a 0 2024-05-27T09:29:17.257 INFO:tasks.workunit.client.0.smithi119.stdout:4/252: creat d4/d48/f51 x:0 0 0 2024-05-27T09:29:17.257 INFO:tasks.workunit.client.0.smithi119.stdout:4/253: readlink d4/de/l1b 0 2024-05-27T09:29:17.259 INFO:tasks.workunit.client.0.smithi119.stdout:7/336: link d7/d1a/f59 d7/f6b 0 2024-05-27T09:29:17.259 INFO:tasks.workunit.client.0.smithi119.stdout:1/342: write f3 [3788687,75158] 0 2024-05-27T09:29:17.261 INFO:tasks.workunit.client.0.smithi119.stdout:4/254: creat d4/d23/f52 x:0 0 0 2024-05-27T09:29:17.265 INFO:tasks.workunit.client.0.smithi119.stdout:7/337: symlink d7/df/d36/l6c 0 2024-05-27T09:29:17.265 INFO:tasks.workunit.client.0.smithi119.stdout:7/338: chown d7/df/d36/d69 73 1 2024-05-27T09:29:17.265 INFO:tasks.workunit.client.0.smithi119.stdout:2/267: dwrite d2/da/f37 [0,4194304] 0 2024-05-27T09:29:17.265 INFO:tasks.workunit.client.0.smithi119.stdout:2/268: stat d2/da/d2c/c49 0 2024-05-27T09:29:17.265 INFO:tasks.workunit.client.0.smithi119.stdout:2/269: dread - d2/d1a/f61 zero size 2024-05-27T09:29:17.271 INFO:tasks.workunit.client.0.smithi119.stdout:4/255: getdents d4/de 0 2024-05-27T09:29:17.272 INFO:tasks.workunit.client.0.smithi119.stdout:7/339: dread d7/d1a/f1c [0,4194304] 0 2024-05-27T09:29:17.272 INFO:tasks.workunit.client.0.smithi119.stdout:7/340: write d7/f66 [730157,122271] 0 2024-05-27T09:29:17.272 INFO:tasks.workunit.client.0.smithi119.stdout:7/341: truncate d7/df/d19/d1d/f4f 139353 0 2024-05-27T09:29:17.273 INFO:tasks.workunit.client.0.smithi119.stdout:2/270: getdents d2/da/d15/d53 0 2024-05-27T09:29:17.275 INFO:tasks.workunit.client.0.smithi119.stdout:2/271: rmdir d2 39 2024-05-27T09:29:17.288 INFO:tasks.workunit.client.0.smithi119.stdout:4/256: dread d4/d23/d31/f33 [0,4194304] 0 2024-05-27T09:29:17.293 INFO:tasks.workunit.client.0.smithi119.stdout:4/257: creat d4/d23/d31/f53 x:0 0 0 2024-05-27T09:29:17.293 INFO:tasks.workunit.client.0.smithi119.stdout:4/258: write d4/f22 [6099252,26455] 0 2024-05-27T09:29:17.293 INFO:tasks.workunit.client.0.smithi119.stdout:4/259: dread - d4/d23/d31/f53 zero size 2024-05-27T09:29:17.294 INFO:tasks.workunit.client.0.smithi119.stdout:4/260: symlink d4/de/l54 0 2024-05-27T09:29:17.296 INFO:tasks.workunit.client.0.smithi119.stdout:4/261: mknod d4/de/d21/c55 0 2024-05-27T09:29:17.300 INFO:tasks.workunit.client.0.smithi119.stdout:4/262: symlink d4/de/l56 0 2024-05-27T09:29:17.300 INFO:tasks.workunit.client.0.smithi119.stdout:4/263: creat d4/d1c/d39/f57 x:0 0 0 2024-05-27T09:29:17.303 INFO:tasks.workunit.client.0.smithi119.stdout:4/264: getdents d4/d6/d28/d2f 0 2024-05-27T09:29:17.312 INFO:tasks.workunit.client.0.smithi119.stdout:7/342: dread d7/df/d19/f14 [0,4194304] 0 2024-05-27T09:29:17.322 INFO:tasks.workunit.client.0.smithi119.stdout:7/343: link d7/df/d19/l34 d7/d1a/d40/d44/d67/l6d 0 2024-05-27T09:29:17.322 INFO:tasks.workunit.client.0.smithi119.stdout:7/344: fdatasync d7/df/d36/f61 0 2024-05-27T09:29:17.326 INFO:tasks.workunit.client.0.smithi119.stdout:2/272: dwrite d2/f47 [0,4194304] 0 2024-05-27T09:29:17.327 INFO:tasks.workunit.client.0.smithi119.stdout:2/273: truncate d2/da/d4e/f4f 4458742 0 2024-05-27T09:29:17.327 INFO:tasks.workunit.client.0.smithi119.stdout:7/345: rmdir d7/d1a/d40/d44/d51 0 2024-05-27T09:29:17.327 INFO:tasks.workunit.client.0.smithi119.stdout:7/346: write d7/d1a/f20 [5099814,119012] 0 2024-05-27T09:29:17.327 INFO:tasks.workunit.client.0.smithi119.stdout:7/347: write d7/db/d37/f55 [329757,81747] 0 2024-05-27T09:29:17.333 INFO:tasks.workunit.client.0.smithi119.stdout:2/274: symlink d2/d1a/d51/l62 0 2024-05-27T09:29:17.337 INFO:tasks.workunit.client.0.smithi119.stdout:7/348: rename d7/d1a/d40/c47 to d7/df/d36/d42/c6e 0 2024-05-27T09:29:17.350 INFO:tasks.workunit.client.0.smithi119.stdout:4/265: dwrite d4/d23/f26 [4194304,4194304] 0 2024-05-27T09:29:17.350 INFO:tasks.workunit.client.0.smithi119.stdout:4/266: chown d4/d6/d28/d2f 350842 1 2024-05-27T09:29:17.350 INFO:tasks.workunit.client.0.smithi119.stdout:4/267: write d4/d23/f52 [878547,101317] 0 2024-05-27T09:29:17.351 INFO:tasks.workunit.client.0.smithi119.stdout:4/268: write d4/de/d21/d40/f42 [805274,102735] 0 2024-05-27T09:29:17.358 INFO:tasks.workunit.client.0.smithi119.stdout:5/257: sync 2024-05-27T09:29:17.358 INFO:tasks.workunit.client.0.smithi119.stdout:7/349: dread d7/fd [0,4194304] 0 2024-05-27T09:29:17.359 INFO:tasks.workunit.client.0.smithi119.stdout:6/230: sync 2024-05-27T09:29:17.361 INFO:tasks.workunit.client.0.smithi119.stdout:5/258: rename d6/db/l23 to d6/dd/d46/l56 0 2024-05-27T09:29:17.365 INFO:tasks.workunit.client.0.smithi119.stdout:7/350: unlink d7/f33 0 2024-05-27T09:29:17.366 INFO:tasks.workunit.client.0.smithi119.stdout:6/231: truncate d0/d2/d4e/f1f 1372105 0 2024-05-27T09:29:17.367 INFO:tasks.workunit.client.0.smithi119.stdout:4/269: write d4/d6/fa [2890959,29347] 0 2024-05-27T09:29:17.367 INFO:tasks.workunit.client.0.smithi119.stdout:4/270: chown d4/de/l54 54 1 2024-05-27T09:29:17.368 INFO:tasks.workunit.client.0.smithi119.stdout:4/271: mknod d4/d1c/c58 0 2024-05-27T09:29:17.378 INFO:tasks.workunit.client.0.smithi119.stdout:5/259: dread d6/db/de/f25 [0,4194304] 0 2024-05-27T09:29:17.379 INFO:tasks.workunit.client.0.smithi119.stdout:5/260: symlink d6/dd/df/d41/l57 0 2024-05-27T09:29:17.379 INFO:tasks.workunit.client.0.smithi119.stdout:5/261: fsync d6/d2f/f3a 0 2024-05-27T09:29:17.381 INFO:tasks.workunit.client.0.smithi119.stdout:5/262: unlink d6/dd/d39/d19/d3c/c45 0 2024-05-27T09:29:17.384 INFO:tasks.workunit.client.0.smithi119.stdout:5/263: creat d6/dd/d46/d47/d4b/f58 x:0 0 0 2024-05-27T09:29:17.384 INFO:tasks.workunit.client.0.smithi119.stdout:5/264: readlink d6/db/l2c 0 2024-05-27T09:29:17.384 INFO:tasks.workunit.client.0.smithi119.stdout:5/265: creat d6/dd/d46/d47/d4b/f59 x:0 0 0 2024-05-27T09:29:17.384 INFO:tasks.workunit.client.0.smithi119.stdout:5/266: chown d6/db/de/d27 37 1 2024-05-27T09:29:17.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:17 smithi138 ceph-mon[34885]: pgmap v6: 65 pgs: 65 active+clean; 473 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:29:17.387 INFO:tasks.workunit.client.0.smithi119.stdout:5/267: rename d6/dd/d39/d19/d3c/l55 to d6/l5a 0 2024-05-27T09:29:17.390 INFO:tasks.workunit.client.0.smithi119.stdout:5/268: creat d6/db/de/f5b x:0 0 0 2024-05-27T09:29:17.394 INFO:tasks.workunit.client.0.smithi119.stdout:5/269: link d6/db/de/d27/f2a d6/dd/df/f5c 0 2024-05-27T09:29:17.394 INFO:tasks.workunit.client.0.smithi119.stdout:5/270: chown d6/d2f/f42 28810 1 2024-05-27T09:29:17.397 INFO:tasks.workunit.client.0.smithi119.stdout:5/271: symlink d6/dd/d39/l5d 0 2024-05-27T09:29:17.437 INFO:tasks.workunit.client.0.smithi119.stdout:7/351: dwrite d7/df/f29 [0,4194304] 0 2024-05-27T09:29:17.440 INFO:tasks.workunit.client.0.smithi119.stdout:7/352: creat d7/d6a/f6f x:0 0 0 2024-05-27T09:29:17.442 INFO:tasks.workunit.client.0.smithi119.stdout:6/232: dwrite d0/d2/d1c/d21/f46 [0,4194304] 0 2024-05-27T09:29:17.442 INFO:tasks.workunit.client.0.smithi119.stdout:6/233: readlink d0/d11/l22 0 2024-05-27T09:29:17.442 INFO:tasks.workunit.client.0.smithi119.stdout:6/234: creat d0/d2/d24/f50 x:0 0 0 2024-05-27T09:29:17.442 INFO:tasks.workunit.client.0.smithi119.stdout:7/353: rmdir d7/df/d36/d69 0 2024-05-27T09:29:17.444 INFO:tasks.workunit.client.0.smithi119.stdout:4/272: dwrite d4/d23/d31/f53 [0,4194304] 0 2024-05-27T09:29:17.446 INFO:tasks.workunit.client.0.smithi119.stdout:5/272: dwrite d6/db/f36 [0,4194304] 0 2024-05-27T09:29:17.447 INFO:tasks.workunit.client.0.smithi119.stdout:6/235: truncate d0/fd 3081977 0 2024-05-27T09:29:17.447 INFO:tasks.workunit.client.0.smithi119.stdout:6/236: chown d0/d2/d1c/l25 90949 1 2024-05-27T09:29:17.447 INFO:tasks.workunit.client.0.smithi119.stdout:7/354: creat d7/df/d36/d42/d64/f70 x:0 0 0 2024-05-27T09:29:17.462 INFO:tasks.workunit.client.0.smithi119.stdout:5/273: truncate d6/dd/f32 3643467 0 2024-05-27T09:29:17.462 INFO:tasks.workunit.client.0.smithi119.stdout:7/355: creat d7/df/d19/d4d/f71 x:0 0 0 2024-05-27T09:29:17.462 INFO:tasks.workunit.client.0.smithi119.stdout:5/274: mknod d6/db/c5e 0 2024-05-27T09:29:17.463 INFO:tasks.workunit.client.0.smithi119.stdout:5/275: mknod d6/dd/c5f 0 2024-05-27T09:29:17.465 INFO:tasks.workunit.client.0.smithi119.stdout:5/276: mknod d6/dd/d46/d47/d4b/c60 0 2024-05-27T09:29:17.465 INFO:tasks.workunit.client.0.smithi119.stdout:5/277: creat d6/db/dc/f61 x:0 0 0 2024-05-27T09:29:17.471 INFO:tasks.workunit.client.0.smithi119.stdout:5/278: mkdir d6/dd/d39/d19/d3c/d62 0 2024-05-27T09:29:17.471 INFO:tasks.workunit.client.0.smithi119.stdout:5/279: truncate d6/dd/f33 646301 0 2024-05-27T09:29:17.471 INFO:tasks.workunit.client.0.smithi119.stdout:5/280: stat d6/db/l2c 0 2024-05-27T09:29:17.480 INFO:tasks.workunit.client.0.smithi119.stdout:5/281: dread f0 [0,4194304] 0 2024-05-27T09:29:17.490 INFO:tasks.workunit.client.0.smithi119.stdout:4/273: dwrite f1 [4194304,4194304] 0 2024-05-27T09:29:17.490 INFO:tasks.workunit.client.0.smithi119.stdout:4/274: truncate d4/d1c/d39/f4b 976473 0 2024-05-27T09:29:17.490 INFO:tasks.workunit.client.0.smithi119.stdout:4/275: getdents d4/d6/d3e 0 2024-05-27T09:29:17.494 INFO:tasks.workunit.client.0.smithi119.stdout:4/276: rmdir d4/d23 39 2024-05-27T09:29:17.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:17 smithi119 ceph-mon[29082]: pgmap v6: 65 pgs: 65 active+clean; 473 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:29:17.542 INFO:tasks.workunit.client.0.smithi119.stdout:0/259: sync 2024-05-27T09:29:17.542 INFO:tasks.workunit.client.0.smithi119.stdout:0/260: fdatasync d2/d6/d9/f2c 0 2024-05-27T09:29:17.542 INFO:tasks.workunit.client.0.smithi119.stdout:0/261: write d2/d6/d9/d12/f32 [1329906,115215] 0 2024-05-27T09:29:17.543 INFO:tasks.workunit.client.0.smithi119.stdout:6/237: dwrite d0/d2/d1c/d21/f3b [0,4194304] 0 2024-05-27T09:29:17.549 INFO:tasks.workunit.client.0.smithi119.stdout:0/262: link d2/d6/d9/f3a d2/d6/d9/d24/d30/d45/f58 0 2024-05-27T09:29:17.567 INFO:tasks.workunit.client.0.smithi119.stdout:0/263: rename d2/d6/d9/d24/d30/d45 to d2/d6/d9/d12/d4b/d59 0 2024-05-27T09:29:17.567 INFO:tasks.workunit.client.0.smithi119.stdout:6/238: dread d0/d2/d4e/f4b [4194304,4194304] 0 2024-05-27T09:29:17.567 INFO:tasks.workunit.client.0.smithi119.stdout:4/277: dwrite d4/d23/f52 [0,4194304] 0 2024-05-27T09:29:17.567 INFO:tasks.workunit.client.0.smithi119.stdout:4/278: stat d4/d6/l15 0 2024-05-27T09:29:17.567 INFO:tasks.workunit.client.0.smithi119.stdout:4/279: truncate d4/de/d21/d40/d47/f49 623647 0 2024-05-27T09:29:17.567 INFO:tasks.workunit.client.0.smithi119.stdout:3/446: sync 2024-05-27T09:29:17.567 INFO:tasks.workunit.client.0.smithi119.stdout:7/356: dwrite d7/df/f10 [0,4194304] 0 2024-05-27T09:29:17.567 INFO:tasks.workunit.client.0.smithi119.stdout:6/239: rename d0/d2/d24/l40 to d0/d2/d1c/d21/l51 0 2024-05-27T09:29:17.568 INFO:tasks.workunit.client.0.smithi119.stdout:4/280: symlink d4/d4e/l59 0 2024-05-27T09:29:17.568 INFO:tasks.workunit.client.0.smithi119.stdout:7/357: mknod d7/df/d19/d46/c72 0 2024-05-27T09:29:17.570 INFO:tasks.workunit.client.0.smithi119.stdout:6/240: creat d0/f52 x:0 0 0 2024-05-27T09:29:17.570 INFO:tasks.workunit.client.0.smithi119.stdout:6/241: fsync d0/d2/f4 0 2024-05-27T09:29:17.585 INFO:tasks.workunit.client.0.smithi119.stdout:2/275: sync 2024-05-27T09:29:17.585 INFO:tasks.workunit.client.0.smithi119.stdout:1/343: sync 2024-05-27T09:29:17.585 INFO:tasks.workunit.client.0.smithi119.stdout:9/220: sync 2024-05-27T09:29:17.586 INFO:tasks.workunit.client.0.smithi119.stdout:2/276: write d2/d1a/f46 [1503138,67874] 0 2024-05-27T09:29:17.586 INFO:tasks.workunit.client.0.smithi119.stdout:2/277: chown d2/da/d15/d33/f41 1545 1 2024-05-27T09:29:17.586 INFO:tasks.workunit.client.0.smithi119.stdout:6/242: link d0/d2/d1c/d21/l32 d0/d2/d24/d37/l53 0 2024-05-27T09:29:17.588 INFO:tasks.workunit.client.0.smithi119.stdout:3/447: rename db/d6f to db/d36/d45/d4c/d62/d76/d98 0 2024-05-27T09:29:17.589 INFO:tasks.workunit.client.0.smithi119.stdout:3/448: chown db/d36/d45/d4c/d62/d76/d98/d7a/f84 425149 1 2024-05-27T09:29:17.589 INFO:tasks.workunit.client.0.smithi119.stdout:3/449: fdatasync db/d7f/f74 0 2024-05-27T09:29:17.594 INFO:tasks.workunit.client.0.smithi119.stdout:3/450: creat db/d36/d45/d4c/d62/d13/f99 x:0 0 0 2024-05-27T09:29:17.597 INFO:tasks.workunit.client.0.smithi119.stdout:7/358: rename d7/df/d19/f50 to d7/df/d36/d42/f73 0 2024-05-27T09:29:17.597 INFO:tasks.workunit.client.0.smithi119.stdout:7/359: dread - d7/df/d19/d4d/f71 zero size 2024-05-27T09:29:17.597 INFO:tasks.workunit.client.0.smithi119.stdout:7/360: chown d7/df/d36/f49 502696706 1 2024-05-27T09:29:17.597 INFO:tasks.workunit.client.0.smithi119.stdout:7/361: write d7/db/d37/f5e [566498,55174] 0 2024-05-27T09:29:17.601 INFO:tasks.workunit.client.0.smithi119.stdout:8/299: sync 2024-05-27T09:29:17.603 INFO:tasks.workunit.client.0.smithi119.stdout:1/344: rmdir d7/db/d12/d15 39 2024-05-27T09:29:17.607 INFO:tasks.workunit.client.0.smithi119.stdout:3/451: symlink db/d36/d45/d4c/d62/d13/d33/l9a 0 2024-05-27T09:29:17.607 INFO:tasks.workunit.client.0.smithi119.stdout:7/362: truncate d7/d1a/f2d 156531 0 2024-05-27T09:29:17.611 INFO:tasks.workunit.client.0.smithi119.stdout:9/221: rename d18/d19/f1c to d18/d19/d1b/d29/f47 0 2024-05-27T09:29:17.611 INFO:tasks.workunit.client.0.smithi119.stdout:9/222: readlink l9 0 2024-05-27T09:29:17.612 INFO:tasks.workunit.client.0.smithi119.stdout:3/452: mkdir db/d36/d45/d4c/d62/d13/d33/d9b 0 2024-05-27T09:29:17.616 INFO:tasks.workunit.client.0.smithi119.stdout:3/453: getdents db/d36/d45/d4c/d62/d76/d98 0 2024-05-27T09:29:17.620 INFO:tasks.workunit.client.0.smithi119.stdout:3/454: creat db/d36/d45/d93/f9c x:0 0 0 2024-05-27T09:29:17.620 INFO:tasks.workunit.client.0.smithi119.stdout:3/455: readlink db/d36/d45/d4c/d62/d13/d33/l34 0 2024-05-27T09:29:17.626 INFO:tasks.workunit.client.0.smithi119.stdout:8/300: write d3/f25 [241259,23654] 0 2024-05-27T09:29:17.627 INFO:tasks.workunit.client.0.smithi119.stdout:8/301: creat d3/de/f64 x:0 0 0 2024-05-27T09:29:17.629 INFO:tasks.workunit.client.0.smithi119.stdout:8/302: symlink d3/d8/d4d/d58/l65 0 2024-05-27T09:29:17.636 INFO:tasks.workunit.client.0.smithi119.stdout:8/303: mkdir d3/d8/d4d/d5d/d66 0 2024-05-27T09:29:17.636 INFO:tasks.workunit.client.0.smithi119.stdout:8/304: creat d3/d8/d4d/f67 x:0 0 0 2024-05-27T09:29:17.636 INFO:tasks.workunit.client.0.smithi119.stdout:8/305: read d3/d8/f34 [446863,14514] 0 2024-05-27T09:29:17.636 INFO:tasks.workunit.client.0.smithi119.stdout:7/363: dread f4 [4194304,4194304] 0 2024-05-27T09:29:17.642 INFO:tasks.workunit.client.0.smithi119.stdout:7/364: symlink d7/d1a/d40/d44/d67/l74 0 2024-05-27T09:29:17.651 INFO:tasks.workunit.client.0.smithi119.stdout:1/345: dread d7/dd/d25/d29/f55 [0,4194304] 0 2024-05-27T09:29:17.651 INFO:tasks.workunit.client.0.smithi119.stdout:4/281: dwrite d4/d1c/f1e [0,4194304] 0 2024-05-27T09:29:17.654 INFO:tasks.workunit.client.0.smithi119.stdout:4/282: readlink d4/d23/d24/l30 0 2024-05-27T09:29:17.654 INFO:tasks.workunit.client.0.smithi119.stdout:4/283: truncate d4/d1c/f1e 4457495 0 2024-05-27T09:29:17.655 INFO:tasks.workunit.client.0.smithi119.stdout:1/346: symlink d7/d40/l6d 0 2024-05-27T09:29:17.655 INFO:tasks.workunit.client.0.smithi119.stdout:1/347: chown d7/db/d12/d15/d1f 725 1 2024-05-27T09:29:17.663 INFO:tasks.workunit.client.0.smithi119.stdout:0/264: dwrite d2/d6/d9/f1c [0,4194304] 0 2024-05-27T09:29:17.671 INFO:tasks.workunit.client.0.smithi119.stdout:6/243: dwrite d0/d2/f31 [0,4194304] 0 2024-05-27T09:29:17.675 INFO:tasks.workunit.client.0.smithi119.stdout:6/244: getdents d0/d11 0 2024-05-27T09:29:17.682 INFO:tasks.workunit.client.0.smithi119.stdout:6/245: rename d0/d2/d4e/c41 to d0/c54 0 2024-05-27T09:29:17.685 INFO:tasks.workunit.client.0.smithi119.stdout:2/278: dwrite d2/da/d2c/f38 [0,4194304] 0 2024-05-27T09:29:17.691 INFO:tasks.workunit.client.0.smithi119.stdout:2/279: symlink d2/da/d4e/l63 0 2024-05-27T09:29:17.693 INFO:tasks.workunit.client.0.smithi119.stdout:2/280: symlink d2/da/d2c/l64 0 2024-05-27T09:29:17.696 INFO:tasks.workunit.client.0.smithi119.stdout:1/348: write d7/dd/d44/f56 [1328968,67007] 0 2024-05-27T09:29:17.699 INFO:tasks.workunit.client.0.smithi119.stdout:1/349: truncate d7/db/d12/d15/f3f 51452 0 2024-05-27T09:29:17.703 INFO:tasks.workunit.client.0.smithi119.stdout:7/365: dwrite f4 [0,4194304] 0 2024-05-27T09:29:17.704 INFO:tasks.workunit.client.0.smithi119.stdout:7/366: write d7/df/d36/d42/f60 [807335,127054] 0 2024-05-27T09:29:17.704 INFO:tasks.workunit.client.0.smithi119.stdout:3/456: dwrite db/f83 [0,4194304] 0 2024-05-27T09:29:17.709 INFO:tasks.workunit.client.0.smithi119.stdout:5/282: sync 2024-05-27T09:29:17.709 INFO:tasks.workunit.client.0.smithi119.stdout:5/283: chown d6/ca 250474534 1 2024-05-27T09:29:17.716 INFO:tasks.workunit.client.0.smithi119.stdout:3/457: unlink db/d36/d45/d4c/d62/c91 0 2024-05-27T09:29:17.716 INFO:tasks.workunit.client.0.smithi119.stdout:7/367: truncate d7/fa 65192 0 2024-05-27T09:29:17.716 INFO:tasks.workunit.client.0.smithi119.stdout:5/284: dread f0 [0,4194304] 0 2024-05-27T09:29:17.717 INFO:tasks.workunit.client.0.smithi119.stdout:6/246: write d0/d2/d1c/d21/f3b [2616634,9329] 0 2024-05-27T09:29:17.717 INFO:tasks.workunit.client.0.smithi119.stdout:4/284: dwrite d4/d6/fd [0,4194304] 0 2024-05-27T09:29:17.720 INFO:tasks.workunit.client.0.smithi119.stdout:5/285: creat d6/f63 x:0 0 0 2024-05-27T09:29:17.721 INFO:tasks.workunit.client.0.smithi119.stdout:5/286: stat d6/dd/d46/d47/d4b/l4f 0 2024-05-27T09:29:17.721 INFO:tasks.workunit.client.0.smithi119.stdout:5/287: stat d6/f1e 0 2024-05-27T09:29:17.721 INFO:tasks.workunit.client.0.smithi119.stdout:4/285: link d4/d1c/l3f d4/de/d21/d40/d47/l5a 0 2024-05-27T09:29:17.721 INFO:tasks.workunit.client.0.smithi119.stdout:4/286: stat d4/d23/d31/l4d 0 2024-05-27T09:29:17.721 INFO:tasks.workunit.client.0.smithi119.stdout:6/247: creat d0/d19/f55 x:0 0 0 2024-05-27T09:29:17.724 INFO:tasks.workunit.client.0.smithi119.stdout:6/248: mknod d0/d19/c56 0 2024-05-27T09:29:17.724 INFO:tasks.workunit.client.0.smithi119.stdout:0/265: dwrite d2/d6/d23/f33 [0,4194304] 0 2024-05-27T09:29:17.724 INFO:tasks.workunit.client.0.smithi119.stdout:2/281: dread d2/f48 [0,4194304] 0 2024-05-27T09:29:17.724 INFO:tasks.workunit.client.0.smithi119.stdout:7/368: dread d7/db/d37/f5e [0,4194304] 0 2024-05-27T09:29:17.725 INFO:tasks.workunit.client.0.smithi119.stdout:2/282: rmdir d2 39 2024-05-27T09:29:17.725 INFO:tasks.workunit.client.0.smithi119.stdout:2/283: fsync d2/d1a/f3d 0 2024-05-27T09:29:17.725 INFO:tasks.workunit.client.0.smithi119.stdout:0/266: creat d2/d6/d9/d12/d4f/f5a x:0 0 0 2024-05-27T09:29:17.725 INFO:tasks.workunit.client.0.smithi119.stdout:0/267: fdatasync d2/f3d 0 2024-05-27T09:29:17.726 INFO:tasks.workunit.client.0.smithi119.stdout:7/369: truncate d7/db/f2f 274294 0 2024-05-27T09:29:17.726 INFO:tasks.workunit.client.0.smithi119.stdout:0/268: truncate d2/d6/f3c 3583314 0 2024-05-27T09:29:17.727 INFO:tasks.workunit.client.0.smithi119.stdout:0/269: chown d2/le 978504 1 2024-05-27T09:29:17.739 INFO:tasks.workunit.client.0.smithi119.stdout:6/249: dread d0/d11/f3f [0,4194304] 0 2024-05-27T09:29:17.783 INFO:tasks.workunit.client.0.smithi119.stdout:5/288: dwrite d6/dd/d46/d47/d4b/f58 [0,4194304] 0 2024-05-27T09:29:17.783 INFO:tasks.workunit.client.0.smithi119.stdout:5/289: write d6/db/f36 [4384749,6616] 0 2024-05-27T09:29:17.783 INFO:tasks.workunit.client.0.smithi119.stdout:5/290: chown d6/dd/df/d41/l57 14 1 2024-05-27T09:29:17.783 INFO:tasks.workunit.client.0.smithi119.stdout:0/270: symlink d2/d6/d9/d24/d30/l5b 0 2024-05-27T09:29:17.783 INFO:tasks.workunit.client.0.smithi119.stdout:7/370: mkdir d7/d6a/d75 0 2024-05-27T09:29:17.784 INFO:tasks.workunit.client.0.smithi119.stdout:6/250: getdents d0/d2/d4e 0 2024-05-27T09:29:17.784 INFO:tasks.workunit.client.0.smithi119.stdout:1/350: dwrite d7/db/d12/d15/d1f/f3a [0,4194304] 0 2024-05-27T09:29:17.794 INFO:tasks.workunit.client.0.smithi119.stdout:3/458: dread db/f12 [0,4194304] 0 2024-05-27T09:29:17.797 INFO:tasks.workunit.client.0.smithi119.stdout:2/284: read d2/f26 [2244726,22863] 0 2024-05-27T09:29:17.799 INFO:tasks.workunit.client.0.smithi119.stdout:4/287: write f3 [2877780,124035] 0 2024-05-27T09:29:17.803 INFO:tasks.workunit.client.0.smithi119.stdout:2/285: dread - d2/d1a/f27 zero size 2024-05-27T09:29:17.803 INFO:tasks.workunit.client.0.smithi119.stdout:2/286: fdatasync d2/f48 0 2024-05-27T09:29:17.803 INFO:tasks.workunit.client.0.smithi119.stdout:2/287: fsync d2/da/d15/f16 0 2024-05-27T09:29:17.812 INFO:tasks.workunit.client.0.smithi119.stdout:7/371: dread d7/fa [0,4194304] 0 2024-05-27T09:29:17.831 INFO:tasks.workunit.client.0.smithi119.stdout:0/271: dread d2/f49 [0,4194304] 0 2024-05-27T09:29:17.849 INFO:tasks.workunit.client.0.smithi119.stdout:2/288: dread d2/d1a/f30 [0,4194304] 0 2024-05-27T09:29:17.861 INFO:tasks.workunit.client.0.smithi119.stdout:3/459: dwrite db/d7f/f5a [0,4194304] 0 2024-05-27T09:29:17.861 INFO:tasks.workunit.client.0.smithi119.stdout:3/460: readlink db/d36/d45/d4c/d62/l25 0 2024-05-27T09:29:17.863 INFO:tasks.workunit.client.0.smithi119.stdout:3/461: write db/d5b/f63 [222860,30119] 0 2024-05-27T09:29:17.863 INFO:tasks.workunit.client.0.smithi119.stdout:6/251: dwrite d0/d2/d1c/d21/f2b [0,4194304] 0 2024-05-27T09:29:18.039 INFO:tasks.workunit.client.0.smithi119.stdout:5/291: creat d6/dd/f64 x:0 0 0 2024-05-27T09:29:18.069 INFO:tasks.workunit.client.0.smithi119.stdout:7/372: mkdir d7/df/d36/d42/d76 0 2024-05-27T09:29:18.069 INFO:tasks.workunit.client.0.smithi119.stdout:5/292: dread d6/f12 [0,4194304] 0 2024-05-27T09:29:18.071 INFO:tasks.workunit.client.0.smithi119.stdout:2/289: creat d2/da/f65 x:0 0 0 2024-05-27T09:29:18.071 INFO:tasks.workunit.client.0.smithi119.stdout:2/290: chown d2/da/d15/f1b 87905 1 2024-05-27T09:29:18.072 INFO:tasks.workunit.client.0.smithi119.stdout:5/293: read d6/dd/df/f5c [754909,73502] 0 2024-05-27T09:29:18.074 INFO:tasks.workunit.client.0.smithi119.stdout:3/462: mkdir db/d36/d45/d49/d9d 0 2024-05-27T09:29:18.079 INFO:tasks.workunit.client.0.smithi119.stdout:4/288: mkdir d4/d23/d24/d5b 0 2024-05-27T09:29:18.083 INFO:tasks.workunit.client.0.smithi119.stdout:7/373: mknod d7/df/d36/d42/d64/c77 0 2024-05-27T09:29:18.083 INFO:tasks.workunit.client.0.smithi119.stdout:7/374: readlink d7/df/d19/l4b 0 2024-05-27T09:29:18.083 INFO:tasks.workunit.client.0.smithi119.stdout:9/223: sync 2024-05-27T09:29:18.083 INFO:tasks.workunit.client.0.smithi119.stdout:9/224: fdatasync d18/d19/f35 0 2024-05-27T09:29:18.083 INFO:tasks.workunit.client.0.smithi119.stdout:9/225: fsync d18/d19/d1b/f38 0 2024-05-27T09:29:18.083 INFO:tasks.workunit.client.0.smithi119.stdout:8/306: sync 2024-05-27T09:29:18.083 INFO:tasks.workunit.client.0.smithi119.stdout:8/307: stat d3/d8/d4d/d5d/l5f 0 2024-05-27T09:29:18.084 INFO:tasks.workunit.client.0.smithi119.stdout:8/308: truncate d3/d8/d1b/f28 1071922 0 2024-05-27T09:29:18.084 INFO:tasks.workunit.client.0.smithi119.stdout:8/309: creat d3/de/f68 x:0 0 0 2024-05-27T09:29:18.084 INFO:tasks.workunit.client.0.smithi119.stdout:8/310: write d3/de/f1c [2778976,55412] 0 2024-05-27T09:29:18.084 INFO:tasks.workunit.client.0.smithi119.stdout:8/311: chown d3/d39/l56 0 1 2024-05-27T09:29:18.084 INFO:tasks.workunit.client.0.smithi119.stdout:8/312: chown d3/d8/f34 2850 1 2024-05-27T09:29:18.085 INFO:tasks.workunit.client.0.smithi119.stdout:2/291: truncate d2/f26 132949 0 2024-05-27T09:29:18.090 INFO:tasks.workunit.client.0.smithi119.stdout:4/289: getdents d4/de/d21 0 2024-05-27T09:29:18.090 INFO:tasks.workunit.client.0.smithi119.stdout:4/290: write d4/d6/f1a [4184212,5470] 0 2024-05-27T09:29:18.091 INFO:tasks.workunit.client.0.smithi119.stdout:9/226: dread d18/d19/d1b/f1e [0,4194304] 0 2024-05-27T09:29:18.092 INFO:tasks.workunit.client.0.smithi119.stdout:9/227: write d18/d19/d1b/f1e [159109,22081] 0 2024-05-27T09:29:18.092 INFO:tasks.workunit.client.0.smithi119.stdout:8/313: link d3/f1a d3/d8/d4d/d5d/d60/f69 0 2024-05-27T09:29:18.092 INFO:tasks.workunit.client.0.smithi119.stdout:8/314: creat d3/d8/d4d/d58/f6a x:0 0 0 2024-05-27T09:29:18.094 INFO:tasks.workunit.client.0.smithi119.stdout:2/292: creat d2/f66 x:0 0 0 2024-05-27T09:29:18.094 INFO:tasks.workunit.client.0.smithi119.stdout:2/293: write d2/d1a/f39 [768255,104955] 0 2024-05-27T09:29:18.095 INFO:tasks.workunit.client.0.smithi119.stdout:2/294: creat d2/da/f67 x:0 0 0 2024-05-27T09:29:18.095 INFO:tasks.workunit.client.0.smithi119.stdout:2/295: creat d2/d1a/d51/f68 x:0 0 0 2024-05-27T09:29:18.096 INFO:tasks.workunit.client.0.smithi119.stdout:0/272: dwrite d2/d6/f3c [0,4194304] 0 2024-05-27T09:29:18.096 INFO:tasks.workunit.client.0.smithi119.stdout:0/273: chown d2/d6/d9/l52 217 1 2024-05-27T09:29:18.096 INFO:tasks.workunit.client.0.smithi119.stdout:0/274: chown d2/d6/d9/f1c 868 1 2024-05-27T09:29:18.096 INFO:tasks.workunit.client.0.smithi119.stdout:0/275: fsync d2/d6/d23/f33 0 2024-05-27T09:29:18.096 INFO:tasks.workunit.client.0.smithi119.stdout:6/252: rename d0/d2/d24 to d0/d2/d57 0 2024-05-27T09:29:18.097 INFO:tasks.workunit.client.0.smithi119.stdout:9/228: getdents d18 0 2024-05-27T09:29:18.102 INFO:tasks.workunit.client.0.smithi119.stdout:8/315: creat d3/d8/d4d/d5d/d60/f6b x:0 0 0 2024-05-27T09:29:18.111 INFO:tasks.workunit.client.0.smithi119.stdout:2/296: mkdir d2/da/d15/d33/d69 0 2024-05-27T09:29:18.111 INFO:tasks.workunit.client.0.smithi119.stdout:9/229: mknod d18/d19/c48 0 2024-05-27T09:29:18.111 INFO:tasks.workunit.client.0.smithi119.stdout:9/230: creat d18/d19/f49 x:0 0 0 2024-05-27T09:29:18.112 INFO:tasks.workunit.client.0.smithi119.stdout:6/253: mkdir d0/d2/d1c/d21/d58 0 2024-05-27T09:29:18.130 INFO:tasks.workunit.client.0.smithi119.stdout:9/231: creat d18/d19/d1b/d40/d44/f4a x:0 0 0 2024-05-27T09:29:18.131 INFO:tasks.workunit.client.0.smithi119.stdout:9/232: stat d18/d19/d1b/l31 0 2024-05-27T09:29:18.132 INFO:tasks.workunit.client.0.smithi119.stdout:1/351: sync 2024-05-27T09:29:18.132 INFO:tasks.workunit.client.0.smithi119.stdout:9/233: mknod d18/d19/d1b/c4b 0 2024-05-27T09:29:18.132 INFO:tasks.workunit.client.0.smithi119.stdout:9/234: write d18/d19/f36 [105400,32666] 0 2024-05-27T09:29:18.132 INFO:tasks.workunit.client.0.smithi119.stdout:9/235: chown d18/d19/f35 33 1 2024-05-27T09:29:18.134 INFO:tasks.workunit.client.0.smithi119.stdout:9/236: creat d18/d19/d1b/f4c x:0 0 0 2024-05-27T09:29:18.135 INFO:tasks.workunit.client.0.smithi119.stdout:9/237: symlink d18/l4d 0 2024-05-27T09:29:18.139 INFO:tasks.workunit.client.0.smithi119.stdout:9/238: rename d18/f46 to d18/d19/f4e 0 2024-05-27T09:29:18.142 INFO:tasks.workunit.client.0.smithi119.stdout:9/239: write d18/f1a [170045,89904] 0 2024-05-27T09:29:18.144 INFO:tasks.workunit.client.0.smithi119.stdout:9/240: link d18/d19/l39 d18/d19/d1b/d40/l4f 0 2024-05-27T09:29:18.156 INFO:tasks.workunit.client.0.smithi119.stdout:4/291: dwrite f3 [0,4194304] 0 2024-05-27T09:29:18.157 INFO:tasks.workunit.client.0.smithi119.stdout:5/294: dwrite d6/d2f/f51 [0,4194304] 0 2024-05-27T09:29:18.160 INFO:tasks.workunit.client.0.smithi119.stdout:8/316: dwrite d3/d8/d4d/f36 [0,4194304] 0 2024-05-27T09:29:18.160 INFO:tasks.workunit.client.0.smithi119.stdout:8/317: creat d3/d8/d1b/f6c x:0 0 0 2024-05-27T09:29:18.160 INFO:tasks.workunit.client.0.smithi119.stdout:8/318: dread - d3/d8/d1b/f6c zero size 2024-05-27T09:29:18.160 INFO:tasks.workunit.client.0.smithi119.stdout:8/319: creat d3/d8/d1b/f6d x:0 0 0 2024-05-27T09:29:18.160 INFO:tasks.workunit.client.0.smithi119.stdout:8/320: write d3/f12 [1439549,126230] 0 2024-05-27T09:29:18.163 INFO:tasks.workunit.client.0.smithi119.stdout:4/292: read d4/de/d21/d40/d47/f49 [15833,73587] 0 2024-05-27T09:29:18.164 INFO:tasks.workunit.client.0.smithi119.stdout:9/241: dread d18/d19/f36 [0,4194304] 0 2024-05-27T09:29:18.173 INFO:tasks.workunit.client.0.smithi119.stdout:4/293: mkdir d4/de/d21/d40/d47/d5c 0 2024-05-27T09:29:18.173 INFO:tasks.workunit.client.0.smithi119.stdout:4/294: creat d4/d1c/d39/f5d x:0 0 0 2024-05-27T09:29:18.174 INFO:tasks.workunit.client.0.smithi119.stdout:8/321: mkdir d3/d6e 0 2024-05-27T09:29:18.193 INFO:tasks.workunit.client.0.smithi119.stdout:9/242: read f2 [4015844,124672] 0 2024-05-27T09:29:18.196 INFO:tasks.workunit.client.0.smithi119.stdout:6/254: dwrite d0/d2/d4e/f2e [0,4194304] 0 2024-05-27T09:29:18.200 INFO:tasks.workunit.client.0.smithi119.stdout:1/352: dread d7/db/d12/d15/f3f [0,4194304] 0 2024-05-27T09:29:18.202 INFO:tasks.workunit.client.0.smithi119.stdout:1/353: mkdir d7/d6e 0 2024-05-27T09:29:18.205 INFO:tasks.workunit.client.0.smithi119.stdout:1/354: mknod d7/dd/d25/d29/d38/d3c/c6f 0 2024-05-27T09:29:18.205 INFO:tasks.workunit.client.0.smithi119.stdout:1/355: truncate d7/db/d12/d15/f3f 1022174 0 2024-05-27T09:29:18.206 INFO:tasks.workunit.client.0.smithi119.stdout:6/255: read d0/d2/f30 [187466,33756] 0 2024-05-27T09:29:18.207 INFO:tasks.workunit.client.0.smithi119.stdout:6/256: write d0/d2/d1c/f2d [502214,19063] 0 2024-05-27T09:29:18.208 INFO:tasks.workunit.client.0.smithi119.stdout:6/257: creat d0/d2/d1c/f59 x:0 0 0 2024-05-27T09:29:18.208 INFO:tasks.workunit.client.0.smithi119.stdout:6/258: chown d0/d2 95 1 2024-05-27T09:29:18.208 INFO:tasks.workunit.client.0.smithi119.stdout:0/276: dwrite d2/d4/d53/f54 [0,4194304] 0 2024-05-27T09:29:18.208 INFO:tasks.workunit.client.0.smithi119.stdout:0/277: write d2/d6/d9/f3a [70994,41610] 0 2024-05-27T09:29:18.209 INFO:tasks.workunit.client.0.smithi119.stdout:1/356: rename d7/dd/d25/d29/l2e to d7/db/d12/d26/d4a/l70 0 2024-05-27T09:29:18.213 INFO:tasks.workunit.client.0.smithi119.stdout:6/259: mkdir d0/d2/d4e/d5a 0 2024-05-27T09:29:18.220 INFO:tasks.workunit.client.0.smithi119.stdout:0/278: link d2/c35 d2/d6/d9/d24/c5c 0 2024-05-27T09:29:18.220 INFO:tasks.workunit.client.0.smithi119.stdout:0/279: chown d2/d6/d9/d24/d30/d4a/c4e 40 1 2024-05-27T09:29:18.220 INFO:tasks.workunit.client.0.smithi119.stdout:0/280: read - d2/d4/f46 zero size 2024-05-27T09:29:18.225 INFO:tasks.workunit.client.0.smithi119.stdout:1/357: symlink d7/dd/d25/d29/d38/d3c/l71 0 2024-05-27T09:29:18.225 INFO:tasks.workunit.client.0.smithi119.stdout:6/260: truncate d0/d2/d4e/f17 4615757 0 2024-05-27T09:29:18.232 INFO:tasks.workunit.client.0.smithi119.stdout:0/281: truncate d2/d6/d9/ff 568093 0 2024-05-27T09:29:18.238 INFO:tasks.workunit.client.0.smithi119.stdout:0/282: rename d2/d6/d9/d24/c2d to d2/d6/c5d 0 2024-05-27T09:29:18.238 INFO:tasks.workunit.client.0.smithi119.stdout:0/283: fdatasync d2/d6/d9/d12/d4b/d59/f58 0 2024-05-27T09:29:18.243 INFO:tasks.workunit.client.0.smithi119.stdout:2/297: dwrite d2/f2d [0,4194304] 0 2024-05-27T09:29:18.243 INFO:tasks.workunit.client.0.smithi119.stdout:2/298: dread - d2/d1a/f3d zero size 2024-05-27T09:29:18.243 INFO:tasks.workunit.client.0.smithi119.stdout:2/299: write d2/d1a/f46 [5209050,110381] 0 2024-05-27T09:29:18.243 INFO:tasks.workunit.client.0.smithi119.stdout:2/300: dread - d2/d1a/f3d zero size 2024-05-27T09:29:18.245 INFO:tasks.workunit.client.0.smithi119.stdout:5/295: dwrite d6/db/de/d27/f2a [0,4194304] 0 2024-05-27T09:29:18.245 INFO:tasks.workunit.client.0.smithi119.stdout:5/296: chown d6/db/f36 32801 1 2024-05-27T09:29:18.251 INFO:tasks.workunit.client.0.smithi119.stdout:0/284: dread d2/d6/f26 [0,4194304] 0 2024-05-27T09:29:18.251 INFO:tasks.workunit.client.0.smithi119.stdout:0/285: chown d2/l3 3068 1 2024-05-27T09:29:18.251 INFO:tasks.workunit.client.0.smithi119.stdout:0/286: creat d2/d6/d9/d24/f5e x:0 0 0 2024-05-27T09:29:18.251 INFO:tasks.workunit.client.0.smithi119.stdout:0/287: fdatasync d2/d4/f46 0 2024-05-27T09:29:18.253 INFO:tasks.workunit.client.0.smithi119.stdout:5/297: symlink d6/dd/l65 0 2024-05-27T09:29:18.253 INFO:tasks.workunit.client.0.smithi119.stdout:5/298: chown d6/ca 571550127 1 2024-05-27T09:29:18.253 INFO:tasks.workunit.client.0.smithi119.stdout:5/299: fdatasync f0 0 2024-05-27T09:29:18.253 INFO:tasks.workunit.client.0.smithi119.stdout:6/261: truncate d0/f6 3861229 0 2024-05-27T09:29:18.254 INFO:tasks.workunit.client.0.smithi119.stdout:1/358: dread f3 [0,4194304] 0 2024-05-27T09:29:18.255 INFO:tasks.workunit.client.0.smithi119.stdout:0/288: truncate d2/f49 2242114 0 2024-05-27T09:29:18.256 INFO:tasks.workunit.client.0.smithi119.stdout:5/300: symlink d6/dd/d39/d19/l66 0 2024-05-27T09:29:18.256 INFO:tasks.workunit.client.0.smithi119.stdout:5/301: creat d6/dd/f67 x:0 0 0 2024-05-27T09:29:18.257 INFO:tasks.workunit.client.0.smithi119.stdout:5/302: fsync d6/f63 0 2024-05-27T09:29:18.258 INFO:tasks.workunit.client.0.smithi119.stdout:8/322: dwrite d3/d8/d4d/f59 [0,4194304] 0 2024-05-27T09:29:18.258 INFO:tasks.workunit.client.0.smithi119.stdout:6/262: creat d0/d2/d57/d37/f5b x:0 0 0 2024-05-27T09:29:18.259 INFO:tasks.workunit.client.0.smithi119.stdout:6/263: truncate d0/d2/d1c/d21/f46 5018582 0 2024-05-27T09:29:18.263 INFO:tasks.workunit.client.0.smithi119.stdout:1/359: link d7/db/d52/f57 d7/dd/d44/d47/f72 0 2024-05-27T09:29:18.271 INFO:tasks.workunit.client.0.smithi119.stdout:5/303: write d6/f9 [2951499,36243] 0 2024-05-27T09:29:18.279 INFO:tasks.workunit.client.0.smithi119.stdout:2/301: dread d2/d1a/d51/f56 [0,4194304] 0 2024-05-27T09:29:18.279 INFO:tasks.workunit.client.0.smithi119.stdout:8/323: mknod d3/d8/d4d/d5d/d60/c6f 0 2024-05-27T09:29:18.281 INFO:tasks.workunit.client.0.smithi119.stdout:8/324: truncate d3/d8/d1b/d46/f2b 1601654 0 2024-05-27T09:29:18.285 INFO:tasks.workunit.client.0.smithi119.stdout:8/325: rename d3/f15 to d3/d8/d4d/d58/f70 0 2024-05-27T09:29:18.286 INFO:tasks.workunit.client.0.smithi119.stdout:2/302: dread d2/d1a/f3b [0,4194304] 0 2024-05-27T09:29:18.288 INFO:tasks.workunit.client.0.smithi119.stdout:2/303: mknod d2/d1a/d51/c6a 0 2024-05-27T09:29:18.293 INFO:tasks.workunit.client.0.smithi119.stdout:9/243: dwrite d18/d19/f49 [0,4194304] 0 2024-05-27T09:29:18.298 INFO:tasks.workunit.client.0.smithi119.stdout:5/304: write d6/d2f/f51 [3320930,60356] 0 2024-05-27T09:29:18.301 INFO:tasks.workunit.client.0.smithi119.stdout:9/244: link d18/f21 d18/d19/f50 0 2024-05-27T09:29:18.325 INFO:tasks.workunit.client.0.smithi119.stdout:5/305: dread d6/dd/df/f5c [0,4194304] 0 2024-05-27T09:29:18.325 INFO:tasks.workunit.client.0.smithi119.stdout:5/306: truncate d6/d2f/f3a 172801 0 2024-05-27T09:29:18.325 INFO:tasks.workunit.client.0.smithi119.stdout:0/289: dwrite d2/d6/d9/d12/d4f/f5a [0,4194304] 0 2024-05-27T09:29:18.325 INFO:tasks.workunit.client.0.smithi119.stdout:0/290: fdatasync d2/d6/f55 0 2024-05-27T09:29:18.328 INFO:tasks.workunit.client.0.smithi119.stdout:1/360: dwrite d7/dd/d44/f4e [0,4194304] 0 2024-05-27T09:29:18.328 INFO:tasks.workunit.client.0.smithi119.stdout:1/361: fdatasync d7/db/d12/d26/f43 0 2024-05-27T09:29:18.328 INFO:tasks.workunit.client.0.smithi119.stdout:0/291: link d2/d6/d9/d24/d30/d4a/f4c d2/d4/d53/f5f 0 2024-05-27T09:29:18.332 INFO:tasks.workunit.client.0.smithi119.stdout:1/362: getdents d7/dd 0 2024-05-27T09:29:18.337 INFO:tasks.workunit.client.0.smithi119.stdout:6/264: dwrite d0/d2/d1c/d21/f2b [4194304,4194304] 0 2024-05-27T09:29:18.337 INFO:tasks.workunit.client.0.smithi119.stdout:6/265: write d0/f52 [883817,8349] 0 2024-05-27T09:29:18.339 INFO:tasks.workunit.client.0.smithi119.stdout:9/245: dread d18/d19/d1b/d29/f47 [0,4194304] 0 2024-05-27T09:29:18.339 INFO:tasks.workunit.client.0.smithi119.stdout:9/246: write d18/d19/f3f [605576,118806] 0 2024-05-27T09:29:18.339 INFO:tasks.workunit.client.0.smithi119.stdout:9/247: chown f2 61724 1 2024-05-27T09:29:18.342 INFO:tasks.workunit.client.0.smithi119.stdout:8/326: read d3/f2d [2945387,39766] 0 2024-05-27T09:29:18.342 INFO:tasks.workunit.client.0.smithi119.stdout:8/327: chown d3/d8/d4d/d5d/l5f 1 1 2024-05-27T09:29:18.351 INFO:tasks.workunit.client.0.smithi119.stdout:6/266: symlink d0/d2/l5c 0 2024-05-27T09:29:18.351 INFO:tasks.workunit.client.0.smithi119.stdout:6/267: chown d0/d2/l5c 0 1 2024-05-27T09:29:18.353 INFO:tasks.workunit.client.0.smithi119.stdout:9/248: rename d18/l42 to d18/l51 0 2024-05-27T09:29:18.353 INFO:tasks.workunit.client.0.smithi119.stdout:9/249: truncate d18/f41 32154 0 2024-05-27T09:29:18.353 INFO:tasks.workunit.client.0.smithi119.stdout:9/250: chown d18/d19/d1b/c25 3195343 1 2024-05-27T09:29:18.354 INFO:tasks.workunit.client.0.smithi119.stdout:6/268: rename d0/d2/f4f to d0/d2/d57/d37/f5d 0 2024-05-27T09:29:18.361 INFO:tasks.workunit.client.0.smithi119.stdout:6/269: truncate d0/d2/d1c/d21/f2b 2669389 0 2024-05-27T09:29:18.384 INFO:tasks.workunit.client.0.smithi119.stdout:2/304: dwrite d2/f66 [0,4194304] 0 2024-05-27T09:29:18.384 INFO:tasks.workunit.client.0.smithi119.stdout:2/305: dread - d2/da/f65 zero size 2024-05-27T09:29:18.384 INFO:tasks.workunit.client.0.smithi119.stdout:2/306: write d2/d1a/f30 [2259611,54187] 0 2024-05-27T09:29:18.388 INFO:tasks.workunit.client.0.smithi119.stdout:2/307: creat d2/da/d15/d33/d69/f6b x:0 0 0 2024-05-27T09:29:18.404 INFO:tasks.workunit.client.0.smithi119.stdout:2/308: read d2/da/fd [5688763,117201] 0 2024-05-27T09:29:18.405 INFO:tasks.workunit.client.0.smithi119.stdout:2/309: mknod d2/d1a/c6c 0 2024-05-27T09:29:18.405 INFO:tasks.workunit.client.0.smithi119.stdout:6/270: write d0/d2/f10 [1201188,65921] 0 2024-05-27T09:29:18.406 INFO:tasks.workunit.client.0.smithi119.stdout:2/310: write d2/d1a/f46 [5472018,34564] 0 2024-05-27T09:29:18.408 INFO:tasks.workunit.client.0.smithi119.stdout:2/311: mknod d2/d1a/c6d 0 2024-05-27T09:29:18.409 INFO:tasks.workunit.client.0.smithi119.stdout:2/312: creat d2/da/d4e/f6e x:0 0 0 2024-05-27T09:29:18.421 INFO:tasks.workunit.client.0.smithi119.stdout:1/363: dwrite d7/db/d12/f4c [0,4194304] 0 2024-05-27T09:29:18.423 INFO:tasks.workunit.client.0.smithi119.stdout:0/292: dwrite d2/d4/f46 [0,4194304] 0 2024-05-27T09:29:18.423 INFO:tasks.workunit.client.0.smithi119.stdout:0/293: write d2/d6/d9/d24/d30/d4a/f4c [566695,52062] 0 2024-05-27T09:29:18.428 INFO:tasks.workunit.client.0.smithi119.stdout:0/294: truncate d2/d4/f42 4523594 0 2024-05-27T09:29:18.430 INFO:tasks.workunit.client.0.smithi119.stdout:2/313: write d2/da/d15/d33/f60 [1968985,78122] 0 2024-05-27T09:29:18.432 INFO:tasks.workunit.client.0.smithi119.stdout:0/295: truncate d2/d6/d9/ff 351534 0 2024-05-27T09:29:18.434 INFO:tasks.workunit.client.0.smithi119.stdout:2/314: mkdir d2/da/d6f 0 2024-05-27T09:29:18.435 INFO:tasks.workunit.client.0.smithi119.stdout:0/296: mknod d2/d6/d23/c60 0 2024-05-27T09:29:18.436 INFO:tasks.workunit.client.0.smithi119.stdout:2/315: creat d2/da/d6f/f70 x:0 0 0 2024-05-27T09:29:18.444 INFO:tasks.workunit.client.0.smithi119.stdout:9/251: dwrite f2 [4194304,4194304] 0 2024-05-27T09:29:18.446 INFO:tasks.workunit.client.0.smithi119.stdout:9/252: unlink fb 0 2024-05-27T09:29:18.453 INFO:tasks.workunit.client.0.smithi119.stdout:5/307: dwrite d6/dd/df/f54 [0,4194304] 0 2024-05-27T09:29:18.453 INFO:tasks.workunit.client.0.smithi119.stdout:5/308: dread - d6/dd/f37 zero size 2024-05-27T09:29:18.454 INFO:tasks.workunit.client.0.smithi119.stdout:5/309: fsync d6/dd/f64 0 2024-05-27T09:29:18.468 INFO:tasks.workunit.client.0.smithi119.stdout:5/310: unlink d6/dd/c5f 0 2024-05-27T09:29:18.470 INFO:tasks.workunit.client.0.smithi119.stdout:9/253: dread d18/d19/d1b/d29/f3a [0,4194304] 0 2024-05-27T09:29:18.471 INFO:tasks.workunit.client.0.smithi119.stdout:5/311: mknod d6/dd/d39/d19/d3c/d62/c68 0 2024-05-27T09:29:18.471 INFO:tasks.workunit.client.0.smithi119.stdout:5/312: chown d6/db/de/d27/f2a 1856 1 2024-05-27T09:29:18.471 INFO:tasks.workunit.client.0.smithi119.stdout:5/313: write d6/db/dc/f61 [444920,52937] 0 2024-05-27T09:29:18.471 INFO:tasks.workunit.client.0.smithi119.stdout:9/254: readlink d18/l51 0 2024-05-27T09:29:18.473 INFO:tasks.workunit.client.0.smithi119.stdout:5/314: rename d6/f8 to d6/dd/d39/f69 0 2024-05-27T09:29:18.473 INFO:tasks.workunit.client.0.smithi119.stdout:5/315: truncate d6/dd/f64 744109 0 2024-05-27T09:29:18.473 INFO:tasks.workunit.client.0.smithi119.stdout:5/316: creat d6/f6a x:0 0 0 2024-05-27T09:29:18.473 INFO:tasks.workunit.client.0.smithi119.stdout:5/317: creat d6/db/de/d27/f6b x:0 0 0 2024-05-27T09:29:18.473 INFO:tasks.workunit.client.0.smithi119.stdout:5/318: write d6/dd/d46/d47/d4b/f58 [5139177,35702] 0 2024-05-27T09:29:18.476 INFO:tasks.workunit.client.0.smithi119.stdout:5/319: mknod d6/db/de/c6c 0 2024-05-27T09:29:18.476 INFO:tasks.workunit.client.0.smithi119.stdout:5/320: readlink d6/dd/df/d41/l57 0 2024-05-27T09:29:18.476 INFO:tasks.workunit.client.0.smithi119.stdout:5/321: readlink d6/dd/df/d41/l57 0 2024-05-27T09:29:18.476 INFO:tasks.workunit.client.0.smithi119.stdout:5/322: truncate d6/db/de/f5b 483250 0 2024-05-27T09:29:18.491 INFO:tasks.workunit.client.0.smithi119.stdout:0/297: write d2/d4/d53/f54 [923949,106083] 0 2024-05-27T09:29:18.491 INFO:tasks.workunit.client.0.smithi119.stdout:6/271: dwrite d0/d2/d1c/f4c [0,4194304] 0 2024-05-27T09:29:18.496 INFO:tasks.workunit.client.0.smithi119.stdout:0/298: dread d2/d6/d9/ff [0,4194304] 0 2024-05-27T09:29:18.496 INFO:tasks.workunit.client.0.smithi119.stdout:9/255: read d18/d19/d1b/f38 [34749,106364] 0 2024-05-27T09:29:18.500 INFO:tasks.workunit.client.0.smithi119.stdout:6/272: link d0/d19/c56 d0/d2/d4e/d5a/c5e 0 2024-05-27T09:29:18.500 INFO:tasks.workunit.client.0.smithi119.stdout:6/273: creat d0/f5f x:0 0 0 2024-05-27T09:29:18.500 INFO:tasks.workunit.client.0.smithi119.stdout:6/274: stat d0/d2/d4e/f2e 0 2024-05-27T09:29:18.501 INFO:tasks.workunit.client.0.smithi119.stdout:6/275: write d0/d2/d57/f50 [492208,33159] 0 2024-05-27T09:29:18.501 INFO:tasks.workunit.client.0.smithi119.stdout:9/256: symlink d18/d19/d1b/d40/l52 0 2024-05-27T09:29:18.501 INFO:tasks.workunit.client.0.smithi119.stdout:9/257: chown l9 716276869 1 2024-05-27T09:29:18.503 INFO:tasks.workunit.client.0.smithi119.stdout:6/276: rename d0/c47 to d0/d2/c60 0 2024-05-27T09:29:18.505 INFO:tasks.workunit.client.0.smithi119.stdout:6/277: rename d0/d11/f14 to d0/d2/d1c/d21/d58/f61 0 2024-05-27T09:29:18.515 INFO:tasks.workunit.client.0.smithi119.stdout:6/278: link d0/d2/d1c/d21/f3c d0/d11/f62 0 2024-05-27T09:29:18.515 INFO:tasks.workunit.client.0.smithi119.stdout:6/279: creat d0/d2/d57/d37/f63 x:0 0 0 2024-05-27T09:29:18.515 INFO:tasks.workunit.client.0.smithi119.stdout:9/258: dread d18/d19/d1b/d29/f47 [0,4194304] 0 2024-05-27T09:29:18.515 INFO:tasks.workunit.client.0.smithi119.stdout:6/280: chown d0/l34 642 1 2024-05-27T09:29:18.520 INFO:tasks.workunit.client.0.smithi119.stdout:2/316: dwrite d2/da/d15/d33/f60 [0,4194304] 0 2024-05-27T09:29:18.520 INFO:tasks.workunit.client.0.smithi119.stdout:2/317: chown d2/da/l20 0 1 2024-05-27T09:29:18.526 INFO:tasks.workunit.client.0.smithi119.stdout:7/375: sync 2024-05-27T09:29:18.526 INFO:tasks.workunit.client.0.smithi119.stdout:4/295: sync 2024-05-27T09:29:18.528 INFO:tasks.workunit.client.0.smithi119.stdout:6/281: mkdir d0/d2/d1c/d64 0 2024-05-27T09:29:18.536 INFO:tasks.workunit.client.0.smithi119.stdout:7/376: getdents d7 0 2024-05-27T09:29:18.536 INFO:tasks.workunit.client.0.smithi119.stdout:7/377: truncate d7/db/f5c 1001346 0 2024-05-27T09:29:18.559 INFO:tasks.workunit.client.0.smithi119.stdout:6/282: rmdir d0/d19 39 2024-05-27T09:29:18.559 INFO:tasks.workunit.client.0.smithi119.stdout:3/463: sync 2024-05-27T09:29:18.561 INFO:tasks.workunit.client.0.smithi119.stdout:3/464: write db/d36/f71 [1750179,68652] 0 2024-05-27T09:29:18.561 INFO:tasks.workunit.client.0.smithi119.stdout:3/465: chown db/d36/d45/d49/d4f/f78 75915890 1 2024-05-27T09:29:18.563 INFO:tasks.workunit.client.0.smithi119.stdout:6/283: unlink d0/d2/d4e/f17 0 2024-05-27T09:29:18.564 INFO:tasks.workunit.client.0.smithi119.stdout:6/284: rmdir d0/d11 39 2024-05-27T09:29:18.564 INFO:tasks.workunit.client.0.smithi119.stdout:6/285: truncate d0/f18 101703 0 2024-05-27T09:29:18.566 INFO:tasks.workunit.client.0.smithi119.stdout:6/286: mkdir d0/d2/d1c/d64/d65 0 2024-05-27T09:29:18.567 INFO:tasks.workunit.client.0.smithi119.stdout:6/287: mknod d0/d2/d1c/d64/d65/c66 0 2024-05-27T09:29:18.583 INFO:tasks.workunit.client.0.smithi119.stdout:0/299: dwrite d2/d4/d53/f54 [4194304,4194304] 0 2024-05-27T09:29:18.586 INFO:tasks.workunit.client.0.smithi119.stdout:5/323: dwrite d6/dd/d46/d47/d4b/f58 [4194304,4194304] 0 2024-05-27T09:29:18.586 INFO:tasks.workunit.client.0.smithi119.stdout:5/324: rename d6/db to d6/db/de/d6d 22 2024-05-27T09:29:18.590 INFO:tasks.workunit.client.0.smithi119.stdout:0/300: symlink d2/d6/d9/d24/d30/d4a/l61 0 2024-05-27T09:29:18.590 INFO:tasks.workunit.client.0.smithi119.stdout:5/325: mkdir d6/dd/d6e 0 2024-05-27T09:29:18.593 INFO:tasks.workunit.client.0.smithi119.stdout:2/318: dwrite d2/da/f10 [0,4194304] 0 2024-05-27T09:29:18.594 INFO:tasks.workunit.client.0.smithi119.stdout:2/319: read - d2/da/f4d zero size 2024-05-27T09:29:18.594 INFO:tasks.workunit.client.0.smithi119.stdout:2/320: write d2/da/d2c/f3c [5111928,38631] 0 2024-05-27T09:29:18.597 INFO:tasks.workunit.client.0.smithi119.stdout:6/288: dread d0/d2/d1c/f4c [0,4194304] 0 2024-05-27T09:29:18.613 INFO:tasks.workunit.client.0.smithi119.stdout:2/321: truncate d2/d1a/f30 402201 0 2024-05-27T09:29:18.614 INFO:tasks.workunit.client.0.smithi119.stdout:2/322: write d2/da/ff [1237300,123367] 0 2024-05-27T09:29:18.614 INFO:tasks.workunit.client.0.smithi119.stdout:2/323: dread - d2/da/d2c/f55 zero size 2024-05-27T09:29:18.629 INFO:tasks.workunit.client.0.smithi119.stdout:3/466: dwrite db/d36/d45/d49/d4f/f7b [0,4194304] 0 2024-05-27T09:29:18.629 INFO:tasks.workunit.client.0.smithi119.stdout:9/259: dwrite fe [4194304,4194304] 0 2024-05-27T09:29:18.637 INFO:tasks.workunit.client.0.smithi119.stdout:9/260: rmdir d18/d19/d1b/d2a 39 2024-05-27T09:29:18.638 INFO:tasks.workunit.client.0.smithi119.stdout:9/261: symlink d18/d19/d1b/l53 0 2024-05-27T09:29:18.643 INFO:tasks.workunit.client.0.smithi119.stdout:0/301: write d2/d6/d23/f33 [690442,99799] 0 2024-05-27T09:29:18.651 INFO:tasks.workunit.client.0.smithi119.stdout:0/302: link d2/d6/d9/d24/d30/d4a/f4c d2/d4/f62 0 2024-05-27T09:29:18.651 INFO:tasks.workunit.client.0.smithi119.stdout:8/328: sync 2024-05-27T09:29:18.651 INFO:tasks.workunit.client.0.smithi119.stdout:8/329: creat d3/d8/d4d/d5d/d60/f71 x:0 0 0 2024-05-27T09:29:18.651 INFO:tasks.workunit.client.0.smithi119.stdout:8/330: chown d3/d8/d1b/f55 7155361 1 2024-05-27T09:29:18.651 INFO:tasks.workunit.client.0.smithi119.stdout:8/331: chown d3/d8/d4d/d5d/d60 3 1 2024-05-27T09:29:18.651 INFO:tasks.workunit.client.0.smithi119.stdout:9/262: truncate fe 3217455 0 2024-05-27T09:29:18.654 INFO:tasks.workunit.client.0.smithi119.stdout:8/332: chown d3/d8/d1b/d46/l4f 1330062727 1 2024-05-27T09:29:18.654 INFO:tasks.workunit.client.0.smithi119.stdout:8/333: truncate d3/d8/d4d/d5d/d60/f6b 967151 0 2024-05-27T09:29:18.658 INFO:tasks.workunit.client.0.smithi119.stdout:9/263: rename d18/d19/d1b/f1f to d18/d19/d1b/d40/d44/f54 0 2024-05-27T09:29:18.662 INFO:tasks.workunit.client.0.smithi119.stdout:8/334: unlink d3/d8/d4d/d5d/c5c 0 2024-05-27T09:29:18.666 INFO:tasks.workunit.client.0.smithi119.stdout:9/264: symlink d18/d19/d1b/d40/l55 0 2024-05-27T09:29:18.670 INFO:tasks.workunit.client.0.smithi119.stdout:4/296: dwrite d4/d1c/f1e [0,4194304] 0 2024-05-27T09:29:18.670 INFO:tasks.workunit.client.0.smithi119.stdout:4/297: fdatasync d4/de/d21/d40/f42 0 2024-05-27T09:29:18.670 INFO:tasks.workunit.client.0.smithi119.stdout:0/303: dread d2/d6/d9/f3a [0,4194304] 0 2024-05-27T09:29:18.671 INFO:tasks.workunit.client.0.smithi119.stdout:0/304: link d2/d6/d9/d24/d30/d4a/l61 d2/d4/l63 0 2024-05-27T09:29:18.673 INFO:tasks.workunit.client.0.smithi119.stdout:0/305: rename d2/f3d to d2/d1a/d20/f64 0 2024-05-27T09:29:18.675 INFO:tasks.workunit.client.0.smithi119.stdout:0/306: mknod d2/d6/d9/d24/d30/d4a/c65 0 2024-05-27T09:29:18.679 INFO:tasks.workunit.client.0.smithi119.stdout:0/307: creat d2/d6/d9/f66 x:0 0 0 2024-05-27T09:29:18.680 INFO:tasks.workunit.client.0.smithi119.stdout:0/308: fsync d2/d6/d9/f1c 0 2024-05-27T09:29:18.683 INFO:tasks.workunit.client.0.smithi119.stdout:5/326: dwrite d6/db/de/d27/f3f [0,4194304] 0 2024-05-27T09:29:18.686 INFO:tasks.workunit.client.0.smithi119.stdout:5/327: fdatasync d6/d2f/f42 0 2024-05-27T09:29:18.693 INFO:tasks.workunit.client.0.smithi119.stdout:1/364: sync 2024-05-27T09:29:18.693 INFO:tasks.workunit.client.0.smithi119.stdout:1/365: fsync d7/db/d12/d15/d1f/f3a 0 2024-05-27T09:29:18.695 INFO:tasks.workunit.client.0.smithi119.stdout:4/298: dread d4/d6/f1a [4194304,4194304] 0 2024-05-27T09:29:18.695 INFO:tasks.workunit.client.0.smithi119.stdout:4/299: fsync d4/de/f2e 0 2024-05-27T09:29:18.695 INFO:tasks.workunit.client.0.smithi119.stdout:4/300: creat d4/d48/f5e x:0 0 0 2024-05-27T09:29:18.701 INFO:tasks.workunit.client.0.smithi119.stdout:1/366: rename d7/d40/l6d to d7/db/d52/l73 0 2024-05-27T09:29:18.715 INFO:tasks.workunit.client.0.smithi119.stdout:0/309: dread d2/d6/f3c [0,4194304] 0 2024-05-27T09:29:18.717 INFO:tasks.workunit.client.0.smithi119.stdout:0/310: symlink d2/d6/d9/d24/d30/l67 0 2024-05-27T09:29:18.719 INFO:tasks.workunit.client.0.smithi119.stdout:0/311: mknod d2/d1a/d20/c68 0 2024-05-27T09:29:18.722 INFO:tasks.workunit.client.0.smithi119.stdout:0/312: mkdir d2/d6/d9/d24/d69 0 2024-05-27T09:29:18.722 INFO:tasks.workunit.client.0.smithi119.stdout:3/467: dwrite db/d36/d45/d4c/d62/d76/d98/d7a/f5f [4194304,4194304] 0 2024-05-27T09:29:18.722 INFO:tasks.workunit.client.0.smithi119.stdout:3/468: write db/d36/d45/d4c/d62/d13/f99 [176493,19520] 0 2024-05-27T09:29:18.726 INFO:tasks.workunit.client.0.smithi119.stdout:1/367: write d7/db/d12/d15/d1f/f3a [3587736,43637] 0 2024-05-27T09:29:18.727 INFO:tasks.workunit.client.0.smithi119.stdout:3/469: creat db/d36/d45/d4c/d62/d76/d98/d7a/f9e x:0 0 0 2024-05-27T09:29:18.735 INFO:tasks.workunit.client.0.smithi119.stdout:1/368: link c1 d7/c74 0 2024-05-27T09:29:18.739 INFO:tasks.workunit.client.0.smithi119.stdout:0/313: dread d2/d6/d9/d24/d30/d4a/f4c [0,4194304] 0 2024-05-27T09:29:18.740 INFO:tasks.workunit.client.0.smithi119.stdout:1/369: link d7/db/d35/l48 d7/dd/d44/l75 0 2024-05-27T09:29:18.742 INFO:tasks.workunit.client.0.smithi119.stdout:0/314: mkdir d2/d6/d9/d12/d4b/d6a 0 2024-05-27T09:29:18.745 INFO:tasks.workunit.client.0.smithi119.stdout:1/370: mknod d7/db/d52/d65/c76 0 2024-05-27T09:29:18.745 INFO:tasks.workunit.client.0.smithi119.stdout:1/371: truncate d7/f5b 909812 0 2024-05-27T09:29:18.745 INFO:tasks.workunit.client.0.smithi119.stdout:1/372: chown d7/dd/f5d 479 1 2024-05-27T09:29:18.745 INFO:tasks.workunit.client.0.smithi119.stdout:1/373: dread - d7/db/d12/f6b zero size 2024-05-27T09:29:18.746 INFO:tasks.workunit.client.0.smithi119.stdout:0/315: truncate d2/d1a/f1e 1521682 0 2024-05-27T09:29:18.746 INFO:tasks.workunit.client.0.smithi119.stdout:0/316: chown d2/d1a/f1e 34 1 2024-05-27T09:29:18.749 INFO:tasks.workunit.client.0.smithi119.stdout:1/374: truncate d7/db/d12/d26/f43 3015482 0 2024-05-27T09:29:18.749 INFO:tasks.workunit.client.0.smithi119.stdout:1/375: readlink d7/dd/d44/d47/l64 0 2024-05-27T09:29:18.751 INFO:tasks.workunit.client.0.smithi119.stdout:1/376: creat d7/dd/f77 x:0 0 0 2024-05-27T09:29:18.753 INFO:tasks.workunit.client.0.smithi119.stdout:1/377: symlink d7/dd/d25/d60/l78 0 2024-05-27T09:29:18.754 INFO:tasks.workunit.client.0.smithi119.stdout:1/378: truncate d7/db/d12/d15/f3f 514052 0 2024-05-27T09:29:18.756 INFO:tasks.workunit.client.0.smithi119.stdout:1/379: symlink d7/db/d12/d15/d1f/l79 0 2024-05-27T09:29:18.756 INFO:tasks.workunit.client.0.smithi119.stdout:1/380: stat d7/dd/d25/d29/d38/d3c/f53 0 2024-05-27T09:29:18.757 INFO:tasks.workunit.client.0.smithi119.stdout:1/381: mknod d7/db/d52/d65/c7a 0 2024-05-27T09:29:18.764 INFO:tasks.workunit.client.0.smithi119.stdout:9/265: dwrite f2 [0,4194304] 0 2024-05-27T09:29:18.769 INFO:tasks.workunit.client.0.smithi119.stdout:9/266: mknod d18/d19/c56 0 2024-05-27T09:29:18.778 INFO:tasks.workunit.client.0.smithi119.stdout:5/328: dwrite d6/d2f/f42 [0,4194304] 0 2024-05-27T09:29:18.778 INFO:tasks.workunit.client.0.smithi119.stdout:4/301: dwrite d4/f4c [0,4194304] 0 2024-05-27T09:29:18.781 INFO:tasks.workunit.client.0.smithi119.stdout:9/267: dread d18/d19/f3f [0,4194304] 0 2024-05-27T09:29:18.782 INFO:tasks.workunit.client.0.smithi119.stdout:9/268: write f6 [2644629,81218] 0 2024-05-27T09:29:18.782 INFO:tasks.workunit.client.0.smithi119.stdout:9/269: dread - d18/d19/f4e zero size 2024-05-27T09:29:18.789 INFO:tasks.workunit.client.0.smithi119.stdout:4/302: mkdir d4/d6/d28/d4a/d5f 0 2024-05-27T09:29:18.794 INFO:tasks.workunit.client.0.smithi119.stdout:9/270: getdents d18/d19/d1b/d40/d44 0 2024-05-27T09:29:18.806 INFO:tasks.workunit.client.0.smithi119.stdout:1/382: dwrite d7/dd/f5d [0,4194304] 0 2024-05-27T09:29:18.844 INFO:tasks.workunit.client.0.smithi119.stdout:3/470: dwrite db/d36/d45/d4c/d62/d13/f2e [0,4194304] 0 2024-05-27T09:29:18.844 INFO:tasks.workunit.client.0.smithi119.stdout:3/471: fsync db/d36/f71 0 2024-05-27T09:29:18.845 INFO:tasks.workunit.client.0.smithi119.stdout:0/317: write d2/d4/f46 [833349,99574] 0 2024-05-27T09:29:18.847 INFO:tasks.workunit.client.0.smithi119.stdout:0/318: mkdir d2/d6/d9/d24/d30/d6b 0 2024-05-27T09:29:18.854 INFO:tasks.workunit.client.0.smithi119.stdout:0/319: fdatasync d2/d4/f46 0 2024-05-27T09:29:18.854 INFO:tasks.workunit.client.0.smithi119.stdout:5/329: dwrite d6/db/dc/f3b [0,4194304] 0 2024-05-27T09:29:18.854 INFO:tasks.workunit.client.0.smithi119.stdout:5/330: chown d6/dd/d46/d47/d4b/c60 256979 1 2024-05-27T09:29:18.858 INFO:tasks.workunit.client.0.smithi119.stdout:0/320: unlink d2/d6/d9/l50 0 2024-05-27T09:29:18.858 INFO:tasks.workunit.client.0.smithi119.stdout:0/321: stat d2/d6/d23/c60 0 2024-05-27T09:29:18.860 INFO:tasks.workunit.client.0.smithi119.stdout:5/331: creat d6/db/f6f x:0 0 0 2024-05-27T09:29:18.864 INFO:tasks.workunit.client.0.smithi119.stdout:1/383: dwrite d7/db/d12/d15/d1f/f3a [0,4194304] 0 2024-05-27T09:29:18.876 INFO:tasks.workunit.client.0.smithi119.stdout:1/384: mknod d7/db/d12/d15/d1f/c7b 0 2024-05-27T09:29:18.877 INFO:tasks.workunit.client.0.smithi119.stdout:1/385: fsync d7/dd/d44/d47/f72 0 2024-05-27T09:29:18.877 INFO:tasks.workunit.client.0.smithi119.stdout:1/386: readlink d7/dd/d25/d29/d38/d3c/l41 0 2024-05-27T09:29:18.877 INFO:tasks.workunit.client.0.smithi119.stdout:2/324: sync 2024-05-27T09:29:18.877 INFO:tasks.workunit.client.0.smithi119.stdout:8/335: sync 2024-05-27T09:29:18.877 INFO:tasks.workunit.client.0.smithi119.stdout:6/289: sync 2024-05-27T09:29:18.877 INFO:tasks.workunit.client.0.smithi119.stdout:6/290: dread - d0/d2/d57/d37/f5b zero size 2024-05-27T09:29:18.877 INFO:tasks.workunit.client.0.smithi119.stdout:7/378: sync 2024-05-27T09:29:18.878 INFO:tasks.workunit.client.0.smithi119.stdout:1/387: link d7/f14 d7/dd/d25/d60/d61/f7c 0 2024-05-27T09:29:18.880 INFO:tasks.workunit.client.0.smithi119.stdout:8/336: getdents d3/d8/d1b/d46 0 2024-05-27T09:29:18.887 INFO:tasks.workunit.client.0.smithi119.stdout:6/291: creat d0/d2/f67 x:0 0 0 2024-05-27T09:29:18.887 INFO:tasks.workunit.client.0.smithi119.stdout:6/292: fdatasync d0/d2/f31 0 2024-05-27T09:29:18.891 INFO:tasks.workunit.client.0.smithi119.stdout:2/325: rmdir d2/d1a/d51 39 2024-05-27T09:29:18.891 INFO:tasks.workunit.client.0.smithi119.stdout:8/337: unlink d3/d8/c18 0 2024-05-27T09:29:18.892 INFO:tasks.workunit.client.0.smithi119.stdout:6/293: link d0/d2/d1c/f4c d0/f68 0 2024-05-27T09:29:18.895 INFO:tasks.workunit.client.0.smithi119.stdout:6/294: creat d0/d19/f69 x:0 0 0 2024-05-27T09:29:18.898 INFO:tasks.workunit.client.0.smithi119.stdout:7/379: dread d7/df/d19/f1b [0,4194304] 0 2024-05-27T09:29:18.898 INFO:tasks.workunit.client.0.smithi119.stdout:7/380: write d7/df/d19/d46/f68 [752423,29944] 0 2024-05-27T09:29:18.908 INFO:tasks.workunit.client.0.smithi119.stdout:6/295: creat d0/d2/d4e/f6a x:0 0 0 2024-05-27T09:29:18.913 INFO:tasks.workunit.client.0.smithi119.stdout:7/381: rmdir d7/d6a/d75 0 2024-05-27T09:29:18.913 INFO:tasks.workunit.client.0.smithi119.stdout:2/326: dread d2/d1a/d51/f56 [0,4194304] 0 2024-05-27T09:29:18.913 INFO:tasks.workunit.client.0.smithi119.stdout:2/327: readlink d2/da/d15/d33/l50 0 2024-05-27T09:29:18.915 INFO:tasks.workunit.client.0.smithi119.stdout:6/296: link d0/d2/d57/d37/l38 d0/d11/l6b 0 2024-05-27T09:29:18.918 INFO:tasks.workunit.client.0.smithi119.stdout:5/332: dwrite d6/db/dc/f3b [0,4194304] 0 2024-05-27T09:29:18.919 INFO:tasks.workunit.client.0.smithi119.stdout:2/328: mknod d2/da/d15/d53/c71 0 2024-05-27T09:29:18.924 INFO:tasks.workunit.client.0.smithi119.stdout:2/329: truncate d2/f2d 3626058 0 2024-05-27T09:29:18.924 INFO:tasks.workunit.client.0.smithi119.stdout:2/330: write d2/da/d15/f5f [14024,23122] 0 2024-05-27T09:29:18.941 INFO:tasks.workunit.client.0.smithi119.stdout:7/382: dread d7/df/d36/d42/f73 [0,4194304] 0 2024-05-27T09:29:18.943 INFO:tasks.workunit.client.0.smithi119.stdout:5/333: dread d6/db/dc/f26 [0,4194304] 0 2024-05-27T09:29:18.943 INFO:tasks.workunit.client.0.smithi119.stdout:5/334: write d6/db/de/d27/f6b [688353,79856] 0 2024-05-27T09:29:18.943 INFO:tasks.workunit.client.0.smithi119.stdout:7/383: mkdir d7/df/d19/d78 0 2024-05-27T09:29:18.947 INFO:tasks.workunit.client.0.smithi119.stdout:5/335: mkdir d6/dd/d39/d19/d70 0 2024-05-27T09:29:18.951 INFO:tasks.workunit.client.0.smithi119.stdout:7/384: mkdir d7/d5f/d79 0 2024-05-27T09:29:18.951 INFO:tasks.workunit.client.0.smithi119.stdout:7/385: creat d7/df/d36/d42/d64/f7a x:0 0 0 2024-05-27T09:29:18.953 INFO:tasks.workunit.client.0.smithi119.stdout:7/386: read d7/d1a/d40/d44/f5d [100800,85404] 0 2024-05-27T09:29:18.953 INFO:tasks.workunit.client.0.smithi119.stdout:7/387: truncate d7/fa 331710 0 2024-05-27T09:29:18.953 INFO:tasks.workunit.client.0.smithi119.stdout:3/472: dwrite db/d36/d45/d4c/d62/d76/f79 [0,4194304] 0 2024-05-27T09:29:18.960 INFO:tasks.workunit.client.0.smithi119.stdout:8/338: dwrite d3/d8/d1b/d46/f4b [0,4194304] 0 2024-05-27T09:29:18.964 INFO:tasks.workunit.client.0.smithi119.stdout:7/388: unlink d7/l27 0 2024-05-27T09:29:18.964 INFO:tasks.workunit.client.0.smithi119.stdout:3/473: creat db/d36/d45/d4c/d62/d76/d98/f9f x:0 0 0 2024-05-27T09:29:18.964 INFO:tasks.workunit.client.0.smithi119.stdout:3/474: fdatasync db/d36/d45/d4d/f58 0 2024-05-27T09:29:18.964 INFO:tasks.workunit.client.0.smithi119.stdout:3/475: creat db/fa0 x:0 0 0 2024-05-27T09:29:18.964 INFO:tasks.workunit.client.0.smithi119.stdout:3/476: readlink db/l26 0 2024-05-27T09:29:18.965 INFO:tasks.workunit.client.0.smithi119.stdout:3/477: read db/d36/d45/d4c/d62/d13/f2a [1280820,74224] 0 2024-05-27T09:29:18.965 INFO:tasks.workunit.client.0.smithi119.stdout:3/478: chown db/d36/d45/d4c/d62/d13/l15 29 1 2024-05-27T09:29:18.968 INFO:tasks.workunit.client.0.smithi119.stdout:3/479: link db/d36/d45/d4c/d62/d76/d98/c8f db/d36/d45/d49/d9d/ca1 0 2024-05-27T09:29:18.976 INFO:tasks.workunit.client.0.smithi119.stdout:7/389: write d7/df/d19/f1b [1981283,49447] 0 2024-05-27T09:29:18.976 INFO:tasks.workunit.client.0.smithi119.stdout:6/297: dwrite d0/d2/d57/d37/f5b [0,4194304] 0 2024-05-27T09:29:18.977 INFO:tasks.workunit.client.0.smithi119.stdout:6/298: chown d0/d2/d57/d37/f43 12 1 2024-05-27T09:29:18.977 INFO:tasks.workunit.client.0.smithi119.stdout:8/339: dread d3/d8/d1b/d46/f37 [0,4194304] 0 2024-05-27T09:29:18.981 INFO:tasks.workunit.client.0.smithi119.stdout:7/390: mkdir d7/d6a/d7b 0 2024-05-27T09:29:18.981 INFO:tasks.workunit.client.0.smithi119.stdout:7/391: chown d7/d1a 3437946 1 2024-05-27T09:29:18.981 INFO:tasks.workunit.client.0.smithi119.stdout:7/392: getdents d7/df/d36/d42/d76 0 2024-05-27T09:29:18.982 INFO:tasks.workunit.client.0.smithi119.stdout:6/299: mknod d0/d11/c6c 0 2024-05-27T09:29:18.982 INFO:tasks.workunit.client.0.smithi119.stdout:8/340: chown d3/c31 2754 1 2024-05-27T09:29:18.983 INFO:tasks.workunit.client.0.smithi119.stdout:8/341: read - d3/de/f64 zero size 2024-05-27T09:29:18.984 INFO:tasks.workunit.client.0.smithi119.stdout:7/393: creat d7/d5f/f7c x:0 0 0 2024-05-27T09:29:18.985 INFO:tasks.workunit.client.0.smithi119.stdout:6/300: link d0/d2/f67 d0/d2/d57/f6d 0 2024-05-27T09:29:18.986 INFO:tasks.workunit.client.0.smithi119.stdout:8/342: creat d3/d39/f72 x:0 0 0 2024-05-27T09:29:18.987 INFO:tasks.workunit.client.0.smithi119.stdout:7/394: symlink d7/df/d19/l7d 0 2024-05-27T09:29:18.989 INFO:tasks.workunit.client.0.smithi119.stdout:6/301: link d0/d2/f4 d0/d2/d57/d37/f6e 0 2024-05-27T09:29:18.990 INFO:tasks.workunit.client.0.smithi119.stdout:8/343: symlink d3/d8/d4d/d5d/d66/l73 0 2024-05-27T09:29:18.990 INFO:tasks.workunit.client.0.smithi119.stdout:8/344: readlink d3/d8/d4d/d5d/l4e 0 2024-05-27T09:29:18.993 INFO:tasks.workunit.client.0.smithi119.stdout:7/395: truncate d7/d1a/d40/f48 7361089 0 2024-05-27T09:29:18.993 INFO:tasks.workunit.client.0.smithi119.stdout:1/388: dwrite d7/dd/d25/d29/d38/d3c/f53 [0,4194304] 0 2024-05-27T09:29:18.993 INFO:tasks.workunit.client.0.smithi119.stdout:1/389: write d7/db/d12/d15/f1d [3470646,29794] 0 2024-05-27T09:29:19.009 INFO:tasks.workunit.client.0.smithi119.stdout:7/396: symlink d7/db/l7e 0 2024-05-27T09:29:19.011 INFO:tasks.workunit.client.0.smithi119.stdout:7/397: mknod d7/db/d37/c7f 0 2024-05-27T09:29:19.042 INFO:tasks.workunit.client.0.smithi119.stdout:5/336: dwrite d6/dd/df/f5c [0,4194304] 0 2024-05-27T09:29:19.042 INFO:tasks.workunit.client.0.smithi119.stdout:5/337: chown d6/dd/d39/d19/d3c/d52 1555 1 2024-05-27T09:29:19.063 INFO:tasks.workunit.client.0.smithi119.stdout:6/302: dwrite d0/d2/f31 [0,4194304] 0 2024-05-27T09:29:19.063 INFO:tasks.workunit.client.0.smithi119.stdout:6/303: stat d0/l34 0 2024-05-27T09:29:19.063 INFO:tasks.workunit.client.0.smithi119.stdout:7/398: dwrite d7/d1a/f2d [0,4194304] 0 2024-05-27T09:29:19.063 INFO:tasks.workunit.client.0.smithi119.stdout:7/399: readlink d7/l58 0 2024-05-27T09:29:19.064 INFO:tasks.workunit.client.0.smithi119.stdout:5/338: write d6/db/de/f25 [8207538,925] 0 2024-05-27T09:29:19.064 INFO:tasks.workunit.client.0.smithi119.stdout:5/339: fsync d6/db/dc/f26 0 2024-05-27T09:29:19.064 INFO:tasks.workunit.client.0.smithi119.stdout:6/304: truncate d0/d2/d1c/d21/f3c 250272 0 2024-05-27T09:29:19.064 INFO:tasks.workunit.client.0.smithi119.stdout:5/340: creat d6/db/f71 x:0 0 0 2024-05-27T09:29:19.064 INFO:tasks.workunit.client.0.smithi119.stdout:5/341: creat d6/db/de/d27/f72 x:0 0 0 2024-05-27T09:29:19.065 INFO:tasks.workunit.client.0.smithi119.stdout:5/342: creat d6/dd/f73 x:0 0 0 2024-05-27T09:29:19.066 INFO:tasks.workunit.client.0.smithi119.stdout:8/345: dwrite d3/d8/d1b/f6c [0,4194304] 0 2024-05-27T09:29:19.066 INFO:tasks.workunit.client.0.smithi119.stdout:8/346: fsync d3/de/f64 0 2024-05-27T09:29:19.066 INFO:tasks.workunit.client.0.smithi119.stdout:7/400: creat d7/d1a/d40/d44/f80 x:0 0 0 2024-05-27T09:29:19.080 INFO:tasks.workunit.client.0.smithi119.stdout:7/401: mkdir d7/db/d81 0 2024-05-27T09:29:19.097 INFO:tasks.workunit.client.0.smithi119.stdout:9/271: sync 2024-05-27T09:29:19.098 INFO:tasks.workunit.client.0.smithi119.stdout:9/272: chown d18/d19/d1b/d29 873276527 1 2024-05-27T09:29:19.098 INFO:tasks.workunit.client.0.smithi119.stdout:0/322: sync 2024-05-27T09:29:19.098 INFO:tasks.workunit.client.0.smithi119.stdout:4/303: sync 2024-05-27T09:29:19.098 INFO:tasks.workunit.client.0.smithi119.stdout:4/304: read - d4/de/d21/f2b zero size 2024-05-27T09:29:19.098 INFO:tasks.workunit.client.0.smithi119.stdout:9/273: rmdir d18/d19/d1b/d40/d44 39 2024-05-27T09:29:19.104 INFO:tasks.workunit.client.0.smithi119.stdout:4/305: mkdir d4/d60 0 2024-05-27T09:29:19.114 INFO:tasks.workunit.client.0.smithi119.stdout:4/306: dread d4/d23/d31/f33 [0,4194304] 0 2024-05-27T09:29:19.114 INFO:tasks.workunit.client.0.smithi119.stdout:4/307: chown d4/f41 22716 1 2024-05-27T09:29:19.114 INFO:tasks.workunit.client.0.smithi119.stdout:4/308: read - d4/de/d21/f2b zero size 2024-05-27T09:29:19.124 INFO:tasks.workunit.client.0.smithi119.stdout:5/343: dwrite d6/dd/f32 [0,4194304] 0 2024-05-27T09:29:19.124 INFO:tasks.workunit.client.0.smithi119.stdout:5/344: chown d6/dd/f1b 245 1 2024-05-27T09:29:19.124 INFO:tasks.workunit.client.0.smithi119.stdout:5/345: chown d6/dd/df 406 1 2024-05-27T09:29:19.135 INFO:tasks.workunit.client.0.smithi119.stdout:4/309: write d4/f32 [269745,12892] 0 2024-05-27T09:29:19.135 INFO:tasks.workunit.client.0.smithi119.stdout:4/310: write d4/f32 [2634264,64456] 0 2024-05-27T09:29:19.137 INFO:tasks.workunit.client.0.smithi119.stdout:5/346: mkdir d6/dd/d39/d74 0 2024-05-27T09:29:19.138 INFO:tasks.workunit.client.0.smithi119.stdout:0/323: write d2/d1a/d20/f64 [2439216,68006] 0 2024-05-27T09:29:19.138 INFO:tasks.workunit.client.0.smithi119.stdout:6/305: dwrite d0/d2/d1c/f3e [0,4194304] 0 2024-05-27T09:29:19.138 INFO:tasks.workunit.client.0.smithi119.stdout:6/306: truncate d0/d11/f62 388916 0 2024-05-27T09:29:19.140 INFO:tasks.workunit.client.0.smithi119.stdout:7/402: dwrite d7/df/d19/f1f [0,4194304] 0 2024-05-27T09:29:19.141 INFO:tasks.workunit.client.0.smithi119.stdout:4/311: rmdir d4/d6/d28/d4a/d5f 0 2024-05-27T09:29:19.141 INFO:tasks.workunit.client.0.smithi119.stdout:5/347: creat d6/dd/df/f75 x:0 0 0 2024-05-27T09:29:19.155 INFO:tasks.workunit.client.0.smithi119.stdout:0/324: link d2/le d2/d6/d9/d12/d4f/l6c 0 2024-05-27T09:29:19.156 INFO:tasks.workunit.client.0.smithi119.stdout:7/403: creat d7/df/d36/d42/d76/f82 x:0 0 0 2024-05-27T09:29:19.156 INFO:tasks.workunit.client.0.smithi119.stdout:4/312: mknod d4/de/d21/d40/d47/d5c/c61 0 2024-05-27T09:29:19.156 INFO:tasks.workunit.client.0.smithi119.stdout:4/313: fdatasync d4/d1c/d39/f4b 0 2024-05-27T09:29:19.156 INFO:tasks.workunit.client.0.smithi119.stdout:4/314: fsync d4/d6/fa 0 2024-05-27T09:29:19.156 INFO:tasks.workunit.client.0.smithi119.stdout:4/315: dread - d4/d6/f3b zero size 2024-05-27T09:29:19.156 INFO:tasks.workunit.client.0.smithi119.stdout:4/316: stat d4/d6/fa 0 2024-05-27T09:29:19.156 INFO:tasks.workunit.client.0.smithi119.stdout:4/317: write d4/de/d21/d40/f42 [1233513,48682] 0 2024-05-27T09:29:19.156 INFO:tasks.workunit.client.0.smithi119.stdout:4/318: write d4/de/d21/d40/d47/f49 [398655,16122] 0 2024-05-27T09:29:19.158 INFO:tasks.workunit.client.0.smithi119.stdout:4/319: rename d4/de/d21/d40/d47/d5c/c61 to d4/d1c/d39/c62 0 2024-05-27T09:29:19.217 INFO:tasks.workunit.client.0.smithi119.stdout:7/404: dwrite d7/df/d19/d4d/f71 [0,4194304] 0 2024-05-27T09:29:19.225 INFO:tasks.workunit.client.0.smithi119.stdout:7/405: dread d7/fa [0,4194304] 0 2024-05-27T09:29:19.228 INFO:tasks.workunit.client.0.smithi119.stdout:5/348: dread d6/db/de/f25 [0,4194304] 0 2024-05-27T09:29:19.228 INFO:tasks.workunit.client.0.smithi119.stdout:6/307: dwrite d0/d2/d1c/d21/f3b [0,4194304] 0 2024-05-27T09:29:19.228 INFO:tasks.workunit.client.0.smithi119.stdout:7/406: link d7/df/d36/d42/c6e d7/d1a/d40/d44/d67/c83 0 2024-05-27T09:29:19.229 INFO:tasks.workunit.client.0.smithi119.stdout:0/325: dwrite d2/d6/d9/f1c [0,4194304] 0 2024-05-27T09:29:19.230 INFO:tasks.workunit.client.0.smithi119.stdout:7/407: mknod d7/c84 0 2024-05-27T09:29:19.232 INFO:tasks.workunit.client.0.smithi119.stdout:0/326: rename d2/d6/d9/ff to d2/d6/d9/f6d 0 2024-05-27T09:29:19.241 INFO:tasks.workunit.client.0.smithi119.stdout:4/320: dwrite d4/d6/f3b [0,4194304] 0 2024-05-27T09:29:19.241 INFO:tasks.workunit.client.0.smithi119.stdout:0/327: mknod d2/d6/d9/d24/d30/c6e 0 2024-05-27T09:29:19.242 INFO:tasks.workunit.client.0.smithi119.stdout:5/349: dread d6/d2f/f51 [0,4194304] 0 2024-05-27T09:29:19.242 INFO:tasks.workunit.client.0.smithi119.stdout:7/408: rmdir d7/df/d19/d4d 39 2024-05-27T09:29:19.244 INFO:tasks.workunit.client.0.smithi119.stdout:6/308: dread d0/d2/d57/d37/f43 [0,4194304] 0 2024-05-27T09:29:19.244 INFO:tasks.workunit.client.0.smithi119.stdout:4/321: mknod d4/d6/d28/d2f/c63 0 2024-05-27T09:29:19.244 INFO:tasks.workunit.client.0.smithi119.stdout:4/322: creat d4/d1c/d39/f64 x:0 0 0 2024-05-27T09:29:19.245 INFO:tasks.workunit.client.0.smithi119.stdout:7/409: rmdir d7/df/d19 39 2024-05-27T09:29:19.249 INFO:tasks.workunit.client.0.smithi119.stdout:4/323: mknod d4/d48/c65 0 2024-05-27T09:29:19.251 INFO:tasks.workunit.client.0.smithi119.stdout:4/324: mkdir d4/de/d66 0 2024-05-27T09:29:19.251 INFO:tasks.workunit.client.0.smithi119.stdout:4/325: chown d4/de/d21/d40/f42 3675 1 2024-05-27T09:29:19.251 INFO:tasks.workunit.client.0.smithi119.stdout:4/326: fsync d4/f22 0 2024-05-27T09:29:19.260 INFO:tasks.workunit.client.0.smithi119.stdout:2/331: sync 2024-05-27T09:29:19.260 INFO:tasks.workunit.client.0.smithi119.stdout:2/332: creat d2/da/d4e/f72 x:0 0 0 2024-05-27T09:29:19.260 INFO:tasks.workunit.client.0.smithi119.stdout:1/390: sync 2024-05-27T09:29:19.262 INFO:tasks.workunit.client.0.smithi119.stdout:0/328: write d2/d6/d9/d12/d4f/f5a [930396,25263] 0 2024-05-27T09:29:19.262 INFO:tasks.workunit.client.0.smithi119.stdout:0/329: chown d2/d1a/f1e 521 1 2024-05-27T09:29:19.263 INFO:tasks.workunit.client.0.smithi119.stdout:4/327: rename d4/f4c to d4/d6/f67 0 2024-05-27T09:29:19.265 INFO:tasks.workunit.client.0.smithi119.stdout:2/333: creat d2/d1a/d51/f73 x:0 0 0 2024-05-27T09:29:19.271 INFO:tasks.workunit.client.0.smithi119.stdout:0/330: link d2/d6/d9/d24/l56 d2/d6/d9/d12/d4f/l6f 0 2024-05-27T09:29:19.271 INFO:tasks.workunit.client.0.smithi119.stdout:0/331: fsync d2/d6/d9/d12/f32 0 2024-05-27T09:29:19.271 INFO:tasks.workunit.client.0.smithi119.stdout:4/328: creat d4/de/d66/f68 x:0 0 0 2024-05-27T09:29:19.271 INFO:tasks.workunit.client.0.smithi119.stdout:0/332: chown d2/l10 105132 1 2024-05-27T09:29:19.272 INFO:tasks.workunit.client.0.smithi119.stdout:0/333: rmdir d2/d4/d53 39 2024-05-27T09:29:19.280 INFO:tasks.workunit.client.0.smithi119.stdout:1/391: chown d7/db/d35/f3e 8799 1 2024-05-27T09:29:19.280 INFO:tasks.workunit.client.0.smithi119.stdout:1/392: creat d7/db/f7d x:0 0 0 2024-05-27T09:29:19.284 INFO:tasks.workunit.client.0.smithi119.stdout:1/393: rmdir d7/dd/d25/d29 39 2024-05-27T09:29:19.297 INFO:tasks.workunit.client.0.smithi119.stdout:3/480: sync 2024-05-27T09:29:19.297 INFO:tasks.workunit.client.0.smithi119.stdout:3/481: dread - db/d36/d45/d49/f85 zero size 2024-05-27T09:29:19.299 INFO:tasks.workunit.client.0.smithi119.stdout:3/482: unlink l5 0 2024-05-27T09:29:19.303 INFO:tasks.workunit.client.0.smithi119.stdout:3/483: truncate db/d36/d45/d4c/d62/f8e 246872 0 2024-05-27T09:29:19.303 INFO:tasks.workunit.client.0.smithi119.stdout:3/484: fdatasync db/d36/d45/d4c/d62/f17 0 2024-05-27T09:29:19.303 INFO:tasks.workunit.client.0.smithi119.stdout:3/485: chown db/d36/d45/d49/f85 34 1 2024-05-27T09:29:19.303 INFO:tasks.workunit.client.0.smithi119.stdout:1/394: dread d7/dd/d25/d60/d61/f7c [0,4194304] 0 2024-05-27T09:29:19.311 INFO:tasks.workunit.client.0.smithi119.stdout:5/350: dwrite d6/dd/f64 [0,4194304] 0 2024-05-27T09:29:19.314 INFO:tasks.workunit.client.0.smithi119.stdout:9/274: sync 2024-05-27T09:29:19.327 INFO:tasks.workunit.client.0.smithi119.stdout:5/351: read d6/db/dc/f26 [2893567,130483] 0 2024-05-27T09:29:19.327 INFO:tasks.workunit.client.0.smithi119.stdout:5/352: fdatasync d6/f6a 0 2024-05-27T09:29:19.327 INFO:tasks.workunit.client.0.smithi119.stdout:5/353: creat d6/db/de/d27/f76 x:0 0 0 2024-05-27T09:29:19.329 INFO:tasks.workunit.client.0.smithi119.stdout:3/486: dread db/d36/d45/d4c/d62/f3f [0,4194304] 0 2024-05-27T09:29:19.329 INFO:tasks.workunit.client.0.smithi119.stdout:3/487: read db/d36/d45/d4c/d62/d13/f21 [3119268,128623] 0 2024-05-27T09:29:19.333 INFO:tasks.workunit.client.0.smithi119.stdout:5/354: mkdir d6/db/de/d27/d77 0 2024-05-27T09:29:19.338 INFO:tasks.workunit.client.0.smithi119.stdout:2/334: read d2/da/d4e/f4f [3066542,43902] 0 2024-05-27T09:29:19.338 INFO:tasks.workunit.client.0.smithi119.stdout:7/410: dwrite d7/df/d36/f49 [0,4194304] 0 2024-05-27T09:29:19.338 INFO:tasks.workunit.client.0.smithi119.stdout:7/411: fdatasync d7/df/d19/d1d/f23 0 2024-05-27T09:29:19.338 INFO:tasks.workunit.client.0.smithi119.stdout:7/412: creat d7/df/d19/d46/f85 x:0 0 0 2024-05-27T09:29:19.349 INFO:tasks.workunit.client.0.smithi119.stdout:2/335: mknod d2/da/d4e/c74 0 2024-05-27T09:29:19.351 INFO:tasks.workunit.client.0.smithi119.stdout:8/347: sync 2024-05-27T09:29:19.351 INFO:tasks.workunit.client.0.smithi119.stdout:8/348: dread - d3/de/f68 zero size 2024-05-27T09:29:19.351 INFO:tasks.workunit.client.0.smithi119.stdout:2/336: mknod d2/c75 0 2024-05-27T09:29:19.351 INFO:tasks.workunit.client.0.smithi119.stdout:2/337: chown d2/da/f2e 11279259 1 2024-05-27T09:29:19.351 INFO:tasks.workunit.client.0.smithi119.stdout:2/338: creat d2/da/d15/d33/d69/f76 x:0 0 0 2024-05-27T09:29:19.351 INFO:tasks.workunit.client.0.smithi119.stdout:8/349: mkdir d3/d8/d4d/d5d/d74 0 2024-05-27T09:29:19.353 INFO:tasks.workunit.client.0.smithi119.stdout:2/339: symlink d2/da/d15/d53/l77 0 2024-05-27T09:29:19.354 INFO:tasks.workunit.client.0.smithi119.stdout:6/309: dwrite d0/f68 [0,4194304] 0 2024-05-27T09:29:19.355 INFO:tasks.workunit.client.0.smithi119.stdout:6/310: truncate d0/f18 1082829 0 2024-05-27T09:29:19.360 INFO:tasks.workunit.client.0.smithi119.stdout:8/350: creat d3/d6e/f75 x:0 0 0 2024-05-27T09:29:19.361 INFO:tasks.workunit.client.0.smithi119.stdout:2/340: symlink d2/da/d2c/l78 0 2024-05-27T09:29:19.363 INFO:tasks.workunit.client.0.smithi119.stdout:2/341: unlink d2/d1a/f27 0 2024-05-27T09:29:19.364 INFO:tasks.workunit.client.0.smithi119.stdout:4/329: dwrite d4/d6/f1a [4194304,4194304] 0 2024-05-27T09:29:19.364 INFO:tasks.workunit.client.0.smithi119.stdout:9/275: dread fa [0,4194304] 0 2024-05-27T09:29:19.364 INFO:tasks.workunit.client.0.smithi119.stdout:9/276: creat d18/d19/d1b/d29/f57 x:0 0 0 2024-05-27T09:29:19.366 INFO:tasks.workunit.client.0.smithi119.stdout:2/342: truncate d2/da/f10 91405 0 2024-05-27T09:29:19.371 INFO:tasks.workunit.client.0.smithi119.stdout:9/277: rename c5 to d18/d19/c58 0 2024-05-27T09:29:19.372 INFO:tasks.workunit.client.0.smithi119.stdout:9/278: write f10 [1637668,113415] 0 2024-05-27T09:29:19.380 INFO:tasks.workunit.client.0.smithi119.stdout:7/413: write d7/df/d36/f3f [38406,105149] 0 2024-05-27T09:29:19.380 INFO:tasks.workunit.client.0.smithi119.stdout:7/414: stat d7/d1a/d40/f48 0 2024-05-27T09:29:19.380 INFO:tasks.workunit.client.0.smithi119.stdout:7/415: fsync d7/f6b 0 2024-05-27T09:29:19.380 INFO:tasks.workunit.client.0.smithi119.stdout:6/311: dread d0/d2/d1c/d21/d58/f61 [0,4194304] 0 2024-05-27T09:29:19.383 INFO:tasks.workunit.client.0.smithi119.stdout:2/343: write d2/f48 [2887144,95291] 0 2024-05-27T09:29:19.389 INFO:tasks.workunit.client.0.smithi119.stdout:6/312: symlink d0/d2/d4e/d5a/l6f 0 2024-05-27T09:29:19.389 INFO:tasks.workunit.client.0.smithi119.stdout:2/344: creat d2/da/d15/d33/f79 x:0 0 0 2024-05-27T09:29:19.389 INFO:tasks.workunit.client.0.smithi119.stdout:2/345: dread - d2/da/d15/d33/d69/f6b zero size 2024-05-27T09:29:19.389 INFO:tasks.workunit.client.0.smithi119.stdout:2/346: stat d2/c11 0 2024-05-27T09:29:19.390 INFO:tasks.workunit.client.0.smithi119.stdout:2/347: rename d2/da/d15/l44 to d2/d1a/d51/l7a 0 2024-05-27T09:29:19.390 INFO:tasks.workunit.client.0.smithi119.stdout:2/348: dread - d2/da/d15/d33/d69/f76 zero size 2024-05-27T09:29:19.390 INFO:tasks.workunit.client.0.smithi119.stdout:2/349: readlink d2/da/l4a 0 2024-05-27T09:29:19.390 INFO:tasks.workunit.client.0.smithi119.stdout:2/350: dread - d2/da/f4d zero size 2024-05-27T09:29:19.390 INFO:tasks.workunit.client.0.smithi119.stdout:7/416: dread d7/d1a/f2b [0,4194304] 0 2024-05-27T09:29:19.390 INFO:tasks.workunit.client.0.smithi119.stdout:2/351: symlink d2/da/d6f/l7b 0 2024-05-27T09:29:19.391 INFO:tasks.workunit.client.0.smithi119.stdout:2/352: fdatasync d2/da/f13 0 2024-05-27T09:29:19.391 INFO:tasks.workunit.client.0.smithi119.stdout:7/417: getdents d7/df/d19/d4d 0 2024-05-27T09:29:19.391 INFO:tasks.workunit.client.0.smithi119.stdout:1/395: dwrite d7/dd/d25/d60/d61/f7c [0,4194304] 0 2024-05-27T09:29:19.391 INFO:tasks.workunit.client.0.smithi119.stdout:2/353: symlink d2/d1a/l7c 0 2024-05-27T09:29:19.392 INFO:tasks.workunit.client.0.smithi119.stdout:2/354: truncate d2/da/f10 523808 0 2024-05-27T09:29:19.393 INFO:tasks.workunit.client.0.smithi119.stdout:7/418: rmdir d7/d1a/d40/d44/d67 39 2024-05-27T09:29:19.393 INFO:tasks.workunit.client.0.smithi119.stdout:7/419: write d7/df/d19/d1d/f4f [235439,91893] 0 2024-05-27T09:29:19.393 INFO:tasks.workunit.client.0.smithi119.stdout:7/420: chown d7/db/cc 125 1 2024-05-27T09:29:19.393 INFO:tasks.workunit.client.0.smithi119.stdout:7/421: fsync d7/d5f/f7c 0 2024-05-27T09:29:19.393 INFO:tasks.workunit.client.0.smithi119.stdout:7/422: write d7/f62 [1679509,77559] 0 2024-05-27T09:29:19.394 INFO:tasks.workunit.client.0.smithi119.stdout:7/423: fdatasync d7/df/f13 0 2024-05-27T09:29:19.395 INFO:tasks.workunit.client.0.smithi119.stdout:1/396: truncate d7/dd/d25/f30 1086219 0 2024-05-27T09:29:19.396 INFO:tasks.workunit.client.0.smithi119.stdout:2/355: rename d2/da/f67 to d2/f7d 0 2024-05-27T09:29:19.396 INFO:tasks.workunit.client.0.smithi119.stdout:2/356: stat d2/d1a/d51/f68 0 2024-05-27T09:29:19.396 INFO:tasks.workunit.client.0.smithi119.stdout:2/357: chown d2/da/ff 154 1 2024-05-27T09:29:19.396 INFO:tasks.workunit.client.0.smithi119.stdout:2/358: creat d2/da/d15/d33/f7e x:0 0 0 2024-05-27T09:29:19.396 INFO:tasks.workunit.client.0.smithi119.stdout:2/359: stat d2/da/f13 0 2024-05-27T09:29:19.397 INFO:tasks.workunit.client.0.smithi119.stdout:1/397: mknod d7/db/d52/c7e 0 2024-05-27T09:29:19.397 INFO:tasks.workunit.client.0.smithi119.stdout:4/330: write d4/d23/d31/f53 [2594762,95164] 0 2024-05-27T09:29:19.397 INFO:tasks.workunit.client.0.smithi119.stdout:4/331: chown d4/d48/f51 2270 1 2024-05-27T09:29:19.398 INFO:tasks.workunit.client.0.smithi119.stdout:4/332: creat d4/d6/d28/f69 x:0 0 0 2024-05-27T09:29:19.399 INFO:tasks.workunit.client.0.smithi119.stdout:4/333: chown d4/d1c/d39 287282822 1 2024-05-27T09:29:19.399 INFO:tasks.workunit.client.0.smithi119.stdout:4/334: creat d4/de/d21/d40/f6a x:0 0 0 2024-05-27T09:29:19.399 INFO:tasks.workunit.client.0.smithi119.stdout:4/335: creat d4/d6/d28/f6b x:0 0 0 2024-05-27T09:29:19.399 INFO:tasks.workunit.client.0.smithi119.stdout:4/336: chown d4/d23/d31 0 1 2024-05-27T09:29:19.401 INFO:tasks.workunit.client.0.smithi119.stdout:4/337: symlink d4/de/l6c 0 2024-05-27T09:29:19.416 INFO:tasks.workunit.client.0.smithi119.stdout:8/351: write d3/de/f2c [2429809,123514] 0 2024-05-27T09:29:19.416 INFO:tasks.workunit.client.0.smithi119.stdout:8/352: truncate d3/de/f2c 5180884 0 2024-05-27T09:29:19.417 INFO:tasks.workunit.client.0.smithi119.stdout:5/355: dwrite d6/dd/f64 [4194304,4194304] 0 2024-05-27T09:29:19.426 INFO:tasks.workunit.client.0.smithi119.stdout:8/353: rename d3/d39/c53 to d3/d39/c76 0 2024-05-27T09:29:19.427 INFO:tasks.workunit.client.0.smithi119.stdout:1/398: write d7/dd/d25/d29/f55 [3846904,110778] 0 2024-05-27T09:29:19.427 INFO:tasks.workunit.client.0.smithi119.stdout:5/356: truncate d6/d2f/f51 4057259 0 2024-05-27T09:29:19.427 INFO:tasks.workunit.client.0.smithi119.stdout:5/357: chown d6/dd/d39/d19/d3c 15653244 1 2024-05-27T09:29:19.427 INFO:tasks.workunit.client.0.smithi119.stdout:5/358: creat d6/d2f/f78 x:0 0 0 2024-05-27T09:29:19.427 INFO:tasks.workunit.client.0.smithi119.stdout:2/360: dread d2/f2d [0,4194304] 0 2024-05-27T09:29:19.427 INFO:tasks.workunit.client.0.smithi119.stdout:2/361: chown l0 32962885 1 2024-05-27T09:29:19.427 INFO:tasks.workunit.client.0.smithi119.stdout:2/362: creat d2/da/d6f/f7f x:0 0 0 2024-05-27T09:29:19.427 INFO:tasks.workunit.client.0.smithi119.stdout:1/399: symlink d7/dd/d25/d29/d38/d3c/l7f 0 2024-05-27T09:29:19.430 INFO:tasks.workunit.client.0.smithi119.stdout:7/424: dread d7/d1a/f2d [0,4194304] 0 2024-05-27T09:29:19.430 INFO:tasks.workunit.client.0.smithi119.stdout:5/359: rename d6/dd/df/d41/l49 to d6/dd/d39/d74/l79 0 2024-05-27T09:29:19.430 INFO:tasks.workunit.client.0.smithi119.stdout:1/400: rmdir d7/dd/d25/d29/d38/d3c 39 2024-05-27T09:29:19.430 INFO:tasks.workunit.client.0.smithi119.stdout:1/401: chown d7/db/d12/d15/d1f 207862 1 2024-05-27T09:29:19.430 INFO:tasks.workunit.client.0.smithi119.stdout:1/402: fdatasync d7/db/d35/f69 0 2024-05-27T09:29:19.434 INFO:tasks.workunit.client.0.smithi119.stdout:6/313: dwrite d0/d11/f62 [0,4194304] 0 2024-05-27T09:29:19.434 INFO:tasks.workunit.client.0.smithi119.stdout:6/314: fsync d0/d2/d4e/f1f 0 2024-05-27T09:29:19.434 INFO:tasks.workunit.client.0.smithi119.stdout:2/363: rename d2/da/f10 to d2/da/d15/f80 0 2024-05-27T09:29:19.439 INFO:tasks.workunit.client.0.smithi119.stdout:5/360: creat d6/dd/d39/d74/f7a x:0 0 0 2024-05-27T09:29:19.439 INFO:tasks.workunit.client.0.smithi119.stdout:6/315: rmdir d0/d2/d1c/d64 39 2024-05-27T09:29:19.440 INFO:tasks.workunit.client.0.smithi119.stdout:9/279: dwrite d18/f1a [0,4194304] 0 2024-05-27T09:29:19.442 INFO:tasks.workunit.client.0.smithi119.stdout:9/280: read d18/d19/d1b/d29/f47 [4259727,14603] 0 2024-05-27T09:29:19.442 INFO:tasks.workunit.client.0.smithi119.stdout:7/425: rename d7/db/f5c to d7/d1a/d40/f86 0 2024-05-27T09:29:19.443 INFO:tasks.workunit.client.0.smithi119.stdout:7/426: truncate d7/d6a/f6f 83195 0 2024-05-27T09:29:19.443 INFO:tasks.workunit.client.0.smithi119.stdout:2/364: mkdir d2/d1a/d51/d81 0 2024-05-27T09:29:19.443 INFO:tasks.workunit.client.0.smithi119.stdout:2/365: write d2/da/d6f/f7f [743471,112209] 0 2024-05-27T09:29:19.443 INFO:tasks.workunit.client.0.smithi119.stdout:2/366: chown d2/da/d15/d33/f7e 66 1 2024-05-27T09:29:19.443 INFO:tasks.workunit.client.0.smithi119.stdout:2/367: chown d2/da/l20 1574277 1 2024-05-27T09:29:19.463 INFO:tasks.workunit.client.0.smithi119.stdout:9/281: dread d18/d19/d1b/f1d [0,4194304] 0 2024-05-27T09:29:19.463 INFO:tasks.workunit.client.0.smithi119.stdout:9/282: creat d18/d19/f59 x:0 0 0 2024-05-27T09:29:19.466 INFO:tasks.workunit.client.0.smithi119.stdout:6/316: write d0/fd [1928140,92155] 0 2024-05-27T09:29:19.466 INFO:tasks.workunit.client.0.smithi119.stdout:6/317: chown d0/d2/d1c/f2d 170 1 2024-05-27T09:29:19.475 INFO:tasks.workunit.client.0.smithi119.stdout:9/283: rename d18/d19/c48 to d18/d19/d1b/d40/d44/c5a 0 2024-05-27T09:29:19.476 INFO:tasks.workunit.client.0.smithi119.stdout:9/284: fdatasync d18/d19/f59 0 2024-05-27T09:29:19.476 INFO:tasks.workunit.client.0.smithi119.stdout:9/285: chown d18/d19/d1b/d29/l33 25003145 1 2024-05-27T09:29:19.476 INFO:tasks.workunit.client.0.smithi119.stdout:6/318: creat d0/d2/d1c/d64/f70 x:0 0 0 2024-05-27T09:29:19.478 INFO:tasks.workunit.client.0.smithi119.stdout:9/286: symlink d18/d19/d1b/d40/d44/l5b 0 2024-05-27T09:29:19.478 INFO:tasks.workunit.client.0.smithi119.stdout:9/287: write d18/d19/f35 [834513,11496] 0 2024-05-27T09:29:19.480 INFO:tasks.workunit.client.0.smithi119.stdout:9/288: symlink d18/l5c 0 2024-05-27T09:29:19.480 INFO:tasks.workunit.client.0.smithi119.stdout:9/289: truncate d18/d19/d1b/d29/f43 605724 0 2024-05-27T09:29:19.480 INFO:tasks.workunit.client.0.smithi119.stdout:9/290: fdatasync d18/f23 0 2024-05-27T09:29:19.482 INFO:tasks.workunit.client.0.smithi119.stdout:9/291: creat d18/d19/d1b/f5d x:0 0 0 2024-05-27T09:29:19.482 INFO:tasks.workunit.client.0.smithi119.stdout:0/334: sync 2024-05-27T09:29:19.482 INFO:tasks.workunit.client.0.smithi119.stdout:0/335: read d2/d6/d9/f2c [666090,120216] 0 2024-05-27T09:29:19.482 INFO:tasks.workunit.client.0.smithi119.stdout:0/336: chown d2/d6/d9/d2f 62 1 2024-05-27T09:29:19.483 INFO:tasks.workunit.client.0.smithi119.stdout:9/292: mknod d18/d19/c5e 0 2024-05-27T09:29:19.486 INFO:tasks.workunit.client.0.smithi119.stdout:0/337: unlink d2/d6/d9/l36 0 2024-05-27T09:29:19.486 INFO:tasks.workunit.client.0.smithi119.stdout:0/338: write d2/d6/d9/f2c [647462,48182] 0 2024-05-27T09:29:19.486 INFO:tasks.workunit.client.0.smithi119.stdout:0/339: dread - d2/d6/d9/d12/f47 zero size 2024-05-27T09:29:19.500 INFO:tasks.workunit.client.0.smithi119.stdout:8/354: dwrite d3/d8/d4d/f59 [0,4194304] 0 2024-05-27T09:29:19.500 INFO:tasks.workunit.client.0.smithi119.stdout:6/319: write d0/f68 [1887491,80355] 0 2024-05-27T09:29:19.500 INFO:tasks.workunit.client.0.smithi119.stdout:6/320: chown d0/d19 72 1 2024-05-27T09:29:19.500 INFO:tasks.workunit.client.0.smithi119.stdout:6/321: stat d0/d2/d57/f50 0 2024-05-27T09:29:19.500 INFO:tasks.workunit.client.0.smithi119.stdout:8/355: mkdir d3/d8/d1b/d77 0 2024-05-27T09:29:19.501 INFO:tasks.workunit.client.0.smithi119.stdout:6/322: truncate d0/d2/f31 3632315 0 2024-05-27T09:29:19.501 INFO:tasks.workunit.client.0.smithi119.stdout:6/323: chown d0/d2/d1c/d64/d65 249 1 2024-05-27T09:29:19.501 INFO:tasks.workunit.client.0.smithi119.stdout:8/356: mknod d3/de/c78 0 2024-05-27T09:29:19.501 INFO:tasks.workunit.client.0.smithi119.stdout:6/324: symlink d0/d2/d1c/l71 0 2024-05-27T09:29:19.501 INFO:tasks.workunit.client.0.smithi119.stdout:6/325: creat d0/d2/d1c/d21/f72 x:0 0 0 2024-05-27T09:29:19.512 INFO:tasks.workunit.client.0.smithi119.stdout:6/326: mknod d0/d2/d57/d37/c73 0 2024-05-27T09:29:19.522 INFO:tasks.workunit.client.0.smithi119.stdout:2/368: write d2/da/d15/d33/f60 [392463,80439] 0 2024-05-27T09:29:19.522 INFO:tasks.workunit.client.0.smithi119.stdout:6/327: rename d0/d2/d1c/d64 to d0/d2/d57/d37/d74 0 2024-05-27T09:29:19.523 INFO:tasks.workunit.client.0.smithi119.stdout:6/328: getdents d0/d2/d1c/d21/d58 0 2024-05-27T09:29:19.523 INFO:tasks.workunit.client.0.smithi119.stdout:6/329: dread - d0/d2/d57/f6d zero size 2024-05-27T09:29:19.562 INFO:tasks.workunit.client.0.smithi119.stdout:6/330: write d0/d2/d1c/d21/f45 [1749149,81180] 0 2024-05-27T09:29:19.563 INFO:tasks.workunit.client.0.smithi119.stdout:8/357: dread d3/d8/d4d/f36 [0,4194304] 0 2024-05-27T09:29:19.566 INFO:tasks.workunit.client.0.smithi119.stdout:5/361: dwrite d6/db/dc/f61 [0,4194304] 0 2024-05-27T09:29:19.569 INFO:tasks.workunit.client.0.smithi119.stdout:8/358: truncate d3/de/f2c 687524 0 2024-05-27T09:29:19.569 INFO:tasks.workunit.client.0.smithi119.stdout:8/359: truncate d3/d8/f47 848870 0 2024-05-27T09:29:19.571 INFO:tasks.workunit.client.0.smithi119.stdout:2/369: dread d2/f47 [0,4194304] 0 2024-05-27T09:29:19.573 INFO:tasks.workunit.client.0.smithi119.stdout:9/293: dwrite d18/d19/d1b/d40/d44/f45 [0,4194304] 0 2024-05-27T09:29:19.577 INFO:tasks.workunit.client.0.smithi119.stdout:5/362: truncate d6/f12 2372527 0 2024-05-27T09:29:19.577 INFO:tasks.workunit.client.0.smithi119.stdout:8/360: mkdir d3/d8/d4d/d79 0 2024-05-27T09:29:19.577 INFO:tasks.workunit.client.0.smithi119.stdout:5/363: write d6/dd/f37 [571483,131047] 0 2024-05-27T09:29:19.577 INFO:tasks.workunit.client.0.smithi119.stdout:2/370: symlink d2/da/d15/d53/l82 0 2024-05-27T09:29:19.577 INFO:tasks.workunit.client.0.smithi119.stdout:5/364: creat d6/db/dc/f7b x:0 0 0 2024-05-27T09:29:19.577 INFO:tasks.workunit.client.0.smithi119.stdout:5/365: write d6/dd/f64 [8694607,107031] 0 2024-05-27T09:29:19.580 INFO:tasks.workunit.client.0.smithi119.stdout:8/361: truncate d3/d8/d1b/d46/f37 3814208 0 2024-05-27T09:29:19.581 INFO:tasks.workunit.client.0.smithi119.stdout:6/331: write d0/d2/d1c/d21/d58/f61 [107114,77852] 0 2024-05-27T09:29:19.581 INFO:tasks.workunit.client.0.smithi119.stdout:6/332: write d0/f18 [1154915,105989] 0 2024-05-27T09:29:19.581 INFO:tasks.workunit.client.0.smithi119.stdout:6/333: fdatasync d0/d2/f31 0 2024-05-27T09:29:19.584 INFO:tasks.workunit.client.0.smithi119.stdout:0/340: dwrite d2/d6/d9/d12/f16 [4194304,4194304] 0 2024-05-27T09:29:19.584 INFO:tasks.workunit.client.0.smithi119.stdout:0/341: chown d2/d4/d53 18688 1 2024-05-27T09:29:19.588 INFO:tasks.workunit.client.0.smithi119.stdout:2/371: dread d2/da/d15/d33/f41 [0,4194304] 0 2024-05-27T09:29:19.589 INFO:tasks.workunit.client.0.smithi119.stdout:2/372: read - d2/da/f2e zero size 2024-05-27T09:29:19.589 INFO:tasks.workunit.client.0.smithi119.stdout:2/373: chown d2/d1a/d51/d81 8997526 1 2024-05-27T09:29:19.593 INFO:tasks.workunit.client.0.smithi119.stdout:8/362: getdents d3/d8 0 2024-05-27T09:29:19.596 INFO:tasks.workunit.client.0.smithi119.stdout:0/342: creat d2/d6/d9/d24/d30/d6b/f70 x:0 0 0 2024-05-27T09:29:19.596 INFO:tasks.workunit.client.0.smithi119.stdout:6/334: rmdir d0/d2/d1c/d21 39 2024-05-27T09:29:19.609 INFO:tasks.workunit.client.0.smithi119.stdout:1/403: dwrite d7/dd/d25/d29/f55 [0,4194304] 0 2024-05-27T09:29:19.609 INFO:tasks.workunit.client.0.smithi119.stdout:1/404: read d7/f8 [1692295,37977] 0 2024-05-27T09:29:19.609 INFO:tasks.workunit.client.0.smithi119.stdout:1/405: read f3 [2179673,87933] 0 2024-05-27T09:29:19.609 INFO:tasks.workunit.client.0.smithi119.stdout:0/343: getdents d2/d6/d9/d12/d4b 0 2024-05-27T09:29:19.609 INFO:tasks.workunit.client.0.smithi119.stdout:6/335: symlink d0/d2/d57/l75 0 2024-05-27T09:29:19.609 INFO:tasks.workunit.client.0.smithi119.stdout:1/406: symlink d7/dd/d25/d29/d38/d49/l80 0 2024-05-27T09:29:19.609 INFO:tasks.workunit.client.0.smithi119.stdout:0/344: mknod d2/d6/d9/d24/d30/c71 0 2024-05-27T09:29:19.610 INFO:tasks.workunit.client.0.smithi119.stdout:1/407: creat d7/db/d12/d26/d4a/f81 x:0 0 0 2024-05-27T09:29:19.610 INFO:tasks.workunit.client.0.smithi119.stdout:1/408: mkdir d7/dd/d25/d29/d38/d82 0 2024-05-27T09:29:19.610 INFO:tasks.workunit.client.0.smithi119.stdout:0/345: truncate d2/d6/d9/f6d 46181 0 2024-05-27T09:29:19.615 INFO:tasks.workunit.client.0.smithi119.stdout:1/409: rename d7/dd/l19 to d7/db/d12/d26/l83 0 2024-05-27T09:29:19.615 INFO:tasks.workunit.client.0.smithi119.stdout:1/410: fsync d7/db/d12/d15/d1f/f34 0 2024-05-27T09:29:19.616 INFO:tasks.workunit.client.0.smithi119.stdout:1/411: truncate d7/db/d12/d26/f43 975185 0 2024-05-27T09:29:19.622 INFO:tasks.workunit.client.0.smithi119.stdout:1/412: unlink d7/l1c 0 2024-05-27T09:29:19.622 INFO:tasks.workunit.client.0.smithi119.stdout:1/413: chown d7/dd/d25/d29/d38/d49 390712066 1 2024-05-27T09:29:19.622 INFO:tasks.workunit.client.0.smithi119.stdout:1/414: fdatasync d7/f27 0 2024-05-27T09:29:19.622 INFO:tasks.workunit.client.0.smithi119.stdout:1/415: rename d7/db/d12/c28 to d7/dd/d36/c84 0 2024-05-27T09:29:19.627 INFO:tasks.workunit.client.0.smithi119.stdout:1/416: symlink d7/dd/d25/d29/l85 0 2024-05-27T09:29:19.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:19 smithi138 ceph-mon[34885]: pgmap v7: 65 pgs: 65 active+clean; 851 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 89 MiB/s rd, 105 MiB/s wr, 532 op/s 2024-05-27T09:29:19.641 INFO:tasks.workunit.client.0.smithi119.stdout:3/488: sync 2024-05-27T09:29:19.641 INFO:tasks.workunit.client.0.smithi119.stdout:4/338: sync 2024-05-27T09:29:19.641 INFO:tasks.workunit.client.0.smithi119.stdout:9/294: dwrite d18/d19/d1b/f5d [0,4194304] 0 2024-05-27T09:29:19.645 INFO:tasks.workunit.client.0.smithi119.stdout:8/363: dread d3/d8/d1b/f6c [0,4194304] 0 2024-05-27T09:29:19.645 INFO:tasks.workunit.client.0.smithi119.stdout:8/364: dread - d3/d8/d4d/f67 zero size 2024-05-27T09:29:19.647 INFO:tasks.workunit.client.0.smithi119.stdout:4/339: mknod d4/de/d66/c6d 0 2024-05-27T09:29:19.649 INFO:tasks.workunit.client.0.smithi119.stdout:4/340: write d4/d48/f51 [321551,51208] 0 2024-05-27T09:29:19.649 INFO:tasks.workunit.client.0.smithi119.stdout:4/341: fdatasync d4/de/f2e 0 2024-05-27T09:29:19.650 INFO:tasks.workunit.client.0.smithi119.stdout:4/342: fsync d4/d6/f1a 0 2024-05-27T09:29:19.650 INFO:tasks.workunit.client.0.smithi119.stdout:4/343: truncate d4/d1c/f29 1624140 0 2024-05-27T09:29:19.664 INFO:tasks.workunit.client.0.smithi119.stdout:5/366: dwrite d6/db/de/f25 [4194304,4194304] 0 2024-05-27T09:29:19.668 INFO:tasks.workunit.client.0.smithi119.stdout:9/295: read d18/d19/d1b/d40/d44/f45 [2118632,56225] 0 2024-05-27T09:29:19.674 INFO:tasks.workunit.client.0.smithi119.stdout:9/296: link fe d18/d19/d1b/d2a/f5f 0 2024-05-27T09:29:19.681 INFO:tasks.workunit.client.0.smithi119.stdout:5/367: rmdir d6/db 39 2024-05-27T09:29:19.682 INFO:tasks.workunit.client.0.smithi119.stdout:5/368: rename d6/dd/d39 to d6/dd/d39/d19/d70/d7c 22 2024-05-27T09:29:19.682 INFO:tasks.workunit.client.0.smithi119.stdout:5/369: write d6/db/de/f25 [2425676,57877] 0 2024-05-27T09:29:19.682 INFO:tasks.workunit.client.0.smithi119.stdout:5/370: truncate d6/db/f6f 424500 0 2024-05-27T09:29:19.682 INFO:tasks.workunit.client.0.smithi119.stdout:5/371: write d6/dd/f67 [118436,51891] 0 2024-05-27T09:29:19.682 INFO:tasks.workunit.client.0.smithi119.stdout:5/372: fdatasync d6/db/de/d27/f3e 0 2024-05-27T09:29:19.682 INFO:tasks.workunit.client.0.smithi119.stdout:5/373: dread - d6/db/dc/f7b zero size 2024-05-27T09:29:19.682 INFO:tasks.workunit.client.0.smithi119.stdout:5/374: write d6/db/de/d27/f6b [1110644,114035] 0 2024-05-27T09:29:19.682 INFO:tasks.workunit.client.0.smithi119.stdout:3/489: read db/f56 [1096369,129228] 0 2024-05-27T09:29:19.683 INFO:tasks.workunit.client.0.smithi119.stdout:5/375: mkdir d6/dd/d39/d19/d7d 0 2024-05-27T09:29:19.683 INFO:tasks.workunit.client.0.smithi119.stdout:5/376: dread - d6/f63 zero size 2024-05-27T09:29:19.684 INFO:tasks.workunit.client.0.smithi119.stdout:3/490: truncate db/d36/d45/d4c/d62/d13/f97 1451261 0 2024-05-27T09:29:19.686 INFO:tasks.workunit.client.0.smithi119.stdout:3/491: symlink db/d7f/la2 0 2024-05-27T09:29:19.687 INFO:tasks.workunit.client.0.smithi119.stdout:3/492: symlink db/d36/d45/d4c/d62/d76/d98/d7a/la3 0 2024-05-27T09:29:19.690 INFO:tasks.workunit.client.0.smithi119.stdout:3/493: link db/d36/d45/d4c/d62/f17 db/d36/d45/d93/fa4 0 2024-05-27T09:29:19.690 INFO:tasks.workunit.client.0.smithi119.stdout:9/297: write d18/d19/d1b/d40/d44/f54 [3040,59695] 0 2024-05-27T09:29:19.690 INFO:tasks.workunit.client.0.smithi119.stdout:3/494: creat db/d36/d45/d4c/d62/d76/d98/fa5 x:0 0 0 2024-05-27T09:29:19.690 INFO:tasks.workunit.client.0.smithi119.stdout:3/495: fdatasync db/d7f/f51 0 2024-05-27T09:29:19.694 INFO:tasks.workunit.client.0.smithi119.stdout:3/496: mknod db/d36/d45/d4c/d62/d76/d98/d7a/ca6 0 2024-05-27T09:29:19.696 INFO:tasks.workunit.client.0.smithi119.stdout:3/497: truncate f6 546920 0 2024-05-27T09:29:19.701 INFO:tasks.workunit.client.0.smithi119.stdout:3/498: creat db/d36/d45/d4c/d62/d13/d33/fa7 x:0 0 0 2024-05-27T09:29:19.701 INFO:tasks.workunit.client.0.smithi119.stdout:3/499: chown db/d36/d45/d4c/d62/d76/d98/f80 114943380 1 2024-05-27T09:29:19.703 INFO:tasks.workunit.client.0.smithi119.stdout:3/500: mknod db/d36/d45/d4c/ca8 0 2024-05-27T09:29:19.704 INFO:tasks.workunit.client.0.smithi119.stdout:3/501: truncate db/d36/d45/d4c/d62/d76/d98/fa5 967012 0 2024-05-27T09:29:19.704 INFO:tasks.workunit.client.0.smithi119.stdout:3/502: write db/d36/d45/d4c/d62/d13/f31 [3015140,52695] 0 2024-05-27T09:29:19.707 INFO:tasks.workunit.client.0.smithi119.stdout:3/503: mkdir db/d7f/da9 0 2024-05-27T09:29:19.710 INFO:tasks.workunit.client.0.smithi119.stdout:4/344: dwrite d4/d1c/d39/f5d [0,4194304] 0 2024-05-27T09:29:19.715 INFO:tasks.workunit.client.0.smithi119.stdout:3/504: mkdir db/d36/d45/d4c/d62/d76/daa 0 2024-05-27T09:29:19.715 INFO:tasks.workunit.client.0.smithi119.stdout:3/505: stat db/d36/d45/d93 0 2024-05-27T09:29:19.715 INFO:tasks.workunit.client.0.smithi119.stdout:4/345: getdents d4/de/d21/d40/d47/d5c 0 2024-05-27T09:29:19.715 INFO:tasks.workunit.client.0.smithi119.stdout:1/417: read d7/dd/d25/f30 [879348,77625] 0 2024-05-27T09:29:19.716 INFO:tasks.workunit.client.0.smithi119.stdout:1/418: fsync d7/f3d 0 2024-05-27T09:29:19.716 INFO:tasks.workunit.client.0.smithi119.stdout:2/374: dwrite d2/d1a/d51/f68 [0,4194304] 0 2024-05-27T09:29:19.716 INFO:tasks.workunit.client.0.smithi119.stdout:2/375: chown d2/da/d15/d33/f60 65 1 2024-05-27T09:29:19.716 INFO:tasks.workunit.client.0.smithi119.stdout:2/376: creat d2/f83 x:0 0 0 2024-05-27T09:29:19.716 INFO:tasks.workunit.client.0.smithi119.stdout:2/377: fdatasync d2/da/d15/d33/f41 0 2024-05-27T09:29:19.728 INFO:tasks.workunit.client.0.smithi119.stdout:0/346: dwrite d2/d6/d9/d24/d30/d4a/f4c [0,4194304] 0 2024-05-27T09:29:19.737 INFO:tasks.workunit.client.0.smithi119.stdout:1/419: dread d7/dd/d44/d47/f72 [0,4194304] 0 2024-05-27T09:29:19.742 INFO:tasks.workunit.client.0.smithi119.stdout:7/427: sync 2024-05-27T09:29:19.743 INFO:tasks.workunit.client.0.smithi119.stdout:7/428: write d7/df/d36/f61 [319930,104810] 0 2024-05-27T09:29:19.743 INFO:tasks.workunit.client.0.smithi119.stdout:0/347: mknod d2/d6/d9/d2f/c72 0 2024-05-27T09:29:19.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:19 smithi119 ceph-mon[29082]: pgmap v7: 65 pgs: 65 active+clean; 851 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 89 MiB/s rd, 105 MiB/s wr, 532 op/s 2024-05-27T09:29:19.748 INFO:tasks.workunit.client.0.smithi119.stdout:7/429: creat d7/df/d19/d1d/f87 x:0 0 0 2024-05-27T09:29:19.749 INFO:tasks.workunit.client.0.smithi119.stdout:0/348: mknod d2/d6/d9/d12/d4f/c73 0 2024-05-27T09:29:19.749 INFO:tasks.workunit.client.0.smithi119.stdout:0/349: write d2/d6/d23/f33 [2044369,81766] 0 2024-05-27T09:29:19.749 INFO:tasks.workunit.client.0.smithi119.stdout:0/350: write d2/d1a/f51 [430300,112985] 0 2024-05-27T09:29:19.749 INFO:tasks.workunit.client.0.smithi119.stdout:0/351: creat d2/d1a/d20/f74 x:0 0 0 2024-05-27T09:29:19.749 INFO:tasks.workunit.client.0.smithi119.stdout:0/352: truncate d2/d6/d9/d12/f47 259211 0 2024-05-27T09:29:19.752 INFO:tasks.workunit.client.0.smithi119.stdout:1/420: rmdir d7/db/d12/d26/d4a 39 2024-05-27T09:29:19.755 INFO:tasks.workunit.client.0.smithi119.stdout:1/421: creat d7/dd/d25/f86 x:0 0 0 2024-05-27T09:29:19.756 INFO:tasks.workunit.client.0.smithi119.stdout:5/377: write d6/f22 [3979888,54852] 0 2024-05-27T09:29:19.756 INFO:tasks.workunit.client.0.smithi119.stdout:6/336: dwrite d0/d2/d57/f6d [0,4194304] 0 2024-05-27T09:29:19.758 INFO:tasks.workunit.client.0.smithi119.stdout:1/422: creat d7/dd/d25/d29/d38/d49/f87 x:0 0 0 2024-05-27T09:29:19.759 INFO:tasks.workunit.client.0.smithi119.stdout:0/353: rmdir d2/d6/d9/d12/d4f 39 2024-05-27T09:29:19.762 INFO:tasks.workunit.client.0.smithi119.stdout:5/378: creat d6/db/de/d27/f7e x:0 0 0 2024-05-27T09:29:19.763 INFO:tasks.workunit.client.0.smithi119.stdout:6/337: mkdir d0/d2/d4e/d5a/d76 0 2024-05-27T09:29:19.764 INFO:tasks.workunit.client.0.smithi119.stdout:5/379: creat d6/db/dc/f7f x:0 0 0 2024-05-27T09:29:19.764 INFO:tasks.workunit.client.0.smithi119.stdout:5/380: creat d6/f80 x:0 0 0 2024-05-27T09:29:19.764 INFO:tasks.workunit.client.0.smithi119.stdout:6/338: mkdir d0/d2/d77 0 2024-05-27T09:29:19.764 INFO:tasks.workunit.client.0.smithi119.stdout:6/339: dread - d0/f1e zero size 2024-05-27T09:29:19.767 INFO:tasks.workunit.client.0.smithi119.stdout:5/381: mknod d6/dd/df/d41/c81 0 2024-05-27T09:29:19.767 INFO:tasks.workunit.client.0.smithi119.stdout:5/382: chown d6/dd/df/d41/l57 36699 1 2024-05-27T09:29:19.768 INFO:tasks.workunit.client.0.smithi119.stdout:6/340: unlink d0/d2/d4e/d5a/c5e 0 2024-05-27T09:29:19.769 INFO:tasks.workunit.client.0.smithi119.stdout:5/383: mknod d6/dd/d39/d74/c82 0 2024-05-27T09:29:19.776 INFO:tasks.workunit.client.0.smithi119.stdout:4/346: write d4/d23/f35 [3099204,114730] 0 2024-05-27T09:29:19.776 INFO:tasks.workunit.client.0.smithi119.stdout:4/347: read - d4/d1c/d39/f64 zero size 2024-05-27T09:29:19.776 INFO:tasks.workunit.client.0.smithi119.stdout:4/348: stat d4/de/d21/f2b 0 2024-05-27T09:29:19.776 INFO:tasks.workunit.client.0.smithi119.stdout:4/349: read d4/d6/fa [4093513,4719] 0 2024-05-27T09:29:19.784 INFO:tasks.workunit.client.0.smithi119.stdout:3/506: dwrite db/d36/d45/d4c/d62/f2d [0,4194304] 0 2024-05-27T09:29:19.787 INFO:tasks.workunit.client.0.smithi119.stdout:3/507: mkdir db/d36/d45/dab 0 2024-05-27T09:29:19.787 INFO:tasks.workunit.client.0.smithi119.stdout:3/508: fdatasync db/d36/d45/d4d/f58 0 2024-05-27T09:29:19.787 INFO:tasks.workunit.client.0.smithi119.stdout:3/509: creat db/d36/d45/d4c/d62/d76/d98/d7a/fac x:0 0 0 2024-05-27T09:29:19.789 INFO:tasks.workunit.client.0.smithi119.stdout:0/354: dread d2/d6/d9/d12/f16 [4194304,4194304] 0 2024-05-27T09:29:19.789 INFO:tasks.workunit.client.0.smithi119.stdout:0/355: chown d2/d6/d9/d12/d4b 142 1 2024-05-27T09:29:19.793 INFO:tasks.workunit.client.0.smithi119.stdout:0/356: mkdir d2/d6/d75 0 2024-05-27T09:29:19.793 INFO:tasks.workunit.client.0.smithi119.stdout:0/357: getdents d2/d6/d9/d12/d4b/d6a 0 2024-05-27T09:29:19.798 INFO:tasks.workunit.client.0.smithi119.stdout:2/378: dwrite d2/d1a/f46 [4194304,4194304] 0 2024-05-27T09:29:19.809 INFO:tasks.workunit.client.0.smithi119.stdout:6/341: dwrite d0/d19/f55 [0,4194304] 0 2024-05-27T09:29:19.811 INFO:tasks.workunit.client.0.smithi119.stdout:0/358: dread d2/d6/f8 [4194304,4194304] 0 2024-05-27T09:29:19.813 INFO:tasks.workunit.client.0.smithi119.stdout:6/342: mknod d0/d2/d1c/c78 0 2024-05-27T09:29:19.817 INFO:tasks.workunit.client.0.smithi119.stdout:0/359: mknod d2/d6/d9/d2f/c76 0 2024-05-27T09:29:19.824 INFO:tasks.workunit.client.0.smithi119.stdout:0/360: unlink d2/d6/d9/d24/f5e 0 2024-05-27T09:29:19.827 INFO:tasks.workunit.client.0.smithi119.stdout:0/361: creat d2/d6/d9/d12/f77 x:0 0 0 2024-05-27T09:29:19.827 INFO:tasks.workunit.client.0.smithi119.stdout:0/362: read d2/d6/d9/d12/f16 [2572248,117384] 0 2024-05-27T09:29:19.832 INFO:tasks.workunit.client.0.smithi119.stdout:6/343: dread d0/f52 [0,4194304] 0 2024-05-27T09:29:19.833 INFO:tasks.workunit.client.0.smithi119.stdout:6/344: creat d0/d19/f79 x:0 0 0 2024-05-27T09:29:19.854 INFO:tasks.workunit.client.0.smithi119.stdout:2/379: dwrite d2/f66 [0,4194304] 0 2024-05-27T09:29:19.914 INFO:tasks.workunit.client.1.smithi138.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-05-27T09:29:19.915 INFO:tasks.workunit.client.1.smithi138.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-27T09:29:19.915 INFO:tasks.workunit.client.1.smithi138.stderr:+ make 2024-05-27T09:29:19.924 INFO:tasks.workunit.client.0.smithi119.stdout:0/363: dwrite d2/d6/f8 [8388608,4194304] 0 2024-05-27T09:29:19.925 INFO:tasks.workunit.client.0.smithi119.stdout:0/364: mknod d2/d6/d9/c78 0 2024-05-27T09:29:19.930 INFO:tasks.workunit.client.0.smithi119.stdout:2/380: dwrite d2/da/d4e/f6e [0,4194304] 0 2024-05-27T09:29:19.930 INFO:tasks.workunit.client.0.smithi119.stdout:2/381: read - d2/da/d15/d33/f7e zero size 2024-05-27T09:29:19.952 INFO:tasks.workunit.client.0.smithi119.stdout:2/382: dread d2/da/d15/d33/f57 [0,4194304] 0 2024-05-27T09:29:19.954 INFO:tasks.workunit.client.0.smithi119.stdout:2/383: link d2/da/d2c/l78 d2/d1a/l84 0 2024-05-27T09:29:19.955 INFO:tasks.workunit.client.0.smithi119.stdout:2/384: read d2/d1a/f3b [119611,8249] 0 2024-05-27T09:29:19.956 INFO:tasks.workunit.client.0.smithi119.stdout:0/365: dwrite d2/d4/f62 [0,4194304] 0 2024-05-27T09:29:19.960 INFO:tasks.workunit.client.0.smithi119.stdout:0/366: link d2/d6/d9/f27 d2/d6/d9/d12/f79 0 2024-05-27T09:29:19.964 INFO:tasks.workunit.client.0.smithi119.stdout:0/367: rename d2/d4 to d2/d6/d23/d7a 0 2024-05-27T09:29:19.966 INFO:tasks.workunit.client.0.smithi119.stdout:0/368: symlink d2/d6/d9/d24/d69/l7b 0 2024-05-27T09:29:19.966 INFO:tasks.workunit.client.0.smithi119.stdout:0/369: write d2/d1a/f51 [1367162,15074] 0 2024-05-27T09:29:19.982 INFO:tasks.workunit.client.0.smithi119.stdout:8/365: sync 2024-05-27T09:29:19.989 INFO:tasks.workunit.client.0.smithi119.stdout:0/370: dread d2/d1a/f51 [0,4194304] 0 2024-05-27T09:29:20.000 INFO:tasks.workunit.client.0.smithi119.stdout:0/371: creat d2/d1a/f7c x:0 0 0 2024-05-27T09:29:20.000 INFO:tasks.workunit.client.0.smithi119.stdout:0/372: truncate d2/d1a/d20/f74 503189 0 2024-05-27T09:29:20.000 INFO:tasks.workunit.client.0.smithi119.stdout:0/373: write d2/d6/d9/d12/f1b [498933,30877] 0 2024-05-27T09:29:20.007 INFO:tasks.workunit.client.0.smithi119.stdout:0/374: rename d2/d6/f26 to d2/d6/d9/d12/d4b/d59/f7d 0 2024-05-27T09:29:20.022 INFO:tasks.workunit.client.0.smithi119.stdout:2/385: dwrite d2/da/d15/d33/d69/f76 [0,4194304] 0 2024-05-27T09:29:20.022 INFO:tasks.workunit.client.0.smithi119.stdout:2/386: chown d2/da/l20 31968 1 2024-05-27T09:29:20.022 INFO:tasks.workunit.client.0.smithi119.stdout:2/387: write d2/da/d15/d33/f41 [728609,23160] 0 2024-05-27T09:29:20.022 INFO:tasks.workunit.client.0.smithi119.stdout:2/388: chown d2/da/f1c 2265 1 2024-05-27T09:29:20.024 INFO:tasks.workunit.client.0.smithi119.stdout:2/389: unlink d2/da/d2c/l64 0 2024-05-27T09:29:20.025 INFO:tasks.workunit.client.0.smithi119.stdout:2/390: creat d2/da/d15/d53/f85 x:0 0 0 2024-05-27T09:29:20.027 INFO:tasks.workunit.client.0.smithi119.stdout:2/391: truncate d2/da/fd 130014 0 2024-05-27T09:29:20.027 INFO:tasks.workunit.client.0.smithi119.stdout:2/392: write d2/da/d2c/f55 [93444,37350] 0 2024-05-27T09:29:20.028 INFO:tasks.workunit.client.0.smithi119.stdout:2/393: fsync d2/da/d15/f80 0 2024-05-27T09:29:20.028 INFO:tasks.workunit.client.0.smithi119.stdout:2/394: symlink d2/d1a/l86 0 2024-05-27T09:29:20.030 INFO:tasks.workunit.client.0.smithi119.stdout:2/395: mkdir d2/d1a/d51/d87 0 2024-05-27T09:29:20.030 INFO:tasks.workunit.client.0.smithi119.stdout:2/396: read - d2/f83 zero size 2024-05-27T09:29:20.030 INFO:tasks.workunit.client.0.smithi119.stdout:2/397: chown d2/da/f65 91870951 1 2024-05-27T09:29:20.030 INFO:tasks.workunit.client.0.smithi119.stdout:2/398: chown d2/da/ff 225 1 2024-05-27T09:29:20.034 INFO:tasks.workunit.client.0.smithi119.stdout:8/366: dwrite d3/d8/d1b/f55 [0,4194304] 0 2024-05-27T09:29:20.037 INFO:tasks.workunit.client.0.smithi119.stdout:8/367: rename d3/d8/d1b/d46/f26 to d3/d8/d1b/f7a 0 2024-05-27T09:29:20.040 INFO:tasks.workunit.client.0.smithi119.stdout:8/368: symlink d3/d8/d1b/d77/l7b 0 2024-05-27T09:29:20.070 INFO:tasks.workunit.client.0.smithi119.stdout:0/375: dwrite d2/d6/d9/d12/f1b [0,4194304] 0 2024-05-27T09:29:20.072 INFO:tasks.workunit.client.0.smithi119.stdout:0/376: getdents d2/d6/d9/d12/d4b/d59 0 2024-05-27T09:29:20.073 INFO:tasks.workunit.client.0.smithi119.stdout:0/377: symlink d2/d1a/l7e 0 2024-05-27T09:29:20.074 INFO:tasks.workunit.client.0.smithi119.stdout:0/378: mknod d2/d6/d9/c7f 0 2024-05-27T09:29:20.076 INFO:tasks.workunit.client.0.smithi119.stdout:0/379: rename d2/d6/d9/l52 to d2/d6/d9/d12/d4f/l80 0 2024-05-27T09:29:20.103 INFO:tasks.workunit.client.0.smithi119.stdout:0/380: dread d2/d6/d9/d12/d4f/f5a [0,4194304] 0 2024-05-27T09:29:20.103 INFO:tasks.workunit.client.0.smithi119.stdout:0/381: chown d2/d6/d23/c17 785997054 1 2024-05-27T09:29:20.103 INFO:tasks.workunit.client.0.smithi119.stdout:0/382: chown d2/d6/d23/d7a/ca 1073163635 1 2024-05-27T09:29:20.124 INFO:tasks.workunit.client.0.smithi119.stdout:1/423: sync 2024-05-27T09:29:20.124 INFO:tasks.workunit.client.0.smithi119.stdout:7/430: sync 2024-05-27T09:29:20.125 INFO:tasks.workunit.client.0.smithi119.stdout:8/369: dwrite d3/d8/d4d/f59 [0,4194304] 0 2024-05-27T09:29:20.126 INFO:tasks.workunit.client.0.smithi119.stdout:1/424: unlink c1 0 2024-05-27T09:29:20.126 INFO:tasks.workunit.client.0.smithi119.stdout:0/383: write d2/d6/d9/d12/f79 [1610295,49904] 0 2024-05-27T09:29:20.126 INFO:tasks.workunit.client.0.smithi119.stdout:0/384: getdents d2/d6/d9/d12/d4b/d59 0 2024-05-27T09:29:20.126 INFO:tasks.workunit.client.0.smithi119.stdout:0/385: read d2/d6/d9/d12/f16 [1214946,51404] 0 2024-05-27T09:29:20.126 INFO:tasks.workunit.client.0.smithi119.stdout:0/386: readlink d2/d6/d9/d24/d30/d4a/l61 0 2024-05-27T09:29:20.126 INFO:tasks.workunit.client.0.smithi119.stdout:7/431: mknod d7/df/d19/d78/c88 0 2024-05-27T09:29:20.129 INFO:tasks.workunit.client.0.smithi119.stdout:8/370: creat d3/d8/d1b/d46/f7c x:0 0 0 2024-05-27T09:29:20.129 INFO:tasks.workunit.client.0.smithi119.stdout:1/425: stat d7/dd/d25/d60/d61/c66 0 2024-05-27T09:29:20.129 INFO:tasks.workunit.client.0.smithi119.stdout:1/426: creat d7/dd/d44/f88 x:0 0 0 2024-05-27T09:29:20.129 INFO:tasks.workunit.client.0.smithi119.stdout:0/387: symlink d2/d6/d9/d2f/l81 0 2024-05-27T09:29:20.131 INFO:tasks.workunit.client.0.smithi119.stdout:1/427: link d7/db/c10 d7/dd/d25/d60/d61/c89 0 2024-05-27T09:29:20.142 INFO:tasks.workunit.client.0.smithi119.stdout:9/298: sync 2024-05-27T09:29:20.142 INFO:tasks.workunit.client.0.smithi119.stdout:0/388: symlink d2/d6/d75/l82 0 2024-05-27T09:29:20.143 INFO:tasks.workunit.client.0.smithi119.stdout:1/428: symlink d7/db/d12/d15/d1f/l8a 0 2024-05-27T09:29:20.145 INFO:tasks.workunit.client.0.smithi119.stdout:8/371: truncate d3/d8/d1b/f41 1522431 0 2024-05-27T09:29:20.145 INFO:tasks.workunit.client.0.smithi119.stdout:8/372: readlink d3/de/l42 0 2024-05-27T09:29:20.147 INFO:tasks.workunit.client.0.smithi119.stdout:1/429: chown d7/db/d52 0 1 2024-05-27T09:29:20.150 INFO:tasks.workunit.client.0.smithi119.stdout:8/373: unlink d3/f12 0 2024-05-27T09:29:20.156 INFO:tasks.workunit.client.0.smithi119.stdout:1/430: unlink d7/dd/f13 0 2024-05-27T09:29:20.158 INFO:tasks.workunit.client.0.smithi119.stdout:1/431: creat d7/db/d12/d15/d1f/f8b x:0 0 0 2024-05-27T09:29:20.164 INFO:tasks.workunit.client.0.smithi119.stdout:1/432: write d7/f5b [1003998,37142] 0 2024-05-27T09:29:20.175 INFO:tasks.workunit.client.0.smithi119.stdout:1/433: getdents d7/dd 0 2024-05-27T09:29:20.178 INFO:tasks.workunit.client.0.smithi119.stdout:8/374: write d3/d8/d1b/f41 [37645,44858] 0 2024-05-27T09:29:20.178 INFO:tasks.workunit.client.0.smithi119.stdout:8/375: write d3/de/f64 [169136,83365] 0 2024-05-27T09:29:20.184 INFO:tasks.workunit.client.0.smithi119.stdout:7/432: dwrite d7/df/d19/d46/f85 [0,4194304] 0 2024-05-27T09:29:20.185 INFO:tasks.workunit.client.0.smithi119.stdout:7/433: mkdir d7/df/d19/d4d/d89 0 2024-05-27T09:29:20.185 INFO:tasks.workunit.client.0.smithi119.stdout:7/434: chown d7/df/d19/l4b 11147455 1 2024-05-27T09:29:20.187 INFO:tasks.workunit.client.0.smithi119.stdout:8/376: write d3/d8/d1b/f28 [447092,19937] 0 2024-05-27T09:29:20.187 INFO:tasks.workunit.client.0.smithi119.stdout:8/377: chown d3/d8/d1b/d77/l7b 129535980 1 2024-05-27T09:29:20.187 INFO:tasks.workunit.client.0.smithi119.stdout:7/435: unlink d7/df/d19/d1d/f4f 0 2024-05-27T09:29:20.190 INFO:tasks.workunit.client.0.smithi119.stdout:8/378: mknod d3/d8/d4d/d79/c7d 0 2024-05-27T09:29:20.190 INFO:tasks.workunit.client.0.smithi119.stdout:7/436: getdents d7/d1a/d40 0 2024-05-27T09:29:20.192 INFO:tasks.workunit.client.0.smithi119.stdout:8/379: creat d3/d8/f7e x:0 0 0 2024-05-27T09:29:20.192 INFO:tasks.workunit.client.0.smithi119.stdout:7/437: creat d7/d5f/d79/f8a x:0 0 0 2024-05-27T09:29:20.195 INFO:tasks.workunit.client.0.smithi119.stdout:7/438: write d7/df/d36/f52 [1017257,124488] 0 2024-05-27T09:29:20.199 INFO:tasks.workunit.client.0.smithi119.stdout:8/380: truncate d3/d8/d4d/f32 8132748 0 2024-05-27T09:29:20.199 INFO:tasks.workunit.client.0.smithi119.stdout:8/381: getdents d3/d39/d5a 0 2024-05-27T09:29:20.199 INFO:tasks.workunit.client.0.smithi119.stdout:8/382: write d3/f1a [1355336,63761] 0 2024-05-27T09:29:20.201 INFO:tasks.workunit.client.0.smithi119.stdout:8/383: mknod d3/d8/d4d/d5d/d66/c7f 0 2024-05-27T09:29:20.203 INFO:tasks.workunit.client.0.smithi119.stdout:8/384: mkdir d3/d6e/d80 0 2024-05-27T09:29:20.206 INFO:tasks.workunit.client.0.smithi119.stdout:9/299: dwrite d18/f41 [0,4194304] 0 2024-05-27T09:29:20.209 INFO:tasks.workunit.client.0.smithi119.stdout:1/434: dread d7/f14 [0,4194304] 0 2024-05-27T09:29:20.211 INFO:tasks.workunit.client.0.smithi119.stdout:7/439: dread d7/db/d37/f5e [0,4194304] 0 2024-05-27T09:29:20.216 INFO:tasks.workunit.client.0.smithi119.stdout:1/435: truncate d7/dd/d44/d47/f72 515128 0 2024-05-27T09:29:20.219 INFO:tasks.workunit.client.0.smithi119.stdout:1/436: truncate d7/f11 702880 0 2024-05-27T09:29:20.231 INFO:tasks.workunit.client.1.smithi138.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-05-27T09:29:20.252 INFO:tasks.workunit.client.0.smithi119.stdout:1/437: write d7/dd/d25/d29/f55 [1422851,64404] 0 2024-05-27T09:29:20.252 INFO:tasks.workunit.client.0.smithi119.stdout:1/438: chown d7/dd/d25/d29/f55 148395250 1 2024-05-27T09:29:20.252 INFO:tasks.workunit.client.0.smithi119.stdout:1/439: fsync d7/db/d12/f6b 0 2024-05-27T09:29:20.257 INFO:tasks.workunit.client.0.smithi119.stdout:1/440: unlink d7/dd/d25/d29/d38/d49/l80 0 2024-05-27T09:29:20.263 INFO:tasks.workunit.client.0.smithi119.stdout:1/441: creat d7/db/d35/f8c x:0 0 0 2024-05-27T09:29:20.263 INFO:tasks.workunit.client.0.smithi119.stdout:1/442: chown d7/dd/d25/d60/l78 110 1 2024-05-27T09:29:20.263 INFO:tasks.workunit.client.0.smithi119.stdout:1/443: fdatasync d7/db/d12/d26/d4a/f81 0 2024-05-27T09:29:20.263 INFO:tasks.workunit.client.0.smithi119.stdout:8/385: dwrite d3/f25 [0,4194304] 0 2024-05-27T09:29:20.282 INFO:tasks.workunit.client.0.smithi119.stdout:0/389: dwrite d2/d6/d9/f1c [0,4194304] 0 2024-05-27T09:29:20.286 INFO:tasks.workunit.client.0.smithi119.stdout:0/390: creat d2/d6/d9/d12/d4b/d59/f83 x:0 0 0 2024-05-27T09:29:20.286 INFO:tasks.workunit.client.0.smithi119.stdout:0/391: dread - d2/d6/d9/d24/d30/d6b/f70 zero size 2024-05-27T09:29:20.287 INFO:tasks.workunit.client.0.smithi119.stdout:7/440: dwrite d7/d1a/d40/d44/f80 [0,4194304] 0 2024-05-27T09:29:20.287 INFO:tasks.workunit.client.0.smithi119.stdout:7/441: chown d7/d1a/f2d 1253992 1 2024-05-27T09:29:20.298 INFO:tasks.workunit.client.0.smithi119.stdout:1/444: write d7/db/d12/d15/d1f/f34 [609430,10405] 0 2024-05-27T09:29:20.299 INFO:tasks.workunit.client.0.smithi119.stdout:6/345: sync 2024-05-27T09:29:20.299 INFO:tasks.workunit.client.0.smithi119.stdout:5/384: sync 2024-05-27T09:29:20.299 INFO:tasks.workunit.client.0.smithi119.stdout:3/510: sync 2024-05-27T09:29:20.299 INFO:tasks.workunit.client.0.smithi119.stdout:3/511: fsync db/d36/d45/d4c/d62/d13/f2e 0 2024-05-27T09:29:20.299 INFO:tasks.workunit.client.0.smithi119.stdout:2/399: sync 2024-05-27T09:29:20.299 INFO:tasks.workunit.client.0.smithi119.stdout:7/442: dread d7/df/d36/d42/f73 [0,4194304] 0 2024-05-27T09:29:20.299 INFO:tasks.workunit.client.0.smithi119.stdout:4/350: sync 2024-05-27T09:29:20.299 INFO:tasks.workunit.client.0.smithi119.stdout:4/351: dread - d4/f41 zero size 2024-05-27T09:29:20.299 INFO:tasks.workunit.client.0.smithi119.stdout:7/443: truncate d7/df/d36/d42/d64/f70 548496 0 2024-05-27T09:29:20.299 INFO:tasks.workunit.client.0.smithi119.stdout:7/444: chown d7/d1a/f2d 30696 1 2024-05-27T09:29:20.299 INFO:tasks.workunit.client.0.smithi119.stdout:1/445: link d7/db/d12/d15/d1f/c7b d7/dd/d44/d47/c8d 0 2024-05-27T09:29:20.300 INFO:tasks.workunit.client.0.smithi119.stdout:6/346: symlink d0/d2/d77/l7a 0 2024-05-27T09:29:20.300 INFO:tasks.workunit.client.0.smithi119.stdout:5/385: truncate d6/db/f6f 331849 0 2024-05-27T09:29:20.301 INFO:tasks.workunit.client.0.smithi119.stdout:3/512: mknod db/d36/d45/d4d/cad 0 2024-05-27T09:29:20.301 INFO:tasks.workunit.client.0.smithi119.stdout:2/400: mknod d2/da/d6f/c88 0 2024-05-27T09:29:20.302 INFO:tasks.workunit.client.0.smithi119.stdout:4/352: creat d4/d6/d3e/f6e x:0 0 0 2024-05-27T09:29:20.302 INFO:tasks.workunit.client.0.smithi119.stdout:4/353: chown d4/de 11 1 2024-05-27T09:29:20.303 INFO:tasks.workunit.client.0.smithi119.stdout:7/445: creat d7/d1a/d40/f8b x:0 0 0 2024-05-27T09:29:20.303 INFO:tasks.workunit.client.0.smithi119.stdout:7/446: read f1 [5920766,82814] 0 2024-05-27T09:29:20.303 INFO:tasks.workunit.client.0.smithi119.stdout:7/447: creat d7/d5f/f8c x:0 0 0 2024-05-27T09:29:20.303 INFO:tasks.workunit.client.0.smithi119.stdout:7/448: dread - d7/d1a/f59 zero size 2024-05-27T09:29:20.303 INFO:tasks.workunit.client.0.smithi119.stdout:7/449: read - d7/d5f/d79/f8a zero size 2024-05-27T09:29:20.304 INFO:tasks.workunit.client.0.smithi119.stdout:1/446: creat d7/db/d12/d15/f8e x:0 0 0 2024-05-27T09:29:20.304 INFO:tasks.workunit.client.0.smithi119.stdout:6/347: mknod d0/d2/d57/d37/d74/d65/c7b 0 2024-05-27T09:29:20.305 INFO:tasks.workunit.client.0.smithi119.stdout:5/386: symlink d6/dd/d46/d47/l83 0 2024-05-27T09:29:20.305 INFO:tasks.workunit.client.0.smithi119.stdout:3/513: mknod db/d36/d45/d4d/cae 0 2024-05-27T09:29:20.308 INFO:tasks.workunit.client.0.smithi119.stdout:9/300: dwrite d18/f21 [4194304,4194304] 0 2024-05-27T09:29:20.308 INFO:tasks.workunit.client.0.smithi119.stdout:2/401: unlink d2/d1a/f36 0 2024-05-27T09:29:20.308 INFO:tasks.workunit.client.0.smithi119.stdout:2/402: dread - d2/da/d15/d53/f85 zero size 2024-05-27T09:29:20.310 INFO:tasks.workunit.client.0.smithi119.stdout:4/354: link d4/d6/l3c d4/l6f 0 2024-05-27T09:29:20.317 INFO:tasks.workunit.client.0.smithi119.stdout:7/450: mkdir d7/df/d19/d8d 0 2024-05-27T09:29:20.317 INFO:tasks.workunit.client.0.smithi119.stdout:7/451: stat d7/df/d36/l3c 0 2024-05-27T09:29:20.318 INFO:tasks.workunit.client.0.smithi119.stdout:1/447: getdents d7/dd/d25/d29 0 2024-05-27T09:29:20.318 INFO:tasks.workunit.client.0.smithi119.stdout:6/348: mknod d0/d2/d1c/d21/d58/c7c 0 2024-05-27T09:29:20.318 INFO:tasks.workunit.client.0.smithi119.stdout:5/387: rename d6/dd/d39/d19/d3c/d62 to d6/dd/d39/d19/d84 0 2024-05-27T09:29:20.318 INFO:tasks.workunit.client.0.smithi119.stdout:3/514: unlink db/d36/d45/d4c/d62/f8e 0 2024-05-27T09:29:20.319 INFO:tasks.workunit.client.0.smithi119.stdout:3/515: chown db/d36/d45/d4c/d62/d13 210522311 1 2024-05-27T09:29:20.319 INFO:tasks.workunit.client.0.smithi119.stdout:2/403: write d2/f2d [1985588,123089] 0 2024-05-27T09:29:20.319 INFO:tasks.workunit.client.0.smithi119.stdout:9/301: unlink d18/d19/d1b/f38 0 2024-05-27T09:29:20.319 INFO:tasks.workunit.client.0.smithi119.stdout:9/302: stat d18/d19/f50 0 2024-05-27T09:29:20.320 INFO:tasks.workunit.client.0.smithi119.stdout:7/452: write d7/d1a/d40/f86 [817293,74993] 0 2024-05-27T09:29:20.321 INFO:tasks.workunit.client.0.smithi119.stdout:3/516: write db/d36/d45/d4d/f70 [553808,122961] 0 2024-05-27T09:29:20.321 INFO:tasks.workunit.client.0.smithi119.stdout:3/517: creat db/d36/d45/d4c/d62/d76/d98/faf x:0 0 0 2024-05-27T09:29:20.321 INFO:tasks.workunit.client.0.smithi119.stdout:6/349: creat d0/d11/f7d x:0 0 0 2024-05-27T09:29:20.322 INFO:tasks.workunit.client.0.smithi119.stdout:2/404: mkdir d2/da/d15/d33/d89 0 2024-05-27T09:29:20.322 INFO:tasks.workunit.client.0.smithi119.stdout:2/405: truncate d2/da/f4d 1004409 0 2024-05-27T09:29:20.323 INFO:tasks.workunit.client.0.smithi119.stdout:9/303: creat d18/d19/d1b/d40/f60 x:0 0 0 2024-05-27T09:29:20.323 INFO:tasks.workunit.client.0.smithi119.stdout:4/355: rmdir d4/de/d21/d40 39 2024-05-27T09:29:20.326 INFO:tasks.workunit.client.0.smithi119.stdout:7/453: rename d7/df/d19/l4b to d7/d1a/d40/l8e 0 2024-05-27T09:29:20.327 INFO:tasks.workunit.client.0.smithi119.stdout:7/454: write d7/d1a/f2d [4602484,63115] 0 2024-05-27T09:29:20.327 INFO:tasks.workunit.client.0.smithi119.stdout:7/455: dread - d7/df/d19/d1d/f87 zero size 2024-05-27T09:29:20.331 INFO:tasks.workunit.client.0.smithi119.stdout:1/448: unlink d7/f8 0 2024-05-27T09:29:20.333 INFO:tasks.workunit.client.0.smithi119.stdout:6/350: symlink d0/d2/d57/d37/l7e 0 2024-05-27T09:29:20.333 INFO:tasks.workunit.client.0.smithi119.stdout:6/351: creat d0/d2/f7f x:0 0 0 2024-05-27T09:29:20.333 INFO:tasks.workunit.client.0.smithi119.stdout:6/352: chown d0/d2/d1c/d21/l49 3645564 1 2024-05-27T09:29:20.333 INFO:tasks.workunit.client.0.smithi119.stdout:6/353: dread - d0/d11/f7d zero size 2024-05-27T09:29:20.333 INFO:tasks.workunit.client.0.smithi119.stdout:6/354: creat d0/d2/f80 x:0 0 0 2024-05-27T09:29:20.335 INFO:tasks.workunit.client.0.smithi119.stdout:2/406: mkdir d2/da/d8a 0 2024-05-27T09:29:20.338 INFO:tasks.workunit.client.0.smithi119.stdout:2/407: creat d2/d1a/d51/f8b x:0 0 0 2024-05-27T09:29:20.338 INFO:tasks.workunit.client.0.smithi119.stdout:2/408: fsync d2/da/d15/f5f 0 2024-05-27T09:29:20.338 INFO:tasks.workunit.client.0.smithi119.stdout:3/518: rename db/d36/d45/d4c/d62/f88 to db/d36/d45/d93/fb0 0 2024-05-27T09:29:20.341 INFO:tasks.workunit.client.0.smithi119.stdout:1/449: mkdir d7/db/d8f 0 2024-05-27T09:29:20.341 INFO:tasks.workunit.client.0.smithi119.stdout:1/450: dread - d7/dd/d44/f88 zero size 2024-05-27T09:29:20.348 INFO:tasks.workunit.client.0.smithi119.stdout:3/519: truncate db/d36/d45/d4c/d62/d76/d98/fa5 953618 0 2024-05-27T09:29:20.349 INFO:tasks.workunit.client.0.smithi119.stdout:3/520: fsync db/d36/d45/d49/d4f/f7b 0 2024-05-27T09:29:20.350 INFO:tasks.workunit.client.0.smithi119.stdout:1/451: creat d7/dd/d25/d29/d38/f90 x:0 0 0 2024-05-27T09:29:20.356 INFO:tasks.workunit.client.0.smithi119.stdout:3/521: creat db/fb1 x:0 0 0 2024-05-27T09:29:20.356 INFO:tasks.workunit.client.0.smithi119.stdout:3/522: stat db/d7f/f74 0 2024-05-27T09:29:20.367 INFO:tasks.workunit.client.0.smithi119.stdout:1/452: rmdir d7/dd/d44 39 2024-05-27T09:29:20.371 INFO:tasks.workunit.client.0.smithi119.stdout:1/453: mknod d7/dd/d25/d29/d38/d49/c91 0 2024-05-27T09:29:20.371 INFO:tasks.workunit.client.0.smithi119.stdout:1/454: write d7/db/d12/d15/d1f/f34 [1547496,116999] 0 2024-05-27T09:29:20.374 INFO:tasks.workunit.client.0.smithi119.stdout:1/455: creat d7/dd/d25/d29/d38/f92 x:0 0 0 2024-05-27T09:29:20.374 INFO:tasks.workunit.client.0.smithi119.stdout:1/456: write f2 [2205414,58500] 0 2024-05-27T09:29:20.378 INFO:tasks.workunit.client.0.smithi119.stdout:1/457: unlink d7/db/d12/d26/d4a/c4d 0 2024-05-27T09:29:20.379 INFO:tasks.workunit.client.0.smithi119.stdout:6/355: rmdir d0/d2 39 2024-05-27T09:29:20.386 INFO:tasks.workunit.client.0.smithi119.stdout:1/458: creat d7/dd/d44/f93 x:0 0 0 2024-05-27T09:29:20.386 INFO:tasks.workunit.client.0.smithi119.stdout:6/356: mknod d0/d2/d57/d37/c81 0 2024-05-27T09:29:20.386 INFO:tasks.workunit.client.0.smithi119.stdout:3/523: write db/d36/d45/d4c/d62/f7c [2529265,131033] 0 2024-05-27T09:29:20.387 INFO:tasks.workunit.client.0.smithi119.stdout:3/524: chown db/d36/d45/d4c/d62/f2d 6204222 1 2024-05-27T09:29:20.388 INFO:tasks.workunit.client.0.smithi119.stdout:3/525: mknod db/d36/d45/d4c/d62/d76/daa/cb2 0 2024-05-27T09:29:20.406 INFO:tasks.workunit.client.0.smithi119.stdout:7/456: dwrite d7/d1a/d40/d44/f5d [0,4194304] 0 2024-05-27T09:29:20.406 INFO:tasks.workunit.client.0.smithi119.stdout:4/356: dwrite d4/d6/fd [0,4194304] 0 2024-05-27T09:29:20.408 INFO:tasks.workunit.client.0.smithi119.stdout:5/388: dwrite d6/db/f36 [4194304,4194304] 0 2024-05-27T09:29:20.412 INFO:tasks.workunit.client.0.smithi119.stdout:6/357: dread d0/d2/d57/d37/f5b [0,4194304] 0 2024-05-27T09:29:20.413 INFO:tasks.workunit.client.0.smithi119.stdout:4/357: truncate d4/de/d21/d40/f42 813022 0 2024-05-27T09:29:20.415 INFO:tasks.workunit.client.0.smithi119.stdout:5/389: symlink d6/db/l85 0 2024-05-27T09:29:20.415 INFO:tasks.workunit.client.0.smithi119.stdout:9/304: dwrite d18/f32 [0,4194304] 0 2024-05-27T09:29:20.416 INFO:tasks.workunit.client.0.smithi119.stdout:6/358: mknod d0/d2/d77/c82 0 2024-05-27T09:29:20.417 INFO:tasks.workunit.client.0.smithi119.stdout:4/358: symlink d4/d23/l70 0 2024-05-27T09:29:20.417 INFO:tasks.workunit.client.0.smithi119.stdout:4/359: truncate d4/d1c/d39/f64 677907 0 2024-05-27T09:29:20.422 INFO:tasks.workunit.client.0.smithi119.stdout:5/390: symlink d6/dd/d46/d47/d4b/l86 0 2024-05-27T09:29:20.422 INFO:tasks.workunit.client.0.smithi119.stdout:5/391: chown d6/dd/d46/d47 4352 1 2024-05-27T09:29:20.422 INFO:tasks.workunit.client.0.smithi119.stdout:5/392: dread - d6/db/de/d27/f72 zero size 2024-05-27T09:29:20.430 INFO:tasks.workunit.client.0.smithi119.stdout:1/459: write d7/f14 [1592311,53581] 0 2024-05-27T09:29:20.430 INFO:tasks.workunit.client.0.smithi119.stdout:6/359: unlink d0/d2/c60 0 2024-05-27T09:29:20.430 INFO:tasks.workunit.client.0.smithi119.stdout:5/393: truncate d6/dd/f1b 761173 0 2024-05-27T09:29:20.431 INFO:tasks.workunit.client.0.smithi119.stdout:5/394: dread - d6/db/dc/f7f zero size 2024-05-27T09:29:20.431 INFO:tasks.workunit.client.0.smithi119.stdout:5/395: fdatasync d6/db/de/d27/f3f 0 2024-05-27T09:29:20.433 INFO:tasks.workunit.client.0.smithi119.stdout:9/305: mkdir d18/d19/d61 0 2024-05-27T09:29:20.445 INFO:tasks.workunit.client.0.smithi119.stdout:1/460: mkdir d7/dd/d44/d94 0 2024-05-27T09:29:20.445 INFO:tasks.workunit.client.0.smithi119.stdout:1/461: chown d7/db/d12/d15/d1f/l8a 1 1 2024-05-27T09:29:20.447 INFO:tasks.workunit.client.0.smithi119.stdout:5/396: creat d6/dd/d39/d19/d3c/f87 x:0 0 0 2024-05-27T09:29:20.447 INFO:tasks.workunit.client.0.smithi119.stdout:9/306: creat d18/d19/d1b/d40/f62 x:0 0 0 2024-05-27T09:29:20.453 INFO:tasks.workunit.client.0.smithi119.stdout:3/526: dwrite db/d36/d45/d93/fb0 [0,4194304] 0 2024-05-27T09:29:20.453 INFO:tasks.workunit.client.0.smithi119.stdout:3/527: write db/d36/d45/d4c/d62/d76/d98/d7a/f9e [520753,95895] 0 2024-05-27T09:29:20.455 INFO:tasks.workunit.client.0.smithi119.stdout:1/462: mkdir d7/dd/d44/d47/d95 0 2024-05-27T09:29:20.456 INFO:tasks.workunit.client.0.smithi119.stdout:1/463: write d7/dd/f77 [766740,122112] 0 2024-05-27T09:29:20.456 INFO:tasks.workunit.client.0.smithi119.stdout:1/464: chown d7/db/d52/d65/c7a 974 1 2024-05-27T09:29:20.459 INFO:tasks.workunit.client.0.smithi119.stdout:9/307: mkdir d18/d19/d1b/d2a/d63 0 2024-05-27T09:29:20.459 INFO:tasks.workunit.client.0.smithi119.stdout:5/397: mknod d6/c88 0 2024-05-27T09:29:20.459 INFO:tasks.workunit.client.0.smithi119.stdout:9/308: chown l14 63916984 1 2024-05-27T09:29:20.459 INFO:tasks.workunit.client.0.smithi119.stdout:5/398: dread - d6/db/dc/f7f zero size 2024-05-27T09:29:20.460 INFO:tasks.workunit.client.0.smithi119.stdout:6/360: read d0/d2/f31 [911510,59565] 0 2024-05-27T09:29:20.460 INFO:tasks.workunit.client.0.smithi119.stdout:9/309: creat d18/d19/d1b/f64 x:0 0 0 2024-05-27T09:29:20.468 INFO:tasks.workunit.client.0.smithi119.stdout:5/399: creat d6/db/de/f89 x:0 0 0 2024-05-27T09:29:20.469 INFO:tasks.workunit.client.0.smithi119.stdout:7/457: dwrite d7/db/d37/f5e [0,4194304] 0 2024-05-27T09:29:20.469 INFO:tasks.workunit.client.0.smithi119.stdout:7/458: creat d7/d6a/f8f x:0 0 0 2024-05-27T09:29:20.470 INFO:tasks.workunit.client.0.smithi119.stdout:7/459: stat d7/df/d36/d42/d76 0 2024-05-27T09:29:20.472 INFO:tasks.workunit.client.0.smithi119.stdout:6/361: creat d0/d2/d1c/d21/f83 x:0 0 0 2024-05-27T09:29:20.476 INFO:tasks.workunit.client.0.smithi119.stdout:6/362: creat d0/f84 x:0 0 0 2024-05-27T09:29:20.487 INFO:tasks.workunit.client.0.smithi119.stdout:7/460: rename d7/df/d19/d1d/c30 to d7/df/d19/d78/c90 0 2024-05-27T09:29:20.487 INFO:tasks.workunit.client.0.smithi119.stdout:7/461: rename d7/df/l3e to d7/df/d19/d4d/l91 0 2024-05-27T09:29:20.496 INFO:tasks.workunit.client.0.smithi119.stdout:5/400: dread d6/db/de/d27/f50 [0,4194304] 0 2024-05-27T09:29:20.525 INFO:tasks.workunit.client.0.smithi119.stdout:8/386: sync 2024-05-27T09:29:20.525 INFO:tasks.workunit.client.0.smithi119.stdout:8/387: stat d3/d8/d4d/d5d/d60 0 2024-05-27T09:29:20.537 INFO:tasks.workunit.client.0.smithi119.stdout:8/388: dread d3/de/f64 [0,4194304] 0 2024-05-27T09:29:20.537 INFO:tasks.workunit.client.0.smithi119.stdout:8/389: chown d3/d8/fd 15 1 2024-05-27T09:29:20.537 INFO:tasks.workunit.client.0.smithi119.stdout:8/390: chown d3/d39/f52 4 1 2024-05-27T09:29:20.538 INFO:tasks.workunit.client.0.smithi119.stdout:8/391: truncate d3/d8/d4d/f36 906582 0 2024-05-27T09:29:20.543 INFO:tasks.workunit.client.0.smithi119.stdout:5/401: dwrite d6/db/de/d27/f72 [0,4194304] 0 2024-05-27T09:29:20.559 INFO:tasks.workunit.client.0.smithi119.stdout:3/528: dwrite db/d36/d45/d4d/f70 [0,4194304] 0 2024-05-27T09:29:20.563 INFO:tasks.workunit.client.0.smithi119.stdout:3/529: unlink f8 0 2024-05-27T09:29:20.563 INFO:tasks.workunit.client.0.smithi119.stdout:3/530: fdatasync db/d36/d45/d49/f85 0 2024-05-27T09:29:20.594 INFO:tasks.workunit.client.0.smithi119.stdout:1/465: dwrite d7/dd/d25/d29/d38/f90 [0,4194304] 0 2024-05-27T09:29:20.597 INFO:tasks.workunit.client.0.smithi119.stdout:1/466: rename d7/db/d12/f4c to d7/db/d12/f96 0 2024-05-27T09:29:20.597 INFO:tasks.workunit.client.0.smithi119.stdout:1/467: fsync d7/db/d12/d26/d4a/f81 0 2024-05-27T09:29:20.599 INFO:tasks.workunit.client.0.smithi119.stdout:1/468: rename d7/dd/d25/d29/d38/d3c/f53 to d7/db/d12/d26/f97 0 2024-05-27T09:29:20.606 INFO:tasks.workunit.client.0.smithi119.stdout:8/392: dwrite d3/d8/d4d/f36 [0,4194304] 0 2024-05-27T09:29:20.612 INFO:tasks.workunit.client.0.smithi119.stdout:6/363: read d0/d2/d1c/d21/f3b [41272,57352] 0 2024-05-27T09:29:20.613 INFO:tasks.workunit.client.0.smithi119.stdout:9/310: dwrite d18/f23 [0,4194304] 0 2024-05-27T09:29:20.617 INFO:tasks.workunit.client.0.smithi119.stdout:9/311: mknod d18/d19/d1b/d40/c65 0 2024-05-27T09:29:20.620 INFO:tasks.workunit.client.0.smithi119.stdout:9/312: getdents d18/d19/d1b 0 2024-05-27T09:29:20.621 INFO:tasks.workunit.client.0.smithi119.stdout:8/393: dread d3/d8/d4d/f32 [0,4194304] 0 2024-05-27T09:29:20.622 INFO:tasks.workunit.client.0.smithi119.stdout:0/392: sync 2024-05-27T09:29:20.622 INFO:tasks.workunit.client.0.smithi119.stdout:0/393: dread - d2/d6/d9/d12/d4b/d59/f83 zero size 2024-05-27T09:29:20.622 INFO:tasks.workunit.client.0.smithi119.stdout:0/394: stat d2/d6/d23/d7a/ca 0 2024-05-27T09:29:20.624 INFO:tasks.workunit.client.0.smithi119.stdout:0/395: mknod d2/d1a/c84 0 2024-05-27T09:29:20.625 INFO:tasks.workunit.client.0.smithi119.stdout:1/469: dread d7/dd/d44/f56 [0,4194304] 0 2024-05-27T09:29:20.626 INFO:tasks.workunit.client.0.smithi119.stdout:1/470: chown d7/f11 283669526 1 2024-05-27T09:29:20.627 INFO:tasks.workunit.client.0.smithi119.stdout:1/471: mknod d7/db/d12/d26/d39/c98 0 2024-05-27T09:29:20.628 INFO:tasks.workunit.client.0.smithi119.stdout:1/472: write d7/f3d [66119,19797] 0 2024-05-27T09:29:20.641 INFO:tasks.workunit.client.0.smithi119.stdout:9/313: dread d18/d19/f50 [4194304,4194304] 0 2024-05-27T09:29:20.642 INFO:tasks.workunit.client.0.smithi119.stdout:9/314: write d18/d19/f36 [319379,28683] 0 2024-05-27T09:29:20.644 INFO:tasks.workunit.client.0.smithi119.stdout:9/315: mknod d18/d19/d1b/d2a/c66 0 2024-05-27T09:29:20.645 INFO:tasks.workunit.client.0.smithi119.stdout:1/473: dread d7/f11 [0,4194304] 0 2024-05-27T09:29:20.645 INFO:tasks.workunit.client.0.smithi119.stdout:1/474: stat d7/db/d35/f3e 0 2024-05-27T09:29:20.646 INFO:tasks.workunit.client.0.smithi119.stdout:9/316: link d18/d19/d1b/l53 d18/l67 0 2024-05-27T09:29:20.646 INFO:tasks.workunit.client.0.smithi119.stdout:9/317: write d18/d19/d1b/d40/d44/f4a [303576,38985] 0 2024-05-27T09:29:20.661 INFO:tasks.workunit.client.0.smithi119.stdout:9/318: dread d18/f21 [0,4194304] 0 2024-05-27T09:29:20.674 INFO:tasks.workunit.client.0.smithi119.stdout:1/475: read d7/db/d12/f96 [2912563,42951] 0 2024-05-27T09:29:20.678 INFO:tasks.workunit.client.0.smithi119.stdout:6/364: dwrite d0/f68 [4194304,4194304] 0 2024-05-27T09:29:20.679 INFO:tasks.workunit.client.0.smithi119.stdout:6/365: truncate d0/d2/d1c/d21/d58/f61 5198541 0 2024-05-27T09:29:20.679 INFO:tasks.workunit.client.0.smithi119.stdout:6/366: truncate d0/d2/d57/f4a 6498 0 2024-05-27T09:29:20.691 INFO:tasks.workunit.client.0.smithi119.stdout:0/396: dwrite d2/d1a/d20/f64 [4194304,4194304] 0 2024-05-27T09:29:20.691 INFO:tasks.workunit.client.0.smithi119.stdout:0/397: write d2/d1a/d20/f64 [274514,41664] 0 2024-05-27T09:29:20.726 INFO:tasks.workunit.client.0.smithi119.stdout:6/367: dwrite d0/d2/d57/d37/f5d [0,4194304] 0 2024-05-27T09:29:20.727 INFO:tasks.workunit.client.0.smithi119.stdout:1/476: dwrite d7/f5b [0,4194304] 0 2024-05-27T09:29:20.727 INFO:tasks.workunit.client.0.smithi119.stdout:9/319: dwrite d18/d19/d1b/d40/f62 [0,4194304] 0 2024-05-27T09:29:20.730 INFO:tasks.workunit.client.0.smithi119.stdout:9/320: write d18/d19/f36 [905829,125757] 0 2024-05-27T09:29:20.731 INFO:tasks.workunit.client.0.smithi119.stdout:6/368: unlink d0/d2/d1c/f59 0 2024-05-27T09:29:20.731 INFO:tasks.workunit.client.0.smithi119.stdout:6/369: write d0/d2/d4e/f6a [481582,5476] 0 2024-05-27T09:29:20.731 INFO:tasks.workunit.client.0.smithi119.stdout:6/370: chown d0 3539826 1 2024-05-27T09:29:20.731 INFO:tasks.workunit.client.0.smithi119.stdout:6/371: write d0/d2/d57/d37/d74/f70 [855687,31291] 0 2024-05-27T09:29:20.731 INFO:tasks.workunit.client.0.smithi119.stdout:6/372: truncate d0/d2/f30 616461 0 2024-05-27T09:29:20.737 INFO:tasks.workunit.client.0.smithi119.stdout:6/373: mkdir d0/d85 0 2024-05-27T09:29:20.743 INFO:tasks.workunit.client.0.smithi119.stdout:6/374: unlink d0/d2/d1c/d21/f28 0 2024-05-27T09:29:20.743 INFO:tasks.workunit.client.0.smithi119.stdout:6/375: readlink d0/d2/d4e/l2f 0 2024-05-27T09:29:20.748 INFO:tasks.workunit.client.0.smithi119.stdout:4/360: sync 2024-05-27T09:29:20.748 INFO:tasks.workunit.client.0.smithi119.stdout:2/409: sync 2024-05-27T09:29:20.750 INFO:tasks.workunit.client.1.smithi138.stderr:++ readlink -f fsstress 2024-05-27T09:29:20.751 INFO:tasks.workunit.client.0.smithi119.stdout:2/410: write d2/da/d15/d33/f79 [305389,52432] 0 2024-05-27T09:29:20.751 INFO:tasks.workunit.client.0.smithi119.stdout:4/361: chown d4/de/d21/d40/f6a 0 1 2024-05-27T09:29:20.751 INFO:tasks.workunit.client.0.smithi119.stdout:4/362: stat d4/d6/c4f 0 2024-05-27T09:29:20.751 INFO:tasks.workunit.client.0.smithi119.stdout:4/363: truncate d4/d1c/d39/f57 507472 0 2024-05-27T09:29:20.751 INFO:tasks.workunit.client.0.smithi119.stdout:7/462: sync 2024-05-27T09:29:20.751 INFO:tasks.workunit.client.0.smithi119.stdout:7/463: truncate d7/df/d19/d1d/f87 518335 0 2024-05-27T09:29:20.752 INFO:tasks.workunit.client.0.smithi119.stdout:9/321: dread d18/f23 [0,4194304] 0 2024-05-27T09:29:20.752 INFO:tasks.workunit.client.0.smithi119.stdout:1/477: read d7/db/d12/d26/f97 [908389,103487] 0 2024-05-27T09:29:20.752 INFO:tasks.workunit.client.0.smithi119.stdout:0/398: dwrite d2/d6/f3c [0,4194304] 0 2024-05-27T09:29:20.753 INFO:tasks.workunit.client.1.smithi138.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-27T09:29:20.754 INFO:tasks.workunit.client.1.smithi138.stdout:~/cephtest/mnt.1/client.1/tmp 2024-05-27T09:29:20.754 INFO:tasks.workunit.client.1.smithi138.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-05-27T09:29:20.754 INFO:tasks.workunit.client.1.smithi138.stderr:+ popd 2024-05-27T09:29:20.754 INFO:tasks.workunit.client.1.smithi138.stderr:+ popd 2024-05-27T09:29:20.754 INFO:tasks.workunit.client.1.smithi138.stderr:++ mktemp -d -p . 2024-05-27T09:29:20.755 INFO:tasks.workunit.client.0.smithi119.stdout:7/464: mkdir d7/db/d92 0 2024-05-27T09:29:20.755 INFO:tasks.workunit.client.0.smithi119.stdout:9/322: getdents d18 0 2024-05-27T09:29:20.755 INFO:tasks.workunit.client.0.smithi119.stdout:7/465: fsync d7/fd 0 2024-05-27T09:29:20.756 INFO:tasks.workunit.client.0.smithi119.stdout:1/478: mknod d7/dd/d25/d29/d38/c99 0 2024-05-27T09:29:20.761 INFO:tasks.workunit.client.0.smithi119.stdout:9/323: symlink d18/d19/d1b/d40/d44/l68 0 2024-05-27T09:29:20.762 INFO:tasks.workunit.client.1.smithi138.stderr:+ T=./tmp.nUPTxQ7sni 2024-05-27T09:29:20.762 INFO:tasks.workunit.client.1.smithi138.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.nUPTxQ7sni -l 1 -n 1000 -p 10 -v 2024-05-27T09:29:20.765 INFO:tasks.workunit.client.1.smithi138.stdout:seed = 1717559301 2024-05-27T09:29:20.770 INFO:tasks.workunit.client.1.smithi138.stdout:0/0: stat - no entries 2024-05-27T09:29:20.771 INFO:tasks.workunit.client.1.smithi138.stdout:0/1: fdatasync - no filename 2024-05-27T09:29:20.771 INFO:tasks.workunit.client.1.smithi138.stdout:0/2: truncate - no filename 2024-05-27T09:29:20.771 INFO:tasks.workunit.client.0.smithi119.stdout:0/399: dread d2/f49 [0,4194304] 0 2024-05-27T09:29:20.771 INFO:tasks.workunit.client.0.smithi119.stdout:0/400: creat d2/d6/d9/f85 x:0 0 0 2024-05-27T09:29:20.771 INFO:tasks.workunit.client.0.smithi119.stdout:0/401: write d2/d6/d9/d24/d30/d6b/f70 [244975,18754] 0 2024-05-27T09:29:20.771 INFO:tasks.workunit.client.0.smithi119.stdout:4/364: dread d4/d23/d31/f33 [0,4194304] 0 2024-05-27T09:29:20.771 INFO:tasks.workunit.client.0.smithi119.stdout:2/411: dread d2/da/f37 [0,4194304] 0 2024-05-27T09:29:20.772 INFO:tasks.workunit.client.0.smithi119.stdout:8/394: sync 2024-05-27T09:29:20.772 INFO:tasks.workunit.client.0.smithi119.stdout:3/531: sync 2024-05-27T09:29:20.773 INFO:tasks.workunit.client.1.smithi138.stdout:0/3: symlink l0 0 2024-05-27T09:29:20.773 INFO:tasks.workunit.client.1.smithi138.stdout:0/4: chown l0 1568 1 2024-05-27T09:29:20.773 INFO:tasks.workunit.client.1.smithi138.stdout:1/0: truncate - no filename 2024-05-27T09:29:20.774 INFO:tasks.workunit.client.1.smithi138.stdout:1/1: fdatasync - no filename 2024-05-27T09:29:20.774 INFO:tasks.workunit.client.1.smithi138.stdout:1/2: write - no filename 2024-05-27T09:29:20.774 INFO:tasks.workunit.client.1.smithi138.stdout:1/3: dwrite - no filename 2024-05-27T09:29:20.774 INFO:tasks.workunit.client.1.smithi138.stdout:1/4: truncate - no filename 2024-05-27T09:29:20.774 INFO:tasks.workunit.client.1.smithi138.stdout:1/5: dread - no filename 2024-05-27T09:29:20.774 INFO:tasks.workunit.client.1.smithi138.stdout:1/6: fsync - no filename 2024-05-27T09:29:20.774 INFO:tasks.workunit.client.1.smithi138.stdout:1/7: dread - no filename 2024-05-27T09:29:20.774 INFO:tasks.workunit.client.0.smithi119.stdout:5/402: sync 2024-05-27T09:29:20.779 INFO:tasks.workunit.client.0.smithi119.stdout:2/412: truncate d2/da/d4e/f4f 941352 0 2024-05-27T09:29:20.779 INFO:tasks.workunit.client.1.smithi138.stdout:0/5: creat f1 x:0 0 0 2024-05-27T09:29:20.779 INFO:tasks.workunit.client.1.smithi138.stdout:0/6: rmdir - no directory 2024-05-27T09:29:20.780 INFO:tasks.workunit.client.1.smithi138.stdout:0/7: read - f1 zero size 2024-05-27T09:29:20.780 INFO:tasks.workunit.client.1.smithi138.stdout:0/8: chown l0 1228 1 2024-05-27T09:29:20.780 INFO:tasks.workunit.client.1.smithi138.stdout:1/8: mknod c0 0 2024-05-27T09:29:20.780 INFO:tasks.workunit.client.1.smithi138.stdout:1/9: stat c0 0 2024-05-27T09:29:20.780 INFO:tasks.workunit.client.1.smithi138.stdout:2/0: mknod c0 0 2024-05-27T09:29:20.781 INFO:tasks.workunit.client.1.smithi138.stdout:1/10: unlink c0 0 2024-05-27T09:29:20.781 INFO:tasks.workunit.client.0.smithi119.stdout:8/395: dread d3/d8/f34 [0,4194304] 0 2024-05-27T09:29:20.782 INFO:tasks.workunit.client.0.smithi119.stdout:8/396: fdatasync d3/d8/d1b/f28 0 2024-05-27T09:29:20.782 INFO:tasks.workunit.client.0.smithi119.stdout:5/403: symlink d6/dd/d39/d19/l8a 0 2024-05-27T09:29:20.782 INFO:tasks.workunit.client.1.smithi138.stdout:2/1: mkdir d1 0 2024-05-27T09:29:20.784 INFO:tasks.workunit.client.0.smithi119.stdout:2/413: rename d2/d1a/l2f to d2/da/d15/l8c 0 2024-05-27T09:29:20.785 INFO:tasks.workunit.client.0.smithi119.stdout:4/365: rename d4/d23/d24/l43 to d4/d23/d24/l71 0 2024-05-27T09:29:20.785 INFO:tasks.workunit.client.0.smithi119.stdout:3/532: read - db/d36/d45/d4c/d62/d76/d98/f9f zero size 2024-05-27T09:29:20.787 INFO:tasks.workunit.client.0.smithi119.stdout:8/397: creat d3/d8/d1b/d46/f81 x:0 0 0 2024-05-27T09:29:20.788 INFO:tasks.workunit.client.0.smithi119.stdout:2/414: creat d2/da/d15/d33/d89/f8d x:0 0 0 2024-05-27T09:29:20.789 INFO:tasks.workunit.client.1.smithi138.stdout:3/0: mkdir d0 0 2024-05-27T09:29:20.790 INFO:tasks.workunit.client.1.smithi138.stdout:2/2: unlink c0 0 2024-05-27T09:29:20.790 INFO:tasks.workunit.client.1.smithi138.stdout:3/1: chown d0 1889 1 2024-05-27T09:29:20.790 INFO:tasks.workunit.client.1.smithi138.stdout:4/0: creat f0 x:0 0 0 2024-05-27T09:29:20.790 INFO:tasks.workunit.client.1.smithi138.stdout:3/2: fsync - no filename 2024-05-27T09:29:20.791 INFO:tasks.workunit.client.1.smithi138.stdout:2/3: dread - no filename 2024-05-27T09:29:20.791 INFO:tasks.workunit.client.1.smithi138.stdout:2/4: fsync - no filename 2024-05-27T09:29:20.791 INFO:tasks.workunit.client.1.smithi138.stdout:2/5: truncate - no filename 2024-05-27T09:29:20.792 INFO:tasks.workunit.client.1.smithi138.stdout:2/6: getdents d1 0 2024-05-27T09:29:20.793 INFO:tasks.workunit.client.0.smithi119.stdout:8/398: rename d3/d8/d1b/f20 to d3/d8/d4d/d58/f82 0 2024-05-27T09:29:20.796 INFO:tasks.workunit.client.1.smithi138.stdout:6/0: unlink - no file 2024-05-27T09:29:20.796 INFO:tasks.workunit.client.1.smithi138.stdout:6/1: dwrite - no filename 2024-05-27T09:29:20.797 INFO:tasks.workunit.client.1.smithi138.stdout:6/2: dwrite - no filename 2024-05-27T09:29:20.797 INFO:tasks.workunit.client.1.smithi138.stdout:5/0: creat f0 x:0 0 0 2024-05-27T09:29:20.797 INFO:tasks.workunit.client.1.smithi138.stdout:5/1: creat f1 x:0 0 0 2024-05-27T09:29:20.798 INFO:tasks.workunit.client.0.smithi119.stdout:9/324: read d18/f30 [6392646,15322] 0 2024-05-27T09:29:20.798 INFO:tasks.workunit.client.0.smithi119.stdout:9/325: truncate d18/d19/d1b/f5d 5196652 0 2024-05-27T09:29:20.798 INFO:tasks.workunit.client.1.smithi138.stdout:2/7: creat d1/f2 x:0 0 0 2024-05-27T09:29:20.798 INFO:tasks.workunit.client.1.smithi138.stdout:2/8: dread - d1/f2 zero size 2024-05-27T09:29:20.798 INFO:tasks.workunit.client.1.smithi138.stdout:6/3: creat f0 x:0 0 0 2024-05-27T09:29:20.799 INFO:tasks.workunit.client.1.smithi138.stdout:6/4: write f0 [551736,3668] 0 2024-05-27T09:29:20.804 INFO:tasks.workunit.client.0.smithi119.stdout:0/402: dread d2/d6/d9/d24/d30/d4a/f4c [0,4194304] 0 2024-05-27T09:29:20.804 INFO:tasks.workunit.client.0.smithi119.stdout:0/403: write d2/d6/d9/d12/d4b/d59/f58 [452856,81636] 0 2024-05-27T09:29:20.804 INFO:tasks.workunit.client.0.smithi119.stdout:8/399: mknod d3/d8/c83 0 2024-05-27T09:29:20.809 INFO:tasks.workunit.client.0.smithi119.stdout:9/326: rename d18/f21 to d18/d19/d1b/d40/d44/f69 0 2024-05-27T09:29:20.810 INFO:tasks.workunit.client.1.smithi138.stdout:2/9: mkdir d1/d3 0 2024-05-27T09:29:20.810 INFO:tasks.workunit.client.1.smithi138.stdout:7/0: getdents . 0 2024-05-27T09:29:20.811 INFO:tasks.workunit.client.0.smithi119.stdout:0/404: link d2/d1a/d20/f74 d2/d6/d75/f86 0 2024-05-27T09:29:20.811 INFO:tasks.workunit.client.1.smithi138.stdout:5/2: mkdir d2 0 2024-05-27T09:29:20.811 INFO:tasks.workunit.client.1.smithi138.stdout:2/10: write d1/f2 [536364,19091] 0 2024-05-27T09:29:20.812 INFO:tasks.workunit.client.0.smithi119.stdout:0/405: creat d2/d6/d9/d24/d30/d6b/f87 x:0 0 0 2024-05-27T09:29:20.812 INFO:tasks.workunit.client.0.smithi119.stdout:0/406: chown d2/d6/d9/ld 1283 1 2024-05-27T09:29:20.812 INFO:tasks.workunit.client.0.smithi119.stdout:8/400: mkdir d3/d6e/d80/d84 0 2024-05-27T09:29:20.812 INFO:tasks.workunit.client.0.smithi119.stdout:5/404: read d6/db/dc/f61 [1734795,2297] 0 2024-05-27T09:29:20.815 INFO:tasks.workunit.client.1.smithi138.stdout:6/5: mknod c1 0 2024-05-27T09:29:20.816 INFO:tasks.workunit.client.0.smithi119.stdout:7/466: dread d7/d1a/d40/f48 [0,4194304] 0 2024-05-27T09:29:20.816 INFO:tasks.workunit.client.0.smithi119.stdout:0/407: rename d2/d6/d9/d24/f40 to d2/d6/d9/f88 0 2024-05-27T09:29:20.816 INFO:tasks.workunit.client.0.smithi119.stdout:0/408: chown d2/d1a/c84 3 1 2024-05-27T09:29:20.816 INFO:tasks.workunit.client.0.smithi119.stdout:0/409: readlink d2/d6/d9/ld 0 2024-05-27T09:29:20.817 INFO:tasks.workunit.client.0.smithi119.stdout:8/401: mkdir d3/d8/d4d/d5d/d66/d85 0 2024-05-27T09:29:20.817 INFO:tasks.workunit.client.0.smithi119.stdout:5/405: mkdir d6/dd/df/d8b 0 2024-05-27T09:29:20.818 INFO:tasks.workunit.client.1.smithi138.stdout:7/1: mkdir d0 0 2024-05-27T09:29:20.819 INFO:tasks.workunit.client.1.smithi138.stdout:7/2: truncate - no filename 2024-05-27T09:29:20.819 INFO:tasks.workunit.client.0.smithi119.stdout:8/402: readlink d3/d8/d1b/d46/l4f 0 2024-05-27T09:29:20.819 INFO:tasks.workunit.client.0.smithi119.stdout:8/403: truncate d3/d39/f52 696555 0 2024-05-27T09:29:20.822 INFO:tasks.workunit.client.0.smithi119.stdout:5/406: symlink d6/db/l8c 0 2024-05-27T09:29:20.825 INFO:tasks.workunit.client.1.smithi138.stdout:2/11: creat d1/d3/f4 x:0 0 0 2024-05-27T09:29:20.831 INFO:tasks.workunit.client.0.smithi119.stdout:8/404: symlink d3/d8/d4d/d5d/d66/l86 0 2024-05-27T09:29:20.835 INFO:tasks.workunit.client.1.smithi138.stdout:7/3: creat d0/f1 x:0 0 0 2024-05-27T09:29:20.838 INFO:tasks.workunit.client.1.smithi138.stdout:2/12: symlink d1/d3/l5 0 2024-05-27T09:29:20.838 INFO:tasks.workunit.client.1.smithi138.stdout:2/13: chown d1 340602205 1 2024-05-27T09:29:20.843 INFO:tasks.workunit.client.0.smithi119.stdout:2/415: rmdir d2/da 39 2024-05-27T09:29:20.854 INFO:tasks.workunit.client.0.smithi119.stdout:4/366: dread d4/f32 [0,4194304] 0 2024-05-27T09:29:20.854 INFO:tasks.workunit.client.1.smithi138.stdout:9/0: creat f0 x:0 0 0 2024-05-27T09:29:20.856 INFO:tasks.workunit.client.1.smithi138.stdout:1/11: getdents . 0 2024-05-27T09:29:20.856 INFO:tasks.workunit.client.1.smithi138.stdout:1/12: fsync - no filename 2024-05-27T09:29:20.870 INFO:tasks.workunit.client.1.smithi138.stdout:9/1: rename f0 to f1 0 2024-05-27T09:29:20.870 INFO:tasks.workunit.client.1.smithi138.stdout:9/2: fdatasync f1 0 2024-05-27T09:29:20.878 INFO:tasks.workunit.client.1.smithi138.stdout:1/13: creat f1 x:0 0 0 2024-05-27T09:29:20.881 INFO:tasks.workunit.client.1.smithi138.stdout:1/14: mknod c2 0 2024-05-27T09:29:20.889 INFO:tasks.workunit.client.1.smithi138.stdout:1/15: unlink c2 0 2024-05-27T09:29:20.889 INFO:tasks.workunit.client.1.smithi138.stdout:1/16: chown f1 280427868 1 2024-05-27T09:29:20.889 INFO:tasks.workunit.client.1.smithi138.stdout:1/17: stat f1 0 2024-05-27T09:29:20.893 INFO:tasks.workunit.client.1.smithi138.stdout:1/18: link f1 f3 0 2024-05-27T09:29:20.893 INFO:tasks.workunit.client.1.smithi138.stdout:1/19: stat f3 0 2024-05-27T09:29:20.893 INFO:tasks.workunit.client.1.smithi138.stdout:1/20: rmdir - no directory 2024-05-27T09:29:20.894 INFO:tasks.workunit.client.1.smithi138.stdout:1/21: mkdir d4 0 2024-05-27T09:29:20.904 INFO:tasks.workunit.client.1.smithi138.stdout:1/22: link f1 d4/f5 0 2024-05-27T09:29:20.908 INFO:tasks.workunit.client.1.smithi138.stdout:1/23: unlink f1 0 2024-05-27T09:29:20.911 INFO:tasks.workunit.client.1.smithi138.stdout:1/24: stat d4/f5 0 2024-05-27T09:29:20.914 INFO:tasks.workunit.client.1.smithi138.stdout:1/25: mknod d4/c6 0 2024-05-27T09:29:20.914 INFO:tasks.workunit.client.1.smithi138.stdout:1/26: dread - d4/f5 zero size 2024-05-27T09:29:20.921 INFO:tasks.workunit.client.1.smithi138.stdout:1/27: creat d4/f7 x:0 0 0 2024-05-27T09:29:20.921 INFO:tasks.workunit.client.1.smithi138.stdout:1/28: dread - d4/f5 zero size 2024-05-27T09:29:20.921 INFO:tasks.workunit.client.0.smithi119.stdout:9/327: read f10 [1558140,115549] 0 2024-05-27T09:29:20.923 INFO:tasks.workunit.client.1.smithi138.stdout:1/29: mkdir d4/d8 0 2024-05-27T09:29:20.923 INFO:tasks.workunit.client.1.smithi138.stdout:1/30: write d4/f5 [321806,37482] 0 2024-05-27T09:29:20.929 INFO:tasks.workunit.client.1.smithi138.stdout:6/6: read f0 [352281,64564] 0 2024-05-27T09:29:20.929 INFO:tasks.workunit.client.1.smithi138.stdout:6/7: creat f2 x:0 0 0 2024-05-27T09:29:20.933 INFO:tasks.workunit.client.0.smithi119.stdout:9/328: rename l9 to d18/d19/l6a 0 2024-05-27T09:29:20.937 INFO:tasks.workunit.client.0.smithi119.stdout:9/329: chown d18/d19/c5e 64357 1 2024-05-27T09:29:20.937 INFO:tasks.workunit.client.0.smithi119.stdout:9/330: stat d18/d19/c5e 0 2024-05-27T09:29:20.942 INFO:tasks.workunit.client.0.smithi119.stdout:9/331: mknod d18/d19/d1b/d2a/c6b 0 2024-05-27T09:29:20.942 INFO:tasks.workunit.client.0.smithi119.stdout:9/332: read - d18/d19/d1b/d40/f60 zero size 2024-05-27T09:29:20.942 INFO:tasks.workunit.client.0.smithi119.stdout:9/333: chown d18/d19/d1b/l53 475865393 1 2024-05-27T09:29:20.966 INFO:tasks.workunit.client.0.smithi119.stdout:0/410: dread d2/d6/d23/d7a/f44 [0,4194304] 0 2024-05-27T09:29:20.968 INFO:tasks.workunit.client.0.smithi119.stdout:0/411: write d2/d6/d23/f33 [3022326,111545] 0 2024-05-27T09:29:20.975 INFO:tasks.workunit.client.0.smithi119.stdout:0/412: truncate d2/d6/f3c 365803 0 2024-05-27T09:29:20.975 INFO:tasks.workunit.client.0.smithi119.stdout:0/413: fsync d2/d6/d23/d7a/f46 0 2024-05-27T09:29:20.975 INFO:tasks.workunit.client.0.smithi119.stdout:0/414: fdatasync d2/d6/d9/f6d 0 2024-05-27T09:29:20.976 INFO:tasks.workunit.client.1.smithi138.stdout:6/8: mkdir d3 0 2024-05-27T09:29:20.976 INFO:tasks.workunit.client.1.smithi138.stdout:6/9: chown c1 6287542 1 2024-05-27T09:29:20.983 INFO:tasks.workunit.client.0.smithi119.stdout:0/415: creat d2/d6/d9/d12/d4f/f89 x:0 0 0 2024-05-27T09:29:20.990 INFO:tasks.workunit.client.0.smithi119.stdout:0/416: creat d2/d6/d75/f8a x:0 0 0 2024-05-27T09:29:21.013 INFO:tasks.workunit.client.0.smithi119.stdout:1/479: dwrite d7/db/d12/d15/d1f/f34 [0,4194304] 0 2024-05-27T09:29:21.014 INFO:tasks.workunit.client.0.smithi119.stdout:1/480: chown d7/db/d12/d15/d1f/l8a 220 1 2024-05-27T09:29:21.014 INFO:tasks.workunit.client.0.smithi119.stdout:1/481: dread - d7/db/d12/f6b zero size 2024-05-27T09:29:21.014 INFO:tasks.workunit.client.0.smithi119.stdout:1/482: chown d7/dd/d44/f56 31252233 1 2024-05-27T09:29:21.014 INFO:tasks.workunit.client.0.smithi119.stdout:1/483: write d7/db/d12/d15/d1f/f8b [527831,71728] 0 2024-05-27T09:29:21.048 INFO:tasks.workunit.client.1.smithi138.stdout:0/9: dwrite f1 [0,4194304] 0 2024-05-27T09:29:21.048 INFO:tasks.workunit.client.1.smithi138.stdout:0/10: creat f2 x:0 0 0 2024-05-27T09:29:21.058 INFO:tasks.workunit.client.0.smithi119.stdout:1/484: dread d7/dd/f77 [0,4194304] 0 2024-05-27T09:29:21.058 INFO:tasks.workunit.client.0.smithi119.stdout:3/533: dwrite db/d36/f71 [0,4194304] 0 2024-05-27T09:29:21.058 INFO:tasks.workunit.client.0.smithi119.stdout:1/485: creat d7/db/d12/f9a x:0 0 0 2024-05-27T09:29:21.125 INFO:tasks.workunit.client.1.smithi138.stdout:2/14: write d1/f2 [506803,64055] 0 2024-05-27T09:29:21.125 INFO:tasks.workunit.client.1.smithi138.stdout:2/15: readlink d1/d3/l5 0 2024-05-27T09:29:21.125 INFO:tasks.workunit.client.1.smithi138.stdout:2/16: dread - d1/d3/f4 zero size 2024-05-27T09:29:21.125 INFO:tasks.workunit.client.1.smithi138.stdout:9/3: getdents . 0 2024-05-27T09:29:21.125 INFO:tasks.workunit.client.1.smithi138.stdout:2/17: creat d1/d3/f6 x:0 0 0 2024-05-27T09:29:21.165 INFO:tasks.workunit.client.0.smithi119.stdout:0/417: fsync d2/d6/f3c 0 2024-05-27T09:29:21.165 INFO:tasks.workunit.client.0.smithi119.stdout:0/418: readlink d2/d6/d9/d24/d30/d4a/l61 0 2024-05-27T09:29:21.166 INFO:tasks.workunit.client.0.smithi119.stdout:5/407: dwrite d6/dd/df/f75 [0,4194304] 0 2024-05-27T09:29:21.169 INFO:tasks.workunit.client.0.smithi119.stdout:8/405: dwrite d3/d8/d1b/d46/f81 [0,4194304] 0 2024-05-27T09:29:21.172 INFO:tasks.workunit.client.0.smithi119.stdout:5/408: creat d6/db/de/d27/d77/f8d x:0 0 0 2024-05-27T09:29:21.172 INFO:tasks.workunit.client.0.smithi119.stdout:5/409: readlink d6/dd/d46/d47/d4b/l86 0 2024-05-27T09:29:21.175 INFO:tasks.workunit.client.0.smithi119.stdout:8/406: write d3/d8/d1b/d46/f29 [3038228,36916] 0 2024-05-27T09:29:21.175 INFO:tasks.workunit.client.0.smithi119.stdout:8/407: dread - d3/d8/d4d/d5d/d60/f71 zero size 2024-05-27T09:29:21.175 INFO:tasks.workunit.client.0.smithi119.stdout:8/408: fdatasync d3/f16 0 2024-05-27T09:29:21.175 INFO:tasks.workunit.client.0.smithi119.stdout:8/409: stat d3/d39/f3e 0 2024-05-27T09:29:21.177 INFO:tasks.workunit.client.0.smithi119.stdout:5/410: symlink d6/dd/l8e 0 2024-05-27T09:29:21.177 INFO:tasks.workunit.client.1.smithi138.stdout:5/3: dwrite f1 [0,4194304] 0 2024-05-27T09:29:21.183 INFO:tasks.workunit.client.0.smithi119.stdout:3/534: dread db/d36/d45/d4c/d62/f3f [0,4194304] 0 2024-05-27T09:29:21.183 INFO:tasks.workunit.client.0.smithi119.stdout:3/535: creat db/d5b/fb3 x:0 0 0 2024-05-27T09:29:21.187 INFO:tasks.workunit.client.0.smithi119.stdout:8/410: fdatasync d3/d8/d1b/d46/f4b 0 2024-05-27T09:29:21.199 INFO:tasks.workunit.client.0.smithi119.stdout:3/536: symlink db/d7f/da9/lb4 0 2024-05-27T09:29:21.199 INFO:tasks.workunit.client.0.smithi119.stdout:8/411: creat d3/f87 x:0 0 0 2024-05-27T09:29:21.199 INFO:tasks.workunit.client.0.smithi119.stdout:8/412: write d3/f2d [2410396,101902] 0 2024-05-27T09:29:21.203 INFO:tasks.workunit.client.0.smithi119.stdout:3/537: creat db/d36/d45/d49/d9d/fb5 x:0 0 0 2024-05-27T09:29:21.204 INFO:tasks.workunit.client.1.smithi138.stdout:4/1: dwrite f0 [0,4194304] 0 2024-05-27T09:29:21.206 INFO:tasks.workunit.client.0.smithi119.stdout:8/413: rename d3/de/f64 to d3/d39/f88 0 2024-05-27T09:29:21.206 INFO:tasks.workunit.client.0.smithi119.stdout:8/414: creat d3/d8/d4d/d58/f89 x:0 0 0 2024-05-27T09:29:21.213 INFO:tasks.workunit.client.0.smithi119.stdout:2/416: dwrite d2/da/d15/d33/f60 [4194304,4194304] 0 2024-05-27T09:29:21.216 INFO:tasks.workunit.client.1.smithi138.stdout:0/11: symlink l3 0 2024-05-27T09:29:21.216 INFO:tasks.workunit.client.1.smithi138.stdout:0/12: creat f4 x:0 0 0 2024-05-27T09:29:21.216 INFO:tasks.workunit.client.1.smithi138.stdout:0/13: creat f5 x:0 0 0 2024-05-27T09:29:21.217 INFO:tasks.workunit.client.0.smithi119.stdout:2/417: mknod d2/da/d8a/c8e 0 2024-05-27T09:29:21.217 INFO:tasks.workunit.client.0.smithi119.stdout:2/418: chown d2/da/d2c/f58 230528445 1 2024-05-27T09:29:21.223 INFO:tasks.workunit.client.0.smithi119.stdout:3/538: rmdir db/d36/d45/d4c/d62/d13 39 2024-05-27T09:29:21.223 INFO:tasks.workunit.client.0.smithi119.stdout:3/539: fsync db/f23 0 2024-05-27T09:29:21.231 INFO:tasks.workunit.client.0.smithi119.stdout:2/419: mkdir d2/da/d8a/d8f 0 2024-05-27T09:29:21.246 INFO:tasks.workunit.client.0.smithi119.stdout:0/419: write d2/d6/f8 [11930398,54418] 0 2024-05-27T09:29:21.246 INFO:tasks.workunit.client.0.smithi119.stdout:0/420: creat d2/d1a/d20/f8b x:0 0 0 2024-05-27T09:29:21.246 INFO:tasks.workunit.client.0.smithi119.stdout:0/421: chown d2/d1a/d20/c68 8481 1 2024-05-27T09:29:21.246 INFO:tasks.workunit.client.0.smithi119.stdout:0/422: readlink d2/d6/d9/d24/l41 0 2024-05-27T09:29:21.252 INFO:tasks.workunit.client.0.smithi119.stdout:0/423: getdents d2/d6/d9/d12/d4b/d59 0 2024-05-27T09:29:21.302 INFO:tasks.workunit.client.1.smithi138.stdout:7/4: dwrite d0/f1 [0,4194304] 0 2024-05-27T09:29:21.364 INFO:tasks.workunit.client.0.smithi119.stdout:6/376: sync 2024-05-27T09:29:21.365 INFO:tasks.workunit.client.1.smithi138.stdout:9/4: creat f2 x:0 0 0 2024-05-27T09:29:21.365 INFO:tasks.workunit.client.1.smithi138.stdout:9/5: creat f3 x:0 0 0 2024-05-27T09:29:21.370 INFO:tasks.workunit.client.0.smithi119.stdout:6/377: getdents d0 0 2024-05-27T09:29:21.371 INFO:tasks.workunit.client.0.smithi119.stdout:6/378: chown d0/d2/d77/l7a 20 1 2024-05-27T09:29:21.371 INFO:tasks.workunit.client.0.smithi119.stdout:6/379: readlink d0/d11/l2c 0 2024-05-27T09:29:21.378 INFO:tasks.workunit.client.0.smithi119.stdout:6/380: link d0/d2/d1c/f2a d0/d2/d57/d37/f86 0 2024-05-27T09:29:21.382 INFO:tasks.workunit.client.0.smithi119.stdout:6/381: mknod d0/d85/c87 0 2024-05-27T09:29:21.393 INFO:tasks.workunit.client.0.smithi119.stdout:3/540: rmdir db/d36/d45/d49/d9d 39 2024-05-27T09:29:21.393 INFO:tasks.workunit.client.0.smithi119.stdout:3/541: chown db/d36/d45/d49/d9d/ca1 501765264 1 2024-05-27T09:29:21.396 INFO:tasks.workunit.client.0.smithi119.stdout:3/542: mknod db/d5b/cb6 0 2024-05-27T09:29:21.399 INFO:tasks.workunit.client.0.smithi119.stdout:9/334: read fa [88177,126435] 0 2024-05-27T09:29:21.400 INFO:tasks.workunit.client.0.smithi119.stdout:3/543: write db/d36/d45/d4c/d62/f2d [611965,15517] 0 2024-05-27T09:29:21.400 INFO:tasks.workunit.client.0.smithi119.stdout:3/544: chown db/d7f/c48 12809 1 2024-05-27T09:29:21.400 INFO:tasks.workunit.client.0.smithi119.stdout:3/545: chown db/fc 551 1 2024-05-27T09:29:21.400 INFO:tasks.workunit.client.0.smithi119.stdout:3/546: fdatasync db/f2f 0 2024-05-27T09:29:21.409 INFO:tasks.workunit.client.0.smithi119.stdout:9/335: rmdir d18/d19 39 2024-05-27T09:29:21.417 INFO:tasks.workunit.client.0.smithi119.stdout:6/382: dread d0/d2/f31 [0,4194304] 0 2024-05-27T09:29:21.420 INFO:tasks.workunit.client.0.smithi119.stdout:6/383: rmdir d0/d2/d57 39 2024-05-27T09:29:21.422 INFO:tasks.workunit.client.1.smithi138.stdout:1/31: dread d4/f5 [0,4194304] 0 2024-05-27T09:29:21.422 INFO:tasks.workunit.client.1.smithi138.stdout:1/32: write f3 [1246564,10026] 0 2024-05-27T09:29:21.461 INFO:tasks.workunit.client.0.smithi119.stdout:6/384: getdents d0/d85 0 2024-05-27T09:29:21.521 INFO:tasks.workunit.client.0.smithi119.stdout:8/415: dread d3/d8/f47 [0,4194304] 0 2024-05-27T09:29:21.521 INFO:tasks.workunit.client.0.smithi119.stdout:8/416: write d3/d8/d4d/d58/f6a [119452,53258] 0 2024-05-27T09:29:21.524 INFO:tasks.workunit.client.0.smithi119.stdout:8/417: creat d3/d8/d4d/d5d/f8a x:0 0 0 2024-05-27T09:29:21.547 INFO:tasks.workunit.client.1.smithi138.stdout:1/33: dread d4/f5 [0,4194304] 0 2024-05-27T09:29:21.547 INFO:tasks.workunit.client.1.smithi138.stdout:1/34: write d4/f7 [843153,85302] 0 2024-05-27T09:29:21.578 INFO:tasks.workunit.client.0.smithi119.stdout:9/336: dread d18/f23 [0,4194304] 0 2024-05-27T09:29:21.581 INFO:tasks.workunit.client.0.smithi119.stdout:9/337: creat d18/d19/d1b/d2a/d63/f6c x:0 0 0 2024-05-27T09:29:21.583 INFO:tasks.workunit.client.0.smithi119.stdout:9/338: creat d18/d19/d61/f6d x:0 0 0 2024-05-27T09:29:21.583 INFO:tasks.workunit.client.0.smithi119.stdout:9/339: write d18/d19/d1b/d29/f47 [9037964,26479] 0 2024-05-27T09:29:21.584 INFO:tasks.workunit.client.0.smithi119.stdout:9/340: chown d18/d19/c56 2749 1 2024-05-27T09:29:21.585 INFO:tasks.workunit.client.0.smithi119.stdout:9/341: mknod d18/c6e 0 2024-05-27T09:29:21.626 INFO:tasks.workunit.client.0.smithi119.stdout:6/385: dwrite d0/d2/d1c/f4c [4194304,4194304] 0 2024-05-27T09:29:21.630 INFO:tasks.workunit.client.0.smithi119.stdout:6/386: rename d0/fb to d0/d2/d4e/f88 0 2024-05-27T09:29:21.648 INFO:tasks.workunit.client.1.smithi138.stdout:9/6: dwrite f2 [0,4194304] 0 2024-05-27T09:29:21.648 INFO:tasks.workunit.client.1.smithi138.stdout:9/7: truncate f1 489128 0 2024-05-27T09:29:21.648 INFO:tasks.workunit.client.1.smithi138.stdout:9/8: read f1 [218357,109202] 0 2024-05-27T09:29:21.649 INFO:tasks.workunit.client.1.smithi138.stdout:9/9: creat f4 x:0 0 0 2024-05-27T09:29:21.655 INFO:tasks.workunit.client.0.smithi119.stdout:4/367: dwrite d4/d1c/d39/f5d [0,4194304] 0 2024-05-27T09:29:21.656 INFO:tasks.workunit.client.0.smithi119.stdout:4/368: write d4/d1c/d39/f64 [1452967,109542] 0 2024-05-27T09:29:21.656 INFO:tasks.workunit.client.0.smithi119.stdout:6/387: dread d0/d2/d1c/f3e [0,4194304] 0 2024-05-27T09:29:21.662 INFO:tasks.workunit.client.0.smithi119.stdout:4/369: creat d4/d6/d28/d2f/d36/f72 x:0 0 0 2024-05-27T09:29:21.663 INFO:tasks.workunit.client.0.smithi119.stdout:4/370: mknod d4/d6/d3e/c73 0 2024-05-27T09:29:21.665 INFO:tasks.workunit.client.0.smithi119.stdout:4/371: creat d4/d6/f74 x:0 0 0 2024-05-27T09:29:21.665 INFO:tasks.workunit.client.0.smithi119.stdout:4/372: fdatasync d4/f32 0 2024-05-27T09:29:21.666 INFO:tasks.workunit.client.0.smithi119.stdout:4/373: truncate d4/d48/f51 1173641 0 2024-05-27T09:29:21.668 INFO:tasks.workunit.client.0.smithi119.stdout:6/388: dread d0/d2/d57/d37/f6e [0,4194304] 0 2024-05-27T09:29:21.672 INFO:tasks.workunit.client.0.smithi119.stdout:4/374: mknod d4/de/d66/c75 0 2024-05-27T09:29:21.672 INFO:tasks.workunit.client.0.smithi119.stdout:4/375: chown d4/de/d21/f2b 1974795 1 2024-05-27T09:29:21.672 INFO:tasks.workunit.client.0.smithi119.stdout:4/376: stat d4/d6/d28/d4a 0 2024-05-27T09:29:21.678 INFO:tasks.workunit.client.0.smithi119.stdout:5/411: write d6/db/de/d27/f72 [851417,92926] 0 2024-05-27T09:29:21.686 INFO:tasks.workunit.client.0.smithi119.stdout:5/412: getdents d6/dd/df 0 2024-05-27T09:29:21.687 INFO:tasks.workunit.client.0.smithi119.stdout:2/420: dread d2/da/d4e/f4f [0,4194304] 0 2024-05-27T09:29:21.697 INFO:tasks.workunit.client.0.smithi119.stdout:3/547: write db/d36/f71 [1039583,19773] 0 2024-05-27T09:29:21.698 INFO:tasks.workunit.client.0.smithi119.stdout:3/548: readlink db/d36/d45/d4c/l92 0 2024-05-27T09:29:21.698 INFO:tasks.workunit.client.0.smithi119.stdout:3/549: chown db/d5b 21703465 1 2024-05-27T09:29:21.698 INFO:tasks.workunit.client.0.smithi119.stdout:3/550: chown db/l7d 2134659352 1 2024-05-27T09:29:21.699 INFO:tasks.workunit.client.0.smithi119.stdout:3/551: creat db/d36/d45/d4c/d62/d76/daa/fb7 x:0 0 0 2024-05-27T09:29:21.700 INFO:tasks.workunit.client.0.smithi119.stdout:3/552: truncate db/d5b/fb3 932772 0 2024-05-27T09:29:21.704 INFO:tasks.workunit.client.0.smithi119.stdout:7/467: sync 2024-05-27T09:29:21.704 INFO:tasks.workunit.client.0.smithi119.stdout:4/377: rmdir d4/d6/d28 39 2024-05-27T09:29:21.709 INFO:tasks.workunit.client.1.smithi138.stdout:0/14: mknod c6 0 2024-05-27T09:29:21.710 INFO:tasks.workunit.client.0.smithi119.stdout:1/486: dwrite d7/dd/d25/f30 [0,4194304] 0 2024-05-27T09:29:21.710 INFO:tasks.workunit.client.0.smithi119.stdout:1/487: mkdir d7/db/d8f/d9b 0 2024-05-27T09:29:21.710 INFO:tasks.workunit.client.0.smithi119.stdout:4/378: symlink d4/d6/d28/l76 0 2024-05-27T09:29:21.712 INFO:tasks.workunit.client.0.smithi119.stdout:4/379: rename d4/d23/d24/l30 to d4/d23/d31/l77 0 2024-05-27T09:29:21.712 INFO:tasks.workunit.client.0.smithi119.stdout:4/380: write d4/d6/d28/d2f/d36/f72 [952263,101502] 0 2024-05-27T09:29:21.712 INFO:tasks.workunit.client.0.smithi119.stdout:4/381: write d4/d6/fa [592254,5336] 0 2024-05-27T09:29:21.712 INFO:tasks.workunit.client.0.smithi119.stdout:4/382: fdatasync d4/d23/d31/f53 0 2024-05-27T09:29:21.712 INFO:tasks.workunit.client.0.smithi119.stdout:5/413: read d6/dd/f64 [6306739,117253] 0 2024-05-27T09:29:21.713 INFO:tasks.workunit.client.0.smithi119.stdout:4/383: creat d4/d23/d24/d5b/f78 x:0 0 0 2024-05-27T09:29:21.715 INFO:tasks.workunit.client.0.smithi119.stdout:4/384: mknod d4/d6/d3e/c79 0 2024-05-27T09:29:21.715 INFO:tasks.workunit.client.0.smithi119.stdout:4/385: read - d4/de/d21/d40/f6a zero size 2024-05-27T09:29:21.732 INFO:tasks.workunit.client.0.smithi119.stdout:6/389: dread d0/f52 [0,4194304] 0 2024-05-27T09:29:21.732 INFO:tasks.workunit.client.0.smithi119.stdout:5/414: mknod d6/dd/df/c8f 0 2024-05-27T09:29:21.732 INFO:tasks.workunit.client.0.smithi119.stdout:5/415: write d6/f6a [443115,105570] 0 2024-05-27T09:29:21.732 INFO:tasks.workunit.client.0.smithi119.stdout:5/416: creat d6/f90 x:0 0 0 2024-05-27T09:29:21.732 INFO:tasks.workunit.client.0.smithi119.stdout:2/421: write d2/da/d15/f16 [2515012,65592] 0 2024-05-27T09:29:21.733 INFO:tasks.workunit.client.0.smithi119.stdout:5/417: symlink d6/dd/d39/d19/d3c/l91 0 2024-05-27T09:29:21.733 INFO:tasks.workunit.client.0.smithi119.stdout:5/418: truncate d6/dd/d39/f69 5193549 0 2024-05-27T09:29:21.762 INFO:tasks.workunit.client.0.smithi119.stdout:0/424: dwrite d2/d6/d9/d12/f47 [0,4194304] 0 2024-05-27T09:29:21.765 INFO:tasks.workunit.client.0.smithi119.stdout:6/390: read d0/d2/d57/d37/f5b [3710185,114079] 0 2024-05-27T09:29:21.765 INFO:tasks.workunit.client.0.smithi119.stdout:5/419: dread d6/d2f/f3a [0,4194304] 0 2024-05-27T09:29:21.771 INFO:tasks.workunit.client.0.smithi119.stdout:6/391: stat d0/d2/f30 0 2024-05-27T09:29:21.771 INFO:tasks.workunit.client.0.smithi119.stdout:0/425: rename d2/d6/f55 to d2/f8c 0 2024-05-27T09:29:21.771 INFO:tasks.workunit.client.0.smithi119.stdout:5/420: truncate d6/f1e 6363723 0 2024-05-27T09:29:21.771 INFO:tasks.workunit.client.0.smithi119.stdout:5/421: write d6/db/de/d27/f7e [581796,71973] 0 2024-05-27T09:29:21.771 INFO:tasks.workunit.client.0.smithi119.stdout:0/426: symlink d2/d6/d9/d2f/l8d 0 2024-05-27T09:29:21.774 INFO:tasks.workunit.client.0.smithi119.stdout:5/422: rename d6/db/dc/f26 to d6/db/de/d27/f92 0 2024-05-27T09:29:21.774 INFO:tasks.workunit.client.0.smithi119.stdout:5/423: read - d6/dd/d39/d74/f7a zero size 2024-05-27T09:29:21.785 INFO:tasks.workunit.client.0.smithi119.stdout:5/424: symlink d6/dd/d39/d19/d3c/d52/l93 0 2024-05-27T09:29:21.799 INFO:tasks.workunit.client.0.smithi119.stdout:7/468: dwrite d7/fa [0,4194304] 0 2024-05-27T09:29:21.805 INFO:tasks.workunit.client.0.smithi119.stdout:5/425: write d6/d2f/f42 [2775126,14651] 0 2024-05-27T09:29:21.805 INFO:tasks.workunit.client.0.smithi119.stdout:2/422: dwrite d2/d1a/f39 [0,4194304] 0 2024-05-27T09:29:21.808 INFO:tasks.workunit.client.0.smithi119.stdout:3/553: dwrite db/d36/d45/d4c/d62/d13/d33/fa7 [0,4194304] 0 2024-05-27T09:29:21.808 INFO:tasks.workunit.client.0.smithi119.stdout:7/469: rmdir d7/d5f 39 2024-05-27T09:29:21.813 INFO:tasks.workunit.client.0.smithi119.stdout:6/392: read d0/d2/d57/f6d [1836215,49241] 0 2024-05-27T09:29:21.814 INFO:tasks.workunit.client.0.smithi119.stdout:9/342: dwrite d18/d19/d1b/d40/d44/f54 [0,4194304] 0 2024-05-27T09:29:21.815 INFO:tasks.workunit.client.0.smithi119.stdout:5/426: mkdir d6/db/de/d27/d94 0 2024-05-27T09:29:21.822 INFO:tasks.workunit.client.0.smithi119.stdout:5/427: truncate d6/d2f/f51 1431978 0 2024-05-27T09:29:21.826 INFO:tasks.workunit.client.0.smithi119.stdout:4/386: dwrite d4/d1c/d39/f4b [0,4194304] 0 2024-05-27T09:29:21.826 INFO:tasks.workunit.client.0.smithi119.stdout:5/428: mknod d6/dd/d39/d19/d3c/c95 0 2024-05-27T09:29:21.834 INFO:tasks.workunit.client.0.smithi119.stdout:7/470: write d7/db/f57 [2685914,12072] 0 2024-05-27T09:29:21.837 INFO:tasks.workunit.client.0.smithi119.stdout:1/488: write d7/db/d35/f3e [2530156,3598] 0 2024-05-27T09:29:21.838 INFO:tasks.workunit.client.0.smithi119.stdout:2/423: read d2/da/d15/f80 [382841,30973] 0 2024-05-27T09:29:21.838 INFO:tasks.workunit.client.0.smithi119.stdout:1/489: chown d7/db/d12/d26/f43 1384 1 2024-05-27T09:29:21.845 INFO:tasks.workunit.client.0.smithi119.stdout:7/471: mkdir d7/df/d19/d46/d93 0 2024-05-27T09:29:21.849 INFO:tasks.workunit.client.0.smithi119.stdout:1/490: mknod d7/dd/c9c 0 2024-05-27T09:29:21.853 INFO:tasks.workunit.client.0.smithi119.stdout:1/491: creat d7/dd/d36/f9d x:0 0 0 2024-05-27T09:29:21.853 INFO:tasks.workunit.client.0.smithi119.stdout:1/492: chown d7/dd/d25/d29/d38/d82 202830583 1 2024-05-27T09:29:21.854 INFO:tasks.workunit.client.0.smithi119.stdout:1/493: fsync d7/db/d12/d26/d4a/f81 0 2024-05-27T09:29:21.857 INFO:tasks.workunit.client.0.smithi119.stdout:1/494: mknod d7/db/d12/d26/d39/c9e 0 2024-05-27T09:29:21.857 INFO:tasks.workunit.client.0.smithi119.stdout:1/495: chown d7/db/lc 1 1 2024-05-27T09:29:21.883 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:21 smithi119 ceph-mon[29082]: pgmap v8: 65 pgs: 65 active+clean; 851 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 69 MiB/s rd, 81 MiB/s wr, 410 op/s 2024-05-27T09:29:21.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:21 smithi138 ceph-mon[34885]: pgmap v8: 65 pgs: 65 active+clean; 851 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 69 MiB/s rd, 81 MiB/s wr, 410 op/s 2024-05-27T09:29:21.891 INFO:tasks.workunit.client.0.smithi119.stdout:7/472: dread d7/df/f13 [4194304,4194304] 0 2024-05-27T09:29:21.896 INFO:tasks.workunit.client.0.smithi119.stdout:4/387: dwrite d4/d1c/d39/f5d [0,4194304] 0 2024-05-27T09:29:21.897 INFO:tasks.workunit.client.0.smithi119.stdout:4/388: fdatasync d4/f22 0 2024-05-27T09:29:21.898 INFO:tasks.workunit.client.0.smithi119.stdout:4/389: truncate d4/de/d21/d40/f42 167138 0 2024-05-27T09:29:21.906 INFO:tasks.workunit.client.0.smithi119.stdout:4/390: unlink d4/d6/f11 0 2024-05-27T09:29:21.915 INFO:tasks.workunit.client.0.smithi119.stdout:4/391: dread d4/d6/d28/d2f/d36/f72 [0,4194304] 0 2024-05-27T09:29:21.923 INFO:tasks.workunit.client.0.smithi119.stdout:1/496: dwrite d7/dd/d44/f4e [0,4194304] 0 2024-05-27T09:29:21.923 INFO:tasks.workunit.client.0.smithi119.stdout:1/497: readlink d7/dd/d25/d29/l85 0 2024-05-27T09:29:21.923 INFO:tasks.workunit.client.0.smithi119.stdout:1/498: dread - d7/db/d12/d15/f8e zero size 2024-05-27T09:29:21.926 INFO:tasks.workunit.client.0.smithi119.stdout:1/499: chown d7/f27 13261 1 2024-05-27T09:29:21.926 INFO:tasks.workunit.client.0.smithi119.stdout:1/500: truncate d7/dd/d25/d29/d38/d49/f87 712750 0 2024-05-27T09:29:21.928 INFO:tasks.workunit.client.0.smithi119.stdout:1/501: symlink d7/dd/l9f 0 2024-05-27T09:29:21.929 INFO:tasks.workunit.client.0.smithi119.stdout:1/502: write d7/f3d [1462235,87132] 0 2024-05-27T09:29:21.938 INFO:tasks.workunit.client.0.smithi119.stdout:7/473: dread d7/df/d19/d46/f85 [0,4194304] 0 2024-05-27T09:29:21.938 INFO:tasks.workunit.client.0.smithi119.stdout:7/474: stat d7/d1a/d40/d44/f5d 0 2024-05-27T09:29:21.940 INFO:tasks.workunit.client.0.smithi119.stdout:3/554: dwrite db/d36/d45/d93/f9c [0,4194304] 0 2024-05-27T09:29:21.943 INFO:tasks.workunit.client.0.smithi119.stdout:4/392: dread d4/d23/f26 [4194304,4194304] 0 2024-05-27T09:29:21.944 INFO:tasks.workunit.client.0.smithi119.stdout:4/393: chown d4/de/d21/f2d 9652 1 2024-05-27T09:29:21.949 INFO:tasks.workunit.client.0.smithi119.stdout:7/475: rename d7/db/d37/f55 to d7/df/d19/d4d/d89/f94 0 2024-05-27T09:29:21.951 INFO:tasks.workunit.client.0.smithi119.stdout:3/555: truncate db/d36/f71 3971376 0 2024-05-27T09:29:21.955 INFO:tasks.workunit.client.0.smithi119.stdout:7/476: getdents d7/d1a/d40 0 2024-05-27T09:29:21.957 INFO:tasks.workunit.client.0.smithi119.stdout:7/477: mknod d7/d1a/d40/d44/d67/c95 0 2024-05-27T09:29:21.958 INFO:tasks.workunit.client.0.smithi119.stdout:4/394: rename d4/de/f2e to d4/d6/d28/f7a 0 2024-05-27T09:29:21.958 INFO:tasks.workunit.client.0.smithi119.stdout:4/395: write d4/de/d66/f68 [965825,8800] 0 2024-05-27T09:29:21.958 INFO:tasks.workunit.client.0.smithi119.stdout:4/396: write d4/d48/f5e [638501,63948] 0 2024-05-27T09:29:21.976 INFO:tasks.workunit.client.0.smithi119.stdout:5/429: dwrite d6/d2f/f78 [0,4194304] 0 2024-05-27T09:29:21.976 INFO:tasks.workunit.client.0.smithi119.stdout:8/418: dwrite d3/d8/d4d/d5d/f8a [0,4194304] 0 2024-05-27T09:29:21.976 INFO:tasks.workunit.client.0.smithi119.stdout:5/430: read d6/db/de/d27/f7e [621644,127680] 0 2024-05-27T09:29:22.014 INFO:tasks.workunit.client.0.smithi119.stdout:9/343: dwrite d18/d19/f50 [4194304,4194304] 0 2024-05-27T09:29:22.014 INFO:tasks.workunit.client.0.smithi119.stdout:8/419: read d3/d8/fd [3034451,7037] 0 2024-05-27T09:29:22.015 INFO:tasks.workunit.client.0.smithi119.stdout:8/420: creat d3/d8/d4d/d5d/d60/f8b x:0 0 0 2024-05-27T09:29:22.015 INFO:tasks.workunit.client.0.smithi119.stdout:8/421: write d3/d8/d4d/d58/f63 [206922,52375] 0 2024-05-27T09:29:22.015 INFO:tasks.workunit.client.0.smithi119.stdout:1/503: dwrite d7/dd/d44/f56 [0,4194304] 0 2024-05-27T09:29:22.035 INFO:tasks.workunit.client.0.smithi119.stdout:7/478: dwrite d7/d5f/d79/f8a [0,4194304] 0 2024-05-27T09:29:22.044 INFO:tasks.workunit.client.0.smithi119.stdout:4/397: creat d4/d6/d28/f7b x:0 0 0 2024-05-27T09:29:22.044 INFO:tasks.workunit.client.0.smithi119.stdout:4/398: fsync d4/d6/d28/f6b 0 2024-05-27T09:29:22.044 INFO:tasks.workunit.client.0.smithi119.stdout:4/399: fdatasync d4/d1c/f29 0 2024-05-27T09:29:22.049 INFO:tasks.workunit.client.0.smithi119.stdout:5/431: rmdir d6/dd/d39/d19/d3c 39 2024-05-27T09:29:22.051 INFO:tasks.workunit.client.0.smithi119.stdout:3/556: dwrite db/f2f [0,4194304] 0 2024-05-27T09:29:22.053 INFO:tasks.workunit.client.0.smithi119.stdout:9/344: write d18/f32 [3277398,86690] 0 2024-05-27T09:29:22.055 INFO:tasks.workunit.client.0.smithi119.stdout:1/504: rename d7/db/d52/f57 to d7/db/d35/fa0 0 2024-05-27T09:29:22.055 INFO:tasks.workunit.client.0.smithi119.stdout:3/557: rename db/d36/d45 to db/d36/d45/d4c/d62/d76/daa/db8 22 2024-05-27T09:29:22.055 INFO:tasks.workunit.client.0.smithi119.stdout:3/558: fsync db/d36/d45/d93/fa4 0 2024-05-27T09:29:22.055 INFO:tasks.workunit.client.0.smithi119.stdout:3/559: fdatasync db/d36/d45/d4c/d62/d76/daa/fb7 0 2024-05-27T09:29:22.055 INFO:tasks.workunit.client.0.smithi119.stdout:3/560: fdatasync db/d36/d45/d4c/d62/d13/f2e 0 2024-05-27T09:29:22.059 INFO:tasks.workunit.client.0.smithi119.stdout:5/432: unlink d6/dd/d39/d19/d3c/f87 0 2024-05-27T09:29:22.063 INFO:tasks.workunit.client.0.smithi119.stdout:1/505: creat d7/db/d12/d15/fa1 x:0 0 0 2024-05-27T09:29:22.067 INFO:tasks.workunit.client.0.smithi119.stdout:9/345: symlink d18/d19/l6f 0 2024-05-27T09:29:22.067 INFO:tasks.workunit.client.0.smithi119.stdout:3/561: read - db/d36/d45/d4c/d62/f89 zero size 2024-05-27T09:29:22.069 INFO:tasks.workunit.client.0.smithi119.stdout:1/506: link d7/dd/d25/d60/d61/c89 d7/db/ca2 0 2024-05-27T09:29:22.073 INFO:tasks.workunit.client.0.smithi119.stdout:9/346: unlink d18/d19/d1b/d2a/c6b 0 2024-05-27T09:29:22.073 INFO:tasks.workunit.client.0.smithi119.stdout:9/347: chown d18/d19/d1b/c4b 235605725 1 2024-05-27T09:29:22.073 INFO:tasks.workunit.client.0.smithi119.stdout:9/348: fdatasync d18/d19/d1b/f4c 0 2024-05-27T09:29:22.074 INFO:tasks.workunit.client.0.smithi119.stdout:8/422: dwrite d3/d8/d4d/d5d/d60/f71 [0,4194304] 0 2024-05-27T09:29:22.078 INFO:tasks.workunit.client.0.smithi119.stdout:9/349: creat d18/d19/d1b/d40/d44/f70 x:0 0 0 2024-05-27T09:29:22.078 INFO:tasks.workunit.client.0.smithi119.stdout:9/350: write d18/d19/d1b/d2a/d63/f6c [527418,33464] 0 2024-05-27T09:29:22.078 INFO:tasks.workunit.client.0.smithi119.stdout:9/351: chown d18/f23 0 1 2024-05-27T09:29:22.078 INFO:tasks.workunit.client.0.smithi119.stdout:9/352: creat d18/d19/d1b/d29/f71 x:0 0 0 2024-05-27T09:29:22.085 INFO:tasks.workunit.client.0.smithi119.stdout:8/423: creat d3/f8c x:0 0 0 2024-05-27T09:29:22.085 INFO:tasks.workunit.client.0.smithi119.stdout:8/424: write d3/d39/f3e [1417412,115354] 0 2024-05-27T09:29:22.087 INFO:tasks.workunit.client.0.smithi119.stdout:7/479: dwrite d7/f66 [0,4194304] 0 2024-05-27T09:29:22.088 INFO:tasks.workunit.client.0.smithi119.stdout:8/425: mknod d3/d6e/d80/d84/c8d 0 2024-05-27T09:29:22.092 INFO:tasks.workunit.client.0.smithi119.stdout:7/480: rename d7/df/d19/d78/c90 to d7/d1a/d40/c96 0 2024-05-27T09:29:22.093 INFO:tasks.workunit.client.0.smithi119.stdout:1/507: link d7/dd/d25/d29/d38/d3c/l7f d7/dd/d25/la3 0 2024-05-27T09:29:22.093 INFO:tasks.workunit.client.0.smithi119.stdout:7/481: rmdir d7/df/d36 39 2024-05-27T09:29:22.094 INFO:tasks.workunit.client.0.smithi119.stdout:3/562: dread db/d36/d45/d4d/f70 [0,4194304] 0 2024-05-27T09:29:22.094 INFO:tasks.workunit.client.0.smithi119.stdout:3/563: fsync db/d36/d45/d4c/d62/d76/f79 0 2024-05-27T09:29:22.095 INFO:tasks.workunit.client.0.smithi119.stdout:1/508: rename d7/db/d12/d15/d1f/f34 to d7/db/d52/d65/fa4 0 2024-05-27T09:29:22.097 INFO:tasks.workunit.client.0.smithi119.stdout:4/400: unlink d4/de/l56 0 2024-05-27T09:29:22.099 INFO:tasks.workunit.client.0.smithi119.stdout:1/509: mkdir d7/dd/d25/d60/d61/da5 0 2024-05-27T09:29:22.100 INFO:tasks.workunit.client.0.smithi119.stdout:4/401: write d4/d48/f5e [174812,113451] 0 2024-05-27T09:29:22.102 INFO:tasks.workunit.client.0.smithi119.stdout:4/402: truncate d4/d23/d31/f33 3754159 0 2024-05-27T09:29:22.102 INFO:tasks.workunit.client.0.smithi119.stdout:4/403: chown d4/d1c/d39/l46 1012569123 1 2024-05-27T09:29:22.105 INFO:tasks.workunit.client.0.smithi119.stdout:4/404: creat d4/de/d21/d40/f7c x:0 0 0 2024-05-27T09:29:22.110 INFO:tasks.workunit.client.0.smithi119.stdout:4/405: creat d4/d6/f7d x:0 0 0 2024-05-27T09:29:22.113 INFO:tasks.workunit.client.0.smithi119.stdout:4/406: rename d4/f41 to d4/f7e 0 2024-05-27T09:29:22.115 INFO:tasks.workunit.client.0.smithi119.stdout:9/353: mknod d18/d19/d1b/d40/c72 0 2024-05-27T09:29:22.117 INFO:tasks.workunit.client.0.smithi119.stdout:9/354: mknod d18/d19/d1b/d40/c73 0 2024-05-27T09:29:22.117 INFO:tasks.workunit.client.0.smithi119.stdout:9/355: read - d18/d19/d1b/d40/d44/f70 zero size 2024-05-27T09:29:22.117 INFO:tasks.workunit.client.0.smithi119.stdout:9/356: fdatasync d18/d19/d1b/d40/d44/f45 0 2024-05-27T09:29:22.121 INFO:tasks.workunit.client.0.smithi119.stdout:1/510: dread d7/f11 [0,4194304] 0 2024-05-27T09:29:22.123 INFO:tasks.workunit.client.0.smithi119.stdout:9/357: rename d18/d19/d1b/d29/f3a to d18/d19/f74 0 2024-05-27T09:29:22.127 INFO:tasks.workunit.client.0.smithi119.stdout:9/358: symlink d18/d19/d1b/d2a/d63/l75 0 2024-05-27T09:29:22.127 INFO:tasks.workunit.client.0.smithi119.stdout:9/359: fsync f6 0 2024-05-27T09:29:22.128 INFO:tasks.workunit.client.0.smithi119.stdout:9/360: creat d18/f76 x:0 0 0 2024-05-27T09:29:22.131 INFO:tasks.workunit.client.0.smithi119.stdout:9/361: symlink d18/d19/d61/l77 0 2024-05-27T09:29:22.132 INFO:tasks.workunit.client.0.smithi119.stdout:4/407: read d4/d23/f52 [1127068,71574] 0 2024-05-27T09:29:22.137 INFO:tasks.workunit.client.0.smithi119.stdout:3/564: dwrite db/d36/d45/d4c/d62/d13/d33/fa7 [0,4194304] 0 2024-05-27T09:29:22.138 INFO:tasks.workunit.client.0.smithi119.stdout:3/565: dread - db/d36/d45/d4c/d62/d76/d98/f80 zero size 2024-05-27T09:29:22.139 INFO:tasks.workunit.client.0.smithi119.stdout:3/566: mkdir db/d36/d45/d4c/d62/d76/d98/db9 0 2024-05-27T09:29:22.140 INFO:tasks.workunit.client.0.smithi119.stdout:3/567: symlink db/d36/d45/d4c/d62/d76/d98/lba 0 2024-05-27T09:29:22.149 INFO:tasks.workunit.client.0.smithi119.stdout:9/362: dread d18/d19/d1b/f5d [0,4194304] 0 2024-05-27T09:29:22.152 INFO:tasks.workunit.client.0.smithi119.stdout:3/568: truncate db/d36/d45/d4c/d62/f17 4766570 0 2024-05-27T09:29:22.152 INFO:tasks.workunit.client.0.smithi119.stdout:3/569: write db/d36/d45/d4c/d62/d13/f2a [4192548,82705] 0 2024-05-27T09:29:22.155 INFO:tasks.workunit.client.0.smithi119.stdout:3/570: rmdir db/d36/d45/d49 39 2024-05-27T09:29:22.155 INFO:tasks.workunit.client.0.smithi119.stdout:4/408: dread d4/d1c/f1e [0,4194304] 0 2024-05-27T09:29:22.155 INFO:tasks.workunit.client.0.smithi119.stdout:4/409: write d4/d23/d31/f33 [4740485,75481] 0 2024-05-27T09:29:22.159 INFO:tasks.workunit.client.0.smithi119.stdout:4/410: link d4/d48/c65 d4/de/d21/d40/d47/d5c/c7f 0 2024-05-27T09:29:22.160 INFO:tasks.workunit.client.0.smithi119.stdout:4/411: dread - d4/de/d21/f2b zero size 2024-05-27T09:29:22.162 INFO:tasks.workunit.client.0.smithi119.stdout:9/363: dread d18/d19/f3f [0,4194304] 0 2024-05-27T09:29:22.163 INFO:tasks.workunit.client.0.smithi119.stdout:4/412: read d4/d1c/d39/f64 [1253355,98667] 0 2024-05-27T09:29:22.163 INFO:tasks.workunit.client.0.smithi119.stdout:4/413: chown d4/d6/c18 444441 1 2024-05-27T09:29:22.166 INFO:tasks.workunit.client.0.smithi119.stdout:9/364: link d18/d19/d1b/d40/d44/l68 d18/d19/d1b/d40/d44/l78 0 2024-05-27T09:29:22.167 INFO:tasks.workunit.client.0.smithi119.stdout:9/365: getdents d18/d19/d61 0 2024-05-27T09:29:22.172 INFO:tasks.workunit.client.0.smithi119.stdout:4/414: dread d4/d1c/f29 [0,4194304] 0 2024-05-27T09:29:22.174 INFO:tasks.workunit.client.0.smithi119.stdout:4/415: rmdir d4/de/d21/d40/d47 39 2024-05-27T09:29:22.174 INFO:tasks.workunit.client.0.smithi119.stdout:4/416: stat d4/d1c/d39 0 2024-05-27T09:29:22.174 INFO:tasks.workunit.client.0.smithi119.stdout:4/417: readlink d4/d6/d28/l76 0 2024-05-27T09:29:22.174 INFO:tasks.workunit.client.0.smithi119.stdout:4/418: write d4/d1c/d39/f64 [230077,113973] 0 2024-05-27T09:29:22.211 INFO:tasks.workunit.client.0.smithi119.stdout:8/426: dwrite d3/d39/f72 [0,4194304] 0 2024-05-27T09:29:22.215 INFO:tasks.workunit.client.0.smithi119.stdout:4/419: dwrite d4/de/d21/f2d [0,4194304] 0 2024-05-27T09:29:22.223 INFO:tasks.workunit.client.0.smithi119.stdout:8/427: mkdir d3/d6e/d8e 0 2024-05-27T09:29:22.256 INFO:tasks.workunit.client.0.smithi119.stdout:3/571: dwrite db/d36/d45/d4c/d62/d13/f99 [0,4194304] 0 2024-05-27T09:29:22.263 INFO:tasks.workunit.client.0.smithi119.stdout:3/572: unlink db/l77 0 2024-05-27T09:29:22.263 INFO:tasks.workunit.client.0.smithi119.stdout:3/573: write db/d36/d45/d4c/d62/d76/d98/f73 [1363371,31842] 0 2024-05-27T09:29:22.267 INFO:tasks.workunit.client.0.smithi119.stdout:3/574: creat db/d36/d45/d49/d9d/fbb x:0 0 0 2024-05-27T09:29:22.269 INFO:tasks.workunit.client.0.smithi119.stdout:4/420: dwrite d4/d23/d31/f33 [0,4194304] 0 2024-05-27T09:29:22.271 INFO:tasks.workunit.client.0.smithi119.stdout:4/421: getdents d4/d4e 0 2024-05-27T09:29:22.272 INFO:tasks.workunit.client.0.smithi119.stdout:4/422: rmdir d4/d23/d31 39 2024-05-27T09:29:22.282 INFO:tasks.workunit.client.0.smithi119.stdout:3/575: dread db/d36/d45/d4c/d62/d13/f97 [0,4194304] 0 2024-05-27T09:29:22.282 INFO:tasks.workunit.client.0.smithi119.stdout:9/366: dwrite d18/d19/f36 [0,4194304] 0 2024-05-27T09:29:22.282 INFO:tasks.workunit.client.0.smithi119.stdout:9/367: dread - d18/d19/d1b/d40/d44/f70 zero size 2024-05-27T09:29:22.284 INFO:tasks.workunit.client.0.smithi119.stdout:3/576: creat db/d36/fbc x:0 0 0 2024-05-27T09:29:22.288 INFO:tasks.workunit.client.0.smithi119.stdout:4/423: dread d4/d6/f1a [4194304,4194304] 0 2024-05-27T09:29:22.288 INFO:tasks.workunit.client.0.smithi119.stdout:4/424: mknod d4/d1c/d39/c80 0 2024-05-27T09:29:22.289 INFO:tasks.workunit.client.0.smithi119.stdout:4/425: rmdir d4/d23/d24 39 2024-05-27T09:29:22.290 INFO:tasks.workunit.client.0.smithi119.stdout:4/426: mkdir d4/d6/d81 0 2024-05-27T09:29:22.290 INFO:tasks.workunit.client.0.smithi119.stdout:4/427: chown d4/d4e/l59 314 1 2024-05-27T09:29:22.332 INFO:tasks.workunit.client.0.smithi119.stdout:3/577: dwrite db/d36/d45/d4d/f58 [0,4194304] 0 2024-05-27T09:29:22.335 INFO:tasks.workunit.client.0.smithi119.stdout:3/578: rmdir db/d36/d45/d49 39 2024-05-27T09:29:22.357 INFO:tasks.workunit.client.0.smithi119.stdout:3/579: write db/d36/d45/d4c/d62/d76/d98/d7a/f5f [5113285,122780] 0 2024-05-27T09:29:22.357 INFO:tasks.workunit.client.0.smithi119.stdout:3/580: fdatasync db/d36/d45/d4c/d62/d76/d98/d7a/fac 0 2024-05-27T09:29:22.357 INFO:tasks.workunit.client.0.smithi119.stdout:3/581: chown db/d36/l4a 0 1 2024-05-27T09:29:22.359 INFO:tasks.workunit.client.0.smithi119.stdout:3/582: write db/d36/d45/d4c/d62/f7c [3396068,10378] 0 2024-05-27T09:29:22.359 INFO:tasks.workunit.client.0.smithi119.stdout:3/583: readlink db/d7f/la2 0 2024-05-27T09:29:22.360 INFO:tasks.workunit.client.0.smithi119.stdout:3/584: rmdir db/d36/d45 39 2024-05-27T09:29:22.362 INFO:tasks.workunit.client.0.smithi119.stdout:3/585: truncate db/d36/d45/d49/d4f/f7b 2385331 0 2024-05-27T09:29:22.367 INFO:tasks.workunit.client.0.smithi119.stdout:9/368: dwrite d18/d19/f35 [0,4194304] 0 2024-05-27T09:29:22.367 INFO:tasks.workunit.client.0.smithi119.stdout:9/369: fsync d18/d19/d1b/d40/d44/f69 0 2024-05-27T09:29:22.373 INFO:tasks.workunit.client.0.smithi119.stdout:9/370: symlink d18/d19/d1b/d29/l79 0 2024-05-27T09:29:22.376 INFO:tasks.workunit.client.0.smithi119.stdout:9/371: creat d18/d19/d1b/d2a/f7a x:0 0 0 2024-05-27T09:29:22.379 INFO:tasks.workunit.client.0.smithi119.stdout:9/372: truncate d18/d19/d1b/d40/d44/f45 2333044 0 2024-05-27T09:29:22.379 INFO:tasks.workunit.client.0.smithi119.stdout:9/373: dread - d18/d19/d1b/d40/d44/f70 zero size 2024-05-27T09:29:22.383 INFO:tasks.workunit.client.0.smithi119.stdout:9/374: unlink d18/d19/d1b/d2a/l3e 0 2024-05-27T09:29:22.384 INFO:tasks.workunit.client.0.smithi119.stdout:3/586: write db/d36/d45/d93/f9c [2919110,16975] 0 2024-05-27T09:29:22.385 INFO:tasks.workunit.client.0.smithi119.stdout:4/428: dwrite d4/d1c/f29 [0,4194304] 0 2024-05-27T09:29:22.385 INFO:tasks.workunit.client.0.smithi119.stdout:4/429: chown d4/d48/c65 345004 1 2024-05-27T09:29:22.388 INFO:tasks.workunit.client.0.smithi119.stdout:9/375: mkdir d18/d19/d1b/d40/d7b 0 2024-05-27T09:29:22.390 INFO:tasks.workunit.client.0.smithi119.stdout:3/587: link db/d36/d45/d93/fa4 db/d36/d45/d4c/d62/d76/d98/db9/fbd 0 2024-05-27T09:29:22.396 INFO:tasks.workunit.client.0.smithi119.stdout:9/376: truncate d18/d19/f59 982460 0 2024-05-27T09:29:22.398 INFO:tasks.workunit.client.0.smithi119.stdout:4/430: write d4/d23/d31/f33 [2702618,17436] 0 2024-05-27T09:29:22.405 INFO:tasks.workunit.client.0.smithi119.stdout:4/431: creat d4/d23/d24/f82 x:0 0 0 2024-05-27T09:29:22.416 INFO:tasks.workunit.client.0.smithi119.stdout:4/432: rename d4/d23/d31/l77 to d4/d1c/l83 0 2024-05-27T09:29:22.416 INFO:tasks.workunit.client.0.smithi119.stdout:3/588: read db/fc [3796659,53054] 0 2024-05-27T09:29:22.417 INFO:tasks.workunit.client.0.smithi119.stdout:3/589: truncate db/d36/d45/d4c/d62/d76/d98/f9f 759805 0 2024-05-27T09:29:22.417 INFO:tasks.workunit.client.0.smithi119.stdout:3/590: chown db/fa0 5781960 1 2024-05-27T09:29:22.417 INFO:tasks.workunit.client.0.smithi119.stdout:3/591: chown db/d36/d45/d4c/d62/d76/d98 194522 1 2024-05-27T09:29:22.429 INFO:tasks.workunit.client.0.smithi119.stdout:4/433: read d4/d1c/f1e [2719603,23732] 0 2024-05-27T09:29:22.429 INFO:tasks.workunit.client.0.smithi119.stdout:4/434: truncate d4/d6/f74 195592 0 2024-05-27T09:29:22.429 INFO:tasks.workunit.client.0.smithi119.stdout:4/435: write d4/d1c/f1e [3656375,64449] 0 2024-05-27T09:29:22.438 INFO:tasks.workunit.client.0.smithi119.stdout:4/436: dread d4/de/d66/f68 [0,4194304] 0 2024-05-27T09:29:22.445 INFO:tasks.workunit.client.0.smithi119.stdout:4/437: write d4/f22 [5434238,32248] 0 2024-05-27T09:29:22.447 INFO:tasks.workunit.client.0.smithi119.stdout:4/438: truncate d4/d6/f1a 7095142 0 2024-05-27T09:29:22.447 INFO:tasks.workunit.client.0.smithi119.stdout:4/439: read - d4/d6/d28/f69 zero size 2024-05-27T09:29:22.447 INFO:tasks.workunit.client.0.smithi119.stdout:4/440: chown d4/d1c/f1e 254878279 1 2024-05-27T09:29:22.449 INFO:tasks.workunit.client.0.smithi119.stdout:4/441: getdents d4 0 2024-05-27T09:29:22.466 INFO:tasks.workunit.client.0.smithi119.stdout:4/442: dread f1 [4194304,4194304] 0 2024-05-27T09:29:22.469 INFO:tasks.workunit.client.0.smithi119.stdout:4/443: symlink d4/l84 0 2024-05-27T09:29:22.490 INFO:tasks.workunit.client.0.smithi119.stdout:4/444: dread d4/d6/fd [0,4194304] 0 2024-05-27T09:29:22.496 INFO:tasks.workunit.client.0.smithi119.stdout:4/445: creat d4/de/d21/f85 x:0 0 0 2024-05-27T09:29:22.530 INFO:tasks.workunit.client.0.smithi119.stdout:4/446: dread d4/de/d21/f2d [0,4194304] 0 2024-05-27T09:29:22.530 INFO:tasks.workunit.client.0.smithi119.stdout:4/447: readlink d4/d6/d28/l3d 0 2024-05-27T09:29:22.530 INFO:tasks.workunit.client.0.smithi119.stdout:4/448: creat d4/d6/f86 x:0 0 0 2024-05-27T09:29:22.530 INFO:tasks.workunit.client.0.smithi119.stdout:4/449: truncate d4/d6/f2a 450256 0 2024-05-27T09:29:22.530 INFO:tasks.workunit.client.0.smithi119.stdout:4/450: dread - d4/de/d21/f85 zero size 2024-05-27T09:29:22.534 INFO:tasks.workunit.client.0.smithi119.stdout:4/451: mkdir d4/de/d66/d87 0 2024-05-27T09:29:22.534 INFO:tasks.workunit.client.0.smithi119.stdout:4/452: dread - d4/de/d21/f85 zero size 2024-05-27T09:29:22.540 INFO:tasks.workunit.client.0.smithi119.stdout:4/453: rmdir d4/de/d21/d40 39 2024-05-27T09:29:22.542 INFO:tasks.workunit.client.0.smithi119.stdout:4/454: read d4/de/d21/d40/f42 [10290,62919] 0 2024-05-27T09:29:22.545 INFO:tasks.workunit.client.0.smithi119.stdout:4/455: mkdir d4/de/d66/d87/d88 0 2024-05-27T09:29:22.775 INFO:tasks.workunit.client.0.smithi119.stdout:6/393: sync 2024-05-27T09:29:22.775 INFO:tasks.workunit.client.0.smithi119.stdout:0/427: sync 2024-05-27T09:29:22.779 INFO:tasks.workunit.client.0.smithi119.stdout:6/394: rename d0/d2/d4e/f2e to d0/d2/d1c/d21/d58/f89 0 2024-05-27T09:29:22.780 INFO:tasks.workunit.client.0.smithi119.stdout:6/395: stat d0/d11 0 2024-05-27T09:29:22.783 INFO:tasks.workunit.client.0.smithi119.stdout:0/428: unlink d2/d6/d9/d24/d30/c6e 0 2024-05-27T09:29:22.785 INFO:tasks.workunit.client.0.smithi119.stdout:6/396: creat d0/d85/f8a x:0 0 0 2024-05-27T09:29:22.788 INFO:tasks.workunit.client.0.smithi119.stdout:0/429: unlink d2/d6/d9/d12/f1b 0 2024-05-27T09:29:22.790 INFO:tasks.workunit.client.0.smithi119.stdout:6/397: mknod d0/d11/c8b 0 2024-05-27T09:29:22.790 INFO:tasks.workunit.client.0.smithi119.stdout:6/398: chown d0/d2/d1c/d21/d58/f61 10541312 1 2024-05-27T09:29:22.814 INFO:tasks.workunit.client.0.smithi119.stdout:0/430: write d2/d6/d23/d7a/d53/f5f [2841181,38725] 0 2024-05-27T09:29:22.815 INFO:tasks.workunit.client.0.smithi119.stdout:0/431: chown d2/d6/d9/d24/l57 9434694 1 2024-05-27T09:29:22.817 INFO:tasks.workunit.client.0.smithi119.stdout:0/432: mknod d2/c8e 0 2024-05-27T09:29:22.818 INFO:tasks.workunit.client.0.smithi119.stdout:0/433: creat d2/d6/d23/f8f x:0 0 0 2024-05-27T09:29:22.859 INFO:tasks.workunit.client.1.smithi138.stdout:5/4: rmdir d2 0 2024-05-27T09:29:22.860 INFO:tasks.workunit.client.1.smithi138.stdout:5/5: write f0 [812447,87690] 0 2024-05-27T09:29:22.868 INFO:tasks.workunit.client.0.smithi119.stdout:2/424: sync 2024-05-27T09:29:22.868 INFO:tasks.workunit.client.0.smithi119.stdout:7/482: sync 2024-05-27T09:29:22.868 INFO:tasks.workunit.client.0.smithi119.stdout:5/433: sync 2024-05-27T09:29:22.868 INFO:tasks.workunit.client.0.smithi119.stdout:5/434: truncate d6/dd/f73 255942 0 2024-05-27T09:29:22.868 INFO:tasks.workunit.client.0.smithi119.stdout:2/425: write d2/da/d15/f1b [1315573,62840] 0 2024-05-27T09:29:22.869 INFO:tasks.workunit.client.0.smithi119.stdout:2/426: write d2/d1a/d51/f73 [897377,125788] 0 2024-05-27T09:29:22.869 INFO:tasks.workunit.client.0.smithi119.stdout:2/427: creat d2/f90 x:0 0 0 2024-05-27T09:29:22.874 INFO:tasks.workunit.client.0.smithi119.stdout:2/428: rename d2/da/d8a/c8e to d2/da/d15/d33/d89/c91 0 2024-05-27T09:29:22.875 INFO:tasks.workunit.client.0.smithi119.stdout:5/435: rmdir d6/dd/d39/d19/d3c 39 2024-05-27T09:29:22.875 INFO:tasks.workunit.client.0.smithi119.stdout:5/436: stat d6/db/de/f89 0 2024-05-27T09:29:22.875 INFO:tasks.workunit.client.0.smithi119.stdout:5/437: rename d6/dd/d46 to d6/dd/d46/d47/d96 22 2024-05-27T09:29:22.879 INFO:tasks.workunit.client.1.smithi138.stdout:5/6: dread f0 [0,4194304] 0 2024-05-27T09:29:22.891 INFO:tasks.workunit.client.0.smithi119.stdout:2/429: dread d2/d1a/f1f [0,4194304] 0 2024-05-27T09:29:22.929 INFO:tasks.workunit.client.0.smithi119.stdout:5/438: dwrite d6/f80 [0,4194304] 0 2024-05-27T09:29:22.935 INFO:tasks.workunit.client.0.smithi119.stdout:5/439: rmdir d6/db/dc 39 2024-05-27T09:29:22.935 INFO:tasks.workunit.client.0.smithi119.stdout:5/440: truncate d6/d2f/f3a 472940 0 2024-05-27T09:29:22.937 INFO:tasks.workunit.client.0.smithi119.stdout:5/441: mknod d6/c97 0 2024-05-27T09:29:22.939 INFO:tasks.workunit.client.0.smithi119.stdout:5/442: truncate d6/dd/f1b 495597 0 2024-05-27T09:29:22.941 INFO:tasks.workunit.client.0.smithi119.stdout:2/430: dwrite d2/da/f4d [0,4194304] 0 2024-05-27T09:29:22.946 INFO:tasks.workunit.client.0.smithi119.stdout:2/431: unlink d2/da/d15/c5d 0 2024-05-27T09:29:22.947 INFO:tasks.workunit.client.0.smithi119.stdout:2/432: creat d2/d1a/d51/f92 x:0 0 0 2024-05-27T09:29:22.947 INFO:tasks.workunit.client.0.smithi119.stdout:2/433: chown d2/da/l20 5 1 2024-05-27T09:29:22.948 INFO:tasks.workunit.client.1.smithi138.stdout:5/7: dwrite f0 [0,4194304] 0 2024-05-27T09:29:22.951 INFO:tasks.workunit.client.0.smithi119.stdout:2/434: symlink d2/da/d6f/l93 0 2024-05-27T09:29:22.959 INFO:tasks.workunit.client.0.smithi119.stdout:2/435: rename d2/d1a/d51 to d2/da/d15/d33/d89/d94 0 2024-05-27T09:29:22.959 INFO:tasks.workunit.client.0.smithi119.stdout:2/436: readlink d2/da/d15/d33/d89/d94/l7a 0 2024-05-27T09:29:22.959 INFO:tasks.workunit.client.0.smithi119.stdout:2/437: stat d2/da/d6f 0 2024-05-27T09:29:22.960 INFO:tasks.workunit.client.0.smithi119.stdout:2/438: truncate d2/f48 4180179 0 2024-05-27T09:29:22.967 INFO:tasks.workunit.client.1.smithi138.stdout:5/8: write f1 [2459695,55345] 0 2024-05-27T09:29:22.967 INFO:tasks.workunit.client.0.smithi119.stdout:5/443: dread d6/db/de/d27/f50 [0,4194304] 0 2024-05-27T09:29:22.967 INFO:tasks.workunit.client.0.smithi119.stdout:5/444: fsync d6/f1d 0 2024-05-27T09:29:22.970 INFO:tasks.workunit.client.0.smithi119.stdout:5/445: creat d6/dd/df/d41/f98 x:0 0 0 2024-05-27T09:29:22.970 INFO:tasks.workunit.client.0.smithi119.stdout:5/446: chown d6/d2f/f78 1990 1 2024-05-27T09:29:22.970 INFO:tasks.workunit.client.0.smithi119.stdout:5/447: fdatasync d6/db/de/d27/f2a 0 2024-05-27T09:29:22.971 INFO:tasks.workunit.client.0.smithi119.stdout:5/448: unlink d6/db/de/d27/f2a 0 2024-05-27T09:29:22.987 INFO:tasks.workunit.client.0.smithi119.stdout:1/511: sync 2024-05-27T09:29:22.987 INFO:tasks.workunit.client.0.smithi119.stdout:1/512: write d7/db/d12/f96 [1265066,48455] 0 2024-05-27T09:29:22.987 INFO:tasks.workunit.client.0.smithi119.stdout:1/513: readlink d7/dd/d25/d60/l78 0 2024-05-27T09:29:22.987 INFO:tasks.workunit.client.0.smithi119.stdout:4/456: sync 2024-05-27T09:29:22.987 INFO:tasks.workunit.client.0.smithi119.stdout:8/428: sync 2024-05-27T09:29:22.987 INFO:tasks.workunit.client.0.smithi119.stdout:8/429: stat d3/d8/d1b/f55 0 2024-05-27T09:29:22.987 INFO:tasks.workunit.client.0.smithi119.stdout:1/514: getdents d7/db/d12/d26/d39 0 2024-05-27T09:29:22.991 INFO:tasks.workunit.client.0.smithi119.stdout:1/515: symlink d7/dd/d25/d29/la6 0 2024-05-27T09:29:22.994 INFO:tasks.workunit.client.0.smithi119.stdout:1/516: unlink f2 0 2024-05-27T09:29:23.006 INFO:tasks.workunit.client.0.smithi119.stdout:8/430: dread d3/d8/d4d/d58/f82 [0,4194304] 0 2024-05-27T09:29:23.006 INFO:tasks.workunit.client.0.smithi119.stdout:8/431: chown d3/d8/f7e 2 1 2024-05-27T09:29:23.006 INFO:tasks.workunit.client.0.smithi119.stdout:8/432: creat d3/d8/d4d/d58/f8f x:0 0 0 2024-05-27T09:29:23.007 INFO:tasks.workunit.client.0.smithi119.stdout:8/433: fdatasync d3/d8/d4d/d58/f8f 0 2024-05-27T09:29:23.009 INFO:tasks.workunit.client.0.smithi119.stdout:1/517: dread d7/db/d12/d26/f43 [0,4194304] 0 2024-05-27T09:29:23.009 INFO:tasks.workunit.client.0.smithi119.stdout:1/518: fdatasync d7/db/d12/d15/fa1 0 2024-05-27T09:29:23.009 INFO:tasks.workunit.client.0.smithi119.stdout:1/519: stat d7/db/d12/d15/f1d 0 2024-05-27T09:29:23.010 INFO:tasks.workunit.client.0.smithi119.stdout:8/434: mkdir d3/d6e/d80/d84/d90 0 2024-05-27T09:29:23.021 INFO:tasks.workunit.client.1.smithi138.stdout:4/2: getdents . 0 2024-05-27T09:29:23.026 INFO:tasks.workunit.client.0.smithi119.stdout:4/457: dread d4/d6/f67 [0,4194304] 0 2024-05-27T09:29:23.026 INFO:tasks.workunit.client.0.smithi119.stdout:4/458: write d4/de/d21/d40/f7c [848276,371] 0 2024-05-27T09:29:23.026 INFO:tasks.workunit.client.0.smithi119.stdout:2/439: dwrite d2/da/d2c/f55 [0,4194304] 0 2024-05-27T09:29:23.027 INFO:tasks.workunit.client.0.smithi119.stdout:2/440: dread - d2/da/d15/d33/d89/d94/f8b zero size 2024-05-27T09:29:23.027 INFO:tasks.workunit.client.0.smithi119.stdout:2/441: fsync d2/da/d15/f18 0 2024-05-27T09:29:23.027 INFO:tasks.workunit.client.0.smithi119.stdout:2/442: dread - d2/f90 zero size 2024-05-27T09:29:23.029 INFO:tasks.workunit.client.0.smithi119.stdout:4/459: mkdir d4/d6/d3e/d89 0 2024-05-27T09:29:23.029 INFO:tasks.workunit.client.0.smithi119.stdout:2/443: creat d2/da/f95 x:0 0 0 2024-05-27T09:29:23.030 INFO:tasks.workunit.client.0.smithi119.stdout:2/444: chown d2/da/d15/d33/d69/f6b 2 1 2024-05-27T09:29:23.037 INFO:tasks.workunit.client.0.smithi119.stdout:4/460: rename d4/d1c/d39/c80 to d4/d6/d28/d4a/c8a 0 2024-05-27T09:29:23.037 INFO:tasks.workunit.client.0.smithi119.stdout:2/445: getdents d2 0 2024-05-27T09:29:23.037 INFO:tasks.workunit.client.0.smithi119.stdout:2/446: fdatasync d2/d1a/f3d 0 2024-05-27T09:29:23.038 INFO:tasks.workunit.client.0.smithi119.stdout:2/447: creat d2/da/d15/d33/d89/d94/d81/f96 x:0 0 0 2024-05-27T09:29:23.038 INFO:tasks.workunit.client.0.smithi119.stdout:2/448: dread - d2/da/d15/d33/d89/f8d zero size 2024-05-27T09:29:23.038 INFO:tasks.workunit.client.1.smithi138.stdout:9/10: mknod c5 0 2024-05-27T09:29:23.038 INFO:tasks.workunit.client.1.smithi138.stdout:9/11: chown f4 54 1 2024-05-27T09:29:23.040 INFO:tasks.workunit.client.0.smithi119.stdout:8/435: read d3/d8/d4d/f59 [1296015,67027] 0 2024-05-27T09:29:23.041 INFO:tasks.workunit.client.0.smithi119.stdout:8/436: write d3/d8/d4d/f59 [1920415,103565] 0 2024-05-27T09:29:23.048 INFO:tasks.workunit.client.0.smithi119.stdout:2/449: rmdir d2/da/d15/d33/d89/d94/d81 39 2024-05-27T09:29:23.055 INFO:tasks.workunit.client.0.smithi119.stdout:2/450: truncate d2/f47 2797930 0 2024-05-27T09:29:23.058 INFO:tasks.workunit.client.0.smithi119.stdout:2/451: link d2/da/d6f/c88 d2/da/d15/d33/c97 0 2024-05-27T09:29:23.059 INFO:tasks.workunit.client.0.smithi119.stdout:2/452: read d2/da/d15/d33/f60 [3483230,16699] 0 2024-05-27T09:29:23.059 INFO:tasks.workunit.client.0.smithi119.stdout:2/453: read - d2/da/d2c/f58 zero size 2024-05-27T09:29:23.059 INFO:tasks.workunit.client.0.smithi119.stdout:2/454: chown d2/c2a 888392570 1 2024-05-27T09:29:23.063 INFO:tasks.workunit.client.1.smithi138.stdout:1/35: symlink d4/l9 0 2024-05-27T09:29:23.066 INFO:tasks.workunit.client.0.smithi119.stdout:5/449: dwrite d6/d2f/f78 [0,4194304] 0 2024-05-27T09:29:23.067 INFO:tasks.workunit.client.0.smithi119.stdout:5/450: write d6/db/f4c [233858,76462] 0 2024-05-27T09:29:23.069 INFO:tasks.workunit.client.0.smithi119.stdout:2/455: dread d2/d1a/f3b [0,4194304] 0 2024-05-27T09:29:23.069 INFO:tasks.workunit.client.0.smithi119.stdout:2/456: stat d2/da/f1c 0 2024-05-27T09:29:23.074 INFO:tasks.workunit.client.0.smithi119.stdout:5/451: creat d6/dd/d39/d19/d3c/d52/f99 x:0 0 0 2024-05-27T09:29:23.074 INFO:tasks.workunit.client.0.smithi119.stdout:2/457: getdents d2/da/d6f 0 2024-05-27T09:29:23.074 INFO:tasks.workunit.client.0.smithi119.stdout:2/458: fdatasync d2/da/d15/d33/d89/d94/f68 0 2024-05-27T09:29:23.074 INFO:tasks.workunit.client.0.smithi119.stdout:2/459: fsync d2/d1a/f61 0 2024-05-27T09:29:23.076 INFO:tasks.workunit.client.1.smithi138.stdout:4/3: dread f0 [0,4194304] 0 2024-05-27T09:29:23.076 INFO:tasks.workunit.client.1.smithi138.stdout:4/4: rmdir - no directory 2024-05-27T09:29:23.078 INFO:tasks.workunit.client.0.smithi119.stdout:5/452: mkdir d6/db/d9a 0 2024-05-27T09:29:23.078 INFO:tasks.workunit.client.0.smithi119.stdout:5/453: creat d6/dd/f9b x:0 0 0 2024-05-27T09:29:23.080 INFO:tasks.workunit.client.0.smithi119.stdout:5/454: fdatasync d6/dd/df/f5c 0 2024-05-27T09:29:23.089 INFO:tasks.workunit.client.0.smithi119.stdout:5/455: mkdir d6/dd/d39/d19/d3c/d52/d9c 0 2024-05-27T09:29:23.089 INFO:tasks.workunit.client.0.smithi119.stdout:4/461: dread d4/d6/fa [0,4194304] 0 2024-05-27T09:29:23.089 INFO:tasks.workunit.client.0.smithi119.stdout:9/377: sync 2024-05-27T09:29:23.089 INFO:tasks.workunit.client.0.smithi119.stdout:3/592: sync 2024-05-27T09:29:23.089 INFO:tasks.workunit.client.0.smithi119.stdout:3/593: write db/d36/d45/d49/f85 [345801,65176] 0 2024-05-27T09:29:23.094 INFO:tasks.workunit.client.0.smithi119.stdout:9/378: dread - d18/d19/d1b/d29/f71 zero size 2024-05-27T09:29:23.094 INFO:tasks.workunit.client.0.smithi119.stdout:9/379: dread - d18/d19/d1b/d40/f60 zero size 2024-05-27T09:29:23.108 INFO:tasks.workunit.client.1.smithi138.stdout:2/18: rename d1/d3 to d1/d7 0 2024-05-27T09:29:23.111 INFO:tasks.workunit.client.1.smithi138.stdout:9/12: dwrite f3 [0,4194304] 0 2024-05-27T09:29:23.115 INFO:tasks.workunit.client.0.smithi119.stdout:3/594: dread db/d36/d45/d4c/d62/d76/d98/fa5 [0,4194304] 0 2024-05-27T09:29:23.117 INFO:tasks.workunit.client.0.smithi119.stdout:3/595: mkdir db/d36/d45/d4c/dbe 0 2024-05-27T09:29:23.117 INFO:tasks.workunit.client.0.smithi119.stdout:3/596: readlink db/d7f/l86 0 2024-05-27T09:29:23.117 INFO:tasks.workunit.client.0.smithi119.stdout:3/597: truncate db/d36/d45/d49/f85 414737 0 2024-05-27T09:29:23.118 INFO:tasks.workunit.client.0.smithi119.stdout:3/598: chown db/d36/d45/d4c/d62/c32 191306619 1 2024-05-27T09:29:23.118 INFO:tasks.workunit.client.0.smithi119.stdout:3/599: truncate db/fb1 388397 0 2024-05-27T09:29:23.129 INFO:tasks.workunit.client.0.smithi119.stdout:8/437: dread d3/d8/d1b/f28 [0,4194304] 0 2024-05-27T09:29:23.135 INFO:tasks.workunit.client.1.smithi138.stdout:4/5: dwrite f0 [4194304,4194304] 0 2024-05-27T09:29:23.146 INFO:tasks.workunit.client.0.smithi119.stdout:5/456: dread d6/dd/f64 [0,4194304] 0 2024-05-27T09:29:23.148 INFO:tasks.workunit.client.1.smithi138.stdout:9/13: write f3 [1358595,53460] 0 2024-05-27T09:29:23.148 INFO:tasks.workunit.client.1.smithi138.stdout:9/14: truncate f1 578700 0 2024-05-27T09:29:23.148 INFO:tasks.workunit.client.1.smithi138.stdout:9/15: truncate f4 377010 0 2024-05-27T09:29:23.149 INFO:tasks.workunit.client.0.smithi119.stdout:1/520: dwrite d7/f5b [4194304,4194304] 0 2024-05-27T09:29:23.149 INFO:tasks.workunit.client.0.smithi119.stdout:1/521: stat d7/dd/d25/d29/d38 0 2024-05-27T09:29:23.150 INFO:tasks.workunit.client.0.smithi119.stdout:5/457: mkdir d6/d2f/d9d 0 2024-05-27T09:29:23.155 INFO:tasks.workunit.client.0.smithi119.stdout:5/458: getdents d6/dd/d46 0 2024-05-27T09:29:23.157 INFO:tasks.workunit.client.0.smithi119.stdout:2/460: dwrite d2/da/fd [0,4194304] 0 2024-05-27T09:29:23.161 INFO:tasks.workunit.client.0.smithi119.stdout:2/461: fsync d2/da/d15/f16 0 2024-05-27T09:29:23.161 INFO:tasks.workunit.client.0.smithi119.stdout:5/459: creat d6/db/d9a/f9e x:0 0 0 2024-05-27T09:29:23.163 INFO:tasks.workunit.client.0.smithi119.stdout:5/460: write d6/dd/df/d41/f98 [376801,31028] 0 2024-05-27T09:29:23.177 INFO:tasks.workunit.client.0.smithi119.stdout:1/522: write d7/dd/d25/d29/d38/d49/f87 [626098,3891] 0 2024-05-27T09:29:23.180 INFO:tasks.workunit.client.0.smithi119.stdout:7/483: sync 2024-05-27T09:29:23.181 INFO:tasks.workunit.client.0.smithi119.stdout:4/462: dwrite d4/d6/d28/d2f/f50 [0,4194304] 0 2024-05-27T09:29:23.188 INFO:tasks.workunit.client.0.smithi119.stdout:6/399: sync 2024-05-27T09:29:23.188 INFO:tasks.workunit.client.0.smithi119.stdout:0/434: sync 2024-05-27T09:29:23.188 INFO:tasks.workunit.client.0.smithi119.stdout:4/463: mknod d4/d6/d3e/c8b 0 2024-05-27T09:29:23.188 INFO:tasks.workunit.client.0.smithi119.stdout:0/435: mknod d2/d6/d9/d24/d30/c90 0 2024-05-27T09:29:23.192 INFO:tasks.workunit.client.0.smithi119.stdout:5/461: dread d6/db/de/d27/f6b [0,4194304] 0 2024-05-27T09:29:23.192 INFO:tasks.workunit.client.0.smithi119.stdout:5/462: chown d6/dd/d46/d47/d4b 125691474 1 2024-05-27T09:29:23.192 INFO:tasks.workunit.client.0.smithi119.stdout:5/463: creat d6/db/de/d27/d77/f9f x:0 0 0 2024-05-27T09:29:23.192 INFO:tasks.workunit.client.0.smithi119.stdout:4/464: unlink d4/d23/d31/f53 0 2024-05-27T09:29:23.195 INFO:tasks.workunit.client.0.smithi119.stdout:5/464: rmdir d6/db/dc 39 2024-05-27T09:29:23.196 INFO:tasks.workunit.client.0.smithi119.stdout:4/465: symlink d4/d6/d81/l8c 0 2024-05-27T09:29:23.198 INFO:tasks.workunit.client.0.smithi119.stdout:5/465: rename d6/db/de/d27/f92 to d6/dd/d39/d74/fa0 0 2024-05-27T09:29:23.198 INFO:tasks.workunit.client.0.smithi119.stdout:5/466: write d6/dd/d39/d19/d3c/f43 [820866,69497] 0 2024-05-27T09:29:23.199 INFO:tasks.workunit.client.0.smithi119.stdout:5/467: getdents d6/dd/d39/d19/d3c 0 2024-05-27T09:29:23.200 INFO:tasks.workunit.client.0.smithi119.stdout:5/468: creat d6/dd/d39/d74/fa1 x:0 0 0 2024-05-27T09:29:23.200 INFO:tasks.workunit.client.0.smithi119.stdout:5/469: chown d6/db/de/d27/f76 1795 1 2024-05-27T09:29:23.205 INFO:tasks.workunit.client.0.smithi119.stdout:6/400: dread d0/d2/f4 [0,4194304] 0 2024-05-27T09:29:23.205 INFO:tasks.workunit.client.0.smithi119.stdout:6/401: dread - d0/d19/f69 zero size 2024-05-27T09:29:23.210 INFO:tasks.workunit.client.0.smithi119.stdout:8/438: dwrite d3/d6e/f75 [0,4194304] 0 2024-05-27T09:29:23.211 INFO:tasks.workunit.client.0.smithi119.stdout:8/439: rmdir d3/d8 39 2024-05-27T09:29:23.213 INFO:tasks.workunit.client.0.smithi119.stdout:8/440: mknod d3/d8/d4d/d5d/d66/d85/c91 0 2024-05-27T09:29:23.216 INFO:tasks.workunit.client.0.smithi119.stdout:4/466: dread d4/de/d66/f68 [0,4194304] 0 2024-05-27T09:29:23.216 INFO:tasks.workunit.client.0.smithi119.stdout:4/467: read - d4/d6/d3e/f6e zero size 2024-05-27T09:29:23.217 INFO:tasks.workunit.client.0.smithi119.stdout:8/441: unlink d3/d8/d1b/d46/f37 0 2024-05-27T09:29:23.218 INFO:tasks.workunit.client.0.smithi119.stdout:8/442: chown d3/d8/d4d/d5d/d60 35835 1 2024-05-27T09:29:23.218 INFO:tasks.workunit.client.0.smithi119.stdout:8/443: write d3/d8/d4d/f32 [6485142,42041] 0 2024-05-27T09:29:23.218 INFO:tasks.workunit.client.0.smithi119.stdout:8/444: chown d3/d6e/d80/d84/c8d 110 1 2024-05-27T09:29:23.218 INFO:tasks.workunit.client.0.smithi119.stdout:8/445: chown d3/de/f1d 5702105 1 2024-05-27T09:29:23.218 INFO:tasks.workunit.client.0.smithi119.stdout:8/446: readlink d3/d8/d1b/d77/l7b 0 2024-05-27T09:29:23.218 INFO:tasks.workunit.client.0.smithi119.stdout:8/447: truncate d3/d8/d1b/f41 2104379 0 2024-05-27T09:29:23.219 INFO:tasks.workunit.client.0.smithi119.stdout:8/448: mkdir d3/d8/d1b/d46/d92 0 2024-05-27T09:29:23.220 INFO:tasks.workunit.client.0.smithi119.stdout:8/449: chown d3/d8/d4d/d5d/d66 127439163 1 2024-05-27T09:29:23.221 INFO:tasks.workunit.client.0.smithi119.stdout:8/450: write d3/f8c [769928,97698] 0 2024-05-27T09:29:23.236 INFO:tasks.workunit.client.0.smithi119.stdout:2/462: dwrite d2/da/d6f/f70 [0,4194304] 0 2024-05-27T09:29:23.243 INFO:tasks.workunit.client.0.smithi119.stdout:2/463: creat d2/da/d2c/f98 x:0 0 0 2024-05-27T09:29:23.243 INFO:tasks.workunit.client.0.smithi119.stdout:2/464: chown d2/da/d15/d53/c5c 983422 1 2024-05-27T09:29:23.267 INFO:tasks.workunit.client.0.smithi119.stdout:0/436: dwrite d2/d6/d9/f6d [0,4194304] 0 2024-05-27T09:29:23.281 INFO:tasks.workunit.client.0.smithi119.stdout:2/465: mkdir d2/da/d15/d33/d89/d94/d99 0 2024-05-27T09:29:23.282 INFO:tasks.workunit.client.0.smithi119.stdout:7/484: dwrite d7/d1a/f59 [0,4194304] 0 2024-05-27T09:29:23.284 INFO:tasks.workunit.client.0.smithi119.stdout:5/470: dwrite d6/f63 [0,4194304] 0 2024-05-27T09:29:23.287 INFO:tasks.workunit.client.0.smithi119.stdout:0/437: mknod d2/d6/d9/d12/c91 0 2024-05-27T09:29:23.287 INFO:tasks.workunit.client.0.smithi119.stdout:0/438: write d2/d6/d9/f66 [692724,105283] 0 2024-05-27T09:29:23.287 INFO:tasks.workunit.client.0.smithi119.stdout:0/439: chown d2/d6/d9/d2f/c72 270 1 2024-05-27T09:29:23.288 INFO:tasks.workunit.client.0.smithi119.stdout:1/523: dwrite d7/dd/d25/f86 [0,4194304] 0 2024-05-27T09:29:23.291 INFO:tasks.workunit.client.0.smithi119.stdout:2/466: creat d2/da/d15/d33/d89/d94/d87/f9a x:0 0 0 2024-05-27T09:29:23.291 INFO:tasks.workunit.client.0.smithi119.stdout:2/467: dread - d2/da/d15/d33/d89/d94/d81/f96 zero size 2024-05-27T09:29:23.299 INFO:tasks.workunit.client.0.smithi119.stdout:7/485: rename d7/db to d7/df/d19/d97 0 2024-05-27T09:29:23.299 INFO:tasks.workunit.client.0.smithi119.stdout:7/486: truncate d7/df/d36/f61 633867 0 2024-05-27T09:29:23.299 INFO:tasks.workunit.client.0.smithi119.stdout:7/487: fdatasync d7/d1a/f2d 0 2024-05-27T09:29:23.302 INFO:tasks.workunit.client.0.smithi119.stdout:5/471: link d6/dd/d39/c30 d6/dd/d39/d19/ca2 0 2024-05-27T09:29:23.305 INFO:tasks.workunit.client.0.smithi119.stdout:0/440: mknod d2/d6/d9/c92 0 2024-05-27T09:29:23.307 INFO:tasks.workunit.client.0.smithi119.stdout:6/402: dwrite d0/d2/d1c/f3e [0,4194304] 0 2024-05-27T09:29:23.312 INFO:tasks.workunit.client.0.smithi119.stdout:1/524: getdents d7/dd 0 2024-05-27T09:29:23.312 INFO:tasks.workunit.client.0.smithi119.stdout:1/525: chown d7/dd/d44/d47 30 1 2024-05-27T09:29:23.313 INFO:tasks.workunit.client.0.smithi119.stdout:1/526: read - d7/db/d12/d26/d4a/f81 zero size 2024-05-27T09:29:23.313 INFO:tasks.workunit.client.0.smithi119.stdout:1/527: write d7/f3d [814997,55854] 0 2024-05-27T09:29:23.314 INFO:tasks.workunit.client.0.smithi119.stdout:7/488: symlink d7/d5f/d79/l98 0 2024-05-27T09:29:23.314 INFO:tasks.workunit.client.0.smithi119.stdout:7/489: truncate d7/d6a/f6f 311121 0 2024-05-27T09:29:23.317 INFO:tasks.workunit.client.0.smithi119.stdout:8/451: dwrite d3/d8/d1b/f6c [0,4194304] 0 2024-05-27T09:29:23.317 INFO:tasks.workunit.client.0.smithi119.stdout:8/452: getdents d3/d8/d4d/d5d/d74 0 2024-05-27T09:29:23.317 INFO:tasks.workunit.client.0.smithi119.stdout:8/453: write d3/d8/d4d/f67 [212285,122092] 0 2024-05-27T09:29:23.318 INFO:tasks.workunit.client.0.smithi119.stdout:5/472: symlink d6/dd/d39/la3 0 2024-05-27T09:29:23.320 INFO:tasks.workunit.client.0.smithi119.stdout:0/441: rename d2/d6/d9/d2f/c76 to d2/d6/d23/c93 0 2024-05-27T09:29:23.335 INFO:tasks.workunit.client.0.smithi119.stdout:4/468: dwrite d4/d1c/f29 [0,4194304] 0 2024-05-27T09:29:23.335 INFO:tasks.workunit.client.0.smithi119.stdout:4/469: truncate d4/d23/d24/d5b/f78 601141 0 2024-05-27T09:29:23.340 INFO:tasks.workunit.client.0.smithi119.stdout:5/473: rename d6/db/dc/f7b to d6/dd/d6e/fa4 0 2024-05-27T09:29:23.343 INFO:tasks.workunit.client.0.smithi119.stdout:4/470: link d4/d6/l3c d4/de/d66/d87/d88/l8d 0 2024-05-27T09:29:23.345 INFO:tasks.workunit.client.0.smithi119.stdout:7/490: rmdir d7/d6a 39 2024-05-27T09:29:23.346 INFO:tasks.workunit.client.0.smithi119.stdout:5/474: mkdir d6/d2f/da5 0 2024-05-27T09:29:23.352 INFO:tasks.workunit.client.0.smithi119.stdout:4/471: link d4/d6/c7 d4/d1c/c8e 0 2024-05-27T09:29:23.352 INFO:tasks.workunit.client.0.smithi119.stdout:5/475: mknod d6/dd/d39/d19/d7d/ca6 0 2024-05-27T09:29:23.353 INFO:tasks.workunit.client.0.smithi119.stdout:1/528: dread d7/f3d [4194304,4194304] 0 2024-05-27T09:29:23.357 INFO:tasks.workunit.client.0.smithi119.stdout:4/472: unlink d4/d6/l15 0 2024-05-27T09:29:23.366 INFO:tasks.workunit.client.0.smithi119.stdout:6/403: dwrite d0/d2/d1c/f3e [0,4194304] 0 2024-05-27T09:29:23.370 INFO:tasks.workunit.client.0.smithi119.stdout:5/476: fdatasync d6/dd/df/f75 0 2024-05-27T09:29:23.371 INFO:tasks.workunit.client.0.smithi119.stdout:4/473: rmdir d4/d60 0 2024-05-27T09:29:23.371 INFO:tasks.workunit.client.0.smithi119.stdout:4/474: chown d4/d6/d28/f7a 427362924 1 2024-05-27T09:29:23.371 INFO:tasks.workunit.client.0.smithi119.stdout:1/529: unlink d7/db/f6c 0 2024-05-27T09:29:23.371 INFO:tasks.workunit.client.0.smithi119.stdout:5/477: link d6/dd/d46/d47/d4b/f58 d6/d2f/da5/fa7 0 2024-05-27T09:29:23.372 INFO:tasks.workunit.client.0.smithi119.stdout:5/478: write d6/d2f/f51 [2251402,128335] 0 2024-05-27T09:29:23.373 INFO:tasks.workunit.client.0.smithi119.stdout:4/475: rename d4/d6/f67 to d4/f8f 0 2024-05-27T09:29:23.375 INFO:tasks.workunit.client.0.smithi119.stdout:1/530: creat d7/dd/d44/fa7 x:0 0 0 2024-05-27T09:29:23.377 INFO:tasks.workunit.client.0.smithi119.stdout:4/476: rename d4/f32 to d4/de/d21/d40/d47/f90 0 2024-05-27T09:29:23.380 INFO:tasks.workunit.client.0.smithi119.stdout:1/531: unlink d7/dd/l9f 0 2024-05-27T09:29:23.380 INFO:tasks.workunit.client.0.smithi119.stdout:1/532: fsync d7/db/d12/d26/f97 0 2024-05-27T09:29:23.381 INFO:tasks.workunit.client.0.smithi119.stdout:1/533: link d7/db/d12/d26/d39/c98 d7/dd/d25/ca8 0 2024-05-27T09:29:23.383 INFO:tasks.workunit.client.0.smithi119.stdout:6/404: dread d0/d2/f4 [0,4194304] 0 2024-05-27T09:29:23.385 INFO:tasks.workunit.client.0.smithi119.stdout:6/405: rmdir d0/d2/d1c/d21 39 2024-05-27T09:29:23.397 INFO:tasks.workunit.client.0.smithi119.stdout:7/491: dwrite d7/df/d19/f14 [0,4194304] 0 2024-05-27T09:29:23.405 INFO:tasks.workunit.client.0.smithi119.stdout:4/477: dread d4/d6/f3b [0,4194304] 0 2024-05-27T09:29:23.407 INFO:tasks.workunit.client.0.smithi119.stdout:4/478: unlink d4/d6/f1a 0 2024-05-27T09:29:23.421 INFO:tasks.workunit.client.0.smithi119.stdout:4/479: rmdir d4/d1c/d39 39 2024-05-27T09:29:23.423 INFO:tasks.workunit.client.0.smithi119.stdout:4/480: mknod d4/de/d66/d87/d88/c91 0 2024-05-27T09:29:23.423 INFO:tasks.workunit.client.0.smithi119.stdout:4/481: dread - d4/d6/d3e/f6e zero size 2024-05-27T09:29:23.430 INFO:tasks.workunit.client.0.smithi119.stdout:1/534: write d7/db/d12/d15/d1f/f8b [460898,76973] 0 2024-05-27T09:29:23.430 INFO:tasks.workunit.client.0.smithi119.stdout:5/479: dwrite d6/dd/df/f75 [4194304,4194304] 0 2024-05-27T09:29:23.437 INFO:tasks.workunit.client.0.smithi119.stdout:4/482: dread d4/de/d66/f68 [0,4194304] 0 2024-05-27T09:29:23.437 INFO:tasks.workunit.client.0.smithi119.stdout:4/483: fdatasync d4/d23/d24/f82 0 2024-05-27T09:29:23.441 INFO:tasks.workunit.client.0.smithi119.stdout:4/484: truncate d4/d6/f3b 3358651 0 2024-05-27T09:29:23.441 INFO:tasks.workunit.client.0.smithi119.stdout:4/485: dread - d4/d6/f7d zero size 2024-05-27T09:29:23.445 INFO:tasks.workunit.client.0.smithi119.stdout:6/406: dwrite d0/d2/d1c/d21/f2b [0,4194304] 0 2024-05-27T09:29:23.446 INFO:tasks.workunit.client.0.smithi119.stdout:6/407: symlink d0/d19/l8c 0 2024-05-27T09:29:23.454 INFO:tasks.workunit.client.0.smithi119.stdout:6/408: dread d0/d2/f4 [0,4194304] 0 2024-05-27T09:29:23.458 INFO:tasks.workunit.client.0.smithi119.stdout:4/486: dread d4/d6/d28/f7a [0,4194304] 0 2024-05-27T09:29:23.473 INFO:tasks.workunit.client.0.smithi119.stdout:6/409: write d0/d2/d57/d37/f5d [3082911,85686] 0 2024-05-27T09:29:23.484 INFO:tasks.workunit.client.0.smithi119.stdout:6/410: unlink d0/fd 0 2024-05-27T09:29:23.509 INFO:tasks.workunit.client.0.smithi119.stdout:7/492: dwrite d7/d1a/d40/f86 [0,4194304] 0 2024-05-27T09:29:23.538 INFO:tasks.workunit.client.0.smithi119.stdout:1/535: dwrite d7/db/d12/d15/f1d [0,4194304] 0 2024-05-27T09:29:23.538 INFO:tasks.workunit.client.0.smithi119.stdout:1/536: creat d7/dd/d44/fa9 x:0 0 0 2024-05-27T09:29:23.544 INFO:tasks.workunit.client.0.smithi119.stdout:1/537: mknod d7/dd/d25/d29/d38/d49/caa 0 2024-05-27T09:29:23.544 INFO:tasks.workunit.client.0.smithi119.stdout:1/538: fdatasync d7/dd/d25/d29/d38/f92 0 2024-05-27T09:29:23.544 INFO:tasks.workunit.client.0.smithi119.stdout:1/539: dread - d7/dd/d36/f9d zero size 2024-05-27T09:29:23.544 INFO:tasks.workunit.client.0.smithi119.stdout:6/411: dwrite d0/d2/d4e/f4b [4194304,4194304] 0 2024-05-27T09:29:23.544 INFO:tasks.workunit.client.0.smithi119.stdout:6/412: stat d0/d2/d57/d37/d74/f70 0 2024-05-27T09:29:23.558 INFO:tasks.workunit.client.0.smithi119.stdout:6/413: dread d0/d2/d4e/f6a [0,4194304] 0 2024-05-27T09:29:23.558 INFO:tasks.workunit.client.0.smithi119.stdout:6/414: dread - d0/d2/f7f zero size 2024-05-27T09:29:23.559 INFO:tasks.workunit.client.0.smithi119.stdout:7/493: dwrite d7/df/d19/f1f [0,4194304] 0 2024-05-27T09:29:23.561 INFO:tasks.workunit.client.0.smithi119.stdout:6/415: rename d0/f6 to d0/d2/d57/d37/f8d 0 2024-05-27T09:29:23.563 INFO:tasks.workunit.client.0.smithi119.stdout:7/494: getdents d7/d1a/d40 0 2024-05-27T09:29:23.563 INFO:tasks.workunit.client.0.smithi119.stdout:7/495: stat d7/d1a/d40/d44/f5d 0 2024-05-27T09:29:23.564 INFO:tasks.workunit.client.0.smithi119.stdout:4/487: dwrite d4/de/d66/f68 [0,4194304] 0 2024-05-27T09:29:23.567 INFO:tasks.workunit.client.0.smithi119.stdout:7/496: mkdir d7/d1a/d99 0 2024-05-27T09:29:23.605 INFO:tasks.workunit.client.0.smithi119.stdout:1/540: dwrite d7/db/d12/f6b [0,4194304] 0 2024-05-27T09:29:23.605 INFO:tasks.workunit.client.0.smithi119.stdout:1/541: dread - d7/dd/d44/f93 zero size 2024-05-27T09:29:23.607 INFO:tasks.workunit.client.0.smithi119.stdout:1/542: stat d7/db/d52/l73 0 2024-05-27T09:29:23.632 INFO:tasks.workunit.client.0.smithi119.stdout:0/442: sync 2024-05-27T09:29:23.632 INFO:tasks.workunit.client.0.smithi119.stdout:2/468: sync 2024-05-27T09:29:23.632 INFO:tasks.workunit.client.0.smithi119.stdout:3/600: sync 2024-05-27T09:29:23.632 INFO:tasks.workunit.client.0.smithi119.stdout:0/443: write d2/d6/d9/f2c [2528642,16518] 0 2024-05-27T09:29:23.635 INFO:tasks.workunit.client.0.smithi119.stdout:6/416: dwrite d0/d2/d4e/f1f [0,4194304] 0 2024-05-27T09:29:23.639 INFO:tasks.workunit.client.0.smithi119.stdout:3/601: rename db/d36/d45/d49/d4f/f60 to db/fbf 0 2024-05-27T09:29:23.639 INFO:tasks.workunit.client.0.smithi119.stdout:3/602: readlink db/d7f/da9/lb4 0 2024-05-27T09:29:23.639 INFO:tasks.workunit.client.0.smithi119.stdout:0/444: rmdir d2/d6/d9/d24/d30/d6b 39 2024-05-27T09:29:23.641 INFO:tasks.workunit.client.0.smithi119.stdout:6/417: mknod d0/d2/d1c/c8e 0 2024-05-27T09:29:23.643 INFO:tasks.workunit.client.0.smithi119.stdout:3/603: unlink db/d36/d45/d93/f9c 0 2024-05-27T09:29:23.643 INFO:tasks.workunit.client.0.smithi119.stdout:3/604: readlink db/l11 0 2024-05-27T09:29:23.650 INFO:tasks.workunit.client.0.smithi119.stdout:6/418: mkdir d0/d8f 0 2024-05-27T09:29:23.652 INFO:tasks.workunit.client.0.smithi119.stdout:6/419: creat d0/d2/d57/f90 x:0 0 0 2024-05-27T09:29:23.652 INFO:tasks.workunit.client.0.smithi119.stdout:6/420: dread - d0/f5f zero size 2024-05-27T09:29:23.652 INFO:tasks.workunit.client.0.smithi119.stdout:9/380: sync 2024-05-27T09:29:23.653 INFO:tasks.workunit.client.0.smithi119.stdout:9/381: creat d18/d19/d1b/d29/f7c x:0 0 0 2024-05-27T09:29:23.655 INFO:tasks.workunit.client.0.smithi119.stdout:4/488: dwrite d4/de/d21/d40/f6a [0,4194304] 0 2024-05-27T09:29:23.661 INFO:tasks.workunit.client.0.smithi119.stdout:1/543: dwrite d7/dd/d25/f86 [0,4194304] 0 2024-05-27T09:29:23.661 INFO:tasks.workunit.client.0.smithi119.stdout:7/497: dwrite d7/df/d19/f1f [0,4194304] 0 2024-05-27T09:29:23.661 INFO:tasks.workunit.client.0.smithi119.stdout:1/544: write d7/db/d35/f3e [3491651,108163] 0 2024-05-27T09:29:23.663 INFO:tasks.workunit.client.0.smithi119.stdout:7/498: rmdir d7/d6a 39 2024-05-27T09:29:23.665 INFO:tasks.workunit.client.0.smithi119.stdout:7/499: mkdir d7/df/d19/d9a 0 2024-05-27T09:29:23.665 INFO:tasks.workunit.client.0.smithi119.stdout:7/500: dread - d7/df/d36/d42/d76/f82 zero size 2024-05-27T09:29:23.667 INFO:tasks.workunit.client.0.smithi119.stdout:1/545: mknod d7/d40/cab 0 2024-05-27T09:29:23.667 INFO:tasks.workunit.client.0.smithi119.stdout:1/546: truncate d7/db/d12/f9a 972306 0 2024-05-27T09:29:23.676 INFO:tasks.workunit.client.0.smithi119.stdout:0/445: write d2/d6/d23/d7a/f42 [1610902,32687] 0 2024-05-27T09:29:23.676 INFO:tasks.workunit.client.0.smithi119.stdout:7/501: creat d7/df/d19/d4d/d89/f9b x:0 0 0 2024-05-27T09:29:23.676 INFO:tasks.workunit.client.0.smithi119.stdout:7/502: chown d7/df/d19/d1d/l2e 3786 1 2024-05-27T09:29:23.676 INFO:tasks.workunit.client.0.smithi119.stdout:1/547: truncate d7/f27 276664 0 2024-05-27T09:29:23.677 INFO:tasks.workunit.client.0.smithi119.stdout:6/421: getdents d0/d2/d57 0 2024-05-27T09:29:23.677 INFO:tasks.workunit.client.0.smithi119.stdout:7/503: truncate d7/df/d19/f14 318445 0 2024-05-27T09:29:23.677 INFO:tasks.workunit.client.0.smithi119.stdout:6/422: read d0/d2/d1c/f2a [4360368,40005] 0 2024-05-27T09:29:23.677 INFO:tasks.workunit.client.0.smithi119.stdout:7/504: readlink d7/d5f/d79/l98 0 2024-05-27T09:29:23.680 INFO:tasks.workunit.client.0.smithi119.stdout:6/423: getdents d0/d2/d57/d37/d74 0 2024-05-27T09:29:23.684 INFO:tasks.workunit.client.0.smithi119.stdout:9/382: dread d18/d19/f49 [0,4194304] 0 2024-05-27T09:29:23.684 INFO:tasks.workunit.client.0.smithi119.stdout:7/505: mknod d7/df/d19/d4d/d89/c9c 0 2024-05-27T09:29:23.684 INFO:tasks.workunit.client.0.smithi119.stdout:7/506: creat d7/df/d36/d42/d76/f9d x:0 0 0 2024-05-27T09:29:23.684 INFO:tasks.workunit.client.0.smithi119.stdout:7/507: stat d7/d6a/f6f 0 2024-05-27T09:29:23.684 INFO:tasks.workunit.client.0.smithi119.stdout:7/508: chown d7/df/d19/d8d 122598 1 2024-05-27T09:29:23.684 INFO:tasks.workunit.client.0.smithi119.stdout:7/509: truncate d7/df/d36/d42/f73 2878278 0 2024-05-27T09:29:23.684 INFO:tasks.workunit.client.0.smithi119.stdout:7/510: chown d7/df/d19/d4d/l91 184238 1 2024-05-27T09:29:23.692 INFO:tasks.workunit.client.0.smithi119.stdout:1/548: getdents d7/dd 0 2024-05-27T09:29:23.718 INFO:tasks.workunit.client.0.smithi119.stdout:5/480: sync 2024-05-27T09:29:23.718 INFO:tasks.workunit.client.0.smithi119.stdout:5/481: dread - d6/dd/d39/d74/fa1 zero size 2024-05-27T09:29:23.718 INFO:tasks.workunit.client.0.smithi119.stdout:8/454: sync 2024-05-27T09:29:23.718 INFO:tasks.workunit.client.0.smithi119.stdout:5/482: getdents d6/dd/d46/d47 0 2024-05-27T09:29:23.723 INFO:tasks.workunit.client.0.smithi119.stdout:5/483: symlink d6/db/de/d27/la8 0 2024-05-27T09:29:23.723 INFO:tasks.workunit.client.0.smithi119.stdout:5/484: fsync d6/db/de/d27/f50 0 2024-05-27T09:29:23.729 INFO:tasks.workunit.client.0.smithi119.stdout:6/424: dread d0/d2/d57/f50 [0,4194304] 0 2024-05-27T09:29:23.729 INFO:tasks.workunit.client.0.smithi119.stdout:6/425: read d0/d2/d4e/f88 [424516,17672] 0 2024-05-27T09:29:23.735 INFO:tasks.workunit.client.0.smithi119.stdout:5/485: rmdir d6 39 2024-05-27T09:29:23.735 INFO:tasks.workunit.client.0.smithi119.stdout:5/486: fsync d6/db/f71 0 2024-05-27T09:29:23.737 INFO:tasks.workunit.client.0.smithi119.stdout:5/487: creat d6/db/de/d27/d94/fa9 x:0 0 0 2024-05-27T09:29:23.743 INFO:tasks.workunit.client.0.smithi119.stdout:0/446: dwrite d2/d6/d9/d12/d4b/d59/f58 [0,4194304] 0 2024-05-27T09:29:23.743 INFO:tasks.workunit.client.0.smithi119.stdout:0/447: fsync d2/d6/f8 0 2024-05-27T09:29:23.743 INFO:tasks.workunit.client.0.smithi119.stdout:7/511: write d7/df/d19/d46/f85 [957272,106098] 0 2024-05-27T09:29:23.748 INFO:tasks.workunit.client.0.smithi119.stdout:5/488: dread d6/db/f4c [0,4194304] 0 2024-05-27T09:29:23.749 INFO:tasks.workunit.client.0.smithi119.stdout:9/383: dwrite d18/d19/d1b/f5d [4194304,4194304] 0 2024-05-27T09:29:23.749 INFO:tasks.workunit.client.0.smithi119.stdout:7/512: symlink d7/df/d36/d42/l9e 0 2024-05-27T09:29:23.750 INFO:tasks.workunit.client.0.smithi119.stdout:5/489: symlink d6/dd/d39/d19/d7d/laa 0 2024-05-27T09:29:23.750 INFO:tasks.workunit.client.0.smithi119.stdout:5/490: read - d6/db/d9a/f9e zero size 2024-05-27T09:29:23.754 INFO:tasks.workunit.client.0.smithi119.stdout:7/513: unlink d7/d1a/d40/d44/d67/c95 0 2024-05-27T09:29:23.754 INFO:tasks.workunit.client.0.smithi119.stdout:9/384: rmdir d18/d19/d1b/d2a/d63 39 2024-05-27T09:29:23.758 INFO:tasks.workunit.client.0.smithi119.stdout:7/514: stat d7/df/d19/l17 0 2024-05-27T09:29:23.763 INFO:tasks.workunit.client.0.smithi119.stdout:9/385: rename d18/d19/c2d to d18/d19/d1b/d2a/c7d 0 2024-05-27T09:29:23.765 INFO:tasks.workunit.client.0.smithi119.stdout:9/386: creat d18/d19/d1b/d40/d7b/f7e x:0 0 0 2024-05-27T09:29:23.766 INFO:tasks.workunit.client.0.smithi119.stdout:9/387: mknod d18/c7f 0 2024-05-27T09:29:23.768 INFO:tasks.workunit.client.0.smithi119.stdout:4/489: write d4/f8f [1925639,65430] 0 2024-05-27T09:29:23.769 INFO:tasks.workunit.client.0.smithi119.stdout:9/388: rename d18/d19/f49 to d18/f80 0 2024-05-27T09:29:23.777 INFO:tasks.workunit.client.0.smithi119.stdout:3/605: dread db/d36/d45/d4c/d62/d13/d33/fa7 [0,4194304] 0 2024-05-27T09:29:23.777 INFO:tasks.workunit.client.0.smithi119.stdout:3/606: chown db/d7f/l86 161304 1 2024-05-27T09:29:23.777 INFO:tasks.workunit.client.0.smithi119.stdout:4/490: truncate d4/d6/f3b 2421441 0 2024-05-27T09:29:23.777 INFO:tasks.workunit.client.0.smithi119.stdout:2/469: dwrite d2/da/d15/d33/f41 [0,4194304] 0 2024-05-27T09:29:23.777 INFO:tasks.workunit.client.0.smithi119.stdout:2/470: truncate d2/da/ff 1451387 0 2024-05-27T09:29:23.777 INFO:tasks.workunit.client.0.smithi119.stdout:4/491: mkdir d4/de/d21/d40/d47/d92 0 2024-05-27T09:29:23.782 INFO:tasks.workunit.client.0.smithi119.stdout:2/471: getdents d2/da/d15/d33/d89/d94 0 2024-05-27T09:29:23.782 INFO:tasks.workunit.client.0.smithi119.stdout:2/472: fsync d2/da/d6f/f7f 0 2024-05-27T09:29:23.782 INFO:tasks.workunit.client.0.smithi119.stdout:1/549: dwrite d7/dd/d44/f93 [0,4194304] 0 2024-05-27T09:29:23.782 INFO:tasks.workunit.client.0.smithi119.stdout:4/492: creat d4/d23/d31/f93 x:0 0 0 2024-05-27T09:29:23.784 INFO:tasks.workunit.client.0.smithi119.stdout:2/473: link d2/da/d15/f1b d2/d1a/f9b 0 2024-05-27T09:29:23.785 INFO:tasks.workunit.client.0.smithi119.stdout:1/550: creat d7/db/d52/fac x:0 0 0 2024-05-27T09:29:23.785 INFO:tasks.workunit.client.0.smithi119.stdout:1/551: fsync d7/f11 0 2024-05-27T09:29:23.786 INFO:tasks.workunit.client.0.smithi119.stdout:2/474: mkdir d2/da/d15/d33/d89/d94/d9c 0 2024-05-27T09:29:23.787 INFO:tasks.workunit.client.0.smithi119.stdout:1/552: mknod d7/dd/d25/d29/d38/d3c/cad 0 2024-05-27T09:29:23.788 INFO:tasks.workunit.client.0.smithi119.stdout:2/475: creat d2/da/d15/d33/d69/f9d x:0 0 0 2024-05-27T09:29:23.790 INFO:tasks.workunit.client.0.smithi119.stdout:1/553: getdents d7/dd/d25/d60/d61 0 2024-05-27T09:29:23.799 INFO:tasks.workunit.client.0.smithi119.stdout:6/426: dread d0/d2/d1c/d21/d58/f89 [0,4194304] 0 2024-05-27T09:29:23.802 INFO:tasks.workunit.client.0.smithi119.stdout:3/607: dread db/d36/d45/d4c/d62/d13/f3d [0,4194304] 0 2024-05-27T09:29:23.802 INFO:tasks.workunit.client.0.smithi119.stdout:2/476: creat d2/da/d15/d33/d89/d94/d81/f9e x:0 0 0 2024-05-27T09:29:23.804 INFO:tasks.workunit.client.0.smithi119.stdout:2/477: truncate d2/d1a/f61 847366 0 2024-05-27T09:29:23.804 INFO:tasks.workunit.client.0.smithi119.stdout:2/478: dread - d2/da/d15/d33/d89/d94/f92 zero size 2024-05-27T09:29:23.804 INFO:tasks.workunit.client.0.smithi119.stdout:0/448: read d2/d6/d9/d12/f79 [2213337,125042] 0 2024-05-27T09:29:23.805 INFO:tasks.workunit.client.0.smithi119.stdout:6/427: creat d0/d2/d4e/d5a/f91 x:0 0 0 2024-05-27T09:29:23.805 INFO:tasks.workunit.client.0.smithi119.stdout:6/428: readlink d0/d2/d1c/l25 0 2024-05-27T09:29:23.809 INFO:tasks.workunit.client.0.smithi119.stdout:9/389: dread fa [0,4194304] 0 2024-05-27T09:29:23.810 INFO:tasks.workunit.client.0.smithi119.stdout:9/390: symlink d18/l81 0 2024-05-27T09:29:23.811 INFO:tasks.workunit.client.0.smithi119.stdout:3/608: symlink db/d36/d45/d4d/lc0 0 2024-05-27T09:29:23.811 INFO:tasks.workunit.client.0.smithi119.stdout:4/493: write d4/d1c/f29 [170182,63488] 0 2024-05-27T09:29:23.814 INFO:tasks.workunit.client.0.smithi119.stdout:9/391: link d18/l67 d18/d19/d61/l82 0 2024-05-27T09:29:23.814 INFO:tasks.workunit.client.0.smithi119.stdout:9/392: read - d18/d19/d1b/d29/f57 zero size 2024-05-27T09:29:23.814 INFO:tasks.workunit.client.0.smithi119.stdout:3/609: creat db/d36/d45/fc1 x:0 0 0 2024-05-27T09:29:23.816 INFO:tasks.workunit.client.0.smithi119.stdout:4/494: creat d4/d6/d28/d2f/d36/f94 x:0 0 0 2024-05-27T09:29:23.816 INFO:tasks.workunit.client.0.smithi119.stdout:4/495: truncate d4/d6/d28/d2f/d36/f72 1121623 0 2024-05-27T09:29:23.816 INFO:tasks.workunit.client.0.smithi119.stdout:4/496: readlink d4/d1c/l3f 0 2024-05-27T09:29:23.817 INFO:tasks.workunit.client.0.smithi119.stdout:3/610: getdents db/d7f 0 2024-05-27T09:29:23.826 INFO:tasks.workunit.client.0.smithi119.stdout:8/455: dwrite d3/d8/d1b/d46/f7c [0,4194304] 0 2024-05-27T09:29:23.826 INFO:tasks.workunit.client.0.smithi119.stdout:8/456: write d3/d8/d4d/f59 [4293142,38565] 0 2024-05-27T09:29:23.826 INFO:tasks.workunit.client.0.smithi119.stdout:8/457: truncate d3/de/f2c 1115896 0 2024-05-27T09:29:23.829 INFO:tasks.workunit.client.0.smithi119.stdout:3/611: mknod db/d36/d45/d4c/d62/d13/d33/cc2 0 2024-05-27T09:29:23.829 INFO:tasks.workunit.client.0.smithi119.stdout:4/497: creat d4/de/d66/d87/d88/f95 x:0 0 0 2024-05-27T09:29:23.829 INFO:tasks.workunit.client.0.smithi119.stdout:3/612: dread - db/d36/d45/d49/d9d/fbb zero size 2024-05-27T09:29:23.834 INFO:tasks.workunit.client.0.smithi119.stdout:6/429: dread d0/d2/d4e/f4b [4194304,4194304] 0 2024-05-27T09:29:23.834 INFO:tasks.workunit.client.0.smithi119.stdout:7/515: dwrite d7/df/d36/d42/d64/f70 [0,4194304] 0 2024-05-27T09:29:23.835 INFO:tasks.workunit.client.0.smithi119.stdout:8/458: symlink d3/d39/d5a/l93 0 2024-05-27T09:29:23.842 INFO:tasks.workunit.client.0.smithi119.stdout:4/498: rename d4/d1c/c8e to d4/d6/d28/c96 0 2024-05-27T09:29:23.843 INFO:tasks.workunit.client.0.smithi119.stdout:3/613: link db/d36/d45/d4c/d62/d13/f99 db/d36/fc3 0 2024-05-27T09:29:23.843 INFO:tasks.workunit.client.0.smithi119.stdout:3/614: truncate db/d36/f71 4110695 0 2024-05-27T09:29:23.846 INFO:tasks.workunit.client.0.smithi119.stdout:6/430: dread d0/d2/f30 [0,4194304] 0 2024-05-27T09:29:23.846 INFO:tasks.workunit.client.0.smithi119.stdout:6/431: write d0/d2/d57/d37/f63 [867146,10440] 0 2024-05-27T09:29:23.846 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:23 smithi138 ceph-mon[34885]: pgmap v9: 65 pgs: 65 active+clean; 1.2 GiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 104 MiB/s rd, 127 MiB/s wr, 627 op/s 2024-05-27T09:29:23.854 INFO:tasks.workunit.client.0.smithi119.stdout:8/459: creat d3/d8/d4d/d5d/d66/f94 x:0 0 0 2024-05-27T09:29:23.862 INFO:tasks.workunit.client.0.smithi119.stdout:1/554: dread d7/f1e [0,4194304] 0 2024-05-27T09:29:23.872 INFO:tasks.workunit.client.0.smithi119.stdout:8/460: mkdir d3/d8/d1b/d77/d95 0 2024-05-27T09:29:23.873 INFO:tasks.workunit.client.0.smithi119.stdout:7/516: rename d7/df/d36/d42/d64/f70 to d7/df/d19/d97/f9f 0 2024-05-27T09:29:23.880 INFO:tasks.workunit.client.0.smithi119.stdout:1/555: link d7/db/d52/d65/c7a d7/dd/d25/d29/d38/d49/cae 0 2024-05-27T09:29:23.880 INFO:tasks.workunit.client.0.smithi119.stdout:1/556: chown d7/db/d35/f8c 3 1 2024-05-27T09:29:23.880 INFO:tasks.workunit.client.0.smithi119.stdout:1/557: fsync d7/db/d35/f8c 0 2024-05-27T09:29:23.880 INFO:tasks.workunit.client.0.smithi119.stdout:1/558: write d7/db/d12/d15/d1f/f8b [359019,72259] 0 2024-05-27T09:29:23.882 INFO:tasks.workunit.client.0.smithi119.stdout:8/461: mknod d3/d8/d1b/c96 0 2024-05-27T09:29:23.882 INFO:tasks.workunit.client.0.smithi119.stdout:3/615: dread db/fc [0,4194304] 0 2024-05-27T09:29:23.882 INFO:tasks.workunit.client.0.smithi119.stdout:0/449: dwrite d2/d6/d23/d7a/f42 [0,4194304] 0 2024-05-27T09:29:23.885 INFO:tasks.workunit.client.0.smithi119.stdout:7/517: link d7/d5f/d79/l98 d7/d6a/la0 0 2024-05-27T09:29:23.885 INFO:tasks.workunit.client.0.smithi119.stdout:7/518: chown d7/d1a/f59 25116272 1 2024-05-27T09:29:23.886 INFO:tasks.workunit.client.0.smithi119.stdout:8/462: link d3/d8/d1b/d46/f81 d3/d8/d4d/d58/f97 0 2024-05-27T09:29:23.886 INFO:tasks.workunit.client.0.smithi119.stdout:0/450: creat d2/f94 x:0 0 0 2024-05-27T09:29:23.886 INFO:tasks.workunit.client.0.smithi119.stdout:0/451: dread - d2/d6/d9/f85 zero size 2024-05-27T09:29:23.886 INFO:tasks.workunit.client.0.smithi119.stdout:0/452: chown d2/d1a/d20/f64 4 1 2024-05-27T09:29:23.887 INFO:tasks.workunit.client.0.smithi119.stdout:7/519: rename d7/df/d36/d42/f60 to d7/d1a/fa1 0 2024-05-27T09:29:23.889 INFO:tasks.workunit.client.0.smithi119.stdout:7/520: rename d7/d1a/d40/d44/d67/l74 to d7/d6a/d7b/la2 0 2024-05-27T09:29:23.889 INFO:tasks.workunit.client.0.smithi119.stdout:7/521: fsync f1 0 2024-05-27T09:29:23.892 INFO:tasks.workunit.client.0.smithi119.stdout:0/453: stat d2/d6/d23/d7a/ca 0 2024-05-27T09:29:23.893 INFO:tasks.workunit.client.0.smithi119.stdout:7/522: rmdir d7/df/d19/d4d/d89 39 2024-05-27T09:29:23.895 INFO:tasks.workunit.client.0.smithi119.stdout:0/454: getdents d2/d6/d9/d24 0 2024-05-27T09:29:23.895 INFO:tasks.workunit.client.0.smithi119.stdout:0/455: chown d2/d6/d9/d12/d4b/d59/f7d 1667721991 1 2024-05-27T09:29:23.895 INFO:tasks.workunit.client.0.smithi119.stdout:0/456: write d2/d6/d9/d12/d4f/f89 [513293,8320] 0 2024-05-27T09:29:23.895 INFO:tasks.workunit.client.0.smithi119.stdout:7/523: truncate d7/d6a/f6f 133243 0 2024-05-27T09:29:23.896 INFO:tasks.workunit.client.0.smithi119.stdout:7/524: write d7/df/d19/d4d/d89/f94 [288642,122483] 0 2024-05-27T09:29:23.897 INFO:tasks.workunit.client.0.smithi119.stdout:7/525: dread - d7/df/d36/d42/d76/f82 zero size 2024-05-27T09:29:23.902 INFO:tasks.workunit.client.0.smithi119.stdout:4/499: dwrite d4/d48/f5e [0,4194304] 0 2024-05-27T09:29:23.903 INFO:tasks.workunit.client.0.smithi119.stdout:8/463: dread d3/d8/d4d/d58/f70 [0,4194304] 0 2024-05-27T09:29:23.903 INFO:tasks.workunit.client.0.smithi119.stdout:8/464: chown d3/d8/lc 525390689 1 2024-05-27T09:29:23.903 INFO:tasks.workunit.client.0.smithi119.stdout:8/465: chown d3/d8/d4d/d5d/d66/d85 6623572 1 2024-05-27T09:29:23.909 INFO:tasks.workunit.client.0.smithi119.stdout:4/500: symlink d4/d1c/l97 0 2024-05-27T09:29:23.909 INFO:tasks.workunit.client.0.smithi119.stdout:4/501: stat d4/de/d21/d40/d47/f90 0 2024-05-27T09:29:23.914 INFO:tasks.workunit.client.0.smithi119.stdout:8/466: getdents d3/d6e 0 2024-05-27T09:29:23.914 INFO:tasks.workunit.client.0.smithi119.stdout:2/479: dwrite d2/f2d [0,4194304] 0 2024-05-27T09:29:23.914 INFO:tasks.workunit.client.0.smithi119.stdout:4/502: symlink d4/d6/d3e/d89/l98 0 2024-05-27T09:29:23.914 INFO:tasks.workunit.client.0.smithi119.stdout:0/457: truncate d2/d6/d23/d7a/f42 3934075 0 2024-05-27T09:29:23.918 INFO:tasks.workunit.client.0.smithi119.stdout:1/559: read d7/f11 [585095,116095] 0 2024-05-27T09:29:23.918 INFO:tasks.workunit.client.0.smithi119.stdout:1/560: write d7/f14 [434270,13890] 0 2024-05-27T09:29:23.926 INFO:tasks.workunit.client.0.smithi119.stdout:2/480: creat d2/da/d8a/f9f x:0 0 0 2024-05-27T09:29:23.927 INFO:tasks.workunit.client.0.smithi119.stdout:4/503: getdents d4/d6/d3e/d89 0 2024-05-27T09:29:23.927 INFO:tasks.workunit.client.0.smithi119.stdout:2/481: write d2/da/d15/f5f [625762,32255] 0 2024-05-27T09:29:23.927 INFO:tasks.workunit.client.0.smithi119.stdout:4/504: write d4/d23/d31/f33 [5619288,7088] 0 2024-05-27T09:29:23.927 INFO:tasks.workunit.client.0.smithi119.stdout:2/482: write d2/d1a/f46 [675723,33352] 0 2024-05-27T09:29:23.927 INFO:tasks.workunit.client.0.smithi119.stdout:0/458: truncate d2/d6/d9/d24/d30/d4a/f4c 1844646 0 2024-05-27T09:29:23.930 INFO:tasks.workunit.client.0.smithi119.stdout:1/561: rename d7/db/d35/f8c to d7/dd/d44/faf 0 2024-05-27T09:29:23.930 INFO:tasks.workunit.client.0.smithi119.stdout:1/562: stat d7/db/d12/d15/d1f/l8a 0 2024-05-27T09:29:23.932 INFO:tasks.workunit.client.0.smithi119.stdout:2/483: creat d2/fa0 x:0 0 0 2024-05-27T09:29:23.932 INFO:tasks.workunit.client.0.smithi119.stdout:2/484: chown d2/da/d15/d33/c97 449 1 2024-05-27T09:29:23.933 INFO:tasks.workunit.client.0.smithi119.stdout:0/459: mkdir d2/d6/d9/d2f/d95 0 2024-05-27T09:29:23.934 INFO:tasks.workunit.client.0.smithi119.stdout:1/563: stat d7/db/d12/d26/l83 0 2024-05-27T09:29:23.937 INFO:tasks.workunit.client.0.smithi119.stdout:4/505: link d4/d6/d28/f7a d4/d23/d24/f99 0 2024-05-27T09:29:23.939 INFO:tasks.workunit.client.0.smithi119.stdout:2/485: symlink d2/da/d4e/la1 0 2024-05-27T09:29:23.939 INFO:tasks.workunit.client.0.smithi119.stdout:2/486: truncate d2/d1a/f30 1099131 0 2024-05-27T09:29:23.939 INFO:tasks.workunit.client.0.smithi119.stdout:2/487: creat d2/da/d15/d33/d89/d94/d81/fa2 x:0 0 0 2024-05-27T09:29:23.939 INFO:tasks.workunit.client.0.smithi119.stdout:2/488: truncate d2/da/d15/d33/d89/d94/d87/f9a 796533 0 2024-05-27T09:29:23.941 INFO:tasks.workunit.client.0.smithi119.stdout:4/506: mknod d4/de/d21/d40/d47/d92/c9a 0 2024-05-27T09:29:23.941 INFO:tasks.workunit.client.0.smithi119.stdout:4/507: stat d4/d1c/d39/f64 0 2024-05-27T09:29:23.941 INFO:tasks.workunit.client.0.smithi119.stdout:2/489: truncate d2/da/f4d 2862118 0 2024-05-27T09:29:23.944 INFO:tasks.workunit.client.0.smithi119.stdout:4/508: rename d4/d6/d28/d4a to d4/de/d9b 0 2024-05-27T09:29:23.948 INFO:tasks.workunit.client.0.smithi119.stdout:2/490: creat d2/da/d15/d33/d89/fa3 x:0 0 0 2024-05-27T09:29:23.949 INFO:tasks.workunit.client.0.smithi119.stdout:2/491: mknod d2/da/d15/ca4 0 2024-05-27T09:29:23.950 INFO:tasks.workunit.client.0.smithi119.stdout:2/492: fsync d2/da/d15/d33/d89/d94/d81/f96 0 2024-05-27T09:29:23.953 INFO:tasks.workunit.client.0.smithi119.stdout:2/493: creat d2/da/d6f/fa5 x:0 0 0 2024-05-27T09:29:23.956 INFO:tasks.workunit.client.0.smithi119.stdout:2/494: unlink d2/da/d15/d33/f41 0 2024-05-27T09:29:23.956 INFO:tasks.workunit.client.0.smithi119.stdout:2/495: chown d2/d1a/l7c 559 1 2024-05-27T09:29:23.956 INFO:tasks.workunit.client.0.smithi119.stdout:2/496: chown d2/da/d15/d33/d89/d94/c6a 40 1 2024-05-27T09:29:23.958 INFO:tasks.workunit.client.0.smithi119.stdout:2/497: unlink d2/da/d4e/f6e 0 2024-05-27T09:29:23.958 INFO:tasks.workunit.client.0.smithi119.stdout:2/498: readlink d2/da/d15/d53/l82 0 2024-05-27T09:29:23.965 INFO:tasks.workunit.client.0.smithi119.stdout:4/509: dread d4/d1c/f1e [0,4194304] 0 2024-05-27T09:29:23.970 INFO:tasks.workunit.client.0.smithi119.stdout:4/510: symlink d4/d6/d81/l9c 0 2024-05-27T09:29:23.970 INFO:tasks.workunit.client.0.smithi119.stdout:6/432: dwrite d0/d2/d1c/f2a [0,4194304] 0 2024-05-27T09:29:23.972 INFO:tasks.workunit.client.0.smithi119.stdout:7/526: dread d7/df/f10 [0,4194304] 0 2024-05-27T09:29:23.974 INFO:tasks.workunit.client.0.smithi119.stdout:3/616: dread db/d36/d45/d4c/d62/d76/d98/d7a/f9e [0,4194304] 0 2024-05-27T09:29:23.974 INFO:tasks.workunit.client.0.smithi119.stdout:3/617: fsync db/fb1 0 2024-05-27T09:29:23.979 INFO:tasks.workunit.client.0.smithi119.stdout:6/433: mknod d0/d19/c92 0 2024-05-27T09:29:23.980 INFO:tasks.workunit.client.0.smithi119.stdout:0/460: dread d2/d6/d9/f27 [0,4194304] 0 2024-05-27T09:29:23.981 INFO:tasks.workunit.client.0.smithi119.stdout:7/527: creat d7/df/d19/d8d/fa3 x:0 0 0 2024-05-27T09:29:23.981 INFO:tasks.workunit.client.0.smithi119.stdout:7/528: stat d7/df/d19/d8d 0 2024-05-27T09:29:23.981 INFO:tasks.workunit.client.0.smithi119.stdout:7/529: fdatasync d7/df/d36/d42/f73 0 2024-05-27T09:29:23.982 INFO:tasks.workunit.client.0.smithi119.stdout:3/618: rename db/d5b/c8c to db/d36/d45/d4c/cc4 0 2024-05-27T09:29:23.983 INFO:tasks.workunit.client.0.smithi119.stdout:6/434: symlink d0/d2/d4e/d5a/l93 0 2024-05-27T09:29:23.986 INFO:tasks.workunit.client.0.smithi119.stdout:2/499: dread d2/da/f37 [0,4194304] 0 2024-05-27T09:29:23.987 INFO:tasks.workunit.client.0.smithi119.stdout:7/530: truncate d7/d1a/d40/f48 2206358 0 2024-05-27T09:29:23.987 INFO:tasks.workunit.client.0.smithi119.stdout:7/531: getdents d7/df/d19/d97/d81 0 2024-05-27T09:29:23.987 INFO:tasks.workunit.client.0.smithi119.stdout:3/619: mknod db/d5b/cc5 0 2024-05-27T09:29:23.987 INFO:tasks.workunit.client.0.smithi119.stdout:6/435: mknod d0/d85/c94 0 2024-05-27T09:29:23.987 INFO:tasks.workunit.client.0.smithi119.stdout:6/436: chown d0/d2/d57/d37/d74 3516 1 2024-05-27T09:29:23.988 INFO:tasks.workunit.client.0.smithi119.stdout:6/437: truncate d0/d2/d4e/f4b 4573871 0 2024-05-27T09:29:23.994 INFO:tasks.workunit.client.0.smithi119.stdout:4/511: dread d4/d6/f3b [0,4194304] 0 2024-05-27T09:29:23.998 INFO:tasks.workunit.client.0.smithi119.stdout:8/467: dwrite d3/de/f1d [0,4194304] 0 2024-05-27T09:29:23.998 INFO:tasks.workunit.client.0.smithi119.stdout:1/564: dwrite d7/dd/d44/d47/f72 [0,4194304] 0 2024-05-27T09:29:23.998 INFO:tasks.workunit.client.0.smithi119.stdout:1/565: chown d7/dd/d44/c45 443 1 2024-05-27T09:29:23.998 INFO:tasks.workunit.client.0.smithi119.stdout:1/566: dread - d7/dd/d44/fa9 zero size 2024-05-27T09:29:24.001 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:23 smithi119 ceph-mon[29082]: pgmap v9: 65 pgs: 65 active+clean; 1.2 GiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 104 MiB/s rd, 127 MiB/s wr, 627 op/s 2024-05-27T09:29:24.017 INFO:tasks.workunit.client.0.smithi119.stdout:8/468: link d3/de/f68 d3/d8/d4d/d5d/d74/f98 0 2024-05-27T09:29:24.021 INFO:tasks.workunit.client.0.smithi119.stdout:8/469: creat d3/d6e/d80/f99 x:0 0 0 2024-05-27T09:29:24.021 INFO:tasks.workunit.client.0.smithi119.stdout:8/470: write d3/de/f1c [7543926,67944] 0 2024-05-27T09:29:24.066 INFO:tasks.workunit.client.0.smithi119.stdout:3/620: dwrite db/f2f [0,4194304] 0 2024-05-27T09:29:24.066 INFO:tasks.workunit.client.0.smithi119.stdout:0/461: dwrite d2/d6/d23/d7a/f46 [4194304,4194304] 0 2024-05-27T09:29:24.073 INFO:tasks.workunit.client.0.smithi119.stdout:7/532: truncate d7/d1a/d40/f48 2067425 0 2024-05-27T09:29:24.075 INFO:tasks.workunit.client.0.smithi119.stdout:1/567: dwrite d7/db/d12/d15/d1f/f8b [0,4194304] 0 2024-05-27T09:29:24.075 INFO:tasks.workunit.client.0.smithi119.stdout:7/533: creat d7/df/d19/d97/d37/fa4 x:0 0 0 2024-05-27T09:29:24.081 INFO:tasks.workunit.client.0.smithi119.stdout:1/568: mkdir d7/dd/db0 0 2024-05-27T09:29:24.084 INFO:tasks.workunit.client.0.smithi119.stdout:3/621: write db/fbf [2155886,66600] 0 2024-05-27T09:29:24.085 INFO:tasks.workunit.client.0.smithi119.stdout:8/471: write d3/d8/d4d/d5d/d60/f71 [4082670,17675] 0 2024-05-27T09:29:24.086 INFO:tasks.workunit.client.0.smithi119.stdout:2/500: dwrite d2/f66 [0,4194304] 0 2024-05-27T09:29:24.096 INFO:tasks.workunit.client.0.smithi119.stdout:8/472: symlink d3/d8/d4d/d79/l9a 0 2024-05-27T09:29:24.096 INFO:tasks.workunit.client.0.smithi119.stdout:8/473: readlink d3/d8/lc 0 2024-05-27T09:29:24.097 INFO:tasks.workunit.client.0.smithi119.stdout:8/474: mkdir d3/d6e/d8e/d9b 0 2024-05-27T09:29:24.098 INFO:tasks.workunit.client.0.smithi119.stdout:8/475: fdatasync d3/d8/d1b/f55 0 2024-05-27T09:29:24.104 INFO:tasks.workunit.client.0.smithi119.stdout:8/476: creat d3/d8/d1b/d77/d95/f9c x:0 0 0 2024-05-27T09:29:24.104 INFO:tasks.workunit.client.0.smithi119.stdout:8/477: fdatasync d3/f87 0 2024-05-27T09:29:24.104 INFO:tasks.workunit.client.0.smithi119.stdout:8/478: chown d3/d8/d4d/d5d/d66/l73 76 1 2024-05-27T09:29:24.116 INFO:tasks.workunit.client.0.smithi119.stdout:8/479: rmdir d3/d8/d4d 39 2024-05-27T09:29:24.130 INFO:tasks.workunit.client.0.smithi119.stdout:1/569: dwrite d7/dd/d44/d47/f72 [0,4194304] 0 2024-05-27T09:29:24.132 INFO:tasks.workunit.client.0.smithi119.stdout:1/570: creat d7/db/d12/d26/d39/fb1 x:0 0 0 2024-05-27T09:29:24.132 INFO:tasks.workunit.client.0.smithi119.stdout:1/571: truncate d7/dd/d44/fa7 305605 0 2024-05-27T09:29:24.134 INFO:tasks.workunit.client.0.smithi119.stdout:1/572: mknod d7/db/d12/d15/cb2 0 2024-05-27T09:29:24.134 INFO:tasks.workunit.client.0.smithi119.stdout:1/573: stat d7/db/d8f 0 2024-05-27T09:29:24.136 INFO:tasks.workunit.client.0.smithi119.stdout:1/574: creat d7/dd/d25/d60/d61/fb3 x:0 0 0 2024-05-27T09:29:24.152 INFO:tasks.workunit.client.0.smithi119.stdout:2/501: dread d2/d1a/f61 [0,4194304] 0 2024-05-27T09:29:24.153 INFO:tasks.workunit.client.0.smithi119.stdout:3/622: dwrite db/d36/d45/d49/d9d/fbb [0,4194304] 0 2024-05-27T09:29:24.153 INFO:tasks.workunit.client.0.smithi119.stdout:2/502: readlink d2/da/d15/d33/d89/d94/l7a 0 2024-05-27T09:29:24.156 INFO:tasks.workunit.client.0.smithi119.stdout:0/462: dwrite d2/d6/d23/d7a/f42 [0,4194304] 0 2024-05-27T09:29:24.161 INFO:tasks.workunit.client.0.smithi119.stdout:2/503: rmdir d2/da/d15/d33/d89/d94/d99 0 2024-05-27T09:29:24.163 INFO:tasks.workunit.client.0.smithi119.stdout:0/463: unlink d2/d1a/c84 0 2024-05-27T09:29:24.173 INFO:tasks.workunit.client.0.smithi119.stdout:3/623: dread db/d36/d45/d4c/d62/f17 [0,4194304] 0 2024-05-27T09:29:24.178 INFO:tasks.workunit.client.0.smithi119.stdout:0/464: truncate d2/d6/d23/d7a/d53/f54 227713 0 2024-05-27T09:29:24.178 INFO:tasks.workunit.client.0.smithi119.stdout:0/465: write d2/d1a/d20/f8b [641514,17848] 0 2024-05-27T09:29:24.178 INFO:tasks.workunit.client.0.smithi119.stdout:0/466: truncate d2/d6/d9/d24/d30/d6b/f87 411866 0 2024-05-27T09:29:24.179 INFO:tasks.workunit.client.0.smithi119.stdout:8/480: dwrite d3/d8/d4d/d58/f89 [0,4194304] 0 2024-05-27T09:29:24.187 INFO:tasks.workunit.client.0.smithi119.stdout:3/624: dread db/d36/d45/d4c/d62/f38 [0,4194304] 0 2024-05-27T09:29:24.188 INFO:tasks.workunit.client.0.smithi119.stdout:3/625: rmdir db/d36/d45/d4c/d62/d13/d33 39 2024-05-27T09:29:24.191 INFO:tasks.workunit.client.0.smithi119.stdout:3/626: link db/d36/d45/d4c/d62/f3f db/d36/d45/d93/fc6 0 2024-05-27T09:29:24.193 INFO:tasks.workunit.client.0.smithi119.stdout:3/627: link db/d36/d45/d4c/d62/d13/c42 db/d36/d45/d49/cc7 0 2024-05-27T09:29:24.193 INFO:tasks.workunit.client.0.smithi119.stdout:3/628: fsync db/d36/d45/d49/d4f/f7b 0 2024-05-27T09:29:24.194 INFO:tasks.workunit.client.0.smithi119.stdout:3/629: mkdir db/d36/d45/d4d/dc8 0 2024-05-27T09:29:24.207 INFO:tasks.workunit.client.0.smithi119.stdout:0/467: write d2/d6/d23/d7a/d53/f54 [109953,61590] 0 2024-05-27T09:29:24.207 INFO:tasks.workunit.client.0.smithi119.stdout:0/468: chown d2/d1a/d20/f74 1453336660 1 2024-05-27T09:29:24.207 INFO:tasks.workunit.client.0.smithi119.stdout:0/469: fdatasync d2/d6/d9/d12/d4b/d59/f58 0 2024-05-27T09:29:24.207 INFO:tasks.workunit.client.0.smithi119.stdout:0/470: chown d2/d6/d9/d24/d30/d4a/c4e 347 1 2024-05-27T09:29:24.207 INFO:tasks.workunit.client.0.smithi119.stdout:1/575: dwrite d7/db/d12/d15/d1f/f8b [0,4194304] 0 2024-05-27T09:29:24.207 INFO:tasks.workunit.client.0.smithi119.stdout:1/576: fsync d7/dd/d44/fa9 0 2024-05-27T09:29:24.216 INFO:tasks.workunit.client.0.smithi119.stdout:0/471: mkdir d2/d6/d9/d12/d4b/d96 0 2024-05-27T09:29:24.217 INFO:tasks.workunit.client.0.smithi119.stdout:1/577: mknod d7/dd/d25/cb4 0 2024-05-27T09:29:24.217 INFO:tasks.workunit.client.0.smithi119.stdout:1/578: dread - d7/db/d12/d26/d39/fb1 zero size 2024-05-27T09:29:24.217 INFO:tasks.workunit.client.0.smithi119.stdout:1/579: chown d7/db/d35 940224624 1 2024-05-27T09:29:24.221 INFO:tasks.workunit.client.0.smithi119.stdout:2/504: dwrite d2/da/d2c/f98 [0,4194304] 0 2024-05-27T09:29:24.224 INFO:tasks.workunit.client.0.smithi119.stdout:1/580: truncate d7/db/d12/d26/f43 44583 0 2024-05-27T09:29:24.230 INFO:tasks.workunit.client.0.smithi119.stdout:2/505: creat d2/da/d15/d53/fa6 x:0 0 0 2024-05-27T09:29:24.231 INFO:tasks.workunit.client.0.smithi119.stdout:8/481: dwrite d3/de/f68 [0,4194304] 0 2024-05-27T09:29:24.234 INFO:tasks.workunit.client.0.smithi119.stdout:1/581: mkdir d7/db/d12/d15/d1f/db5 0 2024-05-27T09:29:24.234 INFO:tasks.workunit.client.0.smithi119.stdout:1/582: stat d7/dd/d44/d94 0 2024-05-27T09:29:24.236 INFO:tasks.workunit.client.0.smithi119.stdout:1/583: read d7/db/d12/d15/f3f [313890,71957] 0 2024-05-27T09:29:24.236 INFO:tasks.workunit.client.0.smithi119.stdout:1/584: write d7/dd/d25/d60/d61/fb3 [1020938,61953] 0 2024-05-27T09:29:24.237 INFO:tasks.workunit.client.0.smithi119.stdout:3/630: rename db/d36/d45/d4c/d62/d76 to db/dc9 0 2024-05-27T09:29:24.240 INFO:tasks.workunit.client.0.smithi119.stdout:8/482: creat d3/f9d x:0 0 0 2024-05-27T09:29:24.242 INFO:tasks.workunit.client.0.smithi119.stdout:0/472: rmdir d2/d6/d9/d12/d4b/d6a 0 2024-05-27T09:29:24.243 INFO:tasks.workunit.client.0.smithi119.stdout:0/473: chown d2/d6/d9/d12/f47 2 1 2024-05-27T09:29:24.244 INFO:tasks.workunit.client.0.smithi119.stdout:3/631: readlink db/d36/d45/d4c/d62/l25 0 2024-05-27T09:29:24.245 INFO:tasks.workunit.client.0.smithi119.stdout:8/483: creat d3/d8/d4d/d5d/d66/d85/f9e x:0 0 0 2024-05-27T09:29:24.245 INFO:tasks.workunit.client.0.smithi119.stdout:8/484: write d3/d8/d4d/d58/f63 [395463,8308] 0 2024-05-27T09:29:24.246 INFO:tasks.workunit.client.0.smithi119.stdout:0/474: truncate d2/d6/d9/d12/f47 3204190 0 2024-05-27T09:29:24.251 INFO:tasks.workunit.client.0.smithi119.stdout:1/585: mkdir d7/db6 0 2024-05-27T09:29:24.251 INFO:tasks.workunit.client.1.smithi138.stdout:5/9: mkdir d3 0 2024-05-27T09:29:24.252 INFO:tasks.workunit.client.0.smithi119.stdout:2/506: dread d2/da/d15/d33/d89/d94/f56 [0,4194304] 0 2024-05-27T09:29:24.253 INFO:tasks.workunit.client.0.smithi119.stdout:3/632: rename db/d36/d45/d4c/d62/f2d to db/d36/d45/d4c/d62/d13/d33/d9b/fca 0 2024-05-27T09:29:24.253 INFO:tasks.workunit.client.0.smithi119.stdout:8/485: link d3/d8/d4d/d5d/f8a d3/d6e/d8e/f9f 0 2024-05-27T09:29:24.253 INFO:tasks.workunit.client.0.smithi119.stdout:3/633: truncate db/d7f/f51 641567 0 2024-05-27T09:29:24.253 INFO:tasks.workunit.client.0.smithi119.stdout:8/486: chown d3/d8/d4d/d58/f97 16 1 2024-05-27T09:29:24.255 INFO:tasks.workunit.client.1.smithi138.stdout:0/15: unlink f2 0 2024-05-27T09:29:24.256 INFO:tasks.workunit.client.0.smithi119.stdout:2/507: mknod d2/da/d4e/ca7 0 2024-05-27T09:29:24.257 INFO:tasks.workunit.client.1.smithi138.stdout:2/19: unlink d1/d7/l5 0 2024-05-27T09:29:24.257 INFO:tasks.workunit.client.1.smithi138.stdout:2/20: chown d1/d7/f4 71 1 2024-05-27T09:29:24.258 INFO:tasks.workunit.client.1.smithi138.stdout:4/6: mknod c1 0 2024-05-27T09:29:24.258 INFO:tasks.workunit.client.0.smithi119.stdout:8/487: creat d3/d6e/fa0 x:0 0 0 2024-05-27T09:29:24.258 INFO:tasks.workunit.client.0.smithi119.stdout:8/488: chown d3/d8/d1b/d46/d92 1 1 2024-05-27T09:29:24.258 INFO:tasks.workunit.client.1.smithi138.stdout:9/16: mkdir d6 0 2024-05-27T09:29:24.259 INFO:tasks.workunit.client.1.smithi138.stdout:5/10: creat d3/f4 x:0 0 0 2024-05-27T09:29:24.260 INFO:tasks.workunit.client.1.smithi138.stdout:0/16: mknod c7 0 2024-05-27T09:29:24.261 INFO:tasks.workunit.client.1.smithi138.stdout:0/17: write f4 [573410,29691] 0 2024-05-27T09:29:24.263 INFO:tasks.workunit.client.1.smithi138.stdout:4/7: rename c1 to c2 0 2024-05-27T09:29:24.264 INFO:tasks.workunit.client.0.smithi119.stdout:2/508: mknod d2/da/d15/d33/d69/ca8 0 2024-05-27T09:29:24.265 INFO:tasks.workunit.client.1.smithi138.stdout:9/17: mknod d6/c7 0 2024-05-27T09:29:24.265 INFO:tasks.workunit.client.1.smithi138.stdout:1/36: mkdir d4/da 0 2024-05-27T09:29:24.267 INFO:tasks.workunit.client.1.smithi138.stdout:9/18: creat d6/f8 x:0 0 0 2024-05-27T09:29:24.270 INFO:tasks.workunit.client.1.smithi138.stdout:1/37: creat d4/fb x:0 0 0 2024-05-27T09:29:24.270 INFO:tasks.workunit.client.0.smithi119.stdout:6/438: sync 2024-05-27T09:29:24.270 INFO:tasks.workunit.client.0.smithi119.stdout:9/393: sync 2024-05-27T09:29:24.270 INFO:tasks.workunit.client.0.smithi119.stdout:2/509: link d2/da/d15/d33/d89/d94/f8b d2/fa9 0 2024-05-27T09:29:24.270 INFO:tasks.workunit.client.0.smithi119.stdout:6/439: creat d0/d2/d4e/d5a/f95 x:0 0 0 2024-05-27T09:29:24.271 INFO:tasks.workunit.client.0.smithi119.stdout:5/491: sync 2024-05-27T09:29:24.271 INFO:tasks.workunit.client.0.smithi119.stdout:5/492: write d6/db/de/d27/f72 [1969256,80969] 0 2024-05-27T09:29:24.276 INFO:tasks.workunit.client.0.smithi119.stdout:6/440: rename d0/d2/d1c/d21/f72 to d0/d2/f96 0 2024-05-27T09:29:24.280 INFO:tasks.workunit.client.1.smithi138.stdout:1/38: link d4/c6 d4/d8/cc 0 2024-05-27T09:29:24.282 INFO:tasks.workunit.client.1.smithi138.stdout:4/8: write f0 [2633968,10239] 0 2024-05-27T09:29:24.283 INFO:tasks.workunit.client.1.smithi138.stdout:1/39: creat d4/fd x:0 0 0 2024-05-27T09:29:24.285 INFO:tasks.workunit.client.1.smithi138.stdout:4/9: link f0 f3 0 2024-05-27T09:29:24.288 INFO:tasks.workunit.client.1.smithi138.stdout:1/40: mknod d4/da/ce 0 2024-05-27T09:29:24.289 INFO:tasks.workunit.client.1.smithi138.stdout:1/41: chown d4/fb 2046455501 1 2024-05-27T09:29:24.392 INFO:tasks.workunit.client.0.smithi119.stdout:5/493: mknod d6/dd/d39/d19/d3c/d52/d9c/cab 0 2024-05-27T09:29:24.395 INFO:tasks.workunit.client.0.smithi119.stdout:5/494: creat d6/dd/d39/d19/fac x:0 0 0 2024-05-27T09:29:24.396 INFO:tasks.workunit.client.0.smithi119.stdout:5/495: dread - d6/db/de/d27/f76 zero size 2024-05-27T09:29:24.417 INFO:tasks.workunit.client.1.smithi138.stdout:4/10: dread f0 [0,4194304] 0 2024-05-27T09:29:24.418 INFO:tasks.workunit.client.1.smithi138.stdout:4/11: read f3 [937261,28217] 0 2024-05-27T09:29:24.435 INFO:tasks.workunit.client.0.smithi119.stdout:3/634: read db/d36/d45/d4c/f6c [1781972,71821] 0 2024-05-27T09:29:24.450 INFO:tasks.workunit.client.0.smithi119.stdout:8/489: dwrite d3/d39/f88 [0,4194304] 0 2024-05-27T09:29:24.458 INFO:tasks.workunit.client.0.smithi119.stdout:0/475: dwrite d2/d6/d23/d7a/f62 [0,4194304] 0 2024-05-27T09:29:24.461 INFO:tasks.workunit.client.0.smithi119.stdout:6/441: dread d0/d2/d1c/f2a [0,4194304] 0 2024-05-27T09:29:24.465 INFO:tasks.workunit.client.0.smithi119.stdout:0/476: truncate d2/d6/d23/d7a/f44 919403 0 2024-05-27T09:29:24.465 INFO:tasks.workunit.client.0.smithi119.stdout:0/477: read d2/d6/d23/f33 [3698105,100929] 0 2024-05-27T09:29:24.467 INFO:tasks.workunit.client.0.smithi119.stdout:5/496: dread d6/db/de/f25 [4194304,4194304] 0 2024-05-27T09:29:24.469 INFO:tasks.workunit.client.0.smithi119.stdout:5/497: link d6/f63 d6/dd/df/d41/fad 0 2024-05-27T09:29:24.471 INFO:tasks.workunit.client.0.smithi119.stdout:1/586: dwrite d7/db/d12/f6b [0,4194304] 0 2024-05-27T09:29:24.473 INFO:tasks.workunit.client.0.smithi119.stdout:5/498: rename d6/dd/df/d8b to d6/dd/d39/d19/d84/dae 0 2024-05-27T09:29:24.473 INFO:tasks.workunit.client.0.smithi119.stdout:5/499: write d6/f63 [4584711,102994] 0 2024-05-27T09:29:24.475 INFO:tasks.workunit.client.0.smithi119.stdout:9/394: dwrite d18/d19/d1b/f1d [0,4194304] 0 2024-05-27T09:29:24.475 INFO:tasks.workunit.client.0.smithi119.stdout:9/395: write f2 [9257010,71045] 0 2024-05-27T09:29:24.476 INFO:tasks.workunit.client.0.smithi119.stdout:5/500: getdents d6/dd/d46/d47/d4b 0 2024-05-27T09:29:24.477 INFO:tasks.workunit.client.0.smithi119.stdout:2/510: dwrite d2/da/d15/d33/f7e [0,4194304] 0 2024-05-27T09:29:24.477 INFO:tasks.workunit.client.0.smithi119.stdout:2/511: truncate d2/da/d15/d33/d89/d94/f73 1500502 0 2024-05-27T09:29:24.479 INFO:tasks.workunit.client.0.smithi119.stdout:1/587: mknod d7/dd/d44/d47/cb7 0 2024-05-27T09:29:24.479 INFO:tasks.workunit.client.0.smithi119.stdout:1/588: truncate d7/db/d12/f9a 1655472 0 2024-05-27T09:29:24.481 INFO:tasks.workunit.client.0.smithi119.stdout:9/396: link d18/d19/d1b/l31 d18/d19/d61/l83 0 2024-05-27T09:29:24.481 INFO:tasks.workunit.client.0.smithi119.stdout:9/397: truncate d18/d19/d1b/d40/d7b/f7e 1027533 0 2024-05-27T09:29:24.481 INFO:tasks.workunit.client.0.smithi119.stdout:9/398: chown d18/d19/f59 48140630 1 2024-05-27T09:29:24.481 INFO:tasks.workunit.client.0.smithi119.stdout:9/399: chown d18/d19/d1b/d29/f2c 1 1 2024-05-27T09:29:24.482 INFO:tasks.workunit.client.0.smithi119.stdout:5/501: rmdir d6/db/d9a 39 2024-05-27T09:29:24.484 INFO:tasks.workunit.client.0.smithi119.stdout:2/512: symlink d2/da/d8a/laa 0 2024-05-27T09:29:24.489 INFO:tasks.workunit.client.0.smithi119.stdout:9/400: rename d18/d19/d61/l77 to d18/d19/d1b/d40/l84 0 2024-05-27T09:29:24.493 INFO:tasks.workunit.client.0.smithi119.stdout:5/502: rmdir d6/dd/d39/d19/d3c/d52 39 2024-05-27T09:29:24.493 INFO:tasks.workunit.client.0.smithi119.stdout:1/589: mknod d7/db/d8f/d9b/cb8 0 2024-05-27T09:29:24.494 INFO:tasks.workunit.client.0.smithi119.stdout:7/534: sync 2024-05-27T09:29:24.494 INFO:tasks.workunit.client.0.smithi119.stdout:7/535: chown d7/d1a/d40/d44/c54 375471 1 2024-05-27T09:29:24.506 INFO:tasks.workunit.client.0.smithi119.stdout:4/512: sync 2024-05-27T09:29:24.507 INFO:tasks.workunit.client.0.smithi119.stdout:4/513: fdatasync d4/f22 0 2024-05-27T09:29:24.507 INFO:tasks.workunit.client.0.smithi119.stdout:4/514: stat d4/d6/d3e/f6e 0 2024-05-27T09:29:24.507 INFO:tasks.workunit.client.0.smithi119.stdout:4/515: write d4/d23/f52 [4949259,97905] 0 2024-05-27T09:29:24.507 INFO:tasks.workunit.client.0.smithi119.stdout:1/590: unlink d7/db/d35/f69 0 2024-05-27T09:29:24.507 INFO:tasks.workunit.client.0.smithi119.stdout:1/591: chown d7/dd/d44/d47/cb7 62867 1 2024-05-27T09:29:24.507 INFO:tasks.workunit.client.0.smithi119.stdout:4/516: symlink d4/de/d9b/l9d 0 2024-05-27T09:29:24.511 INFO:tasks.workunit.client.0.smithi119.stdout:1/592: creat d7/db/d8f/d9b/fb9 x:0 0 0 2024-05-27T09:29:24.515 INFO:tasks.workunit.client.0.smithi119.stdout:1/593: symlink d7/dd/d25/d29/lba 0 2024-05-27T09:29:24.515 INFO:tasks.workunit.client.0.smithi119.stdout:1/594: fsync d7/dd/d44/f88 0 2024-05-27T09:29:24.515 INFO:tasks.workunit.client.0.smithi119.stdout:1/595: write d7/db/d12/d15/f8e [527684,123209] 0 2024-05-27T09:29:24.515 INFO:tasks.workunit.client.0.smithi119.stdout:1/596: creat d7/db/d12/d15/d1f/fbb x:0 0 0 2024-05-27T09:29:24.515 INFO:tasks.workunit.client.0.smithi119.stdout:1/597: write d7/f27 [1173428,15229] 0 2024-05-27T09:29:24.531 INFO:tasks.workunit.client.0.smithi119.stdout:0/478: dread d2/d6/d9/f3a [0,4194304] 0 2024-05-27T09:29:24.531 INFO:tasks.workunit.client.0.smithi119.stdout:8/490: dwrite d3/d8/d4d/d5d/d60/f69 [0,4194304] 0 2024-05-27T09:29:24.531 INFO:tasks.workunit.client.0.smithi119.stdout:0/479: fsync d2/d6/d9/f85 0 2024-05-27T09:29:24.532 INFO:tasks.workunit.client.0.smithi119.stdout:0/480: link d2/d6/d9/d12/d4f/l6c d2/d1a/d20/l97 0 2024-05-27T09:29:24.534 INFO:tasks.workunit.client.0.smithi119.stdout:0/481: creat d2/d6/d9/d12/d4b/d96/f98 x:0 0 0 2024-05-27T09:29:24.534 INFO:tasks.workunit.client.0.smithi119.stdout:0/482: chown d2/d6/d23/d7a/f42 3 1 2024-05-27T09:29:24.535 INFO:tasks.workunit.client.0.smithi119.stdout:0/483: mknod d2/d6/d9/d24/d30/c99 0 2024-05-27T09:29:24.536 INFO:tasks.workunit.client.0.smithi119.stdout:0/484: mknod d2/d6/d9/d24/d30/d6b/c9a 0 2024-05-27T09:29:24.537 INFO:tasks.workunit.client.0.smithi119.stdout:0/485: mknod d2/d6/d9/d2f/c9b 0 2024-05-27T09:29:24.539 INFO:tasks.workunit.client.0.smithi119.stdout:0/486: truncate d2/d1a/d20/f8b 550306 0 2024-05-27T09:29:24.540 INFO:tasks.workunit.client.0.smithi119.stdout:0/487: symlink d2/d6/d9/d12/d4b/d59/l9c 0 2024-05-27T09:29:24.542 INFO:tasks.workunit.client.0.smithi119.stdout:0/488: symlink d2/l9d 0 2024-05-27T09:29:24.543 INFO:tasks.workunit.client.0.smithi119.stdout:0/489: symlink d2/d1a/l9e 0 2024-05-27T09:29:24.558 INFO:tasks.workunit.client.0.smithi119.stdout:7/536: write d7/d1a/f2b [450498,26704] 0 2024-05-27T09:29:24.558 INFO:tasks.workunit.client.0.smithi119.stdout:7/537: chown f4 958 1 2024-05-27T09:29:24.558 INFO:tasks.workunit.client.0.smithi119.stdout:7/538: readlink d7/df/d19/l34 0 2024-05-27T09:29:24.558 INFO:tasks.workunit.client.0.smithi119.stdout:8/491: dread d3/d8/d1b/d46/f81 [0,4194304] 0 2024-05-27T09:29:24.560 INFO:tasks.workunit.client.0.smithi119.stdout:7/539: mknod d7/d5f/ca5 0 2024-05-27T09:29:24.563 INFO:tasks.workunit.client.0.smithi119.stdout:0/490: dread d2/d6/d23/d7a/f44 [0,4194304] 0 2024-05-27T09:29:24.570 INFO:tasks.workunit.client.0.smithi119.stdout:9/401: dwrite d18/d19/d1b/d40/d44/f4a [0,4194304] 0 2024-05-27T09:29:24.570 INFO:tasks.workunit.client.0.smithi119.stdout:9/402: truncate d18/d19/d1b/d40/d44/f70 198705 0 2024-05-27T09:29:24.579 INFO:tasks.workunit.client.0.smithi119.stdout:7/540: mkdir d7/df/d19/d9a/da6 0 2024-05-27T09:29:24.586 INFO:tasks.workunit.client.0.smithi119.stdout:0/491: write d2/d6/d9/d12/f47 [1508357,514] 0 2024-05-27T09:29:24.586 INFO:tasks.workunit.client.0.smithi119.stdout:0/492: fsync d2/d6/d9/d12/d4b/d59/f83 0 2024-05-27T09:29:24.589 INFO:tasks.workunit.client.0.smithi119.stdout:0/493: unlink d2/d6/d9/d24/d30/d4a/f4c 0 2024-05-27T09:29:24.589 INFO:tasks.workunit.client.0.smithi119.stdout:1/598: dread d7/f14 [0,4194304] 0 2024-05-27T09:29:24.600 INFO:tasks.workunit.client.0.smithi119.stdout:9/403: creat d18/d19/f85 x:0 0 0 2024-05-27T09:29:24.608 INFO:tasks.workunit.client.0.smithi119.stdout:0/494: creat d2/d6/d9/d24/f9f x:0 0 0 2024-05-27T09:29:24.609 INFO:tasks.workunit.client.0.smithi119.stdout:1/599: creat d7/db6/fbc x:0 0 0 2024-05-27T09:29:24.609 INFO:tasks.workunit.client.0.smithi119.stdout:1/600: creat d7/db/d12/d15/d1f/fbd x:0 0 0 2024-05-27T09:29:24.609 INFO:tasks.workunit.client.0.smithi119.stdout:0/495: write d2/d6/d75/f86 [1071016,127016] 0 2024-05-27T09:29:24.613 INFO:tasks.workunit.client.0.smithi119.stdout:3/635: dwrite db/d36/d45/d4c/d62/f17 [0,4194304] 0 2024-05-27T09:29:24.620 INFO:tasks.workunit.client.0.smithi119.stdout:0/496: symlink d2/d6/d9/d24/d30/la0 0 2024-05-27T09:29:24.620 INFO:tasks.workunit.client.0.smithi119.stdout:0/497: chown d2/d6/d9/d12/d4f/c73 1 1 2024-05-27T09:29:24.625 INFO:tasks.workunit.client.0.smithi119.stdout:1/601: mkdir d7/db/d12/d15/d1f/dbe 0 2024-05-27T09:29:24.629 INFO:tasks.workunit.client.0.smithi119.stdout:1/602: rename d7/db/d12/f6b to d7/dd/d25/d29/d38/d3c/fbf 0 2024-05-27T09:29:24.629 INFO:tasks.workunit.client.0.smithi119.stdout:1/603: fdatasync d7/dd/d25/d29/f55 0 2024-05-27T09:29:24.631 INFO:tasks.workunit.client.0.smithi119.stdout:1/604: rename d7/db/d52/d65/c76 to d7/dd/d25/d29/d38/d49/cc0 0 2024-05-27T09:29:24.632 INFO:tasks.workunit.client.0.smithi119.stdout:7/541: dread d7/f66 [0,4194304] 0 2024-05-27T09:29:24.632 INFO:tasks.workunit.client.0.smithi119.stdout:7/542: chown f1 1124106 1 2024-05-27T09:29:24.633 INFO:tasks.workunit.client.0.smithi119.stdout:7/543: rmdir d7/df/d19/d4d 39 2024-05-27T09:29:24.649 INFO:tasks.workunit.client.0.smithi119.stdout:4/517: dread d4/f8f [0,4194304] 0 2024-05-27T09:29:24.652 INFO:tasks.workunit.client.0.smithi119.stdout:5/503: dwrite d6/f9 [4194304,4194304] 0 2024-05-27T09:29:24.658 INFO:tasks.workunit.client.0.smithi119.stdout:6/442: dwrite d0/d2/d4e/d5a/f95 [0,4194304] 0 2024-05-27T09:29:24.663 INFO:tasks.workunit.client.0.smithi119.stdout:9/404: dread f6 [0,4194304] 0 2024-05-27T09:29:24.663 INFO:tasks.workunit.client.0.smithi119.stdout:5/504: truncate d6/dd/d39/d19/d3c/f43 852364 0 2024-05-27T09:29:24.663 INFO:tasks.workunit.client.0.smithi119.stdout:6/443: truncate d0/d2/d1c/d21/d58/f61 76671 0 2024-05-27T09:29:24.664 INFO:tasks.workunit.client.0.smithi119.stdout:9/405: chown d18/d19/d1b/c3c 418521973 1 2024-05-27T09:29:24.664 INFO:tasks.workunit.client.0.smithi119.stdout:5/505: truncate d6/db/de/f25 6943152 0 2024-05-27T09:29:24.681 INFO:tasks.workunit.client.0.smithi119.stdout:9/406: symlink d18/d19/d1b/d29/l86 0 2024-05-27T09:29:24.681 INFO:tasks.workunit.client.0.smithi119.stdout:7/544: write d7/df/d36/f49 [2147923,42370] 0 2024-05-27T09:29:24.681 INFO:tasks.workunit.client.0.smithi119.stdout:8/492: dwrite d3/d8/d4d/d5d/d66/f94 [0,4194304] 0 2024-05-27T09:29:24.681 INFO:tasks.workunit.client.0.smithi119.stdout:5/506: write d6/dd/f1b [1127609,124744] 0 2024-05-27T09:29:24.681 INFO:tasks.workunit.client.0.smithi119.stdout:5/507: write d6/db/de/d27/d94/fa9 [202364,55227] 0 2024-05-27T09:29:24.681 INFO:tasks.workunit.client.0.smithi119.stdout:8/493: symlink d3/d6e/d80/la1 0 2024-05-27T09:29:24.681 INFO:tasks.workunit.client.0.smithi119.stdout:5/508: read d6/dd/f37 [499591,10073] 0 2024-05-27T09:29:24.682 INFO:tasks.workunit.client.0.smithi119.stdout:8/494: rename d3/d8/d4d/d5d/l4e to d3/d39/d5a/la2 0 2024-05-27T09:29:24.684 INFO:tasks.workunit.client.0.smithi119.stdout:5/509: mknod d6/dd/d39/d19/d3c/caf 0 2024-05-27T09:29:24.684 INFO:tasks.workunit.client.0.smithi119.stdout:5/510: fdatasync d6/dd/df/d41/fad 0 2024-05-27T09:29:24.684 INFO:tasks.workunit.client.0.smithi119.stdout:5/511: chown d6/dd/d39/d19/d7d/ca6 88683 1 2024-05-27T09:29:24.684 INFO:tasks.workunit.client.0.smithi119.stdout:5/512: readlink d6/db/l85 0 2024-05-27T09:29:24.686 INFO:tasks.workunit.client.0.smithi119.stdout:8/495: truncate d3/d8/d1b/d46/f7c 1926882 0 2024-05-27T09:29:24.687 INFO:tasks.workunit.client.0.smithi119.stdout:5/513: creat d6/dd/df/d41/fb0 x:0 0 0 2024-05-27T09:29:24.687 INFO:tasks.workunit.client.0.smithi119.stdout:5/514: write d6/f22 [3238337,120569] 0 2024-05-27T09:29:24.697 INFO:tasks.workunit.client.0.smithi119.stdout:8/496: creat d3/d8/d4d/d5d/fa3 x:0 0 0 2024-05-27T09:29:24.697 INFO:tasks.workunit.client.0.smithi119.stdout:8/497: chown d3/de/c51 461596 1 2024-05-27T09:29:24.697 INFO:tasks.workunit.client.0.smithi119.stdout:9/407: read d18/d19/f35 [2153286,108811] 0 2024-05-27T09:29:24.697 INFO:tasks.workunit.client.0.smithi119.stdout:5/515: creat d6/d2f/d9d/fb1 x:0 0 0 2024-05-27T09:29:24.700 INFO:tasks.workunit.client.0.smithi119.stdout:9/408: unlink ff 0 2024-05-27T09:29:24.701 INFO:tasks.workunit.client.0.smithi119.stdout:5/516: creat d6/dd/d39/d19/d3c/fb2 x:0 0 0 2024-05-27T09:29:24.706 INFO:tasks.workunit.client.0.smithi119.stdout:9/409: mkdir d18/d19/d1b/d40/d87 0 2024-05-27T09:29:24.706 INFO:tasks.workunit.client.0.smithi119.stdout:7/545: write d7/d1a/fa1 [335936,26239] 0 2024-05-27T09:29:24.706 INFO:tasks.workunit.client.0.smithi119.stdout:7/546: fsync d7/df/d36/f61 0 2024-05-27T09:29:24.710 INFO:tasks.workunit.client.0.smithi119.stdout:5/517: symlink d6/dd/d39/lb3 0 2024-05-27T09:29:24.733 INFO:tasks.workunit.client.0.smithi119.stdout:6/444: dread d0/d2/d57/d37/f43 [0,4194304] 0 2024-05-27T09:29:24.734 INFO:tasks.workunit.client.0.smithi119.stdout:1/605: dwrite d7/dd/f77 [0,4194304] 0 2024-05-27T09:29:24.736 INFO:tasks.workunit.client.0.smithi119.stdout:5/518: dread d6/f63 [0,4194304] 0 2024-05-27T09:29:24.736 INFO:tasks.workunit.client.0.smithi119.stdout:5/519: write d6/d2f/f42 [2073887,49882] 0 2024-05-27T09:29:24.738 INFO:tasks.workunit.client.0.smithi119.stdout:7/547: dread d7/fa [0,4194304] 0 2024-05-27T09:29:24.738 INFO:tasks.workunit.client.0.smithi119.stdout:7/548: dread - d7/df/d36/d42/d76/f9d zero size 2024-05-27T09:29:24.743 INFO:tasks.workunit.client.0.smithi119.stdout:4/518: dwrite d4/d6/d28/d2f/f50 [0,4194304] 0 2024-05-27T09:29:24.743 INFO:tasks.workunit.client.0.smithi119.stdout:5/520: dread d6/dd/f73 [0,4194304] 0 2024-05-27T09:29:24.747 INFO:tasks.workunit.client.0.smithi119.stdout:6/445: rmdir d0/d8f 0 2024-05-27T09:29:24.747 INFO:tasks.workunit.client.0.smithi119.stdout:7/549: mknod d7/df/d19/ca7 0 2024-05-27T09:29:24.750 INFO:tasks.workunit.client.0.smithi119.stdout:7/550: rename d7/df/d36/d42/l9e to d7/df/d19/d4d/d89/la8 0 2024-05-27T09:29:24.750 INFO:tasks.workunit.client.0.smithi119.stdout:7/551: getdents d7/df/d19/d46/d93 0 2024-05-27T09:29:24.762 INFO:tasks.workunit.client.0.smithi119.stdout:6/446: dread d0/d2/d57/d37/f63 [0,4194304] 0 2024-05-27T09:29:24.764 INFO:tasks.workunit.client.0.smithi119.stdout:6/447: rename d0/d2/d4e/f88 to d0/d2/d4e/d5a/f97 0 2024-05-27T09:29:24.764 INFO:tasks.workunit.client.0.smithi119.stdout:6/448: chown d0/f52 14260696 1 2024-05-27T09:29:24.767 INFO:tasks.workunit.client.0.smithi119.stdout:4/519: dread d4/de/d21/d40/f6a [0,4194304] 0 2024-05-27T09:29:24.768 INFO:tasks.workunit.client.0.smithi119.stdout:1/606: dread d7/dd/f5d [0,4194304] 0 2024-05-27T09:29:24.784 INFO:tasks.workunit.client.0.smithi119.stdout:9/410: dread f2 [4194304,4194304] 0 2024-05-27T09:29:24.787 INFO:tasks.workunit.client.0.smithi119.stdout:8/498: dwrite d3/d8/d4d/d5d/d60/f71 [0,4194304] 0 2024-05-27T09:29:24.793 INFO:tasks.workunit.client.0.smithi119.stdout:8/499: rmdir d3/d39/d5a 39 2024-05-27T09:29:24.811 INFO:tasks.workunit.client.0.smithi119.stdout:7/552: write d7/df/d19/d4d/f71 [1564783,83736] 0 2024-05-27T09:29:24.811 INFO:tasks.workunit.client.0.smithi119.stdout:6/449: write d0/d11/f62 [1106384,121846] 0 2024-05-27T09:29:24.811 INFO:tasks.workunit.client.0.smithi119.stdout:6/450: readlink d0/d2/d4e/l2f 0 2024-05-27T09:29:24.811 INFO:tasks.workunit.client.0.smithi119.stdout:6/451: readlink d0/d19/l8c 0 2024-05-27T09:29:24.811 INFO:tasks.workunit.client.0.smithi119.stdout:6/452: dread - d0/f1e zero size 2024-05-27T09:29:24.811 INFO:tasks.workunit.client.0.smithi119.stdout:6/453: write d0/f1e [50192,42378] 0 2024-05-27T09:29:24.815 INFO:tasks.workunit.client.0.smithi119.stdout:7/553: link d7/df/d19/d97/l7e d7/df/d36/d42/d76/la9 0 2024-05-27T09:29:24.815 INFO:tasks.workunit.client.0.smithi119.stdout:7/554: readlink d7/df/d19/d97/l7e 0 2024-05-27T09:29:24.815 INFO:tasks.workunit.client.0.smithi119.stdout:6/454: truncate d0/d2/f67 2461343 0 2024-05-27T09:29:24.816 INFO:tasks.workunit.client.0.smithi119.stdout:7/555: link d7/d5f/f7c d7/faa 0 2024-05-27T09:29:24.818 INFO:tasks.workunit.client.0.smithi119.stdout:6/455: creat d0/d2/d57/d37/d74/f98 x:0 0 0 2024-05-27T09:29:24.822 INFO:tasks.workunit.client.0.smithi119.stdout:7/556: creat d7/df/d19/d97/d92/fab x:0 0 0 2024-05-27T09:29:24.822 INFO:tasks.workunit.client.0.smithi119.stdout:6/456: link d0/d2/d1c/d21/f46 d0/d85/f99 0 2024-05-27T09:29:24.838 INFO:tasks.workunit.client.0.smithi119.stdout:6/457: dread d0/d2/f30 [0,4194304] 0 2024-05-27T09:29:24.840 INFO:tasks.workunit.client.0.smithi119.stdout:6/458: creat d0/f9a x:0 0 0 2024-05-27T09:29:24.846 INFO:tasks.workunit.client.0.smithi119.stdout:8/500: write d3/d8/d4d/f36 [2563667,23573] 0 2024-05-27T09:29:24.848 INFO:tasks.workunit.client.0.smithi119.stdout:4/520: dwrite d4/d23/d31/f33 [0,4194304] 0 2024-05-27T09:29:24.849 INFO:tasks.workunit.client.0.smithi119.stdout:8/501: mkdir d3/d8/d1b/d46/da4 0 2024-05-27T09:29:24.853 INFO:tasks.workunit.client.0.smithi119.stdout:1/607: dwrite d7/db/d12/d15/f8e [0,4194304] 0 2024-05-27T09:29:24.853 INFO:tasks.workunit.client.0.smithi119.stdout:1/608: chown d7/f14 331 1 2024-05-27T09:29:24.857 INFO:tasks.workunit.client.0.smithi119.stdout:1/609: getdents d7/db/d12/d26/d39 0 2024-05-27T09:29:24.867 INFO:tasks.workunit.client.0.smithi119.stdout:4/521: read d4/d23/d24/f99 [1205321,100831] 0 2024-05-27T09:29:24.867 INFO:tasks.workunit.client.0.smithi119.stdout:4/522: chown d4/d23/d24/f82 14 1 2024-05-27T09:29:24.869 INFO:tasks.workunit.client.0.smithi119.stdout:5/521: dwrite d6/dd/f37 [0,4194304] 0 2024-05-27T09:29:24.869 INFO:tasks.workunit.client.0.smithi119.stdout:5/522: write d6/db/de/f89 [450389,29062] 0 2024-05-27T09:29:24.877 INFO:tasks.workunit.client.0.smithi119.stdout:5/523: creat d6/dd/d39/d19/d84/fb4 x:0 0 0 2024-05-27T09:29:24.877 INFO:tasks.workunit.client.0.smithi119.stdout:5/524: chown d6/f1e 3126 1 2024-05-27T09:29:24.877 INFO:tasks.workunit.client.0.smithi119.stdout:5/525: truncate d6/db/d9a/f9e 579915 0 2024-05-27T09:29:24.883 INFO:tasks.workunit.client.0.smithi119.stdout:4/523: dread d4/d6/f2a [0,4194304] 0 2024-05-27T09:29:24.883 INFO:tasks.workunit.client.0.smithi119.stdout:5/526: unlink d6/db/de/d27/f72 0 2024-05-27T09:29:24.883 INFO:tasks.workunit.client.0.smithi119.stdout:5/527: write d6/db/f4c [752667,3938] 0 2024-05-27T09:29:24.883 INFO:tasks.workunit.client.0.smithi119.stdout:5/528: creat d6/dd/d39/d19/fb5 x:0 0 0 2024-05-27T09:29:24.883 INFO:tasks.workunit.client.0.smithi119.stdout:5/529: chown d6/db/f6f 102962432 1 2024-05-27T09:29:24.887 INFO:tasks.workunit.client.0.smithi119.stdout:1/610: dread d7/db/d12/d15/f3f [0,4194304] 0 2024-05-27T09:29:24.901 INFO:tasks.workunit.client.0.smithi119.stdout:3/636: sync 2024-05-27T09:29:24.901 INFO:tasks.workunit.client.0.smithi119.stdout:6/459: dwrite d0/d2/d57/d37/f8d [0,4194304] 0 2024-05-27T09:29:24.903 INFO:tasks.workunit.client.0.smithi119.stdout:2/513: sync 2024-05-27T09:29:24.906 INFO:tasks.workunit.client.0.smithi119.stdout:6/460: rename d0/d2/d1c/d21/f83 to d0/d2/f9b 0 2024-05-27T09:29:24.906 INFO:tasks.workunit.client.0.smithi119.stdout:6/461: read d0/d2/d57/d37/f6e [131793,3526] 0 2024-05-27T09:29:24.907 INFO:tasks.workunit.client.0.smithi119.stdout:6/462: chown d0/d2/f7f 4855 1 2024-05-27T09:29:24.910 INFO:tasks.workunit.client.0.smithi119.stdout:3/637: stat db/d36/d45/d4c/d62/c55 0 2024-05-27T09:29:24.910 INFO:tasks.workunit.client.0.smithi119.stdout:3/638: creat db/d36/d45/fcb x:0 0 0 2024-05-27T09:29:24.910 INFO:tasks.workunit.client.0.smithi119.stdout:3/639: fdatasync db/d36/d45/d4c/d62/d13/d33/fa7 0 2024-05-27T09:29:24.913 INFO:tasks.workunit.client.0.smithi119.stdout:6/463: mknod d0/d2/d1c/d21/c9c 0 2024-05-27T09:29:24.913 INFO:tasks.workunit.client.0.smithi119.stdout:1/611: dread f3 [0,4194304] 0 2024-05-27T09:29:24.913 INFO:tasks.workunit.client.0.smithi119.stdout:1/612: chown d7/dd/d25/d60 719668 1 2024-05-27T09:29:24.913 INFO:tasks.workunit.client.0.smithi119.stdout:3/640: write db/d36/d45/d4c/d62/d13/f2e [5021520,51927] 0 2024-05-27T09:29:24.914 INFO:tasks.workunit.client.0.smithi119.stdout:3/641: truncate db/d36/d45/d49/d9d/fbb 1879877 0 2024-05-27T09:29:24.915 INFO:tasks.workunit.client.0.smithi119.stdout:3/642: truncate db/dc9/d98/f73 489740 0 2024-05-27T09:29:24.916 INFO:tasks.workunit.client.0.smithi119.stdout:3/643: read - db/dc9/d98/d7a/f84 zero size 2024-05-27T09:29:24.918 INFO:tasks.workunit.client.0.smithi119.stdout:0/498: sync 2024-05-27T09:29:24.918 INFO:tasks.workunit.client.0.smithi119.stdout:3/644: symlink db/d36/lcc 0 2024-05-27T09:29:24.918 INFO:tasks.workunit.client.0.smithi119.stdout:3/645: chown db/d36/d45/d4c/d62/c55 9094971 1 2024-05-27T09:29:24.920 INFO:tasks.workunit.client.0.smithi119.stdout:0/499: unlink d2/d6/d9/d12/c3e 0 2024-05-27T09:29:24.922 INFO:tasks.workunit.client.0.smithi119.stdout:6/464: dread d0/d2/d4e/f6a [0,4194304] 0 2024-05-27T09:29:24.924 INFO:tasks.workunit.client.0.smithi119.stdout:3/646: creat db/d36/d45/dab/fcd x:0 0 0 2024-05-27T09:29:24.925 INFO:tasks.workunit.client.0.smithi119.stdout:6/465: symlink d0/d2/d57/l9d 0 2024-05-27T09:29:24.927 INFO:tasks.workunit.client.0.smithi119.stdout:3/647: mkdir db/d5b/dce 0 2024-05-27T09:29:24.927 INFO:tasks.workunit.client.0.smithi119.stdout:3/648: fsync db/d36/d45/d4c/d62/d13/f97 0 2024-05-27T09:29:24.929 INFO:tasks.workunit.client.0.smithi119.stdout:6/466: truncate d0/d2/d57/d37/f5d 3059254 0 2024-05-27T09:29:24.930 INFO:tasks.workunit.client.0.smithi119.stdout:6/467: read d0/d2/d1c/d21/f3c [2168642,39369] 0 2024-05-27T09:29:24.931 INFO:tasks.workunit.client.0.smithi119.stdout:3/649: symlink db/d36/d45/d4d/lcf 0 2024-05-27T09:29:24.932 INFO:tasks.workunit.client.0.smithi119.stdout:3/650: fdatasync db/dc9/d98/f80 0 2024-05-27T09:29:24.936 INFO:tasks.workunit.client.0.smithi119.stdout:0/500: dread d2/d1a/f1e [0,4194304] 0 2024-05-27T09:29:24.936 INFO:tasks.workunit.client.0.smithi119.stdout:6/468: rename d0/d2/d4e/c4d to d0/d2/d1c/d21/d58/c9e 0 2024-05-27T09:29:24.946 INFO:tasks.workunit.client.1.smithi138.stdout:3/3: sync 2024-05-27T09:29:24.946 INFO:tasks.workunit.client.1.smithi138.stdout:6/10: sync 2024-05-27T09:29:24.946 INFO:tasks.workunit.client.1.smithi138.stdout:3/4: dread - no filename 2024-05-27T09:29:24.946 INFO:tasks.workunit.client.1.smithi138.stdout:3/5: truncate - no filename 2024-05-27T09:29:24.946 INFO:tasks.workunit.client.1.smithi138.stdout:8/0: sync 2024-05-27T09:29:24.946 INFO:tasks.workunit.client.1.smithi138.stdout:6/11: fdatasync f0 0 2024-05-27T09:29:24.946 INFO:tasks.workunit.client.1.smithi138.stdout:7/5: sync 2024-05-27T09:29:24.949 INFO:tasks.workunit.client.1.smithi138.stdout:2/21: getdents d1/d7 0 2024-05-27T09:29:24.951 INFO:tasks.workunit.client.1.smithi138.stdout:5/11: getdents d3 0 2024-05-27T09:29:24.951 INFO:tasks.workunit.client.1.smithi138.stdout:3/6: mknod d0/c1 0 2024-05-27T09:29:24.951 INFO:tasks.workunit.client.1.smithi138.stdout:3/7: dwrite - no filename 2024-05-27T09:29:24.951 INFO:tasks.workunit.client.1.smithi138.stdout:3/8: dread - no filename 2024-05-27T09:29:24.964 INFO:tasks.workunit.client.1.smithi138.stdout:6/12: symlink d3/l4 0 2024-05-27T09:29:24.966 INFO:tasks.workunit.client.1.smithi138.stdout:9/19: truncate f3 4084748 0 2024-05-27T09:29:24.966 INFO:tasks.workunit.client.1.smithi138.stdout:1/42: rmdir d4 39 2024-05-27T09:29:24.966 INFO:tasks.workunit.client.1.smithi138.stdout:4/12: truncate f3 1447885 0 2024-05-27T09:29:24.977 INFO:tasks.workunit.client.1.smithi138.stdout:6/13: creat d3/f5 x:0 0 0 2024-05-27T09:29:24.990 INFO:tasks.workunit.client.1.smithi138.stdout:6/14: unlink f0 0 2024-05-27T09:29:24.994 INFO:tasks.workunit.client.0.smithi119.stdout:4/524: read d4/d48/f5e [3161539,1881] 0 2024-05-27T09:29:24.994 INFO:tasks.workunit.client.0.smithi119.stdout:4/525: getdents d4/d4e 0 2024-05-27T09:29:24.994 INFO:tasks.workunit.client.0.smithi119.stdout:4/526: fdatasync d4/de/d66/f68 0 2024-05-27T09:29:24.999 INFO:tasks.workunit.client.1.smithi138.stdout:1/43: mknod d4/cf 0 2024-05-27T09:29:24.999 INFO:tasks.workunit.client.0.smithi119.stdout:4/527: rmdir d4 39 2024-05-27T09:29:24.999 INFO:tasks.workunit.client.0.smithi119.stdout:4/528: fsync d4/d6/f74 0 2024-05-27T09:29:25.002 INFO:tasks.workunit.client.0.smithi119.stdout:6/469: dread d0/d2/d1c/d21/f2b [0,4194304] 0 2024-05-27T09:29:25.003 INFO:tasks.workunit.client.1.smithi138.stdout:1/44: mknod d4/d8/c10 0 2024-05-27T09:29:25.004 INFO:tasks.workunit.client.1.smithi138.stdout:1/45: mknod d4/d8/c11 0 2024-05-27T09:29:25.004 INFO:tasks.workunit.client.1.smithi138.stdout:1/46: chown d4/d8/c11 15365 1 2024-05-27T09:29:25.005 INFO:tasks.workunit.client.1.smithi138.stdout:1/47: chown d4/fb 0 1 2024-05-27T09:29:25.007 INFO:tasks.workunit.client.1.smithi138.stdout:1/48: write d4/f5 [1930981,106535] 0 2024-05-27T09:29:25.009 INFO:tasks.workunit.client.0.smithi119.stdout:4/529: mkdir d4/de/d21/d40/d47/d5c/d9e 0 2024-05-27T09:29:25.009 INFO:tasks.workunit.client.1.smithi138.stdout:1/49: creat d4/f12 x:0 0 0 2024-05-27T09:29:25.010 INFO:tasks.workunit.client.1.smithi138.stdout:1/50: write d4/fb [793789,711] 0 2024-05-27T09:29:25.012 INFO:tasks.workunit.client.1.smithi138.stdout:1/51: mknod d4/c13 0 2024-05-27T09:29:25.014 INFO:tasks.workunit.client.0.smithi119.stdout:4/530: creat d4/de/d66/d87/f9f x:0 0 0 2024-05-27T09:29:25.016 INFO:tasks.workunit.client.1.smithi138.stdout:1/52: creat d4/d8/f14 x:0 0 0 2024-05-27T09:29:25.024 INFO:tasks.workunit.client.0.smithi119.stdout:4/531: getdents d4/de 0 2024-05-27T09:29:25.024 INFO:tasks.workunit.client.0.smithi119.stdout:4/532: write d4/de/d21/d40/d47/f49 [469350,49323] 0 2024-05-27T09:29:25.024 INFO:tasks.workunit.client.0.smithi119.stdout:4/533: fdatasync d4/d23/f35 0 2024-05-27T09:29:25.025 INFO:tasks.workunit.client.0.smithi119.stdout:5/530: dwrite d6/dd/f73 [0,4194304] 0 2024-05-27T09:29:25.025 INFO:tasks.workunit.client.0.smithi119.stdout:5/531: chown d6/db/de/f25 9071 1 2024-05-27T09:29:25.028 INFO:tasks.workunit.client.0.smithi119.stdout:3/651: dread db/d36/d45/d93/fb0 [0,4194304] 0 2024-05-27T09:29:25.029 INFO:tasks.workunit.client.0.smithi119.stdout:4/534: mknod d4/de/d21/d40/d47/d5c/ca0 0 2024-05-27T09:29:25.029 INFO:tasks.workunit.client.0.smithi119.stdout:4/535: fsync d4/d1c/f29 0 2024-05-27T09:29:25.030 INFO:tasks.workunit.client.0.smithi119.stdout:6/470: write d0/d2/d57/d37/f86 [3633511,106071] 0 2024-05-27T09:29:25.031 INFO:tasks.workunit.client.0.smithi119.stdout:1/613: dwrite d7/db/d52/d65/fa4 [0,4194304] 0 2024-05-27T09:29:25.031 INFO:tasks.workunit.client.0.smithi119.stdout:1/614: stat d7/db/lc 0 2024-05-27T09:29:25.031 INFO:tasks.workunit.client.0.smithi119.stdout:1/615: read - d7/db/d12/d15/fa1 zero size 2024-05-27T09:29:25.034 INFO:tasks.workunit.client.0.smithi119.stdout:4/536: unlink d4/d23/d24/f99 0 2024-05-27T09:29:25.034 INFO:tasks.workunit.client.0.smithi119.stdout:4/537: dread - d4/d23/d31/f93 zero size 2024-05-27T09:29:25.036 INFO:tasks.workunit.client.0.smithi119.stdout:1/616: unlink d7/db/d12/d15/f1d 0 2024-05-27T09:29:25.037 INFO:tasks.workunit.client.1.smithi138.stdout:1/53: rename d4/l9 to d4/l15 0 2024-05-27T09:29:25.037 INFO:tasks.workunit.client.1.smithi138.stdout:1/54: write d4/d8/f14 [981814,44727] 0 2024-05-27T09:29:25.038 INFO:tasks.workunit.client.0.smithi119.stdout:4/538: link d4/d6/d28/l3d d4/de/la1 0 2024-05-27T09:29:25.040 INFO:tasks.workunit.client.1.smithi138.stdout:1/55: creat d4/da/f16 x:0 0 0 2024-05-27T09:29:25.040 INFO:tasks.workunit.client.0.smithi119.stdout:1/617: symlink d7/dd/db0/lc1 0 2024-05-27T09:29:25.041 INFO:tasks.workunit.client.1.smithi138.stdout:9/20: dread f2 [0,4194304] 0 2024-05-27T09:29:25.041 INFO:tasks.workunit.client.1.smithi138.stdout:9/21: write f2 [4321408,107255] 0 2024-05-27T09:29:25.044 INFO:tasks.workunit.client.0.smithi119.stdout:1/618: mkdir d7/db6/dc2 0 2024-05-27T09:29:25.045 INFO:tasks.workunit.client.1.smithi138.stdout:9/22: link f4 d6/f9 0 2024-05-27T09:29:25.051 INFO:tasks.workunit.client.0.smithi119.stdout:5/532: write d6/dd/df/f75 [7334653,58497] 0 2024-05-27T09:29:25.052 INFO:tasks.workunit.client.0.smithi119.stdout:5/533: creat d6/dd/d39/d19/fb6 x:0 0 0 2024-05-27T09:29:25.053 INFO:tasks.workunit.client.1.smithi138.stdout:5/12: dwrite d3/f4 [0,4194304] 0 2024-05-27T09:29:25.053 INFO:tasks.workunit.client.1.smithi138.stdout:5/13: chown d3 57509257 1 2024-05-27T09:29:25.057 INFO:tasks.workunit.client.1.smithi138.stdout:5/14: mknod d3/c5 0 2024-05-27T09:29:25.060 INFO:tasks.workunit.client.1.smithi138.stdout:5/15: symlink d3/l6 0 2024-05-27T09:29:25.065 INFO:tasks.workunit.client.1.smithi138.stdout:2/22: dwrite d1/d7/f6 [0,4194304] 0 2024-05-27T09:29:25.066 INFO:tasks.workunit.client.1.smithi138.stdout:2/23: fsync d1/d7/f4 0 2024-05-27T09:29:25.067 INFO:tasks.workunit.client.1.smithi138.stdout:7/6: dwrite d0/f1 [0,4194304] 0 2024-05-27T09:29:25.070 INFO:tasks.workunit.client.1.smithi138.stdout:1/56: rmdir d4/da 39 2024-05-27T09:29:25.074 INFO:tasks.workunit.client.1.smithi138.stdout:1/57: link d4/l15 d4/d8/l17 0 2024-05-27T09:29:25.076 INFO:tasks.workunit.client.1.smithi138.stdout:5/16: dread f1 [0,4194304] 0 2024-05-27T09:29:25.076 INFO:tasks.workunit.client.1.smithi138.stdout:5/17: fsync d3/f4 0 2024-05-27T09:29:25.078 INFO:tasks.workunit.client.0.smithi119.stdout:5/534: dread d6/f1d [0,4194304] 0 2024-05-27T09:29:25.079 INFO:tasks.workunit.client.1.smithi138.stdout:1/58: fsync d4/fb 0 2024-05-27T09:29:25.081 INFO:tasks.workunit.client.1.smithi138.stdout:5/18: rmdir d3 39 2024-05-27T09:29:25.081 INFO:tasks.workunit.client.0.smithi119.stdout:2/514: dwrite d2/da/d15/d53/fa6 [0,4194304] 0 2024-05-27T09:29:25.081 INFO:tasks.workunit.client.0.smithi119.stdout:2/515: write d2/f7d [688270,84736] 0 2024-05-27T09:29:25.086 INFO:tasks.workunit.client.0.smithi119.stdout:2/516: symlink d2/da/d2c/lab 0 2024-05-27T09:29:25.090 INFO:tasks.workunit.client.1.smithi138.stdout:6/15: dwrite f2 [0,4194304] 0 2024-05-27T09:29:25.090 INFO:tasks.workunit.client.1.smithi138.stdout:6/16: truncate d3/f5 72939 0 2024-05-27T09:29:25.093 INFO:tasks.workunit.client.1.smithi138.stdout:7/7: read d0/f1 [972064,75495] 0 2024-05-27T09:29:25.094 INFO:tasks.workunit.client.0.smithi119.stdout:3/652: dread db/f24 [0,4194304] 0 2024-05-27T09:29:25.094 INFO:tasks.workunit.client.0.smithi119.stdout:3/653: creat db/d36/d45/fd0 x:0 0 0 2024-05-27T09:29:25.095 INFO:tasks.workunit.client.1.smithi138.stdout:5/19: creat d3/f7 x:0 0 0 2024-05-27T09:29:25.095 INFO:tasks.workunit.client.0.smithi119.stdout:3/654: write db/d36/d45/d49/d9d/fb5 [75798,46308] 0 2024-05-27T09:29:25.095 INFO:tasks.workunit.client.0.smithi119.stdout:3/655: chown db/d36/d45/d4c/d62/d13/d33/fa7 253 1 2024-05-27T09:29:25.096 INFO:tasks.workunit.client.0.smithi119.stdout:2/517: mkdir d2/da/d15/d53/dac 0 2024-05-27T09:29:25.096 INFO:tasks.workunit.client.0.smithi119.stdout:2/518: stat d2/f48 0 2024-05-27T09:29:25.096 INFO:tasks.workunit.client.0.smithi119.stdout:2/519: stat d2/d1a/c6d 0 2024-05-27T09:29:25.098 INFO:tasks.workunit.client.1.smithi138.stdout:7/8: unlink d0/f1 0 2024-05-27T09:29:25.098 INFO:tasks.workunit.client.0.smithi119.stdout:6/471: dread d0/d2/f67 [0,4194304] 0 2024-05-27T09:29:25.098 INFO:tasks.workunit.client.0.smithi119.stdout:6/472: readlink d0/l13 0 2024-05-27T09:29:25.102 INFO:tasks.workunit.client.0.smithi119.stdout:3/656: write db/dc9/d98/f73 [325318,9628] 0 2024-05-27T09:29:25.103 INFO:tasks.workunit.client.1.smithi138.stdout:7/9: mknod d0/c2 0 2024-05-27T09:29:25.105 INFO:tasks.workunit.client.1.smithi138.stdout:7/10: rename d0/c2 to d0/c3 0 2024-05-27T09:29:25.105 INFO:tasks.workunit.client.1.smithi138.stdout:7/11: write - no filename 2024-05-27T09:29:25.105 INFO:tasks.workunit.client.1.smithi138.stdout:7/12: write - no filename 2024-05-27T09:29:25.105 INFO:tasks.workunit.client.1.smithi138.stdout:7/13: dwrite - no filename 2024-05-27T09:29:25.109 INFO:tasks.workunit.client.1.smithi138.stdout:7/14: creat d0/f4 x:0 0 0 2024-05-27T09:29:25.109 INFO:tasks.workunit.client.1.smithi138.stdout:7/15: write d0/f4 [178956,45547] 0 2024-05-27T09:29:25.109 INFO:tasks.workunit.client.1.smithi138.stdout:7/16: rename d0 to d0/d5 22 2024-05-27T09:29:25.117 INFO:tasks.workunit.client.1.smithi138.stdout:2/24: dread d1/d7/f6 [0,4194304] 0 2024-05-27T09:29:25.119 INFO:tasks.workunit.client.1.smithi138.stdout:2/25: rmdir d1/d7 39 2024-05-27T09:29:25.138 INFO:tasks.workunit.client.0.smithi119.stdout:4/539: dwrite d4/d6/d28/f7a [0,4194304] 0 2024-05-27T09:29:25.140 INFO:tasks.workunit.client.0.smithi119.stdout:1/619: dwrite d7/db/d12/d26/d39/fb1 [0,4194304] 0 2024-05-27T09:29:25.141 INFO:tasks.workunit.client.1.smithi138.stdout:9/23: dwrite f4 [0,4194304] 0 2024-05-27T09:29:25.148 INFO:tasks.workunit.client.0.smithi119.stdout:1/620: creat d7/dd/db0/fc3 x:0 0 0 2024-05-27T09:29:25.154 INFO:tasks.workunit.client.0.smithi119.stdout:6/473: write d0/d2/d4e/d5a/f95 [3215057,105487] 0 2024-05-27T09:29:25.154 INFO:tasks.workunit.client.0.smithi119.stdout:6/474: fdatasync d0/d2/d57/f90 0 2024-05-27T09:29:25.154 INFO:tasks.workunit.client.0.smithi119.stdout:6/475: read d0/d2/d1c/d21/f3c [2276173,57383] 0 2024-05-27T09:29:25.155 INFO:tasks.workunit.client.0.smithi119.stdout:5/535: dwrite d6/db/de/d27/f50 [0,4194304] 0 2024-05-27T09:29:25.158 INFO:tasks.workunit.client.0.smithi119.stdout:0/501: sync 2024-05-27T09:29:25.158 INFO:tasks.workunit.client.0.smithi119.stdout:7/557: sync 2024-05-27T09:29:25.158 INFO:tasks.workunit.client.0.smithi119.stdout:0/502: chown d2/d6/d9/d12/c3f 92 1 2024-05-27T09:29:25.158 INFO:tasks.workunit.client.0.smithi119.stdout:7/558: creat d7/df/d19/d97/d37/fac x:0 0 0 2024-05-27T09:29:25.160 INFO:tasks.workunit.client.0.smithi119.stdout:1/621: creat d7/db/d8f/fc4 x:0 0 0 2024-05-27T09:29:25.160 INFO:tasks.workunit.client.0.smithi119.stdout:8/502: sync 2024-05-27T09:29:25.160 INFO:tasks.workunit.client.0.smithi119.stdout:6/476: link d0/d2/d1c/l71 d0/d2/d1c/l9f 0 2024-05-27T09:29:25.160 INFO:tasks.workunit.client.0.smithi119.stdout:1/622: chown d7/db 3437438 1 2024-05-27T09:29:25.160 INFO:tasks.workunit.client.0.smithi119.stdout:6/477: fsync d0/d11/f7d 0 2024-05-27T09:29:25.160 INFO:tasks.workunit.client.0.smithi119.stdout:6/478: fdatasync d0/d2/d4e/f1f 0 2024-05-27T09:29:25.161 INFO:tasks.workunit.client.0.smithi119.stdout:9/411: sync 2024-05-27T09:29:25.161 INFO:tasks.workunit.client.0.smithi119.stdout:9/412: chown d18/d19/d1b/d2a 269 1 2024-05-27T09:29:25.164 INFO:tasks.workunit.client.0.smithi119.stdout:7/559: mkdir d7/df/d36/d42/d64/dad 0 2024-05-27T09:29:25.164 INFO:tasks.workunit.client.0.smithi119.stdout:1/623: creat d7/dd/d25/fc5 x:0 0 0 2024-05-27T09:29:25.166 INFO:tasks.workunit.client.0.smithi119.stdout:2/520: write d2/f2d [1491060,9648] 0 2024-05-27T09:29:25.167 INFO:tasks.workunit.client.0.smithi119.stdout:3/657: dread f6 [0,4194304] 0 2024-05-27T09:29:25.170 INFO:tasks.workunit.client.0.smithi119.stdout:9/413: fdatasync d18/f30 0 2024-05-27T09:29:25.170 INFO:tasks.workunit.client.0.smithi119.stdout:6/479: creat d0/d2/d1c/fa0 x:0 0 0 2024-05-27T09:29:25.170 INFO:tasks.workunit.client.0.smithi119.stdout:6/480: chown d0/d2/d1c/c8e 362185914 1 2024-05-27T09:29:25.171 INFO:tasks.workunit.client.0.smithi119.stdout:0/503: rmdir d2/d6/d9/d24/d30/d4a 39 2024-05-27T09:29:25.172 INFO:tasks.workunit.client.0.smithi119.stdout:7/560: rmdir d7/df/d19/d4d/d89 39 2024-05-27T09:29:25.174 INFO:tasks.workunit.client.0.smithi119.stdout:2/521: symlink d2/da/d2c/lad 0 2024-05-27T09:29:25.175 INFO:tasks.workunit.client.0.smithi119.stdout:2/522: creat d2/fae x:0 0 0 2024-05-27T09:29:25.177 INFO:tasks.workunit.client.0.smithi119.stdout:3/658: mkdir db/dc9/d98/db9/dd1 0 2024-05-27T09:29:25.178 INFO:tasks.workunit.client.0.smithi119.stdout:6/481: rename d0/f52 to d0/d2/d57/d37/d74/fa1 0 2024-05-27T09:29:25.179 INFO:tasks.workunit.client.0.smithi119.stdout:9/414: unlink d18/d19/d1b/d2a/d63/l75 0 2024-05-27T09:29:25.180 INFO:tasks.workunit.client.0.smithi119.stdout:0/504: symlink d2/d6/d9/d12/d4b/la1 0 2024-05-27T09:29:25.184 INFO:tasks.workunit.client.0.smithi119.stdout:7/561: truncate d7/d1a/f59 3439542 0 2024-05-27T09:29:25.184 INFO:tasks.workunit.client.0.smithi119.stdout:7/562: write d7/d6a/f8f [661279,51076] 0 2024-05-27T09:29:25.186 INFO:tasks.workunit.client.0.smithi119.stdout:3/659: mkdir db/d7f/dd2 0 2024-05-27T09:29:25.186 INFO:tasks.workunit.client.0.smithi119.stdout:3/660: write db/dc9/d98/fa5 [1780055,61161] 0 2024-05-27T09:29:25.196 INFO:tasks.workunit.client.0.smithi119.stdout:1/624: getdents d7/dd/d25/d60/d61 0 2024-05-27T09:29:25.196 INFO:tasks.workunit.client.0.smithi119.stdout:1/625: chown d7/dd/d25/d60/d61/c89 2104 1 2024-05-27T09:29:25.196 INFO:tasks.workunit.client.0.smithi119.stdout:1/626: chown d7/dd/d25/d60/d61/da5 57874 1 2024-05-27T09:29:25.196 INFO:tasks.workunit.client.0.smithi119.stdout:9/415: truncate d18/d19/f36 1260690 0 2024-05-27T09:29:25.196 INFO:tasks.workunit.client.0.smithi119.stdout:9/416: fsync d18/d19/d1b/d29/f47 0 2024-05-27T09:29:25.196 INFO:tasks.workunit.client.0.smithi119.stdout:8/503: write d3/d8/d1b/d46/f7c [1008800,18491] 0 2024-05-27T09:29:25.196 INFO:tasks.workunit.client.0.smithi119.stdout:7/563: mknod d7/df/d19/d4d/cae 0 2024-05-27T09:29:25.196 INFO:tasks.workunit.client.0.smithi119.stdout:7/564: write d7/df/d19/d4d/f71 [2465136,76043] 0 2024-05-27T09:29:25.197 INFO:tasks.workunit.client.1.smithi138.stdout:6/17: dwrite f2 [0,4194304] 0 2024-05-27T09:29:25.197 INFO:tasks.workunit.client.1.smithi138.stdout:6/18: creat d3/f6 x:0 0 0 2024-05-27T09:29:25.203 INFO:tasks.workunit.client.0.smithi119.stdout:9/417: creat d18/d19/f88 x:0 0 0 2024-05-27T09:29:25.204 INFO:tasks.workunit.client.1.smithi138.stdout:6/19: mknod d3/c7 0 2024-05-27T09:29:25.204 INFO:tasks.workunit.client.1.smithi138.stdout:9/24: dread f3 [0,4194304] 0 2024-05-27T09:29:25.204 INFO:tasks.workunit.client.1.smithi138.stdout:9/25: chown f3 199213148 1 2024-05-27T09:29:25.204 INFO:tasks.workunit.client.1.smithi138.stdout:6/20: write d3/f6 [319936,90586] 0 2024-05-27T09:29:25.208 INFO:tasks.workunit.client.0.smithi119.stdout:8/504: creat d3/d6e/d8e/d9b/fa5 x:0 0 0 2024-05-27T09:29:25.209 INFO:tasks.workunit.client.1.smithi138.stdout:9/26: write f2 [938672,88736] 0 2024-05-27T09:29:25.210 INFO:tasks.workunit.client.0.smithi119.stdout:7/565: symlink d7/d1a/d40/d44/d67/laf 0 2024-05-27T09:29:25.213 INFO:tasks.workunit.client.0.smithi119.stdout:9/418: rename fa to d18/d19/d1b/d40/d7b/f89 0 2024-05-27T09:29:25.216 INFO:tasks.workunit.client.1.smithi138.stdout:9/27: mkdir d6/da 0 2024-05-27T09:29:25.216 INFO:tasks.workunit.client.1.smithi138.stdout:5/20: dwrite d3/f4 [0,4194304] 0 2024-05-27T09:29:25.217 INFO:tasks.workunit.client.1.smithi138.stdout:1/59: dwrite d4/d8/f14 [0,4194304] 0 2024-05-27T09:29:25.227 INFO:tasks.workunit.client.1.smithi138.stdout:5/21: symlink d3/l8 0 2024-05-27T09:29:25.237 INFO:tasks.workunit.client.0.smithi119.stdout:0/505: dread d2/d6/d9/d12/f47 [0,4194304] 0 2024-05-27T09:29:25.237 INFO:tasks.workunit.client.0.smithi119.stdout:0/506: creat d2/d6/d75/fa2 x:0 0 0 2024-05-27T09:29:25.237 INFO:tasks.workunit.client.0.smithi119.stdout:9/419: rename d18/f80 to d18/d19/d1b/d2a/f8a 0 2024-05-27T09:29:25.242 INFO:tasks.workunit.client.0.smithi119.stdout:3/661: dread db/dc9/d98/d7a/f5f [4194304,4194304] 0 2024-05-27T09:29:25.242 INFO:tasks.workunit.client.0.smithi119.stdout:3/662: write db/f69 [333123,26696] 0 2024-05-27T09:29:25.244 INFO:tasks.workunit.client.0.smithi119.stdout:9/420: creat d18/f8b x:0 0 0 2024-05-27T09:29:25.247 INFO:tasks.workunit.client.0.smithi119.stdout:7/566: rename d7/df/d19/d1d/c25 to d7/df/d36/d42/d64/cb0 0 2024-05-27T09:29:25.254 INFO:tasks.workunit.client.0.smithi119.stdout:7/567: rename d7/d1a/l63 to d7/df/d19/d97/d37/lb1 0 2024-05-27T09:29:25.255 INFO:tasks.workunit.client.1.smithi138.stdout:6/21: truncate f2 265431 0 2024-05-27T09:29:25.261 INFO:tasks.workunit.client.0.smithi119.stdout:4/540: dwrite d4/d1c/d39/f5d [4194304,4194304] 0 2024-05-27T09:29:25.261 INFO:tasks.workunit.client.0.smithi119.stdout:4/541: creat d4/de/d66/d87/fa2 x:0 0 0 2024-05-27T09:29:25.263 INFO:tasks.workunit.client.0.smithi119.stdout:5/536: dwrite d6/d2f/f51 [0,4194304] 0 2024-05-27T09:29:25.264 INFO:tasks.workunit.client.0.smithi119.stdout:5/537: stat d6/dd/d46/d47/d4b/f59 0 2024-05-27T09:29:25.265 INFO:tasks.workunit.client.0.smithi119.stdout:9/421: symlink d18/d19/l8c 0 2024-05-27T09:29:25.265 INFO:tasks.workunit.client.0.smithi119.stdout:9/422: fdatasync d18/d19/d1b/d40/d44/f4a 0 2024-05-27T09:29:25.270 INFO:tasks.workunit.client.0.smithi119.stdout:7/568: mknod d7/df/d36/d42/d64/cb2 0 2024-05-27T09:29:25.271 INFO:tasks.workunit.client.0.smithi119.stdout:7/569: truncate d7/df/d19/d97/d92/fab 346070 0 2024-05-27T09:29:25.274 INFO:tasks.workunit.client.1.smithi138.stdout:1/60: write - open d4/f7 failed 14 2024-05-27T09:29:25.276 INFO:tasks.workunit.client.1.smithi138.stdout:5/22: write d3/f4 [2783844,36611] 0 2024-05-27T09:29:25.276 INFO:tasks.workunit.client.0.smithi119.stdout:4/542: mkdir d4/de/d21/d40/d47/da3 0 2024-05-27T09:29:25.279 INFO:tasks.workunit.client.1.smithi138.stdout:5/23: mknod d3/c9 0 2024-05-27T09:29:25.279 INFO:tasks.workunit.client.1.smithi138.stdout:5/24: write d3/f4 [2017110,128235] 0 2024-05-27T09:29:25.279 INFO:tasks.workunit.client.1.smithi138.stdout:5/25: creat d3/fa x:0 0 0 2024-05-27T09:29:25.279 INFO:tasks.workunit.client.1.smithi138.stdout:5/26: chown d3/l8 1 1 2024-05-27T09:29:25.279 INFO:tasks.workunit.client.1.smithi138.stdout:5/27: creat d3/fb x:0 0 0 2024-05-27T09:29:25.281 INFO:tasks.workunit.client.1.smithi138.stdout:1/61: read f3 [657202,67437] 0 2024-05-27T09:29:25.289 INFO:tasks.workunit.client.0.smithi119.stdout:5/538: mkdir d6/dd/d39/d19/d7d/db7 0 2024-05-27T09:29:25.290 INFO:tasks.workunit.client.1.smithi138.stdout:1/62: creat d4/f18 x:0 0 0 2024-05-27T09:29:25.290 INFO:tasks.workunit.client.1.smithi138.stdout:1/63: chown d4/d8/l17 1 1 2024-05-27T09:29:25.291 INFO:tasks.workunit.client.0.smithi119.stdout:6/482: dwrite d0/d2/f31 [0,4194304] 0 2024-05-27T09:29:25.300 INFO:tasks.workunit.client.0.smithi119.stdout:7/570: creat d7/df/d36/fb3 x:0 0 0 2024-05-27T09:29:25.302 INFO:tasks.workunit.client.1.smithi138.stdout:1/64: dread d4/fb [0,4194304] 0 2024-05-27T09:29:25.305 INFO:tasks.workunit.client.1.smithi138.stdout:1/65: write d4/f5 [3051110,94010] 0 2024-05-27T09:29:25.307 INFO:tasks.workunit.client.0.smithi119.stdout:8/505: dwrite d3/de/f68 [4194304,4194304] 0 2024-05-27T09:29:25.318 INFO:tasks.workunit.client.0.smithi119.stdout:5/539: rmdir d6/dd/d39/d19/d7d 39 2024-05-27T09:29:25.323 INFO:tasks.workunit.client.0.smithi119.stdout:9/423: rename d18/l51 to d18/d19/d1b/d40/d44/l8d 0 2024-05-27T09:29:25.324 INFO:tasks.workunit.client.1.smithi138.stdout:9/28: dwrite f4 [0,4194304] 0 2024-05-27T09:29:25.324 INFO:tasks.workunit.client.1.smithi138.stdout:9/29: truncate f1 948720 0 2024-05-27T09:29:25.326 INFO:tasks.workunit.client.0.smithi119.stdout:6/483: rmdir d0/d2/d57/d37/d74/d65 39 2024-05-27T09:29:25.326 INFO:tasks.workunit.client.1.smithi138.stdout:6/22: dwrite d3/f6 [0,4194304] 0 2024-05-27T09:29:25.328 INFO:tasks.workunit.client.0.smithi119.stdout:7/571: unlink d7/df/d19/d97/f2f 0 2024-05-27T09:29:25.328 INFO:tasks.workunit.client.0.smithi119.stdout:7/572: fsync d7/df/d36/f49 0 2024-05-27T09:29:25.328 INFO:tasks.workunit.client.0.smithi119.stdout:7/573: readlink d7/d1a/d40/d44/d67/laf 0 2024-05-27T09:29:25.328 INFO:tasks.workunit.client.1.smithi138.stdout:6/23: mknod d3/c8 0 2024-05-27T09:29:25.329 INFO:tasks.workunit.client.1.smithi138.stdout:6/24: fdatasync d3/f6 0 2024-05-27T09:29:25.329 INFO:tasks.workunit.client.1.smithi138.stdout:6/25: fdatasync f2 0 2024-05-27T09:29:25.343 INFO:tasks.workunit.client.0.smithi119.stdout:0/507: dwrite d2/d1a/d20/f8b [0,4194304] 0 2024-05-27T09:29:25.356 INFO:tasks.workunit.client.0.smithi119.stdout:0/508: write d2/d6/d9/d12/f32 [579547,56594] 0 2024-05-27T09:29:25.366 INFO:tasks.workunit.client.1.smithi138.stdout:0/18: sync 2024-05-27T09:29:25.366 INFO:tasks.workunit.client.1.smithi138.stdout:8/1: sync 2024-05-27T09:29:25.366 INFO:tasks.workunit.client.1.smithi138.stdout:8/2: truncate - no filename 2024-05-27T09:29:25.366 INFO:tasks.workunit.client.1.smithi138.stdout:8/3: rename - no filename 2024-05-27T09:29:25.366 INFO:tasks.workunit.client.1.smithi138.stdout:8/4: dwrite - no filename 2024-05-27T09:29:25.366 INFO:tasks.workunit.client.1.smithi138.stdout:8/5: truncate - no filename 2024-05-27T09:29:25.366 INFO:tasks.workunit.client.1.smithi138.stdout:8/6: chown . 6688 1 2024-05-27T09:29:25.366 INFO:tasks.workunit.client.1.smithi138.stdout:8/7: stat - no entries 2024-05-27T09:29:25.367 INFO:tasks.workunit.client.0.smithi119.stdout:9/424: dread fe [0,4194304] 0 2024-05-27T09:29:25.372 INFO:tasks.workunit.client.0.smithi119.stdout:0/509: write d2/d6/d9/d12/d4b/d59/f58 [2942145,123762] 0 2024-05-27T09:29:25.372 INFO:tasks.workunit.client.0.smithi119.stdout:0/510: write d2/f49 [3191843,70650] 0 2024-05-27T09:29:25.373 INFO:tasks.workunit.client.1.smithi138.stdout:8/8: mknod c0 0 2024-05-27T09:29:25.374 INFO:tasks.workunit.client.1.smithi138.stdout:0/19: getdents . 0 2024-05-27T09:29:25.374 INFO:tasks.workunit.client.1.smithi138.stdout:0/20: dread - f5 zero size 2024-05-27T09:29:25.377 INFO:tasks.workunit.client.1.smithi138.stdout:8/9: creat f1 x:0 0 0 2024-05-27T09:29:25.378 INFO:tasks.workunit.client.1.smithi138.stdout:8/10: write f1 [664013,93843] 0 2024-05-27T09:29:25.378 INFO:tasks.workunit.client.0.smithi119.stdout:4/543: rmdir d4/d23/d31 39 2024-05-27T09:29:25.378 INFO:tasks.workunit.client.1.smithi138.stdout:0/21: creat f8 x:0 0 0 2024-05-27T09:29:25.380 INFO:tasks.workunit.client.0.smithi119.stdout:4/544: stat d4/d4e/l59 0 2024-05-27T09:29:25.380 INFO:tasks.workunit.client.0.smithi119.stdout:4/545: write d4/de/d66/d87/f9f [218191,70113] 0 2024-05-27T09:29:25.380 INFO:tasks.workunit.client.0.smithi119.stdout:8/506: symlink d3/d8/d1b/d46/da4/la6 0 2024-05-27T09:29:25.381 INFO:tasks.workunit.client.1.smithi138.stdout:0/22: unlink l3 0 2024-05-27T09:29:25.382 INFO:tasks.workunit.client.0.smithi119.stdout:6/484: mknod d0/d2/d77/ca2 0 2024-05-27T09:29:25.382 INFO:tasks.workunit.client.0.smithi119.stdout:6/485: truncate d0/d2/d57/f90 1008370 0 2024-05-27T09:29:25.382 INFO:tasks.workunit.client.0.smithi119.stdout:7/574: rmdir d7/d5f 39 2024-05-27T09:29:25.384 INFO:tasks.workunit.client.1.smithi138.stdout:0/23: symlink l9 0 2024-05-27T09:29:25.387 INFO:tasks.workunit.client.0.smithi119.stdout:2/523: dwrite d2/f7d [0,4194304] 0 2024-05-27T09:29:25.387 INFO:tasks.workunit.client.0.smithi119.stdout:2/524: write d2/da/d15/f80 [1488631,42241] 0 2024-05-27T09:29:25.387 INFO:tasks.workunit.client.0.smithi119.stdout:2/525: chown d2/c42 596153 1 2024-05-27T09:29:25.387 INFO:tasks.workunit.client.0.smithi119.stdout:2/526: stat d2/d1a/l86 0 2024-05-27T09:29:25.387 INFO:tasks.workunit.client.0.smithi119.stdout:2/527: fdatasync d2/f47 0 2024-05-27T09:29:25.387 INFO:tasks.workunit.client.1.smithi138.stdout:0/24: mkdir da 0 2024-05-27T09:29:25.392 INFO:tasks.workunit.client.0.smithi119.stdout:9/425: dread d18/d19/d1b/d29/f43 [0,4194304] 0 2024-05-27T09:29:25.392 INFO:tasks.workunit.client.0.smithi119.stdout:9/426: readlink d18/d19/d1b/d40/d44/l8d 0 2024-05-27T09:29:25.392 INFO:tasks.workunit.client.1.smithi138.stdout:5/28: dwrite d3/f7 [0,4194304] 0 2024-05-27T09:29:25.392 INFO:tasks.workunit.client.1.smithi138.stdout:5/29: chown d3/c5 7438 1 2024-05-27T09:29:25.392 INFO:tasks.workunit.client.1.smithi138.stdout:5/30: truncate d3/fa 801249 0 2024-05-27T09:29:25.393 INFO:tasks.workunit.client.0.smithi119.stdout:9/427: write d18/d19/f85 [658624,120024] 0 2024-05-27T09:29:25.393 INFO:tasks.workunit.client.0.smithi119.stdout:4/546: symlink d4/de/d9b/la4 0 2024-05-27T09:29:25.393 INFO:tasks.workunit.client.0.smithi119.stdout:8/507: truncate d3/d8/d4d/d58/f82 1944832 0 2024-05-27T09:29:25.393 INFO:tasks.workunit.client.0.smithi119.stdout:6/486: creat d0/d2/d4e/d5a/d76/fa3 x:0 0 0 2024-05-27T09:29:25.397 INFO:tasks.workunit.client.0.smithi119.stdout:9/428: creat d18/d19/d1b/d40/f8e x:0 0 0 2024-05-27T09:29:25.398 INFO:tasks.workunit.client.0.smithi119.stdout:9/429: truncate d18/d19/d1b/d40/f60 506372 0 2024-05-27T09:29:25.400 INFO:tasks.workunit.client.0.smithi119.stdout:3/663: dwrite db/fb1 [0,4194304] 0 2024-05-27T09:29:25.400 INFO:tasks.workunit.client.0.smithi119.stdout:0/511: rmdir d2/d1a/d20 39 2024-05-27T09:29:25.405 INFO:tasks.workunit.client.0.smithi119.stdout:4/547: rename d4/de/d21/d40/d47/d92/c9a to d4/de/d21/ca5 0 2024-05-27T09:29:25.406 INFO:tasks.workunit.client.0.smithi119.stdout:6/487: mknod d0/d19/ca4 0 2024-05-27T09:29:25.410 INFO:tasks.workunit.client.0.smithi119.stdout:2/528: dread d2/d1a/f30 [0,4194304] 0 2024-05-27T09:29:25.413 INFO:tasks.workunit.client.0.smithi119.stdout:4/548: symlink d4/d1c/d39/la6 0 2024-05-27T09:29:25.414 INFO:tasks.workunit.client.0.smithi119.stdout:4/549: write d4/d6/d28/f7b [1010648,50431] 0 2024-05-27T09:29:25.419 INFO:tasks.workunit.client.0.smithi119.stdout:6/488: symlink d0/d2/d77/la5 0 2024-05-27T09:29:25.419 INFO:tasks.workunit.client.0.smithi119.stdout:3/664: link db/d36/fbc db/fd3 0 2024-05-27T09:29:25.419 INFO:tasks.workunit.client.0.smithi119.stdout:6/489: truncate d0/f84 860362 0 2024-05-27T09:29:25.420 INFO:tasks.workunit.client.0.smithi119.stdout:6/490: creat d0/d19/fa6 x:0 0 0 2024-05-27T09:29:25.420 INFO:tasks.workunit.client.0.smithi119.stdout:2/529: symlink d2/da/d15/d33/d69/laf 0 2024-05-27T09:29:25.421 INFO:tasks.workunit.client.0.smithi119.stdout:4/550: mknod d4/d6/d3e/ca7 0 2024-05-27T09:29:25.426 INFO:tasks.workunit.client.1.smithi138.stdout:5/31: write f0 [2685703,28475] 0 2024-05-27T09:29:25.426 INFO:tasks.workunit.client.0.smithi119.stdout:5/540: dwrite d6/f1d [0,4194304] 0 2024-05-27T09:29:25.427 INFO:tasks.workunit.client.1.smithi138.stdout:3/9: sync 2024-05-27T09:29:25.427 INFO:tasks.workunit.client.1.smithi138.stdout:3/10: chown d0 489455 1 2024-05-27T09:29:25.427 INFO:tasks.workunit.client.1.smithi138.stdout:3/11: write - no filename 2024-05-27T09:29:25.427 INFO:tasks.workunit.client.1.smithi138.stdout:3/12: write - no filename 2024-05-27T09:29:25.427 INFO:tasks.workunit.client.1.smithi138.stdout:3/13: dread - no filename 2024-05-27T09:29:25.427 INFO:tasks.workunit.client.1.smithi138.stdout:3/14: write - no filename 2024-05-27T09:29:25.427 INFO:tasks.workunit.client.1.smithi138.stdout:1/66: dwrite f3 [0,4194304] 0 2024-05-27T09:29:25.428 INFO:tasks.workunit.client.1.smithi138.stdout:1/67: write d4/fb [1641805,70274] 0 2024-05-27T09:29:25.428 INFO:tasks.workunit.client.1.smithi138.stdout:1/68: stat d4/d8/c11 0 2024-05-27T09:29:25.429 INFO:tasks.workunit.client.0.smithi119.stdout:3/665: symlink db/d7f/da9/ld4 0 2024-05-27T09:29:25.431 INFO:tasks.workunit.client.0.smithi119.stdout:0/512: read d2/d1a/d20/f8b [1033082,42636] 0 2024-05-27T09:29:25.431 INFO:tasks.workunit.client.0.smithi119.stdout:0/513: truncate d2/d6/d9/d12/f77 25850 0 2024-05-27T09:29:25.432 INFO:tasks.workunit.client.1.smithi138.stdout:6/26: dwrite d3/f6 [0,4194304] 0 2024-05-27T09:29:25.432 INFO:tasks.workunit.client.1.smithi138.stdout:7/17: sync 2024-05-27T09:29:25.432 INFO:tasks.workunit.client.1.smithi138.stdout:2/26: sync 2024-05-27T09:29:25.432 INFO:tasks.workunit.client.1.smithi138.stdout:4/13: sync 2024-05-27T09:29:25.432 INFO:tasks.workunit.client.1.smithi138.stdout:8/11: dread f1 [0,4194304] 0 2024-05-27T09:29:25.432 INFO:tasks.workunit.client.1.smithi138.stdout:8/12: truncate f1 1114571 0 2024-05-27T09:29:25.432 INFO:tasks.workunit.client.1.smithi138.stdout:3/15: mkdir d0/d2 0 2024-05-27T09:29:25.432 INFO:tasks.workunit.client.1.smithi138.stdout:5/32: rename d3/fb to d3/fc 0 2024-05-27T09:29:25.432 INFO:tasks.workunit.client.0.smithi119.stdout:6/491: rename d0/d2/d77/l7a to d0/d2/d4e/la7 0 2024-05-27T09:29:25.433 INFO:tasks.workunit.client.1.smithi138.stdout:6/27: mknod d3/c9 0 2024-05-27T09:29:25.433 INFO:tasks.workunit.client.1.smithi138.stdout:7/18: mknod d0/c6 0 2024-05-27T09:29:25.433 INFO:tasks.workunit.client.1.smithi138.stdout:2/27: mknod d1/c8 0 2024-05-27T09:29:25.436 INFO:tasks.workunit.client.0.smithi119.stdout:8/508: write d3/de/f68 [3151725,101356] 0 2024-05-27T09:29:25.438 INFO:tasks.workunit.client.1.smithi138.stdout:4/14: mknod c4 0 2024-05-27T09:29:25.438 INFO:tasks.workunit.client.1.smithi138.stdout:4/15: fsync f0 0 2024-05-27T09:29:25.440 INFO:tasks.workunit.client.0.smithi119.stdout:2/530: mkdir d2/da/d15/d33/d89/db0 0 2024-05-27T09:29:25.440 INFO:tasks.workunit.client.0.smithi119.stdout:4/551: mkdir d4/d23/d24/d5b/da8 0 2024-05-27T09:29:25.447 INFO:tasks.workunit.client.1.smithi138.stdout:8/13: dread f1 [0,4194304] 0 2024-05-27T09:29:25.447 INFO:tasks.workunit.client.1.smithi138.stdout:3/16: rename d0/c1 to d0/d2/c3 0 2024-05-27T09:29:25.447 INFO:tasks.workunit.client.1.smithi138.stdout:3/17: dread - no filename 2024-05-27T09:29:25.447 INFO:tasks.workunit.client.1.smithi138.stdout:8/14: truncate f1 1382920 0 2024-05-27T09:29:25.447 INFO:tasks.workunit.client.1.smithi138.stdout:8/15: fdatasync f1 0 2024-05-27T09:29:25.455 INFO:tasks.workunit.client.1.smithi138.stdout:5/33: dread d3/f4 [0,4194304] 0 2024-05-27T09:29:25.455 INFO:tasks.workunit.client.1.smithi138.stdout:4/16: creat f5 x:0 0 0 2024-05-27T09:29:25.458 INFO:tasks.workunit.client.0.smithi119.stdout:5/541: stat d6/dd/df/d41/l4d 0 2024-05-27T09:29:25.458 INFO:tasks.workunit.client.0.smithi119.stdout:5/542: chown d6/dd/f64 3672 1 2024-05-27T09:29:25.458 INFO:tasks.workunit.client.1.smithi138.stdout:7/19: mkdir d0/d7 0 2024-05-27T09:29:25.459 INFO:tasks.workunit.client.1.smithi138.stdout:7/20: rename d0/d7 to d0/d7/d8 22 2024-05-27T09:29:25.459 INFO:tasks.workunit.client.1.smithi138.stdout:2/28: dread d1/d7/f6 [0,4194304] 0 2024-05-27T09:29:25.459 INFO:tasks.workunit.client.1.smithi138.stdout:1/69: dread d4/f5 [0,4194304] 0 2024-05-27T09:29:25.459 INFO:tasks.workunit.client.1.smithi138.stdout:1/70: chown d4/d8/c11 32109815 1 2024-05-27T09:29:25.459 INFO:tasks.workunit.client.1.smithi138.stdout:1/71: truncate d4/f18 191252 0 2024-05-27T09:29:25.461 INFO:tasks.workunit.client.0.smithi119.stdout:3/666: creat db/dc9/fd5 x:0 0 0 2024-05-27T09:29:25.462 INFO:tasks.workunit.client.0.smithi119.stdout:0/514: mkdir d2/d6/d9/d12/da3 0 2024-05-27T09:29:25.465 INFO:tasks.workunit.client.0.smithi119.stdout:6/492: mknod d0/d2/d1c/ca8 0 2024-05-27T09:29:25.468 INFO:tasks.workunit.client.0.smithi119.stdout:6/493: creat d0/d2/d57/d37/d74/fa9 x:0 0 0 2024-05-27T09:29:25.471 INFO:tasks.workunit.client.0.smithi119.stdout:8/509: link d3/de/f1c d3/d8/d1b/fa7 0 2024-05-27T09:29:25.471 INFO:tasks.workunit.client.0.smithi119.stdout:0/515: read d2/d6/d9/d24/d30/d6b/f70 [231543,91263] 0 2024-05-27T09:29:25.471 INFO:tasks.workunit.client.0.smithi119.stdout:0/516: fsync d2/d6/d9/d12/d4b/d59/f58 0 2024-05-27T09:29:25.473 INFO:tasks.workunit.client.1.smithi138.stdout:0/25: dwrite f8 [0,4194304] 0 2024-05-27T09:29:25.475 INFO:tasks.workunit.client.0.smithi119.stdout:2/531: mkdir d2/da/d15/d33/d69/db1 0 2024-05-27T09:29:25.483 INFO:tasks.workunit.client.0.smithi119.stdout:4/552: symlink d4/d6/d28/d2f/la9 0 2024-05-27T09:29:25.486 INFO:tasks.workunit.client.0.smithi119.stdout:5/543: creat d6/dd/d46/fb8 x:0 0 0 2024-05-27T09:29:25.487 INFO:tasks.workunit.client.0.smithi119.stdout:3/667: link db/d36/d45/d4c/d62/d13/d33/l94 db/d36/ld6 0 2024-05-27T09:29:25.487 INFO:tasks.workunit.client.0.smithi119.stdout:3/668: fsync db/d36/d45/d93/fa4 0 2024-05-27T09:29:25.487 INFO:tasks.workunit.client.0.smithi119.stdout:3/669: truncate db/dc9/d98/d7a/f9e 949255 0 2024-05-27T09:29:25.502 INFO:tasks.workunit.client.0.smithi119.stdout:4/553: mknod d4/d6/d81/caa 0 2024-05-27T09:29:25.503 INFO:tasks.workunit.client.0.smithi119.stdout:4/554: creat d4/de/d21/fab x:0 0 0 2024-05-27T09:29:25.503 INFO:tasks.workunit.client.0.smithi119.stdout:4/555: write d4/f8f [1591036,115917] 0 2024-05-27T09:29:25.504 INFO:tasks.workunit.client.0.smithi119.stdout:8/510: rmdir d3/d6e/d80 39 2024-05-27T09:29:25.506 INFO:tasks.workunit.client.0.smithi119.stdout:2/532: getdents d2/da/d4e 0 2024-05-27T09:29:25.506 INFO:tasks.workunit.client.0.smithi119.stdout:2/533: write d2/d1a/f3d [587353,38671] 0 2024-05-27T09:29:25.507 INFO:tasks.workunit.client.1.smithi138.stdout:7/21: dread d0/f4 [0,4194304] 0 2024-05-27T09:29:25.507 INFO:tasks.workunit.client.1.smithi138.stdout:7/22: fdatasync d0/f4 0 2024-05-27T09:29:25.507 INFO:tasks.workunit.client.1.smithi138.stdout:3/18: mknod d0/c4 0 2024-05-27T09:29:25.508 INFO:tasks.workunit.client.1.smithi138.stdout:3/19: chown d0/d2 994207851 1 2024-05-27T09:29:25.508 INFO:tasks.workunit.client.1.smithi138.stdout:3/20: write - no filename 2024-05-27T09:29:25.509 INFO:tasks.workunit.client.1.smithi138.stdout:6/28: dwrite d3/f5 [0,4194304] 0 2024-05-27T09:29:25.512 INFO:tasks.workunit.client.0.smithi119.stdout:4/556: symlink d4/de/d66/d87/lac 0 2024-05-27T09:29:25.514 INFO:tasks.workunit.client.1.smithi138.stdout:6/29: chown c1 89851 1 2024-05-27T09:29:25.514 INFO:tasks.workunit.client.1.smithi138.stdout:8/16: link f1 f2 0 2024-05-27T09:29:25.514 INFO:tasks.workunit.client.1.smithi138.stdout:5/34: truncate f1 2518050 0 2024-05-27T09:29:25.514 INFO:tasks.workunit.client.1.smithi138.stdout:4/17: unlink f5 0 2024-05-27T09:29:25.514 INFO:tasks.workunit.client.1.smithi138.stdout:5/35: creat d3/fd x:0 0 0 2024-05-27T09:29:25.515 INFO:tasks.workunit.client.0.smithi119.stdout:9/430: dwrite f15 [0,4194304] 0 2024-05-27T09:29:25.515 INFO:tasks.workunit.client.0.smithi119.stdout:2/534: truncate d2/da/fd 1686435 0 2024-05-27T09:29:25.516 INFO:tasks.workunit.client.1.smithi138.stdout:2/29: rename d1/d7/f4 to d1/d7/f9 0 2024-05-27T09:29:25.518 INFO:tasks.workunit.client.0.smithi119.stdout:4/557: creat d4/d23/d31/fad x:0 0 0 2024-05-27T09:29:25.518 INFO:tasks.workunit.client.0.smithi119.stdout:4/558: fdatasync d4/d6/d28/f34 0 2024-05-27T09:29:25.521 INFO:tasks.workunit.client.0.smithi119.stdout:8/511: mknod d3/d8/d1b/d46/ca8 0 2024-05-27T09:29:25.522 INFO:tasks.workunit.client.0.smithi119.stdout:1/627: sync 2024-05-27T09:29:25.524 INFO:tasks.workunit.client.0.smithi119.stdout:9/431: rmdir d18/d19/d1b/d40/d87 0 2024-05-27T09:29:25.526 INFO:tasks.workunit.client.1.smithi138.stdout:0/26: rename f4 to da/fb 0 2024-05-27T09:29:25.526 INFO:tasks.workunit.client.1.smithi138.stdout:0/27: dread - f5 zero size 2024-05-27T09:29:25.526 INFO:tasks.workunit.client.1.smithi138.stdout:0/28: stat f1 0 2024-05-27T09:29:25.527 INFO:tasks.workunit.client.0.smithi119.stdout:8/512: unlink d3/d8/d4d/d58/f97 0 2024-05-27T09:29:25.528 INFO:tasks.workunit.client.0.smithi119.stdout:9/432: symlink d18/d19/d1b/d40/l8f 0 2024-05-27T09:29:25.528 INFO:tasks.workunit.client.0.smithi119.stdout:9/433: fsync d18/d19/d1b/d29/f2f 0 2024-05-27T09:29:25.528 INFO:tasks.workunit.client.0.smithi119.stdout:9/434: creat d18/d19/d1b/d40/d44/f90 x:0 0 0 2024-05-27T09:29:25.529 INFO:tasks.workunit.client.1.smithi138.stdout:7/23: dread d0/f4 [0,4194304] 0 2024-05-27T09:29:25.531 INFO:tasks.workunit.client.1.smithi138.stdout:8/17: mkdir d3 0 2024-05-27T09:29:25.531 INFO:tasks.workunit.client.1.smithi138.stdout:8/18: rename d3 to d3/d4 22 2024-05-27T09:29:25.532 INFO:tasks.workunit.client.1.smithi138.stdout:8/19: stat f1 0 2024-05-27T09:29:25.534 INFO:tasks.workunit.client.1.smithi138.stdout:4/18: symlink l6 0 2024-05-27T09:29:25.537 INFO:tasks.workunit.client.0.smithi119.stdout:9/435: rename d18/d19/c28 to d18/d19/c91 0 2024-05-27T09:29:25.540 INFO:tasks.workunit.client.1.smithi138.stdout:2/30: mkdir d1/da 0 2024-05-27T09:29:25.540 INFO:tasks.workunit.client.1.smithi138.stdout:2/31: chown d1/c8 143024751 1 2024-05-27T09:29:25.541 INFO:tasks.workunit.client.1.smithi138.stdout:0/29: rmdir da 39 2024-05-27T09:29:25.541 INFO:tasks.workunit.client.1.smithi138.stdout:5/36: unlink d3/f4 0 2024-05-27T09:29:25.541 INFO:tasks.workunit.client.1.smithi138.stdout:6/30: write d3/f5 [3811827,26778] 0 2024-05-27T09:29:25.541 INFO:tasks.workunit.client.1.smithi138.stdout:6/31: creat d3/fa x:0 0 0 2024-05-27T09:29:25.542 INFO:tasks.workunit.client.1.smithi138.stdout:5/37: write d3/fd [737727,124091] 0 2024-05-27T09:29:25.546 INFO:tasks.workunit.client.1.smithi138.stdout:8/20: read f1 [171461,74964] 0 2024-05-27T09:29:25.547 INFO:tasks.workunit.client.1.smithi138.stdout:4/19: truncate f0 1248219 0 2024-05-27T09:29:25.551 INFO:tasks.workunit.client.1.smithi138.stdout:2/32: mkdir d1/d7/db 0 2024-05-27T09:29:25.555 INFO:tasks.workunit.client.0.smithi119.stdout:2/535: write d2/d1a/f30 [876211,5805] 0 2024-05-27T09:29:25.556 INFO:tasks.workunit.client.1.smithi138.stdout:6/32: unlink c1 0 2024-05-27T09:29:25.557 INFO:tasks.workunit.client.1.smithi138.stdout:0/30: mknod da/cc 0 2024-05-27T09:29:25.563 INFO:tasks.workunit.client.0.smithi119.stdout:0/517: dread d2/d6/d23/d7a/f42 [0,4194304] 0 2024-05-27T09:29:25.564 INFO:tasks.workunit.client.1.smithi138.stdout:2/33: mknod d1/cc 0 2024-05-27T09:29:25.566 INFO:tasks.workunit.client.1.smithi138.stdout:0/31: creat da/fd x:0 0 0 2024-05-27T09:29:25.566 INFO:tasks.workunit.client.1.smithi138.stdout:0/32: dread - f5 zero size 2024-05-27T09:29:25.567 INFO:tasks.workunit.client.0.smithi119.stdout:3/670: dwrite db/fa0 [0,4194304] 0 2024-05-27T09:29:25.567 INFO:tasks.workunit.client.0.smithi119.stdout:3/671: write db/dc9/d98/d7a/f84 [76466,119643] 0 2024-05-27T09:29:25.568 INFO:tasks.workunit.client.0.smithi119.stdout:3/672: fsync db/d36/d45/d4c/d62/d13/f3d 0 2024-05-27T09:29:25.573 INFO:tasks.workunit.client.1.smithi138.stdout:1/72: dwrite d4/f5 [0,4194304] 0 2024-05-27T09:29:25.573 INFO:tasks.workunit.client.0.smithi119.stdout:0/518: link d2/d6/d23/f33 d2/d6/d75/fa4 0 2024-05-27T09:29:25.577 INFO:tasks.workunit.client.1.smithi138.stdout:2/34: creat d1/d7/db/fd x:0 0 0 2024-05-27T09:29:25.577 INFO:tasks.workunit.client.1.smithi138.stdout:2/35: getdents d1/da 0 2024-05-27T09:29:25.577 INFO:tasks.workunit.client.1.smithi138.stdout:2/36: truncate d1/d7/db/fd 948300 0 2024-05-27T09:29:25.583 INFO:tasks.workunit.client.1.smithi138.stdout:0/33: rename l9 to da/le 0 2024-05-27T09:29:25.583 INFO:tasks.workunit.client.1.smithi138.stdout:0/34: write f1 [4377623,15173] 0 2024-05-27T09:29:25.586 INFO:tasks.workunit.client.0.smithi119.stdout:4/559: dread d4/d1c/d39/f4b [0,4194304] 0 2024-05-27T09:29:25.587 INFO:tasks.workunit.client.1.smithi138.stdout:2/37: creat d1/d7/fe x:0 0 0 2024-05-27T09:29:25.587 INFO:tasks.workunit.client.1.smithi138.stdout:2/38: rename d1 to d1/d7/db/df 22 2024-05-27T09:29:25.590 INFO:tasks.workunit.client.1.smithi138.stdout:1/73: getdents d4 0 2024-05-27T09:29:25.592 INFO:tasks.workunit.client.0.smithi119.stdout:2/536: dread d2/d1a/f1f [0,4194304] 0 2024-05-27T09:29:25.593 INFO:tasks.workunit.client.1.smithi138.stdout:1/74: rename d4/da/ce to d4/d8/c19 0 2024-05-27T09:29:25.595 INFO:tasks.workunit.client.1.smithi138.stdout:1/75: mkdir d4/d1a 0 2024-05-27T09:29:25.598 INFO:tasks.workunit.client.1.smithi138.stdout:5/38: getdents d3 0 2024-05-27T09:29:25.598 INFO:tasks.workunit.client.1.smithi138.stdout:2/39: read d1/d7/f6 [2457889,126754] 0 2024-05-27T09:29:25.599 INFO:tasks.workunit.client.1.smithi138.stdout:5/39: rmdir d3 39 2024-05-27T09:29:25.599 INFO:tasks.workunit.client.1.smithi138.stdout:5/40: write f0 [944606,35724] 0 2024-05-27T09:29:25.600 INFO:tasks.workunit.client.1.smithi138.stdout:4/20: write f3 [1429409,64525] 0 2024-05-27T09:29:25.600 INFO:tasks.workunit.client.1.smithi138.stdout:4/21: write f0 [422116,71237] 0 2024-05-27T09:29:25.602 INFO:tasks.workunit.client.1.smithi138.stdout:8/21: truncate f2 95653 0 2024-05-27T09:29:25.602 INFO:tasks.workunit.client.1.smithi138.stdout:8/22: fdatasync f1 0 2024-05-27T09:29:25.604 INFO:tasks.workunit.client.1.smithi138.stdout:4/22: mkdir d7 0 2024-05-27T09:29:25.606 INFO:tasks.workunit.client.1.smithi138.stdout:8/23: creat d3/f5 x:0 0 0 2024-05-27T09:29:25.607 INFO:tasks.workunit.client.1.smithi138.stdout:4/23: mknod d7/c8 0 2024-05-27T09:29:25.609 INFO:tasks.workunit.client.0.smithi119.stdout:3/673: dread db/d36/d45/d4c/d62/f3f [0,4194304] 0 2024-05-27T09:29:25.609 INFO:tasks.workunit.client.0.smithi119.stdout:2/537: write d2/d1a/f3d [344864,104246] 0 2024-05-27T09:29:25.611 INFO:tasks.workunit.client.0.smithi119.stdout:2/538: unlink d2/d1a/f9b 0 2024-05-27T09:29:25.612 INFO:tasks.workunit.client.0.smithi119.stdout:2/539: truncate d2/d1a/f61 1299281 0 2024-05-27T09:29:25.613 INFO:tasks.workunit.client.0.smithi119.stdout:6/494: dwrite d0/d2/d1c/f2d [0,4194304] 0 2024-05-27T09:29:25.613 INFO:tasks.workunit.client.0.smithi119.stdout:6/495: dread - d0/d2/f9b zero size 2024-05-27T09:29:25.614 INFO:tasks.workunit.client.0.smithi119.stdout:5/544: dread d6/f9 [4194304,4194304] 0 2024-05-27T09:29:25.625 INFO:tasks.workunit.client.0.smithi119.stdout:2/540: rename d2/d1a/l84 to d2/da/d15/d33/lb2 0 2024-05-27T09:29:25.625 INFO:tasks.workunit.client.0.smithi119.stdout:2/541: chown d2/da/d15/d33/c97 16963 1 2024-05-27T09:29:25.626 INFO:tasks.workunit.client.0.smithi119.stdout:2/542: fdatasync d2/da/fd 0 2024-05-27T09:29:25.628 INFO:tasks.workunit.client.0.smithi119.stdout:5/545: creat d6/db/de/d27/d77/fb9 x:0 0 0 2024-05-27T09:29:25.628 INFO:tasks.workunit.client.0.smithi119.stdout:5/546: chown d6/db/de/f89 1583590 1 2024-05-27T09:29:25.629 INFO:tasks.workunit.client.0.smithi119.stdout:2/543: mknod d2/da/d15/d33/d89/d94/d87/cb3 0 2024-05-27T09:29:25.629 INFO:tasks.workunit.client.0.smithi119.stdout:2/544: truncate d2/da/d4e/f72 228081 0 2024-05-27T09:29:25.632 INFO:tasks.workunit.client.0.smithi119.stdout:2/545: mknod d2/d1a/cb4 0 2024-05-27T09:29:25.634 INFO:tasks.workunit.client.0.smithi119.stdout:2/546: creat d2/da/d15/d33/fb5 x:0 0 0 2024-05-27T09:29:25.634 INFO:tasks.workunit.client.0.smithi119.stdout:2/547: dread - d2/da/d15/d33/d89/d94/d81/f9e zero size 2024-05-27T09:29:25.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:25 smithi138 ceph-mon[34885]: pgmap v10: 65 pgs: 65 active+clean; 1.2 GiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 93 MiB/s rd, 114 MiB/s wr, 560 op/s 2024-05-27T09:29:25.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:25 smithi138 ceph-mon[34885]: Standby manager daemon smithi119.ncwmog started 2024-05-27T09:29:25.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:25 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.119:0/2913104693' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi119.ncwmog/crt"}]: dispatch 2024-05-27T09:29:25.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:25 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.119:0/2913104693' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T09:29:25.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:25 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.119:0/2913104693' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi119.ncwmog/key"}]: dispatch 2024-05-27T09:29:25.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:25 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.119:0/2913104693' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T09:29:25.636 INFO:tasks.workunit.client.0.smithi119.stdout:2/548: unlink d2/da/d2c/c32 0 2024-05-27T09:29:25.636 INFO:tasks.workunit.client.0.smithi119.stdout:2/549: creat d2/da/d15/d53/fb6 x:0 0 0 2024-05-27T09:29:25.640 INFO:tasks.workunit.client.0.smithi119.stdout:2/550: mknod d2/da/d15/d33/d69/cb7 0 2024-05-27T09:29:25.641 INFO:tasks.workunit.client.1.smithi138.stdout:7/24: dwrite d0/f4 [0,4194304] 0 2024-05-27T09:29:25.642 INFO:tasks.workunit.client.0.smithi119.stdout:6/496: dread d0/d2/d1c/f3e [0,4194304] 0 2024-05-27T09:29:25.643 INFO:tasks.workunit.client.0.smithi119.stdout:6/497: creat d0/d2/d4e/d5a/faa x:0 0 0 2024-05-27T09:29:25.644 INFO:tasks.workunit.client.0.smithi119.stdout:6/498: fdatasync d0/d2/d1c/f2d 0 2024-05-27T09:29:25.644 INFO:tasks.workunit.client.0.smithi119.stdout:6/499: truncate d0/d2/d4e/f6a 1002289 0 2024-05-27T09:29:25.647 INFO:tasks.workunit.client.1.smithi138.stdout:6/33: truncate d3/f5 2133726 0 2024-05-27T09:29:25.649 INFO:tasks.workunit.client.1.smithi138.stdout:6/34: mkdir d3/db 0 2024-05-27T09:29:25.649 INFO:tasks.workunit.client.1.smithi138.stdout:5/41: dwrite d3/fd [0,4194304] 0 2024-05-27T09:29:25.649 INFO:tasks.workunit.client.1.smithi138.stdout:6/35: getdents d3/db 0 2024-05-27T09:29:25.650 INFO:tasks.workunit.client.1.smithi138.stdout:6/36: write f2 [791763,102366] 0 2024-05-27T09:29:25.655 INFO:tasks.workunit.client.1.smithi138.stdout:5/42: mknod d3/ce 0 2024-05-27T09:29:25.655 INFO:tasks.workunit.client.1.smithi138.stdout:5/43: dread - d3/fc zero size 2024-05-27T09:29:25.655 INFO:tasks.workunit.client.1.smithi138.stdout:5/44: fdatasync d3/f7 0 2024-05-27T09:29:25.655 INFO:tasks.workunit.client.1.smithi138.stdout:5/45: fsync d3/f7 0 2024-05-27T09:29:25.658 INFO:tasks.workunit.client.1.smithi138.stdout:6/37: link d3/c7 d3/db/cc 0 2024-05-27T09:29:25.659 INFO:tasks.workunit.client.1.smithi138.stdout:1/76: dwrite d4/f18 [0,4194304] 0 2024-05-27T09:29:25.660 INFO:tasks.workunit.client.1.smithi138.stdout:5/46: symlink d3/lf 0 2024-05-27T09:29:25.661 INFO:tasks.workunit.client.1.smithi138.stdout:1/77: write d4/fb [305576,36156] 0 2024-05-27T09:29:25.662 INFO:tasks.workunit.client.0.smithi119.stdout:5/547: dread d6/dd/d39/f69 [0,4194304] 0 2024-05-27T09:29:25.663 INFO:tasks.workunit.client.0.smithi119.stdout:5/548: mkdir d6/dd/d46/dba 0 2024-05-27T09:29:25.663 INFO:tasks.workunit.client.0.smithi119.stdout:5/549: fsync d6/dd/d46/d47/d4b/f58 0 2024-05-27T09:29:25.665 INFO:tasks.workunit.client.1.smithi138.stdout:1/78: read d4/fb [1370735,89761] 0 2024-05-27T09:29:25.665 INFO:tasks.workunit.client.0.smithi119.stdout:5/550: symlink d6/db/de/d27/d94/lbb 0 2024-05-27T09:29:25.666 INFO:tasks.workunit.client.1.smithi138.stdout:6/38: creat d3/db/fd x:0 0 0 2024-05-27T09:29:25.668 INFO:tasks.workunit.client.0.smithi119.stdout:5/551: link d6/dd/df/f75 d6/db/de/d27/d77/fbc 0 2024-05-27T09:29:25.668 INFO:tasks.workunit.client.0.smithi119.stdout:5/552: chown d6/dd/d39/d74/fa1 27182 1 2024-05-27T09:29:25.671 INFO:tasks.workunit.client.0.smithi119.stdout:5/553: link d6/db/f6f d6/dd/d39/d19/d3c/d52/fbd 0 2024-05-27T09:29:25.676 INFO:tasks.workunit.client.1.smithi138.stdout:7/25: write d0/f4 [3825381,45378] 0 2024-05-27T09:29:25.676 INFO:tasks.workunit.client.1.smithi138.stdout:7/26: write d0/f4 [2244556,64746] 0 2024-05-27T09:29:25.677 INFO:tasks.workunit.client.1.smithi138.stdout:1/79: link d4/l15 d4/da/l1b 0 2024-05-27T09:29:25.677 INFO:tasks.workunit.client.1.smithi138.stdout:1/80: truncate d4/f18 4625712 0 2024-05-27T09:29:25.677 INFO:tasks.workunit.client.1.smithi138.stdout:1/81: creat d4/d8/f1c x:0 0 0 2024-05-27T09:29:25.677 INFO:tasks.workunit.client.1.smithi138.stdout:1/82: write d4/fd [447728,7205] 0 2024-05-27T09:29:25.677 INFO:tasks.workunit.client.1.smithi138.stdout:1/83: dread - d4/f12 zero size 2024-05-27T09:29:25.677 INFO:tasks.workunit.client.1.smithi138.stdout:1/84: chown f3 10464119 1 2024-05-27T09:29:25.680 INFO:tasks.workunit.client.1.smithi138.stdout:7/27: link d0/c3 d0/c9 0 2024-05-27T09:29:25.686 INFO:tasks.workunit.client.1.smithi138.stdout:7/28: link d0/c9 d0/ca 0 2024-05-27T09:29:25.694 INFO:tasks.workunit.client.1.smithi138.stdout:6/39: dread f2 [0,4194304] 0 2024-05-27T09:29:25.704 INFO:tasks.workunit.client.1.smithi138.stdout:5/47: getdents d3 0 2024-05-27T09:29:25.704 INFO:tasks.workunit.client.1.smithi138.stdout:5/48: stat d3/fa 0 2024-05-27T09:29:25.704 INFO:tasks.workunit.client.1.smithi138.stdout:5/49: read - d3/fc zero size 2024-05-27T09:29:25.705 INFO:tasks.workunit.client.1.smithi138.stdout:1/85: dread d4/f5 [0,4194304] 0 2024-05-27T09:29:25.706 INFO:tasks.workunit.client.1.smithi138.stdout:1/86: symlink d4/l1d 0 2024-05-27T09:29:25.706 INFO:tasks.workunit.client.1.smithi138.stdout:1/87: creat d4/d8/f1e x:0 0 0 2024-05-27T09:29:25.710 INFO:tasks.workunit.client.1.smithi138.stdout:1/88: creat d4/da/f1f x:0 0 0 2024-05-27T09:29:25.711 INFO:tasks.workunit.client.0.smithi119.stdout:6/500: dwrite d0/d2/f96 [0,4194304] 0 2024-05-27T09:29:25.712 INFO:tasks.workunit.client.0.smithi119.stdout:2/551: dwrite d2/da/d15/d33/d69/f9d [0,4194304] 0 2024-05-27T09:29:25.712 INFO:tasks.workunit.client.0.smithi119.stdout:6/501: write d0/d2/d57/f6d [968234,129656] 0 2024-05-27T09:29:25.718 INFO:tasks.workunit.client.0.smithi119.stdout:8/513: dwrite d3/d8/d1b/d77/d95/f9c [0,4194304] 0 2024-05-27T09:29:25.719 INFO:tasks.workunit.client.0.smithi119.stdout:0/519: dwrite d2/d6/d9/d12/d4b/d96/f98 [0,4194304] 0 2024-05-27T09:29:25.719 INFO:tasks.workunit.client.0.smithi119.stdout:0/520: write d2/d1a/f21 [2009544,34345] 0 2024-05-27T09:29:25.725 INFO:tasks.workunit.client.0.smithi119.stdout:2/552: rmdir d2/da/d15/d33/d89/d94/d9c 0 2024-05-27T09:29:25.725 INFO:tasks.workunit.client.0.smithi119.stdout:6/502: symlink d0/d19/lab 0 2024-05-27T09:29:25.725 INFO:tasks.workunit.client.0.smithi119.stdout:2/553: chown d2/da/d15/d33/f7e 166094635 1 2024-05-27T09:29:25.725 INFO:tasks.workunit.client.0.smithi119.stdout:2/554: chown d2/da/f13 143759 1 2024-05-27T09:29:25.726 INFO:tasks.workunit.client.1.smithi138.stdout:4/24: dwrite f3 [0,4194304] 0 2024-05-27T09:29:25.727 INFO:tasks.workunit.client.1.smithi138.stdout:4/25: creat d7/f9 x:0 0 0 2024-05-27T09:29:25.741 INFO:tasks.workunit.client.0.smithi119.stdout:6/503: getdents d0/d2/d57 0 2024-05-27T09:29:25.741 INFO:tasks.workunit.client.0.smithi119.stdout:2/555: rmdir d2/da/d6f 39 2024-05-27T09:29:25.741 INFO:tasks.workunit.client.0.smithi119.stdout:2/556: fsync d2/da/f37 0 2024-05-27T09:29:25.741 INFO:tasks.workunit.client.0.smithi119.stdout:6/504: chown d0/d19/fa6 184329 1 2024-05-27T09:29:25.741 INFO:tasks.workunit.client.0.smithi119.stdout:8/514: truncate d3/d8/d1b/d46/f2b 438445 0 2024-05-27T09:29:25.746 INFO:tasks.workunit.client.0.smithi119.stdout:2/557: symlink d2/da/d15/d53/dac/lb8 0 2024-05-27T09:29:25.746 INFO:tasks.workunit.client.0.smithi119.stdout:2/558: creat d2/da/d15/d33/d69/fb9 x:0 0 0 2024-05-27T09:29:25.746 INFO:tasks.workunit.client.0.smithi119.stdout:2/559: creat d2/da/d15/d33/d89/d94/d87/fba x:0 0 0 2024-05-27T09:29:25.746 INFO:tasks.workunit.client.0.smithi119.stdout:2/560: fsync d2/da/d6f/f7f 0 2024-05-27T09:29:25.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:25 smithi119 ceph-mon[29082]: pgmap v10: 65 pgs: 65 active+clean; 1.2 GiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 93 MiB/s rd, 114 MiB/s wr, 560 op/s 2024-05-27T09:29:25.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:25 smithi119 ceph-mon[29082]: Standby manager daemon smithi119.ncwmog started 2024-05-27T09:29:25.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:25 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.119:0/2913104693' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi119.ncwmog/crt"}]: dispatch 2024-05-27T09:29:25.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:25 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.119:0/2913104693' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T09:29:25.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:25 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.119:0/2913104693' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi119.ncwmog/key"}]: dispatch 2024-05-27T09:29:25.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:25 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.119:0/2913104693' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T09:29:25.748 INFO:tasks.workunit.client.0.smithi119.stdout:3/674: write db/d36/d45/d4c/d62/d13/d33/fa7 [499172,51832] 0 2024-05-27T09:29:25.748 INFO:tasks.workunit.client.0.smithi119.stdout:4/560: dwrite f1 [0,4194304] 0 2024-05-27T09:29:25.749 INFO:tasks.workunit.client.0.smithi119.stdout:4/561: write d4/d1c/f1e [1092163,66570] 0 2024-05-27T09:29:25.749 INFO:tasks.workunit.client.0.smithi119.stdout:6/505: creat d0/d2/fac x:0 0 0 2024-05-27T09:29:25.751 INFO:tasks.workunit.client.0.smithi119.stdout:8/515: creat d3/d8/d1b/d46/fa9 x:0 0 0 2024-05-27T09:29:25.753 INFO:tasks.workunit.client.0.smithi119.stdout:3/675: dread db/dc9/d98/d7a/f84 [0,4194304] 0 2024-05-27T09:29:25.757 INFO:tasks.workunit.client.1.smithi138.stdout:1/89: dread d4/f18 [0,4194304] 0 2024-05-27T09:29:25.757 INFO:tasks.workunit.client.0.smithi119.stdout:0/521: dread d2/d6/d9/d12/f16 [0,4194304] 0 2024-05-27T09:29:25.774 INFO:tasks.workunit.client.0.smithi119.stdout:2/561: creat d2/da/d15/d53/dac/fbb x:0 0 0 2024-05-27T09:29:25.781 INFO:tasks.workunit.client.0.smithi119.stdout:6/506: mkdir d0/d2/d57/d37/dad 0 2024-05-27T09:29:25.781 INFO:tasks.workunit.client.0.smithi119.stdout:6/507: creat d0/d85/fae x:0 0 0 2024-05-27T09:29:25.781 INFO:tasks.workunit.client.0.smithi119.stdout:6/508: dread - d0/f5f zero size 2024-05-27T09:29:25.781 INFO:tasks.workunit.client.0.smithi119.stdout:8/516: unlink d3/d8/d4d/d79/l9a 0 2024-05-27T09:29:25.781 INFO:tasks.workunit.client.0.smithi119.stdout:8/517: chown d3/d8/d1b/fa7 39133757 1 2024-05-27T09:29:25.781 INFO:tasks.workunit.client.0.smithi119.stdout:8/518: chown d3/de/l5e 26 1 2024-05-27T09:29:25.781 INFO:tasks.workunit.client.0.smithi119.stdout:8/519: readlink d3/d8/lc 0 2024-05-27T09:29:25.782 INFO:tasks.workunit.client.0.smithi119.stdout:3/676: rmdir db/d36/d45/d49/d4f 39 2024-05-27T09:29:25.789 INFO:tasks.workunit.client.1.smithi138.stdout:8/24: dwrite f1 [0,4194304] 0 2024-05-27T09:29:25.791 INFO:tasks.workunit.client.1.smithi138.stdout:4/26: dwrite f3 [4194304,4194304] 0 2024-05-27T09:29:25.791 INFO:tasks.workunit.client.1.smithi138.stdout:6/40: dwrite f2 [0,4194304] 0 2024-05-27T09:29:25.791 INFO:tasks.workunit.client.1.smithi138.stdout:6/41: creat d3/db/fe x:0 0 0 2024-05-27T09:29:25.792 INFO:tasks.workunit.client.1.smithi138.stdout:8/25: symlink d3/l6 0 2024-05-27T09:29:25.795 INFO:tasks.workunit.client.0.smithi119.stdout:6/509: link d0/d11/c8b d0/d85/caf 0 2024-05-27T09:29:25.795 INFO:tasks.workunit.client.0.smithi119.stdout:6/510: creat d0/d19/fb0 x:0 0 0 2024-05-27T09:29:25.796 INFO:tasks.workunit.client.0.smithi119.stdout:6/511: fdatasync d0/d2/d1c/f2a 0 2024-05-27T09:29:25.796 INFO:tasks.workunit.client.0.smithi119.stdout:6/512: creat d0/d2/fb1 x:0 0 0 2024-05-27T09:29:25.796 INFO:tasks.workunit.client.1.smithi138.stdout:6/42: mknod d3/db/cf 0 2024-05-27T09:29:25.796 INFO:tasks.workunit.client.1.smithi138.stdout:6/43: fdatasync d3/db/fe 0 2024-05-27T09:29:25.797 INFO:tasks.workunit.client.1.smithi138.stdout:7/29: dwrite d0/f4 [0,4194304] 0 2024-05-27T09:29:25.798 INFO:tasks.workunit.client.1.smithi138.stdout:6/44: unlink d3/db/fd 0 2024-05-27T09:29:25.800 INFO:tasks.workunit.client.1.smithi138.stdout:7/30: rename d0/f4 to d0/fb 0 2024-05-27T09:29:25.802 INFO:tasks.workunit.client.0.smithi119.stdout:3/677: creat db/d36/d45/d4c/dbe/fd7 x:0 0 0 2024-05-27T09:29:25.802 INFO:tasks.workunit.client.0.smithi119.stdout:2/562: rename d2/da/d2c/f98 to d2/da/d15/fbc 0 2024-05-27T09:29:25.803 INFO:tasks.workunit.client.1.smithi138.stdout:6/45: symlink d3/l10 0 2024-05-27T09:29:25.806 INFO:tasks.workunit.client.0.smithi119.stdout:6/513: mkdir d0/d19/db2 0 2024-05-27T09:29:25.806 INFO:tasks.workunit.client.0.smithi119.stdout:6/514: chown d0/d2/d4e/d5a/f95 3 1 2024-05-27T09:29:25.814 INFO:tasks.workunit.client.0.smithi119.stdout:6/515: unlink d0/l13 0 2024-05-27T09:29:25.814 INFO:tasks.workunit.client.0.smithi119.stdout:6/516: chown d0/d2/d57/d37 6995859 1 2024-05-27T09:29:25.815 INFO:tasks.workunit.client.0.smithi119.stdout:6/517: creat d0/d2/d1c/fb3 x:0 0 0 2024-05-27T09:29:25.815 INFO:tasks.workunit.client.0.smithi119.stdout:6/518: dread - d0/d19/fa6 zero size 2024-05-27T09:29:25.815 INFO:tasks.workunit.client.0.smithi119.stdout:6/519: write d0/d2/d1c/fa0 [173864,28690] 0 2024-05-27T09:29:25.816 INFO:tasks.workunit.client.0.smithi119.stdout:2/563: getdents d2/d1a 0 2024-05-27T09:29:25.821 INFO:tasks.workunit.client.1.smithi138.stdout:8/26: dread f1 [0,4194304] 0 2024-05-27T09:29:25.821 INFO:tasks.workunit.client.0.smithi119.stdout:6/520: unlink d0/d2/f4 0 2024-05-27T09:29:25.822 INFO:tasks.workunit.client.1.smithi138.stdout:5/50: dwrite f0 [4194304,4194304] 0 2024-05-27T09:29:25.823 INFO:tasks.workunit.client.1.smithi138.stdout:5/51: creat d3/f10 x:0 0 0 2024-05-27T09:29:25.824 INFO:tasks.workunit.client.1.smithi138.stdout:7/31: write d0/fb [3540189,95310] 0 2024-05-27T09:29:25.825 INFO:tasks.workunit.client.1.smithi138.stdout:1/90: dwrite d4/f7 [0,4194304] 0 2024-05-27T09:29:25.831 INFO:tasks.workunit.client.1.smithi138.stdout:1/91: symlink d4/d8/l20 0 2024-05-27T09:29:25.837 INFO:tasks.workunit.client.0.smithi119.stdout:6/521: write d0/d2/f96 [1820383,63509] 0 2024-05-27T09:29:25.839 INFO:tasks.workunit.client.1.smithi138.stdout:6/46: rename d3/db to d3/d11 0 2024-05-27T09:29:25.839 INFO:tasks.workunit.client.1.smithi138.stdout:6/47: truncate d3/d11/fe 273219 0 2024-05-27T09:29:25.840 INFO:tasks.workunit.client.0.smithi119.stdout:6/522: getdents d0/d2/d57/d37 0 2024-05-27T09:29:25.848 INFO:tasks.workunit.client.0.smithi119.stdout:6/523: creat d0/d2/d57/d37/dad/fb4 x:0 0 0 2024-05-27T09:29:25.848 INFO:tasks.workunit.client.0.smithi119.stdout:6/524: fdatasync d0/d11/f62 0 2024-05-27T09:29:25.855 INFO:tasks.workunit.client.0.smithi119.stdout:8/520: dwrite d3/d8/d1b/d46/f29 [0,4194304] 0 2024-05-27T09:29:25.855 INFO:tasks.workunit.client.0.smithi119.stdout:8/521: truncate d3/d8/d4d/d58/f6a 720842 0 2024-05-27T09:29:25.855 INFO:tasks.workunit.client.0.smithi119.stdout:8/522: chown d3/d8/d1b/fa7 110449 1 2024-05-27T09:29:25.855 INFO:tasks.workunit.client.0.smithi119.stdout:7/575: sync 2024-05-27T09:29:25.857 INFO:tasks.workunit.client.0.smithi119.stdout:6/525: dread d0/d2/d4e/f6a [0,4194304] 0 2024-05-27T09:29:25.860 INFO:tasks.workunit.client.1.smithi138.stdout:6/48: dread d3/f6 [0,4194304] 0 2024-05-27T09:29:25.861 INFO:tasks.workunit.client.1.smithi138.stdout:6/49: truncate d3/fa 827610 0 2024-05-27T09:29:25.863 INFO:tasks.workunit.client.1.smithi138.stdout:6/50: link d3/c9 d3/c12 0 2024-05-27T09:29:25.863 INFO:tasks.workunit.client.1.smithi138.stdout:6/51: readlink d3/l10 0 2024-05-27T09:29:25.864 INFO:tasks.workunit.client.1.smithi138.stdout:6/52: mknod d3/c13 0 2024-05-27T09:29:25.866 INFO:tasks.workunit.client.0.smithi119.stdout:6/526: symlink d0/d2/lb5 0 2024-05-27T09:29:25.867 INFO:tasks.workunit.client.0.smithi119.stdout:6/527: truncate d0/d19/f55 5163879 0 2024-05-27T09:29:25.867 INFO:tasks.workunit.client.0.smithi119.stdout:6/528: chown d0/d2/d4e/d5a/f91 62596 1 2024-05-27T09:29:25.878 INFO:tasks.workunit.client.1.smithi138.stdout:5/52: write f0 [7853601,98453] 0 2024-05-27T09:29:25.878 INFO:tasks.workunit.client.1.smithi138.stdout:5/53: chown d3/l8 104174 1 2024-05-27T09:29:25.881 INFO:tasks.workunit.client.1.smithi138.stdout:5/54: unlink d3/fc 0 2024-05-27T09:29:25.881 INFO:tasks.workunit.client.1.smithi138.stdout:5/55: fsync d3/f7 0 2024-05-27T09:29:25.881 INFO:tasks.workunit.client.1.smithi138.stdout:1/92: write d4/f18 [2578243,78457] 0 2024-05-27T09:29:25.881 INFO:tasks.workunit.client.1.smithi138.stdout:8/27: dwrite f2 [0,4194304] 0 2024-05-27T09:29:25.881 INFO:tasks.workunit.client.1.smithi138.stdout:8/28: chown d3/f5 105405136 1 2024-05-27T09:29:25.888 INFO:tasks.workunit.client.1.smithi138.stdout:6/53: dread d3/d11/fe [0,4194304] 0 2024-05-27T09:29:25.889 INFO:tasks.workunit.client.0.smithi119.stdout:3/678: dwrite db/fbf [4194304,4194304] 0 2024-05-27T09:29:25.890 INFO:tasks.workunit.client.0.smithi119.stdout:3/679: creat db/d36/d45/dab/fd8 x:0 0 0 2024-05-27T09:29:25.890 INFO:tasks.workunit.client.0.smithi119.stdout:6/529: read d0/d2/d57/d37/f43 [4078696,10257] 0 2024-05-27T09:29:25.893 INFO:tasks.workunit.client.0.smithi119.stdout:3/680: mkdir db/dc9/daa/dd9 0 2024-05-27T09:29:25.893 INFO:tasks.workunit.client.0.smithi119.stdout:3/681: fdatasync db/dc9/f79 0 2024-05-27T09:29:25.893 INFO:tasks.workunit.client.1.smithi138.stdout:6/54: mkdir d3/d14 0 2024-05-27T09:29:25.894 INFO:tasks.workunit.client.0.smithi119.stdout:3/682: write db/d36/d45/d49/d9d/fbb [2477640,78448] 0 2024-05-27T09:29:25.894 INFO:tasks.workunit.client.0.smithi119.stdout:6/530: truncate d0/f68 5921082 0 2024-05-27T09:29:25.902 INFO:tasks.workunit.client.0.smithi119.stdout:6/531: mkdir d0/d2/d57/d37/dad/db6 0 2024-05-27T09:29:25.902 INFO:tasks.workunit.client.1.smithi138.stdout:6/55: mknod d3/d11/c15 0 2024-05-27T09:29:25.905 INFO:tasks.workunit.client.1.smithi138.stdout:5/56: write d3/fd [78233,64156] 0 2024-05-27T09:29:25.905 INFO:tasks.workunit.client.1.smithi138.stdout:5/57: creat d3/f11 x:0 0 0 2024-05-27T09:29:25.909 INFO:tasks.workunit.client.1.smithi138.stdout:5/58: mkdir d3/d12 0 2024-05-27T09:29:25.909 INFO:tasks.workunit.client.1.smithi138.stdout:5/59: creat d3/f13 x:0 0 0 2024-05-27T09:29:25.911 INFO:tasks.workunit.client.0.smithi119.stdout:6/532: symlink d0/d2/d57/d37/d74/d65/lb7 0 2024-05-27T09:29:25.917 INFO:tasks.workunit.client.1.smithi138.stdout:6/56: dread d3/f6 [0,4194304] 0 2024-05-27T09:29:25.917 INFO:tasks.workunit.client.1.smithi138.stdout:5/60: mknod d3/c14 0 2024-05-27T09:29:25.917 INFO:tasks.workunit.client.0.smithi119.stdout:6/533: rename d0/d2/d57/l75 to d0/d2/d1c/d21/lb8 0 2024-05-27T09:29:25.917 INFO:tasks.workunit.client.0.smithi119.stdout:6/534: fdatasync d0/d2/d1c/f3e 0 2024-05-27T09:29:25.920 INFO:tasks.workunit.client.0.smithi119.stdout:6/535: link d0/d2/d4e/f1f d0/d2/d4e/fb9 0 2024-05-27T09:29:25.925 INFO:tasks.workunit.client.0.smithi119.stdout:6/536: rmdir d0/d2/d57/d37/d74/d65 39 2024-05-27T09:29:25.927 INFO:tasks.workunit.client.1.smithi138.stdout:6/57: symlink d3/d11/l16 0 2024-05-27T09:29:25.931 INFO:tasks.workunit.client.0.smithi119.stdout:6/537: mknod d0/d19/db2/cba 0 2024-05-27T09:29:25.932 INFO:tasks.workunit.client.1.smithi138.stdout:7/32: dwrite d0/fb [0,4194304] 0 2024-05-27T09:29:25.932 INFO:tasks.workunit.client.1.smithi138.stdout:7/33: chown d0/c3 13223135 1 2024-05-27T09:29:25.935 INFO:tasks.workunit.client.1.smithi138.stdout:4/27: dwrite f0 [4194304,4194304] 0 2024-05-27T09:29:25.935 INFO:tasks.workunit.client.1.smithi138.stdout:4/28: chown l6 950815199 1 2024-05-27T09:29:25.946 INFO:tasks.workunit.client.1.smithi138.stdout:6/58: dread d3/fa [0,4194304] 0 2024-05-27T09:29:25.946 INFO:tasks.workunit.client.1.smithi138.stdout:6/59: chown d3/f6 82043413 1 2024-05-27T09:29:25.949 INFO:tasks.workunit.client.0.smithi119.stdout:6/538: mkdir d0/dbb 0 2024-05-27T09:29:25.949 INFO:tasks.workunit.client.0.smithi119.stdout:6/539: readlink d0/d2/d4e/la7 0 2024-05-27T09:29:25.949 INFO:tasks.workunit.client.1.smithi138.stdout:5/61: dread f1 [0,4194304] 0 2024-05-27T09:29:25.949 INFO:tasks.workunit.client.1.smithi138.stdout:5/62: write d3/fd [1687058,32344] 0 2024-05-27T09:29:25.949 INFO:tasks.workunit.client.1.smithi138.stdout:5/63: creat d3/f15 x:0 0 0 2024-05-27T09:29:25.963 INFO:tasks.workunit.client.1.smithi138.stdout:8/29: dwrite f1 [0,4194304] 0 2024-05-27T09:29:25.971 INFO:tasks.workunit.client.1.smithi138.stdout:8/30: symlink d3/l7 0 2024-05-27T09:29:25.974 INFO:tasks.workunit.client.1.smithi138.stdout:8/31: unlink f2 0 2024-05-27T09:29:25.977 INFO:tasks.workunit.client.0.smithi119.stdout:2/564: dwrite d2/f90 [0,4194304] 0 2024-05-27T09:29:25.981 INFO:tasks.workunit.client.0.smithi119.stdout:2/565: symlink d2/da/d15/d33/d69/lbd 0 2024-05-27T09:29:25.982 INFO:tasks.workunit.client.0.smithi119.stdout:7/576: dwrite d7/df/d19/d4d/f71 [0,4194304] 0 2024-05-27T09:29:25.990 INFO:tasks.workunit.client.0.smithi119.stdout:2/566: rename d2/da/d15/d33/d89/d94/d81/f96 to d2/fbe 0 2024-05-27T09:29:25.990 INFO:tasks.workunit.client.1.smithi138.stdout:4/29: write f0 [7368887,41872] 0 2024-05-27T09:29:25.992 INFO:tasks.workunit.client.0.smithi119.stdout:2/567: chown d2/c2a 0 1 2024-05-27T09:29:25.992 INFO:tasks.workunit.client.1.smithi138.stdout:4/30: rmdir d7 39 2024-05-27T09:29:25.995 INFO:tasks.workunit.client.0.smithi119.stdout:8/523: dwrite d3/d8/d1b/f41 [0,4194304] 0 2024-05-27T09:29:25.996 INFO:tasks.workunit.client.0.smithi119.stdout:2/568: symlink d2/da/d15/d33/d89/d94/lbf 0 2024-05-27T09:29:25.998 INFO:tasks.workunit.client.0.smithi119.stdout:8/524: mknod d3/d8/d4d/d5d/d66/caa 0 2024-05-27T09:29:25.999 INFO:tasks.workunit.client.1.smithi138.stdout:1/93: dwrite d4/f7 [0,4194304] 0 2024-05-27T09:29:26.000 INFO:tasks.workunit.client.1.smithi138.stdout:7/34: dwrite d0/fb [0,4194304] 0 2024-05-27T09:29:26.003 INFO:tasks.workunit.client.0.smithi119.stdout:2/569: unlink d2/da/d6f/f70 0 2024-05-27T09:29:26.004 INFO:tasks.workunit.client.0.smithi119.stdout:7/577: write d7/d1a/d40/d44/f5d [3974583,107995] 0 2024-05-27T09:29:26.005 INFO:tasks.workunit.client.0.smithi119.stdout:8/525: truncate d3/de/f1d 1313518 0 2024-05-27T09:29:26.005 INFO:tasks.workunit.client.0.smithi119.stdout:8/526: truncate d3/d8/d4d/d5d/d60/f71 5129419 0 2024-05-27T09:29:26.005 INFO:tasks.workunit.client.0.smithi119.stdout:8/527: creat d3/d8/d1b/d77/d95/fab x:0 0 0 2024-05-27T09:29:26.006 INFO:tasks.workunit.client.1.smithi138.stdout:1/94: rename d4/d8/c19 to d4/c21 0 2024-05-27T09:29:26.008 INFO:tasks.workunit.client.0.smithi119.stdout:2/570: mknod d2/da/d15/d33/d89/d94/d81/cc0 0 2024-05-27T09:29:26.009 INFO:tasks.workunit.client.1.smithi138.stdout:1/95: unlink d4/d8/f1e 0 2024-05-27T09:29:26.009 INFO:tasks.workunit.client.1.smithi138.stdout:1/96: write d4/fb [1419843,3086] 0 2024-05-27T09:29:26.010 INFO:tasks.workunit.client.1.smithi138.stdout:2/40: sync 2024-05-27T09:29:26.010 INFO:tasks.workunit.client.1.smithi138.stdout:0/35: sync 2024-05-27T09:29:26.011 INFO:tasks.workunit.client.1.smithi138.stdout:1/97: mkdir d4/d8/d22 0 2024-05-27T09:29:26.011 INFO:tasks.workunit.client.1.smithi138.stdout:1/98: write d4/f12 [763271,48725] 0 2024-05-27T09:29:26.011 INFO:tasks.workunit.client.1.smithi138.stdout:8/32: write f1 [1841975,21375] 0 2024-05-27T09:29:26.015 INFO:tasks.workunit.client.0.smithi119.stdout:3/683: dwrite db/d36/d45/d4c/d62/f3f [0,4194304] 0 2024-05-27T09:29:26.016 INFO:tasks.workunit.client.1.smithi138.stdout:0/36: truncate da/fb 367914 0 2024-05-27T09:29:26.017 INFO:tasks.workunit.client.1.smithi138.stdout:2/41: truncate d1/d7/db/fd 879703 0 2024-05-27T09:29:26.017 INFO:tasks.workunit.client.1.smithi138.stdout:1/99: mknod d4/d1a/c23 0 2024-05-27T09:29:26.022 INFO:tasks.workunit.client.0.smithi119.stdout:1/628: sync 2024-05-27T09:29:26.022 INFO:tasks.workunit.client.0.smithi119.stdout:1/629: creat d7/db6/fc6 x:0 0 0 2024-05-27T09:29:26.024 INFO:tasks.workunit.client.0.smithi119.stdout:2/571: unlink d2/da/d15/d33/c5e 0 2024-05-27T09:29:26.026 INFO:tasks.workunit.client.1.smithi138.stdout:7/35: dread d0/fb [0,4194304] 0 2024-05-27T09:29:26.029 INFO:tasks.workunit.client.1.smithi138.stdout:1/100: rename f3 to d4/f24 0 2024-05-27T09:29:26.031 INFO:tasks.workunit.client.0.smithi119.stdout:3/684: getdents db/d36/d45/d4c 0 2024-05-27T09:29:26.031 INFO:tasks.workunit.client.0.smithi119.stdout:3/685: write db/d36/d45/d4c/d62/d13/f97 [2002710,61898] 0 2024-05-27T09:29:26.031 INFO:tasks.workunit.client.0.smithi119.stdout:1/630: truncate d7/dd/d25/d29/d38/f90 4503703 0 2024-05-27T09:29:26.032 INFO:tasks.workunit.client.0.smithi119.stdout:1/631: chown d7/db/d12/d15/d1f/c7b 1227961 1 2024-05-27T09:29:26.032 INFO:tasks.workunit.client.0.smithi119.stdout:6/540: dwrite d0/d2/d57/d37/f63 [0,4194304] 0 2024-05-27T09:29:26.032 INFO:tasks.workunit.client.0.smithi119.stdout:2/572: link d2/da/f1c d2/da/d6f/fc1 0 2024-05-27T09:29:26.033 INFO:tasks.workunit.client.1.smithi138.stdout:4/31: write f3 [445561,101705] 0 2024-05-27T09:29:26.034 INFO:tasks.workunit.client.1.smithi138.stdout:4/32: read - d7/f9 zero size 2024-05-27T09:29:26.035 INFO:tasks.workunit.client.1.smithi138.stdout:0/37: rmdir da 39 2024-05-27T09:29:26.037 INFO:tasks.workunit.client.1.smithi138.stdout:4/33: mknod d7/ca 0 2024-05-27T09:29:26.037 INFO:tasks.workunit.client.1.smithi138.stdout:4/34: dread - d7/f9 zero size 2024-05-27T09:29:26.037 INFO:tasks.workunit.client.0.smithi119.stdout:1/632: dread d7/db/d12/f9a [0,4194304] 0 2024-05-27T09:29:26.039 INFO:tasks.workunit.client.1.smithi138.stdout:6/60: dwrite d3/fa [0,4194304] 0 2024-05-27T09:29:26.039 INFO:tasks.workunit.client.0.smithi119.stdout:6/541: link d0/d19/db2/cba d0/cbc 0 2024-05-27T09:29:26.039 INFO:tasks.workunit.client.0.smithi119.stdout:1/633: chown d7/dd/d25/f86 157854 1 2024-05-27T09:29:26.040 INFO:tasks.workunit.client.0.smithi119.stdout:3/686: dread db/d36/d45/d4d/f67 [0,4194304] 0 2024-05-27T09:29:26.040 INFO:tasks.workunit.client.0.smithi119.stdout:3/687: rename db to db/dc9/daa/dd9/dda 22 2024-05-27T09:29:26.041 INFO:tasks.workunit.client.1.smithi138.stdout:7/36: write d0/fb [1881209,39587] 0 2024-05-27T09:29:26.042 INFO:tasks.workunit.client.0.smithi119.stdout:3/688: symlink db/d36/d45/d49/ldb 0 2024-05-27T09:29:26.042 INFO:tasks.workunit.client.0.smithi119.stdout:3/689: truncate db/dc9/fd5 158857 0 2024-05-27T09:29:26.042 INFO:tasks.workunit.client.0.smithi119.stdout:3/690: creat db/d36/d45/d49/d9d/fdc x:0 0 0 2024-05-27T09:29:26.043 INFO:tasks.workunit.client.1.smithi138.stdout:0/38: mknod da/cf 0 2024-05-27T09:29:26.043 INFO:tasks.workunit.client.1.smithi138.stdout:2/42: rename d1/d7 to d1/d10 0 2024-05-27T09:29:26.043 INFO:tasks.workunit.client.0.smithi119.stdout:1/634: link d7/dd/d44/l75 d7/db/d35/lc7 0 2024-05-27T09:29:26.044 INFO:tasks.workunit.client.1.smithi138.stdout:5/64: dwrite d3/f11 [0,4194304] 0 2024-05-27T09:29:26.044 INFO:tasks.workunit.client.1.smithi138.stdout:5/65: chown d3/c14 1645 1 2024-05-27T09:29:26.046 INFO:tasks.workunit.client.0.smithi119.stdout:3/691: rename db/d36/d45/d4c/d62/c32 to db/dc9/d98/cdd 0 2024-05-27T09:29:26.049 INFO:tasks.workunit.client.1.smithi138.stdout:0/39: rename f8 to da/f10 0 2024-05-27T09:29:26.050 INFO:tasks.workunit.client.1.smithi138.stdout:2/43: unlink d1/c8 0 2024-05-27T09:29:26.050 INFO:tasks.workunit.client.1.smithi138.stdout:6/61: dread d3/d11/fe [0,4194304] 0 2024-05-27T09:29:26.050 INFO:tasks.workunit.client.1.smithi138.stdout:6/62: creat d3/f17 x:0 0 0 2024-05-27T09:29:26.051 INFO:tasks.workunit.client.0.smithi119.stdout:1/635: rename d7/db/d12/d15/d1f/fbd to d7/dd/d44/fc8 0 2024-05-27T09:29:26.053 INFO:tasks.workunit.client.1.smithi138.stdout:8/33: fdatasync f1 0 2024-05-27T09:29:26.053 INFO:tasks.workunit.client.1.smithi138.stdout:8/34: read f1 [1837227,2708] 0 2024-05-27T09:29:26.054 INFO:tasks.workunit.client.0.smithi119.stdout:1/636: mkdir d7/db/d8f/d9b/dc9 0 2024-05-27T09:29:26.057 INFO:tasks.workunit.client.1.smithi138.stdout:1/101: dread d4/f24 [0,4194304] 0 2024-05-27T09:29:26.057 INFO:tasks.workunit.client.1.smithi138.stdout:1/102: write d4/f12 [900718,85326] 0 2024-05-27T09:29:26.057 INFO:tasks.workunit.client.1.smithi138.stdout:2/44: creat d1/d10/db/f11 x:0 0 0 2024-05-27T09:29:26.058 INFO:tasks.workunit.client.1.smithi138.stdout:2/45: read - d1/d10/db/f11 zero size 2024-05-27T09:29:26.058 INFO:tasks.workunit.client.0.smithi119.stdout:5/554: sync 2024-05-27T09:29:26.058 INFO:tasks.workunit.client.0.smithi119.stdout:5/555: creat d6/db/de/d27/d77/fbe x:0 0 0 2024-05-27T09:29:26.058 INFO:tasks.workunit.client.0.smithi119.stdout:5/556: dread - d6/dd/d46/d47/d4b/f59 zero size 2024-05-27T09:29:26.059 INFO:tasks.workunit.client.0.smithi119.stdout:5/557: creat d6/dd/d46/fbf x:0 0 0 2024-05-27T09:29:26.060 INFO:tasks.workunit.client.0.smithi119.stdout:9/436: sync 2024-05-27T09:29:26.065 INFO:tasks.workunit.client.1.smithi138.stdout:8/35: mkdir d3/d8 0 2024-05-27T09:29:26.065 INFO:tasks.workunit.client.1.smithi138.stdout:8/36: readlink d3/l7 0 2024-05-27T09:29:26.065 INFO:tasks.workunit.client.1.smithi138.stdout:8/37: creat d3/f9 x:0 0 0 2024-05-27T09:29:26.067 INFO:tasks.workunit.client.0.smithi119.stdout:1/637: creat d7/dd/d25/d29/d38/d3c/fca x:0 0 0 2024-05-27T09:29:26.067 INFO:tasks.workunit.client.0.smithi119.stdout:0/522: sync 2024-05-27T09:29:26.067 INFO:tasks.workunit.client.0.smithi119.stdout:0/523: fdatasync d2/d1a/f1e 0 2024-05-27T09:29:26.067 INFO:tasks.workunit.client.0.smithi119.stdout:4/562: sync 2024-05-27T09:29:26.067 INFO:tasks.workunit.client.0.smithi119.stdout:4/563: readlink d4/de/d66/d87/lac 0 2024-05-27T09:29:26.070 INFO:tasks.workunit.client.1.smithi138.stdout:1/103: mknod d4/d8/c25 0 2024-05-27T09:29:26.071 INFO:tasks.workunit.client.0.smithi119.stdout:6/542: dread d0/d2/d57/d37/f43 [0,4194304] 0 2024-05-27T09:29:26.072 INFO:tasks.workunit.client.1.smithi138.stdout:2/46: mknod d1/c12 0 2024-05-27T09:29:26.072 INFO:tasks.workunit.client.1.smithi138.stdout:2/47: chown d1/d10/db/fd 416328 1 2024-05-27T09:29:26.075 INFO:tasks.workunit.client.0.smithi119.stdout:0/524: unlink d2/d6/d23/d7a/f42 0 2024-05-27T09:29:26.075 INFO:tasks.workunit.client.0.smithi119.stdout:4/564: link d4/d1c/d39/c62 d4/de/d21/d40/d47/cae 0 2024-05-27T09:29:26.077 INFO:tasks.workunit.client.1.smithi138.stdout:6/63: truncate d3/fa 2475564 0 2024-05-27T09:29:26.077 INFO:tasks.workunit.client.1.smithi138.stdout:6/64: stat d3/f5 0 2024-05-27T09:29:26.081 INFO:tasks.workunit.client.1.smithi138.stdout:8/38: mknod d3/d8/ca 0 2024-05-27T09:29:26.081 INFO:tasks.workunit.client.1.smithi138.stdout:8/39: write d3/f9 [238962,80765] 0 2024-05-27T09:29:26.083 INFO:tasks.workunit.client.0.smithi119.stdout:5/558: dread d6/dd/f64 [4194304,4194304] 0 2024-05-27T09:29:26.084 INFO:tasks.workunit.client.1.smithi138.stdout:1/104: mknod d4/d1a/c26 0 2024-05-27T09:29:26.084 INFO:tasks.workunit.client.1.smithi138.stdout:6/65: symlink d3/d14/l18 0 2024-05-27T09:29:26.084 INFO:tasks.workunit.client.1.smithi138.stdout:6/66: fsync d3/f5 0 2024-05-27T09:29:26.088 INFO:tasks.workunit.client.1.smithi138.stdout:8/40: unlink d3/l7 0 2024-05-27T09:29:26.090 INFO:tasks.workunit.client.1.smithi138.stdout:8/41: mknod d3/d8/cb 0 2024-05-27T09:29:26.094 INFO:tasks.workunit.client.1.smithi138.stdout:8/42: rename d3/f9 to d3/d8/fc 0 2024-05-27T09:29:26.095 INFO:tasks.workunit.client.1.smithi138.stdout:9/30: sync 2024-05-27T09:29:26.096 INFO:tasks.workunit.client.1.smithi138.stdout:3/21: sync 2024-05-27T09:29:26.096 INFO:tasks.workunit.client.1.smithi138.stdout:8/43: mknod d3/d8/cd 0 2024-05-27T09:29:26.096 INFO:tasks.workunit.client.1.smithi138.stdout:8/44: write d3/d8/fc [775305,121040] 0 2024-05-27T09:29:26.097 INFO:tasks.workunit.client.0.smithi119.stdout:6/543: dread d0/d2/d4e/f1f [0,4194304] 0 2024-05-27T09:29:26.098 INFO:tasks.workunit.client.0.smithi119.stdout:6/544: creat d0/d2/d1c/d21/d58/fbd x:0 0 0 2024-05-27T09:29:26.098 INFO:tasks.workunit.client.1.smithi138.stdout:9/31: mkdir d6/db 0 2024-05-27T09:29:26.099 INFO:tasks.workunit.client.1.smithi138.stdout:3/22: symlink d0/l5 0 2024-05-27T09:29:26.099 INFO:tasks.workunit.client.1.smithi138.stdout:3/23: chown d0/d2/c3 1060572 1 2024-05-27T09:29:26.099 INFO:tasks.workunit.client.1.smithi138.stdout:3/24: fsync - no filename 2024-05-27T09:29:26.099 INFO:tasks.workunit.client.1.smithi138.stdout:3/25: dread - no filename 2024-05-27T09:29:26.100 INFO:tasks.workunit.client.1.smithi138.stdout:9/32: mkdir d6/db/dc 0 2024-05-27T09:29:26.100 INFO:tasks.workunit.client.1.smithi138.stdout:9/33: chown f1 100013459 1 2024-05-27T09:29:26.100 INFO:tasks.workunit.client.1.smithi138.stdout:8/45: rmdir d3/d8 39 2024-05-27T09:29:26.100 INFO:tasks.workunit.client.1.smithi138.stdout:8/46: write f1 [3374750,102146] 0 2024-05-27T09:29:26.101 INFO:tasks.workunit.client.1.smithi138.stdout:3/26: link d0/c4 d0/d2/c6 0 2024-05-27T09:29:26.101 INFO:tasks.workunit.client.1.smithi138.stdout:3/27: write - no filename 2024-05-27T09:29:26.101 INFO:tasks.workunit.client.1.smithi138.stdout:3/28: rename d0 to d0/d2/d7 22 2024-05-27T09:29:26.101 INFO:tasks.workunit.client.1.smithi138.stdout:3/29: truncate - no filename 2024-05-27T09:29:26.102 INFO:tasks.workunit.client.1.smithi138.stdout:9/34: truncate d6/f9 4017205 0 2024-05-27T09:29:26.104 INFO:tasks.workunit.client.1.smithi138.stdout:8/47: mknod d3/d8/ce 0 2024-05-27T09:29:26.105 INFO:tasks.workunit.client.1.smithi138.stdout:3/30: mkdir d0/d8 0 2024-05-27T09:29:26.107 INFO:tasks.workunit.client.0.smithi119.stdout:1/638: dread d7/dd/d44/f56 [0,4194304] 0 2024-05-27T09:29:26.110 INFO:tasks.workunit.client.0.smithi119.stdout:1/639: mkdir d7/dd/dcb 0 2024-05-27T09:29:26.110 INFO:tasks.workunit.client.0.smithi119.stdout:1/640: creat d7/db/d52/fcc x:0 0 0 2024-05-27T09:29:26.110 INFO:tasks.workunit.client.1.smithi138.stdout:3/31: creat d0/d8/f9 x:0 0 0 2024-05-27T09:29:26.111 INFO:tasks.workunit.client.1.smithi138.stdout:3/32: write d0/d8/f9 [783106,61958] 0 2024-05-27T09:29:26.129 INFO:tasks.workunit.client.0.smithi119.stdout:1/641: rmdir d7/dd/d44/d47/d95 0 2024-05-27T09:29:26.129 INFO:tasks.workunit.client.0.smithi119.stdout:1/642: fdatasync d7/db/d12/d26/d4a/f81 0 2024-05-27T09:29:26.131 INFO:tasks.workunit.client.0.smithi119.stdout:0/525: read d2/d6/d9/f6d [2516374,80182] 0 2024-05-27T09:29:26.131 INFO:tasks.workunit.client.0.smithi119.stdout:0/526: chown d2/d6/d9/d12/d4b/d59/f7d 15 1 2024-05-27T09:29:26.131 INFO:tasks.workunit.client.0.smithi119.stdout:0/527: read d2/d6/d9/d12/d4f/f5a [2274207,40452] 0 2024-05-27T09:29:26.133 INFO:tasks.workunit.client.1.smithi138.stdout:1/105: dread d4/f7 [0,4194304] 0 2024-05-27T09:29:26.138 INFO:tasks.workunit.client.1.smithi138.stdout:0/40: dwrite da/f10 [0,4194304] 0 2024-05-27T09:29:26.139 INFO:tasks.workunit.client.1.smithi138.stdout:0/41: rename da/fb to da/f11 0 2024-05-27T09:29:26.140 INFO:tasks.workunit.client.1.smithi138.stdout:0/42: write f5 [37545,52331] 0 2024-05-27T09:29:26.142 INFO:tasks.workunit.client.0.smithi119.stdout:8/528: dwrite d3/d8/d4d/f32 [4194304,4194304] 0 2024-05-27T09:29:26.142 INFO:tasks.workunit.client.0.smithi119.stdout:2/573: dwrite d2/da/d6f/f7f [0,4194304] 0 2024-05-27T09:29:26.144 INFO:tasks.workunit.client.1.smithi138.stdout:0/43: creat da/f12 x:0 0 0 2024-05-27T09:29:26.145 INFO:tasks.workunit.client.1.smithi138.stdout:0/44: fdatasync f1 0 2024-05-27T09:29:26.152 INFO:tasks.workunit.client.0.smithi119.stdout:1/643: getdents d7/db 0 2024-05-27T09:29:26.152 INFO:tasks.workunit.client.0.smithi119.stdout:1/644: chown d7/dd/d44/d47/c59 21130 1 2024-05-27T09:29:26.152 INFO:tasks.workunit.client.0.smithi119.stdout:1/645: readlink d7/db/d12/d26/d4a/l70 0 2024-05-27T09:29:26.153 INFO:tasks.workunit.client.0.smithi119.stdout:1/646: write d7/db/d12/d26/d4a/f81 [783241,64581] 0 2024-05-27T09:29:26.159 INFO:tasks.workunit.client.0.smithi119.stdout:0/528: link d2/d6/d9/d24/d30/d4a/c65 d2/d6/d23/d7a/d53/ca5 0 2024-05-27T09:29:26.160 INFO:tasks.workunit.client.0.smithi119.stdout:0/529: truncate d2/d6/d75/f8a 12558 0 2024-05-27T09:29:26.165 INFO:tasks.workunit.client.0.smithi119.stdout:5/559: dwrite d6/db/f71 [0,4194304] 0 2024-05-27T09:29:26.165 INFO:tasks.workunit.client.0.smithi119.stdout:5/560: dread - d6/f90 zero size 2024-05-27T09:29:26.167 INFO:tasks.workunit.client.0.smithi119.stdout:9/437: dwrite d18/d19/f4e [0,4194304] 0 2024-05-27T09:29:26.167 INFO:tasks.workunit.client.0.smithi119.stdout:9/438: fdatasync d18/d19/d1b/d40/d44/f54 0 2024-05-27T09:29:26.168 INFO:tasks.workunit.client.1.smithi138.stdout:0/45: write da/f10 [2337344,103351] 0 2024-05-27T09:29:26.170 INFO:tasks.workunit.client.0.smithi119.stdout:2/574: mknod d2/da/d15/d33/d89/cc2 0 2024-05-27T09:29:26.170 INFO:tasks.workunit.client.0.smithi119.stdout:2/575: chown d2/da/d15/d33/f7e 6 1 2024-05-27T09:29:26.176 INFO:tasks.workunit.client.1.smithi138.stdout:7/37: dwrite d0/fb [4194304,4194304] 0 2024-05-27T09:29:26.176 INFO:tasks.workunit.client.0.smithi119.stdout:1/647: fdatasync d7/dd/d25/f30 0 2024-05-27T09:29:26.177 INFO:tasks.workunit.client.0.smithi119.stdout:1/648: chown d7/dd/d25/d29 838884 1 2024-05-27T09:29:26.177 INFO:tasks.workunit.client.1.smithi138.stdout:7/38: read d0/fb [2437295,19834] 0 2024-05-27T09:29:26.178 INFO:tasks.workunit.client.1.smithi138.stdout:7/39: symlink d0/d7/lc 0 2024-05-27T09:29:26.178 INFO:tasks.workunit.client.1.smithi138.stdout:7/40: fsync d0/fb 0 2024-05-27T09:29:26.179 INFO:tasks.workunit.client.0.smithi119.stdout:0/530: symlink d2/d6/d9/d2f/la6 0 2024-05-27T09:29:26.180 INFO:tasks.workunit.client.1.smithi138.stdout:7/41: unlink d0/c3 0 2024-05-27T09:29:26.181 INFO:tasks.workunit.client.1.smithi138.stdout:7/42: mknod d0/d7/cd 0 2024-05-27T09:29:26.181 INFO:tasks.workunit.client.1.smithi138.stdout:7/43: write d0/fb [1641608,126145] 0 2024-05-27T09:29:26.190 INFO:tasks.workunit.client.1.smithi138.stdout:3/33: fsync d0/d8/f9 0 2024-05-27T09:29:26.192 INFO:tasks.workunit.client.0.smithi119.stdout:5/561: mkdir d6/db/d9a/dc0 0 2024-05-27T09:29:26.194 INFO:tasks.workunit.client.1.smithi138.stdout:3/34: mkdir d0/d2/da 0 2024-05-27T09:29:26.199 INFO:tasks.workunit.client.1.smithi138.stdout:9/35: dread d6/f9 [0,4194304] 0 2024-05-27T09:29:26.201 INFO:tasks.workunit.client.1.smithi138.stdout:3/35: link d0/c4 d0/d2/da/cb 0 2024-05-27T09:29:26.206 INFO:tasks.workunit.client.1.smithi138.stdout:6/67: dwrite d3/f17 [0,4194304] 0 2024-05-27T09:29:26.210 INFO:tasks.workunit.client.1.smithi138.stdout:3/36: creat d0/d2/fc x:0 0 0 2024-05-27T09:29:26.217 INFO:tasks.workunit.client.1.smithi138.stdout:9/36: dread d6/f9 [0,4194304] 0 2024-05-27T09:29:26.218 INFO:tasks.workunit.client.0.smithi119.stdout:9/439: creat d18/d19/d61/f92 x:0 0 0 2024-05-27T09:29:26.218 INFO:tasks.workunit.client.0.smithi119.stdout:9/440: truncate f2 10245518 0 2024-05-27T09:29:26.219 INFO:tasks.workunit.client.1.smithi138.stdout:9/37: mknod d6/cd 0 2024-05-27T09:29:26.223 INFO:tasks.workunit.client.0.smithi119.stdout:1/649: mkdir d7/dd/db0/dcd 0 2024-05-27T09:29:26.223 INFO:tasks.workunit.client.0.smithi119.stdout:1/650: chown d7/dd/d44/c5c 19221 1 2024-05-27T09:29:26.229 INFO:tasks.workunit.client.1.smithi138.stdout:3/37: dread d0/d8/f9 [0,4194304] 0 2024-05-27T09:29:26.229 INFO:tasks.workunit.client.1.smithi138.stdout:3/38: dread - d0/d2/fc zero size 2024-05-27T09:29:26.234 INFO:tasks.workunit.client.0.smithi119.stdout:1/651: mkdir d7/db/d8f/d9b/dce 0 2024-05-27T09:29:26.239 INFO:tasks.workunit.client.0.smithi119.stdout:5/562: rename d6/dd/df to d6/dd/d39/d19/d70/dc1 0 2024-05-27T09:29:26.239 INFO:tasks.workunit.client.0.smithi119.stdout:5/563: write d6/db/de/d27/d77/fb9 [585339,15117] 0 2024-05-27T09:29:26.242 INFO:tasks.workunit.client.1.smithi138.stdout:4/35: dwrite f0 [4194304,4194304] 0 2024-05-27T09:29:26.242 INFO:tasks.workunit.client.1.smithi138.stdout:4/36: chown d7/f9 6638 1 2024-05-27T09:29:26.243 INFO:tasks.workunit.client.0.smithi119.stdout:8/529: dwrite d3/de/f1d [0,4194304] 0 2024-05-27T09:29:26.247 INFO:tasks.workunit.client.0.smithi119.stdout:9/441: rename d18/d19/d1b/d40/f8e to d18/d19/d1b/d40/d44/f93 0 2024-05-27T09:29:26.248 INFO:tasks.workunit.client.0.smithi119.stdout:5/564: mkdir d6/dd/d39/d19/d7d/dc2 0 2024-05-27T09:29:26.249 INFO:tasks.workunit.client.0.smithi119.stdout:1/652: getdents d7/db/d8f 0 2024-05-27T09:29:26.249 INFO:tasks.workunit.client.0.smithi119.stdout:1/653: stat d7/dd/d25/d29/d38/d49/c67 0 2024-05-27T09:29:26.251 INFO:tasks.workunit.client.0.smithi119.stdout:9/442: rmdir d18/d19/d1b 39 2024-05-27T09:29:26.251 INFO:tasks.workunit.client.0.smithi119.stdout:9/443: fsync d18/d19/d1b/d40/d7b/f7e 0 2024-05-27T09:29:26.251 INFO:tasks.workunit.client.0.smithi119.stdout:5/565: mkdir d6/db/dc/dc3 0 2024-05-27T09:29:26.253 INFO:tasks.workunit.client.0.smithi119.stdout:5/566: mknod d6/dd/d46/cc4 0 2024-05-27T09:29:26.255 INFO:tasks.workunit.client.0.smithi119.stdout:8/530: dread d3/d8/f34 [0,4194304] 0 2024-05-27T09:29:26.255 INFO:tasks.workunit.client.0.smithi119.stdout:5/567: mknod d6/d2f/cc5 0 2024-05-27T09:29:26.256 INFO:tasks.workunit.client.0.smithi119.stdout:5/568: readlink d6/db/dc/l44 0 2024-05-27T09:29:26.256 INFO:tasks.workunit.client.0.smithi119.stdout:5/569: creat d6/dd/d39/d19/d84/fc6 x:0 0 0 2024-05-27T09:29:26.256 INFO:tasks.workunit.client.0.smithi119.stdout:5/570: truncate d6/dd/d39/d19/fb6 683953 0 2024-05-27T09:29:26.260 INFO:tasks.workunit.client.0.smithi119.stdout:8/531: getdents d3/d8 0 2024-05-27T09:29:26.265 INFO:tasks.workunit.client.1.smithi138.stdout:8/48: dwrite d3/d8/fc [0,4194304] 0 2024-05-27T09:29:26.265 INFO:tasks.workunit.client.1.smithi138.stdout:0/46: dwrite f1 [4194304,4194304] 0 2024-05-27T09:29:26.266 INFO:tasks.workunit.client.1.smithi138.stdout:0/47: write da/f11 [827235,94959] 0 2024-05-27T09:29:26.268 INFO:tasks.workunit.client.1.smithi138.stdout:6/68: dwrite d3/f17 [0,4194304] 0 2024-05-27T09:29:26.275 INFO:tasks.workunit.client.1.smithi138.stdout:0/48: rename da/cc to da/c13 0 2024-05-27T09:29:26.280 INFO:tasks.workunit.client.1.smithi138.stdout:0/49: rename c7 to da/c14 0 2024-05-27T09:29:26.280 INFO:tasks.workunit.client.1.smithi138.stdout:0/50: creat da/f15 x:0 0 0 2024-05-27T09:29:26.281 INFO:tasks.workunit.client.0.smithi119.stdout:6/545: dwrite d0/d19/f55 [0,4194304] 0 2024-05-27T09:29:26.281 INFO:tasks.workunit.client.0.smithi119.stdout:2/576: dwrite d2/f66 [4194304,4194304] 0 2024-05-27T09:29:26.282 INFO:tasks.workunit.client.1.smithi138.stdout:6/69: dread d3/f5 [0,4194304] 0 2024-05-27T09:29:26.285 INFO:tasks.workunit.client.1.smithi138.stdout:0/51: mknod da/c16 0 2024-05-27T09:29:26.287 INFO:tasks.workunit.client.1.smithi138.stdout:6/70: mkdir d3/d19 0 2024-05-27T09:29:26.288 INFO:tasks.workunit.client.1.smithi138.stdout:6/71: truncate d3/d11/fe 1046062 0 2024-05-27T09:29:26.290 INFO:tasks.workunit.client.1.smithi138.stdout:0/52: mkdir da/d17 0 2024-05-27T09:29:26.290 INFO:tasks.workunit.client.1.smithi138.stdout:0/53: chown f1 29575906 1 2024-05-27T09:29:26.291 INFO:tasks.workunit.client.1.smithi138.stdout:6/72: rename d3/d11/c15 to d3/d19/c1a 0 2024-05-27T09:29:26.292 INFO:tasks.workunit.client.0.smithi119.stdout:6/546: truncate d0/d2/d1c/d21/f2b 3502490 0 2024-05-27T09:29:26.292 INFO:tasks.workunit.client.0.smithi119.stdout:6/547: fdatasync d0/d2/d1c/d21/d58/f89 0 2024-05-27T09:29:26.292 INFO:tasks.workunit.client.0.smithi119.stdout:6/548: fsync d0/d2/d4e/d5a/f91 0 2024-05-27T09:29:26.294 INFO:tasks.workunit.client.1.smithi138.stdout:0/54: creat da/d17/f18 x:0 0 0 2024-05-27T09:29:26.294 INFO:tasks.workunit.client.1.smithi138.stdout:0/55: creat da/d17/f19 x:0 0 0 2024-05-27T09:29:26.294 INFO:tasks.workunit.client.0.smithi119.stdout:6/549: rename d0/d19/lab to d0/d19/lbe 0 2024-05-27T09:29:26.296 INFO:tasks.workunit.client.1.smithi138.stdout:6/73: creat d3/d14/f1b x:0 0 0 2024-05-27T09:29:26.296 INFO:tasks.workunit.client.1.smithi138.stdout:6/74: chown f2 1777507 1 2024-05-27T09:29:26.297 INFO:tasks.workunit.client.0.smithi119.stdout:6/550: unlink d0/d2/d57/d37/dad/fb4 0 2024-05-27T09:29:26.298 INFO:tasks.workunit.client.0.smithi119.stdout:6/551: mknod d0/d2/d57/d37/d74/d65/cbf 0 2024-05-27T09:29:26.299 INFO:tasks.workunit.client.1.smithi138.stdout:0/56: rename da/cf to da/c1a 0 2024-05-27T09:29:26.302 INFO:tasks.workunit.client.0.smithi119.stdout:0/531: dwrite d2/d6/d9/d12/d4b/d96/f98 [0,4194304] 0 2024-05-27T09:29:26.303 INFO:tasks.workunit.client.1.smithi138.stdout:6/75: mkdir d3/d14/d1c 0 2024-05-27T09:29:26.303 INFO:tasks.workunit.client.1.smithi138.stdout:6/76: dread - d3/d14/f1b zero size 2024-05-27T09:29:26.306 INFO:tasks.workunit.client.0.smithi119.stdout:2/577: dread d2/da/d15/d33/d69/f9d [0,4194304] 0 2024-05-27T09:29:26.307 INFO:tasks.workunit.client.1.smithi138.stdout:4/37: dwrite d7/f9 [0,4194304] 0 2024-05-27T09:29:26.307 INFO:tasks.workunit.client.1.smithi138.stdout:0/57: mknod da/d17/c1b 0 2024-05-27T09:29:26.309 INFO:tasks.workunit.client.0.smithi119.stdout:8/532: dwrite d3/d8/d4d/d58/f82 [0,4194304] 0 2024-05-27T09:29:26.309 INFO:tasks.workunit.client.0.smithi119.stdout:2/578: mknod d2/da/cc3 0 2024-05-27T09:29:26.310 INFO:tasks.workunit.client.1.smithi138.stdout:6/77: rename d3/l4 to d3/d11/l1d 0 2024-05-27T09:29:26.314 INFO:tasks.workunit.client.1.smithi138.stdout:4/38: creat d7/fb x:0 0 0 2024-05-27T09:29:26.314 INFO:tasks.workunit.client.0.smithi119.stdout:8/533: mknod d3/d6e/d80/d84/cac 0 2024-05-27T09:29:26.314 INFO:tasks.workunit.client.0.smithi119.stdout:2/579: symlink d2/da/d15/d33/d69/lc4 0 2024-05-27T09:29:26.314 INFO:tasks.workunit.client.0.smithi119.stdout:2/580: fdatasync d2/da/d15/d33/d89/fa3 0 2024-05-27T09:29:26.316 INFO:tasks.workunit.client.0.smithi119.stdout:2/581: symlink d2/da/d4e/lc5 0 2024-05-27T09:29:26.317 INFO:tasks.workunit.client.0.smithi119.stdout:5/571: dwrite d6/dd/d39/d19/d3c/d52/f99 [0,4194304] 0 2024-05-27T09:29:26.318 INFO:tasks.workunit.client.0.smithi119.stdout:8/534: truncate d3/d8/d4d/d58/f6a 353316 0 2024-05-27T09:29:26.318 INFO:tasks.workunit.client.0.smithi119.stdout:8/535: chown d3/d39/f61 87592 1 2024-05-27T09:29:26.320 INFO:tasks.workunit.client.0.smithi119.stdout:5/572: rmdir d6/db/de 39 2024-05-27T09:29:26.321 INFO:tasks.workunit.client.0.smithi119.stdout:5/573: symlink d6/db/lc7 0 2024-05-27T09:29:26.321 INFO:tasks.workunit.client.0.smithi119.stdout:5/574: fdatasync d6/db/d9a/f9e 0 2024-05-27T09:29:26.322 INFO:tasks.workunit.client.1.smithi138.stdout:0/58: getdents da 0 2024-05-27T09:29:26.322 INFO:tasks.workunit.client.1.smithi138.stdout:0/59: dread - da/fd zero size 2024-05-27T09:29:26.322 INFO:tasks.workunit.client.1.smithi138.stdout:0/60: mknod da/d17/c1c 0 2024-05-27T09:29:26.322 INFO:tasks.workunit.client.1.smithi138.stdout:0/61: chown da/f11 66040268 1 2024-05-27T09:29:26.324 INFO:tasks.workunit.client.1.smithi138.stdout:0/62: unlink da/fd 0 2024-05-27T09:29:26.329 INFO:tasks.workunit.client.1.smithi138.stdout:0/63: mkdir da/d17/d1d 0 2024-05-27T09:29:26.330 INFO:tasks.workunit.client.1.smithi138.stdout:4/39: rmdir d7 39 2024-05-27T09:29:26.333 INFO:tasks.workunit.client.0.smithi119.stdout:5/575: creat d6/fc8 x:0 0 0 2024-05-27T09:29:26.333 INFO:tasks.workunit.client.0.smithi119.stdout:0/532: dread d2/d6/d9/f2c [0,4194304] 0 2024-05-27T09:29:26.336 INFO:tasks.workunit.client.0.smithi119.stdout:0/533: creat d2/d6/d9/d12/d4b/fa7 x:0 0 0 2024-05-27T09:29:26.337 INFO:tasks.workunit.client.1.smithi138.stdout:0/64: dread da/f11 [0,4194304] 0 2024-05-27T09:29:26.337 INFO:tasks.workunit.client.1.smithi138.stdout:0/65: dread - da/d17/f19 zero size 2024-05-27T09:29:26.338 INFO:tasks.workunit.client.1.smithi138.stdout:0/66: mkdir da/d17/d1d/d1e 0 2024-05-27T09:29:26.340 INFO:tasks.workunit.client.1.smithi138.stdout:0/67: rename da/d17/d1d/d1e to da/d17/d1d/d1f 0 2024-05-27T09:29:26.341 INFO:tasks.workunit.client.0.smithi119.stdout:0/534: mknod d2/d6/d9/d24/d30/ca8 0 2024-05-27T09:29:26.341 INFO:tasks.workunit.client.0.smithi119.stdout:0/535: fdatasync d2/d6/d75/fa2 0 2024-05-27T09:29:26.344 INFO:tasks.workunit.client.0.smithi119.stdout:0/536: rename d2/d1a/f7c to d2/d6/d23/fa9 0 2024-05-27T09:29:26.345 INFO:tasks.workunit.client.1.smithi138.stdout:0/68: creat da/f20 x:0 0 0 2024-05-27T09:29:26.346 INFO:tasks.workunit.client.1.smithi138.stdout:0/69: creat da/d17/d1d/d1f/f21 x:0 0 0 2024-05-27T09:29:26.347 INFO:tasks.workunit.client.1.smithi138.stdout:8/49: dwrite d3/f5 [0,4194304] 0 2024-05-27T09:29:26.349 INFO:tasks.workunit.client.0.smithi119.stdout:8/536: dread d3/d8/d1b/f6c [0,4194304] 0 2024-05-27T09:29:26.349 INFO:tasks.workunit.client.1.smithi138.stdout:4/40: read d7/f9 [2564231,81154] 0 2024-05-27T09:29:26.349 INFO:tasks.workunit.client.1.smithi138.stdout:4/41: readlink l6 0 2024-05-27T09:29:26.350 INFO:tasks.workunit.client.0.smithi119.stdout:8/537: mkdir d3/de/dad 0 2024-05-27T09:29:26.351 INFO:tasks.workunit.client.0.smithi119.stdout:8/538: chown d3/d8/d4d/d5d/d60/c6f 70878873 1 2024-05-27T09:29:26.352 INFO:tasks.workunit.client.1.smithi138.stdout:8/50: link d3/f5 d3/ff 0 2024-05-27T09:29:26.352 INFO:tasks.workunit.client.0.smithi119.stdout:8/539: getdents d3/d6e 0 2024-05-27T09:29:26.353 INFO:tasks.workunit.client.0.smithi119.stdout:5/576: dread d6/f80 [0,4194304] 0 2024-05-27T09:29:26.353 INFO:tasks.workunit.client.1.smithi138.stdout:4/42: creat d7/fc x:0 0 0 2024-05-27T09:29:26.355 INFO:tasks.workunit.client.1.smithi138.stdout:8/51: link d3/d8/ca d3/d8/c10 0 2024-05-27T09:29:26.357 INFO:tasks.workunit.client.1.smithi138.stdout:4/43: mknod d7/cd 0 2024-05-27T09:29:26.357 INFO:tasks.workunit.client.1.smithi138.stdout:4/44: chown d7/f9 6575746 1 2024-05-27T09:29:26.357 INFO:tasks.workunit.client.1.smithi138.stdout:4/45: dread - d7/fb zero size 2024-05-27T09:29:26.358 INFO:tasks.workunit.client.0.smithi119.stdout:8/540: write d3/d8/f47 [23557,120053] 0 2024-05-27T09:29:26.359 INFO:tasks.workunit.client.1.smithi138.stdout:4/46: mkdir d7/de 0 2024-05-27T09:29:26.361 INFO:tasks.workunit.client.0.smithi119.stdout:5/577: rename d6/dd/d39/d19/d84 to d6/db/de/d27/dc9 0 2024-05-27T09:29:26.362 INFO:tasks.workunit.client.0.smithi119.stdout:5/578: symlink d6/dd/d39/lca 0 2024-05-27T09:29:26.362 INFO:tasks.workunit.client.0.smithi119.stdout:5/579: truncate d6/dd/d39/d19/d70/dc1/d41/fb0 481750 0 2024-05-27T09:29:26.365 INFO:tasks.workunit.client.0.smithi119.stdout:5/580: rename d6/dd/d39/d74/fa0 to d6/db/d9a/dc0/fcb 0 2024-05-27T09:29:26.367 INFO:tasks.workunit.client.0.smithi119.stdout:5/581: rename d6/c88 to d6/dd/d39/d19/d7d/db7/ccc 0 2024-05-27T09:29:26.372 INFO:tasks.workunit.client.0.smithi119.stdout:7/578: sync 2024-05-27T09:29:26.373 INFO:tasks.workunit.client.0.smithi119.stdout:5/582: link d6/dd/d46/d47/l83 d6/db/de/d27/dc9/lcd 0 2024-05-27T09:29:26.374 INFO:tasks.workunit.client.0.smithi119.stdout:5/583: write d6/dd/d39/d19/fac [529704,91008] 0 2024-05-27T09:29:26.374 INFO:tasks.workunit.client.0.smithi119.stdout:5/584: chown d6 5 1 2024-05-27T09:29:26.376 INFO:tasks.workunit.client.0.smithi119.stdout:7/579: truncate d7/df/d36/d42/f73 2844191 0 2024-05-27T09:29:26.380 INFO:tasks.workunit.client.1.smithi138.stdout:4/47: rmdir d7 39 2024-05-27T09:29:26.380 INFO:tasks.workunit.client.1.smithi138.stdout:4/48: readlink l6 0 2024-05-27T09:29:26.381 INFO:tasks.workunit.client.0.smithi119.stdout:8/541: dread d3/de/f2c [0,4194304] 0 2024-05-27T09:29:26.383 INFO:tasks.workunit.client.0.smithi119.stdout:8/542: stat d3/d8/d1b/d46/f43 0 2024-05-27T09:29:26.383 INFO:tasks.workunit.client.0.smithi119.stdout:8/543: creat d3/d8/d1b/d77/d95/fae x:0 0 0 2024-05-27T09:29:26.383 INFO:tasks.workunit.client.0.smithi119.stdout:8/544: chown d3/d39/d5a 58185284 1 2024-05-27T09:29:26.387 INFO:tasks.workunit.client.0.smithi119.stdout:8/545: truncate d3/d8/fd 1077260 0 2024-05-27T09:29:26.388 INFO:tasks.workunit.client.1.smithi138.stdout:4/49: rmdir d7/de 0 2024-05-27T09:29:26.390 INFO:tasks.workunit.client.0.smithi119.stdout:8/546: unlink d3/d8/d1b/d46/f29 0 2024-05-27T09:29:26.391 INFO:tasks.workunit.client.0.smithi119.stdout:8/547: write d3/d8/d4d/d58/f6a [45100,51391] 0 2024-05-27T09:29:26.394 INFO:tasks.workunit.client.0.smithi119.stdout:8/548: creat d3/faf x:0 0 0 2024-05-27T09:29:26.394 INFO:tasks.workunit.client.0.smithi119.stdout:8/549: read d3/d39/f3e [310293,26937] 0 2024-05-27T09:29:26.402 INFO:tasks.workunit.client.0.smithi119.stdout:7/580: read d7/df/d19/d97/d37/f5e [1926655,43825] 0 2024-05-27T09:29:26.402 INFO:tasks.workunit.client.0.smithi119.stdout:7/581: read f1 [4631523,85970] 0 2024-05-27T09:29:26.415 INFO:tasks.workunit.client.1.smithi138.stdout:8/52: dwrite f1 [4194304,4194304] 0 2024-05-27T09:29:26.442 INFO:tasks.workunit.client.0.smithi119.stdout:8/550: write d3/de/f1d [986322,88625] 0 2024-05-27T09:29:26.444 INFO:tasks.workunit.client.0.smithi119.stdout:8/551: creat d3/d6e/d8e/fb0 x:0 0 0 2024-05-27T09:29:26.445 INFO:tasks.workunit.client.0.smithi119.stdout:5/585: dwrite d6/dd/d46/fbf [0,4194304] 0 2024-05-27T09:29:26.445 INFO:tasks.workunit.client.0.smithi119.stdout:5/586: truncate d6/db/dc/f7f 877893 0 2024-05-27T09:29:26.448 INFO:tasks.workunit.client.0.smithi119.stdout:8/552: mkdir d3/d8/d1b/db1 0 2024-05-27T09:29:26.451 INFO:tasks.workunit.client.0.smithi119.stdout:8/553: mknod d3/d8/d1b/db1/cb2 0 2024-05-27T09:29:26.452 INFO:tasks.workunit.client.0.smithi119.stdout:8/554: write d3/d8/d4d/d58/f82 [4280955,120480] 0 2024-05-27T09:29:26.454 INFO:tasks.workunit.client.0.smithi119.stdout:0/537: dwrite d2/d6/d75/fa2 [0,4194304] 0 2024-05-27T09:29:26.454 INFO:tasks.workunit.client.0.smithi119.stdout:0/538: chown d2/l10 1 1 2024-05-27T09:29:26.454 INFO:tasks.workunit.client.0.smithi119.stdout:8/555: symlink d3/d8/d4d/d5d/d66/lb3 0 2024-05-27T09:29:26.454 INFO:tasks.workunit.client.0.smithi119.stdout:8/556: fsync d3/d39/f72 0 2024-05-27T09:29:26.456 INFO:tasks.workunit.client.0.smithi119.stdout:8/557: symlink d3/d8/d4d/d58/lb4 0 2024-05-27T09:29:26.456 INFO:tasks.workunit.client.0.smithi119.stdout:8/558: readlink d3/d8/d4d/l57 0 2024-05-27T09:29:26.456 INFO:tasks.workunit.client.0.smithi119.stdout:8/559: read d3/de/f1d [3438080,110855] 0 2024-05-27T09:29:26.458 INFO:tasks.workunit.client.0.smithi119.stdout:8/560: creat d3/d39/d5a/fb5 x:0 0 0 2024-05-27T09:29:26.459 INFO:tasks.workunit.client.1.smithi138.stdout:0/70: dwrite f5 [0,4194304] 0 2024-05-27T09:29:26.459 INFO:tasks.workunit.client.1.smithi138.stdout:0/71: fdatasync da/f12 0 2024-05-27T09:29:26.459 INFO:tasks.workunit.client.1.smithi138.stdout:0/72: chown da/le 441390 1 2024-05-27T09:29:26.459 INFO:tasks.workunit.client.0.smithi119.stdout:7/582: dwrite d7/f66 [0,4194304] 0 2024-05-27T09:29:26.460 INFO:tasks.workunit.client.0.smithi119.stdout:8/561: symlink d3/d8/d4d/d5d/d66/d85/lb6 0 2024-05-27T09:29:26.462 INFO:tasks.workunit.client.1.smithi138.stdout:0/73: truncate da/f11 490062 0 2024-05-27T09:29:26.462 INFO:tasks.workunit.client.1.smithi138.stdout:0/74: creat da/f22 x:0 0 0 2024-05-27T09:29:26.463 INFO:tasks.workunit.client.0.smithi119.stdout:0/539: dread d2/d6/d9/d12/f77 [0,4194304] 0 2024-05-27T09:29:26.463 INFO:tasks.workunit.client.0.smithi119.stdout:0/540: chown d2/d6/d9/d24/d30 0 1 2024-05-27T09:29:26.466 INFO:tasks.workunit.client.1.smithi138.stdout:4/50: dwrite f0 [4194304,4194304] 0 2024-05-27T09:29:26.467 INFO:tasks.workunit.client.1.smithi138.stdout:4/51: readlink l6 0 2024-05-27T09:29:26.467 INFO:tasks.workunit.client.1.smithi138.stdout:4/52: chown d7/cd 9864 1 2024-05-27T09:29:26.467 INFO:tasks.workunit.client.0.smithi119.stdout:7/583: link d7/df/d19/f14 d7/df/d19/d78/fb4 0 2024-05-27T09:29:26.468 INFO:tasks.workunit.client.0.smithi119.stdout:0/541: mknod d2/d6/d9/d24/d30/d4a/caa 0 2024-05-27T09:29:26.469 INFO:tasks.workunit.client.0.smithi119.stdout:7/584: mkdir d7/df/d19/d46/db5 0 2024-05-27T09:29:26.471 INFO:tasks.workunit.client.0.smithi119.stdout:0/542: truncate d2/d6/d9/d12/d4b/d59/f58 1656922 0 2024-05-27T09:29:26.471 INFO:tasks.workunit.client.1.smithi138.stdout:4/53: mkdir d7/df 0 2024-05-27T09:29:26.471 INFO:tasks.workunit.client.0.smithi119.stdout:7/585: rmdir d7/df/d19/d8d 39 2024-05-27T09:29:26.472 INFO:tasks.workunit.client.0.smithi119.stdout:7/586: symlink d7/d5f/d79/lb6 0 2024-05-27T09:29:26.478 INFO:tasks.workunit.client.1.smithi138.stdout:8/53: dwrite d3/d8/fc [0,4194304] 0 2024-05-27T09:29:26.484 INFO:tasks.workunit.client.0.smithi119.stdout:8/562: read d3/d8/d4d/f32 [5775965,86217] 0 2024-05-27T09:29:26.486 INFO:tasks.workunit.client.0.smithi119.stdout:7/587: dread d7/df/d19/d78/fb4 [0,4194304] 0 2024-05-27T09:29:26.488 INFO:tasks.workunit.client.0.smithi119.stdout:7/588: mkdir d7/df/d19/d97/d92/db7 0 2024-05-27T09:29:26.490 INFO:tasks.workunit.client.0.smithi119.stdout:7/589: link d7/d1a/d40/d44/d67/c83 d7/d5f/cb8 0 2024-05-27T09:29:26.495 INFO:tasks.workunit.client.0.smithi119.stdout:2/582: sync 2024-05-27T09:29:26.495 INFO:tasks.workunit.client.0.smithi119.stdout:6/552: sync 2024-05-27T09:29:26.495 INFO:tasks.workunit.client.0.smithi119.stdout:1/654: sync 2024-05-27T09:29:26.495 INFO:tasks.workunit.client.0.smithi119.stdout:1/655: chown d7/db/d12/d26/l83 6 1 2024-05-27T09:29:26.495 INFO:tasks.workunit.client.0.smithi119.stdout:4/565: sync 2024-05-27T09:29:26.495 INFO:tasks.workunit.client.0.smithi119.stdout:4/566: fdatasync d4/de/d21/d40/f42 0 2024-05-27T09:29:26.495 INFO:tasks.workunit.client.0.smithi119.stdout:7/590: unlink d7/df/d36/d42/c6e 0 2024-05-27T09:29:26.495 INFO:tasks.workunit.client.0.smithi119.stdout:3/692: sync 2024-05-27T09:29:26.495 INFO:tasks.workunit.client.0.smithi119.stdout:9/444: sync 2024-05-27T09:29:26.496 INFO:tasks.workunit.client.0.smithi119.stdout:6/553: creat d0/d2/d4e/fc0 x:0 0 0 2024-05-27T09:29:26.496 INFO:tasks.workunit.client.0.smithi119.stdout:6/554: truncate d0/d19/fb0 333944 0 2024-05-27T09:29:26.498 INFO:tasks.workunit.client.0.smithi119.stdout:3/693: mknod db/d36/d45/d4c/cde 0 2024-05-27T09:29:26.498 INFO:tasks.workunit.client.0.smithi119.stdout:3/694: chown db/d36/d45/d4c/d62/d13/d33/l9a 44280 1 2024-05-27T09:29:26.499 INFO:tasks.workunit.client.0.smithi119.stdout:4/567: link d4/d6/d28/f34 d4/d6/d3e/faf 0 2024-05-27T09:29:26.503 INFO:tasks.workunit.client.0.smithi119.stdout:1/656: getdents d7 0 2024-05-27T09:29:26.505 INFO:tasks.workunit.client.0.smithi119.stdout:6/555: rename d0/d2/d4e/d5a/f95 to d0/d2/d1c/d21/d58/fc1 0 2024-05-27T09:29:26.506 INFO:tasks.workunit.client.0.smithi119.stdout:3/695: mknod db/dc9/daa/cdf 0 2024-05-27T09:29:26.507 INFO:tasks.workunit.client.0.smithi119.stdout:4/568: truncate d4/de/d21/d40/f7c 128467 0 2024-05-27T09:29:26.508 INFO:tasks.workunit.client.0.smithi119.stdout:7/591: getdents d7/df/d19/d46 0 2024-05-27T09:29:26.512 INFO:tasks.workunit.client.0.smithi119.stdout:3/696: symlink db/d7f/da9/le0 0 2024-05-27T09:29:26.512 INFO:tasks.workunit.client.0.smithi119.stdout:3/697: chown db/dc9/daa/cb2 2963 1 2024-05-27T09:29:26.513 INFO:tasks.workunit.client.0.smithi119.stdout:3/698: write db/d36/d45/d4c/dbe/fd7 [27769,90808] 0 2024-05-27T09:29:26.514 INFO:tasks.workunit.client.0.smithi119.stdout:9/445: read fe [2601314,75197] 0 2024-05-27T09:29:26.514 INFO:tasks.workunit.client.0.smithi119.stdout:9/446: write d18/d19/d61/f6d [272648,57480] 0 2024-05-27T09:29:26.519 INFO:tasks.workunit.client.0.smithi119.stdout:2/583: dread d2/d1a/f39 [0,4194304] 0 2024-05-27T09:29:26.524 INFO:tasks.workunit.client.0.smithi119.stdout:4/569: mknod d4/d6/d28/d2f/d36/cb0 0 2024-05-27T09:29:26.524 INFO:tasks.workunit.client.0.smithi119.stdout:7/592: symlink d7/df/d36/d42/lb9 0 2024-05-27T09:29:26.524 INFO:tasks.workunit.client.0.smithi119.stdout:7/593: readlink d7/l58 0 2024-05-27T09:29:26.524 INFO:tasks.workunit.client.0.smithi119.stdout:5/587: dwrite d6/fc8 [0,4194304] 0 2024-05-27T09:29:26.528 INFO:tasks.workunit.client.0.smithi119.stdout:6/556: rename d0/d2/d57/d37/l53 to d0/d2/d57/d37/d74/d65/lc2 0 2024-05-27T09:29:26.530 INFO:tasks.workunit.client.0.smithi119.stdout:9/447: mknod d18/d19/d61/c94 0 2024-05-27T09:29:26.536 INFO:tasks.workunit.client.0.smithi119.stdout:2/584: mknod d2/da/d15/d33/d89/db0/cc6 0 2024-05-27T09:29:26.536 INFO:tasks.workunit.client.0.smithi119.stdout:2/585: read - d2/da/d8a/f9f zero size 2024-05-27T09:29:26.541 INFO:tasks.workunit.client.1.smithi138.stdout:8/54: dwrite d3/d8/fc [0,4194304] 0 2024-05-27T09:29:26.543 INFO:tasks.workunit.client.0.smithi119.stdout:5/588: mknod d6/db/de/cce 0 2024-05-27T09:29:26.544 INFO:tasks.workunit.client.0.smithi119.stdout:5/589: readlink d6/db/de/d27/l3d 0 2024-05-27T09:29:26.544 INFO:tasks.workunit.client.0.smithi119.stdout:3/699: rename db/d36/d45/d4c/d62/d13/f2a to db/d36/d45/d49/d9d/fe1 0 2024-05-27T09:29:26.544 INFO:tasks.workunit.client.0.smithi119.stdout:6/557: fdatasync d0/d2/d57/d37/f8d 0 2024-05-27T09:29:26.544 INFO:tasks.workunit.client.0.smithi119.stdout:9/448: mkdir d18/d19/d61/d95 0 2024-05-27T09:29:26.544 INFO:tasks.workunit.client.0.smithi119.stdout:9/449: chown l14 8008 1 2024-05-27T09:29:26.549 INFO:tasks.workunit.client.0.smithi119.stdout:1/657: dread d7/dd/d44/f93 [0,4194304] 0 2024-05-27T09:29:26.549 INFO:tasks.workunit.client.0.smithi119.stdout:2/586: link d2/da/d6f/c88 d2/da/d8a/d8f/cc7 0 2024-05-27T09:29:26.556 INFO:tasks.workunit.client.0.smithi119.stdout:5/590: rename d6/db/de/d27/dc9/lcd to d6/dd/d39/d19/d3c/lcf 0 2024-05-27T09:29:26.556 INFO:tasks.workunit.client.0.smithi119.stdout:5/591: stat d6/dd/d39/d19/d3c/d52/l93 0 2024-05-27T09:29:26.556 INFO:tasks.workunit.client.0.smithi119.stdout:3/700: mkdir db/dc9/daa/de2 0 2024-05-27T09:29:26.556 INFO:tasks.workunit.client.0.smithi119.stdout:3/701: truncate db/d36/d45/fc1 1029106 0 2024-05-27T09:29:26.556 INFO:tasks.workunit.client.0.smithi119.stdout:6/558: unlink d0/d2/d1c/c78 0 2024-05-27T09:29:26.556 INFO:tasks.workunit.client.0.smithi119.stdout:6/559: fsync d0/d2/fb1 0 2024-05-27T09:29:26.561 INFO:tasks.workunit.client.0.smithi119.stdout:1/658: mknod d7/ccf 0 2024-05-27T09:29:26.562 INFO:tasks.workunit.client.1.smithi138.stdout:0/75: dwrite da/f11 [0,4194304] 0 2024-05-27T09:29:26.562 INFO:tasks.workunit.client.1.smithi138.stdout:0/76: dread - da/f22 zero size 2024-05-27T09:29:26.563 INFO:tasks.workunit.client.1.smithi138.stdout:0/77: fdatasync da/f22 0 2024-05-27T09:29:26.563 INFO:tasks.workunit.client.0.smithi119.stdout:5/592: write d6/dd/d39/d19/d70/dc1/d41/fad [645994,87622] 0 2024-05-27T09:29:26.563 INFO:tasks.workunit.client.0.smithi119.stdout:5/593: creat d6/dd/d39/d19/fd0 x:0 0 0 2024-05-27T09:29:26.566 INFO:tasks.workunit.client.0.smithi119.stdout:4/570: read f1 [1066448,15509] 0 2024-05-27T09:29:26.566 INFO:tasks.workunit.client.0.smithi119.stdout:8/563: dwrite d3/de/f1d [0,4194304] 0 2024-05-27T09:29:26.567 INFO:tasks.workunit.client.0.smithi119.stdout:1/659: mkdir d7/d6e/dd0 0 2024-05-27T09:29:26.572 INFO:tasks.workunit.client.1.smithi138.stdout:0/78: getdents da/d17/d1d/d1f 0 2024-05-27T09:29:26.572 INFO:tasks.workunit.client.1.smithi138.stdout:0/79: stat da/f15 0 2024-05-27T09:29:26.572 INFO:tasks.workunit.client.1.smithi138.stdout:0/80: truncate da/f15 732992 0 2024-05-27T09:29:26.574 INFO:tasks.workunit.client.0.smithi119.stdout:5/594: truncate d6/db/de/f25 6330276 0 2024-05-27T09:29:26.574 INFO:tasks.workunit.client.0.smithi119.stdout:2/587: dread d2/da/d15/d33/d89/d94/f56 [0,4194304] 0 2024-05-27T09:29:26.577 INFO:tasks.workunit.client.0.smithi119.stdout:6/560: mknod d0/d2/d57/d37/cc3 0 2024-05-27T09:29:26.578 INFO:tasks.workunit.client.0.smithi119.stdout:4/571: mkdir d4/db1 0 2024-05-27T09:29:26.581 INFO:tasks.workunit.client.1.smithi138.stdout:4/54: dwrite f3 [8388608,4194304] 0 2024-05-27T09:29:26.582 INFO:tasks.workunit.client.0.smithi119.stdout:1/660: symlink d7/db/d12/d26/d4a/ld1 0 2024-05-27T09:29:26.587 INFO:tasks.workunit.client.1.smithi138.stdout:4/55: truncate f0 2048459 0 2024-05-27T09:29:26.588 INFO:tasks.workunit.client.0.smithi119.stdout:0/543: dwrite d2/d6/d9/d24/d30/d6b/f70 [0,4194304] 0 2024-05-27T09:29:26.588 INFO:tasks.workunit.client.0.smithi119.stdout:0/544: chown d2/d6/d9/d2f/c72 1 1 2024-05-27T09:29:26.592 INFO:tasks.workunit.client.1.smithi138.stdout:4/56: link f0 d7/df/f10 0 2024-05-27T09:29:26.593 INFO:tasks.workunit.client.0.smithi119.stdout:5/595: creat d6/dd/d39/fd1 x:0 0 0 2024-05-27T09:29:26.594 INFO:tasks.workunit.client.1.smithi138.stdout:4/57: creat d7/df/f11 x:0 0 0 2024-05-27T09:29:26.594 INFO:tasks.workunit.client.1.smithi138.stdout:4/58: dread - d7/fb zero size 2024-05-27T09:29:26.594 INFO:tasks.workunit.client.1.smithi138.stdout:4/59: truncate f0 2721146 0 2024-05-27T09:29:26.594 INFO:tasks.workunit.client.1.smithi138.stdout:4/60: chown d7/df/f11 63 1 2024-05-27T09:29:26.596 INFO:tasks.workunit.client.1.smithi138.stdout:4/61: creat d7/f12 x:0 0 0 2024-05-27T09:29:26.596 INFO:tasks.workunit.client.1.smithi138.stdout:4/62: dread - d7/fc zero size 2024-05-27T09:29:26.596 INFO:tasks.workunit.client.1.smithi138.stdout:4/63: creat d7/f13 x:0 0 0 2024-05-27T09:29:26.597 INFO:tasks.workunit.client.0.smithi119.stdout:1/661: write d7/db/d12/d26/d4a/f81 [170489,23097] 0 2024-05-27T09:29:26.597 INFO:tasks.workunit.client.0.smithi119.stdout:1/662: read - d7/db/d8f/fc4 zero size 2024-05-27T09:29:26.597 INFO:tasks.workunit.client.1.smithi138.stdout:4/64: symlink d7/df/l14 0 2024-05-27T09:29:26.597 INFO:tasks.workunit.client.1.smithi138.stdout:4/65: stat f3 0 2024-05-27T09:29:26.599 INFO:tasks.workunit.client.0.smithi119.stdout:2/588: chown d2/da/d2c/lab 0 1 2024-05-27T09:29:26.601 INFO:tasks.workunit.client.0.smithi119.stdout:7/594: rename d7/df/d36 to d7/df/d19/d97/d92/db7/dba 0 2024-05-27T09:29:26.604 INFO:tasks.workunit.client.0.smithi119.stdout:6/561: unlink d0/d19/lbe 0 2024-05-27T09:29:26.607 INFO:tasks.workunit.client.0.smithi119.stdout:4/572: truncate d4/d23/d31/f33 386438 0 2024-05-27T09:29:26.611 INFO:tasks.workunit.client.0.smithi119.stdout:0/545: mkdir d2/d6/d9/dab 0 2024-05-27T09:29:26.613 INFO:tasks.workunit.client.0.smithi119.stdout:2/589: symlink d2/da/lc8 0 2024-05-27T09:29:26.614 INFO:tasks.workunit.client.0.smithi119.stdout:2/590: creat d2/da/d15/d33/d89/d94/d81/fc9 x:0 0 0 2024-05-27T09:29:26.616 INFO:tasks.workunit.client.0.smithi119.stdout:9/450: dwrite d18/d19/d1b/d40/f60 [0,4194304] 0 2024-05-27T09:29:26.616 INFO:tasks.workunit.client.0.smithi119.stdout:5/596: dread d6/dd/d46/d47/d4b/f58 [4194304,4194304] 0 2024-05-27T09:29:26.617 INFO:tasks.workunit.client.0.smithi119.stdout:5/597: fdatasync d6/db/de/d27/d77/fbc 0 2024-05-27T09:29:26.620 INFO:tasks.workunit.client.0.smithi119.stdout:3/702: rename db/d36/d45/d49/d4f to db/d36/d45/d4c/d62/d13/d33/d9b/de3 0 2024-05-27T09:29:26.620 INFO:tasks.workunit.client.0.smithi119.stdout:7/595: mkdir d7/d1a/d99/dbb 0 2024-05-27T09:29:26.621 INFO:tasks.workunit.client.0.smithi119.stdout:7/596: chown d7/d1a/fa1 3 1 2024-05-27T09:29:26.621 INFO:tasks.workunit.client.0.smithi119.stdout:7/597: read d7/df/d19/d97/f57 [1606061,31900] 0 2024-05-27T09:29:26.621 INFO:tasks.workunit.client.0.smithi119.stdout:4/573: dread d4/de/d21/d40/f6a [0,4194304] 0 2024-05-27T09:29:26.621 INFO:tasks.workunit.client.0.smithi119.stdout:4/574: readlink d4/de/la1 0 2024-05-27T09:29:26.621 INFO:tasks.workunit.client.0.smithi119.stdout:4/575: stat d4/d6/d3e/c79 0 2024-05-27T09:29:26.621 INFO:tasks.workunit.client.0.smithi119.stdout:0/546: symlink d2/d6/d9/d24/d30/lac 0 2024-05-27T09:29:26.627 INFO:tasks.workunit.client.0.smithi119.stdout:5/598: creat d6/db/dc/fd2 x:0 0 0 2024-05-27T09:29:26.634 INFO:tasks.workunit.client.0.smithi119.stdout:1/663: rename d7/db/d12/d15/d1f/db5 to d7/db/d12/d15/dd2 0 2024-05-27T09:29:26.634 INFO:tasks.workunit.client.0.smithi119.stdout:1/664: fsync d7/f1e 0 2024-05-27T09:29:26.634 INFO:tasks.workunit.client.0.smithi119.stdout:6/562: rmdir d0/d2/d57/d37/dad 39 2024-05-27T09:29:26.635 INFO:tasks.workunit.client.0.smithi119.stdout:7/598: symlink d7/df/d19/d46/d93/lbc 0 2024-05-27T09:29:26.638 INFO:tasks.workunit.client.0.smithi119.stdout:9/451: dread d18/d19/d1b/d2a/f5f [0,4194304] 0 2024-05-27T09:29:26.638 INFO:tasks.workunit.client.0.smithi119.stdout:4/576: mkdir d4/d6/d3e/d89/db2 0 2024-05-27T09:29:26.638 INFO:tasks.workunit.client.0.smithi119.stdout:4/577: truncate d4/de/d21/f2b 872975 0 2024-05-27T09:29:26.639 INFO:tasks.workunit.client.0.smithi119.stdout:0/547: truncate d2/d6/d9/d12/f77 154805 0 2024-05-27T09:29:26.639 INFO:tasks.workunit.client.0.smithi119.stdout:0/548: creat d2/d6/d9/d12/d4b/d96/fad x:0 0 0 2024-05-27T09:29:26.639 INFO:tasks.workunit.client.0.smithi119.stdout:0/549: dread - d2/f94 zero size 2024-05-27T09:29:26.639 INFO:tasks.workunit.client.0.smithi119.stdout:0/550: chown d2/d6/d9/d12/d4b/d96/f98 178979 1 2024-05-27T09:29:26.644 INFO:tasks.workunit.client.0.smithi119.stdout:0/551: unlink d2/d6/d9/d2f/c72 0 2024-05-27T09:29:26.644 INFO:tasks.workunit.client.0.smithi119.stdout:0/552: chown d2/d6/d9/d2f/c9b 27200 1 2024-05-27T09:29:26.645 INFO:tasks.workunit.client.0.smithi119.stdout:5/599: getdents d6/db/de/d27/dc9 0 2024-05-27T09:29:26.645 INFO:tasks.workunit.client.0.smithi119.stdout:5/600: chown d6/dd/d39/d19/d3c/caf 1 1 2024-05-27T09:29:26.646 INFO:tasks.workunit.client.0.smithi119.stdout:0/553: mknod d2/d6/d9/d2f/d95/cae 0 2024-05-27T09:29:26.650 INFO:tasks.workunit.client.0.smithi119.stdout:5/601: mknod d6/d2f/d9d/cd3 0 2024-05-27T09:29:26.650 INFO:tasks.workunit.client.0.smithi119.stdout:0/554: getdents d2/d6/d9/d24 0 2024-05-27T09:29:26.652 INFO:tasks.workunit.client.0.smithi119.stdout:1/665: write d7/dd/d44/f56 [1606003,7839] 0 2024-05-27T09:29:26.653 INFO:tasks.workunit.client.1.smithi138.stdout:0/81: dwrite da/f22 [0,4194304] 0 2024-05-27T09:29:26.654 INFO:tasks.workunit.client.1.smithi138.stdout:4/66: dwrite f0 [0,4194304] 0 2024-05-27T09:29:26.655 INFO:tasks.workunit.client.0.smithi119.stdout:1/666: stat d7/db/d35/l48 0 2024-05-27T09:29:26.655 INFO:tasks.workunit.client.0.smithi119.stdout:1/667: read d7/db/d12/f96 [708345,61888] 0 2024-05-27T09:29:26.655 INFO:tasks.workunit.client.0.smithi119.stdout:1/668: fsync d7/db/d12/d26/f97 0 2024-05-27T09:29:26.655 INFO:tasks.workunit.client.0.smithi119.stdout:1/669: write d7/dd/d25/d29/d38/f92 [882637,25210] 0 2024-05-27T09:29:26.656 INFO:tasks.workunit.client.0.smithi119.stdout:1/670: creat d7/dd/d25/d29/d38/d3c/fd3 x:0 0 0 2024-05-27T09:29:26.656 INFO:tasks.workunit.client.1.smithi138.stdout:0/82: symlink da/d17/d1d/l23 0 2024-05-27T09:29:26.661 INFO:tasks.workunit.client.0.smithi119.stdout:1/671: getdents d7/db/d12/d15/d1f 0 2024-05-27T09:29:26.676 INFO:tasks.workunit.client.1.smithi138.stdout:0/83: read da/f11 [357083,28183] 0 2024-05-27T09:29:26.676 INFO:tasks.workunit.client.1.smithi138.stdout:0/84: fsync da/d17/f19 0 2024-05-27T09:29:26.676 INFO:tasks.workunit.client.1.smithi138.stdout:0/85: stat f1 0 2024-05-27T09:29:26.683 INFO:tasks.workunit.client.0.smithi119.stdout:2/591: dwrite d2/f47 [0,4194304] 0 2024-05-27T09:29:26.695 INFO:tasks.workunit.client.0.smithi119.stdout:9/452: dread d18/d19/d1b/d40/f62 [0,4194304] 0 2024-05-27T09:29:26.702 INFO:tasks.workunit.client.0.smithi119.stdout:3/703: dwrite db/f56 [0,4194304] 0 2024-05-27T09:29:26.705 INFO:tasks.workunit.client.0.smithi119.stdout:0/555: dwrite d2/d6/d23/d7a/d53/f5f [0,4194304] 0 2024-05-27T09:29:26.705 INFO:tasks.workunit.client.0.smithi119.stdout:0/556: fdatasync d2/d6/d23/fa9 0 2024-05-27T09:29:26.705 INFO:tasks.workunit.client.0.smithi119.stdout:3/704: symlink db/d36/d45/d4d/le4 0 2024-05-27T09:29:26.705 INFO:tasks.workunit.client.0.smithi119.stdout:3/705: dread - db/d36/d45/dab/fcd zero size 2024-05-27T09:29:26.707 INFO:tasks.workunit.client.1.smithi138.stdout:2/48: sync 2024-05-27T09:29:26.707 INFO:tasks.workunit.client.1.smithi138.stdout:5/66: sync 2024-05-27T09:29:26.709 INFO:tasks.workunit.client.1.smithi138.stdout:1/106: sync 2024-05-27T09:29:26.710 INFO:tasks.workunit.client.0.smithi119.stdout:7/599: dwrite d7/df/d19/d8d/fa3 [0,4194304] 0 2024-05-27T09:29:26.710 INFO:tasks.workunit.client.0.smithi119.stdout:7/600: dread - d7/d1a/d40/f8b zero size 2024-05-27T09:29:26.714 INFO:tasks.workunit.client.1.smithi138.stdout:2/49: rename d1/da to d1/d13 0 2024-05-27T09:29:26.716 INFO:tasks.workunit.client.0.smithi119.stdout:3/706: mkdir db/d5b/dce/de5 0 2024-05-27T09:29:26.723 INFO:tasks.workunit.client.0.smithi119.stdout:9/453: write d18/d19/f50 [1017403,117096] 0 2024-05-27T09:29:26.728 INFO:tasks.workunit.client.1.smithi138.stdout:2/50: dread d1/f2 [0,4194304] 0 2024-05-27T09:29:26.728 INFO:tasks.workunit.client.0.smithi119.stdout:9/454: link d18/d19/d1b/d40/d7b/f89 d18/d19/d1b/f96 0 2024-05-27T09:29:26.728 INFO:tasks.workunit.client.0.smithi119.stdout:9/455: stat d18/d19/d1b/f4c 0 2024-05-27T09:29:26.729 INFO:tasks.workunit.client.1.smithi138.stdout:2/51: mknod d1/d13/c14 0 2024-05-27T09:29:26.736 INFO:tasks.workunit.client.1.smithi138.stdout:5/67: write d3/f11 [3128666,31769] 0 2024-05-27T09:29:26.739 INFO:tasks.workunit.client.0.smithi119.stdout:6/563: dwrite d0/d2/d57/d37/f86 [4194304,4194304] 0 2024-05-27T09:29:26.739 INFO:tasks.workunit.client.0.smithi119.stdout:6/564: dread - d0/d19/f79 zero size 2024-05-27T09:29:26.740 INFO:tasks.workunit.client.1.smithi138.stdout:5/68: mkdir d3/d12/d16 0 2024-05-27T09:29:26.741 INFO:tasks.workunit.client.0.smithi119.stdout:9/456: read d18/d19/d1b/d2a/f8a [2905712,129925] 0 2024-05-27T09:29:26.741 INFO:tasks.workunit.client.0.smithi119.stdout:6/565: link d0/d11/l6b d0/d19/db2/lc4 0 2024-05-27T09:29:26.742 INFO:tasks.workunit.client.0.smithi119.stdout:9/457: truncate d18/d19/f74 261447 0 2024-05-27T09:29:26.743 INFO:tasks.workunit.client.1.smithi138.stdout:0/86: dwrite da/d17/d1d/d1f/f21 [0,4194304] 0 2024-05-27T09:29:26.744 INFO:tasks.workunit.client.1.smithi138.stdout:5/69: unlink d3/ce 0 2024-05-27T09:29:26.746 INFO:tasks.workunit.client.1.smithi138.stdout:0/87: mknod da/d17/d1d/c24 0 2024-05-27T09:29:26.746 INFO:tasks.workunit.client.1.smithi138.stdout:0/88: chown da/d17/c1c 7917 1 2024-05-27T09:29:26.747 INFO:tasks.workunit.client.1.smithi138.stdout:5/70: rename d3/f13 to d3/d12/f17 0 2024-05-27T09:29:26.748 INFO:tasks.workunit.client.1.smithi138.stdout:5/71: fdatasync d3/f15 0 2024-05-27T09:29:26.750 INFO:tasks.workunit.client.1.smithi138.stdout:5/72: link f1 d3/f18 0 2024-05-27T09:29:26.750 INFO:tasks.workunit.client.1.smithi138.stdout:5/73: stat d3/f11 0 2024-05-27T09:29:26.753 INFO:tasks.workunit.client.1.smithi138.stdout:5/74: rename f1 to d3/f19 0 2024-05-27T09:29:26.767 INFO:tasks.workunit.client.0.smithi119.stdout:5/602: dwrite d6/dd/d46/fbf [0,4194304] 0 2024-05-27T09:29:26.767 INFO:tasks.workunit.client.0.smithi119.stdout:5/603: readlink d6/db/de/d27/l3d 0 2024-05-27T09:29:26.769 INFO:tasks.workunit.client.0.smithi119.stdout:0/557: dread d2/d6/d23/d7a/f44 [0,4194304] 0 2024-05-27T09:29:26.771 INFO:tasks.workunit.client.0.smithi119.stdout:0/558: creat d2/d6/d9/d2f/d95/faf x:0 0 0 2024-05-27T09:29:26.774 INFO:tasks.workunit.client.0.smithi119.stdout:7/601: read d7/d1a/d40/f86 [3010158,82908] 0 2024-05-27T09:29:26.775 INFO:tasks.workunit.client.0.smithi119.stdout:0/559: symlink d2/d6/d9/d24/lb0 0 2024-05-27T09:29:26.775 INFO:tasks.workunit.client.0.smithi119.stdout:1/672: dwrite d7/db/d35/f3e [4194304,4194304] 0 2024-05-27T09:29:26.775 INFO:tasks.workunit.client.0.smithi119.stdout:1/673: read - d7/dd/d25/d29/d38/d3c/fca zero size 2024-05-27T09:29:26.776 INFO:tasks.workunit.client.0.smithi119.stdout:5/604: dread d6/dd/d39/d19/d3c/d52/fbd [0,4194304] 0 2024-05-27T09:29:26.776 INFO:tasks.workunit.client.0.smithi119.stdout:0/560: symlink d2/d1a/lb1 0 2024-05-27T09:29:26.779 INFO:tasks.workunit.client.0.smithi119.stdout:1/674: creat d7/db/d35/fd4 x:0 0 0 2024-05-27T09:29:26.782 INFO:tasks.workunit.client.0.smithi119.stdout:0/561: symlink d2/d6/d9/d12/d4f/lb2 0 2024-05-27T09:29:26.800 INFO:tasks.workunit.client.1.smithi138.stdout:5/75: dread d3/f19 [0,4194304] 0 2024-05-27T09:29:26.800 INFO:tasks.workunit.client.1.smithi138.stdout:5/76: dread - d3/d12/f17 zero size 2024-05-27T09:29:26.800 INFO:tasks.workunit.client.1.smithi138.stdout:5/77: dread - d3/f10 zero size 2024-05-27T09:29:26.800 INFO:tasks.workunit.client.1.smithi138.stdout:5/78: stat d3/d12/d16 0 2024-05-27T09:29:26.801 INFO:tasks.workunit.client.0.smithi119.stdout:2/592: dwrite d2/da/d6f/fa5 [0,4194304] 0 2024-05-27T09:29:26.805 INFO:tasks.workunit.client.0.smithi119.stdout:2/593: rmdir d2/da/d8a 39 2024-05-27T09:29:26.808 INFO:tasks.workunit.client.1.smithi138.stdout:5/79: symlink d3/l1a 0 2024-05-27T09:29:26.811 INFO:tasks.workunit.client.1.smithi138.stdout:0/89: dwrite da/d17/f18 [0,4194304] 0 2024-05-27T09:29:26.826 INFO:tasks.workunit.client.0.smithi119.stdout:6/566: dwrite d0/d19/f69 [0,4194304] 0 2024-05-27T09:29:26.829 INFO:tasks.workunit.client.0.smithi119.stdout:6/567: mkdir d0/d2/d4e/dc5 0 2024-05-27T09:29:26.829 INFO:tasks.workunit.client.0.smithi119.stdout:6/568: fdatasync d0/d2/f31 0 2024-05-27T09:29:26.829 INFO:tasks.workunit.client.0.smithi119.stdout:6/569: fsync d0/d19/fa6 0 2024-05-27T09:29:26.830 INFO:tasks.workunit.client.0.smithi119.stdout:6/570: creat d0/d2/d4e/d5a/d76/fc6 x:0 0 0 2024-05-27T09:29:26.830 INFO:tasks.workunit.client.0.smithi119.stdout:9/458: dwrite d18/f32 [0,4194304] 0 2024-05-27T09:29:26.831 INFO:tasks.workunit.client.1.smithi138.stdout:0/90: write f5 [296598,129864] 0 2024-05-27T09:29:26.831 INFO:tasks.workunit.client.0.smithi119.stdout:2/594: write d2/d1a/f1f [90118,49475] 0 2024-05-27T09:29:26.833 INFO:tasks.workunit.client.1.smithi138.stdout:0/91: symlink da/d17/d1d/d1f/l25 0 2024-05-27T09:29:26.834 INFO:tasks.workunit.client.0.smithi119.stdout:6/571: creat d0/d2/d57/d37/d74/d65/fc7 x:0 0 0 2024-05-27T09:29:26.834 INFO:tasks.workunit.client.0.smithi119.stdout:6/572: creat d0/d2/d57/d37/d74/d65/fc8 x:0 0 0 2024-05-27T09:29:26.836 INFO:tasks.workunit.client.1.smithi138.stdout:0/92: creat da/d17/d1d/d1f/f26 x:0 0 0 2024-05-27T09:29:26.837 INFO:tasks.workunit.client.0.smithi119.stdout:2/595: rename d2/da/d15/d53/f85 to d2/da/d15/d33/d89/db0/fca 0 2024-05-27T09:29:26.841 INFO:tasks.workunit.client.0.smithi119.stdout:3/707: dwrite db/d36/d45/d4c/d62/d13/d33/d9b/fca [0,4194304] 0 2024-05-27T09:29:26.841 INFO:tasks.workunit.client.0.smithi119.stdout:9/459: rmdir d18/d19 39 2024-05-27T09:29:26.842 INFO:tasks.workunit.client.0.smithi119.stdout:9/460: readlink d18/d19/d1b/d29/l86 0 2024-05-27T09:29:26.843 INFO:tasks.workunit.client.0.smithi119.stdout:1/675: dwrite d7/dd/d25/fc5 [0,4194304] 0 2024-05-27T09:29:26.846 INFO:tasks.workunit.client.0.smithi119.stdout:6/573: symlink d0/d2/d4e/d5a/lc9 0 2024-05-27T09:29:26.846 INFO:tasks.workunit.client.0.smithi119.stdout:6/574: write d0/d2/d4e/fc0 [212909,741] 0 2024-05-27T09:29:26.852 INFO:tasks.workunit.client.0.smithi119.stdout:6/575: creat d0/d2/fca x:0 0 0 2024-05-27T09:29:26.852 INFO:tasks.workunit.client.0.smithi119.stdout:6/576: stat d0/d11/f62 0 2024-05-27T09:29:26.852 INFO:tasks.workunit.client.0.smithi119.stdout:2/596: write d2/fa0 [269893,76674] 0 2024-05-27T09:29:26.860 INFO:tasks.workunit.client.1.smithi138.stdout:5/80: dwrite d3/f7 [0,4194304] 0 2024-05-27T09:29:26.862 INFO:tasks.workunit.client.0.smithi119.stdout:5/605: dwrite d6/dd/d39/d19/fb5 [0,4194304] 0 2024-05-27T09:29:26.863 INFO:tasks.workunit.client.0.smithi119.stdout:5/606: write d6/db/de/d27/d94/fa9 [686854,119790] 0 2024-05-27T09:29:26.863 INFO:tasks.workunit.client.0.smithi119.stdout:5/607: readlink d6/db/de/d27/d94/lbb 0 2024-05-27T09:29:26.864 INFO:tasks.workunit.client.0.smithi119.stdout:3/708: dread db/d36/d45/d4c/d62/d13/f21 [0,4194304] 0 2024-05-27T09:29:26.878 INFO:tasks.workunit.client.0.smithi119.stdout:7/602: dwrite d7/f62 [0,4194304] 0 2024-05-27T09:29:26.885 INFO:tasks.workunit.client.0.smithi119.stdout:3/709: write db/fb1 [3151011,27277] 0 2024-05-27T09:29:26.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:26 smithi138 ceph-mon[34885]: mgrmap e25: smithi138.dxsuqf(active, since 15s), standbys: smithi119.ncwmog 2024-05-27T09:29:26.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:26 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mgr metadata", "who": "smithi119.ncwmog", "id": "smithi119.ncwmog"}]: dispatch 2024-05-27T09:29:26.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:26 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:29:26.890 INFO:tasks.workunit.client.0.smithi119.stdout:7/603: dread d7/df/d19/f14 [0,4194304] 0 2024-05-27T09:29:26.903 INFO:tasks.workunit.client.0.smithi119.stdout:1/676: dwrite d7/f5b [4194304,4194304] 0 2024-05-27T09:29:26.903 INFO:tasks.workunit.client.0.smithi119.stdout:9/461: creat d18/d19/d1b/d2a/d63/f97 x:0 0 0 2024-05-27T09:29:26.905 INFO:tasks.workunit.client.0.smithi119.stdout:6/577: dwrite d0/d2/d57/d37/f86 [0,4194304] 0 2024-05-27T09:29:26.905 INFO:tasks.workunit.client.0.smithi119.stdout:6/578: chown d0/d2/d4e/fc0 2858332 1 2024-05-27T09:29:26.905 INFO:tasks.workunit.client.0.smithi119.stdout:6/579: write d0/d2/d4e/d5a/f97 [1578338,70713] 0 2024-05-27T09:29:26.905 INFO:tasks.workunit.client.0.smithi119.stdout:6/580: chown d0/d2/f30 0 1 2024-05-27T09:29:26.905 INFO:tasks.workunit.client.0.smithi119.stdout:6/581: read d0/d2/d1c/fa0 [199772,112259] 0 2024-05-27T09:29:26.905 INFO:tasks.workunit.client.0.smithi119.stdout:6/582: write d0/f84 [1652187,30291] 0 2024-05-27T09:29:26.905 INFO:tasks.workunit.client.0.smithi119.stdout:6/583: read d0/d2/d1c/f4c [3082336,52077] 0 2024-05-27T09:29:26.905 INFO:tasks.workunit.client.0.smithi119.stdout:6/584: truncate d0/f18 1528836 0 2024-05-27T09:29:26.912 INFO:tasks.workunit.client.0.smithi119.stdout:5/608: mkdir d6/db/dd4 0 2024-05-27T09:29:26.915 INFO:tasks.workunit.client.0.smithi119.stdout:3/710: symlink db/d36/d45/d49/le6 0 2024-05-27T09:29:26.916 INFO:tasks.workunit.client.0.smithi119.stdout:4/578: sync 2024-05-27T09:29:26.916 INFO:tasks.workunit.client.0.smithi119.stdout:7/604: creat d7/df/d19/d9a/da6/fbd x:0 0 0 2024-05-27T09:29:26.916 INFO:tasks.workunit.client.0.smithi119.stdout:8/564: sync 2024-05-27T09:29:26.917 INFO:tasks.workunit.client.0.smithi119.stdout:1/677: rmdir d7/dd/d44/d94 0 2024-05-27T09:29:26.918 INFO:tasks.workunit.client.0.smithi119.stdout:1/678: chown d7/dd/d25/d60/d61/c66 210172 1 2024-05-27T09:29:26.920 INFO:tasks.workunit.client.0.smithi119.stdout:2/597: rename d2/da/d2c/c49 to d2/da/d15/d33/ccb 0 2024-05-27T09:29:26.923 INFO:tasks.workunit.client.0.smithi119.stdout:5/609: getdents d6/dd/d39/d19/d70/dc1/d41 0 2024-05-27T09:29:26.923 INFO:tasks.workunit.client.0.smithi119.stdout:9/462: dread d18/d19/d1b/d40/d44/f4a [0,4194304] 0 2024-05-27T09:29:26.923 INFO:tasks.workunit.client.0.smithi119.stdout:9/463: read d18/d19/d1b/d40/d7b/f89 [69427,58464] 0 2024-05-27T09:29:26.927 INFO:tasks.workunit.client.0.smithi119.stdout:8/565: dread d3/f8c [0,4194304] 0 2024-05-27T09:29:26.927 INFO:tasks.workunit.client.0.smithi119.stdout:3/711: rmdir db/dc9/daa/dd9 0 2024-05-27T09:29:26.928 INFO:tasks.workunit.client.0.smithi119.stdout:4/579: creat d4/d23/d24/d5b/da8/fb3 x:0 0 0 2024-05-27T09:29:26.928 INFO:tasks.workunit.client.0.smithi119.stdout:4/580: readlink d4/d4e/l59 0 2024-05-27T09:29:26.928 INFO:tasks.workunit.client.0.smithi119.stdout:4/581: write d4/d6/f74 [1193447,25011] 0 2024-05-27T09:29:26.928 INFO:tasks.workunit.client.0.smithi119.stdout:4/582: read d4/d1c/f1e [2622817,115812] 0 2024-05-27T09:29:26.929 INFO:tasks.workunit.client.0.smithi119.stdout:7/605: symlink d7/df/d19/d1d/lbe 0 2024-05-27T09:29:26.930 INFO:tasks.workunit.client.0.smithi119.stdout:1/679: link d7/dd/d25/d29/d38/d49/c4b d7/d40/cd5 0 2024-05-27T09:29:26.931 INFO:tasks.workunit.client.0.smithi119.stdout:6/585: creat d0/d2/d57/d37/dad/db6/fcb x:0 0 0 2024-05-27T09:29:26.933 INFO:tasks.workunit.client.0.smithi119.stdout:5/610: unlink d6/dd/d46/d47/l83 0 2024-05-27T09:29:26.934 INFO:tasks.workunit.client.0.smithi119.stdout:5/611: read d6/db/de/f5b [195645,77467] 0 2024-05-27T09:29:26.936 INFO:tasks.workunit.client.0.smithi119.stdout:9/464: mkdir d18/d19/d61/d98 0 2024-05-27T09:29:26.937 INFO:tasks.workunit.client.0.smithi119.stdout:9/465: chown d18/d19/d1b/f5d 1239646798 1 2024-05-27T09:29:26.937 INFO:tasks.workunit.client.0.smithi119.stdout:8/566: mkdir d3/d8/d4d/d5d/db7 0 2024-05-27T09:29:26.937 INFO:tasks.workunit.client.0.smithi119.stdout:2/598: dread d2/da/d15/d33/f7e [0,4194304] 0 2024-05-27T09:29:26.938 INFO:tasks.workunit.client.0.smithi119.stdout:7/606: truncate d7/df/d19/d97/d92/db7/dba/f3f 1647688 0 2024-05-27T09:29:26.939 INFO:tasks.workunit.client.0.smithi119.stdout:6/586: unlink d0/f5f 0 2024-05-27T09:29:26.941 INFO:tasks.workunit.client.0.smithi119.stdout:5/612: mkdir d6/dd/d39/d19/d3c/dd5 0 2024-05-27T09:29:26.945 INFO:tasks.workunit.client.0.smithi119.stdout:6/587: rename d0/d2/d1c/d21/d58/fbd to d0/d2/d1c/d21/d58/fcc 0 2024-05-27T09:29:26.945 INFO:tasks.workunit.client.0.smithi119.stdout:6/588: dread - d0/d19/f79 zero size 2024-05-27T09:29:26.947 INFO:tasks.workunit.client.0.smithi119.stdout:5/613: creat d6/db/dd4/fd6 x:0 0 0 2024-05-27T09:29:26.948 INFO:tasks.workunit.client.1.smithi138.stdout:3/39: sync 2024-05-27T09:29:26.948 INFO:tasks.workunit.client.1.smithi138.stdout:6/78: sync 2024-05-27T09:29:26.948 INFO:tasks.workunit.client.1.smithi138.stdout:7/44: sync 2024-05-27T09:29:26.948 INFO:tasks.workunit.client.1.smithi138.stdout:3/40: dread - d0/d2/fc zero size 2024-05-27T09:29:26.948 INFO:tasks.workunit.client.1.smithi138.stdout:9/38: sync 2024-05-27T09:29:26.950 INFO:tasks.workunit.client.1.smithi138.stdout:6/79: creat d3/d19/f1e x:0 0 0 2024-05-27T09:29:26.951 INFO:tasks.workunit.client.1.smithi138.stdout:9/39: truncate f3 3136917 0 2024-05-27T09:29:26.951 INFO:tasks.workunit.client.1.smithi138.stdout:3/41: getdents d0/d2/da 0 2024-05-27T09:29:26.952 INFO:tasks.workunit.client.1.smithi138.stdout:3/42: dread - d0/d2/fc zero size 2024-05-27T09:29:26.952 INFO:tasks.workunit.client.0.smithi119.stdout:6/589: link d0/d2/d57/d37/d74/d65/fc8 d0/d2/d57/d37/dad/db6/fcd 0 2024-05-27T09:29:26.953 INFO:tasks.workunit.client.0.smithi119.stdout:6/590: symlink d0/d2/d4e/d5a/d76/lce 0 2024-05-27T09:29:26.954 INFO:tasks.workunit.client.1.smithi138.stdout:6/80: rename d3/d19/c1a to d3/d19/c1f 0 2024-05-27T09:29:26.955 INFO:tasks.workunit.client.1.smithi138.stdout:3/43: creat d0/fd x:0 0 0 2024-05-27T09:29:26.955 INFO:tasks.workunit.client.0.smithi119.stdout:6/591: readlink d0/d2/d57/d37/l38 0 2024-05-27T09:29:26.957 INFO:tasks.workunit.client.1.smithi138.stdout:6/81: symlink d3/d19/l20 0 2024-05-27T09:29:26.958 INFO:tasks.workunit.client.1.smithi138.stdout:3/44: symlink d0/d8/le 0 2024-05-27T09:29:26.958 INFO:tasks.workunit.client.1.smithi138.stdout:3/45: stat d0/d2/fc 0 2024-05-27T09:29:26.960 INFO:tasks.workunit.client.0.smithi119.stdout:7/607: write d7/df/d19/f1f [2514670,58191] 0 2024-05-27T09:29:26.961 INFO:tasks.workunit.client.0.smithi119.stdout:7/608: write d7/df/d19/d1d/f87 [1299275,43633] 0 2024-05-27T09:29:26.961 INFO:tasks.workunit.client.0.smithi119.stdout:7/609: dread - d7/df/d19/d97/d92/db7/dba/fb3 zero size 2024-05-27T09:29:26.961 INFO:tasks.workunit.client.0.smithi119.stdout:7/610: chown d7/df/d19/d97/d37 240707355 1 2024-05-27T09:29:26.976 INFO:tasks.workunit.client.1.smithi138.stdout:9/40: read f3 [2862801,89251] 0 2024-05-27T09:29:26.982 INFO:tasks.workunit.client.1.smithi138.stdout:9/41: mkdir d6/db/de 0 2024-05-27T09:29:26.985 INFO:tasks.workunit.client.1.smithi138.stdout:6/82: dread d3/f6 [0,4194304] 0 2024-05-27T09:29:26.985 INFO:tasks.workunit.client.1.smithi138.stdout:6/83: fdatasync d3/d19/f1e 0 2024-05-27T09:29:26.995 INFO:tasks.workunit.client.1.smithi138.stdout:9/42: write f1 [729716,46791] 0 2024-05-27T09:29:26.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:26 smithi119 ceph-mon[29082]: mgrmap e25: smithi138.dxsuqf(active, since 15s), standbys: smithi119.ncwmog 2024-05-27T09:29:26.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:26 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mgr metadata", "who": "smithi119.ncwmog", "id": "smithi119.ncwmog"}]: dispatch 2024-05-27T09:29:26.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:26 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:29:27.001 INFO:tasks.workunit.client.0.smithi119.stdout:8/567: write d3/de/f1d [1989885,4734] 0 2024-05-27T09:29:27.001 INFO:tasks.workunit.client.0.smithi119.stdout:8/568: chown d3/de/c78 0 1 2024-05-27T09:29:27.012 INFO:tasks.workunit.client.0.smithi119.stdout:9/466: write d18/f23 [1008652,12577] 0 2024-05-27T09:29:27.012 INFO:tasks.workunit.client.0.smithi119.stdout:9/467: chown d18/d19/d1b 73 1 2024-05-27T09:29:27.013 INFO:tasks.workunit.client.0.smithi119.stdout:3/712: dwrite db/d5b/f82 [0,4194304] 0 2024-05-27T09:29:27.013 INFO:tasks.workunit.client.0.smithi119.stdout:3/713: dread - db/d36/d45/fcb zero size 2024-05-27T09:29:27.028 INFO:tasks.workunit.client.0.smithi119.stdout:2/599: dwrite d2/da/f4d [0,4194304] 0 2024-05-27T09:29:27.031 INFO:tasks.workunit.client.1.smithi138.stdout:6/84: dread f2 [0,4194304] 0 2024-05-27T09:29:27.036 INFO:tasks.workunit.client.1.smithi138.stdout:6/85: symlink d3/d11/l21 0 2024-05-27T09:29:27.036 INFO:tasks.workunit.client.1.smithi138.stdout:6/86: readlink d3/d11/l21 0 2024-05-27T09:29:27.036 INFO:tasks.workunit.client.1.smithi138.stdout:6/87: dread - d3/d14/f1b zero size 2024-05-27T09:29:27.045 INFO:tasks.workunit.client.1.smithi138.stdout:9/43: dread f4 [0,4194304] 0 2024-05-27T09:29:27.053 INFO:tasks.workunit.client.1.smithi138.stdout:3/46: dwrite d0/d2/fc [0,4194304] 0 2024-05-27T09:29:27.053 INFO:tasks.workunit.client.1.smithi138.stdout:6/88: dread d3/f17 [0,4194304] 0 2024-05-27T09:29:27.053 INFO:tasks.workunit.client.1.smithi138.stdout:6/89: chown d3/d11/l1d 1680851274 1 2024-05-27T09:29:27.053 INFO:tasks.workunit.client.0.smithi119.stdout:4/583: dwrite d4/d6/d3e/f6e [0,4194304] 0 2024-05-27T09:29:27.053 INFO:tasks.workunit.client.0.smithi119.stdout:4/584: chown d4/de/d21/c38 507 1 2024-05-27T09:29:27.053 INFO:tasks.workunit.client.0.smithi119.stdout:4/585: readlink d4/d1c/d39/l46 0 2024-05-27T09:29:27.054 INFO:tasks.workunit.client.0.smithi119.stdout:4/586: write d4/d1c/d39/f57 [1272113,7103] 0 2024-05-27T09:29:27.054 INFO:tasks.workunit.client.0.smithi119.stdout:6/592: dwrite d0/d85/f8a [0,4194304] 0 2024-05-27T09:29:27.055 INFO:tasks.workunit.client.1.smithi138.stdout:3/47: mkdir d0/d2/da/df 0 2024-05-27T09:29:27.056 INFO:tasks.workunit.client.0.smithi119.stdout:5/614: dwrite d6/f80 [0,4194304] 0 2024-05-27T09:29:27.056 INFO:tasks.workunit.client.0.smithi119.stdout:5/615: fdatasync d6/dd/d39/d19/fb6 0 2024-05-27T09:29:27.056 INFO:tasks.workunit.client.0.smithi119.stdout:5/616: write d6/db/de/d27/d77/f9f [160867,47614] 0 2024-05-27T09:29:27.056 INFO:tasks.workunit.client.0.smithi119.stdout:5/617: write d6/dd/f67 [931542,27607] 0 2024-05-27T09:29:27.057 INFO:tasks.workunit.client.1.smithi138.stdout:6/90: link d3/f5 d3/d19/f22 0 2024-05-27T09:29:27.057 INFO:tasks.workunit.client.1.smithi138.stdout:6/91: fsync d3/fa 0 2024-05-27T09:29:27.057 INFO:tasks.workunit.client.1.smithi138.stdout:6/92: write d3/d14/f1b [998330,126231] 0 2024-05-27T09:29:27.058 INFO:tasks.workunit.client.1.smithi138.stdout:3/48: mkdir d0/d2/da/df/d10 0 2024-05-27T09:29:27.058 INFO:tasks.workunit.client.1.smithi138.stdout:3/49: chown d0/d2/da/cb 1006421553 1 2024-05-27T09:29:27.058 INFO:tasks.workunit.client.1.smithi138.stdout:3/50: fsync d0/d8/f9 0 2024-05-27T09:29:27.059 INFO:tasks.workunit.client.1.smithi138.stdout:3/51: dread - d0/fd zero size 2024-05-27T09:29:27.060 INFO:tasks.workunit.client.1.smithi138.stdout:6/93: link d3/c7 d3/d14/d1c/c23 0 2024-05-27T09:29:27.060 INFO:tasks.workunit.client.1.smithi138.stdout:6/94: fdatasync d3/f5 0 2024-05-27T09:29:27.061 INFO:tasks.workunit.client.1.smithi138.stdout:6/95: write d3/d11/fe [834970,21222] 0 2024-05-27T09:29:27.062 INFO:tasks.workunit.client.1.smithi138.stdout:3/52: creat d0/d2/da/f11 x:0 0 0 2024-05-27T09:29:27.062 INFO:tasks.workunit.client.1.smithi138.stdout:3/53: write d0/d2/da/f11 [163307,62554] 0 2024-05-27T09:29:27.063 INFO:tasks.workunit.client.1.smithi138.stdout:6/96: symlink d3/d14/d1c/l24 0 2024-05-27T09:29:27.064 INFO:tasks.workunit.client.1.smithi138.stdout:6/97: symlink d3/l25 0 2024-05-27T09:29:27.065 INFO:tasks.workunit.client.1.smithi138.stdout:6/98: readlink d3/d11/l1d 0 2024-05-27T09:29:27.065 INFO:tasks.workunit.client.1.smithi138.stdout:6/99: write d3/f6 [4698080,70698] 0 2024-05-27T09:29:27.065 INFO:tasks.workunit.client.1.smithi138.stdout:6/100: read d3/f5 [1474720,38547] 0 2024-05-27T09:29:27.065 INFO:tasks.workunit.client.0.smithi119.stdout:8/569: mknod d3/d39/cb8 0 2024-05-27T09:29:27.081 INFO:tasks.workunit.client.0.smithi119.stdout:3/714: unlink db/d36/d45/d49/f85 0 2024-05-27T09:29:27.081 INFO:tasks.workunit.client.0.smithi119.stdout:3/715: chown db/dc9/d98/f80 25059614 1 2024-05-27T09:29:27.081 INFO:tasks.workunit.client.0.smithi119.stdout:3/716: write db/d36/d45/dab/fcd [1003474,107149] 0 2024-05-27T09:29:27.081 INFO:tasks.workunit.client.0.smithi119.stdout:3/717: chown db/l11 20369839 1 2024-05-27T09:29:27.082 INFO:tasks.workunit.client.0.smithi119.stdout:2/600: creat d2/da/d15/d33/d89/d94/fcc x:0 0 0 2024-05-27T09:29:27.092 INFO:tasks.workunit.client.0.smithi119.stdout:7/611: rmdir d7/df/d19/d97/d92/db7/dba/d42/d64 39 2024-05-27T09:29:27.093 INFO:tasks.workunit.client.0.smithi119.stdout:7/612: write d7/df/d19/d97/f57 [3777140,109431] 0 2024-05-27T09:29:27.093 INFO:tasks.workunit.client.0.smithi119.stdout:7/613: stat d7/df/d19/d4d 0 2024-05-27T09:29:27.098 INFO:tasks.workunit.client.0.smithi119.stdout:2/601: dread d2/da/f2e [0,4194304] 0 2024-05-27T09:29:27.098 INFO:tasks.workunit.client.0.smithi119.stdout:2/602: write d2/da/d15/d53/fb6 [62567,89210] 0 2024-05-27T09:29:27.108 INFO:tasks.workunit.client.1.smithi138.stdout:8/55: sync 2024-05-27T09:29:27.108 INFO:tasks.workunit.client.1.smithi138.stdout:4/67: sync 2024-05-27T09:29:27.108 INFO:tasks.workunit.client.1.smithi138.stdout:4/68: chown f0 70 1 2024-05-27T09:29:27.108 INFO:tasks.workunit.client.1.smithi138.stdout:4/69: readlink l6 0 2024-05-27T09:29:27.112 INFO:tasks.workunit.client.1.smithi138.stdout:8/56: truncate d3/ff 2933122 0 2024-05-27T09:29:27.112 INFO:tasks.workunit.client.1.smithi138.stdout:8/57: chown d3/d8/fc 192 1 2024-05-27T09:29:27.112 INFO:tasks.workunit.client.1.smithi138.stdout:4/70: mknod d7/df/c15 0 2024-05-27T09:29:27.113 INFO:tasks.workunit.client.1.smithi138.stdout:4/71: write d7/fb [62218,14047] 0 2024-05-27T09:29:27.123 INFO:tasks.workunit.client.1.smithi138.stdout:6/101: read d3/fa [1456368,40426] 0 2024-05-27T09:29:27.127 INFO:tasks.workunit.client.0.smithi119.stdout:3/718: unlink db/d7f/da9/lb4 0 2024-05-27T09:29:27.127 INFO:tasks.workunit.client.0.smithi119.stdout:3/719: creat db/d36/d45/fe7 x:0 0 0 2024-05-27T09:29:27.128 INFO:tasks.workunit.client.0.smithi119.stdout:7/614: creat d7/df/d19/d97/fbf x:0 0 0 2024-05-27T09:29:27.132 INFO:tasks.workunit.client.0.smithi119.stdout:2/603: link d2/da/l20 d2/da/d15/d33/d89/db0/lcd 0 2024-05-27T09:29:27.134 INFO:tasks.workunit.client.0.smithi119.stdout:7/615: mknod d7/d1a/d99/cc0 0 2024-05-27T09:29:27.134 INFO:tasks.workunit.client.1.smithi138.stdout:3/54: fdatasync d0/d2/da/f11 0 2024-05-27T09:29:27.135 INFO:tasks.workunit.client.0.smithi119.stdout:2/604: creat d2/da/d15/d33/d69/db1/fce x:0 0 0 2024-05-27T09:29:27.136 INFO:tasks.workunit.client.0.smithi119.stdout:7/616: symlink d7/df/d19/d78/lc1 0 2024-05-27T09:29:27.136 INFO:tasks.workunit.client.0.smithi119.stdout:7/617: fdatasync d7/fd 0 2024-05-27T09:29:27.137 INFO:tasks.workunit.client.0.smithi119.stdout:2/605: stat d2/da/d15/d33/lb2 0 2024-05-27T09:29:27.143 INFO:tasks.workunit.client.0.smithi119.stdout:2/606: mkdir d2/da/d15/d33/d89/d94/d87/dcf 0 2024-05-27T09:29:27.144 INFO:tasks.workunit.client.0.smithi119.stdout:2/607: symlink d2/da/d15/d53/dac/ld0 0 2024-05-27T09:29:27.145 INFO:tasks.workunit.client.0.smithi119.stdout:2/608: rmdir d2/da/d15/d33 39 2024-05-27T09:29:27.155 INFO:tasks.workunit.client.1.smithi138.stdout:6/102: write f2 [1039427,88086] 0 2024-05-27T09:29:27.155 INFO:tasks.workunit.client.1.smithi138.stdout:6/103: chown d3/d14/d1c/c23 0 1 2024-05-27T09:29:27.156 INFO:tasks.workunit.client.1.smithi138.stdout:6/104: getdents d3/d11 0 2024-05-27T09:29:27.159 INFO:tasks.workunit.client.1.smithi138.stdout:6/105: mknod d3/d19/c26 0 2024-05-27T09:29:27.159 INFO:tasks.workunit.client.1.smithi138.stdout:6/106: fsync d3/d19/f22 0 2024-05-27T09:29:27.159 INFO:tasks.workunit.client.1.smithi138.stdout:6/107: chown d3/d14/f1b 0 1 2024-05-27T09:29:27.162 INFO:tasks.workunit.client.1.smithi138.stdout:6/108: mkdir d3/d11/d27 0 2024-05-27T09:29:27.166 INFO:tasks.workunit.client.1.smithi138.stdout:6/109: rmdir d3/d11/d27 0 2024-05-27T09:29:27.166 INFO:tasks.workunit.client.1.smithi138.stdout:6/110: read f2 [142205,32069] 0 2024-05-27T09:29:27.174 INFO:tasks.workunit.client.0.smithi119.stdout:3/720: creat db/d36/d45/d4c/d62/fe8 x:0 0 0 2024-05-27T09:29:27.193 INFO:tasks.workunit.client.0.smithi119.stdout:4/587: dwrite d4/d6/d28/d2f/d36/f72 [0,4194304] 0 2024-05-27T09:29:27.194 INFO:tasks.workunit.client.0.smithi119.stdout:9/468: rename d18/d19/f88 to d18/d19/d61/d95/f99 0 2024-05-27T09:29:27.194 INFO:tasks.workunit.client.0.smithi119.stdout:9/469: fdatasync d18/d19/d1b/d40/d44/f4a 0 2024-05-27T09:29:27.198 INFO:tasks.workunit.client.0.smithi119.stdout:3/721: dread db/d36/d45/d4c/d62/d13/f2e [4194304,4194304] 0 2024-05-27T09:29:27.198 INFO:tasks.workunit.client.0.smithi119.stdout:3/722: fdatasync db/d36/d45/d4c/d62/d13/f2c 0 2024-05-27T09:29:27.200 INFO:tasks.workunit.client.0.smithi119.stdout:4/588: mknod d4/d6/d28/d2f/d36/cb4 0 2024-05-27T09:29:27.202 INFO:tasks.workunit.client.0.smithi119.stdout:3/723: creat db/dc9/d98/d7a/fe9 x:0 0 0 2024-05-27T09:29:27.212 INFO:tasks.workunit.client.0.smithi119.stdout:3/724: unlink db/d36/d45/d4c/d62/d13/l8d 0 2024-05-27T09:29:27.215 INFO:tasks.workunit.client.0.smithi119.stdout:3/725: write db/f23 [492410,86155] 0 2024-05-27T09:29:27.217 INFO:tasks.workunit.client.0.smithi119.stdout:3/726: creat db/d36/d45/dab/fea x:0 0 0 2024-05-27T09:29:27.217 INFO:tasks.workunit.client.0.smithi119.stdout:9/470: rename d18/d19/f35 to d18/f9a 0 2024-05-27T09:29:27.217 INFO:tasks.workunit.client.0.smithi119.stdout:9/471: chown d18/d19/d1b/c3c 482940 1 2024-05-27T09:29:27.217 INFO:tasks.workunit.client.0.smithi119.stdout:9/472: stat d18/d19/d1b/d40/d44/f90 0 2024-05-27T09:29:27.219 INFO:tasks.workunit.client.0.smithi119.stdout:3/727: write db/dc9/d98/fa5 [1086884,50711] 0 2024-05-27T09:29:27.219 INFO:tasks.workunit.client.0.smithi119.stdout:5/618: rename d6/db/de/f18 to d6/dd/d39/d19/d3c/d52/d9c/fd7 0 2024-05-27T09:29:27.219 INFO:tasks.workunit.client.0.smithi119.stdout:9/473: stat d18/f1a 0 2024-05-27T09:29:27.219 INFO:tasks.workunit.client.0.smithi119.stdout:9/474: stat d18/d19/d1b/d40/l55 0 2024-05-27T09:29:27.221 INFO:tasks.workunit.client.0.smithi119.stdout:3/728: unlink db/d36/d45/dab/fd8 0 2024-05-27T09:29:27.221 INFO:tasks.workunit.client.0.smithi119.stdout:5/619: mkdir d6/dd/d46/d47/d4b/dd8 0 2024-05-27T09:29:27.227 INFO:tasks.workunit.client.0.smithi119.stdout:5/620: read d6/db/de/d27/d77/fb9 [332657,27469] 0 2024-05-27T09:29:27.229 INFO:tasks.workunit.client.0.smithi119.stdout:7/618: dwrite d7/d5f/f7c [0,4194304] 0 2024-05-27T09:29:27.231 INFO:tasks.workunit.client.0.smithi119.stdout:4/589: read d4/d6/d28/f7a [3822534,22892] 0 2024-05-27T09:29:27.232 INFO:tasks.workunit.client.0.smithi119.stdout:7/619: mkdir d7/d1a/d99/dbb/dc2 0 2024-05-27T09:29:27.233 INFO:tasks.workunit.client.0.smithi119.stdout:6/593: dwrite d0/d2/d57/d37/f5d [0,4194304] 0 2024-05-27T09:29:27.233 INFO:tasks.workunit.client.0.smithi119.stdout:6/594: write d0/d2/d57/d37/d74/d65/fc7 [632363,92932] 0 2024-05-27T09:29:27.233 INFO:tasks.workunit.client.0.smithi119.stdout:2/609: dwrite d2/d1a/f46 [0,4194304] 0 2024-05-27T09:29:27.233 INFO:tasks.workunit.client.0.smithi119.stdout:2/610: truncate d2/da/d15/d33/d69/f6b 105445 0 2024-05-27T09:29:27.233 INFO:tasks.workunit.client.0.smithi119.stdout:2/611: creat d2/da/d15/d33/d69/db1/fd1 x:0 0 0 2024-05-27T09:29:27.234 INFO:tasks.workunit.client.0.smithi119.stdout:5/621: dread d6/dd/d39/d19/d70/dc1/d41/fb0 [0,4194304] 0 2024-05-27T09:29:27.234 INFO:tasks.workunit.client.0.smithi119.stdout:5/622: truncate d6/d2f/f3a 1251815 0 2024-05-27T09:29:27.237 INFO:tasks.workunit.client.0.smithi119.stdout:9/475: read d18/d19/d1b/f5d [1915190,93563] 0 2024-05-27T09:29:27.237 INFO:tasks.workunit.client.0.smithi119.stdout:4/590: creat d4/de/d21/d40/d47/da3/fb5 x:0 0 0 2024-05-27T09:29:27.241 INFO:tasks.workunit.client.0.smithi119.stdout:7/620: link d7/df/d19/d97/d92/db7/dba/d42/c65 d7/d1a/d99/cc3 0 2024-05-27T09:29:27.248 INFO:tasks.workunit.client.1.smithi138.stdout:8/58: dwrite d3/ff [0,4194304] 0 2024-05-27T09:29:27.248 INFO:tasks.workunit.client.1.smithi138.stdout:8/59: readlink d3/l6 0 2024-05-27T09:29:27.250 INFO:tasks.workunit.client.0.smithi119.stdout:4/591: symlink d4/d1c/lb6 0 2024-05-27T09:29:27.250 INFO:tasks.workunit.client.0.smithi119.stdout:4/592: dread - d4/de/d66/d87/fa2 zero size 2024-05-27T09:29:27.250 INFO:tasks.workunit.client.0.smithi119.stdout:4/593: read d4/d1c/f29 [3869954,53036] 0 2024-05-27T09:29:27.251 INFO:tasks.workunit.client.1.smithi138.stdout:8/60: mkdir d3/d8/d11 0 2024-05-27T09:29:27.252 INFO:tasks.workunit.client.0.smithi119.stdout:6/595: getdents d0/d19 0 2024-05-27T09:29:27.254 INFO:tasks.workunit.client.0.smithi119.stdout:7/621: mkdir d7/df/d19/d4d/d89/dc4 0 2024-05-27T09:29:27.254 INFO:tasks.workunit.client.0.smithi119.stdout:7/622: dread - d7/df/d19/d4d/d89/f9b zero size 2024-05-27T09:29:27.255 INFO:tasks.workunit.client.0.smithi119.stdout:2/612: rename d2/da/d15/d33/d69/db1/fd1 to d2/da/d15/d33/fd2 0 2024-05-27T09:29:27.257 INFO:tasks.workunit.client.0.smithi119.stdout:9/476: mkdir d18/d19/d9b 0 2024-05-27T09:29:27.257 INFO:tasks.workunit.client.0.smithi119.stdout:9/477: creat d18/d19/d1b/d2a/d63/f9c x:0 0 0 2024-05-27T09:29:27.257 INFO:tasks.workunit.client.0.smithi119.stdout:9/478: chown d18/d19/d1b/d40/d7b/f89 289078 1 2024-05-27T09:29:27.257 INFO:tasks.workunit.client.0.smithi119.stdout:9/479: fdatasync d18/d19/d1b/d29/f2f 0 2024-05-27T09:29:27.259 INFO:tasks.workunit.client.0.smithi119.stdout:7/623: mkdir d7/df/d19/d97/d92/db7/dba/d42/dc5 0 2024-05-27T09:29:27.259 INFO:tasks.workunit.client.0.smithi119.stdout:7/624: chown d7/df/d19/d97/d37/l3b 1705277 1 2024-05-27T09:29:27.260 INFO:tasks.workunit.client.0.smithi119.stdout:5/623: rename f0 to d6/dd/fd9 0 2024-05-27T09:29:27.261 INFO:tasks.workunit.client.0.smithi119.stdout:5/624: write d6/dd/d39/d19/fd0 [438422,26490] 0 2024-05-27T09:29:27.262 INFO:tasks.workunit.client.0.smithi119.stdout:2/613: link d2/d1a/f1f d2/da/d15/d33/d89/d94/fd3 0 2024-05-27T09:29:27.263 INFO:tasks.workunit.client.0.smithi119.stdout:9/480: chown d18/d19/l39 242 1 2024-05-27T09:29:27.263 INFO:tasks.workunit.client.0.smithi119.stdout:9/481: fsync d18/f9a 0 2024-05-27T09:29:27.263 INFO:tasks.workunit.client.0.smithi119.stdout:9/482: fsync d18/f9a 0 2024-05-27T09:29:27.264 INFO:tasks.workunit.client.0.smithi119.stdout:0/562: sync 2024-05-27T09:29:27.264 INFO:tasks.workunit.client.0.smithi119.stdout:1/680: sync 2024-05-27T09:29:27.264 INFO:tasks.workunit.client.0.smithi119.stdout:0/563: write d2/d6/d75/f86 [1248373,96867] 0 2024-05-27T09:29:27.264 INFO:tasks.workunit.client.0.smithi119.stdout:0/564: creat d2/d6/d75/fb3 x:0 0 0 2024-05-27T09:29:27.264 INFO:tasks.workunit.client.0.smithi119.stdout:6/596: symlink d0/d2/d57/d37/lcf 0 2024-05-27T09:29:27.265 INFO:tasks.workunit.client.1.smithi138.stdout:6/111: dwrite f2 [0,4194304] 0 2024-05-27T09:29:27.270 INFO:tasks.workunit.client.0.smithi119.stdout:4/594: rename d4/d6/fa to d4/d4e/fb7 0 2024-05-27T09:29:27.270 INFO:tasks.workunit.client.0.smithi119.stdout:4/595: stat d4/d6/d28/d2f/d36/f94 0 2024-05-27T09:29:27.271 INFO:tasks.workunit.client.0.smithi119.stdout:5/625: creat d6/dd/d39/d19/d70/dc1/d41/fda x:0 0 0 2024-05-27T09:29:27.273 INFO:tasks.workunit.client.1.smithi138.stdout:6/112: rename d3/d14/f1b to d3/d19/f28 0 2024-05-27T09:29:27.273 INFO:tasks.workunit.client.1.smithi138.stdout:6/113: fsync d3/fa 0 2024-05-27T09:29:27.274 INFO:tasks.workunit.client.0.smithi119.stdout:1/681: creat d7/db/d12/d26/d4a/fd6 x:0 0 0 2024-05-27T09:29:27.277 INFO:tasks.workunit.client.0.smithi119.stdout:7/625: read d7/df/d19/d4d/f71 [1068620,95970] 0 2024-05-27T09:29:27.277 INFO:tasks.workunit.client.0.smithi119.stdout:7/626: write d7/df/d19/d97/d92/db7/dba/fb3 [448530,17025] 0 2024-05-27T09:29:27.279 INFO:tasks.workunit.client.0.smithi119.stdout:4/596: dread d4/d23/d31/f33 [0,4194304] 0 2024-05-27T09:29:27.279 INFO:tasks.workunit.client.0.smithi119.stdout:4/597: dread - d4/de/d66/d87/fa2 zero size 2024-05-27T09:29:27.284 INFO:tasks.workunit.client.0.smithi119.stdout:9/483: write d18/d19/d1b/f1d [583545,73874] 0 2024-05-27T09:29:27.284 INFO:tasks.workunit.client.0.smithi119.stdout:5/626: getdents d6/db/dd4 0 2024-05-27T09:29:27.287 INFO:tasks.workunit.client.0.smithi119.stdout:7/627: creat d7/d5f/d79/fc6 x:0 0 0 2024-05-27T09:29:27.289 INFO:tasks.workunit.client.0.smithi119.stdout:9/484: chown d18/f41 7466 1 2024-05-27T09:29:27.289 INFO:tasks.workunit.client.0.smithi119.stdout:9/485: fsync d18/f30 0 2024-05-27T09:29:27.291 INFO:tasks.workunit.client.0.smithi119.stdout:2/614: rename d2/da/d15/d33 to d2/da/d8a/dd4 0 2024-05-27T09:29:27.292 INFO:tasks.workunit.client.0.smithi119.stdout:5/627: mkdir d6/dd/d46/d47/d4b/dd8/ddb 0 2024-05-27T09:29:27.294 INFO:tasks.workunit.client.0.smithi119.stdout:9/486: link d18/d19/f85 d18/d19/d61/f9d 0 2024-05-27T09:29:27.297 INFO:tasks.workunit.client.0.smithi119.stdout:5/628: link d6/db/de/d27/f3f d6/db/de/fdc 0 2024-05-27T09:29:27.297 INFO:tasks.workunit.client.0.smithi119.stdout:5/629: chown d6/d2f/f42 84 1 2024-05-27T09:29:27.297 INFO:tasks.workunit.client.0.smithi119.stdout:5/630: chown d6/l5a 500 1 2024-05-27T09:29:27.297 INFO:tasks.workunit.client.0.smithi119.stdout:4/598: dread d4/d23/f52 [0,4194304] 0 2024-05-27T09:29:27.300 INFO:tasks.workunit.client.0.smithi119.stdout:4/599: mkdir d4/d6/d28/d2f/db8 0 2024-05-27T09:29:27.301 INFO:tasks.workunit.client.0.smithi119.stdout:4/600: mknod d4/d6/d81/cb9 0 2024-05-27T09:29:27.308 INFO:tasks.workunit.client.0.smithi119.stdout:9/487: dread fe [0,4194304] 0 2024-05-27T09:29:27.308 INFO:tasks.workunit.client.1.smithi138.stdout:2/52: sync 2024-05-27T09:29:27.308 INFO:tasks.workunit.client.1.smithi138.stdout:1/107: sync 2024-05-27T09:29:27.310 INFO:tasks.workunit.client.1.smithi138.stdout:2/53: unlink d1/d13/c14 0 2024-05-27T09:29:27.312 INFO:tasks.workunit.client.0.smithi119.stdout:7/628: dread d7/df/d19/d97/d92/db7/dba/f49 [0,4194304] 0 2024-05-27T09:29:27.314 INFO:tasks.workunit.client.0.smithi119.stdout:9/488: getdents d18/d19/d1b/d2a/d63 0 2024-05-27T09:29:27.315 INFO:tasks.workunit.client.0.smithi119.stdout:5/631: dread d6/dd/d39/d19/d70/dc1/f75 [0,4194304] 0 2024-05-27T09:29:27.315 INFO:tasks.workunit.client.0.smithi119.stdout:5/632: chown d6/db/dc/fd2 240732 1 2024-05-27T09:29:27.317 INFO:tasks.workunit.client.0.smithi119.stdout:7/629: rename d7/df/d19/d4d/l91 to d7/df/d19/d1d/lc7 0 2024-05-27T09:29:27.317 INFO:tasks.workunit.client.0.smithi119.stdout:7/630: write d7/d6a/f8f [826347,40821] 0 2024-05-27T09:29:27.318 INFO:tasks.workunit.client.1.smithi138.stdout:2/54: dread d1/d10/db/fd [0,4194304] 0 2024-05-27T09:29:27.319 INFO:tasks.workunit.client.0.smithi119.stdout:9/489: creat d18/d19/d1b/d29/f9e x:0 0 0 2024-05-27T09:29:27.319 INFO:tasks.workunit.client.0.smithi119.stdout:9/490: write d18/d19/d1b/d40/d44/f90 [328236,83217] 0 2024-05-27T09:29:27.319 INFO:tasks.workunit.client.0.smithi119.stdout:5/633: rmdir d6/d2f/da5 39 2024-05-27T09:29:27.320 INFO:tasks.workunit.client.1.smithi138.stdout:2/55: rmdir d1/d10/db 39 2024-05-27T09:29:27.320 INFO:tasks.workunit.client.1.smithi138.stdout:2/56: fsync d1/d10/db/fd 0 2024-05-27T09:29:27.321 INFO:tasks.workunit.client.1.smithi138.stdout:2/57: mkdir d1/d15 0 2024-05-27T09:29:27.322 INFO:tasks.workunit.client.0.smithi119.stdout:7/631: creat d7/d1a/d99/fc8 x:0 0 0 2024-05-27T09:29:27.322 INFO:tasks.workunit.client.1.smithi138.stdout:8/61: dwrite d3/d8/fc [0,4194304] 0 2024-05-27T09:29:27.323 INFO:tasks.workunit.client.1.smithi138.stdout:6/114: dwrite d3/d11/fe [0,4194304] 0 2024-05-27T09:29:27.324 INFO:tasks.workunit.client.1.smithi138.stdout:6/115: creat d3/d14/f29 x:0 0 0 2024-05-27T09:29:27.324 INFO:tasks.workunit.client.1.smithi138.stdout:6/116: creat d3/d14/f2a x:0 0 0 2024-05-27T09:29:27.324 INFO:tasks.workunit.client.1.smithi138.stdout:6/117: chown d3/c12 65 1 2024-05-27T09:29:27.326 INFO:tasks.workunit.client.1.smithi138.stdout:2/58: rename d1/d10/f9 to d1/d10/f16 0 2024-05-27T09:29:27.326 INFO:tasks.workunit.client.1.smithi138.stdout:8/62: creat d3/d8/d11/f12 x:0 0 0 2024-05-27T09:29:27.327 INFO:tasks.workunit.client.0.smithi119.stdout:4/601: write d4/d1c/d39/f5d [8015844,91288] 0 2024-05-27T09:29:27.327 INFO:tasks.workunit.client.0.smithi119.stdout:4/602: write d4/de/d21/fab [789375,80881] 0 2024-05-27T09:29:27.328 INFO:tasks.workunit.client.0.smithi119.stdout:4/603: readlink d4/d1c/l97 0 2024-05-27T09:29:27.331 INFO:tasks.workunit.client.1.smithi138.stdout:2/59: rename d1/d10/db to d1/d13/d17 0 2024-05-27T09:29:27.331 INFO:tasks.workunit.client.1.smithi138.stdout:2/60: chown d1/cc 1030921 1 2024-05-27T09:29:27.333 INFO:tasks.workunit.client.0.smithi119.stdout:4/604: creat d4/de/d9b/fba x:0 0 0 2024-05-27T09:29:27.333 INFO:tasks.workunit.client.0.smithi119.stdout:4/605: truncate d4/d23/d31/fad 661707 0 2024-05-27T09:29:27.333 INFO:tasks.workunit.client.0.smithi119.stdout:4/606: chown d4/d48/f5e 0 1 2024-05-27T09:29:27.333 INFO:tasks.workunit.client.0.smithi119.stdout:4/607: dread - d4/d6/f7d zero size 2024-05-27T09:29:27.333 INFO:tasks.workunit.client.0.smithi119.stdout:9/491: dread d18/d19/f74 [0,4194304] 0 2024-05-27T09:29:27.333 INFO:tasks.workunit.client.0.smithi119.stdout:9/492: rmdir d18 39 2024-05-27T09:29:27.335 INFO:tasks.workunit.client.1.smithi138.stdout:1/108: dread d4/f24 [0,4194304] 0 2024-05-27T09:29:27.336 INFO:tasks.workunit.client.0.smithi119.stdout:5/634: write d6/db/f71 [1589681,103678] 0 2024-05-27T09:29:27.338 INFO:tasks.workunit.client.1.smithi138.stdout:2/61: symlink d1/l18 0 2024-05-27T09:29:27.338 INFO:tasks.workunit.client.1.smithi138.stdout:2/62: stat d1/d13/d17/fd 0 2024-05-27T09:29:27.338 INFO:tasks.workunit.client.0.smithi119.stdout:5/635: rename d6/dd/d46/d47/l48 to d6/db/dc/ldd 0 2024-05-27T09:29:27.339 INFO:tasks.workunit.client.0.smithi119.stdout:9/493: fsync d18/d19/d1b/d29/f9e 0 2024-05-27T09:29:27.340 INFO:tasks.workunit.client.0.smithi119.stdout:7/632: dread d7/d1a/f59 [0,4194304] 0 2024-05-27T09:29:27.340 INFO:tasks.workunit.client.1.smithi138.stdout:6/118: dread d3/f17 [0,4194304] 0 2024-05-27T09:29:27.340 INFO:tasks.workunit.client.1.smithi138.stdout:6/119: read - d3/d19/f1e zero size 2024-05-27T09:29:27.341 INFO:tasks.workunit.client.0.smithi119.stdout:7/633: creat d7/df/d19/d97/d92/db7/dba/d42/d64/fc9 x:0 0 0 2024-05-27T09:29:27.341 INFO:tasks.workunit.client.1.smithi138.stdout:1/109: rename d4/da/f16 to d4/d1a/f27 0 2024-05-27T09:29:27.343 INFO:tasks.workunit.client.1.smithi138.stdout:6/120: truncate d3/d19/f28 647314 0 2024-05-27T09:29:27.361 INFO:tasks.workunit.client.0.smithi119.stdout:5/636: write d6/d2f/f51 [236018,110386] 0 2024-05-27T09:29:27.361 INFO:tasks.workunit.client.0.smithi119.stdout:5/637: creat d6/db/dc/fde x:0 0 0 2024-05-27T09:29:27.361 INFO:tasks.workunit.client.0.smithi119.stdout:5/638: stat d6/db/de/l34 0 2024-05-27T09:29:27.361 INFO:tasks.workunit.client.1.smithi138.stdout:1/110: read d4/d8/f14 [1649520,56352] 0 2024-05-27T09:29:27.363 INFO:tasks.workunit.client.1.smithi138.stdout:1/111: mknod d4/d8/d22/c28 0 2024-05-27T09:29:27.380 INFO:tasks.workunit.client.0.smithi119.stdout:4/608: dwrite d4/d6/f7d [0,4194304] 0 2024-05-27T09:29:27.380 INFO:tasks.workunit.client.0.smithi119.stdout:4/609: chown d4/f22 954496 1 2024-05-27T09:29:27.384 INFO:tasks.workunit.client.0.smithi119.stdout:5/639: dread d6/fc8 [0,4194304] 0 2024-05-27T09:29:27.393 INFO:tasks.workunit.client.0.smithi119.stdout:9/494: dwrite fe [4194304,4194304] 0 2024-05-27T09:29:27.393 INFO:tasks.workunit.client.0.smithi119.stdout:9/495: write d18/d19/d1b/d29/f2f [975358,112241] 0 2024-05-27T09:29:27.393 INFO:tasks.workunit.client.0.smithi119.stdout:9/496: chown f6 241322782 1 2024-05-27T09:29:27.445 INFO:tasks.workunit.client.1.smithi138.stdout:1/112: rmdir d4/da 39 2024-05-27T09:29:27.447 INFO:tasks.workunit.client.1.smithi138.stdout:1/113: mkdir d4/d8/d22/d29 0 2024-05-27T09:29:27.449 INFO:tasks.workunit.client.1.smithi138.stdout:1/114: mknod d4/d8/c2a 0 2024-05-27T09:29:27.454 INFO:tasks.workunit.client.1.smithi138.stdout:1/115: unlink d4/d1a/c23 0 2024-05-27T09:29:27.454 INFO:tasks.workunit.client.1.smithi138.stdout:1/116: mkdir d4/d8/d22/d29/d2b 0 2024-05-27T09:29:27.460 INFO:tasks.workunit.client.1.smithi138.stdout:2/63: dwrite d1/d10/f16 [0,4194304] 0 2024-05-27T09:29:27.461 INFO:tasks.workunit.client.1.smithi138.stdout:6/121: dwrite d3/d14/f29 [0,4194304] 0 2024-05-27T09:29:27.461 INFO:tasks.workunit.client.1.smithi138.stdout:6/122: readlink d3/l25 0 2024-05-27T09:29:27.463 INFO:tasks.workunit.client.0.smithi119.stdout:7/634: mknod d7/d1a/d40/d44/d67/cca 0 2024-05-27T09:29:27.466 INFO:tasks.workunit.client.0.smithi119.stdout:4/610: mknod d4/de/d21/cbb 0 2024-05-27T09:29:27.468 INFO:tasks.workunit.client.0.smithi119.stdout:5/640: rename d6/db/f71 to d6/dd/d6e/fdf 0 2024-05-27T09:29:27.469 INFO:tasks.workunit.client.0.smithi119.stdout:9/497: dwrite d18/d19/d1b/d2a/d63/f6c [0,4194304] 0 2024-05-27T09:29:27.475 INFO:tasks.workunit.client.0.smithi119.stdout:7/635: symlink d7/df/d19/d97/lcb 0 2024-05-27T09:29:27.475 INFO:tasks.workunit.client.0.smithi119.stdout:7/636: creat d7/df/d19/d97/fcc x:0 0 0 2024-05-27T09:29:27.475 INFO:tasks.workunit.client.0.smithi119.stdout:7/637: truncate d7/d5f/d79/fc6 29008 0 2024-05-27T09:29:27.479 INFO:tasks.workunit.client.0.smithi119.stdout:4/611: rmdir d4/d6/d81 39 2024-05-27T09:29:27.481 INFO:tasks.workunit.client.1.smithi138.stdout:6/123: dread d3/f6 [0,4194304] 0 2024-05-27T09:29:27.481 INFO:tasks.workunit.client.1.smithi138.stdout:6/124: dread - d3/d19/f1e zero size 2024-05-27T09:29:27.483 INFO:tasks.workunit.client.0.smithi119.stdout:5/641: rmdir d6/d2f/d9d 39 2024-05-27T09:29:27.483 INFO:tasks.workunit.client.0.smithi119.stdout:5/642: stat d6/dd 0 2024-05-27T09:29:27.487 INFO:tasks.workunit.client.0.smithi119.stdout:4/612: truncate d4/d6/d28/d2f/d36/f72 690457 0 2024-05-27T09:29:27.487 INFO:tasks.workunit.client.0.smithi119.stdout:4/613: chown d4/de/c1f 1 1 2024-05-27T09:29:27.490 INFO:tasks.workunit.client.0.smithi119.stdout:4/614: symlink d4/d6/d3e/d89/lbc 0 2024-05-27T09:29:27.494 INFO:tasks.workunit.client.0.smithi119.stdout:4/615: rename d4/de/d21/d40/d47/f49 to d4/d6/d3e/d89/db2/fbd 0 2024-05-27T09:29:27.494 INFO:tasks.workunit.client.0.smithi119.stdout:8/570: sync 2024-05-27T09:29:27.496 INFO:tasks.workunit.client.0.smithi119.stdout:4/616: symlink d4/de/lbe 0 2024-05-27T09:29:27.499 INFO:tasks.workunit.client.0.smithi119.stdout:4/617: creat d4/d6/d81/fbf x:0 0 0 2024-05-27T09:29:27.504 INFO:tasks.workunit.client.1.smithi138.stdout:7/45: sync 2024-05-27T09:29:27.504 INFO:tasks.workunit.client.1.smithi138.stdout:0/93: sync 2024-05-27T09:29:27.504 INFO:tasks.workunit.client.1.smithi138.stdout:5/81: sync 2024-05-27T09:29:27.504 INFO:tasks.workunit.client.1.smithi138.stdout:5/82: creat d3/f1b x:0 0 0 2024-05-27T09:29:27.504 INFO:tasks.workunit.client.1.smithi138.stdout:5/83: readlink d3/l8 0 2024-05-27T09:29:27.504 INFO:tasks.workunit.client.1.smithi138.stdout:5/84: chown d3/lf 15 1 2024-05-27T09:29:27.504 INFO:tasks.workunit.client.0.smithi119.stdout:4/618: symlink d4/d6/d28/d2f/d36/lc0 0 2024-05-27T09:29:27.506 INFO:tasks.workunit.client.1.smithi138.stdout:0/94: symlink da/l27 0 2024-05-27T09:29:27.508 INFO:tasks.workunit.client.0.smithi119.stdout:4/619: link d4/de/d21/f85 d4/d23/d31/fc1 0 2024-05-27T09:29:27.509 INFO:tasks.workunit.client.0.smithi119.stdout:4/620: fdatasync d4/de/d21/d40/f42 0 2024-05-27T09:29:27.509 INFO:tasks.workunit.client.0.smithi119.stdout:4/621: chown d4/d1c/d39/f64 30195064 1 2024-05-27T09:29:27.510 INFO:tasks.workunit.client.1.smithi138.stdout:5/85: creat d3/d12/d16/f1c x:0 0 0 2024-05-27T09:29:27.510 INFO:tasks.workunit.client.1.smithi138.stdout:5/86: readlink d3/l1a 0 2024-05-27T09:29:27.510 INFO:tasks.workunit.client.1.smithi138.stdout:5/87: chown d3/c9 2 1 2024-05-27T09:29:27.511 INFO:tasks.workunit.client.0.smithi119.stdout:4/622: rename d4/de/d9b/la4 to d4/d48/lc2 0 2024-05-27T09:29:27.511 INFO:tasks.workunit.client.0.smithi119.stdout:4/623: creat d4/de/d66/d87/fc3 x:0 0 0 2024-05-27T09:29:27.512 INFO:tasks.workunit.client.1.smithi138.stdout:7/46: creat d0/d7/fe x:0 0 0 2024-05-27T09:29:27.513 INFO:tasks.workunit.client.0.smithi119.stdout:4/624: rename d4/d6/d28/l3d to d4/de/d9b/lc4 0 2024-05-27T09:29:27.516 INFO:tasks.workunit.client.0.smithi119.stdout:4/625: creat d4/d1c/fc5 x:0 0 0 2024-05-27T09:29:27.518 INFO:tasks.workunit.client.0.smithi119.stdout:4/626: creat d4/de/d21/d40/fc6 x:0 0 0 2024-05-27T09:29:27.520 INFO:tasks.workunit.client.0.smithi119.stdout:4/627: symlink d4/de/d21/d40/d47/d5c/d9e/lc7 0 2024-05-27T09:29:27.520 INFO:tasks.workunit.client.0.smithi119.stdout:4/628: truncate d4/d6/d28/f6b 834590 0 2024-05-27T09:29:27.525 INFO:tasks.workunit.client.1.smithi138.stdout:2/64: dwrite d1/d10/f16 [4194304,4194304] 0 2024-05-27T09:29:27.530 INFO:tasks.workunit.client.1.smithi138.stdout:0/95: symlink da/d17/d1d/l28 0 2024-05-27T09:29:27.530 INFO:tasks.workunit.client.1.smithi138.stdout:0/96: truncate da/f12 436277 0 2024-05-27T09:29:27.530 INFO:tasks.workunit.client.1.smithi138.stdout:0/97: dread - da/d17/f19 zero size 2024-05-27T09:29:27.531 INFO:tasks.workunit.client.1.smithi138.stdout:0/98: creat da/f29 x:0 0 0 2024-05-27T09:29:27.531 INFO:tasks.workunit.client.1.smithi138.stdout:1/117: dwrite d4/d1a/f27 [0,4194304] 0 2024-05-27T09:29:27.536 INFO:tasks.workunit.client.1.smithi138.stdout:2/65: unlink d1/f2 0 2024-05-27T09:29:27.539 INFO:tasks.workunit.client.0.smithi119.stdout:8/571: dread d3/d8/d1b/d77/d95/f9c [0,4194304] 0 2024-05-27T09:29:27.540 INFO:tasks.workunit.client.0.smithi119.stdout:8/572: write d3/d8/d1b/d46/f2f [2105616,22556] 0 2024-05-27T09:29:27.547 INFO:tasks.workunit.client.0.smithi119.stdout:8/573: link d3/d6e/d8e/fb0 d3/d8/d1b/d46/da4/fb9 0 2024-05-27T09:29:27.549 INFO:tasks.workunit.client.0.smithi119.stdout:7/638: dwrite d7/df/d19/d97/d92/db7/dba/d42/f73 [0,4194304] 0 2024-05-27T09:29:27.549 INFO:tasks.workunit.client.0.smithi119.stdout:7/639: truncate d7/df/d19/d9a/da6/fbd 943399 0 2024-05-27T09:29:27.552 INFO:tasks.workunit.client.0.smithi119.stdout:8/574: truncate d3/d8/d1b/d77/d95/f9c 773536 0 2024-05-27T09:29:27.552 INFO:tasks.workunit.client.0.smithi119.stdout:8/575: write d3/d39/f3e [710535,12121] 0 2024-05-27T09:29:27.552 INFO:tasks.workunit.client.0.smithi119.stdout:8/576: chown d3/d8/c2e 1 1 2024-05-27T09:29:27.556 INFO:tasks.workunit.client.1.smithi138.stdout:7/47: write d0/fb [5150796,45596] 0 2024-05-27T09:29:27.557 INFO:tasks.workunit.client.0.smithi119.stdout:7/640: truncate d7/d5f/f7c 1956579 0 2024-05-27T09:29:27.557 INFO:tasks.workunit.client.0.smithi119.stdout:8/577: creat d3/d8/fba x:0 0 0 2024-05-27T09:29:27.558 INFO:tasks.workunit.client.0.smithi119.stdout:8/578: mkdir d3/d8/d1b/d46/da4/dbb 0 2024-05-27T09:29:27.562 INFO:tasks.workunit.client.0.smithi119.stdout:7/641: symlink d7/df/d19/d97/d92/db7/dba/d42/d64/dad/lcd 0 2024-05-27T09:29:27.562 INFO:tasks.workunit.client.1.smithi138.stdout:7/48: rename d0/c9 to d0/d7/cf 0 2024-05-27T09:29:27.565 INFO:tasks.workunit.client.0.smithi119.stdout:7/642: rename d7/df/d19/d97/d92/db7/dba/d42/d64/dad/lcd to d7/df/d19/d97/d92/db7/dba/d42/d76/lce 0 2024-05-27T09:29:27.568 INFO:tasks.workunit.client.0.smithi119.stdout:7/643: mkdir d7/d1a/d99/dcf 0 2024-05-27T09:29:27.569 INFO:tasks.workunit.client.1.smithi138.stdout:0/99: dwrite da/f12 [0,4194304] 0 2024-05-27T09:29:27.569 INFO:tasks.workunit.client.1.smithi138.stdout:0/100: chown da/d17/c1b 156644497 1 2024-05-27T09:29:27.569 INFO:tasks.workunit.client.1.smithi138.stdout:0/101: dread - da/d17/d1d/d1f/f26 zero size 2024-05-27T09:29:27.572 INFO:tasks.workunit.client.0.smithi119.stdout:7/644: link d7/df/d19/d97/cc d7/df/d19/d4d/d89/cd0 0 2024-05-27T09:29:27.573 INFO:tasks.workunit.client.1.smithi138.stdout:0/102: mkdir da/d2a 0 2024-05-27T09:29:27.573 INFO:tasks.workunit.client.1.smithi138.stdout:0/103: fsync da/f22 0 2024-05-27T09:29:27.573 INFO:tasks.workunit.client.0.smithi119.stdout:5/643: dwrite d6/dd/d46/d47/d4b/f58 [0,4194304] 0 2024-05-27T09:29:27.580 INFO:tasks.workunit.client.1.smithi138.stdout:5/88: dwrite f0 [8388608,4194304] 0 2024-05-27T09:29:27.583 INFO:tasks.workunit.client.1.smithi138.stdout:1/118: dwrite d4/f18 [0,4194304] 0 2024-05-27T09:29:27.585 INFO:tasks.workunit.client.1.smithi138.stdout:5/89: mkdir d3/d12/d16/d1d 0 2024-05-27T09:29:27.585 INFO:tasks.workunit.client.1.smithi138.stdout:5/90: dread - d3/f15 zero size 2024-05-27T09:29:27.585 INFO:tasks.workunit.client.1.smithi138.stdout:5/91: truncate d3/d12/d16/f1c 115254 0 2024-05-27T09:29:27.585 INFO:tasks.workunit.client.1.smithi138.stdout:5/92: chown d3/l1a 3 1 2024-05-27T09:29:27.585 INFO:tasks.workunit.client.1.smithi138.stdout:5/93: chown d3/d12/d16/d1d 17 1 2024-05-27T09:29:27.585 INFO:tasks.workunit.client.0.smithi119.stdout:8/579: read d3/d8/d4d/d5d/d60/f69 [1898286,1626] 0 2024-05-27T09:29:27.591 INFO:tasks.workunit.client.1.smithi138.stdout:7/49: dread d0/fb [4194304,4194304] 0 2024-05-27T09:29:27.594 INFO:tasks.workunit.client.0.smithi119.stdout:7/645: dread d7/df/d19/d1d/f23 [0,4194304] 0 2024-05-27T09:29:27.595 INFO:tasks.workunit.client.0.smithi119.stdout:7/646: chown d7/df/d19/f1f 10284294 1 2024-05-27T09:29:27.595 INFO:tasks.workunit.client.0.smithi119.stdout:7/647: stat d7/df/d19 0 2024-05-27T09:29:27.595 INFO:tasks.workunit.client.0.smithi119.stdout:7/648: creat d7/df/d19/d97/d92/fd1 x:0 0 0 2024-05-27T09:29:27.595 INFO:tasks.workunit.client.0.smithi119.stdout:4/629: dwrite d4/f8f [0,4194304] 0 2024-05-27T09:29:27.596 INFO:tasks.workunit.client.1.smithi138.stdout:7/50: creat d0/f10 x:0 0 0 2024-05-27T09:29:27.596 INFO:tasks.workunit.client.1.smithi138.stdout:7/51: dread - d0/d7/fe zero size 2024-05-27T09:29:27.597 INFO:tasks.workunit.client.0.smithi119.stdout:8/580: rename d3/d8/d1b/f6c to d3/d8/d1b/db1/fbc 0 2024-05-27T09:29:27.604 INFO:tasks.workunit.client.0.smithi119.stdout:7/649: creat d7/d1a/d99/dcf/fd2 x:0 0 0 2024-05-27T09:29:27.604 INFO:tasks.workunit.client.0.smithi119.stdout:7/650: truncate d7/df/d19/d97/fbf 596439 0 2024-05-27T09:29:27.604 INFO:tasks.workunit.client.0.smithi119.stdout:5/644: dread d6/dd/f73 [0,4194304] 0 2024-05-27T09:29:27.606 INFO:tasks.workunit.client.0.smithi119.stdout:5/645: creat d6/dd/d39/d19/d3c/d52/fe0 x:0 0 0 2024-05-27T09:29:27.607 INFO:tasks.workunit.client.0.smithi119.stdout:5/646: dread - d6/dd/d39/d74/fa1 zero size 2024-05-27T09:29:27.608 INFO:tasks.workunit.client.0.smithi119.stdout:4/630: rmdir d4/de/d21/d40/d47/d92 0 2024-05-27T09:29:27.609 INFO:tasks.workunit.client.0.smithi119.stdout:0/565: sync 2024-05-27T09:29:27.609 INFO:tasks.workunit.client.0.smithi119.stdout:3/729: sync 2024-05-27T09:29:27.609 INFO:tasks.workunit.client.0.smithi119.stdout:2/615: sync 2024-05-27T09:29:27.609 INFO:tasks.workunit.client.0.smithi119.stdout:6/597: sync 2024-05-27T09:29:27.609 INFO:tasks.workunit.client.0.smithi119.stdout:1/682: sync 2024-05-27T09:29:27.610 INFO:tasks.workunit.client.0.smithi119.stdout:5/647: symlink d6/db/de/d27/d94/le1 0 2024-05-27T09:29:27.611 INFO:tasks.workunit.client.0.smithi119.stdout:9/498: sync 2024-05-27T09:29:27.612 INFO:tasks.workunit.client.0.smithi119.stdout:4/631: mkdir d4/d6/d28/d2f/d36/dc8 0 2024-05-27T09:29:27.612 INFO:tasks.workunit.client.0.smithi119.stdout:4/632: stat d4/de/d21/d40/d47/f90 0 2024-05-27T09:29:27.612 INFO:tasks.workunit.client.0.smithi119.stdout:4/633: chown d4/d23/f35 65896137 1 2024-05-27T09:29:27.613 INFO:tasks.workunit.client.0.smithi119.stdout:8/581: write d3/d8/d4d/d5d/d74/f98 [6099379,127822] 0 2024-05-27T09:29:27.613 INFO:tasks.workunit.client.0.smithi119.stdout:4/634: write d4/d23/d31/f33 [1261432,103962] 0 2024-05-27T09:29:27.613 INFO:tasks.workunit.client.0.smithi119.stdout:8/582: dread - d3/d6e/d8e/fb0 zero size 2024-05-27T09:29:27.613 INFO:tasks.workunit.client.0.smithi119.stdout:3/730: mknod db/dc9/d98/ceb 0 2024-05-27T09:29:27.613 INFO:tasks.workunit.client.0.smithi119.stdout:0/566: creat d2/d6/d9/d12/fb4 x:0 0 0 2024-05-27T09:29:27.614 INFO:tasks.workunit.client.0.smithi119.stdout:6/598: mkdir d0/d2/d57/dd0 0 2024-05-27T09:29:27.614 INFO:tasks.workunit.client.0.smithi119.stdout:1/683: mkdir d7/dd7 0 2024-05-27T09:29:27.615 INFO:tasks.workunit.client.0.smithi119.stdout:5/648: link d6/db/de/f25 d6/dd/d6e/fe2 0 2024-05-27T09:29:27.617 INFO:tasks.workunit.client.0.smithi119.stdout:9/499: mkdir d18/d19/d1b/d2a/d9f 0 2024-05-27T09:29:27.618 INFO:tasks.workunit.client.0.smithi119.stdout:5/649: rename d6/d2f/cc5 to d6/d2f/ce3 0 2024-05-27T09:29:27.618 INFO:tasks.workunit.client.0.smithi119.stdout:6/599: mknod d0/d2/d1c/d21/cd1 0 2024-05-27T09:29:27.619 INFO:tasks.workunit.client.0.smithi119.stdout:3/731: creat db/d36/d45/d4c/d62/d13/d33/d9b/fec x:0 0 0 2024-05-27T09:29:27.619 INFO:tasks.workunit.client.0.smithi119.stdout:3/732: fdatasync db/f2f 0 2024-05-27T09:29:27.619 INFO:tasks.workunit.client.0.smithi119.stdout:3/733: fsync db/d36/d45/d4c/d62/d13/f3d 0 2024-05-27T09:29:27.619 INFO:tasks.workunit.client.0.smithi119.stdout:3/734: write db/f37 [4871241,25975] 0 2024-05-27T09:29:27.620 INFO:tasks.workunit.client.0.smithi119.stdout:9/500: getdents d18/d19/d1b/d40 0 2024-05-27T09:29:27.620 INFO:tasks.workunit.client.0.smithi119.stdout:7/651: read d7/d1a/f1c [532790,57340] 0 2024-05-27T09:29:27.622 INFO:tasks.workunit.client.1.smithi138.stdout:1/119: read d4/d1a/f27 [2281187,87002] 0 2024-05-27T09:29:27.622 INFO:tasks.workunit.client.0.smithi119.stdout:6/600: symlink d0/d2/d57/d37/d74/ld2 0 2024-05-27T09:29:27.623 INFO:tasks.workunit.client.0.smithi119.stdout:5/650: rename d6/dd/d39/d19/d3c/d52/fe0 to d6/dd/d46/d47/d4b/fe4 0 2024-05-27T09:29:27.627 INFO:tasks.workunit.client.1.smithi138.stdout:1/120: creat d4/d8/d22/f2c x:0 0 0 2024-05-27T09:29:27.631 INFO:tasks.workunit.client.1.smithi138.stdout:1/121: mkdir d4/d1a/d2d 0 2024-05-27T09:29:27.644 INFO:tasks.workunit.client.0.smithi119.stdout:5/651: truncate d6/dd/f64 2804961 0 2024-05-27T09:29:27.645 INFO:tasks.workunit.client.0.smithi119.stdout:3/735: truncate db/d36/d45/d4c/f87 569204 0 2024-05-27T09:29:27.650 INFO:tasks.workunit.client.0.smithi119.stdout:9/501: rename d18/d19/d1b/d2a/d63/f6c to d18/d19/d9b/fa0 0 2024-05-27T09:29:27.650 INFO:tasks.workunit.client.0.smithi119.stdout:1/684: dread d7/f1e [0,4194304] 0 2024-05-27T09:29:27.653 INFO:tasks.workunit.client.0.smithi119.stdout:1/685: mknod d7/db6/cd8 0 2024-05-27T09:29:27.653 INFO:tasks.workunit.client.0.smithi119.stdout:5/652: rename d6/db/dc/f61 to d6/dd/d46/d47/d4b/dd8/ddb/fe5 0 2024-05-27T09:29:27.655 INFO:tasks.workunit.client.1.smithi138.stdout:9/44: sync 2024-05-27T09:29:27.656 INFO:tasks.workunit.client.1.smithi138.stdout:9/45: chown d6/db/de 18 1 2024-05-27T09:29:27.656 INFO:tasks.workunit.client.1.smithi138.stdout:9/46: getdents d6/db/de 0 2024-05-27T09:29:27.664 INFO:tasks.workunit.client.1.smithi138.stdout:5/94: dwrite d3/f18 [0,4194304] 0 2024-05-27T09:29:27.667 INFO:tasks.workunit.client.1.smithi138.stdout:5/95: creat d3/d12/d16/d1d/f1e x:0 0 0 2024-05-27T09:29:27.669 INFO:tasks.workunit.client.0.smithi119.stdout:3/736: write db/d36/d45/d4c/d62/d13/f3d [2846274,34482] 0 2024-05-27T09:29:27.669 INFO:tasks.workunit.client.0.smithi119.stdout:3/737: chown db/l11 79105 1 2024-05-27T09:29:27.672 INFO:tasks.workunit.client.1.smithi138.stdout:9/47: write f4 [2860396,64439] 0 2024-05-27T09:29:27.672 INFO:tasks.workunit.client.0.smithi119.stdout:3/738: mknod db/d5b/ced 0 2024-05-27T09:29:27.672 INFO:tasks.workunit.client.1.smithi138.stdout:9/48: mknod d6/db/dc/cf 0 2024-05-27T09:29:27.679 INFO:tasks.workunit.client.0.smithi119.stdout:8/583: dwrite d3/f1a [0,4194304] 0 2024-05-27T09:29:27.679 INFO:tasks.workunit.client.0.smithi119.stdout:8/584: truncate d3/d8/d1b/d46/f7c 2577064 0 2024-05-27T09:29:27.682 INFO:tasks.workunit.client.0.smithi119.stdout:7/652: dread d7/faa [0,4194304] 0 2024-05-27T09:29:27.686 INFO:tasks.workunit.client.0.smithi119.stdout:7/653: fdatasync d7/d5f/f8c 0 2024-05-27T09:29:27.686 INFO:tasks.workunit.client.0.smithi119.stdout:7/654: stat d7/d1a/d40/d44/f5d 0 2024-05-27T09:29:27.689 INFO:tasks.workunit.client.1.smithi138.stdout:0/104: dwrite f1 [4194304,4194304] 0 2024-05-27T09:29:27.689 INFO:tasks.workunit.client.1.smithi138.stdout:0/105: dread - da/f20 zero size 2024-05-27T09:29:27.698 INFO:tasks.workunit.client.1.smithi138.stdout:0/106: mkdir da/d2a/d2b 0 2024-05-27T09:29:27.703 INFO:tasks.workunit.client.0.smithi119.stdout:3/739: dread db/d36/d45/d93/fa4 [0,4194304] 0 2024-05-27T09:29:27.703 INFO:tasks.workunit.client.0.smithi119.stdout:3/740: write db/d7f/f74 [621660,49317] 0 2024-05-27T09:29:27.706 INFO:tasks.workunit.client.1.smithi138.stdout:0/107: symlink da/d17/d1d/l2c 0 2024-05-27T09:29:27.706 INFO:tasks.workunit.client.1.smithi138.stdout:0/108: chown f1 13370 1 2024-05-27T09:29:27.709 INFO:tasks.workunit.client.1.smithi138.stdout:0/109: mknod da/d17/c2d 0 2024-05-27T09:29:27.710 INFO:tasks.workunit.client.1.smithi138.stdout:0/110: write f5 [1716714,71038] 0 2024-05-27T09:29:27.716 INFO:tasks.workunit.client.1.smithi138.stdout:0/111: unlink da/d17/c2d 0 2024-05-27T09:29:27.716 INFO:tasks.workunit.client.1.smithi138.stdout:0/112: dread - da/f20 zero size 2024-05-27T09:29:27.718 INFO:tasks.workunit.client.1.smithi138.stdout:0/113: mknod da/d17/d1d/c2e 0 2024-05-27T09:29:27.729 INFO:tasks.workunit.client.1.smithi138.stdout:7/52: dwrite d0/f10 [0,4194304] 0 2024-05-27T09:29:27.733 INFO:tasks.workunit.client.1.smithi138.stdout:9/49: read f1 [904466,40852] 0 2024-05-27T09:29:27.735 INFO:tasks.workunit.client.1.smithi138.stdout:7/53: rename d0/ca to d0/d7/c11 0 2024-05-27T09:29:27.735 INFO:tasks.workunit.client.1.smithi138.stdout:7/54: creat d0/d7/f12 x:0 0 0 2024-05-27T09:29:27.736 INFO:tasks.workunit.client.1.smithi138.stdout:5/96: dwrite d3/f15 [0,4194304] 0 2024-05-27T09:29:27.737 INFO:tasks.workunit.client.0.smithi119.stdout:2/616: dwrite d2/da/d4e/f4f [0,4194304] 0 2024-05-27T09:29:27.740 INFO:tasks.workunit.client.0.smithi119.stdout:4/635: dwrite d4/d6/d28/f34 [0,4194304] 0 2024-05-27T09:29:27.741 INFO:tasks.workunit.client.1.smithi138.stdout:9/50: creat d6/db/f10 x:0 0 0 2024-05-27T09:29:27.741 INFO:tasks.workunit.client.1.smithi138.stdout:9/51: write f4 [2673218,20584] 0 2024-05-27T09:29:27.746 INFO:tasks.workunit.client.1.smithi138.stdout:5/97: mknod d3/d12/c1f 0 2024-05-27T09:29:27.747 INFO:tasks.workunit.client.1.smithi138.stdout:9/52: symlink d6/db/l11 0 2024-05-27T09:29:27.748 INFO:tasks.workunit.client.1.smithi138.stdout:7/55: symlink d0/l13 0 2024-05-27T09:29:27.751 INFO:tasks.workunit.client.0.smithi119.stdout:2/617: rename d2/da/d8a/dd4/d69/fb9 to d2/da/d8a/dd4/d89/d94/fd5 0 2024-05-27T09:29:27.751 INFO:tasks.workunit.client.0.smithi119.stdout:3/741: dread db/d36/d45/d4c/d62/d13/f99 [0,4194304] 0 2024-05-27T09:29:27.752 INFO:tasks.workunit.client.0.smithi119.stdout:6/601: dwrite d0/d2/d1c/d21/f3c [4194304,4194304] 0 2024-05-27T09:29:27.755 INFO:tasks.workunit.client.0.smithi119.stdout:3/742: creat db/d36/d45/d4d/fee x:0 0 0 2024-05-27T09:29:27.761 INFO:tasks.workunit.client.0.smithi119.stdout:4/636: rename d4/d1c/f1e to d4/de/d21/d40/d47/da3/fc9 0 2024-05-27T09:29:27.761 INFO:tasks.workunit.client.0.smithi119.stdout:3/743: creat db/dc9/fef x:0 0 0 2024-05-27T09:29:27.765 INFO:tasks.workunit.client.0.smithi119.stdout:8/585: dread d3/d8/d1b/d46/f2f [0,4194304] 0 2024-05-27T09:29:27.765 INFO:tasks.workunit.client.1.smithi138.stdout:1/122: dwrite d4/f12 [0,4194304] 0 2024-05-27T09:29:27.766 INFO:tasks.workunit.client.0.smithi119.stdout:0/567: dwrite d2/d1a/d20/f64 [4194304,4194304] 0 2024-05-27T09:29:27.766 INFO:tasks.workunit.client.1.smithi138.stdout:1/123: write d4/d8/f1c [372133,103517] 0 2024-05-27T09:29:27.767 INFO:tasks.workunit.client.0.smithi119.stdout:8/586: creat d3/d8/d4d/d79/fbd x:0 0 0 2024-05-27T09:29:27.771 INFO:tasks.workunit.client.0.smithi119.stdout:1/686: dwrite d7/dd/f77 [0,4194304] 0 2024-05-27T09:29:27.772 INFO:tasks.workunit.client.0.smithi119.stdout:1/687: creat d7/dd/d25/d29/d38/d49/fd9 x:0 0 0 2024-05-27T09:29:27.773 INFO:tasks.workunit.client.0.smithi119.stdout:1/688: fdatasync d7/dd/d25/d29/d38/d3c/fbf 0 2024-05-27T09:29:27.773 INFO:tasks.workunit.client.0.smithi119.stdout:1/689: dread - d7/db/d8f/fc4 zero size 2024-05-27T09:29:27.783 INFO:tasks.workunit.client.1.smithi138.stdout:9/53: read f2 [4034971,23840] 0 2024-05-27T09:29:27.783 INFO:tasks.workunit.client.1.smithi138.stdout:9/54: read f3 [241440,34944] 0 2024-05-27T09:29:27.783 INFO:tasks.workunit.client.1.smithi138.stdout:5/98: dread d3/f18 [0,4194304] 0 2024-05-27T09:29:27.783 INFO:tasks.workunit.client.1.smithi138.stdout:0/114: dwrite da/f15 [0,4194304] 0 2024-05-27T09:29:27.785 INFO:tasks.workunit.client.1.smithi138.stdout:0/115: rename da/d17/c1c to da/d2a/d2b/c2f 0 2024-05-27T09:29:27.786 INFO:tasks.workunit.client.0.smithi119.stdout:0/568: symlink d2/d1a/lb5 0 2024-05-27T09:29:27.786 INFO:tasks.workunit.client.0.smithi119.stdout:8/587: mknod d3/d8/d1b/d77/d95/cbe 0 2024-05-27T09:29:27.791 INFO:tasks.workunit.client.0.smithi119.stdout:4/637: dread d4/d6/f74 [0,4194304] 0 2024-05-27T09:29:27.791 INFO:tasks.workunit.client.0.smithi119.stdout:7/655: dwrite d7/df/d19/d97/fbf [0,4194304] 0 2024-05-27T09:29:27.795 INFO:tasks.workunit.client.0.smithi119.stdout:8/588: mknod d3/d8/d4d/d5d/d66/d85/cbf 0 2024-05-27T09:29:27.795 INFO:tasks.workunit.client.0.smithi119.stdout:0/569: rename d2/d6/d9/d24/l41 to d2/d6/d9/d12/d4f/lb6 0 2024-05-27T09:29:27.800 INFO:tasks.workunit.client.0.smithi119.stdout:7/656: mkdir d7/df/d19/d97/d92/db7/dba/d42/d64/dad/dd3 0 2024-05-27T09:29:27.801 INFO:tasks.workunit.client.0.smithi119.stdout:7/657: truncate d7/df/d19/d97/d92/db7/dba/d42/d76/f82 348808 0 2024-05-27T09:29:27.802 INFO:tasks.workunit.client.0.smithi119.stdout:0/570: link d2/d6/d9/d12/d4b/la1 d2/d6/d9/d2f/d95/lb7 0 2024-05-27T09:29:27.806 INFO:tasks.workunit.client.0.smithi119.stdout:8/589: dread d3/d8/d4d/f67 [0,4194304] 0 2024-05-27T09:29:27.808 INFO:tasks.workunit.client.0.smithi119.stdout:0/571: mkdir d2/d6/d23/db8 0 2024-05-27T09:29:27.809 INFO:tasks.workunit.client.1.smithi138.stdout:7/56: dwrite d0/d7/fe [0,4194304] 0 2024-05-27T09:29:27.813 INFO:tasks.workunit.client.0.smithi119.stdout:6/602: read d0/d85/f8a [1376225,51584] 0 2024-05-27T09:29:27.816 INFO:tasks.workunit.client.0.smithi119.stdout:8/590: truncate d3/d8/d1b/d46/f81 3839153 0 2024-05-27T09:29:27.816 INFO:tasks.workunit.client.1.smithi138.stdout:6/125: sync 2024-05-27T09:29:27.816 INFO:tasks.workunit.client.1.smithi138.stdout:3/55: sync 2024-05-27T09:29:27.816 INFO:tasks.workunit.client.1.smithi138.stdout:3/56: creat d0/f12 x:0 0 0 2024-05-27T09:29:27.816 INFO:tasks.workunit.client.1.smithi138.stdout:3/57: creat d0/d2/f13 x:0 0 0 2024-05-27T09:29:27.817 INFO:tasks.workunit.client.1.smithi138.stdout:7/57: truncate d0/fb 3734931 0 2024-05-27T09:29:27.817 INFO:tasks.workunit.client.1.smithi138.stdout:7/58: readlink d0/l13 0 2024-05-27T09:29:27.817 INFO:tasks.workunit.client.1.smithi138.stdout:4/72: sync 2024-05-27T09:29:27.820 INFO:tasks.workunit.client.1.smithi138.stdout:8/63: sync 2024-05-27T09:29:27.820 INFO:tasks.workunit.client.1.smithi138.stdout:6/126: symlink d3/d14/d1c/l2b 0 2024-05-27T09:29:27.820 INFO:tasks.workunit.client.1.smithi138.stdout:6/127: stat d3/d14/d1c/l24 0 2024-05-27T09:29:27.824 INFO:tasks.workunit.client.1.smithi138.stdout:4/73: rename d7/df/f10 to d7/f16 0 2024-05-27T09:29:27.824 INFO:tasks.workunit.client.1.smithi138.stdout:0/116: write da/f22 [3831817,40911] 0 2024-05-27T09:29:27.827 INFO:tasks.workunit.client.1.smithi138.stdout:7/59: getdents d0 0 2024-05-27T09:29:27.827 INFO:tasks.workunit.client.1.smithi138.stdout:6/128: mknod d3/d14/c2c 0 2024-05-27T09:29:27.827 INFO:tasks.workunit.client.0.smithi119.stdout:9/502: dwrite d18/d19/d1b/d40/d44/f70 [0,4194304] 0 2024-05-27T09:29:27.827 INFO:tasks.workunit.client.0.smithi119.stdout:9/503: rename d18/d19/d1b to d18/d19/d1b/da1 22 2024-05-27T09:29:27.830 INFO:tasks.workunit.client.1.smithi138.stdout:0/117: link f5 da/d2a/f30 0 2024-05-27T09:29:27.834 INFO:tasks.workunit.client.0.smithi119.stdout:8/591: creat d3/de/dad/fc0 x:0 0 0 2024-05-27T09:29:27.834 INFO:tasks.workunit.client.0.smithi119.stdout:8/592: fdatasync d3/d8/fba 0 2024-05-27T09:29:27.835 INFO:tasks.workunit.client.0.smithi119.stdout:8/593: stat d3/d8/d4d/d5d/d60/f6b 0 2024-05-27T09:29:27.835 INFO:tasks.workunit.client.1.smithi138.stdout:0/118: rmdir da/d2a 39 2024-05-27T09:29:27.835 INFO:tasks.workunit.client.1.smithi138.stdout:0/119: readlink da/d17/d1d/l2c 0 2024-05-27T09:29:27.842 INFO:tasks.workunit.client.0.smithi119.stdout:4/638: dread d4/de/d21/fab [0,4194304] 0 2024-05-27T09:29:27.844 INFO:tasks.workunit.client.1.smithi138.stdout:2/66: sync 2024-05-27T09:29:27.844 INFO:tasks.workunit.client.1.smithi138.stdout:2/67: fdatasync d1/d13/d17/f11 0 2024-05-27T09:29:27.844 INFO:tasks.workunit.client.1.smithi138.stdout:2/68: write d1/d10/f6 [1125229,50219] 0 2024-05-27T09:29:27.844 INFO:tasks.workunit.client.1.smithi138.stdout:2/69: write d1/d13/d17/fd [1013218,74313] 0 2024-05-27T09:29:27.846 INFO:tasks.workunit.client.0.smithi119.stdout:8/594: dread d3/d8/d4d/d5d/d60/f6b [0,4194304] 0 2024-05-27T09:29:27.847 INFO:tasks.workunit.client.1.smithi138.stdout:2/70: creat d1/d15/f19 x:0 0 0 2024-05-27T09:29:27.851 INFO:tasks.workunit.client.1.smithi138.stdout:8/64: write d3/d8/fc [74330,8055] 0 2024-05-27T09:29:27.852 INFO:tasks.workunit.client.1.smithi138.stdout:8/65: link c0 d3/c13 0 2024-05-27T09:29:27.852 INFO:tasks.workunit.client.1.smithi138.stdout:8/66: readlink d3/l6 0 2024-05-27T09:29:27.852 INFO:tasks.workunit.client.1.smithi138.stdout:8/67: readlink d3/l6 0 2024-05-27T09:29:27.852 INFO:tasks.workunit.client.1.smithi138.stdout:8/68: fsync f1 0 2024-05-27T09:29:27.852 INFO:tasks.workunit.client.1.smithi138.stdout:8/69: readlink d3/l6 0 2024-05-27T09:29:27.854 INFO:tasks.workunit.client.1.smithi138.stdout:8/70: rmdir d3/d8/d11 39 2024-05-27T09:29:27.856 INFO:tasks.workunit.client.1.smithi138.stdout:8/71: creat d3/d8/f14 x:0 0 0 2024-05-27T09:29:27.857 INFO:tasks.workunit.client.1.smithi138.stdout:8/72: fsync d3/ff 0 2024-05-27T09:29:27.861 INFO:tasks.workunit.client.1.smithi138.stdout:8/73: truncate f1 5190847 0 2024-05-27T09:29:27.861 INFO:tasks.workunit.client.1.smithi138.stdout:8/74: chown d3/d8/d11 460440 1 2024-05-27T09:29:27.862 INFO:tasks.workunit.client.0.smithi119.stdout:0/572: rmdir d2/d6/d9/d12 39 2024-05-27T09:29:27.862 INFO:tasks.workunit.client.0.smithi119.stdout:0/573: dread - d2/d6/d9/d12/d4b/d96/fad zero size 2024-05-27T09:29:27.863 INFO:tasks.workunit.client.1.smithi138.stdout:6/129: dread d3/f6 [0,4194304] 0 2024-05-27T09:29:27.863 INFO:tasks.workunit.client.1.smithi138.stdout:6/130: chown d3/f5 126877 1 2024-05-27T09:29:27.863 INFO:tasks.workunit.client.1.smithi138.stdout:8/75: symlink d3/d8/l15 0 2024-05-27T09:29:27.863 INFO:tasks.workunit.client.1.smithi138.stdout:8/76: fdatasync f1 0 2024-05-27T09:29:27.864 INFO:tasks.workunit.client.1.smithi138.stdout:6/131: mkdir d3/d11/d2d 0 2024-05-27T09:29:27.864 INFO:tasks.workunit.client.1.smithi138.stdout:6/132: chown d3/d11 0 1 2024-05-27T09:29:27.869 INFO:tasks.workunit.client.1.smithi138.stdout:8/77: mknod d3/d8/c16 0 2024-05-27T09:29:27.869 INFO:tasks.workunit.client.1.smithi138.stdout:8/78: dread - d3/d8/f14 zero size 2024-05-27T09:29:27.869 INFO:tasks.workunit.client.0.smithi119.stdout:9/504: read d18/d19/d1b/d40/f62 [4171017,22610] 0 2024-05-27T09:29:27.869 INFO:tasks.workunit.client.0.smithi119.stdout:2/618: dwrite d2/da/d8a/f9f [0,4194304] 0 2024-05-27T09:29:27.875 INFO:tasks.workunit.client.1.smithi138.stdout:0/120: dread da/f12 [0,4194304] 0 2024-05-27T09:29:27.875 INFO:tasks.workunit.client.1.smithi138.stdout:0/121: fdatasync da/f20 0 2024-05-27T09:29:27.875 INFO:tasks.workunit.client.1.smithi138.stdout:0/122: write da/d17/f19 [458025,33866] 0 2024-05-27T09:29:27.875 INFO:tasks.workunit.client.1.smithi138.stdout:4/74: dread f0 [0,4194304] 0 2024-05-27T09:29:27.879 INFO:tasks.workunit.client.1.smithi138.stdout:2/71: dread d1/d10/f16 [4194304,4194304] 0 2024-05-27T09:29:27.887 INFO:tasks.workunit.client.0.smithi119.stdout:4/639: write d4/f22 [748699,30278] 0 2024-05-27T09:29:27.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:27 smithi138 ceph-mon[34885]: pgmap v11: 65 pgs: 65 active+clean; 1.2 GiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 93 MiB/s rd, 114 MiB/s wr, 560 op/s 2024-05-27T09:29:27.889 INFO:tasks.workunit.client.0.smithi119.stdout:0/574: dread d2/d1a/d20/f8b [0,4194304] 0 2024-05-27T09:29:27.892 INFO:tasks.workunit.client.0.smithi119.stdout:4/640: mknod d4/d6/d28/d2f/db8/cca 0 2024-05-27T09:29:27.892 INFO:tasks.workunit.client.0.smithi119.stdout:4/641: write d4/f7e [969579,60622] 0 2024-05-27T09:29:27.892 INFO:tasks.workunit.client.0.smithi119.stdout:9/505: dread d18/d19/f36 [0,4194304] 0 2024-05-27T09:29:27.894 INFO:tasks.workunit.client.1.smithi138.stdout:5/99: dwrite d3/f18 [4194304,4194304] 0 2024-05-27T09:29:27.894 INFO:tasks.workunit.client.1.smithi138.stdout:2/72: creat d1/d10/f1a x:0 0 0 2024-05-27T09:29:27.894 INFO:tasks.workunit.client.1.smithi138.stdout:2/73: write d1/d13/d17/f11 [522310,42506] 0 2024-05-27T09:29:27.896 INFO:tasks.workunit.client.0.smithi119.stdout:6/603: dwrite d0/d2/d1c/fa0 [0,4194304] 0 2024-05-27T09:29:27.898 INFO:tasks.workunit.client.0.smithi119.stdout:4/642: rename d4/d6/f74 to d4/de/d21/d40/d47/d5c/d9e/fcb 0 2024-05-27T09:29:27.899 INFO:tasks.workunit.client.1.smithi138.stdout:8/79: write d3/f5 [1790405,90251] 0 2024-05-27T09:29:27.899 INFO:tasks.workunit.client.1.smithi138.stdout:8/80: readlink d3/l6 0 2024-05-27T09:29:27.899 INFO:tasks.workunit.client.1.smithi138.stdout:5/100: mknod d3/c20 0 2024-05-27T09:29:27.899 INFO:tasks.workunit.client.1.smithi138.stdout:5/101: creat d3/f21 x:0 0 0 2024-05-27T09:29:27.899 INFO:tasks.workunit.client.1.smithi138.stdout:5/102: write d3/f10 [445081,56433] 0 2024-05-27T09:29:27.900 INFO:tasks.workunit.client.1.smithi138.stdout:5/103: mkdir d3/d12/d16/d1d/d22 0 2024-05-27T09:29:27.902 INFO:tasks.workunit.client.1.smithi138.stdout:1/124: dwrite d4/f7 [0,4194304] 0 2024-05-27T09:29:27.909 INFO:tasks.workunit.client.1.smithi138.stdout:3/58: fsync d0/f12 0 2024-05-27T09:29:27.909 INFO:tasks.workunit.client.1.smithi138.stdout:3/59: dread - d0/f12 zero size 2024-05-27T09:29:27.909 INFO:tasks.workunit.client.1.smithi138.stdout:4/75: getdents d7/df 0 2024-05-27T09:29:27.909 INFO:tasks.workunit.client.1.smithi138.stdout:4/76: readlink d7/df/l14 0 2024-05-27T09:29:27.910 INFO:tasks.workunit.client.1.smithi138.stdout:4/77: creat d7/f17 x:0 0 0 2024-05-27T09:29:27.910 INFO:tasks.workunit.client.1.smithi138.stdout:4/78: truncate d7/f12 46745 0 2024-05-27T09:29:27.910 INFO:tasks.workunit.client.1.smithi138.stdout:4/79: creat d7/f18 x:0 0 0 2024-05-27T09:29:27.923 INFO:tasks.workunit.client.0.smithi119.stdout:6/604: mkdir d0/d2/d1c/d21/d58/dd3 0 2024-05-27T09:29:27.926 INFO:tasks.workunit.client.1.smithi138.stdout:9/55: dwrite f1 [0,4194304] 0 2024-05-27T09:29:27.936 INFO:tasks.workunit.client.1.smithi138.stdout:4/80: truncate f3 2554358 0 2024-05-27T09:29:27.939 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:27 smithi119 ceph-mon[29082]: pgmap v11: 65 pgs: 65 active+clean; 1.2 GiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 93 MiB/s rd, 114 MiB/s wr, 560 op/s 2024-05-27T09:29:27.943 INFO:tasks.workunit.client.1.smithi138.stdout:4/81: creat d7/df/f19 x:0 0 0 2024-05-27T09:29:27.950 INFO:tasks.workunit.client.1.smithi138.stdout:4/82: unlink d7/f16 0 2024-05-27T09:29:27.951 INFO:tasks.workunit.client.1.smithi138.stdout:4/83: creat d7/df/f1a x:0 0 0 2024-05-27T09:29:27.956 INFO:tasks.workunit.client.1.smithi138.stdout:4/84: unlink d7/f9 0 2024-05-27T09:29:27.959 INFO:tasks.workunit.client.0.smithi119.stdout:0/575: dwrite d2/d6/d9/d12/f47 [0,4194304] 0 2024-05-27T09:29:27.959 INFO:tasks.workunit.client.0.smithi119.stdout:0/576: stat d2/d6/d9/d24/d30 0 2024-05-27T09:29:27.960 INFO:tasks.workunit.client.1.smithi138.stdout:0/123: dwrite da/f20 [0,4194304] 0 2024-05-27T09:29:27.960 INFO:tasks.workunit.client.1.smithi138.stdout:0/124: creat da/d17/f31 x:0 0 0 2024-05-27T09:29:27.960 INFO:tasks.workunit.client.1.smithi138.stdout:0/125: truncate da/f29 219459 0 2024-05-27T09:29:27.960 INFO:tasks.workunit.client.0.smithi119.stdout:3/744: dwrite db/fbf [4194304,4194304] 0 2024-05-27T09:29:27.960 INFO:tasks.workunit.client.0.smithi119.stdout:3/745: readlink db/l1c 0 2024-05-27T09:29:27.962 INFO:tasks.workunit.client.1.smithi138.stdout:4/85: mkdir d7/df/d1b 0 2024-05-27T09:29:27.963 INFO:tasks.workunit.client.0.smithi119.stdout:8/595: dwrite d3/de/f1d [0,4194304] 0 2024-05-27T09:29:27.963 INFO:tasks.workunit.client.0.smithi119.stdout:8/596: write d3/d8/d1b/d46/fa9 [424458,117717] 0 2024-05-27T09:29:27.966 INFO:tasks.workunit.client.1.smithi138.stdout:0/126: rename da/d17/d1d/d1f/l25 to da/d17/l32 0 2024-05-27T09:29:27.966 INFO:tasks.workunit.client.1.smithi138.stdout:8/81: dread d3/f5 [0,4194304] 0 2024-05-27T09:29:27.968 INFO:tasks.workunit.client.0.smithi119.stdout:8/597: mknod d3/d8/d4d/d58/cc1 0 2024-05-27T09:29:27.969 INFO:tasks.workunit.client.1.smithi138.stdout:4/86: rmdir d7/df/d1b 0 2024-05-27T09:29:27.970 INFO:tasks.workunit.client.1.smithi138.stdout:4/87: write d7/df/f11 [339562,64473] 0 2024-05-27T09:29:27.970 INFO:tasks.workunit.client.1.smithi138.stdout:4/88: chown d7/df/l14 0 1 2024-05-27T09:29:27.970 INFO:tasks.workunit.client.1.smithi138.stdout:4/89: write d7/fb [286715,31736] 0 2024-05-27T09:29:27.978 INFO:tasks.workunit.client.1.smithi138.stdout:0/127: rmdir da/d17/d1d 39 2024-05-27T09:29:27.978 INFO:tasks.workunit.client.1.smithi138.stdout:0/128: fdatasync da/f11 0 2024-05-27T09:29:27.981 INFO:tasks.workunit.client.1.smithi138.stdout:7/60: dread d0/fb [0,4194304] 0 2024-05-27T09:29:27.985 INFO:tasks.workunit.client.1.smithi138.stdout:7/61: symlink d0/d7/l14 0 2024-05-27T09:29:27.985 INFO:tasks.workunit.client.1.smithi138.stdout:7/62: fsync d0/f10 0 2024-05-27T09:29:27.985 INFO:tasks.workunit.client.1.smithi138.stdout:2/74: dread d1/d13/d17/f11 [0,4194304] 0 2024-05-27T09:29:27.988 INFO:tasks.workunit.client.0.smithi119.stdout:6/605: dwrite d0/d2/fac [0,4194304] 0 2024-05-27T09:29:27.988 INFO:tasks.workunit.client.0.smithi119.stdout:6/606: chown d0/d85/c87 82997809 1 2024-05-27T09:29:27.988 INFO:tasks.workunit.client.0.smithi119.stdout:6/607: creat d0/d2/d1c/d21/d58/fd4 x:0 0 0 2024-05-27T09:29:27.991 INFO:tasks.workunit.client.0.smithi119.stdout:6/608: link d0/d2/d57/d37/d74/d65/lb7 d0/d2/d1c/d21/ld5 0 2024-05-27T09:29:27.992 INFO:tasks.workunit.client.1.smithi138.stdout:7/63: rename d0/d7/cd to d0/c15 0 2024-05-27T09:29:27.992 INFO:tasks.workunit.client.1.smithi138.stdout:7/64: truncate d0/d7/f12 1039893 0 2024-05-27T09:29:27.993 INFO:tasks.workunit.client.0.smithi119.stdout:6/609: mknod d0/d2/cd6 0 2024-05-27T09:29:27.994 INFO:tasks.workunit.client.0.smithi119.stdout:6/610: stat d0/d2/d57/f90 0 2024-05-27T09:29:27.994 INFO:tasks.workunit.client.0.smithi119.stdout:5/653: sync 2024-05-27T09:29:27.994 INFO:tasks.workunit.client.0.smithi119.stdout:5/654: write d6/dd/d39/d19/d3c/d52/fbd [958236,42855] 0 2024-05-27T09:29:27.995 INFO:tasks.workunit.client.0.smithi119.stdout:6/611: symlink d0/d2/d1c/d21/d58/ld7 0 2024-05-27T09:29:27.996 INFO:tasks.workunit.client.0.smithi119.stdout:6/612: creat d0/d2/d1c/d21/d58/fd8 x:0 0 0 2024-05-27T09:29:27.996 INFO:tasks.workunit.client.0.smithi119.stdout:6/613: dread - d0/d11/f7d zero size 2024-05-27T09:29:27.996 INFO:tasks.workunit.client.0.smithi119.stdout:6/614: dread - d0/d2/d1c/d21/d58/fd4 zero size 2024-05-27T09:29:27.996 INFO:tasks.workunit.client.0.smithi119.stdout:6/615: creat d0/d2/d1c/fd9 x:0 0 0 2024-05-27T09:29:27.996 INFO:tasks.workunit.client.0.smithi119.stdout:5/655: mknod d6/db/d9a/ce6 0 2024-05-27T09:29:27.996 INFO:tasks.workunit.client.0.smithi119.stdout:5/656: truncate d6/dd/d39/d74/fa1 421619 0 2024-05-27T09:29:27.997 INFO:tasks.workunit.client.1.smithi138.stdout:2/75: rename d1/d13/d17/fd to d1/f1b 0 2024-05-27T09:29:27.999 INFO:tasks.workunit.client.0.smithi119.stdout:6/616: creat d0/d2/d57/d37/dad/fda x:0 0 0 2024-05-27T09:29:28.002 INFO:tasks.workunit.client.0.smithi119.stdout:6/617: creat d0/d11/fdb x:0 0 0 2024-05-27T09:29:28.002 INFO:tasks.workunit.client.0.smithi119.stdout:6/618: chown d0/d2/d57/d37/d74/d65/fc7 310 1 2024-05-27T09:29:28.009 INFO:tasks.workunit.client.0.smithi119.stdout:2/619: dwrite d2/da/d8a/dd4/fb5 [0,4194304] 0 2024-05-27T09:29:28.009 INFO:tasks.workunit.client.1.smithi138.stdout:6/133: dwrite d3/d14/f2a [0,4194304] 0 2024-05-27T09:29:28.012 INFO:tasks.workunit.client.0.smithi119.stdout:0/577: write d2/d1a/f51 [4212968,113721] 0 2024-05-27T09:29:28.015 INFO:tasks.workunit.client.0.smithi119.stdout:2/620: mknod d2/da/d8a/dd4/d89/d94/d81/cd6 0 2024-05-27T09:29:28.017 INFO:tasks.workunit.client.0.smithi119.stdout:0/578: mknod d2/d6/d9/dab/cb9 0 2024-05-27T09:29:28.021 INFO:tasks.workunit.client.0.smithi119.stdout:2/621: creat d2/da/d15/fd7 x:0 0 0 2024-05-27T09:29:28.021 INFO:tasks.workunit.client.0.smithi119.stdout:2/622: fsync d2/da/f37 0 2024-05-27T09:29:28.021 INFO:tasks.workunit.client.0.smithi119.stdout:2/623: fsync d2/da/d15/f80 0 2024-05-27T09:29:28.021 INFO:tasks.workunit.client.0.smithi119.stdout:2/624: write d2/da/d8a/dd4/d89/d94/d81/fa2 [1021224,75740] 0 2024-05-27T09:29:28.023 INFO:tasks.workunit.client.1.smithi138.stdout:9/56: dwrite d6/f8 [0,4194304] 0 2024-05-27T09:29:28.025 INFO:tasks.workunit.client.1.smithi138.stdout:9/57: dread - d6/db/f10 zero size 2024-05-27T09:29:28.032 INFO:tasks.workunit.client.0.smithi119.stdout:5/657: dread d6/f1d [0,4194304] 0 2024-05-27T09:29:28.032 INFO:tasks.workunit.client.0.smithi119.stdout:5/658: creat d6/dd/d46/fe7 x:0 0 0 2024-05-27T09:29:28.036 INFO:tasks.workunit.client.0.smithi119.stdout:5/659: symlink d6/dd/d39/d19/d70/le8 0 2024-05-27T09:29:28.036 INFO:tasks.workunit.client.0.smithi119.stdout:5/660: read - d6/db/dc/fde zero size 2024-05-27T09:29:28.040 INFO:tasks.workunit.client.0.smithi119.stdout:5/661: symlink d6/dd/d46/d47/d4b/dd8/le9 0 2024-05-27T09:29:28.041 INFO:tasks.workunit.client.0.smithi119.stdout:5/662: fsync d6/dd/d39/d19/d70/dc1/f5c 0 2024-05-27T09:29:28.049 INFO:tasks.workunit.client.1.smithi138.stdout:4/90: dwrite d7/f18 [0,4194304] 0 2024-05-27T09:29:28.052 INFO:tasks.workunit.client.0.smithi119.stdout:3/746: dwrite db/d36/d45/dab/fea [0,4194304] 0 2024-05-27T09:29:28.053 INFO:tasks.workunit.client.0.smithi119.stdout:6/619: dwrite d0/d2/d1c/f2d [0,4194304] 0 2024-05-27T09:29:28.053 INFO:tasks.workunit.client.0.smithi119.stdout:6/620: write d0/d2/d57/d37/dad/fda [101708,21387] 0 2024-05-27T09:29:28.056 INFO:tasks.workunit.client.0.smithi119.stdout:3/747: rename db/fa0 to db/d36/d45/ff0 0 2024-05-27T09:29:28.056 INFO:tasks.workunit.client.0.smithi119.stdout:6/621: mknod d0/d19/cdc 0 2024-05-27T09:29:28.058 INFO:tasks.workunit.client.0.smithi119.stdout:3/748: creat db/d36/d45/d4c/dbe/ff1 x:0 0 0 2024-05-27T09:29:28.058 INFO:tasks.workunit.client.0.smithi119.stdout:6/622: creat d0/d2/d77/fdd x:0 0 0 2024-05-27T09:29:28.058 INFO:tasks.workunit.client.0.smithi119.stdout:6/623: chown d0/f18 229 1 2024-05-27T09:29:28.063 INFO:tasks.workunit.client.0.smithi119.stdout:3/749: getdents db/d36 0 2024-05-27T09:29:28.063 INFO:tasks.workunit.client.0.smithi119.stdout:1/690: sync 2024-05-27T09:29:28.063 INFO:tasks.workunit.client.0.smithi119.stdout:7/658: sync 2024-05-27T09:29:28.065 INFO:tasks.workunit.client.0.smithi119.stdout:7/659: write d7/df/d19/d1d/f23 [4435437,68381] 0 2024-05-27T09:29:28.065 INFO:tasks.workunit.client.0.smithi119.stdout:7/660: write d7/df/d19/d97/f57 [293615,97598] 0 2024-05-27T09:29:28.065 INFO:tasks.workunit.client.0.smithi119.stdout:7/661: creat d7/df/d19/d97/d92/db7/dba/d42/d64/fd4 x:0 0 0 2024-05-27T09:29:28.066 INFO:tasks.workunit.client.0.smithi119.stdout:7/662: fsync d7/d1a/f59 0 2024-05-27T09:29:28.066 INFO:tasks.workunit.client.0.smithi119.stdout:7/663: fsync d7/df/d19/d97/fcc 0 2024-05-27T09:29:28.066 INFO:tasks.workunit.client.0.smithi119.stdout:7/664: creat d7/df/d19/d97/d92/fd5 x:0 0 0 2024-05-27T09:29:28.066 INFO:tasks.workunit.client.0.smithi119.stdout:7/665: fsync d7/d1a/f4a 0 2024-05-27T09:29:28.066 INFO:tasks.workunit.client.0.smithi119.stdout:7/666: fsync d7/df/d19/f14 0 2024-05-27T09:29:28.066 INFO:tasks.workunit.client.0.smithi119.stdout:3/750: mknod db/dc9/daa/cf2 0 2024-05-27T09:29:28.066 INFO:tasks.workunit.client.0.smithi119.stdout:3/751: readlink db/d36/l4e 0 2024-05-27T09:29:28.067 INFO:tasks.workunit.client.1.smithi138.stdout:2/76: write d1/d10/f16 [77127,54081] 0 2024-05-27T09:29:28.068 INFO:tasks.workunit.client.0.smithi119.stdout:3/752: link db/l7d db/d5b/dce/lf3 0 2024-05-27T09:29:28.068 INFO:tasks.workunit.client.0.smithi119.stdout:3/753: getdents db/d36/d45/d4d/dc8 0 2024-05-27T09:29:28.069 INFO:tasks.workunit.client.1.smithi138.stdout:2/77: symlink d1/l1c 0 2024-05-27T09:29:28.069 INFO:tasks.workunit.client.1.smithi138.stdout:2/78: truncate d1/d10/f16 8822098 0 2024-05-27T09:29:28.069 INFO:tasks.workunit.client.1.smithi138.stdout:2/79: chown d1/d10/f1a 52366 1 2024-05-27T09:29:28.090 INFO:tasks.workunit.client.1.smithi138.stdout:3/60: dwrite d0/d2/fc [0,4194304] 0 2024-05-27T09:29:28.091 INFO:tasks.workunit.client.1.smithi138.stdout:5/104: dwrite d3/d12/d16/f1c [0,4194304] 0 2024-05-27T09:29:28.093 INFO:tasks.workunit.client.1.smithi138.stdout:3/61: mknod d0/d8/c14 0 2024-05-27T09:29:28.093 INFO:tasks.workunit.client.1.smithi138.stdout:3/62: write d0/d2/f13 [548783,74506] 0 2024-05-27T09:29:28.094 INFO:tasks.workunit.client.0.smithi119.stdout:9/506: dwrite d18/d19/d1b/f5d [8388608,4194304] 0 2024-05-27T09:29:28.094 INFO:tasks.workunit.client.1.smithi138.stdout:8/82: dwrite d3/d8/d11/f12 [0,4194304] 0 2024-05-27T09:29:28.095 INFO:tasks.workunit.client.1.smithi138.stdout:5/105: mkdir d3/d12/d16/d1d/d23 0 2024-05-27T09:29:28.096 INFO:tasks.workunit.client.0.smithi119.stdout:9/507: write d18/d19/d1b/d40/d44/f54 [5212036,68768] 0 2024-05-27T09:29:28.097 INFO:tasks.workunit.client.0.smithi119.stdout:2/625: dread d2/da/d4e/f4f [0,4194304] 0 2024-05-27T09:29:28.099 INFO:tasks.workunit.client.1.smithi138.stdout:0/129: dwrite da/d17/f18 [0,4194304] 0 2024-05-27T09:29:28.101 INFO:tasks.workunit.client.0.smithi119.stdout:4/643: dwrite d4/d23/f52 [4194304,4194304] 0 2024-05-27T09:29:28.101 INFO:tasks.workunit.client.0.smithi119.stdout:4/644: chown d4/d6/d3e/d89/db2/fbd 2 1 2024-05-27T09:29:28.101 INFO:tasks.workunit.client.0.smithi119.stdout:4/645: truncate d4/d1c/d39/f64 2599267 0 2024-05-27T09:29:28.101 INFO:tasks.workunit.client.0.smithi119.stdout:4/646: creat d4/d23/d31/fcc x:0 0 0 2024-05-27T09:29:28.101 INFO:tasks.workunit.client.0.smithi119.stdout:4/647: readlink d4/d6/l3c 0 2024-05-27T09:29:28.102 INFO:tasks.workunit.client.1.smithi138.stdout:3/63: symlink d0/d2/da/df/d10/l15 0 2024-05-27T09:29:28.102 INFO:tasks.workunit.client.1.smithi138.stdout:3/64: chown d0/d8/le 75784 1 2024-05-27T09:29:28.105 INFO:tasks.workunit.client.0.smithi119.stdout:4/648: write d4/d23/d31/fad [1487465,124072] 0 2024-05-27T09:29:28.107 INFO:tasks.workunit.client.1.smithi138.stdout:8/83: unlink d3/f5 0 2024-05-27T09:29:28.107 INFO:tasks.workunit.client.1.smithi138.stdout:7/65: dread d0/fb [0,4194304] 0 2024-05-27T09:29:28.108 INFO:tasks.workunit.client.1.smithi138.stdout:5/106: rename d3/fa to d3/d12/d16/d1d/f24 0 2024-05-27T09:29:28.115 INFO:tasks.workunit.client.0.smithi119.stdout:1/691: dread d7/f1e [0,4194304] 0 2024-05-27T09:29:28.115 INFO:tasks.workunit.client.0.smithi119.stdout:1/692: dread - d7/db/d8f/fc4 zero size 2024-05-27T09:29:28.115 INFO:tasks.workunit.client.0.smithi119.stdout:1/693: stat d7/db/d12/d26/d4a/fd6 0 2024-05-27T09:29:28.115 INFO:tasks.workunit.client.0.smithi119.stdout:1/694: chown d7/db/d12 666 1 2024-05-27T09:29:28.115 INFO:tasks.workunit.client.0.smithi119.stdout:5/663: dread d6/dd/d39/d19/d3c/d52/fbd [0,4194304] 0 2024-05-27T09:29:28.118 INFO:tasks.workunit.client.1.smithi138.stdout:0/130: link da/d17/f19 da/d2a/d2b/f33 0 2024-05-27T09:29:28.118 INFO:tasks.workunit.client.1.smithi138.stdout:0/131: fdatasync da/d17/f18 0 2024-05-27T09:29:28.125 INFO:tasks.workunit.client.0.smithi119.stdout:4/649: dread d4/d1c/d39/f64 [0,4194304] 0 2024-05-27T09:29:28.127 INFO:tasks.workunit.client.1.smithi138.stdout:5/107: rename d3/d12/d16/f1c to d3/d12/f25 0 2024-05-27T09:29:28.127 INFO:tasks.workunit.client.1.smithi138.stdout:5/108: truncate d3/fd 4824744 0 2024-05-27T09:29:28.129 INFO:tasks.workunit.client.1.smithi138.stdout:5/109: mkdir d3/d12/d26 0 2024-05-27T09:29:28.131 INFO:tasks.workunit.client.0.smithi119.stdout:4/650: truncate d4/d23/f26 2766161 0 2024-05-27T09:29:28.131 INFO:tasks.workunit.client.0.smithi119.stdout:4/651: write d4/d48/f5e [3463221,114869] 0 2024-05-27T09:29:28.132 INFO:tasks.workunit.client.0.smithi119.stdout:4/652: dread - d4/d6/d28/f69 zero size 2024-05-27T09:29:28.132 INFO:tasks.workunit.client.0.smithi119.stdout:0/579: dwrite d2/d6/d9/d12/f77 [0,4194304] 0 2024-05-27T09:29:28.132 INFO:tasks.workunit.client.0.smithi119.stdout:0/580: write d2/d6/d9/f6d [2902589,33965] 0 2024-05-27T09:29:28.136 INFO:tasks.workunit.client.1.smithi138.stdout:5/110: rename d3/c9 to d3/d12/d16/d1d/d22/c27 0 2024-05-27T09:29:28.136 INFO:tasks.workunit.client.1.smithi138.stdout:8/84: write d3/d8/d11/f12 [645025,70894] 0 2024-05-27T09:29:28.136 INFO:tasks.workunit.client.1.smithi138.stdout:5/111: symlink d3/d12/d16/d1d/d23/l28 0 2024-05-27T09:29:28.147 INFO:tasks.workunit.client.1.smithi138.stdout:0/132: dread da/f12 [0,4194304] 0 2024-05-27T09:29:28.156 INFO:tasks.workunit.client.1.smithi138.stdout:7/66: dwrite d0/d7/fe [0,4194304] 0 2024-05-27T09:29:28.157 INFO:tasks.workunit.client.1.smithi138.stdout:7/67: fdatasync d0/fb 0 2024-05-27T09:29:28.159 INFO:tasks.workunit.client.0.smithi119.stdout:4/653: rmdir d4/d23/d31 39 2024-05-27T09:29:28.160 INFO:tasks.workunit.client.0.smithi119.stdout:4/654: creat d4/d6/d28/d2f/db8/fcd x:0 0 0 2024-05-27T09:29:28.161 INFO:tasks.workunit.client.1.smithi138.stdout:7/68: unlink d0/fb 0 2024-05-27T09:29:28.161 INFO:tasks.workunit.client.1.smithi138.stdout:7/69: chown d0/d7/cf 64391 1 2024-05-27T09:29:28.166 INFO:tasks.workunit.client.0.smithi119.stdout:6/624: dwrite d0/d2/d1c/d21/f2b [0,4194304] 0 2024-05-27T09:29:28.166 INFO:tasks.workunit.client.0.smithi119.stdout:6/625: truncate d0/d2/fb1 710751 0 2024-05-27T09:29:28.167 INFO:tasks.workunit.client.0.smithi119.stdout:7/667: dwrite d7/df/d19/d97/d92/fd5 [0,4194304] 0 2024-05-27T09:29:28.173 INFO:tasks.workunit.client.0.smithi119.stdout:7/668: read d7/d5f/f7c [1231717,53453] 0 2024-05-27T09:29:28.173 INFO:tasks.workunit.client.0.smithi119.stdout:4/655: rename d4/de/d66/d87 to d4/d6/d81/dce 0 2024-05-27T09:29:28.174 INFO:tasks.workunit.client.0.smithi119.stdout:6/626: chown d0/d2/d57/d37/f5b 21 1 2024-05-27T09:29:28.176 INFO:tasks.workunit.client.0.smithi119.stdout:6/627: mknod d0/d2/d57/cde 0 2024-05-27T09:29:28.178 INFO:tasks.workunit.client.0.smithi119.stdout:4/656: write d4/d1c/d39/f57 [1046805,93404] 0 2024-05-27T09:29:28.182 INFO:tasks.workunit.client.1.smithi138.stdout:8/85: dread d3/d8/fc [0,4194304] 0 2024-05-27T09:29:28.182 INFO:tasks.workunit.client.1.smithi138.stdout:2/80: dwrite d1/f1b [0,4194304] 0 2024-05-27T09:29:28.182 INFO:tasks.workunit.client.1.smithi138.stdout:2/81: read - d1/d10/fe zero size 2024-05-27T09:29:28.182 INFO:tasks.workunit.client.1.smithi138.stdout:2/82: truncate d1/d10/f16 9289401 0 2024-05-27T09:29:28.189 INFO:tasks.workunit.client.1.smithi138.stdout:2/83: mkdir d1/d13/d1d 0 2024-05-27T09:29:28.192 INFO:tasks.workunit.client.1.smithi138.stdout:2/84: symlink d1/d15/l1e 0 2024-05-27T09:29:28.194 INFO:tasks.workunit.client.1.smithi138.stdout:2/85: write d1/d10/f6 [3848494,116228] 0 2024-05-27T09:29:28.195 INFO:tasks.workunit.client.1.smithi138.stdout:9/58: dwrite d6/db/f10 [0,4194304] 0 2024-05-27T09:29:28.198 INFO:tasks.workunit.client.1.smithi138.stdout:9/59: creat d6/db/dc/f12 x:0 0 0 2024-05-27T09:29:28.205 INFO:tasks.workunit.client.1.smithi138.stdout:2/86: dread d1/d13/d17/f11 [0,4194304] 0 2024-05-27T09:29:28.205 INFO:tasks.workunit.client.1.smithi138.stdout:2/87: write d1/d15/f19 [201382,108754] 0 2024-05-27T09:29:28.205 INFO:tasks.workunit.client.1.smithi138.stdout:2/88: read d1/d10/f16 [257894,69549] 0 2024-05-27T09:29:28.215 INFO:tasks.workunit.client.0.smithi119.stdout:9/508: dwrite f15 [0,4194304] 0 2024-05-27T09:29:28.215 INFO:tasks.workunit.client.0.smithi119.stdout:9/509: truncate d18/d19/d1b/f96 754719 0 2024-05-27T09:29:28.219 INFO:tasks.workunit.client.0.smithi119.stdout:3/754: dwrite db/dc9/d98/d7a/f84 [0,4194304] 0 2024-05-27T09:29:28.220 INFO:tasks.workunit.client.0.smithi119.stdout:3/755: write db/d36/d45/dab/fea [5198403,90289] 0 2024-05-27T09:29:28.222 INFO:tasks.workunit.client.0.smithi119.stdout:9/510: truncate d18/f23 1679972 0 2024-05-27T09:29:28.224 INFO:tasks.workunit.client.1.smithi138.stdout:9/60: dread f4 [0,4194304] 0 2024-05-27T09:29:28.224 INFO:tasks.workunit.client.1.smithi138.stdout:0/133: write da/d17/d1d/d1f/f21 [3032844,79151] 0 2024-05-27T09:29:28.226 INFO:tasks.workunit.client.0.smithi119.stdout:9/511: truncate d18/d19/d1b/d40/f60 591408 0 2024-05-27T09:29:28.227 INFO:tasks.workunit.client.1.smithi138.stdout:9/61: symlink d6/db/de/l13 0 2024-05-27T09:29:28.231 INFO:tasks.workunit.client.1.smithi138.stdout:0/134: rename da/l27 to da/d17/d1d/l34 0 2024-05-27T09:29:28.231 INFO:tasks.workunit.client.1.smithi138.stdout:0/135: write da/f10 [2312539,85210] 0 2024-05-27T09:29:28.234 INFO:tasks.workunit.client.1.smithi138.stdout:0/136: rename da/d17/f18 to da/d2a/f35 0 2024-05-27T09:29:28.234 INFO:tasks.workunit.client.1.smithi138.stdout:5/112: dwrite d3/fd [4194304,4194304] 0 2024-05-27T09:29:28.237 INFO:tasks.workunit.client.1.smithi138.stdout:3/65: dwrite d0/d2/f13 [0,4194304] 0 2024-05-27T09:29:28.237 INFO:tasks.workunit.client.1.smithi138.stdout:3/66: truncate d0/fd 698170 0 2024-05-27T09:29:28.237 INFO:tasks.workunit.client.1.smithi138.stdout:7/70: dread d0/f10 [0,4194304] 0 2024-05-27T09:29:28.240 INFO:tasks.workunit.client.0.smithi119.stdout:9/512: dread d18/d19/d61/f9d [0,4194304] 0 2024-05-27T09:29:28.249 INFO:tasks.workunit.client.1.smithi138.stdout:0/137: symlink da/d2a/d2b/l36 0 2024-05-27T09:29:28.249 INFO:tasks.workunit.client.1.smithi138.stdout:0/138: readlink da/d17/d1d/l2c 0 2024-05-27T09:29:28.254 INFO:tasks.workunit.client.1.smithi138.stdout:3/67: unlink d0/d2/da/f11 0 2024-05-27T09:29:28.255 INFO:tasks.workunit.client.1.smithi138.stdout:7/71: unlink d0/c6 0 2024-05-27T09:29:28.255 INFO:tasks.workunit.client.1.smithi138.stdout:7/72: stat d0/d7/lc 0 2024-05-27T09:29:28.261 INFO:tasks.workunit.client.1.smithi138.stdout:3/68: symlink d0/d2/da/l16 0 2024-05-27T09:29:28.271 INFO:tasks.workunit.client.1.smithi138.stdout:7/73: link d0/d7/fe d0/d7/f16 0 2024-05-27T09:29:28.279 INFO:tasks.workunit.client.1.smithi138.stdout:7/74: truncate d0/d7/f12 250403 0 2024-05-27T09:29:28.281 INFO:tasks.workunit.client.1.smithi138.stdout:6/134: sync 2024-05-27T09:29:28.281 INFO:tasks.workunit.client.1.smithi138.stdout:7/75: mknod d0/d7/c17 0 2024-05-27T09:29:28.282 INFO:tasks.workunit.client.1.smithi138.stdout:1/125: sync 2024-05-27T09:29:28.284 INFO:tasks.workunit.client.1.smithi138.stdout:0/139: dread da/f15 [0,4194304] 0 2024-05-27T09:29:28.284 INFO:tasks.workunit.client.1.smithi138.stdout:0/140: chown da/c13 2 1 2024-05-27T09:29:28.285 INFO:tasks.workunit.client.0.smithi119.stdout:3/756: dwrite db/dc9/d98/d7a/f84 [0,4194304] 0 2024-05-27T09:29:28.285 INFO:tasks.workunit.client.0.smithi119.stdout:3/757: write db/d36/d45/d4c/d62/f89 [198508,64202] 0 2024-05-27T09:29:28.286 INFO:tasks.workunit.client.1.smithi138.stdout:7/76: symlink d0/l18 0 2024-05-27T09:29:28.286 INFO:tasks.workunit.client.1.smithi138.stdout:7/77: stat d0/d7/l14 0 2024-05-27T09:29:28.286 INFO:tasks.workunit.client.1.smithi138.stdout:7/78: chown d0/d7/c17 669 1 2024-05-27T09:29:28.286 INFO:tasks.workunit.client.1.smithi138.stdout:7/79: chown d0/d7/f16 17 1 2024-05-27T09:29:28.287 INFO:tasks.workunit.client.1.smithi138.stdout:1/126: mkdir d4/d1a/d2e 0 2024-05-27T09:29:28.291 INFO:tasks.workunit.client.1.smithi138.stdout:7/80: creat d0/f19 x:0 0 0 2024-05-27T09:29:28.291 INFO:tasks.workunit.client.1.smithi138.stdout:0/141: symlink da/l37 0 2024-05-27T09:29:28.291 INFO:tasks.workunit.client.1.smithi138.stdout:0/142: readlink da/d17/d1d/l2c 0 2024-05-27T09:29:28.291 INFO:tasks.workunit.client.1.smithi138.stdout:0/143: write da/f11 [518120,41148] 0 2024-05-27T09:29:28.291 INFO:tasks.workunit.client.1.smithi138.stdout:0/144: fdatasync da/f22 0 2024-05-27T09:29:28.291 INFO:tasks.workunit.client.1.smithi138.stdout:0/145: readlink da/d17/d1d/l23 0 2024-05-27T09:29:28.298 INFO:tasks.workunit.client.1.smithi138.stdout:0/146: rename da/f12 to da/d17/d1d/d1f/f38 0 2024-05-27T09:29:28.298 INFO:tasks.workunit.client.1.smithi138.stdout:0/147: fsync da/d2a/f35 0 2024-05-27T09:29:28.300 INFO:tasks.workunit.client.1.smithi138.stdout:0/148: rename da/c1a to da/c39 0 2024-05-27T09:29:28.303 INFO:tasks.workunit.client.0.smithi119.stdout:2/626: dwrite d2/da/f1c [0,4194304] 0 2024-05-27T09:29:28.304 INFO:tasks.workunit.client.0.smithi119.stdout:2/627: chown d2/da/d8a/d8f/cc7 0 1 2024-05-27T09:29:28.304 INFO:tasks.workunit.client.0.smithi119.stdout:2/628: stat d2/da/d8a/dd4/d89/d94/fcc 0 2024-05-27T09:29:28.304 INFO:tasks.workunit.client.0.smithi119.stdout:2/629: readlink d2/da/d4e/la1 0 2024-05-27T09:29:28.304 INFO:tasks.workunit.client.1.smithi138.stdout:0/149: mknod da/c3a 0 2024-05-27T09:29:28.306 INFO:tasks.workunit.client.0.smithi119.stdout:1/695: dwrite d7/f3d [4194304,4194304] 0 2024-05-27T09:29:28.306 INFO:tasks.workunit.client.0.smithi119.stdout:0/581: dwrite d2/f49 [0,4194304] 0 2024-05-27T09:29:28.310 INFO:tasks.workunit.client.0.smithi119.stdout:3/758: dread db/d36/d45/fc1 [0,4194304] 0 2024-05-27T09:29:28.310 INFO:tasks.workunit.client.0.smithi119.stdout:3/759: fdatasync db/d36/d45/d93/fa4 0 2024-05-27T09:29:28.310 INFO:tasks.workunit.client.0.smithi119.stdout:3/760: write db/dc9/fd5 [684344,106553] 0 2024-05-27T09:29:28.310 INFO:tasks.workunit.client.0.smithi119.stdout:7/669: dwrite d7/d1a/d99/dcf/fd2 [0,4194304] 0 2024-05-27T09:29:28.311 INFO:tasks.workunit.client.1.smithi138.stdout:6/135: read d3/d14/f2a [430188,39875] 0 2024-05-27T09:29:28.313 INFO:tasks.workunit.client.0.smithi119.stdout:3/761: rename db/d36/d45/d4c/d62/d13/d33/l94 to db/dc9/d98/db9/dd1/lf4 0 2024-05-27T09:29:28.313 INFO:tasks.workunit.client.0.smithi119.stdout:3/762: truncate db/d36/d45/d4c/d62/d13/f97 2977823 0 2024-05-27T09:29:28.314 INFO:tasks.workunit.client.0.smithi119.stdout:3/763: fsync db/d5b/f63 0 2024-05-27T09:29:28.314 INFO:tasks.workunit.client.0.smithi119.stdout:7/670: creat d7/df/d19/d4d/d89/fd6 x:0 0 0 2024-05-27T09:29:28.315 INFO:tasks.workunit.client.1.smithi138.stdout:6/136: creat d3/d11/f2e x:0 0 0 2024-05-27T09:29:28.315 INFO:tasks.workunit.client.1.smithi138.stdout:6/137: stat d3/f17 0 2024-05-27T09:29:28.316 INFO:tasks.workunit.client.1.smithi138.stdout:1/127: dread d4/d8/f14 [0,4194304] 0 2024-05-27T09:29:28.317 INFO:tasks.workunit.client.1.smithi138.stdout:6/138: write d3/f6 [4636295,31845] 0 2024-05-27T09:29:28.318 INFO:tasks.workunit.client.1.smithi138.stdout:1/128: rename d4/fb to d4/da/f2f 0 2024-05-27T09:29:28.320 INFO:tasks.workunit.client.1.smithi138.stdout:6/139: rename d3/c8 to d3/d19/c2f 0 2024-05-27T09:29:28.320 INFO:tasks.workunit.client.1.smithi138.stdout:6/140: fdatasync d3/d19/f1e 0 2024-05-27T09:29:28.320 INFO:tasks.workunit.client.1.smithi138.stdout:6/141: chown d3/d11/fe 266997900 1 2024-05-27T09:29:28.320 INFO:tasks.workunit.client.1.smithi138.stdout:6/142: write d3/d11/f2e [666174,80288] 0 2024-05-27T09:29:28.321 INFO:tasks.workunit.client.1.smithi138.stdout:6/143: chown d3/d11/d2d 285 1 2024-05-27T09:29:28.326 INFO:tasks.workunit.client.0.smithi119.stdout:3/764: symlink db/d36/d45/lf5 0 2024-05-27T09:29:28.327 INFO:tasks.workunit.client.0.smithi119.stdout:3/765: chown db/d36/d45/d4c/d62/d13/d33/l34 1991551 1 2024-05-27T09:29:28.328 INFO:tasks.workunit.client.1.smithi138.stdout:1/129: symlink d4/d1a/d2d/l30 0 2024-05-27T09:29:28.328 INFO:tasks.workunit.client.1.smithi138.stdout:1/130: chown d4/c21 18 1 2024-05-27T09:29:28.328 INFO:tasks.workunit.client.0.smithi119.stdout:7/671: symlink d7/d5f/d79/ld7 0 2024-05-27T09:29:28.329 INFO:tasks.workunit.client.1.smithi138.stdout:7/81: dread d0/d7/f16 [0,4194304] 0 2024-05-27T09:29:28.329 INFO:tasks.workunit.client.1.smithi138.stdout:6/144: mknod d3/d14/c30 0 2024-05-27T09:29:28.329 INFO:tasks.workunit.client.1.smithi138.stdout:9/62: dwrite d6/f8 [0,4194304] 0 2024-05-27T09:29:28.330 INFO:tasks.workunit.client.1.smithi138.stdout:9/63: creat d6/db/f14 x:0 0 0 2024-05-27T09:29:28.330 INFO:tasks.workunit.client.0.smithi119.stdout:3/766: mknod db/d36/d45/d4c/d62/cf6 0 2024-05-27T09:29:28.331 INFO:tasks.workunit.client.1.smithi138.stdout:1/131: unlink d4/f5 0 2024-05-27T09:29:28.333 INFO:tasks.workunit.client.1.smithi138.stdout:1/132: read d4/d8/f1c [280866,25235] 0 2024-05-27T09:29:28.333 INFO:tasks.workunit.client.1.smithi138.stdout:1/133: fsync d4/d1a/f27 0 2024-05-27T09:29:28.334 INFO:tasks.workunit.client.1.smithi138.stdout:9/64: mknod d6/db/c15 0 2024-05-27T09:29:28.334 INFO:tasks.workunit.client.1.smithi138.stdout:9/65: write f2 [1926279,74867] 0 2024-05-27T09:29:28.334 INFO:tasks.workunit.client.1.smithi138.stdout:9/66: write d6/f8 [4413506,77748] 0 2024-05-27T09:29:28.334 INFO:tasks.workunit.client.1.smithi138.stdout:9/67: chown c5 7 1 2024-05-27T09:29:28.335 INFO:tasks.workunit.client.1.smithi138.stdout:1/134: symlink d4/d8/l31 0 2024-05-27T09:29:28.336 INFO:tasks.workunit.client.0.smithi119.stdout:4/657: dwrite d4/d23/d24/f82 [0,4194304] 0 2024-05-27T09:29:28.338 INFO:tasks.workunit.client.1.smithi138.stdout:8/86: dwrite d3/d8/fc [0,4194304] 0 2024-05-27T09:29:28.340 INFO:tasks.workunit.client.0.smithi119.stdout:0/582: read d2/d6/d9/d24/d30/d6b/f70 [190224,61410] 0 2024-05-27T09:29:28.340 INFO:tasks.workunit.client.0.smithi119.stdout:0/583: fsync d2/d6/d9/f3a 0 2024-05-27T09:29:28.340 INFO:tasks.workunit.client.0.smithi119.stdout:0/584: read - d2/d6/d23/fa9 zero size 2024-05-27T09:29:28.341 INFO:tasks.workunit.client.0.smithi119.stdout:4/658: rmdir d4/d48 39 2024-05-27T09:29:28.343 INFO:tasks.workunit.client.0.smithi119.stdout:9/513: dwrite d18/d19/f85 [0,4194304] 0 2024-05-27T09:29:28.343 INFO:tasks.workunit.client.0.smithi119.stdout:9/514: write d18/d19/f3f [971492,118807] 0 2024-05-27T09:29:28.344 INFO:tasks.workunit.client.0.smithi119.stdout:6/628: dwrite d0/d2/d57/d37/f8d [0,4194304] 0 2024-05-27T09:29:28.344 INFO:tasks.workunit.client.0.smithi119.stdout:6/629: write d0/d2/d1c/d21/f3b [1773514,64095] 0 2024-05-27T09:29:28.345 INFO:tasks.workunit.client.0.smithi119.stdout:0/585: truncate d2/d6/d9/d12/f79 435000 0 2024-05-27T09:29:28.357 INFO:tasks.workunit.client.1.smithi138.stdout:7/82: read d0/d7/f16 [3879082,88061] 0 2024-05-27T09:29:28.357 INFO:tasks.workunit.client.1.smithi138.stdout:7/83: creat d0/f1a x:0 0 0 2024-05-27T09:29:28.357 INFO:tasks.workunit.client.1.smithi138.stdout:7/84: creat d0/d7/f1b x:0 0 0 2024-05-27T09:29:28.357 INFO:tasks.workunit.client.1.smithi138.stdout:7/85: write d0/d7/f12 [1286675,19354] 0 2024-05-27T09:29:28.357 INFO:tasks.workunit.client.1.smithi138.stdout:0/150: getdents da 0 2024-05-27T09:29:28.357 INFO:tasks.workunit.client.1.smithi138.stdout:5/113: dwrite f0 [4194304,4194304] 0 2024-05-27T09:29:28.358 INFO:tasks.workunit.client.0.smithi119.stdout:9/515: mkdir d18/d19/d1b/d2a/da2 0 2024-05-27T09:29:28.358 INFO:tasks.workunit.client.1.smithi138.stdout:2/89: dwrite d1/d10/f1a [0,4194304] 0 2024-05-27T09:29:28.361 INFO:tasks.workunit.client.0.smithi119.stdout:6/630: write d0/d2/d57/d37/d74/fa1 [837502,78835] 0 2024-05-27T09:29:28.361 INFO:tasks.workunit.client.0.smithi119.stdout:6/631: write d0/d85/fae [275107,52699] 0 2024-05-27T09:29:28.361 INFO:tasks.workunit.client.0.smithi119.stdout:0/586: symlink d2/d6/d9/d12/d4b/d96/lba 0 2024-05-27T09:29:28.362 INFO:tasks.workunit.client.0.smithi119.stdout:6/632: chown d0/dbb 270791 1 2024-05-27T09:29:28.366 INFO:tasks.workunit.client.0.smithi119.stdout:6/633: mknod d0/cdf 0 2024-05-27T09:29:28.366 INFO:tasks.workunit.client.0.smithi119.stdout:9/516: dread d18/d19/f74 [0,4194304] 0 2024-05-27T09:29:28.366 INFO:tasks.workunit.client.0.smithi119.stdout:9/517: fsync d18/d19/d1b/d29/f57 0 2024-05-27T09:29:28.368 INFO:tasks.workunit.client.0.smithi119.stdout:0/587: creat d2/d6/d9/fbb x:0 0 0 2024-05-27T09:29:28.371 INFO:tasks.workunit.client.0.smithi119.stdout:6/634: symlink d0/d2/d4e/d5a/le0 0 2024-05-27T09:29:28.371 INFO:tasks.workunit.client.0.smithi119.stdout:6/635: chown d0/d2/d57 55 1 2024-05-27T09:29:28.371 INFO:tasks.workunit.client.0.smithi119.stdout:6/636: stat d0/d19/f55 0 2024-05-27T09:29:28.373 INFO:tasks.workunit.client.0.smithi119.stdout:9/518: mknod d18/d19/d61/ca3 0 2024-05-27T09:29:28.373 INFO:tasks.workunit.client.1.smithi138.stdout:1/135: dread d4/da/f2f [0,4194304] 0 2024-05-27T09:29:28.374 INFO:tasks.workunit.client.0.smithi119.stdout:9/519: chown d18/d19/d1b/d2a/f5f 0 1 2024-05-27T09:29:28.376 INFO:tasks.workunit.client.1.smithi138.stdout:7/86: mkdir d0/d7/d1c 0 2024-05-27T09:29:28.377 INFO:tasks.workunit.client.0.smithi119.stdout:0/588: link d2/d1a/l7e d2/d6/d9/lbc 0 2024-05-27T09:29:28.379 INFO:tasks.workunit.client.1.smithi138.stdout:5/114: symlink d3/d12/l29 0 2024-05-27T09:29:28.383 INFO:tasks.workunit.client.1.smithi138.stdout:7/87: rmdir d0/d7/d1c 0 2024-05-27T09:29:28.383 INFO:tasks.workunit.client.1.smithi138.stdout:7/88: chown d0 388772597 1 2024-05-27T09:29:28.383 INFO:tasks.workunit.client.1.smithi138.stdout:7/89: creat d0/f1d x:0 0 0 2024-05-27T09:29:28.383 INFO:tasks.workunit.client.1.smithi138.stdout:7/90: rename d0 to d0/d7/d1e 22 2024-05-27T09:29:28.383 INFO:tasks.workunit.client.0.smithi119.stdout:6/637: mkdir d0/d2/de1 0 2024-05-27T09:29:28.384 INFO:tasks.workunit.client.1.smithi138.stdout:6/145: dwrite f2 [4194304,4194304] 0 2024-05-27T09:29:28.385 INFO:tasks.workunit.client.1.smithi138.stdout:6/146: creat d3/f31 x:0 0 0 2024-05-27T09:29:28.385 INFO:tasks.workunit.client.1.smithi138.stdout:5/115: creat d3/d12/d16/d1d/d22/f2a x:0 0 0 2024-05-27T09:29:28.389 INFO:tasks.workunit.client.1.smithi138.stdout:7/91: symlink d0/d7/l1f 0 2024-05-27T09:29:28.391 INFO:tasks.workunit.client.0.smithi119.stdout:0/589: getdents d2/d6/d9/d12/d4f 0 2024-05-27T09:29:28.392 INFO:tasks.workunit.client.1.smithi138.stdout:5/116: mkdir d3/d12/d16/d1d/d22/d2b 0 2024-05-27T09:29:28.392 INFO:tasks.workunit.client.1.smithi138.stdout:5/117: chown d3/d12 220444 1 2024-05-27T09:29:28.392 INFO:tasks.workunit.client.1.smithi138.stdout:5/118: chown d3/d12/d16/d1d/f1e 32691 1 2024-05-27T09:29:28.393 INFO:tasks.workunit.client.1.smithi138.stdout:5/119: write d3/d12/d16/d1d/f1e [243405,76680] 0 2024-05-27T09:29:28.395 INFO:tasks.workunit.client.0.smithi119.stdout:9/520: dread d18/d19/d1b/d29/f2f [0,4194304] 0 2024-05-27T09:29:28.396 INFO:tasks.workunit.client.0.smithi119.stdout:6/638: rename d0/d2/d1c/d21/f2b to d0/d2/de1/fe2 0 2024-05-27T09:29:28.396 INFO:tasks.workunit.client.0.smithi119.stdout:6/639: readlink d0/d2/l1a 0 2024-05-27T09:29:28.398 INFO:tasks.workunit.client.1.smithi138.stdout:6/147: mkdir d3/d14/d1c/d32 0 2024-05-27T09:29:28.398 INFO:tasks.workunit.client.1.smithi138.stdout:6/148: chown d3/d14/c30 33854 1 2024-05-27T09:29:28.398 INFO:tasks.workunit.client.1.smithi138.stdout:6/149: fdatasync d3/f17 0 2024-05-27T09:29:28.399 INFO:tasks.workunit.client.0.smithi119.stdout:3/767: write db/d36/d45/d4c/f87 [583632,65581] 0 2024-05-27T09:29:28.399 INFO:tasks.workunit.client.0.smithi119.stdout:6/640: mknod d0/d19/db2/ce3 0 2024-05-27T09:29:28.399 INFO:tasks.workunit.client.0.smithi119.stdout:6/641: chown d0/d2/d1c/fd9 31023325 1 2024-05-27T09:29:28.400 INFO:tasks.workunit.client.0.smithi119.stdout:1/696: dwrite d7/dd/f77 [0,4194304] 0 2024-05-27T09:29:28.400 INFO:tasks.workunit.client.0.smithi119.stdout:1/697: chown d7/dd/d44/d47 216335024 1 2024-05-27T09:29:28.400 INFO:tasks.workunit.client.0.smithi119.stdout:1/698: stat d7/l46 0 2024-05-27T09:29:28.400 INFO:tasks.workunit.client.0.smithi119.stdout:1/699: fdatasync d7/db/d8f/fc4 0 2024-05-27T09:29:28.402 INFO:tasks.workunit.client.0.smithi119.stdout:3/768: rename db/dc9/d98/d7a/fac to db/d7f/da9/ff7 0 2024-05-27T09:29:28.404 INFO:tasks.workunit.client.0.smithi119.stdout:6/642: mknod d0/d2/d57/ce4 0 2024-05-27T09:29:28.404 INFO:tasks.workunit.client.0.smithi119.stdout:6/643: write d0/d2/d4e/d5a/faa [237564,42365] 0 2024-05-27T09:29:28.404 INFO:tasks.workunit.client.0.smithi119.stdout:6/644: fdatasync d0/d2/d57/d37/d74/d65/fc7 0 2024-05-27T09:29:28.405 INFO:tasks.workunit.client.1.smithi138.stdout:0/151: dwrite da/d2a/f35 [0,4194304] 0 2024-05-27T09:29:28.405 INFO:tasks.workunit.client.1.smithi138.stdout:0/152: fsync da/d17/d1d/d1f/f21 0 2024-05-27T09:29:28.405 INFO:tasks.workunit.client.1.smithi138.stdout:5/120: mknod d3/d12/c2c 0 2024-05-27T09:29:28.405 INFO:tasks.workunit.client.0.smithi119.stdout:6/645: readlink d0/d2/d1c/d21/d58/ld7 0 2024-05-27T09:29:28.406 INFO:tasks.workunit.client.0.smithi119.stdout:2/630: dwrite d2/da/d8a/dd4/d89/d94/fcc [0,4194304] 0 2024-05-27T09:29:28.408 INFO:tasks.workunit.client.0.smithi119.stdout:1/700: truncate d7/db/d12/d26/d39/fb1 1270261 0 2024-05-27T09:29:28.408 INFO:tasks.workunit.client.0.smithi119.stdout:1/701: getdents d7/db 0 2024-05-27T09:29:28.411 INFO:tasks.workunit.client.1.smithi138.stdout:1/136: write d4/f18 [2537198,73506] 0 2024-05-27T09:29:28.411 INFO:tasks.workunit.client.1.smithi138.stdout:8/87: dwrite d3/ff [4194304,4194304] 0 2024-05-27T09:29:28.412 INFO:tasks.workunit.client.1.smithi138.stdout:5/121: symlink d3/l2d 0 2024-05-27T09:29:28.417 INFO:tasks.workunit.client.1.smithi138.stdout:1/137: mkdir d4/d1a/d2e/d32 0 2024-05-27T09:29:28.418 INFO:tasks.workunit.client.1.smithi138.stdout:8/88: write d3/d8/d11/f12 [476945,37748] 0 2024-05-27T09:29:28.418 INFO:tasks.workunit.client.1.smithi138.stdout:8/89: write d3/d8/fc [4343011,60777] 0 2024-05-27T09:29:28.419 INFO:tasks.workunit.client.1.smithi138.stdout:1/138: unlink d4/d1a/c26 0 2024-05-27T09:29:28.419 INFO:tasks.workunit.client.1.smithi138.stdout:6/150: write d3/d11/fe [2235956,2018] 0 2024-05-27T09:29:28.421 INFO:tasks.workunit.client.0.smithi119.stdout:2/631: rmdir d2/da/d15 39 2024-05-27T09:29:28.421 INFO:tasks.workunit.client.1.smithi138.stdout:8/90: readlink d3/l6 0 2024-05-27T09:29:28.421 INFO:tasks.workunit.client.1.smithi138.stdout:8/91: write f1 [2062107,38499] 0 2024-05-27T09:29:28.422 INFO:tasks.workunit.client.1.smithi138.stdout:8/92: chown d3/d8/ca 389700 1 2024-05-27T09:29:28.424 INFO:tasks.workunit.client.0.smithi119.stdout:2/632: mkdir d2/da/d15/dd8 0 2024-05-27T09:29:28.426 INFO:tasks.workunit.client.0.smithi119.stdout:2/633: link d2/da/d8a/dd4/d89/d94/fcc d2/da/d8a/dd4/d89/d94/fd9 0 2024-05-27T09:29:28.427 INFO:tasks.workunit.client.1.smithi138.stdout:1/139: truncate d4/d1a/f27 3482984 0 2024-05-27T09:29:28.429 INFO:tasks.workunit.client.1.smithi138.stdout:4/91: sync 2024-05-27T09:29:28.429 INFO:tasks.workunit.client.1.smithi138.stdout:6/151: mkdir d3/d11/d33 0 2024-05-27T09:29:28.430 INFO:tasks.workunit.client.1.smithi138.stdout:4/92: readlink l6 0 2024-05-27T09:29:28.430 INFO:tasks.workunit.client.0.smithi119.stdout:2/634: symlink d2/da/d2c/lda 0 2024-05-27T09:29:28.438 INFO:tasks.workunit.client.0.smithi119.stdout:9/521: write f2 [1846622,32609] 0 2024-05-27T09:29:28.438 INFO:tasks.workunit.client.0.smithi119.stdout:4/659: dwrite d4/de/d21/d40/f42 [0,4194304] 0 2024-05-27T09:29:28.438 INFO:tasks.workunit.client.1.smithi138.stdout:9/68: dwrite d6/f8 [0,4194304] 0 2024-05-27T09:29:28.438 INFO:tasks.workunit.client.1.smithi138.stdout:9/69: chown f4 2203 1 2024-05-27T09:29:28.439 INFO:tasks.workunit.client.1.smithi138.stdout:9/70: stat d6/f9 0 2024-05-27T09:29:28.439 INFO:tasks.workunit.client.1.smithi138.stdout:8/93: unlink d3/ff 0 2024-05-27T09:29:28.454 INFO:tasks.workunit.client.1.smithi138.stdout:9/71: rename d6/db/c15 to d6/da/c16 0 2024-05-27T09:29:28.457 INFO:tasks.workunit.client.0.smithi119.stdout:8/598: sync 2024-05-27T09:29:28.457 INFO:tasks.workunit.client.0.smithi119.stdout:8/599: fsync d3/de/f68 0 2024-05-27T09:29:28.459 INFO:tasks.workunit.client.1.smithi138.stdout:8/94: unlink d3/l6 0 2024-05-27T09:29:28.460 INFO:tasks.workunit.client.1.smithi138.stdout:8/95: stat d3/d8 0 2024-05-27T09:29:28.460 INFO:tasks.workunit.client.0.smithi119.stdout:8/600: link d3/d6e/f75 d3/d6e/d8e/d9b/fc2 0 2024-05-27T09:29:28.462 INFO:tasks.workunit.client.1.smithi138.stdout:9/72: mknod d6/db/de/c17 0 2024-05-27T09:29:28.462 INFO:tasks.workunit.client.0.smithi119.stdout:8/601: mknod d3/d6e/d80/cc3 0 2024-05-27T09:29:28.463 INFO:tasks.workunit.client.0.smithi119.stdout:5/664: sync 2024-05-27T09:29:28.463 INFO:tasks.workunit.client.0.smithi119.stdout:8/602: rmdir d3/d6e/d80/d84/d90 0 2024-05-27T09:29:28.464 INFO:tasks.workunit.client.0.smithi119.stdout:5/665: mknod d6/dd/d46/cea 0 2024-05-27T09:29:28.465 INFO:tasks.workunit.client.1.smithi138.stdout:6/152: dread f2 [0,4194304] 0 2024-05-27T09:29:28.465 INFO:tasks.workunit.client.1.smithi138.stdout:6/153: creat d3/d19/f34 x:0 0 0 2024-05-27T09:29:28.466 INFO:tasks.workunit.client.1.smithi138.stdout:6/154: write d3/f5 [1697415,51249] 0 2024-05-27T09:29:28.469 INFO:tasks.workunit.client.0.smithi119.stdout:5/666: rmdir d6/dd/d39/d74 39 2024-05-27T09:29:28.469 INFO:tasks.workunit.client.0.smithi119.stdout:8/603: getdents d3/d8/d1b/d46 0 2024-05-27T09:29:28.483 INFO:tasks.workunit.client.1.smithi138.stdout:8/96: rmdir d3/d8 39 2024-05-27T09:29:28.484 INFO:tasks.workunit.client.1.smithi138.stdout:8/97: chown d3 2 1 2024-05-27T09:29:28.490 INFO:tasks.workunit.client.1.smithi138.stdout:9/73: dread d6/f8 [0,4194304] 0 2024-05-27T09:29:28.490 INFO:tasks.workunit.client.1.smithi138.stdout:9/74: fsync f1 0 2024-05-27T09:29:28.493 INFO:tasks.workunit.client.1.smithi138.stdout:9/75: truncate d6/db/f10 1568511 0 2024-05-27T09:29:28.493 INFO:tasks.workunit.client.1.smithi138.stdout:8/98: fsync d3/d8/d11/f12 0 2024-05-27T09:29:28.508 INFO:tasks.workunit.client.0.smithi119.stdout:4/660: dwrite d4/d1c/d39/f5d [0,4194304] 0 2024-05-27T09:29:28.516 INFO:tasks.workunit.client.0.smithi119.stdout:4/661: mkdir d4/d6/d3e/d89/dcf 0 2024-05-27T09:29:28.520 INFO:tasks.workunit.client.0.smithi119.stdout:4/662: creat d4/d6/d28/d2f/d36/dc8/fd0 x:0 0 0 2024-05-27T09:29:28.537 INFO:tasks.workunit.client.0.smithi119.stdout:3/769: dwrite db/d36/d45/d4d/f58 [0,4194304] 0 2024-05-27T09:29:28.540 INFO:tasks.workunit.client.1.smithi138.stdout:0/153: dwrite da/f29 [0,4194304] 0 2024-05-27T09:29:28.540 INFO:tasks.workunit.client.1.smithi138.stdout:0/154: readlink da/d17/d1d/l28 0 2024-05-27T09:29:28.540 INFO:tasks.workunit.client.1.smithi138.stdout:0/155: write da/d17/d1d/d1f/f21 [3379106,68669] 0 2024-05-27T09:29:28.540 INFO:tasks.workunit.client.1.smithi138.stdout:0/156: fsync da/f15 0 2024-05-27T09:29:28.542 INFO:tasks.workunit.client.0.smithi119.stdout:3/770: symlink db/d36/d45/d93/lf8 0 2024-05-27T09:29:28.542 INFO:tasks.workunit.client.0.smithi119.stdout:3/771: readlink db/l11 0 2024-05-27T09:29:28.547 INFO:tasks.workunit.client.1.smithi138.stdout:6/155: dread d3/d14/f29 [0,4194304] 0 2024-05-27T09:29:28.547 INFO:tasks.workunit.client.1.smithi138.stdout:5/122: dwrite d3/f15 [0,4194304] 0 2024-05-27T09:29:28.547 INFO:tasks.workunit.client.1.smithi138.stdout:5/123: write d3/f21 [364382,5257] 0 2024-05-27T09:29:28.547 INFO:tasks.workunit.client.1.smithi138.stdout:5/124: dread - d3/d12/f17 zero size 2024-05-27T09:29:28.549 INFO:tasks.workunit.client.0.smithi119.stdout:3/772: link db/d36/d45/d4c/d62/d13/d33/d9b/de3/f7b db/d36/d45/d4d/dc8/ff9 0 2024-05-27T09:29:28.551 INFO:tasks.workunit.client.0.smithi119.stdout:5/667: dwrite d6/dd/d39/d19/fd0 [0,4194304] 0 2024-05-27T09:29:28.552 INFO:tasks.workunit.client.0.smithi119.stdout:5/668: dread - d6/dd/d39/d74/f7a zero size 2024-05-27T09:29:28.552 INFO:tasks.workunit.client.0.smithi119.stdout:6/646: dwrite d0/d2/d1c/fb3 [0,4194304] 0 2024-05-27T09:29:28.552 INFO:tasks.workunit.client.0.smithi119.stdout:9/522: dwrite f16 [0,4194304] 0 2024-05-27T09:29:28.552 INFO:tasks.workunit.client.0.smithi119.stdout:2/635: dwrite d2/f19 [0,4194304] 0 2024-05-27T09:29:28.553 INFO:tasks.workunit.client.1.smithi138.stdout:1/140: dwrite d4/d8/f14 [0,4194304] 0 2024-05-27T09:29:28.554 INFO:tasks.workunit.client.1.smithi138.stdout:1/141: write d4/d8/d22/f2c [439550,44195] 0 2024-05-27T09:29:28.555 INFO:tasks.workunit.client.1.smithi138.stdout:4/93: dwrite f0 [0,4194304] 0 2024-05-27T09:29:28.565 INFO:tasks.workunit.client.0.smithi119.stdout:3/773: symlink db/d5b/dce/de5/lfa 0 2024-05-27T09:29:28.573 INFO:tasks.workunit.client.1.smithi138.stdout:1/142: symlink d4/d8/l33 0 2024-05-27T09:29:28.574 INFO:tasks.workunit.client.1.smithi138.stdout:1/143: chown d4/d1a/d2e 1014 1 2024-05-27T09:29:28.574 INFO:tasks.workunit.client.1.smithi138.stdout:1/144: stat d4/cf 0 2024-05-27T09:29:28.574 INFO:tasks.workunit.client.0.smithi119.stdout:9/523: symlink d18/d19/d1b/d29/la4 0 2024-05-27T09:29:28.578 INFO:tasks.workunit.client.0.smithi119.stdout:2/636: rmdir d2/da/d15/dd8 0 2024-05-27T09:29:28.578 INFO:tasks.workunit.client.0.smithi119.stdout:3/774: creat db/ffb x:0 0 0 2024-05-27T09:29:28.585 INFO:tasks.workunit.client.0.smithi119.stdout:9/524: rename d18/d19/f50 to d18/d19/d61/fa5 0 2024-05-27T09:29:28.592 INFO:tasks.workunit.client.0.smithi119.stdout:2/637: creat d2/da/d8a/dd4/fdb x:0 0 0 2024-05-27T09:29:28.601 INFO:tasks.workunit.client.0.smithi119.stdout:3/775: rename db/dc9/daa to db/d36/d45/d4c/d62/d13/d33/dfc 0 2024-05-27T09:29:28.601 INFO:tasks.workunit.client.0.smithi119.stdout:3/776: dread - db/dc9/fef zero size 2024-05-27T09:29:28.601 INFO:tasks.workunit.client.0.smithi119.stdout:2/638: creat d2/da/d8a/dd4/d89/d94/d81/fdc x:0 0 0 2024-05-27T09:29:28.601 INFO:tasks.workunit.client.0.smithi119.stdout:2/639: dread - d2/da/d8a/dd4/d69/db1/fce zero size 2024-05-27T09:29:28.606 INFO:tasks.workunit.client.0.smithi119.stdout:9/525: rename d18/d19/d61/d95/f99 to d18/fa6 0 2024-05-27T09:29:28.606 INFO:tasks.workunit.client.0.smithi119.stdout:9/526: fsync d18/d19/d61/f92 0 2024-05-27T09:29:28.606 INFO:tasks.workunit.client.0.smithi119.stdout:9/527: creat d18/d19/d1b/d29/fa7 x:0 0 0 2024-05-27T09:29:28.606 INFO:tasks.workunit.client.0.smithi119.stdout:9/528: creat d18/d19/d1b/d29/fa8 x:0 0 0 2024-05-27T09:29:28.618 INFO:tasks.workunit.client.0.smithi119.stdout:6/647: dread d0/d2/d1c/fa0 [0,4194304] 0 2024-05-27T09:29:28.618 INFO:tasks.workunit.client.0.smithi119.stdout:5/669: dwrite d6/db/dc/fd2 [0,4194304] 0 2024-05-27T09:29:28.618 INFO:tasks.workunit.client.0.smithi119.stdout:5/670: stat d6/db/de/c2b 0 2024-05-27T09:29:28.619 INFO:tasks.workunit.client.0.smithi119.stdout:2/640: dread d2/da/d8a/dd4/d89/d94/f56 [0,4194304] 0 2024-05-27T09:29:28.619 INFO:tasks.workunit.client.0.smithi119.stdout:2/641: truncate d2/da/d8a/dd4/d89/d94/d81/f9e 278619 0 2024-05-27T09:29:28.619 INFO:tasks.workunit.client.0.smithi119.stdout:4/663: dwrite d4/d1c/fc5 [0,4194304] 0 2024-05-27T09:29:28.619 INFO:tasks.workunit.client.0.smithi119.stdout:2/642: write d2/da/d8a/dd4/d89/fa3 [1019508,29784] 0 2024-05-27T09:29:28.620 INFO:tasks.workunit.client.0.smithi119.stdout:5/671: write d6/dd/d39/d19/d3c/d52/fbd [52477,35131] 0 2024-05-27T09:29:28.620 INFO:tasks.workunit.client.0.smithi119.stdout:5/672: write d6/d2f/d9d/fb1 [405486,60326] 0 2024-05-27T09:29:28.623 INFO:tasks.workunit.client.1.smithi138.stdout:6/156: dwrite d3/d19/f22 [0,4194304] 0 2024-05-27T09:29:28.624 INFO:tasks.workunit.client.1.smithi138.stdout:4/94: dread f3 [0,4194304] 0 2024-05-27T09:29:28.624 INFO:tasks.workunit.client.1.smithi138.stdout:6/157: read d3/d11/fe [412718,84970] 0 2024-05-27T09:29:28.628 INFO:tasks.workunit.client.0.smithi119.stdout:2/643: read - d2/fae zero size 2024-05-27T09:29:28.628 INFO:tasks.workunit.client.0.smithi119.stdout:4/664: getdents d4/de/d21/d40/d47 0 2024-05-27T09:29:28.631 INFO:tasks.workunit.client.0.smithi119.stdout:5/673: creat d6/dd/d39/d74/feb x:0 0 0 2024-05-27T09:29:28.631 INFO:tasks.workunit.client.0.smithi119.stdout:2/644: mknod d2/da/cdd 0 2024-05-27T09:29:28.632 INFO:tasks.workunit.client.0.smithi119.stdout:4/665: link d4/d1c/d39/f4b d4/d6/fd1 0 2024-05-27T09:29:28.633 INFO:tasks.workunit.client.0.smithi119.stdout:3/777: dread db/d36/d45/d4c/d62/f7c [0,4194304] 0 2024-05-27T09:29:28.633 INFO:tasks.workunit.client.0.smithi119.stdout:3/778: chown db/d36/d45/d4c/d62/f17 193 1 2024-05-27T09:29:28.634 INFO:tasks.workunit.client.0.smithi119.stdout:5/674: creat d6/d2f/fec x:0 0 0 2024-05-27T09:29:28.636 INFO:tasks.workunit.client.0.smithi119.stdout:4/666: mkdir d4/d6/d3e/dd2 0 2024-05-27T09:29:28.637 INFO:tasks.workunit.client.0.smithi119.stdout:6/648: rename d0/d2/d77 to d0/d2/d57/d37/de5 0 2024-05-27T09:29:28.637 INFO:tasks.workunit.client.0.smithi119.stdout:5/675: symlink d6/db/de/led 0 2024-05-27T09:29:28.637 INFO:tasks.workunit.client.0.smithi119.stdout:5/676: chown d6/dd/d39/d19/d7d 206397 1 2024-05-27T09:29:28.639 INFO:tasks.workunit.client.0.smithi119.stdout:6/649: unlink d0/d2/d1c/d21/l49 0 2024-05-27T09:29:28.639 INFO:tasks.workunit.client.0.smithi119.stdout:6/650: chown d0/cdf 5348 1 2024-05-27T09:29:28.639 INFO:tasks.workunit.client.0.smithi119.stdout:6/651: dread - d0/f9a zero size 2024-05-27T09:29:28.639 INFO:tasks.workunit.client.0.smithi119.stdout:5/677: symlink d6/db/d9a/lee 0 2024-05-27T09:29:28.641 INFO:tasks.workunit.client.0.smithi119.stdout:5/678: symlink d6/dd/d39/d19/d3c/d52/d9c/lef 0 2024-05-27T09:29:28.641 INFO:tasks.workunit.client.0.smithi119.stdout:6/652: rmdir d0/d2/d1c/d21/d58 39 2024-05-27T09:29:28.646 INFO:tasks.workunit.client.0.smithi119.stdout:5/679: mknod d6/dd/d39/d19/cf0 0 2024-05-27T09:29:28.653 INFO:tasks.workunit.client.0.smithi119.stdout:5/680: rename d6/d2f/fec to d6/dd/d6e/ff1 0 2024-05-27T09:29:28.655 INFO:tasks.workunit.client.0.smithi119.stdout:5/681: truncate d6/db/dc/f3b 2082401 0 2024-05-27T09:29:28.655 INFO:tasks.workunit.client.0.smithi119.stdout:5/682: stat d6/c7 0 2024-05-27T09:29:28.655 INFO:tasks.workunit.client.1.smithi138.stdout:6/158: dread d3/d14/f29 [0,4194304] 0 2024-05-27T09:29:28.656 INFO:tasks.workunit.client.1.smithi138.stdout:6/159: fsync d3/f6 0 2024-05-27T09:29:28.657 INFO:tasks.workunit.client.1.smithi138.stdout:6/160: mkdir d3/d35 0 2024-05-27T09:29:28.660 INFO:tasks.workunit.client.1.smithi138.stdout:6/161: mknod d3/d11/c36 0 2024-05-27T09:29:28.662 INFO:tasks.workunit.client.1.smithi138.stdout:0/157: dwrite da/d17/d1d/d1f/f21 [0,4194304] 0 2024-05-27T09:29:28.663 INFO:tasks.workunit.client.1.smithi138.stdout:6/162: mkdir d3/d11/d2d/d37 0 2024-05-27T09:29:28.664 INFO:tasks.workunit.client.1.smithi138.stdout:6/163: creat d3/d19/f38 x:0 0 0 2024-05-27T09:29:28.664 INFO:tasks.workunit.client.0.smithi119.stdout:7/672: sync 2024-05-27T09:29:28.664 INFO:tasks.workunit.client.0.smithi119.stdout:7/673: fsync d7/df/d19/d97/d92/db7/dba/f52 0 2024-05-27T09:29:28.669 INFO:tasks.workunit.client.1.smithi138.stdout:0/158: rename da/d17/f19 to da/d17/d1d/d1f/f3b 0 2024-05-27T09:29:28.669 INFO:tasks.workunit.client.1.smithi138.stdout:5/125: dwrite f0 [4194304,4194304] 0 2024-05-27T09:29:28.670 INFO:tasks.workunit.client.1.smithi138.stdout:5/126: dread - d3/d12/d16/d1d/d22/f2a zero size 2024-05-27T09:29:28.670 INFO:tasks.workunit.client.1.smithi138.stdout:5/127: dread - d3/d12/f17 zero size 2024-05-27T09:29:28.670 INFO:tasks.workunit.client.1.smithi138.stdout:6/164: read d3/d19/f28 [557,95098] 0 2024-05-27T09:29:28.676 INFO:tasks.workunit.client.1.smithi138.stdout:0/159: creat da/f3c x:0 0 0 2024-05-27T09:29:28.682 INFO:tasks.workunit.client.1.smithi138.stdout:6/165: creat d3/d11/d2d/f39 x:0 0 0 2024-05-27T09:29:28.682 INFO:tasks.workunit.client.1.smithi138.stdout:6/166: dread - d3/d19/f34 zero size 2024-05-27T09:29:28.686 INFO:tasks.workunit.client.1.smithi138.stdout:5/128: getdents d3/d12/d16/d1d 0 2024-05-27T09:29:28.686 INFO:tasks.workunit.client.1.smithi138.stdout:5/129: chown d3/d12/d26 29930 1 2024-05-27T09:29:28.688 INFO:tasks.workunit.client.0.smithi119.stdout:4/667: dread d4/d48/f5e [0,4194304] 0 2024-05-27T09:29:28.688 INFO:tasks.workunit.client.0.smithi119.stdout:2/645: dread d2/da/f1c [0,4194304] 0 2024-05-27T09:29:28.689 INFO:tasks.workunit.client.0.smithi119.stdout:7/674: dread d7/d1a/f2d [0,4194304] 0 2024-05-27T09:29:28.689 INFO:tasks.workunit.client.0.smithi119.stdout:7/675: getdents d7/df/d19/d97/d81 0 2024-05-27T09:29:28.690 INFO:tasks.workunit.client.0.smithi119.stdout:5/683: dread d6/db/f36 [4194304,4194304] 0 2024-05-27T09:29:28.690 INFO:tasks.workunit.client.0.smithi119.stdout:5/684: fsync d6/dd/d46/d47/d4b/f58 0 2024-05-27T09:29:28.690 INFO:tasks.workunit.client.0.smithi119.stdout:5/685: readlink d6/db/d9a/lee 0 2024-05-27T09:29:28.691 INFO:tasks.workunit.client.0.smithi119.stdout:5/686: chown d6/db/de/d27/d77/f9f 11206 1 2024-05-27T09:29:28.691 INFO:tasks.workunit.client.0.smithi119.stdout:2/646: truncate d2/d1a/f46 7729246 0 2024-05-27T09:29:28.691 INFO:tasks.workunit.client.0.smithi119.stdout:2/647: readlink d2/da/d6f/l7b 0 2024-05-27T09:29:28.691 INFO:tasks.workunit.client.1.smithi138.stdout:0/160: symlink da/d17/l3d 0 2024-05-27T09:29:28.691 INFO:tasks.workunit.client.1.smithi138.stdout:6/167: rename d3/d14/c30 to d3/d11/d2d/c3a 0 2024-05-27T09:29:28.691 INFO:tasks.workunit.client.1.smithi138.stdout:5/130: getdents d3 0 2024-05-27T09:29:28.692 INFO:tasks.workunit.client.1.smithi138.stdout:5/131: creat d3/d12/d16/f2e x:0 0 0 2024-05-27T09:29:28.692 INFO:tasks.workunit.client.1.smithi138.stdout:5/132: getdents d3/d12/d16/d1d/d22/d2b 0 2024-05-27T09:29:28.692 INFO:tasks.workunit.client.0.smithi119.stdout:7/676: truncate f1 7361140 0 2024-05-27T09:29:28.698 INFO:tasks.workunit.client.0.smithi119.stdout:2/648: mknod d2/da/d8a/dd4/d89/d94/cde 0 2024-05-27T09:29:28.698 INFO:tasks.workunit.client.0.smithi119.stdout:7/677: creat d7/df/d19/d97/d92/db7/dba/d42/d64/dad/fd8 x:0 0 0 2024-05-27T09:29:28.700 INFO:tasks.workunit.client.0.smithi119.stdout:5/687: getdents d6/db/dc 0 2024-05-27T09:29:28.701 INFO:tasks.workunit.client.0.smithi119.stdout:2/649: mknod d2/da/d8a/dd4/d89/d94/d81/cdf 0 2024-05-27T09:29:28.701 INFO:tasks.workunit.client.0.smithi119.stdout:2/650: readlink d2/da/d15/l8c 0 2024-05-27T09:29:28.701 INFO:tasks.workunit.client.0.smithi119.stdout:9/529: dwrite d18/d19/d1b/d2a/d63/f97 [0,4194304] 0 2024-05-27T09:29:28.702 INFO:tasks.workunit.client.1.smithi138.stdout:3/69: sync 2024-05-27T09:29:28.702 INFO:tasks.workunit.client.0.smithi119.stdout:3/779: dwrite db/d36/d45/d93/fa4 [0,4194304] 0 2024-05-27T09:29:28.702 INFO:tasks.workunit.client.0.smithi119.stdout:3/780: fdatasync db/dc9/f79 0 2024-05-27T09:29:28.703 INFO:tasks.workunit.client.0.smithi119.stdout:6/653: dwrite d0/d2/d1c/d21/f45 [0,4194304] 0 2024-05-27T09:29:28.704 INFO:tasks.workunit.client.0.smithi119.stdout:5/688: unlink d6/dd/d39/d74/l79 0 2024-05-27T09:29:28.710 INFO:tasks.workunit.client.1.smithi138.stdout:4/95: dwrite d7/f17 [0,4194304] 0 2024-05-27T09:29:28.714 INFO:tasks.workunit.client.1.smithi138.stdout:5/133: dread d3/f11 [0,4194304] 0 2024-05-27T09:29:28.714 INFO:tasks.workunit.client.1.smithi138.stdout:5/134: stat d3/d12/f25 0 2024-05-27T09:29:28.715 INFO:tasks.workunit.client.0.smithi119.stdout:2/651: rename d2/da/d8a/dd4/d69/db1/fce to d2/da/d8a/dd4/d89/d94/d87/dcf/fe0 0 2024-05-27T09:29:28.717 INFO:tasks.workunit.client.0.smithi119.stdout:5/689: mkdir d6/dd/d46/d47/d4b/dd8/df2 0 2024-05-27T09:29:28.721 INFO:tasks.workunit.client.1.smithi138.stdout:4/96: mknod d7/df/c1c 0 2024-05-27T09:29:28.721 INFO:tasks.workunit.client.1.smithi138.stdout:4/97: creat d7/df/f1d x:0 0 0 2024-05-27T09:29:28.726 INFO:tasks.workunit.client.0.smithi119.stdout:3/781: rename db/d36/d45/d49/l8a to db/d36/d45/d4c/d62/d13/d33/d9b/de3/lfd 0 2024-05-27T09:29:28.731 INFO:tasks.workunit.client.0.smithi119.stdout:2/652: creat d2/da/d2c/fe1 x:0 0 0 2024-05-27T09:29:28.733 INFO:tasks.workunit.client.0.smithi119.stdout:6/654: getdents d0/d2/d57/d37/dad 0 2024-05-27T09:29:28.739 INFO:tasks.workunit.client.0.smithi119.stdout:2/653: mkdir d2/da/d8a/dd4/d89/d94/de2 0 2024-05-27T09:29:28.744 INFO:tasks.workunit.client.0.smithi119.stdout:6/655: truncate d0/d2/d57/d37/f43 1330444 0 2024-05-27T09:29:28.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:28 smithi119 ceph-mon[29082]: pgmap v12: 65 pgs: 65 active+clean; 1.5 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 152 MiB/s rd, 182 MiB/s wr, 678 op/s 2024-05-27T09:29:28.747 INFO:tasks.workunit.client.0.smithi119.stdout:3/782: symlink db/d36/d45/d4c/lfe 0 2024-05-27T09:29:28.748 INFO:tasks.workunit.client.1.smithi138.stdout:7/92: sync 2024-05-27T09:29:28.748 INFO:tasks.workunit.client.1.smithi138.stdout:9/76: sync 2024-05-27T09:29:28.749 INFO:tasks.workunit.client.1.smithi138.stdout:9/77: chown d6/db 0 1 2024-05-27T09:29:28.749 INFO:tasks.workunit.client.1.smithi138.stdout:9/78: read - d6/db/f14 zero size 2024-05-27T09:29:28.749 INFO:tasks.workunit.client.1.smithi138.stdout:8/99: sync 2024-05-27T09:29:28.749 INFO:tasks.workunit.client.1.smithi138.stdout:1/145: sync 2024-05-27T09:29:28.749 INFO:tasks.workunit.client.1.smithi138.stdout:1/146: getdents d4/d8/d22/d29/d2b 0 2024-05-27T09:29:28.749 INFO:tasks.workunit.client.1.smithi138.stdout:1/147: chown d4/d8/cc 786528 1 2024-05-27T09:29:28.749 INFO:tasks.workunit.client.1.smithi138.stdout:2/90: sync 2024-05-27T09:29:28.750 INFO:tasks.workunit.client.1.smithi138.stdout:9/79: unlink d6/db/l11 0 2024-05-27T09:29:28.752 INFO:tasks.workunit.client.0.smithi119.stdout:2/654: creat d2/da/d4e/fe3 x:0 0 0 2024-05-27T09:29:28.753 INFO:tasks.workunit.client.1.smithi138.stdout:2/91: mknod d1/d13/d17/c1f 0 2024-05-27T09:29:28.755 INFO:tasks.workunit.client.0.smithi119.stdout:1/702: sync 2024-05-27T09:29:28.755 INFO:tasks.workunit.client.0.smithi119.stdout:8/604: sync 2024-05-27T09:29:28.756 INFO:tasks.workunit.client.1.smithi138.stdout:9/80: mkdir d6/db/de/d18 0 2024-05-27T09:29:28.756 INFO:tasks.workunit.client.1.smithi138.stdout:7/93: getdents d0 0 2024-05-27T09:29:28.756 INFO:tasks.workunit.client.1.smithi138.stdout:7/94: read - d0/f19 zero size 2024-05-27T09:29:28.756 INFO:tasks.workunit.client.0.smithi119.stdout:6/656: symlink d0/d2/d57/d37/dad/db6/le6 0 2024-05-27T09:29:28.756 INFO:tasks.workunit.client.0.smithi119.stdout:6/657: write d0/d2/d1c/d21/d58/fd4 [1042191,100495] 0 2024-05-27T09:29:28.757 INFO:tasks.workunit.client.0.smithi119.stdout:6/658: dread - d0/d2/d57/d37/d74/f98 zero size 2024-05-27T09:29:28.761 INFO:tasks.workunit.client.1.smithi138.stdout:9/81: getdents d6/db/dc 0 2024-05-27T09:29:28.761 INFO:tasks.workunit.client.1.smithi138.stdout:2/92: symlink d1/d13/l20 0 2024-05-27T09:29:28.761 INFO:tasks.workunit.client.1.smithi138.stdout:7/95: truncate d0/d7/fe 970186 0 2024-05-27T09:29:28.762 INFO:tasks.workunit.client.0.smithi119.stdout:3/783: rename db/d5b/ced to db/d5b/dce/cff 0 2024-05-27T09:29:28.762 INFO:tasks.workunit.client.0.smithi119.stdout:0/590: sync 2024-05-27T09:29:28.762 INFO:tasks.workunit.client.0.smithi119.stdout:0/591: chown d2/d6/d9/d12/d4f/f89 0 1 2024-05-27T09:29:28.762 INFO:tasks.workunit.client.0.smithi119.stdout:0/592: write d2/d6/d9/f27 [1008328,80809] 0 2024-05-27T09:29:28.763 INFO:tasks.workunit.client.1.smithi138.stdout:2/93: mknod d1/c21 0 2024-05-27T09:29:28.763 INFO:tasks.workunit.client.1.smithi138.stdout:2/94: mknod d1/c22 0 2024-05-27T09:29:28.763 INFO:tasks.workunit.client.1.smithi138.stdout:2/95: readlink d1/d13/l20 0 2024-05-27T09:29:28.765 INFO:tasks.workunit.client.1.smithi138.stdout:2/96: mknod d1/d13/d17/c23 0 2024-05-27T09:29:28.766 INFO:tasks.workunit.client.0.smithi119.stdout:2/655: creat d2/da/d8a/fe4 x:0 0 0 2024-05-27T09:29:28.769 INFO:tasks.workunit.client.0.smithi119.stdout:1/703: truncate d7/f5b 4161638 0 2024-05-27T09:29:28.769 INFO:tasks.workunit.client.0.smithi119.stdout:6/659: mkdir d0/d85/de7 0 2024-05-27T09:29:28.769 INFO:tasks.workunit.client.1.smithi138.stdout:2/97: write d1/d10/fe [150413,82098] 0 2024-05-27T09:29:28.771 INFO:tasks.workunit.client.0.smithi119.stdout:8/605: rename d3/d39/l56 to d3/d8/d4d/d5d/lc4 0 2024-05-27T09:29:28.773 INFO:tasks.workunit.client.1.smithi138.stdout:7/96: read d0/d7/f12 [30857,110562] 0 2024-05-27T09:29:28.774 INFO:tasks.workunit.client.1.smithi138.stdout:7/97: fdatasync d0/d7/fe 0 2024-05-27T09:29:28.774 INFO:tasks.workunit.client.1.smithi138.stdout:7/98: write d0/d7/f12 [1614032,103934] 0 2024-05-27T09:29:28.776 INFO:tasks.workunit.client.0.smithi119.stdout:2/656: getdents d2/da 0 2024-05-27T09:29:28.776 INFO:tasks.workunit.client.0.smithi119.stdout:6/660: creat d0/d2/d57/dd0/fe8 x:0 0 0 2024-05-27T09:29:28.778 INFO:tasks.workunit.client.1.smithi138.stdout:6/168: dwrite f2 [8388608,4194304] 0 2024-05-27T09:29:28.778 INFO:tasks.workunit.client.1.smithi138.stdout:6/169: stat d3/d19/f28 0 2024-05-27T09:29:28.778 INFO:tasks.workunit.client.1.smithi138.stdout:2/98: symlink d1/d15/l24 0 2024-05-27T09:29:28.781 INFO:tasks.workunit.client.0.smithi119.stdout:2/657: mkdir d2/da/d8a/dd4/d89/d94/d87/dcf/de5 0 2024-05-27T09:29:28.782 INFO:tasks.workunit.client.0.smithi119.stdout:3/784: dread db/d36/d45/d49/d9d/fbb [0,4194304] 0 2024-05-27T09:29:28.783 INFO:tasks.workunit.client.1.smithi138.stdout:7/99: link d0/d7/l14 d0/d7/l20 0 2024-05-27T09:29:28.786 INFO:tasks.workunit.client.1.smithi138.stdout:6/170: rename d3/d14/c2c to d3/d11/d2d/d37/c3b 0 2024-05-27T09:29:28.786 INFO:tasks.workunit.client.0.smithi119.stdout:7/678: dwrite d7/df/d19/d46/f85 [0,4194304] 0 2024-05-27T09:29:28.787 INFO:tasks.workunit.client.1.smithi138.stdout:2/99: mkdir d1/d13/d17/d25 0 2024-05-27T09:29:28.787 INFO:tasks.workunit.client.0.smithi119.stdout:6/661: getdents d0/d2/d1c 0 2024-05-27T09:29:28.789 INFO:tasks.workunit.client.0.smithi119.stdout:2/658: write d2/da/d2c/f58 [537636,6299] 0 2024-05-27T09:29:28.789 INFO:tasks.workunit.client.0.smithi119.stdout:1/704: dread d7/dd/d25/fc5 [0,4194304] 0 2024-05-27T09:29:28.789 INFO:tasks.workunit.client.0.smithi119.stdout:2/659: creat d2/da/d8a/dd4/d89/d94/d81/fe6 x:0 0 0 2024-05-27T09:29:28.790 INFO:tasks.workunit.client.1.smithi138.stdout:7/100: creat d0/f21 x:0 0 0 2024-05-27T09:29:28.790 INFO:tasks.workunit.client.1.smithi138.stdout:7/101: chown d0/d7/l14 70 1 2024-05-27T09:29:28.790 INFO:tasks.workunit.client.1.smithi138.stdout:7/102: fdatasync d0/f1d 0 2024-05-27T09:29:28.791 INFO:tasks.workunit.client.1.smithi138.stdout:6/171: unlink d3/l10 0 2024-05-27T09:29:28.792 INFO:tasks.workunit.client.0.smithi119.stdout:9/530: dwrite fc [0,4194304] 0 2024-05-27T09:29:28.793 INFO:tasks.workunit.client.0.smithi119.stdout:3/785: unlink db/dc9/d98/f80 0 2024-05-27T09:29:28.794 INFO:tasks.workunit.client.1.smithi138.stdout:7/103: rename d0/d7/c11 to d0/c22 0 2024-05-27T09:29:28.794 INFO:tasks.workunit.client.1.smithi138.stdout:7/104: fsync d0/d7/f1b 0 2024-05-27T09:29:28.794 INFO:tasks.workunit.client.1.smithi138.stdout:7/105: readlink d0/d7/l20 0 2024-05-27T09:29:28.795 INFO:tasks.workunit.client.0.smithi119.stdout:4/668: dwrite d4/d6/d81/dce/f9f [0,4194304] 0 2024-05-27T09:29:28.798 INFO:tasks.workunit.client.0.smithi119.stdout:1/705: dread d7/db/d12/d15/f3f [0,4194304] 0 2024-05-27T09:29:28.799 INFO:tasks.workunit.client.0.smithi119.stdout:9/531: dread d18/d19/d1b/d40/d7b/f7e [0,4194304] 0 2024-05-27T09:29:28.800 INFO:tasks.workunit.client.1.smithi138.stdout:6/172: creat d3/d11/d2d/d37/f3c x:0 0 0 2024-05-27T09:29:28.800 INFO:tasks.workunit.client.1.smithi138.stdout:0/161: dwrite da/f3c [0,4194304] 0 2024-05-27T09:29:28.807 INFO:tasks.workunit.client.0.smithi119.stdout:3/786: creat db/d5b/dce/f100 x:0 0 0 2024-05-27T09:29:28.807 INFO:tasks.workunit.client.0.smithi119.stdout:3/787: chown db/d36/d45/d4c/d62/d13/d33/l9a 1 1 2024-05-27T09:29:28.807 INFO:tasks.workunit.client.0.smithi119.stdout:3/788: creat db/d36/d45/dab/f101 x:0 0 0 2024-05-27T09:29:28.807 INFO:tasks.workunit.client.0.smithi119.stdout:2/660: symlink d2/da/d15/le7 0 2024-05-27T09:29:28.807 INFO:tasks.workunit.client.0.smithi119.stdout:3/789: chown db/fb1 173921 1 2024-05-27T09:29:28.807 INFO:tasks.workunit.client.0.smithi119.stdout:2/661: chown d2/da/d8a/dd4/d89/d94/fd3 0 1 2024-05-27T09:29:28.807 INFO:tasks.workunit.client.0.smithi119.stdout:2/662: stat d2/da/d6f/fa5 0 2024-05-27T09:29:28.807 INFO:tasks.workunit.client.0.smithi119.stdout:5/690: dwrite d6/db/f36 [4194304,4194304] 0 2024-05-27T09:29:28.817 INFO:tasks.workunit.client.0.smithi119.stdout:7/679: mknod d7/d1a/d40/d44/d67/cd9 0 2024-05-27T09:29:28.818 INFO:tasks.workunit.client.1.smithi138.stdout:2/100: rmdir d1/d13 39 2024-05-27T09:29:28.818 INFO:tasks.workunit.client.0.smithi119.stdout:9/532: creat d18/d19/d1b/d40/d7b/fa9 x:0 0 0 2024-05-27T09:29:28.819 INFO:tasks.workunit.client.0.smithi119.stdout:1/706: creat d7/dd/db0/dcd/fda x:0 0 0 2024-05-27T09:29:28.820 INFO:tasks.workunit.client.0.smithi119.stdout:1/707: truncate d7/dd/d44/f88 326427 0 2024-05-27T09:29:28.826 INFO:tasks.workunit.client.1.smithi138.stdout:2/101: truncate d1/d13/d17/f11 38334 0 2024-05-27T09:29:28.826 INFO:tasks.workunit.client.1.smithi138.stdout:2/102: fdatasync d1/d13/d17/f11 0 2024-05-27T09:29:28.827 INFO:tasks.workunit.client.1.smithi138.stdout:1/148: dwrite d4/f18 [4194304,4194304] 0 2024-05-27T09:29:28.831 INFO:tasks.workunit.client.0.smithi119.stdout:3/790: symlink db/dc9/d98/db9/dd1/l102 0 2024-05-27T09:29:28.831 INFO:tasks.workunit.client.0.smithi119.stdout:3/791: creat db/d36/d45/dab/f103 x:0 0 0 2024-05-27T09:29:28.837 INFO:tasks.workunit.client.1.smithi138.stdout:2/103: link d1/d15/l24 d1/d13/d17/d25/l26 0 2024-05-27T09:29:28.837 INFO:tasks.workunit.client.1.smithi138.stdout:1/149: mknod d4/d1a/d2e/d32/c34 0 2024-05-27T09:29:28.840 INFO:tasks.workunit.client.0.smithi119.stdout:5/691: symlink d6/dd/d39/d19/d3c/d52/lf3 0 2024-05-27T09:29:28.841 INFO:tasks.workunit.client.1.smithi138.stdout:2/104: rename d1/d10/f16 to d1/d15/f27 0 2024-05-27T09:29:28.843 INFO:tasks.workunit.client.1.smithi138.stdout:8/100: dwrite d3/d8/fc [0,4194304] 0 2024-05-27T09:29:28.843 INFO:tasks.workunit.client.0.smithi119.stdout:4/669: rmdir d4/d1c 39 2024-05-27T09:29:28.859 INFO:tasks.workunit.client.0.smithi119.stdout:9/533: symlink d18/d19/d1b/d2a/d9f/laa 0 2024-05-27T09:29:28.860 INFO:tasks.workunit.client.0.smithi119.stdout:6/662: dwrite d0/d2/d57/d37/f8d [4194304,4194304] 0 2024-05-27T09:29:28.874 INFO:tasks.workunit.client.1.smithi138.stdout:1/150: dread d4/f7 [0,4194304] 0 2024-05-27T09:29:28.874 INFO:tasks.workunit.client.1.smithi138.stdout:1/151: write d4/d8/f1c [1275577,107479] 0 2024-05-27T09:29:28.875 INFO:tasks.workunit.client.1.smithi138.stdout:8/101: dread d3/d8/d11/f12 [0,4194304] 0 2024-05-27T09:29:28.875 INFO:tasks.workunit.client.0.smithi119.stdout:5/692: getdents d6/db/d9a 0 2024-05-27T09:29:28.875 INFO:tasks.workunit.client.0.smithi119.stdout:5/693: chown d6/dd/d39/l5d 176 1 2024-05-27T09:29:28.875 INFO:tasks.workunit.client.0.smithi119.stdout:0/593: dwrite d2/d6/d9/f1c [0,4194304] 0 2024-05-27T09:29:28.878 INFO:tasks.workunit.client.0.smithi119.stdout:4/670: mkdir d4/de/d21/d40/d47/d5c/d9e/dd3 0 2024-05-27T09:29:28.878 INFO:tasks.workunit.client.0.smithi119.stdout:9/534: chown d18/l5c 5 1 2024-05-27T09:29:28.878 INFO:tasks.workunit.client.0.smithi119.stdout:9/535: truncate d18/f8b 366934 0 2024-05-27T09:29:28.878 INFO:tasks.workunit.client.1.smithi138.stdout:0/162: dwrite da/f20 [0,4194304] 0 2024-05-27T09:29:28.878 INFO:tasks.workunit.client.1.smithi138.stdout:8/102: rmdir d3/d8/d11 39 2024-05-27T09:29:28.880 INFO:tasks.workunit.client.0.smithi119.stdout:0/594: rmdir d2/d6 39 2024-05-27T09:29:28.880 INFO:tasks.workunit.client.0.smithi119.stdout:6/663: creat d0/d2/d4e/fe9 x:0 0 0 2024-05-27T09:29:28.880 INFO:tasks.workunit.client.0.smithi119.stdout:0/595: creat d2/d6/d9/fbd x:0 0 0 2024-05-27T09:29:28.882 INFO:tasks.workunit.client.0.smithi119.stdout:7/680: rename d7/d1a/d40 to d7/df/d19/d9a/da6/dda 0 2024-05-27T09:29:28.882 INFO:tasks.workunit.client.1.smithi138.stdout:6/173: dwrite d3/f31 [0,4194304] 0 2024-05-27T09:29:28.883 INFO:tasks.workunit.client.0.smithi119.stdout:4/671: dread d4/d23/d24/d5b/f78 [0,4194304] 0 2024-05-27T09:29:28.883 INFO:tasks.workunit.client.0.smithi119.stdout:9/536: dread d18/d19/d1b/d29/f43 [0,4194304] 0 2024-05-27T09:29:28.885 INFO:tasks.workunit.client.1.smithi138.stdout:8/103: write f1 [4200225,56776] 0 2024-05-27T09:29:28.885 INFO:tasks.workunit.client.1.smithi138.stdout:8/104: dread - d3/d8/f14 zero size 2024-05-27T09:29:28.885 INFO:tasks.workunit.client.1.smithi138.stdout:8/105: write d3/d8/d11/f12 [1056238,59927] 0 2024-05-27T09:29:28.885 INFO:tasks.workunit.client.1.smithi138.stdout:8/106: read - d3/d8/f14 zero size 2024-05-27T09:29:28.887 INFO:tasks.workunit.client.0.smithi119.stdout:0/596: mknod d2/d1a/cbe 0 2024-05-27T09:29:28.889 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:28 smithi138 ceph-mon[34885]: pgmap v12: 65 pgs: 65 active+clean; 1.5 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 152 MiB/s rd, 182 MiB/s wr, 678 op/s 2024-05-27T09:29:28.890 INFO:tasks.workunit.client.0.smithi119.stdout:6/664: dread d0/d2/d4e/fc0 [0,4194304] 0 2024-05-27T09:29:28.892 INFO:tasks.workunit.client.1.smithi138.stdout:0/163: dread da/d17/d1d/d1f/f3b [0,4194304] 0 2024-05-27T09:29:28.896 INFO:tasks.workunit.client.1.smithi138.stdout:1/152: dread d4/fd [0,4194304] 0 2024-05-27T09:29:28.896 INFO:tasks.workunit.client.1.smithi138.stdout:1/153: creat d4/d8/d22/f35 x:0 0 0 2024-05-27T09:29:28.896 INFO:tasks.workunit.client.1.smithi138.stdout:1/154: fsync d4/da/f2f 0 2024-05-27T09:29:28.896 INFO:tasks.workunit.client.1.smithi138.stdout:7/106: dwrite d0/d7/f1b [0,4194304] 0 2024-05-27T09:29:28.896 INFO:tasks.workunit.client.1.smithi138.stdout:1/155: chown d4/c13 186322163 1 2024-05-27T09:29:28.896 INFO:tasks.workunit.client.1.smithi138.stdout:7/107: chown d0/f1a 1060362741 1 2024-05-27T09:29:28.897 INFO:tasks.workunit.client.1.smithi138.stdout:7/108: mkdir d0/d23 0 2024-05-27T09:29:28.897 INFO:tasks.workunit.client.1.smithi138.stdout:7/109: creat d0/f24 x:0 0 0 2024-05-27T09:29:28.897 INFO:tasks.workunit.client.1.smithi138.stdout:7/110: chown d0/f19 391433 1 2024-05-27T09:29:28.898 INFO:tasks.workunit.client.1.smithi138.stdout:7/111: mknod d0/c25 0 2024-05-27T09:29:28.899 INFO:tasks.workunit.client.1.smithi138.stdout:7/112: symlink d0/d23/l26 0 2024-05-27T09:29:28.900 INFO:tasks.workunit.client.1.smithi138.stdout:7/113: truncate d0/f19 1025402 0 2024-05-27T09:29:28.900 INFO:tasks.workunit.client.0.smithi119.stdout:1/708: rename d7/dd/d25/d29 to d7/db/d12/ddb 0 2024-05-27T09:29:28.900 INFO:tasks.workunit.client.1.smithi138.stdout:7/114: write d0/d7/fe [1654756,128830] 0 2024-05-27T09:29:28.901 INFO:tasks.workunit.client.0.smithi119.stdout:2/663: dwrite d2/da/d8a/dd4/d89/fa3 [0,4194304] 0 2024-05-27T09:29:28.904 INFO:tasks.workunit.client.1.smithi138.stdout:2/105: dwrite d1/d10/f6 [0,4194304] 0 2024-05-27T09:29:28.906 INFO:tasks.workunit.client.0.smithi119.stdout:5/694: rename d6/db/dc/ldd to d6/db/d9a/lf4 0 2024-05-27T09:29:28.908 INFO:tasks.workunit.client.1.smithi138.stdout:2/106: creat d1/d13/d17/d25/f28 x:0 0 0 2024-05-27T09:29:28.910 INFO:tasks.workunit.client.1.smithi138.stdout:6/174: read d3/d19/f22 [2046554,113429] 0 2024-05-27T09:29:28.911 INFO:tasks.workunit.client.1.smithi138.stdout:6/175: creat d3/d19/f3d x:0 0 0 2024-05-27T09:29:28.911 INFO:tasks.workunit.client.1.smithi138.stdout:6/176: write d3/fa [2097025,120603] 0 2024-05-27T09:29:28.912 INFO:tasks.workunit.client.0.smithi119.stdout:7/681: read d7/d1a/f59 [880598,3021] 0 2024-05-27T09:29:28.912 INFO:tasks.workunit.client.0.smithi119.stdout:7/682: write d7/d1a/f1c [3515867,38982] 0 2024-05-27T09:29:28.915 INFO:tasks.workunit.client.1.smithi138.stdout:6/177: symlink d3/d14/l3e 0 2024-05-27T09:29:28.915 INFO:tasks.workunit.client.1.smithi138.stdout:6/178: dread - d3/d19/f38 zero size 2024-05-27T09:29:28.916 INFO:tasks.workunit.client.0.smithi119.stdout:1/709: mkdir d7/db/d8f/d9b/dc9/ddc 0 2024-05-27T09:29:28.916 INFO:tasks.workunit.client.0.smithi119.stdout:1/710: write d7/dd/d44/fc8 [89116,19909] 0 2024-05-27T09:29:28.916 INFO:tasks.workunit.client.0.smithi119.stdout:1/711: dread - d7/dd/d44/fa9 zero size 2024-05-27T09:29:28.920 INFO:tasks.workunit.client.0.smithi119.stdout:2/664: mkdir d2/da/d8a/dd4/d89/de8 0 2024-05-27T09:29:28.920 INFO:tasks.workunit.client.0.smithi119.stdout:4/672: rename d4/d48/c65 to d4/d1c/d39/cd4 0 2024-05-27T09:29:28.920 INFO:tasks.workunit.client.0.smithi119.stdout:2/665: fdatasync d2/da/d8a/dd4/d69/f6b 0 2024-05-27T09:29:28.920 INFO:tasks.workunit.client.0.smithi119.stdout:2/666: stat d2/fa0 0 2024-05-27T09:29:28.921 INFO:tasks.workunit.client.0.smithi119.stdout:5/695: symlink d6/dd/d46/d47/d4b/lf5 0 2024-05-27T09:29:28.927 INFO:tasks.workunit.client.0.smithi119.stdout:1/712: creat d7/dd/d44/d47/fdd x:0 0 0 2024-05-27T09:29:28.927 INFO:tasks.workunit.client.0.smithi119.stdout:0/597: write d2/d6/d9/d12/d4b/d96/f98 [1955768,51180] 0 2024-05-27T09:29:28.930 INFO:tasks.workunit.client.1.smithi138.stdout:0/164: write da/f29 [1431222,115963] 0 2024-05-27T09:29:28.930 INFO:tasks.workunit.client.1.smithi138.stdout:0/165: write da/d17/f31 [536431,40295] 0 2024-05-27T09:29:28.931 INFO:tasks.workunit.client.0.smithi119.stdout:6/665: rename d0/d2/d57/d37/d74/fa9 to d0/d19/db2/fea 0 2024-05-27T09:29:28.932 INFO:tasks.workunit.client.0.smithi119.stdout:6/666: creat d0/d2/d4e/feb x:0 0 0 2024-05-27T09:29:28.937 INFO:tasks.workunit.client.0.smithi119.stdout:4/673: creat d4/de/d21/d40/d47/d5c/fd5 x:0 0 0 2024-05-27T09:29:28.941 INFO:tasks.workunit.client.0.smithi119.stdout:5/696: symlink d6/d2f/da5/lf6 0 2024-05-27T09:29:28.941 INFO:tasks.workunit.client.0.smithi119.stdout:5/697: fsync d6/db/de/d27/d94/fa9 0 2024-05-27T09:29:28.941 INFO:tasks.workunit.client.0.smithi119.stdout:5/698: write d6/db/de/d27/dc9/fb4 [506349,29974] 0 2024-05-27T09:29:28.941 INFO:tasks.workunit.client.0.smithi119.stdout:1/713: creat d7/db/d8f/d9b/dc9/fde x:0 0 0 2024-05-27T09:29:28.950 INFO:tasks.workunit.client.0.smithi119.stdout:5/699: mkdir d6/dd/d46/d47/d4b/dd8/ddb/df7 0 2024-05-27T09:29:28.962 INFO:tasks.workunit.client.1.smithi138.stdout:8/107: dwrite d3/d8/f14 [0,4194304] 0 2024-05-27T09:29:28.963 INFO:tasks.workunit.client.1.smithi138.stdout:8/108: stat d3/d8 0 2024-05-27T09:29:28.969 INFO:tasks.workunit.client.0.smithi119.stdout:5/700: write d6/db/de/d27/d77/f9f [107239,35589] 0 2024-05-27T09:29:28.969 INFO:tasks.workunit.client.0.smithi119.stdout:5/701: stat d6/db/de/d27/f3e 0 2024-05-27T09:29:28.969 INFO:tasks.workunit.client.0.smithi119.stdout:1/714: write d7/f1e [2890894,67383] 0 2024-05-27T09:29:28.975 INFO:tasks.workunit.client.0.smithi119.stdout:6/667: dread d0/d2/d1c/d21/d58/f61 [0,4194304] 0 2024-05-27T09:29:28.985 INFO:tasks.workunit.client.0.smithi119.stdout:1/715: unlink d7/dd/d44/d47/fdd 0 2024-05-27T09:29:28.992 INFO:tasks.workunit.client.0.smithi119.stdout:6/668: unlink d0/d2/de1/fe2 0 2024-05-27T09:29:28.992 INFO:tasks.workunit.client.0.smithi119.stdout:6/669: truncate d0/d85/fae 1249174 0 2024-05-27T09:29:28.993 INFO:tasks.workunit.client.0.smithi119.stdout:1/716: creat d7/db/d12/d15/dd2/fdf x:0 0 0 2024-05-27T09:29:28.994 INFO:tasks.workunit.client.1.smithi138.stdout:1/156: dwrite d4/da/f2f [0,4194304] 0 2024-05-27T09:29:28.996 INFO:tasks.workunit.client.1.smithi138.stdout:1/157: link d4/d8/l17 d4/d8/l36 0 2024-05-27T09:29:28.996 INFO:tasks.workunit.client.0.smithi119.stdout:6/670: mkdir d0/d2/dec 0 2024-05-27T09:29:28.998 INFO:tasks.workunit.client.1.smithi138.stdout:1/158: getdents d4/da 0 2024-05-27T09:29:28.999 INFO:tasks.workunit.client.1.smithi138.stdout:7/115: dwrite d0/d7/f1b [4194304,4194304] 0 2024-05-27T09:29:28.999 INFO:tasks.workunit.client.1.smithi138.stdout:7/116: write d0/f21 [620188,20206] 0 2024-05-27T09:29:29.000 INFO:tasks.workunit.client.1.smithi138.stdout:6/179: dwrite d3/d19/f34 [0,4194304] 0 2024-05-27T09:29:29.000 INFO:tasks.workunit.client.1.smithi138.stdout:6/180: creat d3/d14/f3f x:0 0 0 2024-05-27T09:29:29.004 INFO:tasks.workunit.client.0.smithi119.stdout:4/674: dread d4/d1c/d39/f5d [4194304,4194304] 0 2024-05-27T09:29:29.006 INFO:tasks.workunit.client.0.smithi119.stdout:4/675: symlink d4/de/d66/ld6 0 2024-05-27T09:29:29.007 INFO:tasks.workunit.client.1.smithi138.stdout:7/117: rename d0/d7/cf to d0/d23/c27 0 2024-05-27T09:29:29.009 INFO:tasks.workunit.client.0.smithi119.stdout:2/667: dwrite d2/da/d6f/fa5 [0,4194304] 0 2024-05-27T09:29:29.010 INFO:tasks.workunit.client.0.smithi119.stdout:4/676: mknod d4/db1/cd7 0 2024-05-27T09:29:29.010 INFO:tasks.workunit.client.0.smithi119.stdout:4/677: read d4/d1c/d39/f64 [31734,38906] 0 2024-05-27T09:29:29.011 INFO:tasks.workunit.client.1.smithi138.stdout:6/181: symlink d3/d11/d2d/d37/l40 0 2024-05-27T09:29:29.011 INFO:tasks.workunit.client.1.smithi138.stdout:6/182: creat d3/d19/f41 x:0 0 0 2024-05-27T09:29:29.012 INFO:tasks.workunit.client.1.smithi138.stdout:6/183: write d3/d19/f41 [279171,6709] 0 2024-05-27T09:29:29.012 INFO:tasks.workunit.client.1.smithi138.stdout:6/184: chown d3/d14/f3f 1289 1 2024-05-27T09:29:29.012 INFO:tasks.workunit.client.1.smithi138.stdout:6/185: truncate d3/d19/f28 1185707 0 2024-05-27T09:29:29.012 INFO:tasks.workunit.client.0.smithi119.stdout:4/678: link d4/d6/c18 d4/d6/d3e/d89/cd8 0 2024-05-27T09:29:29.012 INFO:tasks.workunit.client.0.smithi119.stdout:9/537: dwrite d18/d19/f59 [0,4194304] 0 2024-05-27T09:29:29.013 INFO:tasks.workunit.client.0.smithi119.stdout:2/668: write d2/d1a/f61 [396015,32029] 0 2024-05-27T09:29:29.013 INFO:tasks.workunit.client.0.smithi119.stdout:1/717: dread d7/db/d12/d15/d1f/f8b [0,4194304] 0 2024-05-27T09:29:29.016 INFO:tasks.workunit.client.0.smithi119.stdout:1/718: readlink d7/db/d12/ddb/l85 0 2024-05-27T09:29:29.016 INFO:tasks.workunit.client.0.smithi119.stdout:0/598: dwrite d2/d6/d9/fbd [0,4194304] 0 2024-05-27T09:29:29.016 INFO:tasks.workunit.client.0.smithi119.stdout:1/719: fdatasync d7/f5b 0 2024-05-27T09:29:29.016 INFO:tasks.workunit.client.0.smithi119.stdout:1/720: chown d7/db/d12/d26/d39 18001 1 2024-05-27T09:29:29.016 INFO:tasks.workunit.client.0.smithi119.stdout:1/721: chown d7/f11 15455317 1 2024-05-27T09:29:29.016 INFO:tasks.workunit.client.0.smithi119.stdout:7/683: dwrite d7/df/d19/d97/d92/fab [0,4194304] 0 2024-05-27T09:29:29.017 INFO:tasks.workunit.client.0.smithi119.stdout:4/679: rename d4/de/c19 to d4/d6/d28/d2f/d36/dc8/cd9 0 2024-05-27T09:29:29.017 INFO:tasks.workunit.client.0.smithi119.stdout:4/680: write d4/d6/d81/dce/fc3 [730249,22838] 0 2024-05-27T09:29:29.017 INFO:tasks.workunit.client.0.smithi119.stdout:9/538: creat d18/d19/fab x:0 0 0 2024-05-27T09:29:29.018 INFO:tasks.workunit.client.0.smithi119.stdout:9/539: chown d18/d19/d9b/fa0 1906 1 2024-05-27T09:29:29.020 INFO:tasks.workunit.client.0.smithi119.stdout:2/669: symlink d2/da/d8a/dd4/d89/de8/le9 0 2024-05-27T09:29:29.020 INFO:tasks.workunit.client.0.smithi119.stdout:2/670: chown d2/da/d6f/fa5 107 1 2024-05-27T09:29:29.020 INFO:tasks.workunit.client.0.smithi119.stdout:2/671: read - d2/da/d2c/fe1 zero size 2024-05-27T09:29:29.020 INFO:tasks.workunit.client.0.smithi119.stdout:2/672: write d2/da/d15/f80 [2210996,93711] 0 2024-05-27T09:29:29.020 INFO:tasks.workunit.client.1.smithi138.stdout:0/166: dwrite da/d2a/d2b/f33 [0,4194304] 0 2024-05-27T09:29:29.020 INFO:tasks.workunit.client.1.smithi138.stdout:0/167: creat da/d17/d1d/d1f/f3e x:0 0 0 2024-05-27T09:29:29.026 INFO:tasks.workunit.client.1.smithi138.stdout:1/159: dread d4/d1a/f27 [0,4194304] 0 2024-05-27T09:29:29.028 INFO:tasks.workunit.client.1.smithi138.stdout:1/160: creat d4/f37 x:0 0 0 2024-05-27T09:29:29.030 INFO:tasks.workunit.client.1.smithi138.stdout:1/161: mkdir d4/d38 0 2024-05-27T09:29:29.033 INFO:tasks.workunit.client.0.smithi119.stdout:5/702: dwrite d6/db/de/d27/f3f [0,4194304] 0 2024-05-27T09:29:29.033 INFO:tasks.workunit.client.0.smithi119.stdout:5/703: readlink d6/dd/d39/d19/l1c 0 2024-05-27T09:29:29.033 INFO:tasks.workunit.client.1.smithi138.stdout:1/162: rename d4/d1a/d2d to d4/d8/d39 0 2024-05-27T09:29:29.038 INFO:tasks.workunit.client.1.smithi138.stdout:7/118: rmdir d0 39 2024-05-27T09:29:29.039 INFO:tasks.workunit.client.0.smithi119.stdout:7/684: creat d7/fdb x:0 0 0 2024-05-27T09:29:29.039 INFO:tasks.workunit.client.0.smithi119.stdout:1/722: write d7/dd/d25/f30 [923299,41759] 0 2024-05-27T09:29:29.040 INFO:tasks.workunit.client.0.smithi119.stdout:4/681: mkdir d4/d23/d24/d5b/da8/dda 0 2024-05-27T09:29:29.044 INFO:tasks.workunit.client.0.smithi119.stdout:2/673: fsync d2/da/d2c/f55 0 2024-05-27T09:29:29.044 INFO:tasks.workunit.client.0.smithi119.stdout:2/674: stat d2/d1a/l7c 0 2024-05-27T09:29:29.047 INFO:tasks.workunit.client.1.smithi138.stdout:4/98: sync 2024-05-27T09:29:29.047 INFO:tasks.workunit.client.1.smithi138.stdout:5/135: sync 2024-05-27T09:29:29.047 INFO:tasks.workunit.client.1.smithi138.stdout:5/136: dread - d3/d12/d16/d1d/d22/f2a zero size 2024-05-27T09:29:29.047 INFO:tasks.workunit.client.0.smithi119.stdout:0/599: rmdir d2/d6/d9/d24/d30 39 2024-05-27T09:29:29.047 INFO:tasks.workunit.client.0.smithi119.stdout:0/600: read d2/d6/f8 [2220865,17391] 0 2024-05-27T09:29:29.048 INFO:tasks.workunit.client.1.smithi138.stdout:4/99: mknod d7/c1e 0 2024-05-27T09:29:29.049 INFO:tasks.workunit.client.1.smithi138.stdout:3/70: sync 2024-05-27T09:29:29.050 INFO:tasks.workunit.client.1.smithi138.stdout:4/100: mknod d7/c1f 0 2024-05-27T09:29:29.051 INFO:tasks.workunit.client.0.smithi119.stdout:1/723: symlink d7/dd7/le0 0 2024-05-27T09:29:29.051 INFO:tasks.workunit.client.0.smithi119.stdout:4/682: mkdir d4/d23/d24/d5b/da8/dda/ddb 0 2024-05-27T09:29:29.052 INFO:tasks.workunit.client.1.smithi138.stdout:3/71: rename d0/d2/c3 to d0/d2/c17 0 2024-05-27T09:29:29.053 INFO:tasks.workunit.client.1.smithi138.stdout:5/137: read d3/d12/d16/d1d/f1e [239762,36909] 0 2024-05-27T09:29:29.053 INFO:tasks.workunit.client.1.smithi138.stdout:5/138: chown d3/f7 45136091 1 2024-05-27T09:29:29.053 INFO:tasks.workunit.client.1.smithi138.stdout:3/72: mknod d0/c18 0 2024-05-27T09:29:29.053 INFO:tasks.workunit.client.0.smithi119.stdout:9/540: getdents d18/d19/d1b/d29 0 2024-05-27T09:29:29.053 INFO:tasks.workunit.client.0.smithi119.stdout:7/685: dread d7/d6a/f8f [0,4194304] 0 2024-05-27T09:29:29.058 INFO:tasks.workunit.client.0.smithi119.stdout:4/683: write d4/de/d21/f2b [416616,32844] 0 2024-05-27T09:29:29.068 INFO:tasks.workunit.client.0.smithi119.stdout:4/684: symlink d4/de/d66/ldc 0 2024-05-27T09:29:29.072 INFO:tasks.workunit.client.1.smithi138.stdout:1/163: dread d4/f24 [0,4194304] 0 2024-05-27T09:29:29.072 INFO:tasks.workunit.client.1.smithi138.stdout:1/164: chown d4/d8/l31 11674 1 2024-05-27T09:29:29.072 INFO:tasks.workunit.client.1.smithi138.stdout:1/165: write d4/f18 [2800152,30845] 0 2024-05-27T09:29:29.082 INFO:tasks.workunit.client.1.smithi138.stdout:5/139: dread d3/d12/d16/d1d/f24 [0,4194304] 0 2024-05-27T09:29:29.084 INFO:tasks.workunit.client.0.smithi119.stdout:4/685: rmdir d4/d6/d28/d2f/db8 39 2024-05-27T09:29:29.084 INFO:tasks.workunit.client.0.smithi119.stdout:4/686: read - d4/de/d21/d40/d47/d5c/fd5 zero size 2024-05-27T09:29:29.084 INFO:tasks.workunit.client.0.smithi119.stdout:0/601: dread d2/d6/d9/d12/d4f/f5a [0,4194304] 0 2024-05-27T09:29:29.089 INFO:tasks.workunit.client.0.smithi119.stdout:4/687: rename d4/d6/d81/dce/fc3 to d4/d6/d3e/d89/dcf/fdd 0 2024-05-27T09:29:29.089 INFO:tasks.workunit.client.0.smithi119.stdout:4/688: chown d4/d6/f2a 336992 1 2024-05-27T09:29:29.089 INFO:tasks.workunit.client.1.smithi138.stdout:6/186: write d3/f31 [3647111,77267] 0 2024-05-27T09:29:29.094 INFO:tasks.workunit.client.0.smithi119.stdout:0/602: mknod d2/d6/d9/dab/cbf 0 2024-05-27T09:29:29.095 INFO:tasks.workunit.client.0.smithi119.stdout:4/689: truncate d4/de/d66/f68 1716702 0 2024-05-27T09:29:29.095 INFO:tasks.workunit.client.0.smithi119.stdout:4/690: dread - d4/d23/d24/d5b/da8/fb3 zero size 2024-05-27T09:29:29.095 INFO:tasks.workunit.client.0.smithi119.stdout:4/691: getdents d4/de/d21/d40/d47 0 2024-05-27T09:29:29.096 INFO:tasks.workunit.client.0.smithi119.stdout:1/724: dread d7/db/d52/d65/fa4 [0,4194304] 0 2024-05-27T09:29:29.096 INFO:tasks.workunit.client.0.smithi119.stdout:1/725: fsync d7/db/d8f/d9b/dc9/fde 0 2024-05-27T09:29:29.098 INFO:tasks.workunit.client.0.smithi119.stdout:4/692: symlink d4/d6/d81/dce/lde 0 2024-05-27T09:29:29.103 INFO:tasks.workunit.client.1.smithi138.stdout:6/187: rename d3/f6 to d3/d19/f42 0 2024-05-27T09:29:29.104 INFO:tasks.workunit.client.0.smithi119.stdout:4/693: creat d4/d23/fdf x:0 0 0 2024-05-27T09:29:29.106 INFO:tasks.workunit.client.1.smithi138.stdout:1/166: truncate d4/f7 535476 0 2024-05-27T09:29:29.112 INFO:tasks.workunit.client.0.smithi119.stdout:3/792: sync 2024-05-27T09:29:29.112 INFO:tasks.workunit.client.0.smithi119.stdout:8/606: sync 2024-05-27T09:29:29.116 INFO:tasks.workunit.client.1.smithi138.stdout:1/167: symlink d4/d8/l3a 0 2024-05-27T09:29:29.116 INFO:tasks.workunit.client.0.smithi119.stdout:8/607: creat d3/d8/d1b/db1/fc5 x:0 0 0 2024-05-27T09:29:29.119 INFO:tasks.workunit.client.0.smithi119.stdout:8/608: getdents d3/d6e/d8e/d9b 0 2024-05-27T09:29:29.119 INFO:tasks.workunit.client.1.smithi138.stdout:1/168: symlink d4/d8/l3b 0 2024-05-27T09:29:29.120 INFO:tasks.workunit.client.1.smithi138.stdout:1/169: write d4/f37 [375681,104992] 0 2024-05-27T09:29:29.120 INFO:tasks.workunit.client.1.smithi138.stdout:6/188: rename d3/d14 to d3/d35/d43 0 2024-05-27T09:29:29.120 INFO:tasks.workunit.client.1.smithi138.stdout:6/189: truncate d3/d11/d2d/d37/f3c 551967 0 2024-05-27T09:29:29.121 INFO:tasks.workunit.client.1.smithi138.stdout:1/170: mkdir d4/d3c 0 2024-05-27T09:29:29.126 INFO:tasks.workunit.client.1.smithi138.stdout:6/190: link d3/d11/d2d/c3a d3/d19/c44 0 2024-05-27T09:29:29.126 INFO:tasks.workunit.client.1.smithi138.stdout:6/191: creat d3/d11/d2d/f45 x:0 0 0 2024-05-27T09:29:29.128 INFO:tasks.workunit.client.1.smithi138.stdout:6/192: symlink d3/d19/l46 0 2024-05-27T09:29:29.129 INFO:tasks.workunit.client.1.smithi138.stdout:6/193: mkdir d3/d35/d43/d1c/d32/d47 0 2024-05-27T09:29:29.129 INFO:tasks.workunit.client.1.smithi138.stdout:6/194: chown d3/d11/l21 25540521 1 2024-05-27T09:29:29.129 INFO:tasks.workunit.client.1.smithi138.stdout:6/195: readlink d3/d19/l46 0 2024-05-27T09:29:29.131 INFO:tasks.workunit.client.1.smithi138.stdout:6/196: getdents d3/d35/d43 0 2024-05-27T09:29:29.134 INFO:tasks.workunit.client.1.smithi138.stdout:6/197: creat d3/d35/d43/f48 x:0 0 0 2024-05-27T09:29:29.134 INFO:tasks.workunit.client.1.smithi138.stdout:7/119: dwrite d0/d7/fe [0,4194304] 0 2024-05-27T09:29:29.135 INFO:tasks.workunit.client.1.smithi138.stdout:7/120: chown d0/f21 45039510 1 2024-05-27T09:29:29.139 INFO:tasks.workunit.client.1.smithi138.stdout:0/168: dwrite da/d17/d1d/d1f/f26 [0,4194304] 0 2024-05-27T09:29:29.141 INFO:tasks.workunit.client.0.smithi119.stdout:9/541: dwrite d18/d19/d61/f6d [0,4194304] 0 2024-05-27T09:29:29.143 INFO:tasks.workunit.client.0.smithi119.stdout:5/704: dwrite d6/dd/f64 [0,4194304] 0 2024-05-27T09:29:29.146 INFO:tasks.workunit.client.1.smithi138.stdout:7/121: mkdir d0/d23/d28 0 2024-05-27T09:29:29.150 INFO:tasks.workunit.client.1.smithi138.stdout:1/171: dread d4/d8/d22/f2c [0,4194304] 0 2024-05-27T09:29:29.152 INFO:tasks.workunit.client.1.smithi138.stdout:1/172: mkdir d4/d1a/d2e/d32/d3d 0 2024-05-27T09:29:29.153 INFO:tasks.workunit.client.0.smithi119.stdout:9/542: dread - d18/f76 zero size 2024-05-27T09:29:29.154 INFO:tasks.workunit.client.1.smithi138.stdout:1/173: symlink d4/d8/d22/d29/d2b/l3e 0 2024-05-27T09:29:29.154 INFO:tasks.workunit.client.1.smithi138.stdout:1/174: readlink d4/l15 0 2024-05-27T09:29:29.154 INFO:tasks.workunit.client.1.smithi138.stdout:1/175: stat d4/l1d 0 2024-05-27T09:29:29.157 INFO:tasks.workunit.client.1.smithi138.stdout:1/176: rename d4/d8/f1c to d4/d8/d22/d29/f3f 0 2024-05-27T09:29:29.159 INFO:tasks.workunit.client.0.smithi119.stdout:5/705: mknod d6/dd/d46/d47/d4b/dd8/ddb/df7/cf8 0 2024-05-27T09:29:29.159 INFO:tasks.workunit.client.0.smithi119.stdout:5/706: write d6/dd/d46/d47/d4b/dd8/ddb/fe5 [1158876,35573] 0 2024-05-27T09:29:29.159 INFO:tasks.workunit.client.0.smithi119.stdout:5/707: chown d6/dd/d39/d19/d3c/d52/f99 9 1 2024-05-27T09:29:29.159 INFO:tasks.workunit.client.1.smithi138.stdout:1/177: rename d4/d8/cc to d4/da/c40 0 2024-05-27T09:29:29.162 INFO:tasks.workunit.client.1.smithi138.stdout:6/198: write f2 [1680168,51846] 0 2024-05-27T09:29:29.163 INFO:tasks.workunit.client.0.smithi119.stdout:1/726: dread d7/dd/d44/f93 [0,4194304] 0 2024-05-27T09:29:29.163 INFO:tasks.workunit.client.0.smithi119.stdout:7/686: dwrite d7/df/d19/d46/f68 [0,4194304] 0 2024-05-27T09:29:29.164 INFO:tasks.workunit.client.1.smithi138.stdout:6/199: symlink d3/l49 0 2024-05-27T09:29:29.167 INFO:tasks.workunit.client.0.smithi119.stdout:9/543: unlink d18/d19/d61/ca3 0 2024-05-27T09:29:29.167 INFO:tasks.workunit.client.0.smithi119.stdout:9/544: dread - d18/d19/d1b/d2a/d63/f9c zero size 2024-05-27T09:29:29.167 INFO:tasks.workunit.client.0.smithi119.stdout:9/545: fsync d18/d19/d1b/d40/d44/f93 0 2024-05-27T09:29:29.170 INFO:tasks.workunit.client.1.smithi138.stdout:1/178: dread d4/d8/d22/f2c [0,4194304] 0 2024-05-27T09:29:29.171 INFO:tasks.workunit.client.0.smithi119.stdout:5/708: symlink d6/dd/d46/d47/d4b/dd8/lf9 0 2024-05-27T09:29:29.171 INFO:tasks.workunit.client.0.smithi119.stdout:5/709: chown d6/db/dd4 47 1 2024-05-27T09:29:29.171 INFO:tasks.workunit.client.0.smithi119.stdout:5/710: stat d6/db/c5e 0 2024-05-27T09:29:29.171 INFO:tasks.workunit.client.0.smithi119.stdout:5/711: chown d6/dd/d39/d74/f7a 1550508168 1 2024-05-27T09:29:29.172 INFO:tasks.workunit.client.0.smithi119.stdout:7/687: mkdir d7/df/d19/d97/d92/db7/ddc 0 2024-05-27T09:29:29.172 INFO:tasks.workunit.client.0.smithi119.stdout:2/675: dwrite d2/f2d [0,4194304] 0 2024-05-27T09:29:29.173 INFO:tasks.workunit.client.1.smithi138.stdout:0/169: dread da/f11 [0,4194304] 0 2024-05-27T09:29:29.173 INFO:tasks.workunit.client.1.smithi138.stdout:0/170: stat da/c14 0 2024-05-27T09:29:29.173 INFO:tasks.workunit.client.0.smithi119.stdout:9/546: creat d18/d19/d1b/d29/fac x:0 0 0 2024-05-27T09:29:29.173 INFO:tasks.workunit.client.0.smithi119.stdout:9/547: readlink d18/d19/l8c 0 2024-05-27T09:29:29.175 INFO:tasks.workunit.client.0.smithi119.stdout:5/712: symlink d6/dd/d39/d19/d7d/db7/lfa 0 2024-05-27T09:29:29.175 INFO:tasks.workunit.client.0.smithi119.stdout:1/727: dread d7/db/d12/d26/d39/fb1 [0,4194304] 0 2024-05-27T09:29:29.176 INFO:tasks.workunit.client.0.smithi119.stdout:7/688: rmdir d7/df/d19/d8d 39 2024-05-27T09:29:29.177 INFO:tasks.workunit.client.0.smithi119.stdout:2/676: read - d2/fa9 zero size 2024-05-27T09:29:29.178 INFO:tasks.workunit.client.0.smithi119.stdout:9/548: symlink d18/d19/d1b/d2a/lad 0 2024-05-27T09:29:29.178 INFO:tasks.workunit.client.0.smithi119.stdout:9/549: stat d18/d19/d1b/d29/la4 0 2024-05-27T09:29:29.178 INFO:tasks.workunit.client.1.smithi138.stdout:5/140: dwrite d3/d12/d16/f2e [0,4194304] 0 2024-05-27T09:29:29.178 INFO:tasks.workunit.client.0.smithi119.stdout:5/713: unlink d6/db/c5e 0 2024-05-27T09:29:29.179 INFO:tasks.workunit.client.0.smithi119.stdout:5/714: read d6/db/de/d27/d77/f9f [155918,18088] 0 2024-05-27T09:29:29.179 INFO:tasks.workunit.client.0.smithi119.stdout:5/715: chown d6/dd/d46/d47/d4b/dd8/lf9 1 1 2024-05-27T09:29:29.179 INFO:tasks.workunit.client.0.smithi119.stdout:2/677: creat d2/da/d8a/dd4/d89/d94/d87/dcf/de5/fea x:0 0 0 2024-05-27T09:29:29.180 INFO:tasks.workunit.client.1.smithi138.stdout:7/122: write d0/d7/fe [3888183,105649] 0 2024-05-27T09:29:29.180 INFO:tasks.workunit.client.1.smithi138.stdout:4/101: dwrite d7/df/f1a [0,4194304] 0 2024-05-27T09:29:29.180 INFO:tasks.workunit.client.1.smithi138.stdout:4/102: fsync d7/fb 0 2024-05-27T09:29:29.181 INFO:tasks.workunit.client.1.smithi138.stdout:3/73: dwrite d0/d2/fc [0,4194304] 0 2024-05-27T09:29:29.181 INFO:tasks.workunit.client.1.smithi138.stdout:5/141: creat d3/d12/d26/f2f x:0 0 0 2024-05-27T09:29:29.182 INFO:tasks.workunit.client.0.smithi119.stdout:2/678: rmdir d2/da/d8a/d8f 39 2024-05-27T09:29:29.188 INFO:tasks.workunit.client.1.smithi138.stdout:7/123: unlink d0/d23/c27 0 2024-05-27T09:29:29.188 INFO:tasks.workunit.client.1.smithi138.stdout:7/124: chown d0/d7/l14 62579108 1 2024-05-27T09:29:29.190 INFO:tasks.workunit.client.0.smithi119.stdout:0/603: dwrite d2/d6/d9/f27 [4194304,4194304] 0 2024-05-27T09:29:29.191 INFO:tasks.workunit.client.0.smithi119.stdout:5/716: truncate d6/dd/d39/d19/d70/dc1/f54 2008812 0 2024-05-27T09:29:29.191 INFO:tasks.workunit.client.0.smithi119.stdout:5/717: write d6/dd/d39/d74/fa1 [584591,63140] 0 2024-05-27T09:29:29.191 INFO:tasks.workunit.client.0.smithi119.stdout:5/718: chown d6/db/de/d27/l38 1534150 1 2024-05-27T09:29:29.191 INFO:tasks.workunit.client.0.smithi119.stdout:5/719: dread - d6/dd/d39/d74/f7a zero size 2024-05-27T09:29:29.191 INFO:tasks.workunit.client.0.smithi119.stdout:4/694: dwrite d4/d23/d31/fcc [0,4194304] 0 2024-05-27T09:29:29.191 INFO:tasks.workunit.client.0.smithi119.stdout:4/695: truncate d4/d48/f51 1643741 0 2024-05-27T09:29:29.197 INFO:tasks.workunit.client.1.smithi138.stdout:1/179: write d4/f18 [6637871,93892] 0 2024-05-27T09:29:29.197 INFO:tasks.workunit.client.0.smithi119.stdout:0/604: link d2/d6/d9/d2f/l81 d2/d6/d75/lc0 0 2024-05-27T09:29:29.199 INFO:tasks.workunit.client.1.smithi138.stdout:3/74: mknod d0/d2/da/df/c19 0 2024-05-27T09:29:29.200 INFO:tasks.workunit.client.0.smithi119.stdout:0/605: rename d2/d6/d23/d7a/d53/f54 to d2/d6/d9/d2f/fc1 0 2024-05-27T09:29:29.201 INFO:tasks.workunit.client.1.smithi138.stdout:5/142: mkdir d3/d12/d16/d1d/d30 0 2024-05-27T09:29:29.201 INFO:tasks.workunit.client.1.smithi138.stdout:5/143: stat d3/f1b 0 2024-05-27T09:29:29.206 INFO:tasks.workunit.client.1.smithi138.stdout:3/75: read d0/fd [544608,114077] 0 2024-05-27T09:29:29.207 INFO:tasks.workunit.client.0.smithi119.stdout:0/606: creat d2/d6/d9/d12/d4f/fc2 x:0 0 0 2024-05-27T09:29:29.211 INFO:tasks.workunit.client.0.smithi119.stdout:8/609: dwrite d3/d8/d1b/d46/f2f [0,4194304] 0 2024-05-27T09:29:29.215 INFO:tasks.workunit.client.1.smithi138.stdout:0/171: write da/d17/d1d/d1f/f26 [1742495,85699] 0 2024-05-27T09:29:29.216 INFO:tasks.workunit.client.1.smithi138.stdout:6/200: getdents d3/d35/d43/d1c/d32 0 2024-05-27T09:29:29.216 INFO:tasks.workunit.client.1.smithi138.stdout:6/201: write d3/d19/f34 [4944159,91858] 0 2024-05-27T09:29:29.216 INFO:tasks.workunit.client.0.smithi119.stdout:9/550: dread d18/d19/d1b/d40/d44/f54 [0,4194304] 0 2024-05-27T09:29:29.217 INFO:tasks.workunit.client.0.smithi119.stdout:9/551: truncate d18/d19/d1b/d2a/f7a 5908 0 2024-05-27T09:29:29.222 INFO:tasks.workunit.client.0.smithi119.stdout:5/720: dread d6/dd/f73 [0,4194304] 0 2024-05-27T09:29:29.223 INFO:tasks.workunit.client.1.smithi138.stdout:6/202: link d3/d19/f1e d3/d11/d2d/d37/f4a 0 2024-05-27T09:29:29.223 INFO:tasks.workunit.client.1.smithi138.stdout:6/203: fsync d3/f17 0 2024-05-27T09:29:29.223 INFO:tasks.workunit.client.1.smithi138.stdout:6/204: chown d3/c9 7919453 1 2024-05-27T09:29:29.223 INFO:tasks.workunit.client.1.smithi138.stdout:6/205: chown d3/d35/d43/d1c/d32/d47 100 1 2024-05-27T09:29:29.224 INFO:tasks.workunit.client.0.smithi119.stdout:4/696: dread d4/d6/d28/f7a [0,4194304] 0 2024-05-27T09:29:29.224 INFO:tasks.workunit.client.0.smithi119.stdout:4/697: truncate d4/de/d21/d40/d47/d5c/fd5 583522 0 2024-05-27T09:29:29.224 INFO:tasks.workunit.client.1.smithi138.stdout:4/103: write d7/df/f1a [3911668,23178] 0 2024-05-27T09:29:29.229 INFO:tasks.workunit.client.1.smithi138.stdout:6/206: truncate d3/d19/f42 3336100 0 2024-05-27T09:29:29.229 INFO:tasks.workunit.client.0.smithi119.stdout:0/607: truncate d2/d6/d9/d2f/fc1 156454 0 2024-05-27T09:29:29.231 INFO:tasks.workunit.client.1.smithi138.stdout:6/207: write d3/d19/f28 [488048,32334] 0 2024-05-27T09:29:29.231 INFO:tasks.workunit.client.1.smithi138.stdout:6/208: write d3/d19/f1e [937003,4755] 0 2024-05-27T09:29:29.233 INFO:tasks.workunit.client.0.smithi119.stdout:8/610: link d3/d8/c27 d3/d8/d1b/db1/cc6 0 2024-05-27T09:29:29.234 INFO:tasks.workunit.client.0.smithi119.stdout:3/793: dwrite db/fc [0,4194304] 0 2024-05-27T09:29:29.234 INFO:tasks.workunit.client.0.smithi119.stdout:8/611: creat d3/d8/d4d/d79/fc7 x:0 0 0 2024-05-27T09:29:29.241 INFO:tasks.workunit.client.0.smithi119.stdout:9/552: rename d18/d19/l6f to d18/d19/d1b/d40/lae 0 2024-05-27T09:29:29.243 INFO:tasks.workunit.client.1.smithi138.stdout:4/104: mkdir d7/df/d20 0 2024-05-27T09:29:29.243 INFO:tasks.workunit.client.0.smithi119.stdout:5/721: symlink d6/db/de/d27/dc9/dae/lfb 0 2024-05-27T09:29:29.249 INFO:tasks.workunit.client.0.smithi119.stdout:8/612: symlink d3/de/lc8 0 2024-05-27T09:29:29.253 INFO:tasks.workunit.client.0.smithi119.stdout:5/722: creat d6/dd/d46/dba/ffc x:0 0 0 2024-05-27T09:29:29.254 INFO:tasks.workunit.client.0.smithi119.stdout:8/613: mkdir d3/d39/d5a/dc9 0 2024-05-27T09:29:29.256 INFO:tasks.workunit.client.1.smithi138.stdout:4/105: link d7/df/c1c d7/df/c21 0 2024-05-27T09:29:29.256 INFO:tasks.workunit.client.1.smithi138.stdout:4/106: fsync d7/f17 0 2024-05-27T09:29:29.258 INFO:tasks.workunit.client.0.smithi119.stdout:9/553: rmdir d18/d19/d61 39 2024-05-27T09:29:29.262 INFO:tasks.workunit.client.0.smithi119.stdout:5/723: mkdir d6/d2f/dfd 0 2024-05-27T09:29:29.262 INFO:tasks.workunit.client.0.smithi119.stdout:4/698: dread d4/d1c/f29 [0,4194304] 0 2024-05-27T09:29:29.263 INFO:tasks.workunit.client.0.smithi119.stdout:9/554: write d18/d19/d1b/f1e [136478,80073] 0 2024-05-27T09:29:29.263 INFO:tasks.workunit.client.0.smithi119.stdout:7/689: dwrite d7/df/d19/d8d/fa3 [0,4194304] 0 2024-05-27T09:29:29.264 INFO:tasks.workunit.client.0.smithi119.stdout:7/690: chown d7/d1a/f2d 1312 1 2024-05-27T09:29:29.264 INFO:tasks.workunit.client.0.smithi119.stdout:1/728: dwrite d7/db/d35/fa0 [4194304,4194304] 0 2024-05-27T09:29:29.265 INFO:tasks.workunit.client.1.smithi138.stdout:9/82: sync 2024-05-27T09:29:29.266 INFO:tasks.workunit.client.1.smithi138.stdout:2/107: sync 2024-05-27T09:29:29.266 INFO:tasks.workunit.client.1.smithi138.stdout:8/109: sync 2024-05-27T09:29:29.266 INFO:tasks.workunit.client.1.smithi138.stdout:9/83: truncate d6/f8 4772441 0 2024-05-27T09:29:29.267 INFO:tasks.workunit.client.0.smithi119.stdout:5/724: symlink d6/db/de/d27/dc9/dae/lfe 0 2024-05-27T09:29:29.269 INFO:tasks.workunit.client.1.smithi138.stdout:4/107: rename d7/f13 to d7/df/d20/f22 0 2024-05-27T09:29:29.272 INFO:tasks.workunit.client.0.smithi119.stdout:4/699: rename d4/d6/d81/dce/d88/f95 to d4/d6/fe0 0 2024-05-27T09:29:29.272 INFO:tasks.workunit.client.0.smithi119.stdout:4/700: getdents d4/de/d21/d40/d47/d5c/d9e/dd3 0 2024-05-27T09:29:29.272 INFO:tasks.workunit.client.0.smithi119.stdout:4/701: readlink d4/d1c/d39/la6 0 2024-05-27T09:29:29.272 INFO:tasks.workunit.client.0.smithi119.stdout:4/702: chown d4/d6/d28/d2f/d36/f94 791673 1 2024-05-27T09:29:29.272 INFO:tasks.workunit.client.0.smithi119.stdout:4/703: chown d4/de/d21/d40/fc6 665457 1 2024-05-27T09:29:29.274 INFO:tasks.workunit.client.0.smithi119.stdout:9/555: mknod d18/d19/d1b/d40/d7b/caf 0 2024-05-27T09:29:29.274 INFO:tasks.workunit.client.0.smithi119.stdout:9/556: readlink d18/d19/d61/l82 0 2024-05-27T09:29:29.275 INFO:tasks.workunit.client.0.smithi119.stdout:2/679: dwrite d2/da/d2c/fe1 [0,4194304] 0 2024-05-27T09:29:29.277 INFO:tasks.workunit.client.1.smithi138.stdout:5/144: dwrite d3/d12/f17 [0,4194304] 0 2024-05-27T09:29:29.282 INFO:tasks.workunit.client.1.smithi138.stdout:3/76: dwrite d0/fd [0,4194304] 0 2024-05-27T09:29:29.283 INFO:tasks.workunit.client.0.smithi119.stdout:1/729: truncate d7/f5b 2255909 0 2024-05-27T09:29:29.283 INFO:tasks.workunit.client.0.smithi119.stdout:4/704: getdents d4/db1 0 2024-05-27T09:29:29.285 INFO:tasks.workunit.client.1.smithi138.stdout:6/209: rename d3/d19/f42 to d3/d19/f4b 0 2024-05-27T09:29:29.286 INFO:tasks.workunit.client.1.smithi138.stdout:8/110: truncate d3/d8/fc 2702473 0 2024-05-27T09:29:29.287 INFO:tasks.workunit.client.1.smithi138.stdout:4/108: dread f0 [0,4194304] 0 2024-05-27T09:29:29.287 INFO:tasks.workunit.client.1.smithi138.stdout:4/109: readlink d7/df/l14 0 2024-05-27T09:29:29.287 INFO:tasks.workunit.client.1.smithi138.stdout:4/110: read - d7/df/f1d zero size 2024-05-27T09:29:29.288 INFO:tasks.workunit.client.0.smithi119.stdout:7/691: rename d7/df/d19/ca7 to d7/df/d19/d4d/cdd 0 2024-05-27T09:29:29.288 INFO:tasks.workunit.client.0.smithi119.stdout:7/692: write d7/df/d19/d97/d92/db7/dba/d42/d64/f7a [103534,101225] 0 2024-05-27T09:29:29.289 INFO:tasks.workunit.client.1.smithi138.stdout:5/145: creat d3/d12/f31 x:0 0 0 2024-05-27T09:29:29.289 INFO:tasks.workunit.client.1.smithi138.stdout:5/146: write d3/f11 [3863322,67317] 0 2024-05-27T09:29:29.289 INFO:tasks.workunit.client.1.smithi138.stdout:2/108: mknod d1/d10/c29 0 2024-05-27T09:29:29.290 INFO:tasks.workunit.client.1.smithi138.stdout:2/109: stat d1/d13/d17/d25 0 2024-05-27T09:29:29.290 INFO:tasks.workunit.client.0.smithi119.stdout:9/557: creat d18/d19/d1b/d40/d44/fb0 x:0 0 0 2024-05-27T09:29:29.290 INFO:tasks.workunit.client.0.smithi119.stdout:9/558: chown d18/d19/f85 586278 1 2024-05-27T09:29:29.292 INFO:tasks.workunit.client.1.smithi138.stdout:3/77: getdents d0/d2/da/df 0 2024-05-27T09:29:29.292 INFO:tasks.workunit.client.1.smithi138.stdout:6/210: creat d3/d35/d43/d1c/f4c x:0 0 0 2024-05-27T09:29:29.292 INFO:tasks.workunit.client.1.smithi138.stdout:6/211: creat d3/d35/d43/d1c/f4d x:0 0 0 2024-05-27T09:29:29.293 INFO:tasks.workunit.client.1.smithi138.stdout:8/111: getdents d3/d8 0 2024-05-27T09:29:29.295 INFO:tasks.workunit.client.1.smithi138.stdout:9/84: link d6/db/dc/cf d6/db/de/c19 0 2024-05-27T09:29:29.296 INFO:tasks.workunit.client.1.smithi138.stdout:9/85: fsync f4 0 2024-05-27T09:29:29.296 INFO:tasks.workunit.client.1.smithi138.stdout:5/147: mkdir d3/d12/d32 0 2024-05-27T09:29:29.299 INFO:tasks.workunit.client.1.smithi138.stdout:6/212: mknod d3/d11/c4e 0 2024-05-27T09:29:29.303 INFO:tasks.workunit.client.1.smithi138.stdout:9/86: mkdir d6/db/de/d18/d1a 0 2024-05-27T09:29:29.303 INFO:tasks.workunit.client.1.smithi138.stdout:9/87: creat d6/db/f1b x:0 0 0 2024-05-27T09:29:29.303 INFO:tasks.workunit.client.1.smithi138.stdout:5/148: link d3/d12/d26/f2f d3/d12/d26/f33 0 2024-05-27T09:29:29.308 INFO:tasks.workunit.client.1.smithi138.stdout:6/213: link d3/d11/d2d/d37/f3c d3/d11/f4f 0 2024-05-27T09:29:29.309 INFO:tasks.workunit.client.1.smithi138.stdout:0/172: dwrite da/f20 [0,4194304] 0 2024-05-27T09:29:29.310 INFO:tasks.workunit.client.1.smithi138.stdout:9/88: unlink d6/db/de/l13 0 2024-05-27T09:29:29.311 INFO:tasks.workunit.client.1.smithi138.stdout:8/112: unlink d3/c13 0 2024-05-27T09:29:29.311 INFO:tasks.workunit.client.1.smithi138.stdout:8/113: chown d3 634458 1 2024-05-27T09:29:29.311 INFO:tasks.workunit.client.1.smithi138.stdout:6/214: symlink d3/d19/l50 0 2024-05-27T09:29:29.313 INFO:tasks.workunit.client.0.smithi119.stdout:0/608: dwrite d2/d6/d9/f3a [0,4194304] 0 2024-05-27T09:29:29.318 INFO:tasks.workunit.client.1.smithi138.stdout:9/89: creat d6/db/dc/f1c x:0 0 0 2024-05-27T09:29:29.318 INFO:tasks.workunit.client.1.smithi138.stdout:6/215: creat d3/d35/d43/d1c/d32/d47/f51 x:0 0 0 2024-05-27T09:29:29.318 INFO:tasks.workunit.client.1.smithi138.stdout:9/90: chown c5 9 1 2024-05-27T09:29:29.320 INFO:tasks.workunit.client.1.smithi138.stdout:4/111: rename d7/df/d20/f22 to d7/f23 0 2024-05-27T09:29:29.325 INFO:tasks.workunit.client.1.smithi138.stdout:9/91: symlink d6/db/dc/l1d 0 2024-05-27T09:29:29.325 INFO:tasks.workunit.client.1.smithi138.stdout:9/92: creat d6/db/dc/f1e x:0 0 0 2024-05-27T09:29:29.325 INFO:tasks.workunit.client.1.smithi138.stdout:9/93: dread - d6/db/dc/f1c zero size 2024-05-27T09:29:29.326 INFO:tasks.workunit.client.1.smithi138.stdout:9/94: dread - d6/db/dc/f1e zero size 2024-05-27T09:29:29.329 INFO:tasks.workunit.client.1.smithi138.stdout:2/110: dread d1/d15/f27 [4194304,4194304] 0 2024-05-27T09:29:29.329 INFO:tasks.workunit.client.1.smithi138.stdout:3/78: dread d0/d2/f13 [0,4194304] 0 2024-05-27T09:29:29.329 INFO:tasks.workunit.client.1.smithi138.stdout:9/95: rename f4 to d6/f1f 0 2024-05-27T09:29:29.330 INFO:tasks.workunit.client.1.smithi138.stdout:2/111: fsync d1/d10/f1a 0 2024-05-27T09:29:29.330 INFO:tasks.workunit.client.1.smithi138.stdout:2/112: creat d1/d13/d17/d25/f2a x:0 0 0 2024-05-27T09:29:29.331 INFO:tasks.workunit.client.1.smithi138.stdout:2/113: write d1/d13/d17/d25/f28 [882904,124995] 0 2024-05-27T09:29:29.331 INFO:tasks.workunit.client.1.smithi138.stdout:2/114: truncate d1/d13/d17/d25/f28 1948166 0 2024-05-27T09:29:29.331 INFO:tasks.workunit.client.1.smithi138.stdout:2/115: creat d1/d13/d17/d25/f2b x:0 0 0 2024-05-27T09:29:29.331 INFO:tasks.workunit.client.1.smithi138.stdout:2/116: truncate d1/d13/d17/d25/f2a 370819 0 2024-05-27T09:29:29.335 INFO:tasks.workunit.client.1.smithi138.stdout:9/96: rmdir d6/db/de/d18/d1a 0 2024-05-27T09:29:29.344 INFO:tasks.workunit.client.0.smithi119.stdout:3/794: dwrite db/dc9/d98/d7a/f5c [0,4194304] 0 2024-05-27T09:29:29.344 INFO:tasks.workunit.client.1.smithi138.stdout:2/117: unlink d1/d10/c29 0 2024-05-27T09:29:29.344 INFO:tasks.workunit.client.1.smithi138.stdout:4/112: dread f3 [0,4194304] 0 2024-05-27T09:29:29.345 INFO:tasks.workunit.client.0.smithi119.stdout:1/730: creat d7/db/fe1 x:0 0 0 2024-05-27T09:29:29.346 INFO:tasks.workunit.client.1.smithi138.stdout:4/113: rename d7/fc to d7/df/f24 0 2024-05-27T09:29:29.346 INFO:tasks.workunit.client.1.smithi138.stdout:4/114: write d7/df/f24 [489842,130741] 0 2024-05-27T09:29:29.346 INFO:tasks.workunit.client.1.smithi138.stdout:4/115: read d7/df/f1a [3033482,22330] 0 2024-05-27T09:29:29.346 INFO:tasks.workunit.client.1.smithi138.stdout:4/116: readlink d7/df/l14 0 2024-05-27T09:29:29.346 INFO:tasks.workunit.client.1.smithi138.stdout:4/117: fsync d7/df/f1d 0 2024-05-27T09:29:29.353 INFO:tasks.workunit.client.1.smithi138.stdout:0/173: dread da/d2a/f30 [0,4194304] 0 2024-05-27T09:29:29.353 INFO:tasks.workunit.client.1.smithi138.stdout:4/118: unlink d7/df/f11 0 2024-05-27T09:29:29.354 INFO:tasks.workunit.client.0.smithi119.stdout:4/705: rename d4/d1c/fc5 to d4/de/d21/d40/d47/d5c/d9e/fe1 0 2024-05-27T09:29:29.355 INFO:tasks.workunit.client.0.smithi119.stdout:4/706: readlink d4/de/d66/ld6 0 2024-05-27T09:29:29.355 INFO:tasks.workunit.client.0.smithi119.stdout:0/609: getdents d2/d6/d9/d24 0 2024-05-27T09:29:29.363 INFO:tasks.workunit.client.1.smithi138.stdout:3/79: dread d0/d2/f13 [0,4194304] 0 2024-05-27T09:29:29.369 INFO:tasks.workunit.client.0.smithi119.stdout:5/725: dwrite d6/dd/f73 [4194304,4194304] 0 2024-05-27T09:29:29.370 INFO:tasks.workunit.client.1.smithi138.stdout:0/174: write da/d17/d1d/d1f/f3b [1067456,82344] 0 2024-05-27T09:29:29.370 INFO:tasks.workunit.client.1.smithi138.stdout:0/175: chown da/c39 1157656 1 2024-05-27T09:29:29.370 INFO:tasks.workunit.client.1.smithi138.stdout:0/176: chown da/d2a/f30 19 1 2024-05-27T09:29:29.374 INFO:tasks.workunit.client.1.smithi138.stdout:1/180: dwrite d4/f7 [0,4194304] 0 2024-05-27T09:29:29.375 INFO:tasks.workunit.client.1.smithi138.stdout:1/181: getdents d4/d1a 0 2024-05-27T09:29:29.379 INFO:tasks.workunit.client.1.smithi138.stdout:1/182: fdatasync d4/f37 0 2024-05-27T09:29:29.382 INFO:tasks.workunit.client.1.smithi138.stdout:6/216: read d3/f17 [3810286,118891] 0 2024-05-27T09:29:29.383 INFO:tasks.workunit.client.0.smithi119.stdout:2/680: rmdir d2/da/d8a 39 2024-05-27T09:29:29.384 INFO:tasks.workunit.client.1.smithi138.stdout:3/80: dread d0/d2/f13 [0,4194304] 0 2024-05-27T09:29:29.384 INFO:tasks.workunit.client.1.smithi138.stdout:6/217: mknod d3/d35/d43/d1c/c52 0 2024-05-27T09:29:29.388 INFO:tasks.workunit.client.1.smithi138.stdout:3/81: rename d0/d2/da/df/d10/l15 to d0/d2/da/df/l1a 0 2024-05-27T09:29:29.388 INFO:tasks.workunit.client.0.smithi119.stdout:9/559: dwrite d18/d19/d1b/d2a/d63/f97 [0,4194304] 0 2024-05-27T09:29:29.390 INFO:tasks.workunit.client.0.smithi119.stdout:3/795: link db/d36/d45/d4d/f58 db/d36/d45/d4c/d62/d13/d33/f104 0 2024-05-27T09:29:29.390 INFO:tasks.workunit.client.0.smithi119.stdout:3/796: stat db/d36/d45/d4c/d62/f38 0 2024-05-27T09:29:29.390 INFO:tasks.workunit.client.0.smithi119.stdout:3/797: fsync db/f83 0 2024-05-27T09:29:29.392 INFO:tasks.workunit.client.1.smithi138.stdout:6/218: stat d3/d19/c2f 0 2024-05-27T09:29:29.394 INFO:tasks.workunit.client.1.smithi138.stdout:6/219: creat d3/f53 x:0 0 0 2024-05-27T09:29:29.394 INFO:tasks.workunit.client.1.smithi138.stdout:6/220: dread - d3/d19/f3d zero size 2024-05-27T09:29:29.395 INFO:tasks.workunit.client.1.smithi138.stdout:6/221: write d3/d19/f1e [1281571,97622] 0 2024-05-27T09:29:29.396 INFO:tasks.workunit.client.1.smithi138.stdout:9/97: dread f1 [0,4194304] 0 2024-05-27T09:29:29.396 INFO:tasks.workunit.client.1.smithi138.stdout:9/98: dread - d6/db/f14 zero size 2024-05-27T09:29:29.397 INFO:tasks.workunit.client.0.smithi119.stdout:1/731: creat d7/d6e/fe2 x:0 0 0 2024-05-27T09:29:29.397 INFO:tasks.workunit.client.0.smithi119.stdout:1/732: fdatasync d7/d6e/fe2 0 2024-05-27T09:29:29.397 INFO:tasks.workunit.client.0.smithi119.stdout:1/733: truncate d7/db/d8f/fc4 653002 0 2024-05-27T09:29:29.405 INFO:tasks.workunit.client.1.smithi138.stdout:4/119: rmdir d7 39 2024-05-27T09:29:29.408 INFO:tasks.workunit.client.1.smithi138.stdout:4/120: creat d7/f25 x:0 0 0 2024-05-27T09:29:29.410 INFO:tasks.workunit.client.0.smithi119.stdout:3/798: dread db/dc9/d98/f73 [0,4194304] 0 2024-05-27T09:29:29.410 INFO:tasks.workunit.client.0.smithi119.stdout:2/681: write d2/f19 [1548824,79738] 0 2024-05-27T09:29:29.411 INFO:tasks.workunit.client.0.smithi119.stdout:7/693: rename d7/df/d19/f1b to d7/df/d19/d9a/da6/dda/d44/fde 0 2024-05-27T09:29:29.411 INFO:tasks.workunit.client.0.smithi119.stdout:7/694: chown d7/fa 387 1 2024-05-27T09:29:29.411 INFO:tasks.workunit.client.0.smithi119.stdout:7/695: readlink d7/df/d19/d1d/lc7 0 2024-05-27T09:29:29.411 INFO:tasks.workunit.client.0.smithi119.stdout:7/696: truncate d7/d5f/d79/fc6 632844 0 2024-05-27T09:29:29.412 INFO:tasks.workunit.client.1.smithi138.stdout:4/121: unlink l6 0 2024-05-27T09:29:29.412 INFO:tasks.workunit.client.1.smithi138.stdout:4/122: creat d7/f26 x:0 0 0 2024-05-27T09:29:29.412 INFO:tasks.workunit.client.1.smithi138.stdout:9/99: write d6/f8 [2236103,110998] 0 2024-05-27T09:29:29.413 INFO:tasks.workunit.client.1.smithi138.stdout:9/100: fdatasync f1 0 2024-05-27T09:29:29.413 INFO:tasks.workunit.client.1.smithi138.stdout:6/222: dread d3/d19/f22 [0,4194304] 0 2024-05-27T09:29:29.415 INFO:tasks.workunit.client.1.smithi138.stdout:4/123: mkdir d7/d27 0 2024-05-27T09:29:29.415 INFO:tasks.workunit.client.1.smithi138.stdout:4/124: getdents d7/d27 0 2024-05-27T09:29:29.416 INFO:tasks.workunit.client.1.smithi138.stdout:9/101: symlink d6/db/de/l20 0 2024-05-27T09:29:29.416 INFO:tasks.workunit.client.1.smithi138.stdout:9/102: dread - d6/db/f1b zero size 2024-05-27T09:29:29.419 INFO:tasks.workunit.client.0.smithi119.stdout:4/707: creat d4/d23/d24/d5b/da8/dda/fe2 x:0 0 0 2024-05-27T09:29:29.419 INFO:tasks.workunit.client.0.smithi119.stdout:4/708: chown d4/d4e/fb7 1375143590 1 2024-05-27T09:29:29.421 INFO:tasks.workunit.client.0.smithi119.stdout:0/610: unlink d2/d6/d9/f88 0 2024-05-27T09:29:29.422 INFO:tasks.workunit.client.0.smithi119.stdout:0/611: chown d2/d6/d9/c7f 58 1 2024-05-27T09:29:29.422 INFO:tasks.workunit.client.1.smithi138.stdout:6/223: rmdir d3/d35/d43 39 2024-05-27T09:29:29.422 INFO:tasks.workunit.client.1.smithi138.stdout:6/224: truncate d3/d19/f34 5796944 0 2024-05-27T09:29:29.422 INFO:tasks.workunit.client.1.smithi138.stdout:5/149: dwrite d3/f15 [0,4194304] 0 2024-05-27T09:29:29.422 INFO:tasks.workunit.client.1.smithi138.stdout:5/150: dread - d3/d12/d26/f2f zero size 2024-05-27T09:29:29.422 INFO:tasks.workunit.client.1.smithi138.stdout:9/103: symlink d6/db/l21 0 2024-05-27T09:29:29.424 INFO:tasks.workunit.client.1.smithi138.stdout:1/183: dread d4/d8/f14 [0,4194304] 0 2024-05-27T09:29:29.429 INFO:tasks.workunit.client.0.smithi119.stdout:5/726: creat d6/dd/fff x:0 0 0 2024-05-27T09:29:29.430 INFO:tasks.workunit.client.0.smithi119.stdout:6/671: sync 2024-05-27T09:29:29.433 INFO:tasks.workunit.client.1.smithi138.stdout:3/82: dwrite d0/fd [0,4194304] 0 2024-05-27T09:29:29.435 INFO:tasks.workunit.client.0.smithi119.stdout:1/734: symlink d7/db6/le3 0 2024-05-27T09:29:29.435 INFO:tasks.workunit.client.0.smithi119.stdout:1/735: write d7/db/d52/fcc [539414,69599] 0 2024-05-27T09:29:29.438 INFO:tasks.workunit.client.1.smithi138.stdout:8/114: dwrite f1 [0,4194304] 0 2024-05-27T09:29:29.439 INFO:tasks.workunit.client.0.smithi119.stdout:3/799: mkdir db/dc9/d98/d105 0 2024-05-27T09:29:29.442 INFO:tasks.workunit.client.1.smithi138.stdout:5/151: dread d3/f18 [4194304,4194304] 0 2024-05-27T09:29:29.445 INFO:tasks.workunit.client.1.smithi138.stdout:3/83: truncate d0/d2/f13 3397104 0 2024-05-27T09:29:29.445 INFO:tasks.workunit.client.1.smithi138.stdout:3/84: creat d0/d2/da/f1b x:0 0 0 2024-05-27T09:29:29.446 INFO:tasks.workunit.client.1.smithi138.stdout:4/125: getdents d7 0 2024-05-27T09:29:29.446 INFO:tasks.workunit.client.1.smithi138.stdout:4/126: truncate d7/f23 39196 0 2024-05-27T09:29:29.446 INFO:tasks.workunit.client.1.smithi138.stdout:4/127: readlink d7/df/l14 0 2024-05-27T09:29:29.447 INFO:tasks.workunit.client.1.smithi138.stdout:9/104: rmdir d6 39 2024-05-27T09:29:29.447 INFO:tasks.workunit.client.1.smithi138.stdout:9/105: write f3 [915250,123358] 0 2024-05-27T09:29:29.447 INFO:tasks.workunit.client.1.smithi138.stdout:9/106: chown d6/c7 457035 1 2024-05-27T09:29:29.447 INFO:tasks.workunit.client.1.smithi138.stdout:4/128: fsync d7/df/f19 0 2024-05-27T09:29:29.448 INFO:tasks.workunit.client.0.smithi119.stdout:7/697: link d7/df/d19/d46/f85 d7/df/d19/d97/d92/db7/dba/d42/fdf 0 2024-05-27T09:29:29.449 INFO:tasks.workunit.client.1.smithi138.stdout:6/225: dread d3/d35/d43/f29 [0,4194304] 0 2024-05-27T09:29:29.449 INFO:tasks.workunit.client.1.smithi138.stdout:6/226: chown d3/d11/d2d/f39 98 1 2024-05-27T09:29:29.449 INFO:tasks.workunit.client.1.smithi138.stdout:5/152: symlink d3/l34 0 2024-05-27T09:29:29.454 INFO:tasks.workunit.client.0.smithi119.stdout:1/736: dread d7/dd/d44/f93 [0,4194304] 0 2024-05-27T09:29:29.460 INFO:tasks.workunit.client.0.smithi119.stdout:0/612: mknod d2/d6/d75/cc3 0 2024-05-27T09:29:29.464 INFO:tasks.workunit.client.0.smithi119.stdout:5/727: mkdir d6/d2f/d9d/d100 0 2024-05-27T09:29:29.464 INFO:tasks.workunit.client.0.smithi119.stdout:5/728: chown d6/dd/d39/d19/d7d/db7/ccc 831596 1 2024-05-27T09:29:29.464 INFO:tasks.workunit.client.0.smithi119.stdout:5/729: stat d6/dd/d39/d19/d70 0 2024-05-27T09:29:29.464 INFO:tasks.workunit.client.0.smithi119.stdout:5/730: chown d6/dd/d39/d19/d7d/ca6 1 1 2024-05-27T09:29:29.469 INFO:tasks.workunit.client.1.smithi138.stdout:3/85: mknod d0/d2/da/c1c 0 2024-05-27T09:29:29.469 INFO:tasks.workunit.client.1.smithi138.stdout:3/86: stat d0/d2/fc 0 2024-05-27T09:29:29.469 INFO:tasks.workunit.client.1.smithi138.stdout:0/177: dwrite da/d17/d1d/d1f/f26 [4194304,4194304] 0 2024-05-27T09:29:29.470 INFO:tasks.workunit.client.0.smithi119.stdout:9/560: link d18/d19/d1b/d29/f9e d18/d19/fb1 0 2024-05-27T09:29:29.472 INFO:tasks.workunit.client.0.smithi119.stdout:3/800: creat db/d36/d45/d4c/d62/f106 x:0 0 0 2024-05-27T09:29:29.474 INFO:tasks.workunit.client.0.smithi119.stdout:2/682: mkdir d2/deb 0 2024-05-27T09:29:29.477 INFO:tasks.workunit.client.0.smithi119.stdout:7/698: mknod d7/df/d19/d97/d92/db7/dba/d42/dc5/ce0 0 2024-05-27T09:29:29.480 INFO:tasks.workunit.client.1.smithi138.stdout:7/125: sync 2024-05-27T09:29:29.481 INFO:tasks.workunit.client.0.smithi119.stdout:0/613: symlink d2/d6/d9/lc4 0 2024-05-27T09:29:29.481 INFO:tasks.workunit.client.0.smithi119.stdout:0/614: stat d2/d6/d23/d7a/f44 0 2024-05-27T09:29:29.483 INFO:tasks.workunit.client.1.smithi138.stdout:0/178: mkdir da/d2a/d3f 0 2024-05-27T09:29:29.483 INFO:tasks.workunit.client.1.smithi138.stdout:7/126: getdents d0/d23 0 2024-05-27T09:29:29.485 INFO:tasks.workunit.client.0.smithi119.stdout:5/731: truncate d6/dd/d6e/fdf 222926 0 2024-05-27T09:29:29.486 INFO:tasks.workunit.client.0.smithi119.stdout:5/732: chown d6/dd/d39/d19/d70/dc1 688 1 2024-05-27T09:29:29.486 INFO:tasks.workunit.client.1.smithi138.stdout:6/227: truncate d3/d11/d2d/d37/f4a 1364029 0 2024-05-27T09:29:29.486 INFO:tasks.workunit.client.1.smithi138.stdout:0/179: symlink da/d17/l40 0 2024-05-27T09:29:29.486 INFO:tasks.workunit.client.1.smithi138.stdout:6/228: write d3/d35/d43/f48 [910152,62653] 0 2024-05-27T09:29:29.486 INFO:tasks.workunit.client.1.smithi138.stdout:6/229: fdatasync d3/d35/d43/f29 0 2024-05-27T09:29:29.487 INFO:tasks.workunit.client.0.smithi119.stdout:9/561: symlink d18/d19/d1b/d2a/lb2 0 2024-05-27T09:29:29.487 INFO:tasks.workunit.client.1.smithi138.stdout:0/180: rename da/d17/d1d/c24 to da/d17/d1d/d1f/c41 0 2024-05-27T09:29:29.487 INFO:tasks.workunit.client.1.smithi138.stdout:7/127: truncate d0/f10 573631 0 2024-05-27T09:29:29.489 INFO:tasks.workunit.client.1.smithi138.stdout:0/181: truncate da/d2a/f30 1607398 0 2024-05-27T09:29:29.490 INFO:tasks.workunit.client.0.smithi119.stdout:3/801: write db/d36/d45/d4d/dc8/ff9 [740439,15053] 0 2024-05-27T09:29:29.492 INFO:tasks.workunit.client.1.smithi138.stdout:0/182: mkdir da/d17/d1d/d1f/d42 0 2024-05-27T09:29:29.492 INFO:tasks.workunit.client.1.smithi138.stdout:0/183: fsync da/f3c 0 2024-05-27T09:29:29.492 INFO:tasks.workunit.client.1.smithi138.stdout:0/184: read - da/d17/d1d/d1f/f3e zero size 2024-05-27T09:29:29.493 INFO:tasks.workunit.client.1.smithi138.stdout:7/128: symlink d0/l29 0 2024-05-27T09:29:29.494 INFO:tasks.workunit.client.0.smithi119.stdout:3/802: read db/d36/f71 [2676616,82268] 0 2024-05-27T09:29:29.494 INFO:tasks.workunit.client.0.smithi119.stdout:7/699: symlink d7/df/d19/d9a/da6/dda/d44/le1 0 2024-05-27T09:29:29.494 INFO:tasks.workunit.client.0.smithi119.stdout:0/615: creat d2/d6/d9/d24/d30/d4a/fc5 x:0 0 0 2024-05-27T09:29:29.495 INFO:tasks.workunit.client.0.smithi119.stdout:2/683: write d2/da/f28 [308435,57056] 0 2024-05-27T09:29:29.496 INFO:tasks.workunit.client.0.smithi119.stdout:2/684: write d2/da/d8a/dd4/d69/f9d [4546619,93388] 0 2024-05-27T09:29:29.498 INFO:tasks.workunit.client.0.smithi119.stdout:7/700: mknod d7/df/d19/d97/d92/db7/dba/ce2 0 2024-05-27T09:29:29.499 INFO:tasks.workunit.client.1.smithi138.stdout:7/129: rename d0/d7/l1f to d0/d23/l2a 0 2024-05-27T09:29:29.499 INFO:tasks.workunit.client.0.smithi119.stdout:9/562: getdents d18/d19/d1b/d29 0 2024-05-27T09:29:29.500 INFO:tasks.workunit.client.0.smithi119.stdout:3/803: dread db/d36/d45/d4c/f87 [0,4194304] 0 2024-05-27T09:29:29.503 INFO:tasks.workunit.client.1.smithi138.stdout:7/130: rmdir d0/d23/d28 0 2024-05-27T09:29:29.503 INFO:tasks.workunit.client.0.smithi119.stdout:9/563: creat d18/d19/d9b/fb3 x:0 0 0 2024-05-27T09:29:29.503 INFO:tasks.workunit.client.0.smithi119.stdout:9/564: write d18/f23 [2606636,6777] 0 2024-05-27T09:29:29.507 INFO:tasks.workunit.client.0.smithi119.stdout:7/701: mkdir d7/df/d19/d97/d92/db7/dba/d42/d64/de3 0 2024-05-27T09:29:29.508 INFO:tasks.workunit.client.1.smithi138.stdout:7/131: mkdir d0/d2b 0 2024-05-27T09:29:29.512 INFO:tasks.workunit.client.0.smithi119.stdout:3/804: creat db/d36/d45/d4d/dc8/f107 x:0 0 0 2024-05-27T09:29:29.512 INFO:tasks.workunit.client.0.smithi119.stdout:3/805: truncate db/dc9/d98/faf 89854 0 2024-05-27T09:29:29.515 INFO:tasks.workunit.client.1.smithi138.stdout:8/115: dwrite d3/d8/f14 [0,4194304] 0 2024-05-27T09:29:29.517 INFO:tasks.workunit.client.0.smithi119.stdout:9/565: rmdir d18 39 2024-05-27T09:29:29.520 INFO:tasks.workunit.client.0.smithi119.stdout:7/702: unlink d7/df/d19/d1d/f23 0 2024-05-27T09:29:29.520 INFO:tasks.workunit.client.0.smithi119.stdout:7/703: stat d7/df/d19/d97/d92 0 2024-05-27T09:29:29.520 INFO:tasks.workunit.client.0.smithi119.stdout:7/704: fsync d7/df/d19/d97/d92/fab 0 2024-05-27T09:29:29.520 INFO:tasks.workunit.client.0.smithi119.stdout:7/705: stat d7/df/d19/d97/d81 0 2024-05-27T09:29:29.524 INFO:tasks.workunit.client.0.smithi119.stdout:3/806: truncate db/d36/d45/d4c/d62/d13/f21 104346 0 2024-05-27T09:29:29.525 INFO:tasks.workunit.client.0.smithi119.stdout:3/807: write db/dc9/d98/f73 [1082059,102608] 0 2024-05-27T09:29:29.533 INFO:tasks.workunit.client.0.smithi119.stdout:3/808: mkdir db/d36/d45/d4d/d108 0 2024-05-27T09:29:29.539 INFO:tasks.workunit.client.0.smithi119.stdout:9/566: mkdir d18/db4 0 2024-05-27T09:29:29.539 INFO:tasks.workunit.client.0.smithi119.stdout:9/567: truncate d18/d19/d1b/d40/d44/fb0 739912 0 2024-05-27T09:29:29.543 INFO:tasks.workunit.client.0.smithi119.stdout:9/568: write d18/d19/d1b/d40/d7b/f89 [256845,110137] 0 2024-05-27T09:29:29.543 INFO:tasks.workunit.client.0.smithi119.stdout:6/672: dwrite d0/d2/d1c/fd9 [0,4194304] 0 2024-05-27T09:29:29.544 INFO:tasks.workunit.client.1.smithi138.stdout:3/87: write d0/d2/f13 [3342477,90978] 0 2024-05-27T09:29:29.544 INFO:tasks.workunit.client.1.smithi138.stdout:3/88: dread - d0/f12 zero size 2024-05-27T09:29:29.559 INFO:tasks.workunit.client.0.smithi119.stdout:3/809: rmdir db/d36/d45/d4c/d62/d13/d33/dfc 39 2024-05-27T09:29:29.559 INFO:tasks.workunit.client.0.smithi119.stdout:1/737: dread d7/db/d12/ddb/d38/f90 [0,4194304] 0 2024-05-27T09:29:29.560 INFO:tasks.workunit.client.0.smithi119.stdout:1/738: readlink d7/dd/d44/d47/l64 0 2024-05-27T09:29:29.560 INFO:tasks.workunit.client.0.smithi119.stdout:1/739: stat d7/db/d12/ddb/d38/d49/caa 0 2024-05-27T09:29:29.560 INFO:tasks.workunit.client.0.smithi119.stdout:1/740: write d7/db/d12/ddb/d38/d49/fd9 [80901,9420] 0 2024-05-27T09:29:29.560 INFO:tasks.workunit.client.0.smithi119.stdout:1/741: creat d7/d6e/fe4 x:0 0 0 2024-05-27T09:29:29.561 INFO:tasks.workunit.client.1.smithi138.stdout:8/116: dread d3/d8/d11/f12 [0,4194304] 0 2024-05-27T09:29:29.561 INFO:tasks.workunit.client.1.smithi138.stdout:8/117: write d3/d8/d11/f12 [1010933,114952] 0 2024-05-27T09:29:29.573 INFO:tasks.workunit.client.0.smithi119.stdout:9/569: symlink d18/d19/d9b/lb5 0 2024-05-27T09:29:29.573 INFO:tasks.workunit.client.0.smithi119.stdout:3/810: creat db/d36/d45/d4d/d108/f109 x:0 0 0 2024-05-27T09:29:29.573 INFO:tasks.workunit.client.0.smithi119.stdout:3/811: fsync db/d36/d45/d4c/d62/d13/d33/d9b/de3/f78 0 2024-05-27T09:29:29.582 INFO:tasks.workunit.client.0.smithi119.stdout:3/812: mknod db/d36/c10a 0 2024-05-27T09:29:29.582 INFO:tasks.workunit.client.0.smithi119.stdout:3/813: truncate db/d36/d45/d49/d9d/fdc 942179 0 2024-05-27T09:29:29.586 INFO:tasks.workunit.client.1.smithi138.stdout:8/118: write d3/d8/fc [516126,103528] 0 2024-05-27T09:29:29.586 INFO:tasks.workunit.client.0.smithi119.stdout:3/814: mknod db/d36/d45/d4c/dbe/c10b 0 2024-05-27T09:29:29.586 INFO:tasks.workunit.client.0.smithi119.stdout:0/616: dwrite d2/d6/d9/d24/d30/d6b/f87 [0,4194304] 0 2024-05-27T09:29:29.587 INFO:tasks.workunit.client.0.smithi119.stdout:0/617: chown d2/d6/d9/d24/d30/d4a 16711481 1 2024-05-27T09:29:29.591 INFO:tasks.workunit.client.1.smithi138.stdout:4/129: dwrite d7/df/f1a [0,4194304] 0 2024-05-27T09:29:29.598 INFO:tasks.workunit.client.1.smithi138.stdout:1/184: dwrite d4/f12 [0,4194304] 0 2024-05-27T09:29:29.598 INFO:tasks.workunit.client.1.smithi138.stdout:1/185: fsync d4/f24 0 2024-05-27T09:29:29.600 INFO:tasks.workunit.client.0.smithi119.stdout:3/815: unlink db/d36/l4e 0 2024-05-27T09:29:29.600 INFO:tasks.workunit.client.0.smithi119.stdout:3/816: readlink db/l7d 0 2024-05-27T09:29:29.600 INFO:tasks.workunit.client.0.smithi119.stdout:3/817: dread - db/d36/d45/fcb zero size 2024-05-27T09:29:29.600 INFO:tasks.workunit.client.0.smithi119.stdout:3/818: dread - db/d36/d45/d4c/d62/d13/d33/dfc/fb7 zero size 2024-05-27T09:29:29.600 INFO:tasks.workunit.client.0.smithi119.stdout:3/819: creat db/d36/d45/d4d/d108/f10c x:0 0 0 2024-05-27T09:29:29.601 INFO:tasks.workunit.client.0.smithi119.stdout:3/820: write db/d7f/f5a [5149132,121622] 0 2024-05-27T09:29:29.602 INFO:tasks.workunit.client.1.smithi138.stdout:8/119: rename d3/d8/l15 to d3/d8/l17 0 2024-05-27T09:29:29.603 INFO:tasks.workunit.client.1.smithi138.stdout:4/130: link d7/c1f d7/d27/c28 0 2024-05-27T09:29:29.604 INFO:tasks.workunit.client.1.smithi138.stdout:4/131: write d7/df/f1a [5015979,36586] 0 2024-05-27T09:29:29.606 INFO:tasks.workunit.client.0.smithi119.stdout:3/821: symlink db/d5b/dce/l10d 0 2024-05-27T09:29:29.607 INFO:tasks.workunit.client.1.smithi138.stdout:7/132: dwrite d0/f19 [0,4194304] 0 2024-05-27T09:29:29.608 INFO:tasks.workunit.client.1.smithi138.stdout:1/186: creat d4/d1a/d2e/f41 x:0 0 0 2024-05-27T09:29:29.609 INFO:tasks.workunit.client.0.smithi119.stdout:0/618: creat d2/d6/d9/d24/d30/d6b/fc6 x:0 0 0 2024-05-27T09:29:29.609 INFO:tasks.workunit.client.0.smithi119.stdout:0/619: chown d2/l3 35173 1 2024-05-27T09:29:29.609 INFO:tasks.workunit.client.0.smithi119.stdout:0/620: dread - d2/d6/d9/fbb zero size 2024-05-27T09:29:29.612 INFO:tasks.workunit.client.0.smithi119.stdout:9/570: dread d18/d19/d61/fa5 [4194304,4194304] 0 2024-05-27T09:29:29.614 INFO:tasks.workunit.client.1.smithi138.stdout:4/132: dread d7/f23 [0,4194304] 0 2024-05-27T09:29:29.614 INFO:tasks.workunit.client.1.smithi138.stdout:4/133: write d7/df/f19 [853058,61149] 0 2024-05-27T09:29:29.615 INFO:tasks.workunit.client.0.smithi119.stdout:0/621: rename d2/d6/d23/c13 to d2/d6/d9/cc7 0 2024-05-27T09:29:29.615 INFO:tasks.workunit.client.0.smithi119.stdout:9/571: mknod d18/d19/d1b/d40/d44/cb6 0 2024-05-27T09:29:29.616 INFO:tasks.workunit.client.0.smithi119.stdout:9/572: stat d18/d19/d1b/d29/f7c 0 2024-05-27T09:29:29.621 INFO:tasks.workunit.client.0.smithi119.stdout:9/573: getdents d18/d19/d1b/d40 0 2024-05-27T09:29:29.621 INFO:tasks.workunit.client.0.smithi119.stdout:9/574: write d18/d19/d1b/f64 [544223,109313] 0 2024-05-27T09:29:29.621 INFO:tasks.workunit.client.0.smithi119.stdout:0/622: dread d2/d6/d75/f8a [0,4194304] 0 2024-05-27T09:29:29.621 INFO:tasks.workunit.client.0.smithi119.stdout:0/623: truncate d2/d6/d9/fbb 697462 0 2024-05-27T09:29:29.622 INFO:tasks.workunit.client.1.smithi138.stdout:7/133: symlink d0/d2b/l2c 0 2024-05-27T09:29:29.623 INFO:tasks.workunit.client.0.smithi119.stdout:9/575: mkdir d18/d19/d1b/d40/db7 0 2024-05-27T09:29:29.623 INFO:tasks.workunit.client.0.smithi119.stdout:9/576: truncate d18/d19/d1b/f1e 1222595 0 2024-05-27T09:29:29.623 INFO:tasks.workunit.client.0.smithi119.stdout:9/577: read - d18/d19/d1b/d29/f9e zero size 2024-05-27T09:29:29.623 INFO:tasks.workunit.client.0.smithi119.stdout:9/578: dread - d18/d19/fb1 zero size 2024-05-27T09:29:29.623 INFO:tasks.workunit.client.0.smithi119.stdout:9/579: readlink d18/d19/d1b/d40/l52 0 2024-05-27T09:29:29.624 INFO:tasks.workunit.client.1.smithi138.stdout:1/187: creat d4/d3c/f42 x:0 0 0 2024-05-27T09:29:29.624 INFO:tasks.workunit.client.1.smithi138.stdout:1/188: chown d4 121947056 1 2024-05-27T09:29:29.627 INFO:tasks.workunit.client.0.smithi119.stdout:0/624: rename d2/d6/d23/d7a/f44 to d2/d6/d9/d24/fc8 0 2024-05-27T09:29:29.628 INFO:tasks.workunit.client.1.smithi138.stdout:4/134: unlink d7/f12 0 2024-05-27T09:29:29.628 INFO:tasks.workunit.client.1.smithi138.stdout:4/135: chown c2 118 1 2024-05-27T09:29:29.631 INFO:tasks.workunit.client.0.smithi119.stdout:2/685: dwrite d2/da/d8a/dd4/f57 [4194304,4194304] 0 2024-05-27T09:29:29.632 INFO:tasks.workunit.client.1.smithi138.stdout:9/107: dwrite f3 [0,4194304] 0 2024-05-27T09:29:29.632 INFO:tasks.workunit.client.1.smithi138.stdout:9/108: fsync d6/f9 0 2024-05-27T09:29:29.632 INFO:tasks.workunit.client.1.smithi138.stdout:9/109: truncate d6/db/dc/f1e 429818 0 2024-05-27T09:29:29.632 INFO:tasks.workunit.client.1.smithi138.stdout:9/110: fdatasync f1 0 2024-05-27T09:29:29.633 INFO:tasks.workunit.client.0.smithi119.stdout:4/709: dwrite d4/d23/f26 [0,4194304] 0 2024-05-27T09:29:29.635 INFO:tasks.workunit.client.0.smithi119.stdout:7/706: dwrite d7/faa [0,4194304] 0 2024-05-27T09:29:29.636 INFO:tasks.workunit.client.1.smithi138.stdout:3/89: dwrite d0/d2/da/f1b [0,4194304] 0 2024-05-27T09:29:29.642 INFO:tasks.workunit.client.0.smithi119.stdout:0/625: unlink d2/d1a/f51 0 2024-05-27T09:29:29.642 INFO:tasks.workunit.client.0.smithi119.stdout:0/626: readlink d2/d6/d9/d2f/l8d 0 2024-05-27T09:29:29.643 INFO:tasks.workunit.client.0.smithi119.stdout:2/686: mkdir d2/da/d8a/dec 0 2024-05-27T09:29:29.644 INFO:tasks.workunit.client.1.smithi138.stdout:9/111: dread d6/f8 [0,4194304] 0 2024-05-27T09:29:29.645 INFO:tasks.workunit.client.0.smithi119.stdout:7/707: unlink f4 0 2024-05-27T09:29:29.648 INFO:tasks.workunit.client.0.smithi119.stdout:0/627: rename d2/d6/d9/d12/d4f/l6c to d2/d6/d9/lc9 0 2024-05-27T09:29:29.649 INFO:tasks.workunit.client.1.smithi138.stdout:8/120: truncate d3/d8/fc 2678705 0 2024-05-27T09:29:29.649 INFO:tasks.workunit.client.1.smithi138.stdout:7/134: fdatasync d0/f21 0 2024-05-27T09:29:29.649 INFO:tasks.workunit.client.1.smithi138.stdout:8/121: fdatasync d3/d8/f14 0 2024-05-27T09:29:29.649 INFO:tasks.workunit.client.1.smithi138.stdout:7/135: stat d0/l29 0 2024-05-27T09:29:29.652 INFO:tasks.workunit.client.1.smithi138.stdout:1/189: unlink d4/d8/l20 0 2024-05-27T09:29:29.656 INFO:tasks.workunit.client.1.smithi138.stdout:0/185: truncate da/d2a/f30 513122 0 2024-05-27T09:29:29.659 INFO:tasks.workunit.client.1.smithi138.stdout:9/112: creat d6/db/f22 x:0 0 0 2024-05-27T09:29:29.661 INFO:tasks.workunit.client.0.smithi119.stdout:7/708: rename d7/df/d19/d46/d93/lbc to d7/df/d19/d97/d92/db7/ddc/le4 0 2024-05-27T09:29:29.662 INFO:tasks.workunit.client.1.smithi138.stdout:1/190: dread d4/fd [0,4194304] 0 2024-05-27T09:29:29.662 INFO:tasks.workunit.client.1.smithi138.stdout:1/191: chown d4/l15 199745030 1 2024-05-27T09:29:29.662 INFO:tasks.workunit.client.1.smithi138.stdout:8/122: mknod d3/d8/c18 0 2024-05-27T09:29:29.669 INFO:tasks.workunit.client.1.smithi138.stdout:3/90: read d0/d2/da/f1b [216452,59485] 0 2024-05-27T09:29:29.669 INFO:tasks.workunit.client.1.smithi138.stdout:3/91: write d0/d2/f13 [2055959,114020] 0 2024-05-27T09:29:29.669 INFO:tasks.workunit.client.1.smithi138.stdout:4/136: truncate d7/df/f19 32435 0 2024-05-27T09:29:29.671 INFO:tasks.workunit.client.1.smithi138.stdout:9/113: symlink d6/db/de/d18/l23 0 2024-05-27T09:29:29.671 INFO:tasks.workunit.client.1.smithi138.stdout:3/92: creat d0/d8/f1d x:0 0 0 2024-05-27T09:29:29.672 INFO:tasks.workunit.client.0.smithi119.stdout:6/673: dwrite d0/d2/f31 [0,4194304] 0 2024-05-27T09:29:29.672 INFO:tasks.workunit.client.0.smithi119.stdout:5/733: dwrite d6/db/de/d27/dc9/fc6 [0,4194304] 0 2024-05-27T09:29:29.672 INFO:tasks.workunit.client.1.smithi138.stdout:9/114: write d6/db/dc/f1e [294554,127176] 0 2024-05-27T09:29:29.672 INFO:tasks.workunit.client.1.smithi138.stdout:9/115: chown d6/f1f 522652 1 2024-05-27T09:29:29.673 INFO:tasks.workunit.client.1.smithi138.stdout:9/116: dread - d6/db/f14 zero size 2024-05-27T09:29:29.673 INFO:tasks.workunit.client.1.smithi138.stdout:9/117: truncate d6/db/f1b 497921 0 2024-05-27T09:29:29.673 INFO:tasks.workunit.client.0.smithi119.stdout:1/742: dwrite d7/dd/d25/fc5 [4194304,4194304] 0 2024-05-27T09:29:29.674 INFO:tasks.workunit.client.0.smithi119.stdout:0/628: rmdir d2/d6/d23/d7a 39 2024-05-27T09:29:29.675 INFO:tasks.workunit.client.1.smithi138.stdout:1/192: mknod d4/d1a/d2e/d32/c43 0 2024-05-27T09:29:29.677 INFO:tasks.workunit.client.1.smithi138.stdout:0/186: dread da/f29 [0,4194304] 0 2024-05-27T09:29:29.680 INFO:tasks.workunit.client.1.smithi138.stdout:7/136: unlink d0/l18 0 2024-05-27T09:29:29.683 INFO:tasks.workunit.client.0.smithi119.stdout:5/734: symlink d6/d2f/l101 0 2024-05-27T09:29:29.692 INFO:tasks.workunit.client.1.smithi138.stdout:2/118: sync 2024-05-27T09:29:29.693 INFO:tasks.workunit.client.1.smithi138.stdout:2/119: write d1/d10/f6 [4642234,35305] 0 2024-05-27T09:29:29.694 INFO:tasks.workunit.client.1.smithi138.stdout:9/118: mkdir d6/db/de/d18/d24 0 2024-05-27T09:29:29.696 INFO:tasks.workunit.client.1.smithi138.stdout:7/137: symlink d0/l2d 0 2024-05-27T09:29:29.697 INFO:tasks.workunit.client.0.smithi119.stdout:3/822: dwrite db/f37 [0,4194304] 0 2024-05-27T09:29:29.701 INFO:tasks.workunit.client.1.smithi138.stdout:9/119: mkdir d6/d25 0 2024-05-27T09:29:29.701 INFO:tasks.workunit.client.1.smithi138.stdout:4/137: rmdir d7/df/d20 0 2024-05-27T09:29:29.701 INFO:tasks.workunit.client.1.smithi138.stdout:4/138: dread - d7/f26 zero size 2024-05-27T09:29:29.701 INFO:tasks.workunit.client.1.smithi138.stdout:7/138: read - d0/f1a zero size 2024-05-27T09:29:29.702 INFO:tasks.workunit.client.1.smithi138.stdout:4/139: creat d7/df/f29 x:0 0 0 2024-05-27T09:29:29.703 INFO:tasks.workunit.client.0.smithi119.stdout:7/709: dread d7/df/d19/d8d/fa3 [0,4194304] 0 2024-05-27T09:29:29.704 INFO:tasks.workunit.client.0.smithi119.stdout:4/710: dwrite d4/de/d9b/fba [0,4194304] 0 2024-05-27T09:29:29.706 INFO:tasks.workunit.client.1.smithi138.stdout:4/140: mknod d7/df/c2a 0 2024-05-27T09:29:29.707 INFO:tasks.workunit.client.0.smithi119.stdout:5/735: symlink d6/dd/d46/d47/d4b/dd8/l102 0 2024-05-27T09:29:29.707 INFO:tasks.workunit.client.0.smithi119.stdout:0/629: unlink d2/d6/d9/d24/l57 0 2024-05-27T09:29:29.707 INFO:tasks.workunit.client.0.smithi119.stdout:0/630: fdatasync d2/d6/d9/d12/fb4 0 2024-05-27T09:29:29.709 INFO:tasks.workunit.client.1.smithi138.stdout:4/141: symlink d7/d27/l2b 0 2024-05-27T09:29:29.709 INFO:tasks.workunit.client.1.smithi138.stdout:4/142: chown d7/df/f1d 491892 1 2024-05-27T09:29:29.709 INFO:tasks.workunit.client.1.smithi138.stdout:1/193: dread d4/f24 [0,4194304] 0 2024-05-27T09:29:29.710 INFO:tasks.workunit.client.0.smithi119.stdout:0/631: write d2/d6/d9/d24/f9f [256813,44209] 0 2024-05-27T09:29:29.710 INFO:tasks.workunit.client.0.smithi119.stdout:9/580: dwrite d18/d19/d9b/fa0 [0,4194304] 0 2024-05-27T09:29:29.711 INFO:tasks.workunit.client.1.smithi138.stdout:4/143: creat d7/d27/f2c x:0 0 0 2024-05-27T09:29:29.712 INFO:tasks.workunit.client.1.smithi138.stdout:1/194: readlink d4/d8/l36 0 2024-05-27T09:29:29.714 INFO:tasks.workunit.client.1.smithi138.stdout:4/144: rename d7/df/c21 to d7/df/c2d 0 2024-05-27T09:29:29.714 INFO:tasks.workunit.client.1.smithi138.stdout:4/145: creat d7/df/f2e x:0 0 0 2024-05-27T09:29:29.716 INFO:tasks.workunit.client.0.smithi119.stdout:5/736: write d6/dd/d39/d19/fac [175880,14800] 0 2024-05-27T09:29:29.717 INFO:tasks.workunit.client.1.smithi138.stdout:8/123: dread d3/d8/fc [0,4194304] 0 2024-05-27T09:29:29.718 INFO:tasks.workunit.client.1.smithi138.stdout:8/124: stat d3/d8/c18 0 2024-05-27T09:29:29.723 INFO:tasks.workunit.client.0.smithi119.stdout:3/823: link db/d36/d45/d4c/d62/f3f db/dc9/d98/d105/f10e 0 2024-05-27T09:29:29.724 INFO:tasks.workunit.client.0.smithi119.stdout:3/824: dread - db/d36/d45/dab/f101 zero size 2024-05-27T09:29:29.725 INFO:tasks.workunit.client.0.smithi119.stdout:3/825: readlink db/d36/d45/d4c/d62/d13/d33/l34 0 2024-05-27T09:29:29.726 INFO:tasks.workunit.client.1.smithi138.stdout:4/146: mknod d7/c2f 0 2024-05-27T09:29:29.728 INFO:tasks.workunit.client.0.smithi119.stdout:0/632: symlink d2/d6/d9/d12/d4f/lca 0 2024-05-27T09:29:29.729 INFO:tasks.workunit.client.0.smithi119.stdout:2/687: dwrite d2/da/d8a/dd4/f79 [0,4194304] 0 2024-05-27T09:29:29.729 INFO:tasks.workunit.client.0.smithi119.stdout:7/710: readlink d7/df/d19/d97/d92/db7/dba/l3c 0 2024-05-27T09:29:29.730 INFO:tasks.workunit.client.0.smithi119.stdout:7/711: truncate d7/d1a/d99/fc8 662660 0 2024-05-27T09:29:29.731 INFO:tasks.workunit.client.0.smithi119.stdout:5/737: rename d6/f80 to d6/d2f/d9d/f103 0 2024-05-27T09:29:29.732 INFO:tasks.workunit.client.1.smithi138.stdout:0/187: dread da/d2a/f30 [0,4194304] 0 2024-05-27T09:29:29.741 INFO:tasks.workunit.client.1.smithi138.stdout:8/125: mknod d3/d8/c19 0 2024-05-27T09:29:29.741 INFO:tasks.workunit.client.1.smithi138.stdout:8/126: chown d3/d8/cd 6 1 2024-05-27T09:29:29.741 INFO:tasks.workunit.client.1.smithi138.stdout:0/188: mkdir da/d17/d1d/d43 0 2024-05-27T09:29:29.741 INFO:tasks.workunit.client.1.smithi138.stdout:9/120: dread f1 [0,4194304] 0 2024-05-27T09:29:29.741 INFO:tasks.workunit.client.0.smithi119.stdout:0/633: creat d2/d6/d9/d24/fcb x:0 0 0 2024-05-27T09:29:29.745 INFO:tasks.workunit.client.1.smithi138.stdout:8/127: link d3/d8/fc d3/d8/f1a 0 2024-05-27T09:29:29.746 INFO:tasks.workunit.client.0.smithi119.stdout:2/688: creat d2/da/d8a/d8f/fed x:0 0 0 2024-05-27T09:29:29.747 INFO:tasks.workunit.client.0.smithi119.stdout:8/614: sync 2024-05-27T09:29:29.748 INFO:tasks.workunit.client.0.smithi119.stdout:8/615: truncate d3/d8/d4d/d58/f70 840079 0 2024-05-27T09:29:29.748 INFO:tasks.workunit.client.0.smithi119.stdout:8/616: chown d3/d8/d1b/db1/fc5 1 1 2024-05-27T09:29:29.749 INFO:tasks.workunit.client.1.smithi138.stdout:9/121: mkdir d6/da/d26 0 2024-05-27T09:29:29.749 INFO:tasks.workunit.client.0.smithi119.stdout:0/634: mkdir d2/d6/d9/d24/d30/dcc 0 2024-05-27T09:29:29.750 INFO:tasks.workunit.client.0.smithi119.stdout:7/712: rmdir d7/df/d19/d97/d92/db7/dba 39 2024-05-27T09:29:29.751 INFO:tasks.workunit.client.1.smithi138.stdout:5/153: sync 2024-05-27T09:29:29.751 INFO:tasks.workunit.client.1.smithi138.stdout:6/230: sync 2024-05-27T09:29:29.752 INFO:tasks.workunit.client.0.smithi119.stdout:5/738: getdents d6/d2f 0 2024-05-27T09:29:29.757 INFO:tasks.workunit.client.0.smithi119.stdout:0/635: rename d2/d6/d9/d2f/d95/lb7 to d2/d6/d9/dab/lcd 0 2024-05-27T09:29:29.757 INFO:tasks.workunit.client.0.smithi119.stdout:0/636: chown d2/d6/d9/d24/d30/d4a/l61 2896 1 2024-05-27T09:29:29.757 INFO:tasks.workunit.client.1.smithi138.stdout:8/128: symlink d3/d8/d11/l1b 0 2024-05-27T09:29:29.758 INFO:tasks.workunit.client.0.smithi119.stdout:2/689: getdents d2/da/d4e 0 2024-05-27T09:29:29.758 INFO:tasks.workunit.client.0.smithi119.stdout:2/690: fdatasync d2/da/d15/f18 0 2024-05-27T09:29:29.760 INFO:tasks.workunit.client.1.smithi138.stdout:9/122: mkdir d6/db/de/d18/d27 0 2024-05-27T09:29:29.761 INFO:tasks.workunit.client.1.smithi138.stdout:5/154: mknod d3/d12/d16/d1d/c35 0 2024-05-27T09:29:29.761 INFO:tasks.workunit.client.1.smithi138.stdout:9/123: write d6/db/dc/f1c [647432,72210] 0 2024-05-27T09:29:29.761 INFO:tasks.workunit.client.1.smithi138.stdout:6/231: mknod d3/d11/d2d/c54 0 2024-05-27T09:29:29.762 INFO:tasks.workunit.client.1.smithi138.stdout:4/147: getdents d7 0 2024-05-27T09:29:29.763 INFO:tasks.workunit.client.0.smithi119.stdout:7/713: mkdir d7/df/d19/d97/d92/db7/dba/d42/d64/de3/de5 0 2024-05-27T09:29:29.763 INFO:tasks.workunit.client.0.smithi119.stdout:7/714: chown d7/df/d19/d8d/fa3 686316577 1 2024-05-27T09:29:29.765 INFO:tasks.workunit.client.0.smithi119.stdout:0/637: creat d2/d6/d9/d12/fce x:0 0 0 2024-05-27T09:29:29.767 INFO:tasks.workunit.client.0.smithi119.stdout:2/691: creat d2/da/d8a/dd4/d89/d94/d87/dcf/fee x:0 0 0 2024-05-27T09:29:29.767 INFO:tasks.workunit.client.0.smithi119.stdout:2/692: write d2/d1a/f39 [4625650,30704] 0 2024-05-27T09:29:29.767 INFO:tasks.workunit.client.0.smithi119.stdout:2/693: chown d2/da/d8a/d8f/fed 63582589 1 2024-05-27T09:29:29.767 INFO:tasks.workunit.client.0.smithi119.stdout:2/694: readlink d2/da/d8a/dd4/l4b 0 2024-05-27T09:29:29.767 INFO:tasks.workunit.client.0.smithi119.stdout:2/695: chown d2/c2a 472216969 1 2024-05-27T09:29:29.769 INFO:tasks.workunit.client.1.smithi138.stdout:8/129: unlink d3/d8/d11/f12 0 2024-05-27T09:29:29.769 INFO:tasks.workunit.client.0.smithi119.stdout:7/715: write d7/d6a/f8f [129165,79902] 0 2024-05-27T09:29:29.769 INFO:tasks.workunit.client.0.smithi119.stdout:7/716: truncate d7/df/d19/d97/d92/fd1 600633 0 2024-05-27T09:29:29.770 INFO:tasks.workunit.client.0.smithi119.stdout:5/739: creat d6/dd/d39/d19/d3c/dd5/f104 x:0 0 0 2024-05-27T09:29:29.770 INFO:tasks.workunit.client.0.smithi119.stdout:5/740: getdents d6/d2f 0 2024-05-27T09:29:29.773 INFO:tasks.workunit.client.0.smithi119.stdout:0/638: symlink d2/d6/d9/d12/d4f/lcf 0 2024-05-27T09:29:29.773 INFO:tasks.workunit.client.0.smithi119.stdout:0/639: chown d2/d6/d9/d12/c91 3 1 2024-05-27T09:29:29.773 INFO:tasks.workunit.client.1.smithi138.stdout:5/155: truncate d3/f21 245429 0 2024-05-27T09:29:29.777 INFO:tasks.workunit.client.0.smithi119.stdout:5/741: mkdir d6/dd/d46/d47/d105 0 2024-05-27T09:29:29.778 INFO:tasks.workunit.client.1.smithi138.stdout:6/232: mknod d3/d35/d43/d1c/d32/d47/c55 0 2024-05-27T09:29:29.778 INFO:tasks.workunit.client.1.smithi138.stdout:6/233: readlink d3/d11/l16 0 2024-05-27T09:29:29.778 INFO:tasks.workunit.client.1.smithi138.stdout:6/234: fsync d3/d35/d43/d1c/f4c 0 2024-05-27T09:29:29.778 INFO:tasks.workunit.client.0.smithi119.stdout:0/640: mknod d2/d6/d23/db8/cd0 0 2024-05-27T09:29:29.778 INFO:tasks.workunit.client.0.smithi119.stdout:0/641: chown d2/d6/d9/d12/d4b/d96/f98 24 1 2024-05-27T09:29:29.781 INFO:tasks.workunit.client.1.smithi138.stdout:4/148: rename d7/c8 to d7/d27/c30 0 2024-05-27T09:29:29.781 INFO:tasks.workunit.client.1.smithi138.stdout:4/149: stat d7/ca 0 2024-05-27T09:29:29.781 INFO:tasks.workunit.client.0.smithi119.stdout:5/742: write d6/db/f4c [497391,90139] 0 2024-05-27T09:29:29.781 INFO:tasks.workunit.client.0.smithi119.stdout:7/717: unlink d7/d5f/cb8 0 2024-05-27T09:29:29.782 INFO:tasks.workunit.client.0.smithi119.stdout:2/696: dread d2/d1a/f3d [0,4194304] 0 2024-05-27T09:29:29.782 INFO:tasks.workunit.client.0.smithi119.stdout:2/697: fsync d2/da/f65 0 2024-05-27T09:29:29.784 INFO:tasks.workunit.client.1.smithi138.stdout:5/156: mkdir d3/d12/d16/d1d/d22/d2b/d36 0 2024-05-27T09:29:29.786 INFO:tasks.workunit.client.0.smithi119.stdout:9/581: dwrite d18/f41 [0,4194304] 0 2024-05-27T09:29:29.786 INFO:tasks.workunit.client.0.smithi119.stdout:0/642: mkdir d2/d6/d23/d7a/d53/dd1 0 2024-05-27T09:29:29.786 INFO:tasks.workunit.client.0.smithi119.stdout:5/743: write d6/f6a [348145,37694] 0 2024-05-27T09:29:29.786 INFO:tasks.workunit.client.0.smithi119.stdout:5/744: chown d6/c28 4032708 1 2024-05-27T09:29:29.787 INFO:tasks.workunit.client.0.smithi119.stdout:2/698: unlink d2/da/d8a/d8f/cc7 0 2024-05-27T09:29:29.787 INFO:tasks.workunit.client.0.smithi119.stdout:5/745: creat d6/dd/f106 x:0 0 0 2024-05-27T09:29:29.787 INFO:tasks.workunit.client.0.smithi119.stdout:5/746: chown d6 249 1 2024-05-27T09:29:29.787 INFO:tasks.workunit.client.0.smithi119.stdout:5/747: creat d6/dd/d39/d74/f107 x:0 0 0 2024-05-27T09:29:29.787 INFO:tasks.workunit.client.0.smithi119.stdout:5/748: write d6/f22 [4162975,127208] 0 2024-05-27T09:29:29.789 INFO:tasks.workunit.client.0.smithi119.stdout:0/643: getdents d2/d6 0 2024-05-27T09:29:29.790 INFO:tasks.workunit.client.0.smithi119.stdout:0/644: readlink d2/d6/d9/d24/d30/lac 0 2024-05-27T09:29:29.795 INFO:tasks.workunit.client.1.smithi138.stdout:9/124: getdents d6/db 0 2024-05-27T09:29:29.796 INFO:tasks.workunit.client.1.smithi138.stdout:8/130: rename d3/d8/ce to d3/d8/d11/c1c 0 2024-05-27T09:29:29.796 INFO:tasks.workunit.client.0.smithi119.stdout:5/749: creat d6/db/de/d27/f108 x:0 0 0 2024-05-27T09:29:29.797 INFO:tasks.workunit.client.0.smithi119.stdout:0/645: symlink d2/d6/d9/d24/ld2 0 2024-05-27T09:29:29.797 INFO:tasks.workunit.client.1.smithi138.stdout:4/150: link f0 d7/d27/f31 0 2024-05-27T09:29:29.799 INFO:tasks.workunit.client.0.smithi119.stdout:0/646: creat d2/d6/d9/d12/da3/fd3 x:0 0 0 2024-05-27T09:29:29.799 INFO:tasks.workunit.client.1.smithi138.stdout:5/157: mknod d3/d12/d16/d1d/d22/d2b/c37 0 2024-05-27T09:29:29.802 INFO:tasks.workunit.client.0.smithi119.stdout:8/617: dread d3/f25 [0,4194304] 0 2024-05-27T09:29:29.802 INFO:tasks.workunit.client.0.smithi119.stdout:5/750: rename d6/dd/d39/d19/d3c/caf to d6/dd/d39/d19/d3c/d52/d9c/c109 0 2024-05-27T09:29:29.802 INFO:tasks.workunit.client.1.smithi138.stdout:2/120: dwrite d1/d13/d17/d25/f28 [0,4194304] 0 2024-05-27T09:29:29.804 INFO:tasks.workunit.client.1.smithi138.stdout:6/235: rmdir d3/d11/d2d/d37 39 2024-05-27T09:29:29.806 INFO:tasks.workunit.client.1.smithi138.stdout:6/236: creat d3/d35/d43/d1c/d32/d47/f56 x:0 0 0 2024-05-27T09:29:29.808 INFO:tasks.workunit.client.0.smithi119.stdout:0/647: creat d2/d6/d23/d7a/fd4 x:0 0 0 2024-05-27T09:29:29.808 INFO:tasks.workunit.client.0.smithi119.stdout:4/711: dwrite d4/d6/d3e/d89/db2/fbd [0,4194304] 0 2024-05-27T09:29:29.808 INFO:tasks.workunit.client.0.smithi119.stdout:4/712: write d4/d6/f86 [830540,114748] 0 2024-05-27T09:29:29.808 INFO:tasks.workunit.client.0.smithi119.stdout:8/618: unlink d3/d39/f3e 0 2024-05-27T09:29:29.808 INFO:tasks.workunit.client.0.smithi119.stdout:8/619: write d3/d6e/d8e/d9b/fa5 [995614,82330] 0 2024-05-27T09:29:29.810 INFO:tasks.workunit.client.1.smithi138.stdout:9/125: mkdir d6/da/d26/d28 0 2024-05-27T09:29:29.811 INFO:tasks.workunit.client.1.smithi138.stdout:5/158: getdents d3/d12/d16/d1d/d23 0 2024-05-27T09:29:29.819 INFO:tasks.workunit.client.1.smithi138.stdout:6/237: unlink d3/d19/f4b 0 2024-05-27T09:29:29.820 INFO:tasks.workunit.client.1.smithi138.stdout:6/238: readlink d3/l49 0 2024-05-27T09:29:29.823 INFO:tasks.workunit.client.0.smithi119.stdout:4/713: symlink d4/d23/d24/d5b/da8/le3 0 2024-05-27T09:29:29.823 INFO:tasks.workunit.client.1.smithi138.stdout:2/121: fdatasync d1/d10/f6 0 2024-05-27T09:29:29.824 INFO:tasks.workunit.client.1.smithi138.stdout:5/159: unlink d3/d12/d16/d1d/d22/c27 0 2024-05-27T09:29:29.824 INFO:tasks.workunit.client.1.smithi138.stdout:5/160: chown d3/d12/l29 1 1 2024-05-27T09:29:29.824 INFO:tasks.workunit.client.0.smithi119.stdout:4/714: fsync d4/d6/d81/dce/f9f 0 2024-05-27T09:29:29.824 INFO:tasks.workunit.client.1.smithi138.stdout:5/161: write d3/d12/d26/f2f [327633,54109] 0 2024-05-27T09:29:29.825 INFO:tasks.workunit.client.0.smithi119.stdout:4/715: dread - d4/d6/d28/d2f/d36/dc8/fd0 zero size 2024-05-27T09:29:29.825 INFO:tasks.workunit.client.1.smithi138.stdout:9/126: mkdir d6/da/d29 0 2024-05-27T09:29:29.826 INFO:tasks.workunit.client.0.smithi119.stdout:8/620: creat d3/d8/d4d/d5d/d66/fca x:0 0 0 2024-05-27T09:29:29.827 INFO:tasks.workunit.client.1.smithi138.stdout:8/131: symlink d3/l1d 0 2024-05-27T09:29:29.828 INFO:tasks.workunit.client.1.smithi138.stdout:9/127: mknod d6/db/de/d18/c2a 0 2024-05-27T09:29:29.831 INFO:tasks.workunit.client.1.smithi138.stdout:8/132: creat d3/d8/f1e x:0 0 0 2024-05-27T09:29:29.832 INFO:tasks.workunit.client.0.smithi119.stdout:4/716: mkdir d4/d1c/de4 0 2024-05-27T09:29:29.832 INFO:tasks.workunit.client.0.smithi119.stdout:4/717: write d4/d4e/fb7 [143849,118482] 0 2024-05-27T09:29:29.832 INFO:tasks.workunit.client.0.smithi119.stdout:4/718: chown d4/d6/d28/d2f/c63 29987880 1 2024-05-27T09:29:29.832 INFO:tasks.workunit.client.1.smithi138.stdout:1/195: dwrite d4/f18 [4194304,4194304] 0 2024-05-27T09:29:29.833 INFO:tasks.workunit.client.1.smithi138.stdout:8/133: symlink d3/d8/d11/l1f 0 2024-05-27T09:29:29.833 INFO:tasks.workunit.client.1.smithi138.stdout:8/134: write d3/d8/f1a [51007,71891] 0 2024-05-27T09:29:29.835 INFO:tasks.workunit.client.1.smithi138.stdout:8/135: symlink d3/d8/d11/l20 0 2024-05-27T09:29:29.838 INFO:tasks.workunit.client.0.smithi119.stdout:8/621: rename d3/d39/d5a/la2 to d3/d6e/d80/d84/lcb 0 2024-05-27T09:29:29.838 INFO:tasks.workunit.client.0.smithi119.stdout:8/622: write d3/d8/fd [874449,104480] 0 2024-05-27T09:29:29.841 INFO:tasks.workunit.client.1.smithi138.stdout:9/128: dread d6/db/dc/f1e [0,4194304] 0 2024-05-27T09:29:29.844 INFO:tasks.workunit.client.0.smithi119.stdout:7/718: dread d7/df/d19/d9a/da6/dda/f86 [0,4194304] 0 2024-05-27T09:29:29.844 INFO:tasks.workunit.client.0.smithi119.stdout:4/719: truncate d4/de/d66/f68 1021881 0 2024-05-27T09:29:29.849 INFO:tasks.workunit.client.1.smithi138.stdout:7/139: dwrite d0/f24 [0,4194304] 0 2024-05-27T09:29:29.849 INFO:tasks.workunit.client.1.smithi138.stdout:7/140: fdatasync d0/d7/f1b 0 2024-05-27T09:29:29.849 INFO:tasks.workunit.client.1.smithi138.stdout:8/136: symlink d3/d8/l21 0 2024-05-27T09:29:29.849 INFO:tasks.workunit.client.1.smithi138.stdout:5/162: write f0 [6775196,62726] 0 2024-05-27T09:29:29.850 INFO:tasks.workunit.client.1.smithi138.stdout:9/129: creat d6/db/de/f2b x:0 0 0 2024-05-27T09:29:29.850 INFO:tasks.workunit.client.0.smithi119.stdout:8/623: mknod d3/d8/ccc 0 2024-05-27T09:29:29.850 INFO:tasks.workunit.client.0.smithi119.stdout:8/624: chown d3/d8/d1b/db1/cc6 8438408 1 2024-05-27T09:29:29.850 INFO:tasks.workunit.client.0.smithi119.stdout:8/625: chown d3/de/dad 2 1 2024-05-27T09:29:29.851 INFO:tasks.workunit.client.0.smithi119.stdout:0/648: write d2/d6/d9/fbd [2842300,6520] 0 2024-05-27T09:29:29.852 INFO:tasks.workunit.client.0.smithi119.stdout:9/582: dwrite d18/d19/d1b/d2a/d63/f97 [0,4194304] 0 2024-05-27T09:29:29.854 INFO:tasks.workunit.client.1.smithi138.stdout:9/130: truncate d6/f8 620523 0 2024-05-27T09:29:29.856 INFO:tasks.workunit.client.1.smithi138.stdout:9/131: write d6/db/f10 [2161438,42737] 0 2024-05-27T09:29:29.856 INFO:tasks.workunit.client.1.smithi138.stdout:5/163: creat d3/d12/d16/d1d/d22/d2b/f38 x:0 0 0 2024-05-27T09:29:29.856 INFO:tasks.workunit.client.1.smithi138.stdout:5/164: read - d3/f1b zero size 2024-05-27T09:29:29.856 INFO:tasks.workunit.client.1.smithi138.stdout:5/165: chown d3/l8 5142499 1 2024-05-27T09:29:29.857 INFO:tasks.workunit.client.1.smithi138.stdout:5/166: link d3/f18 d3/d12/d16/f39 0 2024-05-27T09:29:29.858 INFO:tasks.workunit.client.1.smithi138.stdout:5/167: stat d3/d12/c1f 0 2024-05-27T09:29:29.868 INFO:tasks.workunit.client.1.smithi138.stdout:8/137: fsync d3/d8/f1e 0 2024-05-27T09:29:29.871 INFO:tasks.workunit.client.1.smithi138.stdout:8/138: getdents d3/d8/d11 0 2024-05-27T09:29:29.871 INFO:tasks.workunit.client.1.smithi138.stdout:8/139: write d3/d8/f1e [342811,31504] 0 2024-05-27T09:29:29.871 INFO:tasks.workunit.client.1.smithi138.stdout:8/140: chown d3/d8/f14 5612 1 2024-05-27T09:29:29.872 INFO:tasks.workunit.client.0.smithi119.stdout:4/720: mkdir d4/de/de5 0 2024-05-27T09:29:29.872 INFO:tasks.workunit.client.0.smithi119.stdout:4/721: chown d4/l6f 901315 1 2024-05-27T09:29:29.874 INFO:tasks.workunit.client.1.smithi138.stdout:8/141: rename d3/d8/f1e to d3/d8/d11/f22 0 2024-05-27T09:29:29.876 INFO:tasks.workunit.client.0.smithi119.stdout:8/626: mknod d3/d8/d4d/ccd 0 2024-05-27T09:29:29.876 INFO:tasks.workunit.client.1.smithi138.stdout:8/142: mknod d3/d8/c23 0 2024-05-27T09:29:29.878 INFO:tasks.workunit.client.1.smithi138.stdout:2/122: write d1/f1b [1025542,1723] 0 2024-05-27T09:29:29.879 INFO:tasks.workunit.client.1.smithi138.stdout:8/143: mkdir d3/d8/d24 0 2024-05-27T09:29:29.879 INFO:tasks.workunit.client.1.smithi138.stdout:8/144: chown d3/l1d 367562 1 2024-05-27T09:29:29.885 INFO:tasks.workunit.client.1.smithi138.stdout:6/239: write d3/d35/d43/f29 [843688,81905] 0 2024-05-27T09:29:29.885 INFO:tasks.workunit.client.1.smithi138.stdout:6/240: stat d3/d19/c26 0 2024-05-27T09:29:29.891 INFO:tasks.workunit.client.1.smithi138.stdout:2/123: truncate d1/d13/d17/d25/f28 3102105 0 2024-05-27T09:29:29.896 INFO:tasks.workunit.client.0.smithi119.stdout:0/649: mknod d2/d6/d23/d7a/cd5 0 2024-05-27T09:29:29.896 INFO:tasks.workunit.client.1.smithi138.stdout:6/241: mknod d3/d35/d43/d1c/c57 0 2024-05-27T09:29:29.897 INFO:tasks.workunit.client.1.smithi138.stdout:2/124: truncate d1/d10/f1a 918686 0 2024-05-27T09:29:29.897 INFO:tasks.workunit.client.0.smithi119.stdout:0/650: write d2/d6/d23/d7a/d53/f5f [4890259,96614] 0 2024-05-27T09:29:29.897 INFO:tasks.workunit.client.0.smithi119.stdout:0/651: write d2/d6/d9/d12/d4f/fc2 [106376,44602] 0 2024-05-27T09:29:29.897 INFO:tasks.workunit.client.0.smithi119.stdout:0/652: readlink d2/d1a/d20/l97 0 2024-05-27T09:29:29.899 INFO:tasks.workunit.client.0.smithi119.stdout:8/627: mkdir d3/d6e/d8e/d9b/dce 0 2024-05-27T09:29:29.899 INFO:tasks.workunit.client.0.smithi119.stdout:8/628: chown d3/f16 5 1 2024-05-27T09:29:29.901 INFO:tasks.workunit.client.1.smithi138.stdout:9/132: dread d6/f9 [0,4194304] 0 2024-05-27T09:29:29.902 INFO:tasks.workunit.client.1.smithi138.stdout:6/242: mknod d3/d11/d2d/d37/c58 0 2024-05-27T09:29:29.902 INFO:tasks.workunit.client.1.smithi138.stdout:6/243: fdatasync d3/d19/f38 0 2024-05-27T09:29:29.903 INFO:tasks.workunit.client.1.smithi138.stdout:6/244: read - d3/d35/d43/d1c/f4c zero size 2024-05-27T09:29:29.903 INFO:tasks.workunit.client.1.smithi138.stdout:6/245: creat d3/d35/d43/d1c/f59 x:0 0 0 2024-05-27T09:29:29.903 INFO:tasks.workunit.client.0.smithi119.stdout:0/653: rmdir d2/d1a/d20 39 2024-05-27T09:29:29.905 INFO:tasks.workunit.client.1.smithi138.stdout:5/168: dread d3/d12/d16/f2e [0,4194304] 0 2024-05-27T09:29:29.905 INFO:tasks.workunit.client.1.smithi138.stdout:5/169: creat d3/d12/d16/d1d/d22/f3a x:0 0 0 2024-05-27T09:29:29.905 INFO:tasks.workunit.client.1.smithi138.stdout:2/125: link d1/cc d1/d13/c2c 0 2024-05-27T09:29:29.911 INFO:tasks.workunit.client.1.smithi138.stdout:4/151: dwrite d7/f25 [0,4194304] 0 2024-05-27T09:29:29.911 INFO:tasks.workunit.client.1.smithi138.stdout:4/152: chown d7/df/f1a 42007124 1 2024-05-27T09:29:29.911 INFO:tasks.workunit.client.1.smithi138.stdout:2/126: rmdir d1/d13 39 2024-05-27T09:29:29.913 INFO:tasks.workunit.client.1.smithi138.stdout:2/127: write d1/d13/d17/f11 [480393,61037] 0 2024-05-27T09:29:29.914 INFO:tasks.workunit.client.1.smithi138.stdout:9/133: rename d6/f8 to d6/db/dc/f2c 0 2024-05-27T09:29:29.914 INFO:tasks.workunit.client.1.smithi138.stdout:1/196: dwrite d4/d1a/d2e/f41 [0,4194304] 0 2024-05-27T09:29:29.914 INFO:tasks.workunit.client.1.smithi138.stdout:2/128: getdents d1 0 2024-05-27T09:29:29.914 INFO:tasks.workunit.client.1.smithi138.stdout:2/129: fsync d1/d13/d17/d25/f2a 0 2024-05-27T09:29:29.914 INFO:tasks.workunit.client.1.smithi138.stdout:2/130: truncate d1/d13/d17/f11 685542 0 2024-05-27T09:29:29.924 INFO:tasks.workunit.client.1.smithi138.stdout:5/170: write d3/f7 [2175450,93207] 0 2024-05-27T09:29:29.926 INFO:tasks.workunit.client.1.smithi138.stdout:5/171: mkdir d3/d12/d16/d1d/d23/d3b 0 2024-05-27T09:29:29.926 INFO:tasks.workunit.client.1.smithi138.stdout:5/172: readlink d3/l6 0 2024-05-27T09:29:29.926 INFO:tasks.workunit.client.1.smithi138.stdout:5/173: chown d3/l6 125 1 2024-05-27T09:29:29.928 INFO:tasks.workunit.client.1.smithi138.stdout:8/145: dread d3/d8/f1a [0,4194304] 0 2024-05-27T09:29:29.928 INFO:tasks.workunit.client.1.smithi138.stdout:5/174: unlink d3/lf 0 2024-05-27T09:29:29.931 INFO:tasks.workunit.client.0.smithi119.stdout:8/629: rename d3/d6e/d8e/d9b to d3/d6e/dcf 0 2024-05-27T09:29:29.937 INFO:tasks.workunit.client.0.smithi119.stdout:5/751: dwrite d6/db/dd4/fd6 [0,4194304] 0 2024-05-27T09:29:29.939 INFO:tasks.workunit.client.0.smithi119.stdout:8/630: symlink d3/d8/d1b/d46/d92/ld0 0 2024-05-27T09:29:29.939 INFO:tasks.workunit.client.0.smithi119.stdout:8/631: readlink d3/de/lc8 0 2024-05-27T09:29:29.942 INFO:tasks.workunit.client.1.smithi138.stdout:2/131: rename d1/d13 to d1/d15/d2d 0 2024-05-27T09:29:29.945 INFO:tasks.workunit.client.1.smithi138.stdout:1/197: dread d4/f18 [4194304,4194304] 0 2024-05-27T09:29:29.945 INFO:tasks.workunit.client.1.smithi138.stdout:1/198: write d4/da/f1f [618232,104579] 0 2024-05-27T09:29:29.945 INFO:tasks.workunit.client.1.smithi138.stdout:1/199: chown d4/l15 2716 1 2024-05-27T09:29:29.945 INFO:tasks.workunit.client.0.smithi119.stdout:5/752: creat d6/dd/d46/d47/d4b/dd8/df2/f10a x:0 0 0 2024-05-27T09:29:29.946 INFO:tasks.workunit.client.0.smithi119.stdout:8/632: rmdir d3/d8/d1b/d46/d92 39 2024-05-27T09:29:29.946 INFO:tasks.workunit.client.1.smithi138.stdout:2/132: mkdir d1/d15/d2d/d2e 0 2024-05-27T09:29:29.947 INFO:tasks.workunit.client.0.smithi119.stdout:7/719: dwrite d7/df/d19/d97/d92/db7/dba/f49 [0,4194304] 0 2024-05-27T09:29:29.947 INFO:tasks.workunit.client.0.smithi119.stdout:5/753: mkdir d6/db/dc/d10b 0 2024-05-27T09:29:29.948 INFO:tasks.workunit.client.0.smithi119.stdout:5/754: write d6/db/de/f89 [772191,124147] 0 2024-05-27T09:29:29.948 INFO:tasks.workunit.client.1.smithi138.stdout:2/133: mkdir d1/d2f 0 2024-05-27T09:29:29.952 INFO:tasks.workunit.client.0.smithi119.stdout:8/633: rename d3/d8/lc to d3/d8/d1b/d46/ld1 0 2024-05-27T09:29:29.953 INFO:tasks.workunit.client.0.smithi119.stdout:4/722: dwrite d4/d6/d28/d2f/d36/f72 [0,4194304] 0 2024-05-27T09:29:29.954 INFO:tasks.workunit.client.0.smithi119.stdout:7/720: mknod d7/ce6 0 2024-05-27T09:29:29.961 INFO:tasks.workunit.client.0.smithi119.stdout:5/755: unlink d6/db/de/cce 0 2024-05-27T09:29:29.966 INFO:tasks.workunit.client.0.smithi119.stdout:4/723: mkdir d4/d6/d28/d2f/de6 0 2024-05-27T09:29:29.966 INFO:tasks.workunit.client.0.smithi119.stdout:7/721: creat d7/df/d19/d97/d92/db7/dba/d42/d64/de3/fe7 x:0 0 0 2024-05-27T09:29:29.967 INFO:tasks.workunit.client.0.smithi119.stdout:7/722: write d7/df/d19/d9a/da6/dda/d44/fde [1780156,103380] 0 2024-05-27T09:29:29.967 INFO:tasks.workunit.client.0.smithi119.stdout:7/723: write d7/df/d19/d97/d92/db7/dba/d42/d64/de3/fe7 [963205,36413] 0 2024-05-27T09:29:29.971 INFO:tasks.workunit.client.0.smithi119.stdout:4/724: unlink c0 0 2024-05-27T09:29:29.971 INFO:tasks.workunit.client.0.smithi119.stdout:4/725: creat d4/d23/d24/d5b/da8/fe7 x:0 0 0 2024-05-27T09:29:29.975 INFO:tasks.workunit.client.0.smithi119.stdout:7/724: link d7/df/d19/d97/f9f d7/df/d19/d4d/d89/fe8 0 2024-05-27T09:29:29.976 INFO:tasks.workunit.client.0.smithi119.stdout:8/634: dread d3/d8/d1b/f7a [0,4194304] 0 2024-05-27T09:29:29.979 INFO:tasks.workunit.client.0.smithi119.stdout:4/726: mkdir d4/de/de8 0 2024-05-27T09:29:29.986 INFO:tasks.workunit.client.0.smithi119.stdout:5/756: dread d6/db/dc/f3b [0,4194304] 0 2024-05-27T09:29:29.992 INFO:tasks.workunit.client.1.smithi138.stdout:7/141: dwrite d0/f21 [0,4194304] 0 2024-05-27T09:29:29.993 INFO:tasks.workunit.client.0.smithi119.stdout:5/757: creat d6/d2f/d9d/d100/f10c x:0 0 0 2024-05-27T09:29:29.995 INFO:tasks.workunit.client.1.smithi138.stdout:6/246: dwrite d3/f31 [4194304,4194304] 0 2024-05-27T09:29:29.996 INFO:tasks.workunit.client.0.smithi119.stdout:5/758: mknod d6/dd/d39/d19/d3c/d52/d9c/c10d 0 2024-05-27T09:29:29.996 INFO:tasks.workunit.client.1.smithi138.stdout:7/142: symlink d0/d7/l2e 0 2024-05-27T09:29:30.002 INFO:tasks.workunit.client.0.smithi119.stdout:5/759: creat d6/db/de/f10e x:0 0 0 2024-05-27T09:29:30.006 INFO:tasks.workunit.client.1.smithi138.stdout:8/146: dread d3/d8/f14 [0,4194304] 0 2024-05-27T09:29:30.011 INFO:tasks.workunit.client.0.smithi119.stdout:4/727: write d4/d23/d31/fcc [1146729,105657] 0 2024-05-27T09:29:30.013 INFO:tasks.workunit.client.1.smithi138.stdout:8/147: unlink d3/d8/f1a 0 2024-05-27T09:29:30.013 INFO:tasks.workunit.client.1.smithi138.stdout:8/148: fsync d3/d8/d11/f22 0 2024-05-27T09:29:30.014 INFO:tasks.workunit.client.0.smithi119.stdout:4/728: truncate d4/d6/d3e/f6e 1800794 0 2024-05-27T09:29:30.016 INFO:tasks.workunit.client.0.smithi119.stdout:4/729: getdents d4/d6/d3e/d89/db2 0 2024-05-27T09:29:30.018 INFO:tasks.workunit.client.0.smithi119.stdout:4/730: mknod d4/de/d21/d40/d47/d5c/d9e/ce9 0 2024-05-27T09:29:30.026 INFO:tasks.workunit.client.0.smithi119.stdout:0/654: dwrite d2/d6/d9/f66 [0,4194304] 0 2024-05-27T09:29:30.034 INFO:tasks.workunit.client.1.smithi138.stdout:7/143: write d0/f21 [2077003,124361] 0 2024-05-27T09:29:30.035 INFO:tasks.workunit.client.0.smithi119.stdout:7/725: dread d7/df/f13 [0,4194304] 0 2024-05-27T09:29:30.036 INFO:tasks.workunit.client.1.smithi138.stdout:1/200: dwrite d4/d8/d22/f35 [0,4194304] 0 2024-05-27T09:29:30.036 INFO:tasks.workunit.client.1.smithi138.stdout:4/153: dwrite d7/df/f19 [0,4194304] 0 2024-05-27T09:29:30.037 INFO:tasks.workunit.client.1.smithi138.stdout:7/144: stat d0/c22 0 2024-05-27T09:29:30.037 INFO:tasks.workunit.client.1.smithi138.stdout:7/145: chown d0/d7/l14 50091053 1 2024-05-27T09:29:30.039 INFO:tasks.workunit.client.1.smithi138.stdout:9/134: dwrite d6/db/dc/f2c [0,4194304] 0 2024-05-27T09:29:30.039 INFO:tasks.workunit.client.1.smithi138.stdout:9/135: dread - d6/db/de/f2b zero size 2024-05-27T09:29:30.039 INFO:tasks.workunit.client.1.smithi138.stdout:9/136: fdatasync f3 0 2024-05-27T09:29:30.039 INFO:tasks.workunit.client.1.smithi138.stdout:9/137: write d6/db/dc/f12 [305256,68384] 0 2024-05-27T09:29:30.041 INFO:tasks.workunit.client.0.smithi119.stdout:8/635: dwrite d3/d8/d1b/d77/d95/fab [0,4194304] 0 2024-05-27T09:29:30.041 INFO:tasks.workunit.client.0.smithi119.stdout:0/655: write d2/d6/d75/f86 [596505,82056] 0 2024-05-27T09:29:30.041 INFO:tasks.workunit.client.0.smithi119.stdout:0/656: dread - d2/d6/d9/d24/d30/d4a/fc5 zero size 2024-05-27T09:29:30.042 INFO:tasks.workunit.client.1.smithi138.stdout:4/154: mknod d7/d27/c32 0 2024-05-27T09:29:30.042 INFO:tasks.workunit.client.1.smithi138.stdout:1/201: creat d4/d38/f44 x:0 0 0 2024-05-27T09:29:30.044 INFO:tasks.workunit.client.0.smithi119.stdout:8/636: symlink d3/d8/d4d/d5d/db7/ld2 0 2024-05-27T09:29:30.045 INFO:tasks.workunit.client.0.smithi119.stdout:1/743: sync 2024-05-27T09:29:30.046 INFO:tasks.workunit.client.0.smithi119.stdout:8/637: mknod d3/d6e/cd3 0 2024-05-27T09:29:30.046 INFO:tasks.workunit.client.0.smithi119.stdout:8/638: stat d3/d8/d1b/d46/da4/la6 0 2024-05-27T09:29:30.049 INFO:tasks.workunit.client.0.smithi119.stdout:8/639: mknod d3/d8/d4d/d5d/cd4 0 2024-05-27T09:29:30.049 INFO:tasks.workunit.client.1.smithi138.stdout:7/146: symlink d0/d7/l2f 0 2024-05-27T09:29:30.049 INFO:tasks.workunit.client.1.smithi138.stdout:9/138: mknod d6/da/c2d 0 2024-05-27T09:29:30.050 INFO:tasks.workunit.client.1.smithi138.stdout:1/202: creat d4/d8/d39/f45 x:0 0 0 2024-05-27T09:29:30.055 INFO:tasks.workunit.client.1.smithi138.stdout:3/93: sync 2024-05-27T09:29:30.055 INFO:tasks.workunit.client.1.smithi138.stdout:0/189: sync 2024-05-27T09:29:30.059 INFO:tasks.workunit.client.1.smithi138.stdout:2/134: dwrite d1/d10/f6 [0,4194304] 0 2024-05-27T09:29:30.059 INFO:tasks.workunit.client.0.smithi119.stdout:0/657: read d2/d1a/f21 [364684,64900] 0 2024-05-27T09:29:30.062 INFO:tasks.workunit.client.0.smithi119.stdout:0/658: truncate d2/d6/d9/d12/d4f/fc2 321304 0 2024-05-27T09:29:30.062 INFO:tasks.workunit.client.0.smithi119.stdout:8/640: mknod d3/d8/d1b/d77/cd5 0 2024-05-27T09:29:30.062 INFO:tasks.workunit.client.0.smithi119.stdout:8/641: truncate d3/d8/d1b/f6d 187829 0 2024-05-27T09:29:30.063 INFO:tasks.workunit.client.0.smithi119.stdout:1/744: write d7/db/d12/f9a [486668,114302] 0 2024-05-27T09:29:30.063 INFO:tasks.workunit.client.0.smithi119.stdout:1/745: creat d7/db/d12/d26/d4a/fe5 x:0 0 0 2024-05-27T09:29:30.063 INFO:tasks.workunit.client.0.smithi119.stdout:7/726: write d7/df/d19/d4d/d89/fe8 [1103662,112490] 0 2024-05-27T09:29:30.067 INFO:tasks.workunit.client.1.smithi138.stdout:4/155: dread d7/df/f19 [0,4194304] 0 2024-05-27T09:29:30.067 INFO:tasks.workunit.client.1.smithi138.stdout:4/156: dread - d7/d27/f2c zero size 2024-05-27T09:29:30.068 INFO:tasks.workunit.client.0.smithi119.stdout:8/642: truncate d3/d8/d4d/d5d/d60/f6b 351862 0 2024-05-27T09:29:30.072 INFO:tasks.workunit.client.1.smithi138.stdout:2/135: write d1/d15/d2d/d17/f11 [104807,39621] 0 2024-05-27T09:29:30.075 INFO:tasks.workunit.client.0.smithi119.stdout:1/746: mknod d7/db/ce6 0 2024-05-27T09:29:30.077 INFO:tasks.workunit.client.0.smithi119.stdout:9/583: sync 2024-05-27T09:29:30.077 INFO:tasks.workunit.client.0.smithi119.stdout:2/699: sync 2024-05-27T09:29:30.077 INFO:tasks.workunit.client.0.smithi119.stdout:6/674: sync 2024-05-27T09:29:30.077 INFO:tasks.workunit.client.0.smithi119.stdout:3/826: sync 2024-05-27T09:29:30.077 INFO:tasks.workunit.client.0.smithi119.stdout:3/827: stat db/dc9/d98/d7a/f5c 0 2024-05-27T09:29:30.077 INFO:tasks.workunit.client.0.smithi119.stdout:3/828: readlink db/l26 0 2024-05-27T09:29:30.077 INFO:tasks.workunit.client.0.smithi119.stdout:3/829: write db/d36/d45/d49/d9d/fdc [1930487,73681] 0 2024-05-27T09:29:30.077 INFO:tasks.workunit.client.0.smithi119.stdout:3/830: readlink db/dc9/d98/d7a/la3 0 2024-05-27T09:29:30.077 INFO:tasks.workunit.client.0.smithi119.stdout:3/831: fdatasync db/dc9/d98/fa5 0 2024-05-27T09:29:30.077 INFO:tasks.workunit.client.0.smithi119.stdout:3/832: chown db/f12 0 1 2024-05-27T09:29:30.077 INFO:tasks.workunit.client.0.smithi119.stdout:3/833: write db/d36/d45/dab/f103 [574594,74917] 0 2024-05-27T09:29:30.077 INFO:tasks.workunit.client.0.smithi119.stdout:3/834: write db/fbf [3663291,82876] 0 2024-05-27T09:29:30.079 INFO:tasks.workunit.client.0.smithi119.stdout:7/727: rmdir d7/df/d19/d97/d92/db7 39 2024-05-27T09:29:30.079 INFO:tasks.workunit.client.0.smithi119.stdout:7/728: readlink d7/df/d19/d97/d92/db7/dba/l3c 0 2024-05-27T09:29:30.079 INFO:tasks.workunit.client.0.smithi119.stdout:7/729: readlink d7/df/d19/d9a/da6/dda/l8e 0 2024-05-27T09:29:30.079 INFO:tasks.workunit.client.0.smithi119.stdout:7/730: chown d7/d1a/fa1 272009109 1 2024-05-27T09:29:30.079 INFO:tasks.workunit.client.0.smithi119.stdout:7/731: chown d7/df/d19/d97/d92/db7/dba/d42 50800 1 2024-05-27T09:29:30.083 INFO:tasks.workunit.client.1.smithi138.stdout:9/139: truncate d6/db/dc/f2c 1655159 0 2024-05-27T09:29:30.083 INFO:tasks.workunit.client.0.smithi119.stdout:1/747: rename d7/db/d12/ddb/d38/d3c/fca to d7/db/d52/fe7 0 2024-05-27T09:29:30.083 INFO:tasks.workunit.client.0.smithi119.stdout:1/748: chown d7/db/d8f/d9b 54360 1 2024-05-27T09:29:30.085 INFO:tasks.workunit.client.1.smithi138.stdout:9/140: truncate f3 4027878 0 2024-05-27T09:29:30.085 INFO:tasks.workunit.client.1.smithi138.stdout:9/141: write d6/f9 [459751,31963] 0 2024-05-27T09:29:30.089 INFO:tasks.workunit.client.0.smithi119.stdout:3/835: mknod db/d36/d45/d4d/d108/c10f 0 2024-05-27T09:29:30.089 INFO:tasks.workunit.client.0.smithi119.stdout:3/836: fdatasync db/f83 0 2024-05-27T09:29:30.090 INFO:tasks.workunit.client.0.smithi119.stdout:5/760: dwrite f1 [4194304,4194304] 0 2024-05-27T09:29:30.091 INFO:tasks.workunit.client.0.smithi119.stdout:7/732: unlink d7/d1a/d99/fc8 0 2024-05-27T09:29:30.097 INFO:tasks.workunit.client.0.smithi119.stdout:9/584: rename d18/d19/f85 to d18/d19/d1b/d40/db7/fb8 0 2024-05-27T09:29:30.097 INFO:tasks.workunit.client.0.smithi119.stdout:3/837: unlink db/d36/d45/d4c/d62/d13/c29 0 2024-05-27T09:29:30.097 INFO:tasks.workunit.client.0.smithi119.stdout:9/585: truncate d18/d19/d1b/d29/fa7 173921 0 2024-05-27T09:29:30.097 INFO:tasks.workunit.client.0.smithi119.stdout:5/761: mknod d6/dd/d46/d47/d4b/dd8/df2/c10f 0 2024-05-27T09:29:30.097 INFO:tasks.workunit.client.0.smithi119.stdout:9/586: write d18/d19/fb1 [833324,92131] 0 2024-05-27T09:29:30.097 INFO:tasks.workunit.client.0.smithi119.stdout:9/587: chown d18/d19/d1b/d2a/c66 1 1 2024-05-27T09:29:30.098 INFO:tasks.workunit.client.0.smithi119.stdout:2/700: rmdir d2/da/d8a/dd4/d69/db1 0 2024-05-27T09:29:30.098 INFO:tasks.workunit.client.0.smithi119.stdout:2/701: chown d2/da/d8a/dd4/d89/d94/f8b 789 1 2024-05-27T09:29:30.099 INFO:tasks.workunit.client.0.smithi119.stdout:2/702: fdatasync d2/da/d8a/dd4/fd2 0 2024-05-27T09:29:30.101 INFO:tasks.workunit.client.0.smithi119.stdout:3/838: creat db/d7f/da9/f110 x:0 0 0 2024-05-27T09:29:30.102 INFO:tasks.workunit.client.1.smithi138.stdout:2/136: dread d1/d10/f1a [0,4194304] 0 2024-05-27T09:29:30.104 INFO:tasks.workunit.client.0.smithi119.stdout:1/749: write d7/db/d12/d15/d1f/f3a [1125009,17436] 0 2024-05-27T09:29:30.105 INFO:tasks.workunit.client.0.smithi119.stdout:5/762: symlink d6/dd/d39/d19/d3c/dd5/l110 0 2024-05-27T09:29:30.106 INFO:tasks.workunit.client.1.smithi138.stdout:2/137: truncate d1/d10/fe 61099 0 2024-05-27T09:29:30.109 INFO:tasks.workunit.client.1.smithi138.stdout:2/138: getdents d1/d15/d2d 0 2024-05-27T09:29:30.114 INFO:tasks.workunit.client.0.smithi119.stdout:5/763: rename d6/d2f/da5/fa7 to d6/d2f/da5/f111 0 2024-05-27T09:29:30.114 INFO:tasks.workunit.client.0.smithi119.stdout:5/764: mknod d6/dd/d39/d19/d7d/c112 0 2024-05-27T09:29:30.120 INFO:tasks.workunit.client.0.smithi119.stdout:5/765: symlink d6/dd/d46/l113 0 2024-05-27T09:29:30.120 INFO:tasks.workunit.client.0.smithi119.stdout:5/766: readlink d6/dd/l65 0 2024-05-27T09:29:30.121 INFO:tasks.workunit.client.0.smithi119.stdout:5/767: mkdir d6/dd/d39/d19/d70/dc1/d41/d114 0 2024-05-27T09:29:30.123 INFO:tasks.workunit.client.0.smithi119.stdout:5/768: creat d6/db/d9a/f115 x:0 0 0 2024-05-27T09:29:30.126 INFO:tasks.workunit.client.0.smithi119.stdout:5/769: link d6/dd/d39/d19/d3c/d52/f99 d6/dd/d46/d47/f116 0 2024-05-27T09:29:30.126 INFO:tasks.workunit.client.0.smithi119.stdout:5/770: chown d6/f90 2560 1 2024-05-27T09:29:30.127 INFO:tasks.workunit.client.1.smithi138.stdout:2/139: write d1/d15/d2d/d17/d25/f28 [235833,46017] 0 2024-05-27T09:29:30.136 INFO:tasks.workunit.client.0.smithi119.stdout:9/588: dread d18/d19/d1b/d40/d44/f69 [4194304,4194304] 0 2024-05-27T09:29:30.136 INFO:tasks.workunit.client.0.smithi119.stdout:9/589: write f16 [4382577,87935] 0 2024-05-27T09:29:30.137 INFO:tasks.workunit.client.0.smithi119.stdout:4/731: dwrite d4/de/d21/f85 [0,4194304] 0 2024-05-27T09:29:30.138 INFO:tasks.workunit.client.1.smithi138.stdout:6/247: dwrite d3/d11/d2d/d37/f3c [0,4194304] 0 2024-05-27T09:29:30.140 INFO:tasks.workunit.client.0.smithi119.stdout:4/732: read d4/d6/f2a [336760,70695] 0 2024-05-27T09:29:30.142 INFO:tasks.workunit.client.1.smithi138.stdout:6/248: mkdir d3/d35/d5a 0 2024-05-27T09:29:30.144 INFO:tasks.workunit.client.0.smithi119.stdout:1/750: write d7/f5b [1231426,110765] 0 2024-05-27T09:29:30.145 INFO:tasks.workunit.client.0.smithi119.stdout:1/751: stat d7/db/d12/ddb/d38/d49/c91 0 2024-05-27T09:29:30.145 INFO:tasks.workunit.client.1.smithi138.stdout:6/249: symlink d3/d35/l5b 0 2024-05-27T09:29:30.153 INFO:tasks.workunit.client.0.smithi119.stdout:6/675: dwrite d0/d2/f7f [0,4194304] 0 2024-05-27T09:29:30.154 INFO:tasks.workunit.client.0.smithi119.stdout:6/676: chown d0/d2/d1c/f2a 25019452 1 2024-05-27T09:29:30.174 INFO:tasks.workunit.client.0.smithi119.stdout:3/839: dread db/dc9/f79 [0,4194304] 0 2024-05-27T09:29:30.181 INFO:tasks.workunit.client.0.smithi119.stdout:8/643: dwrite d3/d8/d4d/d5d/d60/f71 [4194304,4194304] 0 2024-05-27T09:29:30.181 INFO:tasks.workunit.client.0.smithi119.stdout:8/644: dread - d3/d8/d4d/d5d/d66/fca zero size 2024-05-27T09:29:30.186 INFO:tasks.workunit.client.0.smithi119.stdout:9/590: rename d18/d19/d1b/d40/c72 to d18/d19/d1b/d40/d44/cb9 0 2024-05-27T09:29:30.189 INFO:tasks.workunit.client.0.smithi119.stdout:0/659: dwrite d2/d6/d75/f8a [0,4194304] 0 2024-05-27T09:29:30.189 INFO:tasks.workunit.client.0.smithi119.stdout:0/660: chown d2/d1a/lb5 4982 1 2024-05-27T09:29:30.189 INFO:tasks.workunit.client.0.smithi119.stdout:0/661: fdatasync d2/f49 0 2024-05-27T09:29:30.202 INFO:tasks.workunit.client.1.smithi138.stdout:7/147: dwrite d0/d7/f1b [4194304,4194304] 0 2024-05-27T09:29:30.203 INFO:tasks.workunit.client.0.smithi119.stdout:4/733: mkdir d4/d23/d24/d5b/da8/dda/dea 0 2024-05-27T09:29:30.205 INFO:tasks.workunit.client.1.smithi138.stdout:8/149: dwrite f1 [0,4194304] 0 2024-05-27T09:29:30.206 INFO:tasks.workunit.client.1.smithi138.stdout:8/150: chown d3 0 1 2024-05-27T09:29:30.210 INFO:tasks.workunit.client.0.smithi119.stdout:1/752: mkdir d7/db/d12/d15/dd2/de8 0 2024-05-27T09:29:30.210 INFO:tasks.workunit.client.0.smithi119.stdout:1/753: chown d7/db/d12/ddb/d38/d3c/cad 278658 1 2024-05-27T09:29:30.211 INFO:tasks.workunit.client.1.smithi138.stdout:8/151: rename d3/d8/c19 to d3/c25 0 2024-05-27T09:29:30.211 INFO:tasks.workunit.client.0.smithi119.stdout:7/733: dwrite d7/df/d19/d97/f9f [4194304,4194304] 0 2024-05-27T09:29:30.213 INFO:tasks.workunit.client.1.smithi138.stdout:6/250: dwrite d3/d19/f1e [0,4194304] 0 2024-05-27T09:29:30.213 INFO:tasks.workunit.client.1.smithi138.stdout:6/251: stat d3/d19/f28 0 2024-05-27T09:29:30.217 INFO:tasks.workunit.client.0.smithi119.stdout:6/677: unlink d0/d2/d57/d37/dad/fda 0 2024-05-27T09:29:30.222 INFO:tasks.workunit.client.1.smithi138.stdout:2/140: dread d1/d10/f6 [0,4194304] 0 2024-05-27T09:29:30.222 INFO:tasks.workunit.client.1.smithi138.stdout:2/141: unlink d1/l1c 0 2024-05-27T09:29:30.225 INFO:tasks.workunit.client.1.smithi138.stdout:3/94: dwrite d0/d8/f1d [0,4194304] 0 2024-05-27T09:29:30.226 INFO:tasks.workunit.client.1.smithi138.stdout:3/95: symlink d0/d2/da/df/d10/l1e 0 2024-05-27T09:29:30.230 INFO:tasks.workunit.client.0.smithi119.stdout:9/591: dread f15 [0,4194304] 0 2024-05-27T09:29:30.230 INFO:tasks.workunit.client.0.smithi119.stdout:9/592: fsync d18/d19/d1b/f1d 0 2024-05-27T09:29:30.230 INFO:tasks.workunit.client.0.smithi119.stdout:9/593: readlink d18/d19/d1b/d2a/lad 0 2024-05-27T09:29:30.231 INFO:tasks.workunit.client.1.smithi138.stdout:1/203: dwrite d4/f12 [0,4194304] 0 2024-05-27T09:29:30.231 INFO:tasks.workunit.client.1.smithi138.stdout:3/96: rename d0/d2/c17 to d0/d2/da/df/d10/c1f 0 2024-05-27T09:29:30.231 INFO:tasks.workunit.client.1.smithi138.stdout:3/97: write d0/f12 [152909,45620] 0 2024-05-27T09:29:30.250 INFO:tasks.workunit.client.0.smithi119.stdout:8/645: unlink d3/d8/d4d/d5d/db7/ld2 0 2024-05-27T09:29:30.259 INFO:tasks.workunit.client.1.smithi138.stdout:4/157: dwrite d7/f17 [0,4194304] 0 2024-05-27T09:29:30.259 INFO:tasks.workunit.client.1.smithi138.stdout:1/204: write d4/f18 [4723111,43038] 0 2024-05-27T09:29:30.264 INFO:tasks.workunit.client.1.smithi138.stdout:4/158: link d7/df/f29 d7/df/f33 0 2024-05-27T09:29:30.264 INFO:tasks.workunit.client.1.smithi138.stdout:4/159: truncate d7/d27/f2c 589234 0 2024-05-27T09:29:30.264 INFO:tasks.workunit.client.1.smithi138.stdout:4/160: fsync d7/df/f1d 0 2024-05-27T09:29:30.265 INFO:tasks.workunit.client.1.smithi138.stdout:4/161: fdatasync d7/f18 0 2024-05-27T09:29:30.269 INFO:tasks.workunit.client.0.smithi119.stdout:4/734: rename d4/d6/d28/d2f/d36/lc0 to d4/d6/d81/dce/d88/leb 0 2024-05-27T09:29:30.269 INFO:tasks.workunit.client.0.smithi119.stdout:4/735: chown d4/d23/d24/l71 22155751 1 2024-05-27T09:29:30.270 INFO:tasks.workunit.client.1.smithi138.stdout:0/190: dwrite da/d17/d1d/d1f/f21 [0,4194304] 0 2024-05-27T09:29:30.289 INFO:tasks.workunit.client.1.smithi138.stdout:9/142: dwrite d6/db/de/f2b [0,4194304] 0 2024-05-27T09:29:30.295 INFO:tasks.workunit.client.0.smithi119.stdout:5/771: write d6/db/dc/f3b [1508787,42108] 0 2024-05-27T09:29:30.303 INFO:tasks.workunit.client.0.smithi119.stdout:4/736: dread d4/d23/d24/f82 [0,4194304] 0 2024-05-27T09:29:30.303 INFO:tasks.workunit.client.0.smithi119.stdout:9/594: dwrite d18/d19/d1b/d40/d44/f93 [0,4194304] 0 2024-05-27T09:29:30.312 INFO:tasks.workunit.client.0.smithi119.stdout:1/754: dwrite d7/db/d35/fa0 [4194304,4194304] 0 2024-05-27T09:29:30.312 INFO:tasks.workunit.client.0.smithi119.stdout:7/734: creat d7/df/d19/d97/d92/db7/dba/d42/fe9 x:0 0 0 2024-05-27T09:29:30.312 INFO:tasks.workunit.client.0.smithi119.stdout:6/678: mknod d0/d2/d1c/d21/d58/dd3/ced 0 2024-05-27T09:29:30.312 INFO:tasks.workunit.client.0.smithi119.stdout:3/840: unlink db/d36/d45/d4c/l92 0 2024-05-27T09:29:30.313 INFO:tasks.workunit.client.1.smithi138.stdout:8/152: dwrite d3/d8/f14 [0,4194304] 0 2024-05-27T09:29:30.318 INFO:tasks.workunit.client.1.smithi138.stdout:8/153: creat d3/d8/d11/f26 x:0 0 0 2024-05-27T09:29:30.321 INFO:tasks.workunit.client.0.smithi119.stdout:8/646: link d3/f16 d3/d8/d4d/d5d/fd6 0 2024-05-27T09:29:30.321 INFO:tasks.workunit.client.0.smithi119.stdout:0/662: symlink d2/d6/d23/d7a/ld6 0 2024-05-27T09:29:30.321 INFO:tasks.workunit.client.0.smithi119.stdout:5/772: mknod d6/dd/d6e/c117 0 2024-05-27T09:29:30.321 INFO:tasks.workunit.client.0.smithi119.stdout:9/595: unlink d18/d19/d1b/c25 0 2024-05-27T09:29:30.321 INFO:tasks.workunit.client.0.smithi119.stdout:0/663: fdatasync d2/d6/d9/d24/d30/d6b/f70 0 2024-05-27T09:29:30.321 INFO:tasks.workunit.client.0.smithi119.stdout:0/664: dread - d2/d6/d9/d12/d4b/fa7 zero size 2024-05-27T09:29:30.323 INFO:tasks.workunit.client.1.smithi138.stdout:8/154: truncate d3/d8/fc 646087 0 2024-05-27T09:29:30.326 INFO:tasks.workunit.client.0.smithi119.stdout:4/737: truncate f3 23423 0 2024-05-27T09:29:30.326 INFO:tasks.workunit.client.0.smithi119.stdout:4/738: truncate d4/de/d21/d40/fc6 68817 0 2024-05-27T09:29:30.326 INFO:tasks.workunit.client.0.smithi119.stdout:4/739: readlink d4/d6/d3e/d89/lbc 0 2024-05-27T09:29:30.327 INFO:tasks.workunit.client.0.smithi119.stdout:7/735: rename d7/c39 to d7/d1a/d99/dbb/cea 0 2024-05-27T09:29:30.328 INFO:tasks.workunit.client.0.smithi119.stdout:6/679: creat d0/d2/d4e/dc5/fee x:0 0 0 2024-05-27T09:29:30.329 INFO:tasks.workunit.client.0.smithi119.stdout:9/596: write d18/d19/d1b/d40/d7b/f7e [881157,123274] 0 2024-05-27T09:29:30.330 INFO:tasks.workunit.client.1.smithi138.stdout:3/98: dwrite d0/d2/da/f1b [0,4194304] 0 2024-05-27T09:29:30.333 INFO:tasks.workunit.client.0.smithi119.stdout:6/680: symlink d0/d2/d1c/d21/d58/dd3/lef 0 2024-05-27T09:29:30.333 INFO:tasks.workunit.client.1.smithi138.stdout:3/99: mknod d0/d2/c20 0 2024-05-27T09:29:30.334 INFO:tasks.workunit.client.1.smithi138.stdout:3/100: mknod d0/d2/da/c21 0 2024-05-27T09:29:30.337 INFO:tasks.workunit.client.0.smithi119.stdout:1/755: write d7/dd/d25/f86 [1598864,92978] 0 2024-05-27T09:29:30.337 INFO:tasks.workunit.client.0.smithi119.stdout:1/756: chown d7/dd/d44/faf 33 1 2024-05-27T09:29:30.337 INFO:tasks.workunit.client.0.smithi119.stdout:1/757: fsync d7/d6e/fe4 0 2024-05-27T09:29:30.337 INFO:tasks.workunit.client.0.smithi119.stdout:9/597: getdents d18/d19/d1b/d2a 0 2024-05-27T09:29:30.340 INFO:tasks.workunit.client.0.smithi119.stdout:0/665: rename d2/d6/d9/d24/d30/d6b to d2/d6/d9/d12/d4b/d59/dd7 0 2024-05-27T09:29:30.343 INFO:tasks.workunit.client.1.smithi138.stdout:4/162: dwrite d7/df/f19 [0,4194304] 0 2024-05-27T09:29:30.350 INFO:tasks.workunit.client.1.smithi138.stdout:0/191: write da/f3c [973543,91044] 0 2024-05-27T09:29:30.354 INFO:tasks.workunit.client.1.smithi138.stdout:7/148: dwrite d0/f10 [0,4194304] 0 2024-05-27T09:29:30.358 INFO:tasks.workunit.client.0.smithi119.stdout:8/647: dread d3/f1a [0,4194304] 0 2024-05-27T09:29:30.358 INFO:tasks.workunit.client.0.smithi119.stdout:8/648: write d3/d6e/d8e/fb0 [395411,50473] 0 2024-05-27T09:29:30.358 INFO:tasks.workunit.client.0.smithi119.stdout:8/649: write d3/d8/f7e [417254,51363] 0 2024-05-27T09:29:30.359 INFO:tasks.workunit.client.1.smithi138.stdout:4/163: rename d7/f17 to d7/d27/f34 0 2024-05-27T09:29:30.359 INFO:tasks.workunit.client.1.smithi138.stdout:4/164: read - d7/f26 zero size 2024-05-27T09:29:30.359 INFO:tasks.workunit.client.0.smithi119.stdout:8/650: creat d3/d8/d4d/d5d/d66/d85/fd7 x:0 0 0 2024-05-27T09:29:30.363 INFO:tasks.workunit.client.1.smithi138.stdout:0/192: truncate da/d17/f31 3223 0 2024-05-27T09:29:30.365 INFO:tasks.workunit.client.1.smithi138.stdout:4/165: mkdir d7/df/d35 0 2024-05-27T09:29:30.367 INFO:tasks.workunit.client.1.smithi138.stdout:3/101: write d0/d2/da/f1b [2704796,41303] 0 2024-05-27T09:29:30.367 INFO:tasks.workunit.client.1.smithi138.stdout:3/102: creat d0/d2/f22 x:0 0 0 2024-05-27T09:29:30.373 INFO:tasks.workunit.client.1.smithi138.stdout:2/142: dwrite d1/d15/d2d/d17/f11 [0,4194304] 0 2024-05-27T09:29:30.380 INFO:tasks.workunit.client.0.smithi119.stdout:8/651: creat d3/d6e/dcf/dce/fd8 x:0 0 0 2024-05-27T09:29:30.380 INFO:tasks.workunit.client.1.smithi138.stdout:5/175: sync 2024-05-27T09:29:30.380 INFO:tasks.workunit.client.1.smithi138.stdout:5/176: chown d3/l34 1543 1 2024-05-27T09:29:30.380 INFO:tasks.workunit.client.1.smithi138.stdout:5/177: write f0 [8288723,12898] 0 2024-05-27T09:29:30.384 INFO:tasks.workunit.client.1.smithi138.stdout:1/205: dwrite d4/d38/f44 [0,4194304] 0 2024-05-27T09:29:30.386 INFO:tasks.workunit.client.1.smithi138.stdout:2/143: unlink d1/d15/l24 0 2024-05-27T09:29:30.387 INFO:tasks.workunit.client.1.smithi138.stdout:2/144: chown d1/d15/f19 1573 1 2024-05-27T09:29:30.392 INFO:tasks.workunit.client.0.smithi119.stdout:2/703: sync 2024-05-27T09:29:30.396 INFO:tasks.workunit.client.1.smithi138.stdout:5/178: dread d3/f21 [0,4194304] 0 2024-05-27T09:29:30.399 INFO:tasks.workunit.client.1.smithi138.stdout:5/179: unlink d3/f7 0 2024-05-27T09:29:30.400 INFO:tasks.workunit.client.0.smithi119.stdout:5/773: dread d6/db/de/d27/f6b [0,4194304] 0 2024-05-27T09:29:30.401 INFO:tasks.workunit.client.1.smithi138.stdout:5/180: link d3/d12/f25 d3/d12/d16/f3c 0 2024-05-27T09:29:30.403 INFO:tasks.workunit.client.0.smithi119.stdout:5/774: rename d6/db/f4c to d6/dd/d6e/f118 0 2024-05-27T09:29:30.403 INFO:tasks.workunit.client.0.smithi119.stdout:5/775: fdatasync d6/db/de/fdc 0 2024-05-27T09:29:30.404 INFO:tasks.workunit.client.0.smithi119.stdout:5/776: fsync d6/d2f/da5/f111 0 2024-05-27T09:29:30.411 INFO:tasks.workunit.client.0.smithi119.stdout:2/704: write d2/f47 [1939510,74521] 0 2024-05-27T09:29:30.414 INFO:tasks.workunit.client.0.smithi119.stdout:8/652: dread d3/d8/d4d/d5d/fd6 [0,4194304] 0 2024-05-27T09:29:30.417 INFO:tasks.workunit.client.0.smithi119.stdout:2/705: mknod d2/da/d8a/dd4/d69/cef 0 2024-05-27T09:29:30.420 INFO:tasks.workunit.client.0.smithi119.stdout:3/841: dwrite db/d36/d45/d4c/d62/d13/d33/f104 [4194304,4194304] 0 2024-05-27T09:29:30.420 INFO:tasks.workunit.client.0.smithi119.stdout:3/842: write db/d36/d45/d4c/d62/d13/f3d [5213024,87396] 0 2024-05-27T09:29:30.420 INFO:tasks.workunit.client.0.smithi119.stdout:3/843: creat db/d7f/da9/f111 x:0 0 0 2024-05-27T09:29:30.425 INFO:tasks.workunit.client.1.smithi138.stdout:4/166: dread d7/df/f19 [0,4194304] 0 2024-05-27T09:29:30.431 INFO:tasks.workunit.client.0.smithi119.stdout:4/740: dwrite d4/d1c/d39/f57 [0,4194304] 0 2024-05-27T09:29:30.441 INFO:tasks.workunit.client.0.smithi119.stdout:4/741: getdents d4/de/d66 0 2024-05-27T09:29:30.441 INFO:tasks.workunit.client.0.smithi119.stdout:4/742: write f1 [2101206,8519] 0 2024-05-27T09:29:30.442 INFO:tasks.workunit.client.1.smithi138.stdout:9/143: dwrite d6/f9 [0,4194304] 0 2024-05-27T09:29:30.442 INFO:tasks.workunit.client.1.smithi138.stdout:9/144: fdatasync d6/db/f10 0 2024-05-27T09:29:30.445 INFO:tasks.workunit.client.1.smithi138.stdout:0/193: write da/d17/d1d/d1f/f38 [2845591,35979] 0 2024-05-27T09:29:30.445 INFO:tasks.workunit.client.1.smithi138.stdout:0/194: creat da/d17/f44 x:0 0 0 2024-05-27T09:29:30.446 INFO:tasks.workunit.client.1.smithi138.stdout:8/155: dwrite d3/d8/f14 [0,4194304] 0 2024-05-27T09:29:30.446 INFO:tasks.workunit.client.1.smithi138.stdout:8/156: chown d3/d8/c23 2307 1 2024-05-27T09:29:30.452 INFO:tasks.workunit.client.0.smithi119.stdout:4/743: dread d4/d6/d28/f7b [0,4194304] 0 2024-05-27T09:29:30.453 INFO:tasks.workunit.client.1.smithi138.stdout:1/206: dread d4/f12 [0,4194304] 0 2024-05-27T09:29:30.461 INFO:tasks.workunit.client.1.smithi138.stdout:7/149: dwrite d0/f24 [0,4194304] 0 2024-05-27T09:29:30.464 INFO:tasks.workunit.client.0.smithi119.stdout:6/681: dwrite d0/d2/fac [0,4194304] 0 2024-05-27T09:29:30.465 INFO:tasks.workunit.client.0.smithi119.stdout:2/706: write d2/da/d8a/dd4/d89/d94/f68 [2679771,130954] 0 2024-05-27T09:29:30.465 INFO:tasks.workunit.client.0.smithi119.stdout:2/707: chown l0 11341 1 2024-05-27T09:29:30.466 INFO:tasks.workunit.client.0.smithi119.stdout:6/682: symlink d0/d2/d57/d37/d74/lf0 0 2024-05-27T09:29:30.468 INFO:tasks.workunit.client.0.smithi119.stdout:6/683: dread - d0/d2/d57/d37/d74/d65/fc8 zero size 2024-05-27T09:29:30.468 INFO:tasks.workunit.client.0.smithi119.stdout:6/684: write d0/d2/d4e/d5a/d76/fc6 [556534,26552] 0 2024-05-27T09:29:30.469 INFO:tasks.workunit.client.0.smithi119.stdout:2/708: symlink d2/da/d8a/dd4/d89/d94/lf0 0 2024-05-27T09:29:30.481 INFO:tasks.workunit.client.1.smithi138.stdout:0/195: dread da/f22 [0,4194304] 0 2024-05-27T09:29:30.483 INFO:tasks.workunit.client.0.smithi119.stdout:5/777: dwrite d6/db/de/d27/dc9/fb4 [0,4194304] 0 2024-05-27T09:29:30.485 INFO:tasks.workunit.client.1.smithi138.stdout:5/181: write d3/fd [8270050,62405] 0 2024-05-27T09:29:30.498 INFO:tasks.workunit.client.1.smithi138.stdout:9/145: truncate f2 1977985 0 2024-05-27T09:29:30.498 INFO:tasks.workunit.client.1.smithi138.stdout:9/146: readlink d6/db/de/l20 0 2024-05-27T09:29:30.510 INFO:tasks.workunit.client.1.smithi138.stdout:7/150: dread - d0/f1d zero size 2024-05-27T09:29:30.510 INFO:tasks.workunit.client.1.smithi138.stdout:7/151: fsync d0/f21 0 2024-05-27T09:29:30.510 INFO:tasks.workunit.client.1.smithi138.stdout:7/152: chown d0/d7/c17 1 1 2024-05-27T09:29:30.511 INFO:tasks.workunit.client.0.smithi119.stdout:8/653: read d3/d8/d4d/d5d/d60/f71 [7755739,104822] 0 2024-05-27T09:29:30.511 INFO:tasks.workunit.client.0.smithi119.stdout:8/654: stat d3/f8c 0 2024-05-27T09:29:30.511 INFO:tasks.workunit.client.0.smithi119.stdout:8/655: fsync d3/f25 0 2024-05-27T09:29:30.511 INFO:tasks.workunit.client.0.smithi119.stdout:8/656: chown d3 1763799362 1 2024-05-27T09:29:30.512 INFO:tasks.workunit.client.0.smithi119.stdout:8/657: fsync d3/f8c 0 2024-05-27T09:29:30.512 INFO:tasks.workunit.client.1.smithi138.stdout:4/167: dwrite f0 [0,4194304] 0 2024-05-27T09:29:30.512 INFO:tasks.workunit.client.0.smithi119.stdout:6/685: write d0/f84 [624894,64898] 0 2024-05-27T09:29:30.512 INFO:tasks.workunit.client.0.smithi119.stdout:6/686: readlink d0/d2/d1c/l71 0 2024-05-27T09:29:30.514 INFO:tasks.workunit.client.0.smithi119.stdout:8/658: symlink d3/ld9 0 2024-05-27T09:29:30.516 INFO:tasks.workunit.client.1.smithi138.stdout:0/196: dread da/f10 [0,4194304] 0 2024-05-27T09:29:30.516 INFO:tasks.workunit.client.1.smithi138.stdout:0/197: write f5 [220082,89756] 0 2024-05-27T09:29:30.517 INFO:tasks.workunit.client.1.smithi138.stdout:0/198: chown da/d17/f31 2021974 1 2024-05-27T09:29:30.523 INFO:tasks.workunit.client.0.smithi119.stdout:8/659: creat d3/d8/d4d/d5d/d60/fda x:0 0 0 2024-05-27T09:29:30.525 INFO:tasks.workunit.client.1.smithi138.stdout:4/168: symlink d7/d27/l36 0 2024-05-27T09:29:30.525 INFO:tasks.workunit.client.1.smithi138.stdout:4/169: chown d7/c1f 0 1 2024-05-27T09:29:30.538 INFO:tasks.workunit.client.1.smithi138.stdout:8/157: dwrite d3/d8/f14 [4194304,4194304] 0 2024-05-27T09:29:30.538 INFO:tasks.workunit.client.1.smithi138.stdout:8/158: write d3/d8/d11/f26 [842887,58898] 0 2024-05-27T09:29:30.543 INFO:tasks.workunit.client.1.smithi138.stdout:8/159: unlink d3/c25 0 2024-05-27T09:29:30.544 INFO:tasks.workunit.client.1.smithi138.stdout:3/103: dwrite d0/d2/f22 [0,4194304] 0 2024-05-27T09:29:30.547 INFO:tasks.workunit.client.1.smithi138.stdout:3/104: symlink d0/d2/l23 0 2024-05-27T09:29:30.547 INFO:tasks.workunit.client.1.smithi138.stdout:3/105: chown d0/d2/da/cb 5239038 1 2024-05-27T09:29:30.548 INFO:tasks.workunit.client.1.smithi138.stdout:4/170: write f0 [534606,32025] 0 2024-05-27T09:29:30.570 INFO:tasks.workunit.client.1.smithi138.stdout:7/153: write d0/f10 [3210960,7625] 0 2024-05-27T09:29:30.573 INFO:tasks.workunit.client.1.smithi138.stdout:9/147: dwrite d6/f9 [0,4194304] 0 2024-05-27T09:29:30.573 INFO:tasks.workunit.client.1.smithi138.stdout:9/148: write d6/db/dc/f1e [1220911,73585] 0 2024-05-27T09:29:30.573 INFO:tasks.workunit.client.1.smithi138.stdout:9/149: dread - d6/db/f14 zero size 2024-05-27T09:29:30.574 INFO:tasks.workunit.client.1.smithi138.stdout:9/150: truncate d6/db/f1b 1135129 0 2024-05-27T09:29:30.575 INFO:tasks.workunit.client.1.smithi138.stdout:9/151: truncate d6/db/dc/f12 871419 0 2024-05-27T09:29:30.575 INFO:tasks.workunit.client.1.smithi138.stdout:0/199: dread f1 [4194304,4194304] 0 2024-05-27T09:29:30.577 INFO:tasks.workunit.client.1.smithi138.stdout:0/200: mknod da/d17/d1d/d43/c45 0 2024-05-27T09:29:30.578 INFO:tasks.workunit.client.1.smithi138.stdout:0/201: symlink da/d2a/d2b/l46 0 2024-05-27T09:29:30.580 INFO:tasks.workunit.client.1.smithi138.stdout:4/171: dread d7/d27/f34 [0,4194304] 0 2024-05-27T09:29:30.580 INFO:tasks.workunit.client.1.smithi138.stdout:8/160: fsync d3/d8/d11/f26 0 2024-05-27T09:29:30.580 INFO:tasks.workunit.client.1.smithi138.stdout:2/145: dwrite d1/d10/f6 [0,4194304] 0 2024-05-27T09:29:30.581 INFO:tasks.workunit.client.1.smithi138.stdout:0/202: unlink da/d17/d1d/l28 0 2024-05-27T09:29:30.584 INFO:tasks.workunit.client.1.smithi138.stdout:5/182: dwrite d3/f10 [0,4194304] 0 2024-05-27T09:29:30.588 INFO:tasks.workunit.client.1.smithi138.stdout:4/172: unlink f0 0 2024-05-27T09:29:30.588 INFO:tasks.workunit.client.1.smithi138.stdout:8/161: mknod d3/d8/d24/c27 0 2024-05-27T09:29:30.588 INFO:tasks.workunit.client.1.smithi138.stdout:4/173: creat d7/d27/f37 x:0 0 0 2024-05-27T09:29:30.588 INFO:tasks.workunit.client.1.smithi138.stdout:2/146: symlink d1/d15/d2d/d17/l30 0 2024-05-27T09:29:30.588 INFO:tasks.workunit.client.1.smithi138.stdout:2/147: fdatasync d1/f1b 0 2024-05-27T09:29:30.591 INFO:tasks.workunit.client.0.smithi119.stdout:0/666: dwrite d2/f8c [0,4194304] 0 2024-05-27T09:29:30.591 INFO:tasks.workunit.client.0.smithi119.stdout:7/736: dwrite d7/df/d19/d1d/f87 [0,4194304] 0 2024-05-27T09:29:30.593 INFO:tasks.workunit.client.1.smithi138.stdout:8/162: mkdir d3/d8/d28 0 2024-05-27T09:29:30.594 INFO:tasks.workunit.client.1.smithi138.stdout:2/148: mkdir d1/d15/d2d/d31 0 2024-05-27T09:29:30.596 INFO:tasks.workunit.client.1.smithi138.stdout:2/149: creat d1/d15/d2d/d17/d25/f32 x:0 0 0 2024-05-27T09:29:30.597 INFO:tasks.workunit.client.0.smithi119.stdout:8/660: dwrite d3/de/f68 [0,4194304] 0 2024-05-27T09:29:30.604 INFO:tasks.workunit.client.1.smithi138.stdout:7/154: dread d0/f24 [0,4194304] 0 2024-05-27T09:29:30.606 INFO:tasks.workunit.client.1.smithi138.stdout:1/207: rename d4/c21 to d4/d38/c46 0 2024-05-27T09:29:30.606 INFO:tasks.workunit.client.1.smithi138.stdout:1/208: readlink d4/d8/d22/d29/d2b/l3e 0 2024-05-27T09:29:30.608 INFO:tasks.workunit.client.1.smithi138.stdout:1/209: rmdir d4/d8 39 2024-05-27T09:29:30.615 INFO:tasks.workunit.client.1.smithi138.stdout:0/203: dread da/d17/d1d/d1f/f38 [0,4194304] 0 2024-05-27T09:29:30.618 INFO:tasks.workunit.client.0.smithi119.stdout:7/737: rmdir d7/df/d19/d78 39 2024-05-27T09:29:30.619 INFO:tasks.workunit.client.0.smithi119.stdout:8/661: creat d3/d8/d1b/d77/fdb x:0 0 0 2024-05-27T09:29:30.621 INFO:tasks.workunit.client.0.smithi119.stdout:7/738: mkdir d7/df/d19/d4d/d89/deb 0 2024-05-27T09:29:30.623 INFO:tasks.workunit.client.0.smithi119.stdout:2/709: dwrite d2/da/f2e [0,4194304] 0 2024-05-27T09:29:30.624 INFO:tasks.workunit.client.1.smithi138.stdout:2/150: dread d1/d10/f1a [0,4194304] 0 2024-05-27T09:29:30.624 INFO:tasks.workunit.client.1.smithi138.stdout:2/151: creat d1/d2f/f33 x:0 0 0 2024-05-27T09:29:30.626 INFO:tasks.workunit.client.1.smithi138.stdout:2/152: mkdir d1/d15/d2d/d17/d25/d34 0 2024-05-27T09:29:30.631 INFO:tasks.workunit.client.0.smithi119.stdout:7/739: creat d7/df/d19/d4d/d89/dc4/fec x:0 0 0 2024-05-27T09:29:30.632 INFO:tasks.workunit.client.0.smithi119.stdout:2/710: rmdir d2/da/d8a 39 2024-05-27T09:29:30.633 INFO:tasks.workunit.client.0.smithi119.stdout:7/740: readlink d7/df/l26 0 2024-05-27T09:29:30.641 INFO:tasks.workunit.client.0.smithi119.stdout:2/711: creat d2/da/d15/d53/ff1 x:0 0 0 2024-05-27T09:29:30.647 INFO:tasks.workunit.client.0.smithi119.stdout:2/712: rename d2/da/d15/f5f to d2/da/d4e/ff2 0 2024-05-27T09:29:30.651 INFO:tasks.workunit.client.0.smithi119.stdout:6/687: dread d0/d2/d1c/fd9 [0,4194304] 0 2024-05-27T09:29:30.660 INFO:tasks.workunit.client.0.smithi119.stdout:6/688: unlink d0/d2/d1c/d21/d58/f61 0 2024-05-27T09:29:30.661 INFO:tasks.workunit.client.0.smithi119.stdout:6/689: mkdir d0/d2/d57/d37/d74/df1 0 2024-05-27T09:29:30.661 INFO:tasks.workunit.client.0.smithi119.stdout:6/690: write d0/d2/d1c/f2a [8458630,49305] 0 2024-05-27T09:29:30.661 INFO:tasks.workunit.client.0.smithi119.stdout:6/691: readlink d0/d2/d1c/l71 0 2024-05-27T09:29:30.662 INFO:tasks.workunit.client.0.smithi119.stdout:6/692: write d0/d2/d57/d37/f43 [1018825,23174] 0 2024-05-27T09:29:30.663 INFO:tasks.workunit.client.0.smithi119.stdout:6/693: symlink d0/d2/d57/d37/d74/lf2 0 2024-05-27T09:29:30.667 INFO:tasks.workunit.client.0.smithi119.stdout:8/662: getdents d3/d8/d4d/d5d/d60 0 2024-05-27T09:29:30.668 INFO:tasks.workunit.client.0.smithi119.stdout:6/694: symlink d0/d2/d1c/d21/lf3 0 2024-05-27T09:29:30.669 INFO:tasks.workunit.client.1.smithi138.stdout:0/204: write f1 [5644885,26778] 0 2024-05-27T09:29:30.669 INFO:tasks.workunit.client.0.smithi119.stdout:3/844: dwrite db/d36/d45/d4c/d62/d13/f99 [0,4194304] 0 2024-05-27T09:29:30.669 INFO:tasks.workunit.client.0.smithi119.stdout:3/845: write db/d7f/da9/ff7 [176452,92663] 0 2024-05-27T09:29:30.670 INFO:tasks.workunit.client.0.smithi119.stdout:8/663: symlink d3/d6e/d80/d84/ldc 0 2024-05-27T09:29:30.673 INFO:tasks.workunit.client.0.smithi119.stdout:0/667: write d2/d6/d9/d2f/fc1 [495172,20748] 0 2024-05-27T09:29:30.676 INFO:tasks.workunit.client.0.smithi119.stdout:8/664: link d3/d8/d4d/d5d/d66/l73 d3/d8/d1b/ldd 0 2024-05-27T09:29:30.680 INFO:tasks.workunit.client.0.smithi119.stdout:6/695: getdents d0/d11 0 2024-05-27T09:29:30.681 INFO:tasks.workunit.client.0.smithi119.stdout:0/668: creat d2/d6/d9/d12/d4b/d59/dd7/fd8 x:0 0 0 2024-05-27T09:29:30.684 INFO:tasks.workunit.client.1.smithi138.stdout:2/153: dread d1/d15/f19 [0,4194304] 0 2024-05-27T09:29:30.684 INFO:tasks.workunit.client.1.smithi138.stdout:2/154: chown d1/d2f 875051 1 2024-05-27T09:29:30.684 INFO:tasks.workunit.client.1.smithi138.stdout:7/155: dread d0/d7/f16 [0,4194304] 0 2024-05-27T09:29:30.684 INFO:tasks.workunit.client.0.smithi119.stdout:6/696: link d0/d2/d1c/d21/lf3 d0/d2/d4e/d5a/lf4 0 2024-05-27T09:29:30.684 INFO:tasks.workunit.client.0.smithi119.stdout:6/697: write d0/d2/d4e/d5a/f91 [962452,59841] 0 2024-05-27T09:29:30.687 INFO:tasks.workunit.client.1.smithi138.stdout:5/183: dwrite d3/d12/d16/f3c [0,4194304] 0 2024-05-27T09:29:30.688 INFO:tasks.workunit.client.0.smithi119.stdout:6/698: creat d0/ff5 x:0 0 0 2024-05-27T09:29:30.689 INFO:tasks.workunit.client.1.smithi138.stdout:5/184: unlink d3/f11 0 2024-05-27T09:29:30.689 INFO:tasks.workunit.client.1.smithi138.stdout:5/185: stat d3/l1a 0 2024-05-27T09:29:30.689 INFO:tasks.workunit.client.1.smithi138.stdout:5/186: dread - d3/d12/d16/d1d/d22/d2b/f38 zero size 2024-05-27T09:29:30.690 INFO:tasks.workunit.client.0.smithi119.stdout:6/699: creat d0/d19/ff6 x:0 0 0 2024-05-27T09:29:30.691 INFO:tasks.workunit.client.1.smithi138.stdout:5/187: symlink d3/d12/d16/d1d/d23/d3b/l3d 0 2024-05-27T09:29:30.693 INFO:tasks.workunit.client.1.smithi138.stdout:4/174: dwrite d7/fb [0,4194304] 0 2024-05-27T09:29:30.694 INFO:tasks.workunit.client.0.smithi119.stdout:6/700: symlink d0/d85/lf7 0 2024-05-27T09:29:30.696 INFO:tasks.workunit.client.1.smithi138.stdout:0/205: rmdir da 39 2024-05-27T09:29:30.697 INFO:tasks.workunit.client.0.smithi119.stdout:6/701: truncate d0/d2/fb1 19733 0 2024-05-27T09:29:30.699 INFO:tasks.workunit.client.0.smithi119.stdout:8/665: write d3/d8/d1b/d46/f81 [1113626,84562] 0 2024-05-27T09:29:30.699 INFO:tasks.workunit.client.0.smithi119.stdout:8/666: stat d3/d8/d4d/d5d/d60/fda 0 2024-05-27T09:29:30.699 INFO:tasks.workunit.client.0.smithi119.stdout:8/667: write d3/d8/d4d/d5d/fd6 [2418457,79076] 0 2024-05-27T09:29:30.700 INFO:tasks.workunit.client.1.smithi138.stdout:9/152: dwrite f2 [0,4194304] 0 2024-05-27T09:29:30.713 INFO:tasks.workunit.client.1.smithi138.stdout:2/155: dread d1/d15/d2d/d17/f11 [0,4194304] 0 2024-05-27T09:29:30.714 INFO:tasks.workunit.client.0.smithi119.stdout:7/741: dwrite d7/d1a/f59 [0,4194304] 0 2024-05-27T09:29:30.716 INFO:tasks.workunit.client.1.smithi138.stdout:1/210: dread d4/d8/d22/f35 [0,4194304] 0 2024-05-27T09:29:30.724 INFO:tasks.workunit.client.1.smithi138.stdout:1/211: mknod d4/d38/c47 0 2024-05-27T09:29:30.731 INFO:tasks.workunit.client.1.smithi138.stdout:8/163: dwrite d3/d8/d11/f26 [0,4194304] 0 2024-05-27T09:29:30.734 INFO:tasks.workunit.client.0.smithi119.stdout:2/713: dwrite d2/da/d8a/dd4/d89/d94/fd3 [0,4194304] 0 2024-05-27T09:29:30.737 INFO:tasks.workunit.client.0.smithi119.stdout:6/702: write d0/d2/fb1 [128457,94023] 0 2024-05-27T09:29:30.741 INFO:tasks.workunit.client.0.smithi119.stdout:6/703: mknod d0/cf8 0 2024-05-27T09:29:30.745 INFO:tasks.workunit.client.1.smithi138.stdout:0/206: write f1 [1954154,96815] 0 2024-05-27T09:29:30.745 INFO:tasks.workunit.client.1.smithi138.stdout:0/207: read da/d17/d1d/d1f/f38 [481718,61336] 0 2024-05-27T09:29:30.745 INFO:tasks.workunit.client.1.smithi138.stdout:8/164: creat d3/f29 x:0 0 0 2024-05-27T09:29:30.746 INFO:tasks.workunit.client.0.smithi119.stdout:3/846: dwrite db/d5b/f63 [0,4194304] 0 2024-05-27T09:29:30.750 INFO:tasks.workunit.client.1.smithi138.stdout:6/252: sync 2024-05-27T09:29:30.750 INFO:tasks.workunit.client.1.smithi138.stdout:6/253: chown d3/d11/fe 19108 1 2024-05-27T09:29:30.750 INFO:tasks.workunit.client.1.smithi138.stdout:6/254: write d3/d11/f4f [4746410,90575] 0 2024-05-27T09:29:30.750 INFO:tasks.workunit.client.1.smithi138.stdout:8/165: truncate f1 4163151 0 2024-05-27T09:29:30.752 INFO:tasks.workunit.client.1.smithi138.stdout:6/255: symlink d3/l5c 0 2024-05-27T09:29:30.752 INFO:tasks.workunit.client.1.smithi138.stdout:6/256: creat d3/f5d x:0 0 0 2024-05-27T09:29:30.757 INFO:tasks.workunit.client.1.smithi138.stdout:6/257: link d3/d11/f4f d3/d35/d43/d1c/d32/d47/f5e 0 2024-05-27T09:29:30.763 INFO:tasks.workunit.client.0.smithi119.stdout:3/847: mkdir db/d5b/d112 0 2024-05-27T09:29:30.769 INFO:tasks.workunit.client.0.smithi119.stdout:6/704: dread d0/d2/d57/d37/f5d [0,4194304] 0 2024-05-27T09:29:30.770 INFO:tasks.workunit.client.0.smithi119.stdout:0/669: dwrite d2/d6/d9/d12/d4b/d59/dd7/fc6 [0,4194304] 0 2024-05-27T09:29:30.770 INFO:tasks.workunit.client.0.smithi119.stdout:0/670: chown d2/d6/d9/fbd 0 1 2024-05-27T09:29:30.778 INFO:tasks.workunit.client.1.smithi138.stdout:9/153: write d6/db/f10 [1083905,130084] 0 2024-05-27T09:29:30.778 INFO:tasks.workunit.client.0.smithi119.stdout:3/848: mknod db/d36/d45/d4c/d62/d13/d33/d9b/de3/c113 0 2024-05-27T09:29:30.778 INFO:tasks.workunit.client.0.smithi119.stdout:3/849: chown db/d36/d45/d4c/d62/cf6 9 1 2024-05-27T09:29:30.779 INFO:tasks.workunit.client.1.smithi138.stdout:9/154: chown d6/db/f10 172808 1 2024-05-27T09:29:30.783 INFO:tasks.workunit.client.1.smithi138.stdout:6/258: rmdir d3/d35/d43/d1c 39 2024-05-27T09:29:30.785 INFO:tasks.workunit.client.1.smithi138.stdout:6/259: creat d3/d35/f5f x:0 0 0 2024-05-27T09:29:30.786 INFO:tasks.workunit.client.0.smithi119.stdout:8/668: dwrite d3/d8/d4d/d5d/d66/f94 [0,4194304] 0 2024-05-27T09:29:30.786 INFO:tasks.workunit.client.0.smithi119.stdout:2/714: dread d2/f19 [0,4194304] 0 2024-05-27T09:29:30.790 INFO:tasks.workunit.client.0.smithi119.stdout:6/705: rename d0/d2/d1c/d21/f3c to d0/d2/d1c/d21/ff9 0 2024-05-27T09:29:30.793 INFO:tasks.workunit.client.0.smithi119.stdout:0/671: write d2/d6/d9/d12/d4b/d59/f58 [1326701,114443] 0 2024-05-27T09:29:30.794 INFO:tasks.workunit.client.1.smithi138.stdout:0/208: dread da/f29 [0,4194304] 0 2024-05-27T09:29:30.794 INFO:tasks.workunit.client.1.smithi138.stdout:0/209: fsync da/f11 0 2024-05-27T09:29:30.794 INFO:tasks.workunit.client.1.smithi138.stdout:9/155: write d6/db/dc/f1c [185770,3568] 0 2024-05-27T09:29:30.795 INFO:tasks.workunit.client.1.smithi138.stdout:0/210: symlink da/d17/d1d/d43/l47 0 2024-05-27T09:29:30.801 INFO:tasks.workunit.client.1.smithi138.stdout:0/211: rename da/f11 to da/d17/d1d/d43/f48 0 2024-05-27T09:29:30.803 INFO:tasks.workunit.client.1.smithi138.stdout:0/212: symlink da/d17/l49 0 2024-05-27T09:29:30.804 INFO:tasks.workunit.client.1.smithi138.stdout:2/156: dwrite d1/f1b [4194304,4194304] 0 2024-05-27T09:29:30.805 INFO:tasks.workunit.client.0.smithi119.stdout:7/742: dwrite d7/df/d19/d97/d92/db7/dba/f52 [0,4194304] 0 2024-05-27T09:29:30.805 INFO:tasks.workunit.client.0.smithi119.stdout:7/743: dread - d7/df/d19/d97/d92/db7/dba/d42/d64/fd4 zero size 2024-05-27T09:29:30.805 INFO:tasks.workunit.client.1.smithi138.stdout:0/213: mkdir da/d17/d1d/d1f/d42/d4a 0 2024-05-27T09:29:30.816 INFO:tasks.workunit.client.0.smithi119.stdout:3/850: symlink db/d5b/d112/l114 0 2024-05-27T09:29:30.822 INFO:tasks.workunit.client.0.smithi119.stdout:0/672: dread d2/d6/d23/f33 [0,4194304] 0 2024-05-27T09:29:30.823 INFO:tasks.workunit.client.1.smithi138.stdout:1/212: dwrite d4/da/f1f [0,4194304] 0 2024-05-27T09:29:30.823 INFO:tasks.workunit.client.1.smithi138.stdout:1/213: stat d4/d8/d39/l30 0 2024-05-27T09:29:30.823 INFO:tasks.workunit.client.1.smithi138.stdout:1/214: read - d4/d3c/f42 zero size 2024-05-27T09:29:30.823 INFO:tasks.workunit.client.1.smithi138.stdout:1/215: stat d4/d1a/d2e 0 2024-05-27T09:29:30.823 INFO:tasks.workunit.client.1.smithi138.stdout:4/175: dwrite f3 [0,4194304] 0 2024-05-27T09:29:30.827 INFO:tasks.workunit.client.1.smithi138.stdout:8/166: dwrite d3/d8/d11/f26 [0,4194304] 0 2024-05-27T09:29:30.827 INFO:tasks.workunit.client.0.smithi119.stdout:8/669: unlink d3/d8/d1b/d77/l7b 0 2024-05-27T09:29:30.828 INFO:tasks.workunit.client.1.smithi138.stdout:0/214: dread da/d17/f31 [0,4194304] 0 2024-05-27T09:29:30.831 INFO:tasks.workunit.client.1.smithi138.stdout:1/216: rename d4/d8 to d4/d48 0 2024-05-27T09:29:30.831 INFO:tasks.workunit.client.0.smithi119.stdout:5/778: sync 2024-05-27T09:29:30.833 INFO:tasks.workunit.client.0.smithi119.stdout:7/744: chown d7/df/f29 5 1 2024-05-27T09:29:30.833 INFO:tasks.workunit.client.0.smithi119.stdout:7/745: write d7/df/d19/d97/d92/db7/dba/d42/d64/fc9 [622660,48683] 0 2024-05-27T09:29:30.833 INFO:tasks.workunit.client.0.smithi119.stdout:9/598: sync 2024-05-27T09:29:30.833 INFO:tasks.workunit.client.0.smithi119.stdout:4/744: sync 2024-05-27T09:29:30.833 INFO:tasks.workunit.client.0.smithi119.stdout:9/599: chown d18/d19/d1b/d40/d44/f70 6718 1 2024-05-27T09:29:30.834 INFO:tasks.workunit.client.0.smithi119.stdout:9/600: chown d18/d19/d1b/d40/d7b/fa9 2883 1 2024-05-27T09:29:30.837 INFO:tasks.workunit.client.0.smithi119.stdout:0/673: truncate d2/d6/d9/d24/fc8 769957 0 2024-05-27T09:29:30.839 INFO:tasks.workunit.client.0.smithi119.stdout:1/758: sync 2024-05-27T09:29:30.839 INFO:tasks.workunit.client.0.smithi119.stdout:1/759: dread - d7/db/d8f/d9b/fb9 zero size 2024-05-27T09:29:30.840 INFO:tasks.workunit.client.1.smithi138.stdout:9/156: dread f3 [0,4194304] 0 2024-05-27T09:29:30.843 INFO:tasks.workunit.client.0.smithi119.stdout:8/670: truncate d3/d8/d4d/d5d/d74/f98 3380212 0 2024-05-27T09:29:30.843 INFO:tasks.workunit.client.0.smithi119.stdout:8/671: chown d3/d8/d4d/d5d/lc4 6 1 2024-05-27T09:29:30.843 INFO:tasks.workunit.client.0.smithi119.stdout:5/779: symlink d6/d2f/d9d/d100/l119 0 2024-05-27T09:29:30.843 INFO:tasks.workunit.client.0.smithi119.stdout:5/780: truncate d6/dd/d6e/fa4 881429 0 2024-05-27T09:29:30.843 INFO:tasks.workunit.client.0.smithi119.stdout:3/851: dread db/d36/d45/d49/d9d/fdc [0,4194304] 0 2024-05-27T09:29:30.843 INFO:tasks.workunit.client.0.smithi119.stdout:3/852: creat db/d36/d45/d4c/dbe/f115 x:0 0 0 2024-05-27T09:29:30.845 INFO:tasks.workunit.client.0.smithi119.stdout:7/746: getdents d7/df/d19/d8d 0 2024-05-27T09:29:30.846 INFO:tasks.workunit.client.1.smithi138.stdout:2/157: dread d1/f1b [0,4194304] 0 2024-05-27T09:29:30.847 INFO:tasks.workunit.client.0.smithi119.stdout:0/674: creat d2/d6/d9/d24/d30/dcc/fd9 x:0 0 0 2024-05-27T09:29:30.850 INFO:tasks.workunit.client.0.smithi119.stdout:5/781: rename d6/dd/f37 to d6/db/de/d27/d77/f11a 0 2024-05-27T09:29:30.851 INFO:tasks.workunit.client.1.smithi138.stdout:0/215: creat da/d2a/d2b/f4b x:0 0 0 2024-05-27T09:29:30.857 INFO:tasks.workunit.client.1.smithi138.stdout:1/217: creat d4/d1a/d2e/d32/f49 x:0 0 0 2024-05-27T09:29:30.857 INFO:tasks.workunit.client.1.smithi138.stdout:8/167: dread f1 [0,4194304] 0 2024-05-27T09:29:30.859 INFO:tasks.workunit.client.0.smithi119.stdout:7/747: symlink d7/df/d19/d97/d92/db7/led 0 2024-05-27T09:29:30.859 INFO:tasks.workunit.client.0.smithi119.stdout:7/748: creat d7/d1a/d99/fee x:0 0 0 2024-05-27T09:29:30.860 INFO:tasks.workunit.client.0.smithi119.stdout:0/675: creat d2/d6/d23/db8/fda x:0 0 0 2024-05-27T09:29:30.860 INFO:tasks.workunit.client.0.smithi119.stdout:2/715: dwrite d2/da/d8a/fe4 [0,4194304] 0 2024-05-27T09:29:30.861 INFO:tasks.workunit.client.1.smithi138.stdout:6/260: dwrite d3/d11/f2e [0,4194304] 0 2024-05-27T09:29:30.861 INFO:tasks.workunit.client.1.smithi138.stdout:6/261: creat d3/d35/f60 x:0 0 0 2024-05-27T09:29:30.861 INFO:tasks.workunit.client.1.smithi138.stdout:6/262: chown d3/d35/f60 21 1 2024-05-27T09:29:30.864 INFO:tasks.workunit.client.1.smithi138.stdout:6/263: creat d3/d11/d2d/f61 x:0 0 0 2024-05-27T09:29:30.865 INFO:tasks.workunit.client.1.smithi138.stdout:4/176: mknod d7/d27/c38 0 2024-05-27T09:29:30.865 INFO:tasks.workunit.client.1.smithi138.stdout:4/177: readlink d7/df/l14 0 2024-05-27T09:29:30.866 INFO:tasks.workunit.client.1.smithi138.stdout:2/158: unlink d1/c22 0 2024-05-27T09:29:30.866 INFO:tasks.workunit.client.1.smithi138.stdout:2/159: truncate d1/d10/f1a 1136288 0 2024-05-27T09:29:30.866 INFO:tasks.workunit.client.1.smithi138.stdout:2/160: fsync d1/d2f/f33 0 2024-05-27T09:29:30.868 INFO:tasks.workunit.client.0.smithi119.stdout:3/853: rmdir db/d36/d45/d93 39 2024-05-27T09:29:30.869 INFO:tasks.workunit.client.1.smithi138.stdout:9/157: dread f3 [0,4194304] 0 2024-05-27T09:29:30.872 INFO:tasks.workunit.client.1.smithi138.stdout:4/178: stat c2 0 2024-05-27T09:29:30.873 INFO:tasks.workunit.client.1.smithi138.stdout:9/158: creat d6/f2e x:0 0 0 2024-05-27T09:29:30.873 INFO:tasks.workunit.client.1.smithi138.stdout:9/159: readlink d6/db/l21 0 2024-05-27T09:29:30.874 INFO:tasks.workunit.client.1.smithi138.stdout:4/179: truncate d7/df/f1a 2838277 0 2024-05-27T09:29:30.875 INFO:tasks.workunit.client.1.smithi138.stdout:1/218: dread d4/d48/d22/d29/f3f [0,4194304] 0 2024-05-27T09:29:30.875 INFO:tasks.workunit.client.1.smithi138.stdout:9/160: mkdir d6/da/d2f 0 2024-05-27T09:29:30.878 INFO:tasks.workunit.client.1.smithi138.stdout:4/180: link d7/df/f33 d7/f39 0 2024-05-27T09:29:30.880 INFO:tasks.workunit.client.1.smithi138.stdout:9/161: symlink d6/db/de/l30 0 2024-05-27T09:29:30.880 INFO:tasks.workunit.client.1.smithi138.stdout:1/219: rename d4/d1a/d2e/d32/d3d to d4/d1a/d2e/d4a 0 2024-05-27T09:29:30.880 INFO:tasks.workunit.client.1.smithi138.stdout:1/220: readlink d4/d48/d22/d29/d2b/l3e 0 2024-05-27T09:29:30.882 INFO:tasks.workunit.client.0.smithi119.stdout:2/716: unlink d2/d1a/f39 0 2024-05-27T09:29:30.884 INFO:tasks.workunit.client.1.smithi138.stdout:9/162: rmdir d6/da/d26 39 2024-05-27T09:29:30.884 INFO:tasks.workunit.client.1.smithi138.stdout:9/163: creat d6/db/de/f31 x:0 0 0 2024-05-27T09:29:30.886 INFO:tasks.workunit.client.0.smithi119.stdout:3/854: unlink db/d36/d45/d4c/d62/d13/f3d 0 2024-05-27T09:29:30.887 INFO:tasks.workunit.client.1.smithi138.stdout:1/221: symlink d4/d1a/d2e/l4b 0 2024-05-27T09:29:30.887 INFO:tasks.workunit.client.1.smithi138.stdout:1/222: chown d4/f18 17580454 1 2024-05-27T09:29:30.887 INFO:tasks.workunit.client.1.smithi138.stdout:1/223: dread - d4/d3c/f42 zero size 2024-05-27T09:29:30.887 INFO:tasks.workunit.client.1.smithi138.stdout:1/224: fdatasync d4/f18 0 2024-05-27T09:29:30.887 INFO:tasks.workunit.client.1.smithi138.stdout:1/225: write d4/f18 [5295764,68648] 0 2024-05-27T09:29:30.887 INFO:tasks.workunit.client.1.smithi138.stdout:1/226: dread - d4/d1a/d2e/d32/f49 zero size 2024-05-27T09:29:30.895 INFO:tasks.workunit.client.1.smithi138.stdout:0/216: fsync da/d2a/d2b/f4b 0 2024-05-27T09:29:30.898 INFO:tasks.workunit.client.1.smithi138.stdout:0/217: creat da/d2a/d3f/f4c x:0 0 0 2024-05-27T09:29:30.899 INFO:tasks.workunit.client.0.smithi119.stdout:2/717: unlink d2/da/d8a/dd4/d89/d94/d81/fe6 0 2024-05-27T09:29:30.899 INFO:tasks.workunit.client.0.smithi119.stdout:6/706: dwrite d0/d2/d57/d37/d74/d65/fc7 [0,4194304] 0 2024-05-27T09:29:30.906 INFO:tasks.workunit.client.0.smithi119.stdout:6/707: rmdir d0/d85 39 2024-05-27T09:29:30.906 INFO:tasks.workunit.client.0.smithi119.stdout:6/708: write d0/d2/f80 [1044753,100337] 0 2024-05-27T09:29:30.906 INFO:tasks.workunit.client.0.smithi119.stdout:2/718: dread d2/d1a/f3b [0,4194304] 0 2024-05-27T09:29:30.912 INFO:tasks.workunit.client.1.smithi138.stdout:3/106: sync 2024-05-27T09:29:30.912 INFO:tasks.workunit.client.1.smithi138.stdout:8/168: read d3/d8/f14 [2985921,74476] 0 2024-05-27T09:29:30.912 INFO:tasks.workunit.client.1.smithi138.stdout:1/227: rename d4/f7 to d4/d3c/f4c 0 2024-05-27T09:29:30.913 INFO:tasks.workunit.client.0.smithi119.stdout:9/601: dwrite d18/d19/d1b/d40/d44/f4a [0,4194304] 0 2024-05-27T09:29:30.913 INFO:tasks.workunit.client.0.smithi119.stdout:9/602: read d18/d19/d61/fa5 [1221295,28568] 0 2024-05-27T09:29:30.913 INFO:tasks.workunit.client.0.smithi119.stdout:9/603: fsync d18/d19/d1b/d29/f2f 0 2024-05-27T09:29:30.914 INFO:tasks.workunit.client.1.smithi138.stdout:8/169: symlink d3/d8/l2a 0 2024-05-27T09:29:30.914 INFO:tasks.workunit.client.1.smithi138.stdout:8/170: chown d3/d8/cd 39325 1 2024-05-27T09:29:30.917 INFO:tasks.workunit.client.1.smithi138.stdout:8/171: creat d3/d8/d28/f2b x:0 0 0 2024-05-27T09:29:30.920 INFO:tasks.workunit.client.0.smithi119.stdout:5/782: dread d6/db/de/d27/f50 [0,4194304] 0 2024-05-27T09:29:30.920 INFO:tasks.workunit.client.0.smithi119.stdout:5/783: readlink d6/dd/d39/l4e 0 2024-05-27T09:29:30.921 INFO:tasks.workunit.client.1.smithi138.stdout:8/172: symlink d3/d8/d28/l2c 0 2024-05-27T09:29:30.926 INFO:tasks.workunit.client.1.smithi138.stdout:8/173: rename d3/d8/d11/l1f to d3/d8/l2d 0 2024-05-27T09:29:30.926 INFO:tasks.workunit.client.0.smithi119.stdout:5/784: link d6/dd/d39/d19/d3c/d52/lf3 d6/dd/d39/d74/l11b 0 2024-05-27T09:29:30.932 INFO:tasks.workunit.client.0.smithi119.stdout:5/785: unlink d6/dd/d46/d47/d4b/f59 0 2024-05-27T09:29:30.932 INFO:tasks.workunit.client.0.smithi119.stdout:5/786: write d6/db/dc/f3b [2173935,72657] 0 2024-05-27T09:29:30.933 INFO:tasks.workunit.client.0.smithi119.stdout:7/749: dread d7/df/d19/d9a/da6/dda/d44/f5d [0,4194304] 0 2024-05-27T09:29:30.933 INFO:tasks.workunit.client.0.smithi119.stdout:7/750: stat d7/fd 0 2024-05-27T09:29:30.946 INFO:tasks.workunit.client.0.smithi119.stdout:8/672: dwrite d3/de/f62 [0,4194304] 0 2024-05-27T09:29:30.946 INFO:tasks.workunit.client.0.smithi119.stdout:8/673: chown d3/d8/d1b/d46/d92 3 1 2024-05-27T09:29:30.946 INFO:tasks.workunit.client.0.smithi119.stdout:4/745: dwrite d4/de/d21/d40/f42 [0,4194304] 0 2024-05-27T09:29:30.949 INFO:tasks.workunit.client.1.smithi138.stdout:4/181: write f3 [2630886,86542] 0 2024-05-27T09:29:30.950 INFO:tasks.workunit.client.0.smithi119.stdout:6/709: write d0/d2/d4e/fb9 [943322,96315] 0 2024-05-27T09:29:30.951 INFO:tasks.workunit.client.0.smithi119.stdout:3/855: dwrite db/d7f/da9/ff7 [0,4194304] 0 2024-05-27T09:29:30.954 INFO:tasks.workunit.client.1.smithi138.stdout:4/182: readlink d7/d27/l36 0 2024-05-27T09:29:30.960 INFO:tasks.workunit.client.1.smithi138.stdout:1/228: read d4/d38/f44 [2776029,35528] 0 2024-05-27T09:29:30.961 INFO:tasks.workunit.client.0.smithi119.stdout:2/719: dread d2/da/d6f/fc1 [0,4194304] 0 2024-05-27T09:29:30.961 INFO:tasks.workunit.client.0.smithi119.stdout:2/720: truncate d2/fae 935550 0 2024-05-27T09:29:30.961 INFO:tasks.workunit.client.0.smithi119.stdout:7/751: getdents d7/df/d19/d9a/da6/dda/d44 0 2024-05-27T09:29:30.961 INFO:tasks.workunit.client.0.smithi119.stdout:7/752: chown d7/df/d19/d97/d92/db7/dba/d42/d76/f82 29241487 1 2024-05-27T09:29:30.961 INFO:tasks.workunit.client.0.smithi119.stdout:7/753: chown d7/d6a/la0 42 1 2024-05-27T09:29:30.961 INFO:tasks.workunit.client.0.smithi119.stdout:7/754: creat d7/df/d19/d97/d92/db7/dba/d42/d64/de3/fef x:0 0 0 2024-05-27T09:29:30.962 INFO:tasks.workunit.client.1.smithi138.stdout:9/164: dwrite d6/db/de/f31 [0,4194304] 0 2024-05-27T09:29:30.970 INFO:tasks.workunit.client.1.smithi138.stdout:1/229: unlink d4/d48/d22/d29/d2b/l3e 0 2024-05-27T09:29:30.980 INFO:tasks.workunit.client.1.smithi138.stdout:9/165: rename d6/db/f1b to d6/da/d2f/f32 0 2024-05-27T09:29:30.983 INFO:tasks.workunit.client.1.smithi138.stdout:1/230: rmdir d4/d38 39 2024-05-27T09:29:30.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:30 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:30.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:30 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:30.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:30 smithi119 ceph-mon[29082]: pgmap v13: 65 pgs: 65 active+clean; 1.5 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 102 MiB/s rd, 123 MiB/s wr, 380 op/s 2024-05-27T09:29:31.004 INFO:tasks.workunit.client.1.smithi138.stdout:6/264: dwrite d3/d11/f2e [0,4194304] 0 2024-05-27T09:29:31.006 INFO:tasks.workunit.client.0.smithi119.stdout:8/674: symlink d3/d8/d4d/d5d/d66/d85/lde 0 2024-05-27T09:29:31.007 INFO:tasks.workunit.client.1.smithi138.stdout:6/265: unlink d3/d35/f5f 0 2024-05-27T09:29:31.011 INFO:tasks.workunit.client.1.smithi138.stdout:6/266: symlink d3/d11/l62 0 2024-05-27T09:29:31.012 INFO:tasks.workunit.client.0.smithi119.stdout:6/710: creat d0/d85/de7/ffa x:0 0 0 2024-05-27T09:29:31.014 INFO:tasks.workunit.client.0.smithi119.stdout:9/604: dwrite d18/f9a [0,4194304] 0 2024-05-27T09:29:31.015 INFO:tasks.workunit.client.1.smithi138.stdout:9/166: read d6/f1f [2874904,63139] 0 2024-05-27T09:29:31.017 INFO:tasks.workunit.client.1.smithi138.stdout:9/167: mknod d6/db/dc/c33 0 2024-05-27T09:29:31.023 INFO:tasks.workunit.client.1.smithi138.stdout:8/174: dwrite d3/d8/f14 [0,4194304] 0 2024-05-27T09:29:31.023 INFO:tasks.workunit.client.1.smithi138.stdout:8/175: creat d3/d8/d28/f2e x:0 0 0 2024-05-27T09:29:31.024 INFO:tasks.workunit.client.1.smithi138.stdout:2/161: dwrite d1/d15/f27 [0,4194304] 0 2024-05-27T09:29:31.028 INFO:tasks.workunit.client.1.smithi138.stdout:2/162: mknod d1/d15/d2d/d17/d25/c35 0 2024-05-27T09:29:31.040 INFO:tasks.workunit.client.1.smithi138.stdout:2/163: dread d1/d10/f1a [0,4194304] 0 2024-05-27T09:29:31.044 INFO:tasks.workunit.client.1.smithi138.stdout:3/107: dwrite d0/d2/f13 [0,4194304] 0 2024-05-27T09:29:31.045 INFO:tasks.workunit.client.0.smithi119.stdout:9/605: read d18/f41 [3929586,57025] 0 2024-05-27T09:29:31.045 INFO:tasks.workunit.client.0.smithi119.stdout:5/787: dwrite d6/f22 [4194304,4194304] 0 2024-05-27T09:29:31.049 INFO:tasks.workunit.client.0.smithi119.stdout:9/606: creat d18/d19/d61/d98/fba x:0 0 0 2024-05-27T09:29:31.050 INFO:tasks.workunit.client.0.smithi119.stdout:5/788: rmdir d6/dd/d46/d47/d4b/dd8 39 2024-05-27T09:29:31.055 INFO:tasks.workunit.client.0.smithi119.stdout:3/856: dwrite db/d36/d45/dab/fea [0,4194304] 0 2024-05-27T09:29:31.055 INFO:tasks.workunit.client.0.smithi119.stdout:3/857: write db/d36/d45/d4d/dc8/f107 [292219,15269] 0 2024-05-27T09:29:31.057 INFO:tasks.workunit.client.0.smithi119.stdout:8/675: dread d3/f8c [0,4194304] 0 2024-05-27T09:29:31.060 INFO:tasks.workunit.client.0.smithi119.stdout:9/607: creat d18/d19/d1b/d40/db7/fbb x:0 0 0 2024-05-27T09:29:31.065 INFO:tasks.workunit.client.1.smithi138.stdout:9/168: dread d6/db/de/f2b [0,4194304] 0 2024-05-27T09:29:31.067 INFO:tasks.workunit.client.0.smithi119.stdout:4/746: dwrite d4/de/d21/f2b [0,4194304] 0 2024-05-27T09:29:31.070 INFO:tasks.workunit.client.0.smithi119.stdout:8/676: rename d3/d39/c3a to d3/d8/cdf 0 2024-05-27T09:29:31.070 INFO:tasks.workunit.client.0.smithi119.stdout:9/608: mknod d18/d19/d1b/d2a/cbc 0 2024-05-27T09:29:31.072 INFO:tasks.workunit.client.0.smithi119.stdout:2/721: dwrite d2/da/d15/fbc [0,4194304] 0 2024-05-27T09:29:31.072 INFO:tasks.workunit.client.0.smithi119.stdout:2/722: dread - d2/da/d8a/dd4/d89/d94/d87/fba zero size 2024-05-27T09:29:31.083 INFO:tasks.workunit.client.1.smithi138.stdout:1/231: rename d4/d48/d22/d29 to d4/da/d4d 0 2024-05-27T09:29:31.083 INFO:tasks.workunit.client.1.smithi138.stdout:1/232: fdatasync d4/f12 0 2024-05-27T09:29:31.085 INFO:tasks.workunit.client.1.smithi138.stdout:6/267: rename d3/f5d to d3/d19/f63 0 2024-05-27T09:29:31.093 INFO:tasks.workunit.client.1.smithi138.stdout:1/233: link d4/d48/d22/f2c d4/d3c/f4e 0 2024-05-27T09:29:31.093 INFO:tasks.workunit.client.1.smithi138.stdout:1/234: getdents d4/d1a 0 2024-05-27T09:29:31.093 INFO:tasks.workunit.client.1.smithi138.stdout:1/235: fdatasync d4/d1a/d2e/d32/f49 0 2024-05-27T09:29:31.093 INFO:tasks.workunit.client.1.smithi138.stdout:1/236: fdatasync d4/d3c/f4c 0 2024-05-27T09:29:31.093 INFO:tasks.workunit.client.1.smithi138.stdout:1/237: chown d4/d38 63 1 2024-05-27T09:29:31.094 INFO:tasks.workunit.client.1.smithi138.stdout:6/268: symlink d3/d19/l64 0 2024-05-27T09:29:31.094 INFO:tasks.workunit.client.1.smithi138.stdout:3/108: write d0/d8/f1d [1285467,50826] 0 2024-05-27T09:29:31.096 INFO:tasks.workunit.client.0.smithi119.stdout:7/755: dwrite d7/fa [0,4194304] 0 2024-05-27T09:29:31.099 INFO:tasks.workunit.client.1.smithi138.stdout:6/269: mknod d3/d35/d5a/c65 0 2024-05-27T09:29:31.103 INFO:tasks.workunit.client.1.smithi138.stdout:6/270: stat d3/d11/cc 0 2024-05-27T09:29:31.104 INFO:tasks.workunit.client.0.smithi119.stdout:6/711: dwrite d0/d2/d1c/f2a [0,4194304] 0 2024-05-27T09:29:31.106 INFO:tasks.workunit.client.1.smithi138.stdout:6/271: creat d3/d11/d33/f66 x:0 0 0 2024-05-27T09:29:31.107 INFO:tasks.workunit.client.1.smithi138.stdout:1/238: dread d4/d3c/f4e [0,4194304] 0 2024-05-27T09:29:31.108 INFO:tasks.workunit.client.1.smithi138.stdout:6/272: mknod d3/d19/c67 0 2024-05-27T09:29:31.111 INFO:tasks.workunit.client.1.smithi138.stdout:6/273: truncate d3/d19/f34 2756897 0 2024-05-27T09:29:31.111 INFO:tasks.workunit.client.1.smithi138.stdout:6/274: chown d3/c12 21220506 1 2024-05-27T09:29:31.115 INFO:tasks.workunit.client.1.smithi138.stdout:0/218: dwrite f1 [4194304,4194304] 0 2024-05-27T09:29:31.116 INFO:tasks.workunit.client.1.smithi138.stdout:0/219: creat da/d2a/d2b/f4d x:0 0 0 2024-05-27T09:29:31.121 INFO:tasks.workunit.client.1.smithi138.stdout:3/109: dread d0/d2/da/f1b [0,4194304] 0 2024-05-27T09:29:31.121 INFO:tasks.workunit.client.1.smithi138.stdout:3/110: creat d0/d2/f24 x:0 0 0 2024-05-27T09:29:31.122 INFO:tasks.workunit.client.0.smithi119.stdout:2/723: dread d2/da/d8a/dd4/d89/d94/f73 [0,4194304] 0 2024-05-27T09:29:31.122 INFO:tasks.workunit.client.0.smithi119.stdout:2/724: truncate d2/da/d8a/d8f/fed 656524 0 2024-05-27T09:29:31.122 INFO:tasks.workunit.client.0.smithi119.stdout:7/756: mkdir d7/df/d19/d97/df0 0 2024-05-27T09:29:31.126 INFO:tasks.workunit.client.1.smithi138.stdout:0/220: creat da/d17/d1d/d1f/d42/d4a/f4e x:0 0 0 2024-05-27T09:29:31.129 INFO:tasks.workunit.client.1.smithi138.stdout:0/221: mkdir da/d17/d1d/d1f/d4f 0 2024-05-27T09:29:31.130 INFO:tasks.workunit.client.1.smithi138.stdout:0/222: creat da/d2a/d2b/f50 x:0 0 0 2024-05-27T09:29:31.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:30 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:31.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:30 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:31.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:30 smithi138 ceph-mon[34885]: pgmap v13: 65 pgs: 65 active+clean; 1.5 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 102 MiB/s rd, 123 MiB/s wr, 380 op/s 2024-05-27T09:29:31.137 INFO:tasks.workunit.client.0.smithi119.stdout:8/677: dread d3/d8/d1b/db1/fbc [0,4194304] 0 2024-05-27T09:29:31.150 INFO:tasks.workunit.client.0.smithi119.stdout:5/789: dwrite d6/db/d9a/dc0/fcb [0,4194304] 0 2024-05-27T09:29:31.151 INFO:tasks.workunit.client.0.smithi119.stdout:8/678: mkdir d3/d8/d4d/d5d/db7/de0 0 2024-05-27T09:29:31.152 INFO:tasks.workunit.client.0.smithi119.stdout:7/757: link d7/df/f10 d7/d1a/d99/dbb/ff1 0 2024-05-27T09:29:31.154 INFO:tasks.workunit.client.0.smithi119.stdout:7/758: mknod d7/df/d19/d97/d81/cf2 0 2024-05-27T09:29:31.156 INFO:tasks.workunit.client.0.smithi119.stdout:7/759: unlink d7/df/d19/l7d 0 2024-05-27T09:29:31.156 INFO:tasks.workunit.client.0.smithi119.stdout:3/858: dwrite db/d36/d45/d4c/d62/d13/f31 [0,4194304] 0 2024-05-27T09:29:31.159 INFO:tasks.workunit.client.1.smithi138.stdout:3/111: write d0/d2/f22 [4064773,100422] 0 2024-05-27T09:29:31.159 INFO:tasks.workunit.client.1.smithi138.stdout:3/112: chown d0/d2/da/df 52767568 1 2024-05-27T09:29:31.167 INFO:tasks.workunit.client.0.smithi119.stdout:4/747: dwrite d4/f7e [0,4194304] 0 2024-05-27T09:29:31.168 INFO:tasks.workunit.client.0.smithi119.stdout:4/748: creat d4/de/d21/fec x:0 0 0 2024-05-27T09:29:31.169 INFO:tasks.workunit.client.1.smithi138.stdout:7/156: sync 2024-05-27T09:29:31.169 INFO:tasks.workunit.client.1.smithi138.stdout:5/188: sync 2024-05-27T09:29:31.171 INFO:tasks.workunit.client.1.smithi138.stdout:8/176: dwrite d3/d8/d28/f2b [0,4194304] 0 2024-05-27T09:29:31.171 INFO:tasks.workunit.client.1.smithi138.stdout:0/223: dread da/d2a/f30 [0,4194304] 0 2024-05-27T09:29:31.172 INFO:tasks.workunit.client.1.smithi138.stdout:8/177: chown d3/d8/d28/l2c 424 1 2024-05-27T09:29:31.176 INFO:tasks.workunit.client.0.smithi119.stdout:4/749: truncate d4/d6/f3b 428305 0 2024-05-27T09:29:31.176 INFO:tasks.workunit.client.1.smithi138.stdout:1/239: dwrite d4/d48/d22/f35 [4194304,4194304] 0 2024-05-27T09:29:31.176 INFO:tasks.workunit.client.1.smithi138.stdout:5/189: rename d3/c5 to d3/d12/d16/d1d/d30/c3e 0 2024-05-27T09:29:31.176 INFO:tasks.workunit.client.1.smithi138.stdout:1/240: chown d4/l15 23236019 1 2024-05-27T09:29:31.176 INFO:tasks.workunit.client.1.smithi138.stdout:1/241: readlink d4/d48/l31 0 2024-05-27T09:29:31.180 INFO:tasks.workunit.client.1.smithi138.stdout:7/157: rmdir d0/d2b 39 2024-05-27T09:29:31.180 INFO:tasks.workunit.client.1.smithi138.stdout:7/158: readlink d0/d7/l14 0 2024-05-27T09:29:31.180 INFO:tasks.workunit.client.1.smithi138.stdout:0/224: creat da/d2a/f51 x:0 0 0 2024-05-27T09:29:31.181 INFO:tasks.workunit.client.1.smithi138.stdout:0/225: rename da/d17 to da/d17/d1d/d1f/d42/d4a/d52 22 2024-05-27T09:29:31.181 INFO:tasks.workunit.client.1.smithi138.stdout:0/226: write da/f29 [3869925,106732] 0 2024-05-27T09:29:31.181 INFO:tasks.workunit.client.0.smithi119.stdout:9/609: dwrite f16 [0,4194304] 0 2024-05-27T09:29:31.181 INFO:tasks.workunit.client.0.smithi119.stdout:4/750: creat d4/de/d21/d40/d47/fed x:0 0 0 2024-05-27T09:29:31.181 INFO:tasks.workunit.client.0.smithi119.stdout:4/751: readlink d4/d6/d28/d2f/la9 0 2024-05-27T09:29:31.182 INFO:tasks.workunit.client.0.smithi119.stdout:4/752: readlink d4/d6/d81/dce/lde 0 2024-05-27T09:29:31.182 INFO:tasks.workunit.client.0.smithi119.stdout:6/712: dwrite d0/d2/d4e/d5a/faa [0,4194304] 0 2024-05-27T09:29:31.182 INFO:tasks.workunit.client.0.smithi119.stdout:9/610: creat d18/d19/d9b/fbd x:0 0 0 2024-05-27T09:29:31.185 INFO:tasks.workunit.client.0.smithi119.stdout:9/611: mkdir d18/db4/dbe 0 2024-05-27T09:29:31.188 INFO:tasks.workunit.client.1.smithi138.stdout:8/178: rename d3/d8/d28/f2b to d3/d8/d28/f2f 0 2024-05-27T09:29:31.188 INFO:tasks.workunit.client.1.smithi138.stdout:1/242: truncate d4/da/f2f 2740585 0 2024-05-27T09:29:31.188 INFO:tasks.workunit.client.1.smithi138.stdout:8/179: rename d3 to d3/d8/d28/d30 22 2024-05-27T09:29:31.188 INFO:tasks.workunit.client.1.smithi138.stdout:7/159: link d0/d7/l2f d0/d7/l30 0 2024-05-27T09:29:31.189 INFO:tasks.workunit.client.1.smithi138.stdout:0/227: mkdir da/d17/d1d/d1f/d42/d4a/d53 0 2024-05-27T09:29:31.189 INFO:tasks.workunit.client.1.smithi138.stdout:0/228: read - da/d17/d1d/d1f/d42/d4a/f4e zero size 2024-05-27T09:29:31.189 INFO:tasks.workunit.client.1.smithi138.stdout:0/229: chown da/f10 347385220 1 2024-05-27T09:29:31.189 INFO:tasks.workunit.client.1.smithi138.stdout:9/169: dwrite d6/f9 [0,4194304] 0 2024-05-27T09:29:31.192 INFO:tasks.workunit.client.0.smithi119.stdout:5/790: dread d6/db/de/d27/d77/f11a [0,4194304] 0 2024-05-27T09:29:31.195 INFO:tasks.workunit.client.1.smithi138.stdout:1/243: truncate d4/f18 4960568 0 2024-05-27T09:29:31.196 INFO:tasks.workunit.client.1.smithi138.stdout:1/244: fdatasync d4/da/f2f 0 2024-05-27T09:29:31.200 INFO:tasks.workunit.client.1.smithi138.stdout:9/170: getdents d6/db 0 2024-05-27T09:29:31.200 INFO:tasks.workunit.client.1.smithi138.stdout:0/230: rename da/d17/l32 to da/d17/d1d/d1f/d42/d4a/l54 0 2024-05-27T09:29:31.200 INFO:tasks.workunit.client.1.smithi138.stdout:0/231: chown da/f29 3343 1 2024-05-27T09:29:31.200 INFO:tasks.workunit.client.1.smithi138.stdout:9/171: write d6/db/f14 [388639,68307] 0 2024-05-27T09:29:31.201 INFO:tasks.workunit.client.1.smithi138.stdout:3/113: dread d0/d2/fc [0,4194304] 0 2024-05-27T09:29:31.202 INFO:tasks.workunit.client.1.smithi138.stdout:6/275: dwrite d3/d11/d2d/d37/f3c [0,4194304] 0 2024-05-27T09:29:31.202 INFO:tasks.workunit.client.1.smithi138.stdout:6/276: stat d3/f5 0 2024-05-27T09:29:31.206 INFO:tasks.workunit.client.1.smithi138.stdout:3/114: mknod d0/d2/da/df/c25 0 2024-05-27T09:29:31.206 INFO:tasks.workunit.client.1.smithi138.stdout:6/277: rename d3/d11/d2d/f45 to d3/d35/d5a/f68 0 2024-05-27T09:29:31.207 INFO:tasks.workunit.client.1.smithi138.stdout:3/115: symlink d0/l26 0 2024-05-27T09:29:31.207 INFO:tasks.workunit.client.1.smithi138.stdout:3/116: stat d0/d8/c14 0 2024-05-27T09:29:31.208 INFO:tasks.workunit.client.1.smithi138.stdout:6/278: mknod d3/d35/d43/d1c/c69 0 2024-05-27T09:29:31.209 INFO:tasks.workunit.client.0.smithi119.stdout:7/760: write d7/df/d19/d97/d92/db7/dba/f49 [2046086,31] 0 2024-05-27T09:29:31.209 INFO:tasks.workunit.client.0.smithi119.stdout:7/761: write d7/df/d19/d97/d92/db7/dba/d42/d64/f7a [936461,107481] 0 2024-05-27T09:29:31.209 INFO:tasks.workunit.client.0.smithi119.stdout:2/725: dwrite d2/da/f2e [0,4194304] 0 2024-05-27T09:29:31.212 INFO:tasks.workunit.client.0.smithi119.stdout:5/791: dread d6/d2f/f78 [0,4194304] 0 2024-05-27T09:29:31.212 INFO:tasks.workunit.client.0.smithi119.stdout:5/792: write d6/dd/d39/d19/d3c/dd5/f104 [401056,32036] 0 2024-05-27T09:29:31.215 INFO:tasks.workunit.client.0.smithi119.stdout:4/753: dread d4/d23/d24/f82 [0,4194304] 0 2024-05-27T09:29:31.221 INFO:tasks.workunit.client.1.smithi138.stdout:5/190: unlink d3/d12/d16/d1d/d30/c3e 0 2024-05-27T09:29:31.222 INFO:tasks.workunit.client.0.smithi119.stdout:2/726: creat d2/da/d8a/dd4/d89/d94/d87/ff3 x:0 0 0 2024-05-27T09:29:31.225 INFO:tasks.workunit.client.0.smithi119.stdout:4/754: read d4/d6/f86 [387449,114066] 0 2024-05-27T09:29:31.225 INFO:tasks.workunit.client.0.smithi119.stdout:4/755: readlink d4/d6/d81/dce/lde 0 2024-05-27T09:29:31.243 INFO:tasks.workunit.client.0.smithi119.stdout:4/756: creat d4/d1c/d39/fee x:0 0 0 2024-05-27T09:29:31.243 INFO:tasks.workunit.client.0.smithi119.stdout:4/757: chown d4/l44 4945 1 2024-05-27T09:29:31.249 INFO:tasks.workunit.client.0.smithi119.stdout:4/758: rmdir d4/de/de5 0 2024-05-27T09:29:31.254 INFO:tasks.workunit.client.0.smithi119.stdout:9/612: dread d18/f32 [0,4194304] 0 2024-05-27T09:29:31.254 INFO:tasks.workunit.client.0.smithi119.stdout:3/859: dwrite db/d36/fc3 [0,4194304] 0 2024-05-27T09:29:31.260 INFO:tasks.workunit.client.0.smithi119.stdout:3/860: symlink db/d36/d45/d4c/d62/d13/l116 0 2024-05-27T09:29:31.266 INFO:tasks.workunit.client.0.smithi119.stdout:8/679: dwrite d3/d8/d4d/d5d/d60/f71 [0,4194304] 0 2024-05-27T09:29:31.268 INFO:tasks.workunit.client.1.smithi138.stdout:7/160: write d0/f19 [4054885,58821] 0 2024-05-27T09:29:31.275 INFO:tasks.workunit.client.0.smithi119.stdout:8/680: symlink d3/d8/d1b/db1/le1 0 2024-05-27T09:29:31.277 INFO:tasks.workunit.client.0.smithi119.stdout:3/861: symlink db/d36/d45/d4c/d62/l117 0 2024-05-27T09:29:31.277 INFO:tasks.workunit.client.0.smithi119.stdout:3/862: truncate db/d36/d45/d4c/dbe/f115 144893 0 2024-05-27T09:29:31.279 INFO:tasks.workunit.client.1.smithi138.stdout:8/180: dread d3/d8/d11/f26 [0,4194304] 0 2024-05-27T09:29:31.281 INFO:tasks.workunit.client.0.smithi119.stdout:7/762: dwrite d7/df/d19/d97/d92/fd1 [0,4194304] 0 2024-05-27T09:29:31.283 INFO:tasks.workunit.client.1.smithi138.stdout:3/117: rmdir d0/d2 39 2024-05-27T09:29:31.283 INFO:tasks.workunit.client.1.smithi138.stdout:7/161: mknod d0/d2b/c31 0 2024-05-27T09:29:31.283 INFO:tasks.workunit.client.1.smithi138.stdout:7/162: dread - d0/f1a zero size 2024-05-27T09:29:31.283 INFO:tasks.workunit.client.1.smithi138.stdout:7/163: fdatasync d0/d7/fe 0 2024-05-27T09:29:31.287 INFO:tasks.workunit.client.0.smithi119.stdout:8/681: unlink d3/d8/d4d/d5d/d60/f6b 0 2024-05-27T09:29:31.288 INFO:tasks.workunit.client.1.smithi138.stdout:8/181: rename c0 to d3/d8/d24/c31 0 2024-05-27T09:29:31.290 INFO:tasks.workunit.client.1.smithi138.stdout:3/118: creat d0/d2/f27 x:0 0 0 2024-05-27T09:29:31.291 INFO:tasks.workunit.client.1.smithi138.stdout:3/119: creat d0/d8/f28 x:0 0 0 2024-05-27T09:29:31.294 INFO:tasks.workunit.client.1.smithi138.stdout:2/164: sync 2024-05-27T09:29:31.295 INFO:tasks.workunit.client.1.smithi138.stdout:4/183: sync 2024-05-27T09:29:31.296 INFO:tasks.workunit.client.1.smithi138.stdout:8/182: write d3/d8/d11/f22 [122521,73483] 0 2024-05-27T09:29:31.304 INFO:tasks.workunit.client.0.smithi119.stdout:7/763: symlink d7/df/d19/d9a/da6/dda/lf3 0 2024-05-27T09:29:31.304 INFO:tasks.workunit.client.0.smithi119.stdout:8/682: mknod d3/d8/d4d/d79/ce2 0 2024-05-27T09:29:31.304 INFO:tasks.workunit.client.0.smithi119.stdout:4/759: fsync d4/d6/f3b 0 2024-05-27T09:29:31.304 INFO:tasks.workunit.client.0.smithi119.stdout:8/683: chown d3/d8/d4d/d5d/d66/d85/lde 898603521 1 2024-05-27T09:29:31.305 INFO:tasks.workunit.client.1.smithi138.stdout:2/165: write d1/d15/d2d/d17/d25/f28 [2764354,97821] 0 2024-05-27T09:29:31.305 INFO:tasks.workunit.client.1.smithi138.stdout:2/166: write d1/f1b [8435959,102848] 0 2024-05-27T09:29:31.305 INFO:tasks.workunit.client.1.smithi138.stdout:2/167: write d1/d10/fe [977402,113571] 0 2024-05-27T09:29:31.305 INFO:tasks.workunit.client.1.smithi138.stdout:2/168: getdents d1/d15/d2d/d2e 0 2024-05-27T09:29:31.312 INFO:tasks.workunit.client.0.smithi119.stdout:9/613: dwrite d18/d19/d1b/d40/d44/f45 [0,4194304] 0 2024-05-27T09:29:31.313 INFO:tasks.workunit.client.1.smithi138.stdout:1/245: dwrite d4/d48/d22/f35 [4194304,4194304] 0 2024-05-27T09:29:31.313 INFO:tasks.workunit.client.1.smithi138.stdout:1/246: chown d4/da/d4d/d2b 919386 1 2024-05-27T09:29:31.313 INFO:tasks.workunit.client.1.smithi138.stdout:1/247: write d4/d1a/f27 [4191974,109036] 0 2024-05-27T09:29:31.314 INFO:tasks.workunit.client.1.smithi138.stdout:1/248: readlink d4/d48/d39/l30 0 2024-05-27T09:29:31.314 INFO:tasks.workunit.client.1.smithi138.stdout:7/164: link d0/f1a d0/f32 0 2024-05-27T09:29:31.314 INFO:tasks.workunit.client.1.smithi138.stdout:9/172: dwrite d6/db/de/f2b [4194304,4194304] 0 2024-05-27T09:29:31.315 INFO:tasks.workunit.client.0.smithi119.stdout:7/764: rename d7/df/d19/d97/d92/db7/dba/f49 to d7/df/d19/d46/d93/ff4 0 2024-05-27T09:29:31.317 INFO:tasks.workunit.client.1.smithi138.stdout:0/232: dwrite da/d17/d1d/d1f/d42/d4a/f4e [0,4194304] 0 2024-05-27T09:29:31.318 INFO:tasks.workunit.client.0.smithi119.stdout:9/614: creat d18/d19/d1b/d2a/d63/fbf x:0 0 0 2024-05-27T09:29:31.319 INFO:tasks.workunit.client.1.smithi138.stdout:1/249: link d4/da/l1b d4/d38/l4f 0 2024-05-27T09:29:31.321 INFO:tasks.workunit.client.1.smithi138.stdout:7/165: link d0/d7/c17 d0/d23/c33 0 2024-05-27T09:29:31.321 INFO:tasks.workunit.client.0.smithi119.stdout:2/727: dwrite d2/da/d15/d53/ff1 [0,4194304] 0 2024-05-27T09:29:31.324 INFO:tasks.workunit.client.1.smithi138.stdout:0/233: getdents da/d17 0 2024-05-27T09:29:31.326 INFO:tasks.workunit.client.1.smithi138.stdout:1/250: creat d4/da/f50 x:0 0 0 2024-05-27T09:29:31.326 INFO:tasks.workunit.client.1.smithi138.stdout:1/251: chown d4/d38/l4f 1866 1 2024-05-27T09:29:31.329 INFO:tasks.workunit.client.1.smithi138.stdout:7/166: link d0/d7/lc d0/d23/l34 0 2024-05-27T09:29:31.329 INFO:tasks.workunit.client.1.smithi138.stdout:0/234: mkdir da/d2a/d55 0 2024-05-27T09:29:31.329 INFO:tasks.workunit.client.0.smithi119.stdout:8/684: dread d3/d8/d4d/f32 [0,4194304] 0 2024-05-27T09:29:31.332 INFO:tasks.workunit.client.1.smithi138.stdout:7/167: creat d0/d2b/f35 x:0 0 0 2024-05-27T09:29:31.332 INFO:tasks.workunit.client.0.smithi119.stdout:2/728: symlink d2/da/d8a/d8f/lf4 0 2024-05-27T09:29:31.332 INFO:tasks.workunit.client.0.smithi119.stdout:2/729: chown d2/da/d15/ca4 443501191 1 2024-05-27T09:29:31.336 INFO:tasks.workunit.client.1.smithi138.stdout:0/235: mknod da/d17/d1d/d1f/c56 0 2024-05-27T09:29:31.336 INFO:tasks.workunit.client.1.smithi138.stdout:0/236: stat da/d2a/d2b/l36 0 2024-05-27T09:29:31.338 INFO:tasks.workunit.client.0.smithi119.stdout:8/685: getdents d3/d8/d1b/d46 0 2024-05-27T09:29:31.338 INFO:tasks.workunit.client.0.smithi119.stdout:8/686: creat d3/d8/d1b/d77/fe3 x:0 0 0 2024-05-27T09:29:31.342 INFO:tasks.workunit.client.1.smithi138.stdout:9/173: dread d6/db/dc/f1c [0,4194304] 0 2024-05-27T09:29:31.342 INFO:tasks.workunit.client.1.smithi138.stdout:9/174: chown d6/da 10698194 1 2024-05-27T09:29:31.342 INFO:tasks.workunit.client.1.smithi138.stdout:9/175: write d6/db/f10 [1912070,4339] 0 2024-05-27T09:29:31.343 INFO:tasks.workunit.client.1.smithi138.stdout:7/168: creat d0/f36 x:0 0 0 2024-05-27T09:29:31.343 INFO:tasks.workunit.client.0.smithi119.stdout:2/730: mkdir d2/da/d8a/dd4/d69/df5 0 2024-05-27T09:29:31.343 INFO:tasks.workunit.client.0.smithi119.stdout:2/731: write d2/da/d8a/dd4/d89/f8d [643621,98842] 0 2024-05-27T09:29:31.347 INFO:tasks.workunit.client.0.smithi119.stdout:1/760: sync 2024-05-27T09:29:31.347 INFO:tasks.workunit.client.0.smithi119.stdout:0/676: sync 2024-05-27T09:29:31.347 INFO:tasks.workunit.client.0.smithi119.stdout:0/677: chown d2/d6/d9/dab/lcd 2387 1 2024-05-27T09:29:31.347 INFO:tasks.workunit.client.0.smithi119.stdout:1/761: chown d7/ccf 5619 1 2024-05-27T09:29:31.349 INFO:tasks.workunit.client.1.smithi138.stdout:0/237: creat da/d17/d1d/d1f/d42/d4a/d53/f57 x:0 0 0 2024-05-27T09:29:31.349 INFO:tasks.workunit.client.1.smithi138.stdout:0/238: write f5 [1032593,124356] 0 2024-05-27T09:29:31.349 INFO:tasks.workunit.client.1.smithi138.stdout:1/252: dread d4/d48/d22/f2c [0,4194304] 0 2024-05-27T09:29:31.349 INFO:tasks.workunit.client.1.smithi138.stdout:7/169: creat d0/f37 x:0 0 0 2024-05-27T09:29:31.349 INFO:tasks.workunit.client.1.smithi138.stdout:7/170: chown d0/f36 834705 1 2024-05-27T09:29:31.349 INFO:tasks.workunit.client.1.smithi138.stdout:9/176: read d6/db/dc/f12 [805802,17871] 0 2024-05-27T09:29:31.349 INFO:tasks.workunit.client.1.smithi138.stdout:7/171: chown d0/d23/l2a 25668 1 2024-05-27T09:29:31.351 INFO:tasks.workunit.client.1.smithi138.stdout:6/279: dwrite d3/d19/f41 [0,4194304] 0 2024-05-27T09:29:31.351 INFO:tasks.workunit.client.0.smithi119.stdout:8/687: mknod d3/d39/ce4 0 2024-05-27T09:29:31.352 INFO:tasks.workunit.client.1.smithi138.stdout:9/177: link d6/db/de/l30 d6/d25/l34 0 2024-05-27T09:29:31.357 INFO:tasks.workunit.client.0.smithi119.stdout:2/732: getdents d2/da/d8a/dd4 0 2024-05-27T09:29:31.358 INFO:tasks.workunit.client.0.smithi119.stdout:1/762: dread d7/dd/d25/d60/d61/fb3 [0,4194304] 0 2024-05-27T09:29:31.360 INFO:tasks.workunit.client.1.smithi138.stdout:9/178: mkdir d6/da/d2f/d35 0 2024-05-27T09:29:31.360 INFO:tasks.workunit.client.1.smithi138.stdout:9/179: read - d6/f2e zero size 2024-05-27T09:29:31.360 INFO:tasks.workunit.client.1.smithi138.stdout:7/172: symlink d0/l38 0 2024-05-27T09:29:31.364 INFO:tasks.workunit.client.0.smithi119.stdout:3/863: dwrite db/d36/d45/d93/fb0 [0,4194304] 0 2024-05-27T09:29:31.365 INFO:tasks.workunit.client.1.smithi138.stdout:9/180: rename d6/db/de/l30 to d6/da/l36 0 2024-05-27T09:29:31.365 INFO:tasks.workunit.client.1.smithi138.stdout:7/173: rename d0/d7/f16 to d0/d7/f39 0 2024-05-27T09:29:31.365 INFO:tasks.workunit.client.1.smithi138.stdout:7/174: chown d0/d7/l2f 49 1 2024-05-27T09:29:31.366 INFO:tasks.workunit.client.1.smithi138.stdout:7/175: truncate d0/d2b/f35 107093 0 2024-05-27T09:29:31.366 INFO:tasks.workunit.client.0.smithi119.stdout:3/864: write db/d36/d45/dab/fea [5130956,120916] 0 2024-05-27T09:29:31.372 INFO:tasks.workunit.client.1.smithi138.stdout:5/191: dwrite d3/f18 [4194304,4194304] 0 2024-05-27T09:29:31.372 INFO:tasks.workunit.client.1.smithi138.stdout:2/169: dread d1/d15/f27 [4194304,4194304] 0 2024-05-27T09:29:31.373 INFO:tasks.workunit.client.1.smithi138.stdout:9/181: rename f2 to d6/da/d26/f37 0 2024-05-27T09:29:31.373 INFO:tasks.workunit.client.1.smithi138.stdout:9/182: chown d6/db/dc/f2c 1 1 2024-05-27T09:29:31.375 INFO:tasks.workunit.client.0.smithi119.stdout:2/733: creat d2/da/d8a/dec/ff6 x:0 0 0 2024-05-27T09:29:31.377 INFO:tasks.workunit.client.1.smithi138.stdout:5/192: link d3/d12/l29 d3/d12/d16/d1d/l3f 0 2024-05-27T09:29:31.377 INFO:tasks.workunit.client.1.smithi138.stdout:5/193: fsync f0 0 2024-05-27T09:29:31.377 INFO:tasks.workunit.client.1.smithi138.stdout:2/170: mknod d1/c36 0 2024-05-27T09:29:31.379 INFO:tasks.workunit.client.1.smithi138.stdout:1/253: dread d4/da/f2f [0,4194304] 0 2024-05-27T09:29:31.380 INFO:tasks.workunit.client.1.smithi138.stdout:9/183: creat d6/da/d26/d28/f38 x:0 0 0 2024-05-27T09:29:31.384 INFO:tasks.workunit.client.1.smithi138.stdout:2/171: rename d1/d15/f27 to d1/d15/d2d/d2e/f37 0 2024-05-27T09:29:31.391 INFO:tasks.workunit.client.1.smithi138.stdout:6/280: dread d3/d19/f34 [0,4194304] 0 2024-05-27T09:29:31.393 INFO:tasks.workunit.client.0.smithi119.stdout:1/763: symlink d7/db/d52/d65/le9 0 2024-05-27T09:29:31.394 INFO:tasks.workunit.client.0.smithi119.stdout:1/764: readlink d7/dd/d44/d47/l64 0 2024-05-27T09:29:31.395 INFO:tasks.workunit.client.1.smithi138.stdout:2/172: symlink d1/d15/d2d/d17/d25/l38 0 2024-05-27T09:29:31.395 INFO:tasks.workunit.client.1.smithi138.stdout:2/173: dread - d1/d2f/f33 zero size 2024-05-27T09:29:31.399 INFO:tasks.workunit.client.1.smithi138.stdout:6/281: mkdir d3/d35/d5a/d6a 0 2024-05-27T09:29:31.399 INFO:tasks.workunit.client.1.smithi138.stdout:6/282: write d3/d35/d43/d1c/f4c [545619,112876] 0 2024-05-27T09:29:31.402 INFO:tasks.workunit.client.0.smithi119.stdout:0/678: mknod d2/d6/d23/d7a/cdb 0 2024-05-27T09:29:31.402 INFO:tasks.workunit.client.0.smithi119.stdout:0/679: creat d2/d6/d23/db8/fdc x:0 0 0 2024-05-27T09:29:31.412 INFO:tasks.workunit.client.1.smithi138.stdout:7/176: dread d0/f10 [0,4194304] 0 2024-05-27T09:29:31.414 INFO:tasks.workunit.client.1.smithi138.stdout:2/174: dread d1/d10/fe [0,4194304] 0 2024-05-27T09:29:31.414 INFO:tasks.workunit.client.1.smithi138.stdout:2/175: write d1/d10/f6 [5059811,116503] 0 2024-05-27T09:29:31.415 INFO:tasks.workunit.client.0.smithi119.stdout:8/688: getdents d3/d6e/d8e 0 2024-05-27T09:29:31.415 INFO:tasks.workunit.client.0.smithi119.stdout:9/615: dwrite d18/f9a [0,4194304] 0 2024-05-27T09:29:31.419 INFO:tasks.workunit.client.0.smithi119.stdout:7/765: dwrite d7/df/d19/d9a/da6/dda/f48 [0,4194304] 0 2024-05-27T09:29:31.420 INFO:tasks.workunit.client.1.smithi138.stdout:2/176: creat d1/d15/d2d/f39 x:0 0 0 2024-05-27T09:29:31.422 INFO:tasks.workunit.client.1.smithi138.stdout:2/177: read d1/d10/f6 [4488723,51595] 0 2024-05-27T09:29:31.422 INFO:tasks.workunit.client.1.smithi138.stdout:2/178: dread - d1/d2f/f33 zero size 2024-05-27T09:29:31.422 INFO:tasks.workunit.client.0.smithi119.stdout:2/734: mknod d2/deb/cf7 0 2024-05-27T09:29:31.422 INFO:tasks.workunit.client.0.smithi119.stdout:2/735: chown d2/da/d8a/dd4/d89/db0/fca 13491926 1 2024-05-27T09:29:31.423 INFO:tasks.workunit.client.0.smithi119.stdout:2/736: truncate d2/da/d8a/dd4/d89/f8d 1140142 0 2024-05-27T09:29:31.423 INFO:tasks.workunit.client.1.smithi138.stdout:2/179: creat d1/d15/d2d/d17/d25/f3a x:0 0 0 2024-05-27T09:29:31.424 INFO:tasks.workunit.client.1.smithi138.stdout:0/239: dwrite da/d2a/f35 [0,4194304] 0 2024-05-27T09:29:31.424 INFO:tasks.workunit.client.1.smithi138.stdout:0/240: readlink da/le 0 2024-05-27T09:29:31.425 INFO:tasks.workunit.client.0.smithi119.stdout:1/765: symlink d7/db/d8f/lea 0 2024-05-27T09:29:31.426 INFO:tasks.workunit.client.1.smithi138.stdout:8/183: dwrite d3/d8/fc [0,4194304] 0 2024-05-27T09:29:31.426 INFO:tasks.workunit.client.1.smithi138.stdout:8/184: fdatasync d3/d8/d11/f22 0 2024-05-27T09:29:31.426 INFO:tasks.workunit.client.1.smithi138.stdout:4/184: dwrite d7/d27/f34 [0,4194304] 0 2024-05-27T09:29:31.429 INFO:tasks.workunit.client.1.smithi138.stdout:9/184: dwrite d6/da/d26/d28/f38 [0,4194304] 0 2024-05-27T09:29:31.435 INFO:tasks.workunit.client.1.smithi138.stdout:8/185: mkdir d3/d8/d32 0 2024-05-27T09:29:31.439 INFO:tasks.workunit.client.0.smithi119.stdout:0/680: rename d2/d1a/l4d to d2/d6/d9/d24/d69/ldd 0 2024-05-27T09:29:31.439 INFO:tasks.workunit.client.0.smithi119.stdout:4/760: dwrite d4/d23/d31/f33 [0,4194304] 0 2024-05-27T09:29:31.440 INFO:tasks.workunit.client.1.smithi138.stdout:7/177: dread d0/d7/f39 [0,4194304] 0 2024-05-27T09:29:31.441 INFO:tasks.workunit.client.1.smithi138.stdout:7/178: chown d0/d7/c17 59 1 2024-05-27T09:29:31.441 INFO:tasks.workunit.client.1.smithi138.stdout:7/179: write d0/d7/f39 [4436225,128713] 0 2024-05-27T09:29:31.443 INFO:tasks.workunit.client.1.smithi138.stdout:4/185: mkdir d7/d27/d3a 0 2024-05-27T09:29:31.444 INFO:tasks.workunit.client.0.smithi119.stdout:8/689: creat d3/fe5 x:0 0 0 2024-05-27T09:29:31.444 INFO:tasks.workunit.client.0.smithi119.stdout:4/761: write d4/d1c/d39/f64 [250558,102042] 0 2024-05-27T09:29:31.448 INFO:tasks.workunit.client.1.smithi138.stdout:0/241: dread da/f15 [0,4194304] 0 2024-05-27T09:29:31.449 INFO:tasks.workunit.client.0.smithi119.stdout:2/737: creat d2/deb/ff8 x:0 0 0 2024-05-27T09:29:31.449 INFO:tasks.workunit.client.0.smithi119.stdout:2/738: truncate d2/da/d8a/dd4/d89/d94/f92 509069 0 2024-05-27T09:29:31.450 INFO:tasks.workunit.client.1.smithi138.stdout:7/180: mkdir d0/d23/d3a 0 2024-05-27T09:29:31.450 INFO:tasks.workunit.client.1.smithi138.stdout:5/194: truncate d3/d12/d16/f39 4303778 0 2024-05-27T09:29:31.450 INFO:tasks.workunit.client.1.smithi138.stdout:5/195: creat d3/d12/d16/d1d/d22/d2b/f40 x:0 0 0 2024-05-27T09:29:31.456 INFO:tasks.workunit.client.1.smithi138.stdout:6/283: dwrite f2 [4194304,4194304] 0 2024-05-27T09:29:31.457 INFO:tasks.workunit.client.1.smithi138.stdout:4/186: link c2 d7/d27/c3b 0 2024-05-27T09:29:31.457 INFO:tasks.workunit.client.1.smithi138.stdout:4/187: fdatasync d7/f39 0 2024-05-27T09:29:31.457 INFO:tasks.workunit.client.1.smithi138.stdout:4/188: write d7/df/f29 [788592,4851] 0 2024-05-27T09:29:31.457 INFO:tasks.workunit.client.1.smithi138.stdout:0/242: rename da/c16 to da/d17/d1d/d43/c58 0 2024-05-27T09:29:31.458 INFO:tasks.workunit.client.1.smithi138.stdout:0/243: creat da/d2a/d3f/f59 x:0 0 0 2024-05-27T09:29:31.458 INFO:tasks.workunit.client.1.smithi138.stdout:0/244: fsync f1 0 2024-05-27T09:29:31.458 INFO:tasks.workunit.client.1.smithi138.stdout:4/189: creat d7/df/f3c x:0 0 0 2024-05-27T09:29:31.461 INFO:tasks.workunit.client.0.smithi119.stdout:9/616: rmdir d18/d19/d1b/d40/d7b 39 2024-05-27T09:29:31.465 INFO:tasks.workunit.client.0.smithi119.stdout:8/690: dread d3/d8/d1b/d46/f81 [0,4194304] 0 2024-05-27T09:29:31.466 INFO:tasks.workunit.client.1.smithi138.stdout:5/196: link d3/d12/d16/d1d/f24 d3/d12/d16/d1d/d22/d2b/d36/f41 0 2024-05-27T09:29:31.466 INFO:tasks.workunit.client.1.smithi138.stdout:5/197: fdatasync d3/d12/f17 0 2024-05-27T09:29:31.466 INFO:tasks.workunit.client.1.smithi138.stdout:6/284: creat d3/d35/d5a/d6a/f6b x:0 0 0 2024-05-27T09:29:31.467 INFO:tasks.workunit.client.1.smithi138.stdout:7/181: creat d0/d7/f3b x:0 0 0 2024-05-27T09:29:31.468 INFO:tasks.workunit.client.0.smithi119.stdout:0/681: read d2/d6/d9/d12/d4b/d59/dd7/f87 [1433838,17881] 0 2024-05-27T09:29:31.468 INFO:tasks.workunit.client.1.smithi138.stdout:8/186: dread d3/d8/f14 [0,4194304] 0 2024-05-27T09:29:31.469 INFO:tasks.workunit.client.0.smithi119.stdout:2/739: write d2/da/d8a/dd4/d89/d94/f92 [230648,44115] 0 2024-05-27T09:29:31.469 INFO:tasks.workunit.client.1.smithi138.stdout:8/187: creat d3/d8/f33 x:0 0 0 2024-05-27T09:29:31.472 INFO:tasks.workunit.client.0.smithi119.stdout:3/865: dwrite db/d36/d45/d4c/d62/fe8 [0,4194304] 0 2024-05-27T09:29:31.472 INFO:tasks.workunit.client.0.smithi119.stdout:3/866: read - db/d36/d45/dab/f101 zero size 2024-05-27T09:29:31.472 INFO:tasks.workunit.client.0.smithi119.stdout:3/867: creat db/d36/d45/f118 x:0 0 0 2024-05-27T09:29:31.472 INFO:tasks.workunit.client.0.smithi119.stdout:9/617: rename d18/d19/d1b/d40/c73 to d18/d19/d1b/d40/cc0 0 2024-05-27T09:29:31.472 INFO:tasks.workunit.client.0.smithi119.stdout:9/618: stat d18/d19/d1b/d29/f2f 0 2024-05-27T09:29:31.472 INFO:tasks.workunit.client.0.smithi119.stdout:9/619: chown d18/f30 379563 1 2024-05-27T09:29:31.472 INFO:tasks.workunit.client.0.smithi119.stdout:9/620: fsync d18/f76 0 2024-05-27T09:29:31.476 INFO:tasks.workunit.client.0.smithi119.stdout:8/691: mknod d3/d8/d4d/d5d/db7/ce6 0 2024-05-27T09:29:31.476 INFO:tasks.workunit.client.0.smithi119.stdout:8/692: creat d3/d8/fe7 x:0 0 0 2024-05-27T09:29:31.477 INFO:tasks.workunit.client.1.smithi138.stdout:4/190: rename d7/d27/f37 to d7/f3d 0 2024-05-27T09:29:31.478 INFO:tasks.workunit.client.1.smithi138.stdout:7/182: unlink d0/d7/lc 0 2024-05-27T09:29:31.488 INFO:tasks.workunit.client.0.smithi119.stdout:4/762: dread d4/d6/fd [0,4194304] 0 2024-05-27T09:29:31.488 INFO:tasks.workunit.client.0.smithi119.stdout:1/766: dwrite d7/db/d52/d65/fa4 [0,4194304] 0 2024-05-27T09:29:31.489 INFO:tasks.workunit.client.0.smithi119.stdout:3/868: rename db/d36/d45/d49/d9d/fb5 to db/d36/d45/d4c/d62/d13/d33/d9b/de3/f119 0 2024-05-27T09:29:31.489 INFO:tasks.workunit.client.0.smithi119.stdout:3/869: write db/d36/d45/d4c/d62/d13/f97 [3229409,12426] 0 2024-05-27T09:29:31.493 INFO:tasks.workunit.client.1.smithi138.stdout:8/188: truncate f1 505438 0 2024-05-27T09:29:31.494 INFO:tasks.workunit.client.0.smithi119.stdout:0/682: rmdir d2/d6/d9/d24 39 2024-05-27T09:29:31.497 INFO:tasks.workunit.client.1.smithi138.stdout:5/198: dread d3/fd [0,4194304] 0 2024-05-27T09:29:31.500 INFO:tasks.workunit.client.0.smithi119.stdout:7/766: dwrite d7/df/d19/d46/d93/ff4 [0,4194304] 0 2024-05-27T09:29:31.500 INFO:tasks.workunit.client.0.smithi119.stdout:7/767: write d7/df/d19/f14 [833172,5599] 0 2024-05-27T09:29:31.502 INFO:tasks.workunit.client.1.smithi138.stdout:7/183: readlink d0/l13 0 2024-05-27T09:29:31.502 INFO:tasks.workunit.client.1.smithi138.stdout:8/189: mkdir d3/d8/d32/d34 0 2024-05-27T09:29:31.516 INFO:tasks.workunit.client.0.smithi119.stdout:0/683: mknod d2/d6/d23/d7a/d53/cde 0 2024-05-27T09:29:31.516 INFO:tasks.workunit.client.0.smithi119.stdout:4/763: getdents d4/d6/d81 0 2024-05-27T09:29:31.517 INFO:tasks.workunit.client.0.smithi119.stdout:7/768: dread d7/df/d19/d9a/da6/dda/d44/f5d [0,4194304] 0 2024-05-27T09:29:31.522 INFO:tasks.workunit.client.1.smithi138.stdout:6/285: write d3/d19/f1e [566795,89452] 0 2024-05-27T09:29:31.522 INFO:tasks.workunit.client.1.smithi138.stdout:6/286: read - d3/d35/d43/f3f zero size 2024-05-27T09:29:31.522 INFO:tasks.workunit.client.1.smithi138.stdout:6/287: chown d3/d19/f3d 246228684 1 2024-05-27T09:29:31.522 INFO:tasks.workunit.client.1.smithi138.stdout:6/288: dread - d3/f53 zero size 2024-05-27T09:29:31.522 INFO:tasks.workunit.client.1.smithi138.stdout:6/289: stat d3/d35/d5a/d6a 0 2024-05-27T09:29:31.522 INFO:tasks.workunit.client.1.smithi138.stdout:6/290: write d3/d19/f3d [252652,78630] 0 2024-05-27T09:29:31.522 INFO:tasks.workunit.client.1.smithi138.stdout:6/291: read d3/d19/f28 [548568,112249] 0 2024-05-27T09:29:31.522 INFO:tasks.workunit.client.1.smithi138.stdout:6/292: readlink d3/d35/d43/d1c/l24 0 2024-05-27T09:29:31.522 INFO:tasks.workunit.client.1.smithi138.stdout:6/293: read - d3/d11/d2d/f61 zero size 2024-05-27T09:29:31.522 INFO:tasks.workunit.client.1.smithi138.stdout:5/199: link d3/d12/f17 d3/d12/d32/f42 0 2024-05-27T09:29:31.526 INFO:tasks.workunit.client.0.smithi119.stdout:0/684: mknod d2/d6/d9/d12/cdf 0 2024-05-27T09:29:31.526 INFO:tasks.workunit.client.0.smithi119.stdout:0/685: read - d2/d6/d23/f8f zero size 2024-05-27T09:29:31.527 INFO:tasks.workunit.client.0.smithi119.stdout:4/764: mkdir d4/d6/d3e/d89/db2/def 0 2024-05-27T09:29:31.527 INFO:tasks.workunit.client.0.smithi119.stdout:4/765: write d4/d23/d24/d5b/da8/fb3 [943272,128431] 0 2024-05-27T09:29:31.527 INFO:tasks.workunit.client.0.smithi119.stdout:4/766: read - d4/d23/fdf zero size 2024-05-27T09:29:31.528 INFO:tasks.workunit.client.0.smithi119.stdout:7/769: mknod d7/df/d19/d97/d92/db7/dba/d42/d64/dad/dd3/cf5 0 2024-05-27T09:29:31.530 INFO:tasks.workunit.client.0.smithi119.stdout:0/686: truncate d2/d6/d9/f2c 64649 0 2024-05-27T09:29:31.531 INFO:tasks.workunit.client.0.smithi119.stdout:0/687: creat d2/d6/d9/d12/da3/fe0 x:0 0 0 2024-05-27T09:29:31.531 INFO:tasks.workunit.client.0.smithi119.stdout:0/688: stat d2/d6/d23/d7a/cd5 0 2024-05-27T09:29:31.531 INFO:tasks.workunit.client.0.smithi119.stdout:0/689: chown d2/d6/d9/d12/c3f 0 1 2024-05-27T09:29:31.531 INFO:tasks.workunit.client.0.smithi119.stdout:0/690: stat d2/d6/d9/d24/d30/d4a/caa 0 2024-05-27T09:29:31.531 INFO:tasks.workunit.client.0.smithi119.stdout:0/691: readlink d2/d6/d9/d12/d4f/l80 0 2024-05-27T09:29:31.536 INFO:tasks.workunit.client.0.smithi119.stdout:4/767: symlink d4/d6/d81/dce/d88/lf0 0 2024-05-27T09:29:31.538 INFO:tasks.workunit.client.0.smithi119.stdout:7/770: creat d7/df/d19/d4d/d89/deb/ff6 x:0 0 0 2024-05-27T09:29:31.545 INFO:tasks.workunit.client.1.smithi138.stdout:7/184: mkdir d0/d23/d3c 0 2024-05-27T09:29:31.545 INFO:tasks.workunit.client.1.smithi138.stdout:7/185: readlink d0/l13 0 2024-05-27T09:29:31.545 INFO:tasks.workunit.client.1.smithi138.stdout:8/190: symlink d3/d8/d11/l35 0 2024-05-27T09:29:31.545 INFO:tasks.workunit.client.0.smithi119.stdout:0/692: creat d2/d6/d9/d24/d69/fe1 x:0 0 0 2024-05-27T09:29:31.545 INFO:tasks.workunit.client.0.smithi119.stdout:0/693: fdatasync d2/d6/d9/d2f/fc1 0 2024-05-27T09:29:31.549 INFO:tasks.workunit.client.0.smithi119.stdout:4/768: unlink d4/de/d66/ld6 0 2024-05-27T09:29:31.549 INFO:tasks.workunit.client.0.smithi119.stdout:4/769: chown d4/d1c/d39/f57 84362 1 2024-05-27T09:29:31.549 INFO:tasks.workunit.client.0.smithi119.stdout:4/770: creat d4/de/d9b/ff1 x:0 0 0 2024-05-27T09:29:31.552 INFO:tasks.workunit.client.0.smithi119.stdout:7/771: creat d7/df/d19/d9a/da6/dda/d44/ff7 x:0 0 0 2024-05-27T09:29:31.557 INFO:tasks.workunit.client.1.smithi138.stdout:0/245: dwrite da/d17/d1d/d1f/d42/d4a/f4e [0,4194304] 0 2024-05-27T09:29:31.557 INFO:tasks.workunit.client.0.smithi119.stdout:7/772: symlink d7/df/d19/d46/lf8 0 2024-05-27T09:29:31.558 INFO:tasks.workunit.client.0.smithi119.stdout:7/773: chown d7/d1a/d99/cc3 1211132206 1 2024-05-27T09:29:31.558 INFO:tasks.workunit.client.0.smithi119.stdout:7/774: write d7/df/d19/d97/d92/db7/dba/d42/fe9 [122249,110170] 0 2024-05-27T09:29:31.558 INFO:tasks.workunit.client.0.smithi119.stdout:7/775: fsync d7/d1a/f59 0 2024-05-27T09:29:31.566 INFO:tasks.workunit.client.0.smithi119.stdout:7/776: getdents d7/d1a/d99/dcf 0 2024-05-27T09:29:31.572 INFO:tasks.workunit.client.0.smithi119.stdout:3/870: dwrite db/fb1 [0,4194304] 0 2024-05-27T09:29:31.572 INFO:tasks.workunit.client.0.smithi119.stdout:7/777: rename d7/f6b to d7/df/d19/d78/ff9 0 2024-05-27T09:29:31.572 INFO:tasks.workunit.client.0.smithi119.stdout:7/778: creat d7/d5f/d79/ffa x:0 0 0 2024-05-27T09:29:31.575 INFO:tasks.workunit.client.0.smithi119.stdout:0/694: dread d2/d6/d23/d7a/f46 [4194304,4194304] 0 2024-05-27T09:29:31.575 INFO:tasks.workunit.client.0.smithi119.stdout:6/713: sync 2024-05-27T09:29:31.578 INFO:tasks.workunit.client.0.smithi119.stdout:5/793: sync 2024-05-27T09:29:31.578 INFO:tasks.workunit.client.0.smithi119.stdout:8/693: write d3/d8/d1b/d77/d95/fab [3540029,39863] 0 2024-05-27T09:29:31.578 INFO:tasks.workunit.client.0.smithi119.stdout:6/714: fdatasync d0/d85/de7/ffa 0 2024-05-27T09:29:31.578 INFO:tasks.workunit.client.0.smithi119.stdout:7/779: link d7/d1a/d99/cc3 d7/df/d19/d97/d92/db7/dba/d42/dc5/cfb 0 2024-05-27T09:29:31.578 INFO:tasks.workunit.client.0.smithi119.stdout:2/740: dwrite d2/f47 [0,4194304] 0 2024-05-27T09:29:31.579 INFO:tasks.workunit.client.0.smithi119.stdout:4/771: write d4/f8f [2166393,29582] 0 2024-05-27T09:29:31.582 INFO:tasks.workunit.client.0.smithi119.stdout:1/767: dwrite d7/db/d12/d15/f3f [0,4194304] 0 2024-05-27T09:29:31.582 INFO:tasks.workunit.client.0.smithi119.stdout:0/695: rename d2/d1a/d20/c68 to d2/d6/d9/d24/ce2 0 2024-05-27T09:29:31.582 INFO:tasks.workunit.client.0.smithi119.stdout:8/694: unlink d3/de/c11 0 2024-05-27T09:29:31.586 INFO:tasks.workunit.client.0.smithi119.stdout:5/794: mknod d6/dd/d39/c11c 0 2024-05-27T09:29:31.587 INFO:tasks.workunit.client.0.smithi119.stdout:7/780: symlink d7/df/d19/d97/d37/lfc 0 2024-05-27T09:29:31.589 INFO:tasks.workunit.client.0.smithi119.stdout:3/871: dread db/dc9/d98/f73 [0,4194304] 0 2024-05-27T09:29:31.590 INFO:tasks.workunit.client.0.smithi119.stdout:1/768: dread d7/db/d52/fcc [0,4194304] 0 2024-05-27T09:29:31.590 INFO:tasks.workunit.client.0.smithi119.stdout:2/741: mkdir d2/da/d8a/dd4/d89/d94/de2/df9 0 2024-05-27T09:29:31.590 INFO:tasks.workunit.client.0.smithi119.stdout:2/742: readlink d2/da/d8a/dd4/l50 0 2024-05-27T09:29:31.596 INFO:tasks.workunit.client.1.smithi138.stdout:0/246: dread da/d2a/f35 [0,4194304] 0 2024-05-27T09:29:31.596 INFO:tasks.workunit.client.0.smithi119.stdout:9/621: dwrite d18/d19/d1b/d40/d44/fb0 [0,4194304] 0 2024-05-27T09:29:31.597 INFO:tasks.workunit.client.0.smithi119.stdout:9/622: creat d18/d19/d1b/d40/db7/fc1 x:0 0 0 2024-05-27T09:29:31.597 INFO:tasks.workunit.client.0.smithi119.stdout:9/623: fsync d18/d19/d1b/d29/f47 0 2024-05-27T09:29:31.599 INFO:tasks.workunit.client.0.smithi119.stdout:0/696: mkdir d2/d6/d9/d24/d30/dcc/de3 0 2024-05-27T09:29:31.599 INFO:tasks.workunit.client.0.smithi119.stdout:0/697: write d2/d6/d9/d24/d30/d4a/fc5 [57220,53422] 0 2024-05-27T09:29:31.600 INFO:tasks.workunit.client.0.smithi119.stdout:0/698: write d2/d6/d9/d12/d4b/d96/f98 [4656284,102297] 0 2024-05-27T09:29:31.600 INFO:tasks.workunit.client.0.smithi119.stdout:0/699: chown d2/d6/d9/f85 2298 1 2024-05-27T09:29:31.600 INFO:tasks.workunit.client.0.smithi119.stdout:8/695: symlink d3/d8/d1b/d46/da4/dbb/le8 0 2024-05-27T09:29:31.600 INFO:tasks.workunit.client.0.smithi119.stdout:0/700: write d2/d6/d23/fa9 [682529,43045] 0 2024-05-27T09:29:31.603 INFO:tasks.workunit.client.1.smithi138.stdout:5/200: dwrite d3/d12/d32/f42 [0,4194304] 0 2024-05-27T09:29:31.603 INFO:tasks.workunit.client.1.smithi138.stdout:5/201: fdatasync d3/f19 0 2024-05-27T09:29:31.608 INFO:tasks.workunit.client.0.smithi119.stdout:6/715: write d0/d2/d57/d37/f5d [786309,35360] 0 2024-05-27T09:29:31.611 INFO:tasks.workunit.client.0.smithi119.stdout:1/769: creat d7/db/d12/feb x:0 0 0 2024-05-27T09:29:31.612 INFO:tasks.workunit.client.0.smithi119.stdout:5/795: truncate d6/db/de/d27/f3f 2445892 0 2024-05-27T09:29:31.613 INFO:tasks.workunit.client.1.smithi138.stdout:5/202: rmdir d3/d12/d16/d1d/d23 39 2024-05-27T09:29:31.614 INFO:tasks.workunit.client.1.smithi138.stdout:4/191: dwrite d7/df/f19 [4194304,4194304] 0 2024-05-27T09:29:31.617 INFO:tasks.workunit.client.0.smithi119.stdout:8/696: creat d3/d8/d4d/fe9 x:0 0 0 2024-05-27T09:29:31.620 INFO:tasks.workunit.client.0.smithi119.stdout:6/716: mknod d0/d2/d1c/cfb 0 2024-05-27T09:29:31.620 INFO:tasks.workunit.client.0.smithi119.stdout:6/717: write d0/d2/d57/d37/d74/f98 [623590,97268] 0 2024-05-27T09:29:31.622 INFO:tasks.workunit.client.0.smithi119.stdout:7/781: write d7/df/d19/d46/d93/ff4 [31492,117342] 0 2024-05-27T09:29:31.626 INFO:tasks.workunit.client.0.smithi119.stdout:5/796: creat d6/db/de/d27/dc9/dae/f11d x:0 0 0 2024-05-27T09:29:31.626 INFO:tasks.workunit.client.0.smithi119.stdout:5/797: chown d6/dd/d39/d19/fb6 0 1 2024-05-27T09:29:31.626 INFO:tasks.workunit.client.0.smithi119.stdout:5/798: chown d6/dd/d6e/fe2 238299297 1 2024-05-27T09:29:31.627 INFO:tasks.workunit.client.0.smithi119.stdout:5/799: fsync d6/f12 0 2024-05-27T09:29:31.627 INFO:tasks.workunit.client.0.smithi119.stdout:1/770: rename d7/db/d12/f96 to d7/db/d12/d26/d39/fec 0 2024-05-27T09:29:31.628 INFO:tasks.workunit.client.1.smithi138.stdout:5/203: creat d3/d12/d16/d1d/d22/d2b/d36/f43 x:0 0 0 2024-05-27T09:29:31.629 INFO:tasks.workunit.client.1.smithi138.stdout:5/204: chown d3/d12/d16/d1d/f24 0 1 2024-05-27T09:29:31.629 INFO:tasks.workunit.client.1.smithi138.stdout:4/192: chown d7/d27/c28 2316608 1 2024-05-27T09:29:31.631 INFO:tasks.workunit.client.0.smithi119.stdout:8/697: creat d3/d8/d1b/d46/da4/dbb/fea x:0 0 0 2024-05-27T09:29:31.633 INFO:tasks.workunit.client.1.smithi138.stdout:4/193: chown d7/d27/f2c 13657214 1 2024-05-27T09:29:31.633 INFO:tasks.workunit.client.1.smithi138.stdout:4/194: chown d7/d27/c3b 0 1 2024-05-27T09:29:31.633 INFO:tasks.workunit.client.0.smithi119.stdout:7/782: write d7/df/d19/d97/d92/db7/dba/d42/d64/fc9 [455222,72666] 0 2024-05-27T09:29:31.636 INFO:tasks.workunit.client.0.smithi119.stdout:0/701: dread d2/f8c [0,4194304] 0 2024-05-27T09:29:31.637 INFO:tasks.workunit.client.0.smithi119.stdout:0/702: dread - d2/d6/d9/d24/fcb zero size 2024-05-27T09:29:31.637 INFO:tasks.workunit.client.1.smithi138.stdout:0/247: write da/f20 [1961440,103668] 0 2024-05-27T09:29:31.637 INFO:tasks.workunit.client.0.smithi119.stdout:0/703: write d2/d6/d9/d12/d4b/d59/f83 [598575,16259] 0 2024-05-27T09:29:31.639 INFO:tasks.workunit.client.1.smithi138.stdout:6/294: dwrite d3/d19/f22 [0,4194304] 0 2024-05-27T09:29:31.646 INFO:tasks.workunit.client.0.smithi119.stdout:6/718: creat d0/d2/dec/ffc x:0 0 0 2024-05-27T09:29:31.647 INFO:tasks.workunit.client.1.smithi138.stdout:7/186: dwrite d0/f24 [0,4194304] 0 2024-05-27T09:29:31.648 INFO:tasks.workunit.client.1.smithi138.stdout:4/195: dread d7/df/f24 [0,4194304] 0 2024-05-27T09:29:31.650 INFO:tasks.workunit.client.1.smithi138.stdout:4/196: getdents d7/d27 0 2024-05-27T09:29:31.652 INFO:tasks.workunit.client.1.smithi138.stdout:7/187: symlink d0/d23/l3d 0 2024-05-27T09:29:31.652 INFO:tasks.workunit.client.1.smithi138.stdout:4/197: symlink d7/d27/d3a/l3e 0 2024-05-27T09:29:31.654 INFO:tasks.workunit.client.1.smithi138.stdout:4/198: symlink d7/df/d35/l3f 0 2024-05-27T09:29:31.654 INFO:tasks.workunit.client.1.smithi138.stdout:4/199: write d7/df/f29 [882809,103384] 0 2024-05-27T09:29:31.654 INFO:tasks.workunit.client.1.smithi138.stdout:4/200: fsync d7/df/f1a 0 2024-05-27T09:29:31.654 INFO:tasks.workunit.client.1.smithi138.stdout:7/188: symlink d0/d23/d3a/l3e 0 2024-05-27T09:29:31.656 INFO:tasks.workunit.client.1.smithi138.stdout:4/201: rename d7/df/c2d to d7/d27/c40 0 2024-05-27T09:29:31.656 INFO:tasks.workunit.client.1.smithi138.stdout:4/202: fsync d7/d27/f2c 0 2024-05-27T09:29:31.657 INFO:tasks.workunit.client.1.smithi138.stdout:4/203: write d7/df/f33 [1839258,75139] 0 2024-05-27T09:29:31.658 INFO:tasks.workunit.client.1.smithi138.stdout:7/189: unlink d0/d7/l2f 0 2024-05-27T09:29:31.658 INFO:tasks.workunit.client.1.smithi138.stdout:7/190: write d0/f10 [1137988,58948] 0 2024-05-27T09:29:31.660 INFO:tasks.workunit.client.1.smithi138.stdout:4/204: unlink d7/df/c2a 0 2024-05-27T09:29:31.660 INFO:tasks.workunit.client.1.smithi138.stdout:4/205: creat d7/df/f41 x:0 0 0 2024-05-27T09:29:31.661 INFO:tasks.workunit.client.0.smithi119.stdout:6/719: write d0/d19/f69 [2167464,28041] 0 2024-05-27T09:29:31.663 INFO:tasks.workunit.client.1.smithi138.stdout:4/206: rename d7/df/f1d to d7/f42 0 2024-05-27T09:29:31.665 INFO:tasks.workunit.client.1.smithi138.stdout:4/207: symlink d7/df/l43 0 2024-05-27T09:29:31.667 INFO:tasks.workunit.client.0.smithi119.stdout:5/800: mknod d6/dd/d46/d47/d4b/dd8/ddb/c11e 0 2024-05-27T09:29:31.667 INFO:tasks.workunit.client.1.smithi138.stdout:8/191: dwrite d3/d8/d28/f2f [0,4194304] 0 2024-05-27T09:29:31.668 INFO:tasks.workunit.client.0.smithi119.stdout:1/771: creat d7/db/d12/d15/d1f/fed x:0 0 0 2024-05-27T09:29:31.668 INFO:tasks.workunit.client.0.smithi119.stdout:1/772: chown d7/dd/d44/fa9 225837 1 2024-05-27T09:29:31.669 INFO:tasks.workunit.client.1.smithi138.stdout:4/208: rename d7/df/f33 to d7/f44 0 2024-05-27T09:29:31.672 INFO:tasks.workunit.client.1.smithi138.stdout:4/209: mknod d7/d27/d3a/c45 0 2024-05-27T09:29:31.675 INFO:tasks.workunit.client.0.smithi119.stdout:2/743: dwrite d2/da/d2c/fe1 [0,4194304] 0 2024-05-27T09:29:31.678 INFO:tasks.workunit.client.1.smithi138.stdout:4/210: unlink d7/df/f3c 0 2024-05-27T09:29:31.679 INFO:tasks.workunit.client.0.smithi119.stdout:8/698: link d3/d6e/dcf/dce/fd8 d3/d8/d4d/d5d/feb 0 2024-05-27T09:29:31.679 INFO:tasks.workunit.client.0.smithi119.stdout:8/699: creat d3/d8/d4d/d79/fec x:0 0 0 2024-05-27T09:29:31.680 INFO:tasks.workunit.client.0.smithi119.stdout:8/700: dread - d3/f9d zero size 2024-05-27T09:29:31.680 INFO:tasks.workunit.client.0.smithi119.stdout:8/701: chown d3/d8/d1b/f28 997447224 1 2024-05-27T09:29:31.680 INFO:tasks.workunit.client.0.smithi119.stdout:8/702: write d3/d8/d1b/db1/fc5 [634739,59132] 0 2024-05-27T09:29:31.680 INFO:tasks.workunit.client.0.smithi119.stdout:8/703: dread - d3/d8/d4d/d5d/feb zero size 2024-05-27T09:29:31.680 INFO:tasks.workunit.client.0.smithi119.stdout:2/744: read d2/da/f13 [129987,5282] 0 2024-05-27T09:29:31.681 INFO:tasks.workunit.client.1.smithi138.stdout:4/211: rename d7/d27/c28 to d7/d27/d3a/c46 0 2024-05-27T09:29:31.682 INFO:tasks.workunit.client.0.smithi119.stdout:7/783: unlink c2 0 2024-05-27T09:29:31.682 INFO:tasks.workunit.client.1.smithi138.stdout:4/212: rmdir d7/d27/d3a 39 2024-05-27T09:29:31.682 INFO:tasks.workunit.client.1.smithi138.stdout:4/213: chown d7/df/d35 27965309 1 2024-05-27T09:29:31.683 INFO:tasks.workunit.client.0.smithi119.stdout:0/704: mkdir d2/d6/d9/d24/d30/dcc/de3/de4 0 2024-05-27T09:29:31.684 INFO:tasks.workunit.client.1.smithi138.stdout:7/191: dread d0/f24 [0,4194304] 0 2024-05-27T09:29:31.684 INFO:tasks.workunit.client.1.smithi138.stdout:7/192: write d0/f10 [132722,16648] 0 2024-05-27T09:29:31.690 INFO:tasks.workunit.client.1.smithi138.stdout:7/193: mkdir d0/d23/d3c/d3f 0 2024-05-27T09:29:31.690 INFO:tasks.workunit.client.1.smithi138.stdout:7/194: read - d0/f37 zero size 2024-05-27T09:29:31.690 INFO:tasks.workunit.client.1.smithi138.stdout:7/195: fdatasync d0/f19 0 2024-05-27T09:29:31.690 INFO:tasks.workunit.client.1.smithi138.stdout:7/196: write d0/f36 [580552,60189] 0 2024-05-27T09:29:31.694 INFO:tasks.workunit.client.0.smithi119.stdout:4/772: dwrite d4/d6/f2a [0,4194304] 0 2024-05-27T09:29:31.694 INFO:tasks.workunit.client.1.smithi138.stdout:8/192: dread d3/d8/d28/f2f [0,4194304] 0 2024-05-27T09:29:31.694 INFO:tasks.workunit.client.1.smithi138.stdout:8/193: fsync d3/d8/d11/f22 0 2024-05-27T09:29:31.694 INFO:tasks.workunit.client.1.smithi138.stdout:7/197: rename d0/f37 to d0/d2b/f40 0 2024-05-27T09:29:31.697 INFO:tasks.workunit.client.1.smithi138.stdout:8/194: rename d3/d8/cd to d3/d8/c36 0 2024-05-27T09:29:31.697 INFO:tasks.workunit.client.1.smithi138.stdout:8/195: write d3/d8/d28/f2e [1045131,124852] 0 2024-05-27T09:29:31.697 INFO:tasks.workunit.client.1.smithi138.stdout:8/196: chown d3/d8/l2a 210 1 2024-05-27T09:29:31.697 INFO:tasks.workunit.client.1.smithi138.stdout:8/197: chown d3 9 1 2024-05-27T09:29:31.698 INFO:tasks.workunit.client.1.smithi138.stdout:8/198: rename d3 to d3/d8/d32/d37 22 2024-05-27T09:29:31.699 INFO:tasks.workunit.client.0.smithi119.stdout:6/720: creat d0/d2/d1c/d21/ffd x:0 0 0 2024-05-27T09:29:31.699 INFO:tasks.workunit.client.0.smithi119.stdout:6/721: fdatasync d0/d2/d57/f6d 0 2024-05-27T09:29:31.703 INFO:tasks.workunit.client.0.smithi119.stdout:5/801: symlink d6/dd/d39/d19/d70/l11f 0 2024-05-27T09:29:31.706 INFO:tasks.workunit.client.0.smithi119.stdout:1/773: link d7/db/d12/ddb/la6 d7/dd/d44/lee 0 2024-05-27T09:29:31.710 INFO:tasks.workunit.client.0.smithi119.stdout:1/774: read d7/f27 [981010,22232] 0 2024-05-27T09:29:31.710 INFO:tasks.workunit.client.1.smithi138.stdout:7/198: dread d0/d7/f12 [0,4194304] 0 2024-05-27T09:29:31.710 INFO:tasks.workunit.client.1.smithi138.stdout:8/199: write d3/d8/d11/f26 [45636,14056] 0 2024-05-27T09:29:31.711 INFO:tasks.workunit.client.1.smithi138.stdout:7/199: chown d0/f1d 3722 1 2024-05-27T09:29:31.711 INFO:tasks.workunit.client.0.smithi119.stdout:8/704: creat d3/d8/d1b/fed x:0 0 0 2024-05-27T09:29:31.711 INFO:tasks.workunit.client.0.smithi119.stdout:2/745: symlink d2/da/d8a/d8f/lfa 0 2024-05-27T09:29:31.715 INFO:tasks.workunit.client.0.smithi119.stdout:5/802: readlink d6/dd/d39/d19/d3c/l91 0 2024-05-27T09:29:31.717 INFO:tasks.workunit.client.0.smithi119.stdout:1/775: rename d7/db/d12/d15/d1f/f3a to d7/fef 0 2024-05-27T09:29:31.717 INFO:tasks.workunit.client.0.smithi119.stdout:1/776: chown d7/db/d8f/d9b/fb9 27930 1 2024-05-27T09:29:31.717 INFO:tasks.workunit.client.0.smithi119.stdout:1/777: dread - d7/dd/d44/fa9 zero size 2024-05-27T09:29:31.717 INFO:tasks.workunit.client.0.smithi119.stdout:6/722: getdents d0/d85/de7 0 2024-05-27T09:29:31.719 INFO:tasks.workunit.client.0.smithi119.stdout:1/778: creat d7/dd/d25/d60/ff0 x:0 0 0 2024-05-27T09:29:31.721 INFO:tasks.workunit.client.1.smithi138.stdout:8/200: truncate d3/d8/d11/f22 166398 0 2024-05-27T09:29:31.721 INFO:tasks.workunit.client.1.smithi138.stdout:8/201: readlink d3/d8/l17 0 2024-05-27T09:29:31.723 INFO:tasks.workunit.client.0.smithi119.stdout:8/705: rename d3/d6e/d80/f99 to d3/d8/d4d/d5d/db7/fee 0 2024-05-27T09:29:31.724 INFO:tasks.workunit.client.0.smithi119.stdout:6/723: mkdir d0/d2/d57/d37/d74/dfe 0 2024-05-27T09:29:31.728 INFO:tasks.workunit.client.1.smithi138.stdout:7/200: unlink d0/l29 0 2024-05-27T09:29:31.728 INFO:tasks.workunit.client.1.smithi138.stdout:7/201: write d0/f1d [103943,32904] 0 2024-05-27T09:29:31.728 INFO:tasks.workunit.client.1.smithi138.stdout:7/202: fdatasync d0/f1a 0 2024-05-27T09:29:31.737 INFO:tasks.workunit.client.0.smithi119.stdout:1/779: rename d7/dd/c9c to d7/d6e/cf1 0 2024-05-27T09:29:31.740 INFO:tasks.workunit.client.1.smithi138.stdout:8/202: truncate d3/d8/d11/f26 641123 0 2024-05-27T09:29:31.745 INFO:tasks.workunit.client.0.smithi119.stdout:6/724: symlink d0/d19/lff 0 2024-05-27T09:29:31.745 INFO:tasks.workunit.client.0.smithi119.stdout:0/705: dwrite d2/d6/d23/db8/fdc [0,4194304] 0 2024-05-27T09:29:31.756 INFO:tasks.workunit.client.1.smithi138.stdout:8/203: dread d3/d8/d11/f22 [0,4194304] 0 2024-05-27T09:29:31.756 INFO:tasks.workunit.client.1.smithi138.stdout:8/204: write f1 [846125,105859] 0 2024-05-27T09:29:31.756 INFO:tasks.workunit.client.0.smithi119.stdout:6/725: mkdir d0/d85/d100 0 2024-05-27T09:29:31.761 INFO:tasks.workunit.client.0.smithi119.stdout:6/726: symlink d0/d19/l101 0 2024-05-27T09:29:31.761 INFO:tasks.workunit.client.0.smithi119.stdout:6/727: chown d0/d2/d1c/d21/d58/fd4 7597 1 2024-05-27T09:29:31.762 INFO:tasks.workunit.client.1.smithi138.stdout:8/205: mkdir d3/d8/d32/d34/d38 0 2024-05-27T09:29:31.777 INFO:tasks.workunit.client.0.smithi119.stdout:7/784: dwrite d7/d5f/f8c [0,4194304] 0 2024-05-27T09:29:31.778 INFO:tasks.workunit.client.0.smithi119.stdout:7/785: creat d7/df/d19/d9a/da6/dda/d44/ffd x:0 0 0 2024-05-27T09:29:31.788 INFO:tasks.workunit.client.0.smithi119.stdout:7/786: mkdir d7/d1a/d99/dbb/dc2/dfe 0 2024-05-27T09:29:31.788 INFO:tasks.workunit.client.0.smithi119.stdout:7/787: dread - d7/df/d19/d9a/da6/dda/d44/ff7 zero size 2024-05-27T09:29:31.788 INFO:tasks.workunit.client.0.smithi119.stdout:7/788: readlink d7/df/d19/l3a 0 2024-05-27T09:29:31.796 INFO:tasks.workunit.client.0.smithi119.stdout:7/789: truncate d7/df/d19/d97/d37/f5e 3055159 0 2024-05-27T09:29:31.805 INFO:tasks.workunit.client.0.smithi119.stdout:5/803: dwrite d6/d2f/d9d/f103 [4194304,4194304] 0 2024-05-27T09:29:31.807 INFO:tasks.workunit.client.0.smithi119.stdout:7/790: rename d7/df/d19/d1d/c2c to d7/df/d19/d97/d92/db7/dba/d42/d64/dad/cff 0 2024-05-27T09:29:31.808 INFO:tasks.workunit.client.0.smithi119.stdout:5/804: creat d6/dd/d46/d47/d4b/f120 x:0 0 0 2024-05-27T09:29:31.810 INFO:tasks.workunit.client.0.smithi119.stdout:5/805: mkdir d6/dd/d6e/d121 0 2024-05-27T09:29:31.810 INFO:tasks.workunit.client.0.smithi119.stdout:5/806: chown d6/dd/f9b 15488989 1 2024-05-27T09:29:31.811 INFO:tasks.workunit.client.1.smithi138.stdout:8/206: dread d3/d8/fc [0,4194304] 0 2024-05-27T09:29:31.812 INFO:tasks.workunit.client.1.smithi138.stdout:8/207: fdatasync d3/d8/d11/f22 0 2024-05-27T09:29:31.821 INFO:tasks.workunit.client.0.smithi119.stdout:7/791: dread d7/d1a/f2b [0,4194304] 0 2024-05-27T09:29:31.822 INFO:tasks.workunit.client.1.smithi138.stdout:7/203: dwrite d0/f10 [0,4194304] 0 2024-05-27T09:29:31.822 INFO:tasks.workunit.client.0.smithi119.stdout:4/773: dwrite d4/d1c/f29 [0,4194304] 0 2024-05-27T09:29:31.829 INFO:tasks.workunit.client.0.smithi119.stdout:8/706: dwrite d3/de/dad/fc0 [0,4194304] 0 2024-05-27T09:29:31.829 INFO:tasks.workunit.client.0.smithi119.stdout:8/707: chown d3/d8/d4d/d5d/cd4 11512 1 2024-05-27T09:29:31.829 INFO:tasks.workunit.client.0.smithi119.stdout:8/708: stat d3/d8/l35 0 2024-05-27T09:29:31.840 INFO:tasks.workunit.client.1.smithi138.stdout:7/204: fdatasync d0/d7/f12 0 2024-05-27T09:29:31.842 INFO:tasks.workunit.client.0.smithi119.stdout:7/792: mknod d7/df/d19/d9a/da6/dda/c100 0 2024-05-27T09:29:31.843 INFO:tasks.workunit.client.0.smithi119.stdout:2/746: dwrite d2/da/d8a/f9f [0,4194304] 0 2024-05-27T09:29:31.847 INFO:tasks.workunit.client.0.smithi119.stdout:1/780: dwrite d7/dd/d25/f30 [0,4194304] 0 2024-05-27T09:29:31.848 INFO:tasks.workunit.client.0.smithi119.stdout:7/793: mknod d7/df/d19/d97/d92/db7/c101 0 2024-05-27T09:29:31.848 INFO:tasks.workunit.client.0.smithi119.stdout:7/794: chown d7/d6a/f6f 51412 1 2024-05-27T09:29:31.855 INFO:tasks.workunit.client.0.smithi119.stdout:0/706: dwrite d2/d6/d9/d24/d69/fe1 [0,4194304] 0 2024-05-27T09:29:31.855 INFO:tasks.workunit.client.0.smithi119.stdout:0/707: stat d2/d6/d23/d7a/d53/f5f 0 2024-05-27T09:29:31.855 INFO:tasks.workunit.client.0.smithi119.stdout:0/708: chown d2/d6/d9/d2f/d95/cae 0 1 2024-05-27T09:29:31.855 INFO:tasks.workunit.client.0.smithi119.stdout:0/709: fdatasync d2/d6/d9/fbb 0 2024-05-27T09:29:31.855 INFO:tasks.workunit.client.0.smithi119.stdout:0/710: chown d2/d1a/f1e 292656781 1 2024-05-27T09:29:31.857 INFO:tasks.workunit.client.0.smithi119.stdout:0/711: mkdir d2/d6/d9/d12/de5 0 2024-05-27T09:29:31.859 INFO:tasks.workunit.client.0.smithi119.stdout:6/728: dwrite d0/d2/d4e/d5a/f97 [0,4194304] 0 2024-05-27T09:29:31.859 INFO:tasks.workunit.client.0.smithi119.stdout:5/807: dread d6/dd/d46/d47/f116 [0,4194304] 0 2024-05-27T09:29:31.860 INFO:tasks.workunit.client.0.smithi119.stdout:5/808: creat d6/dd/d46/d47/d4b/f122 x:0 0 0 2024-05-27T09:29:31.865 INFO:tasks.workunit.client.1.smithi138.stdout:7/205: stat d0/c15 0 2024-05-27T09:29:31.865 INFO:tasks.workunit.client.0.smithi119.stdout:0/712: mkdir d2/d6/d9/d24/d30/dcc/de6 0 2024-05-27T09:29:31.868 INFO:tasks.workunit.client.0.smithi119.stdout:6/729: creat d0/d2/d57/f102 x:0 0 0 2024-05-27T09:29:31.868 INFO:tasks.workunit.client.0.smithi119.stdout:6/730: truncate d0/f1e 572230 0 2024-05-27T09:29:31.871 INFO:tasks.workunit.client.0.smithi119.stdout:6/731: write d0/d2/d4e/dc5/fee [244521,14927] 0 2024-05-27T09:29:31.876 INFO:tasks.workunit.client.0.smithi119.stdout:5/809: rename d6/dd/d39/d74/f107 to d6/db/dd4/f123 0 2024-05-27T09:29:31.877 INFO:tasks.workunit.client.0.smithi119.stdout:5/810: chown d6/dd/f73 996 1 2024-05-27T09:29:31.877 INFO:tasks.workunit.client.0.smithi119.stdout:5/811: truncate d6/db/de/fdc 3453231 0 2024-05-27T09:29:31.877 INFO:tasks.workunit.client.0.smithi119.stdout:5/812: dread - d6/dd/d39/d74/feb zero size 2024-05-27T09:29:31.882 INFO:tasks.workunit.client.0.smithi119.stdout:0/713: rmdir d2/d6/d9/d24/d30/dcc/de3/de4 0 2024-05-27T09:29:31.882 INFO:tasks.workunit.client.0.smithi119.stdout:0/714: write d2/d6/d9/fbb [737554,103246] 0 2024-05-27T09:29:31.883 INFO:tasks.workunit.client.1.smithi138.stdout:7/206: mknod d0/d23/c41 0 2024-05-27T09:29:31.887 INFO:tasks.workunit.client.0.smithi119.stdout:5/813: getdents d6/dd/d39/d19/d70/dc1 0 2024-05-27T09:29:31.887 INFO:tasks.workunit.client.0.smithi119.stdout:5/814: dread - d6/db/de/f10e zero size 2024-05-27T09:29:31.888 INFO:tasks.workunit.client.1.smithi138.stdout:7/207: truncate d0/f21 1939946 0 2024-05-27T09:29:31.888 INFO:tasks.workunit.client.1.smithi138.stdout:7/208: fsync d0/d2b/f35 0 2024-05-27T09:29:31.888 INFO:tasks.workunit.client.1.smithi138.stdout:7/209: read - d0/f1a zero size 2024-05-27T09:29:31.888 INFO:tasks.workunit.client.1.smithi138.stdout:7/210: write d0/f1a [102318,71360] 0 2024-05-27T09:29:31.890 INFO:tasks.workunit.client.1.smithi138.stdout:8/208: dwrite d3/d8/fc [0,4194304] 0 2024-05-27T09:29:31.891 INFO:tasks.workunit.client.0.smithi119.stdout:0/715: rename d2/d6/d9/d12/c3f to d2/d6/d9/d2f/d95/ce7 0 2024-05-27T09:29:31.891 INFO:tasks.workunit.client.0.smithi119.stdout:0/716: write d2/d1a/d20/f74 [400642,91960] 0 2024-05-27T09:29:31.894 INFO:tasks.workunit.client.1.smithi138.stdout:0/248: mknod da/d17/d1d/d1f/d42/d4a/d53/c5a 0 2024-05-27T09:29:31.896 INFO:tasks.workunit.client.1.smithi138.stdout:8/209: rename d3/d8/l17 to d3/d8/d11/l39 0 2024-05-27T09:29:31.899 INFO:tasks.workunit.client.1.smithi138.stdout:2/180: sync 2024-05-27T09:29:31.899 INFO:tasks.workunit.client.1.smithi138.stdout:3/120: sync 2024-05-27T09:29:31.899 INFO:tasks.workunit.client.1.smithi138.stdout:1/254: sync 2024-05-27T09:29:31.899 INFO:tasks.workunit.client.1.smithi138.stdout:9/185: sync 2024-05-27T09:29:31.899 INFO:tasks.workunit.client.1.smithi138.stdout:9/186: fsync d6/db/f10 0 2024-05-27T09:29:31.900 INFO:tasks.workunit.client.1.smithi138.stdout:3/121: creat d0/d8/f29 x:0 0 0 2024-05-27T09:29:31.901 INFO:tasks.workunit.client.1.smithi138.stdout:8/210: mkdir d3/d3a 0 2024-05-27T09:29:31.903 INFO:tasks.workunit.client.1.smithi138.stdout:9/187: creat d6/db/f39 x:0 0 0 2024-05-27T09:29:31.904 INFO:tasks.workunit.client.1.smithi138.stdout:3/122: mkdir d0/d2/da/df/d10/d2a 0 2024-05-27T09:29:31.904 INFO:tasks.workunit.client.1.smithi138.stdout:3/123: write d0/d2/f24 [852941,60682] 0 2024-05-27T09:29:31.905 INFO:tasks.workunit.client.1.smithi138.stdout:8/211: creat d3/d8/d32/f3b x:0 0 0 2024-05-27T09:29:31.907 INFO:tasks.workunit.client.1.smithi138.stdout:3/124: mknod d0/d2/c2b 0 2024-05-27T09:29:31.907 INFO:tasks.workunit.client.1.smithi138.stdout:9/188: creat d6/d25/f3a x:0 0 0 2024-05-27T09:29:31.908 INFO:tasks.workunit.client.1.smithi138.stdout:9/189: write d6/d25/f3a [54723,100584] 0 2024-05-27T09:29:31.908 INFO:tasks.workunit.client.1.smithi138.stdout:9/190: creat d6/f3b x:0 0 0 2024-05-27T09:29:31.908 INFO:tasks.workunit.client.1.smithi138.stdout:9/191: rename d6/db to d6/db/de/d18/d3c 22 2024-05-27T09:29:31.908 INFO:tasks.workunit.client.1.smithi138.stdout:9/192: write d6/db/dc/f1c [223922,77360] 0 2024-05-27T09:29:31.909 INFO:tasks.workunit.client.1.smithi138.stdout:0/249: symlink da/d17/d1d/d1f/d4f/l5b 0 2024-05-27T09:29:31.911 INFO:tasks.workunit.client.0.smithi119.stdout:4/774: dwrite d4/de/d21/d40/fc6 [0,4194304] 0 2024-05-27T09:29:31.911 INFO:tasks.workunit.client.1.smithi138.stdout:3/125: mknod d0/d2/da/df/c2c 0 2024-05-27T09:29:31.911 INFO:tasks.workunit.client.1.smithi138.stdout:3/126: chown d0/d2/f22 117102416 1 2024-05-27T09:29:31.912 INFO:tasks.workunit.client.1.smithi138.stdout:3/127: chown d0/l26 20571525 1 2024-05-27T09:29:31.912 INFO:tasks.workunit.client.0.smithi119.stdout:8/709: dwrite d3/d8/d1b/fa7 [8388608,4194304] 0 2024-05-27T09:29:31.914 INFO:tasks.workunit.client.1.smithi138.stdout:3/128: creat d0/d2/da/f2d x:0 0 0 2024-05-27T09:29:31.921 INFO:tasks.workunit.client.1.smithi138.stdout:6/295: dwrite d3/d19/f22 [0,4194304] 0 2024-05-27T09:29:31.921 INFO:tasks.workunit.client.0.smithi119.stdout:9/624: sync 2024-05-27T09:29:31.926 INFO:tasks.workunit.client.0.smithi119.stdout:2/747: dwrite d2/da/d8a/dec/ff6 [0,4194304] 0 2024-05-27T09:29:31.933 INFO:tasks.workunit.client.0.smithi119.stdout:5/815: rename d6/d2f/d9d to d6/dd/d46/d47/d4b/dd8/d124 0 2024-05-27T09:29:31.935 INFO:tasks.workunit.client.0.smithi119.stdout:0/717: truncate d2/d6/d9/d24/d69/fe1 1578846 0 2024-05-27T09:29:31.936 INFO:tasks.workunit.client.1.smithi138.stdout:3/129: mknod d0/d2/da/df/d10/d2a/c2e 0 2024-05-27T09:29:31.936 INFO:tasks.workunit.client.1.smithi138.stdout:6/296: creat d3/d35/d43/d1c/d32/f6c x:0 0 0 2024-05-27T09:29:31.940 INFO:tasks.workunit.client.1.smithi138.stdout:8/212: dread d3/d8/d28/f2f [0,4194304] 0 2024-05-27T09:29:31.941 INFO:tasks.workunit.client.1.smithi138.stdout:8/213: fsync f1 0 2024-05-27T09:29:31.944 INFO:tasks.workunit.client.0.smithi119.stdout:2/748: write d2/f66 [4292083,42209] 0 2024-05-27T09:29:31.946 INFO:tasks.workunit.client.0.smithi119.stdout:5/816: rename d6/db/de/d27/d77/f8d to d6/db/f125 0 2024-05-27T09:29:31.956 INFO:tasks.workunit.client.1.smithi138.stdout:6/297: creat d3/d11/d2d/d37/f6d x:0 0 0 2024-05-27T09:29:31.956 INFO:tasks.workunit.client.1.smithi138.stdout:6/298: fsync d3/d19/f38 0 2024-05-27T09:29:31.956 INFO:tasks.workunit.client.1.smithi138.stdout:8/214: link d3/d8/d11/l1b d3/d8/d28/l3c 0 2024-05-27T09:29:31.957 INFO:tasks.workunit.client.0.smithi119.stdout:1/781: dwrite d7/fef [4194304,4194304] 0 2024-05-27T09:29:31.958 INFO:tasks.workunit.client.0.smithi119.stdout:0/718: symlink d2/d6/d75/le8 0 2024-05-27T09:29:31.958 INFO:tasks.workunit.client.0.smithi119.stdout:0/719: chown d2/d6/d9/f85 438139 1 2024-05-27T09:29:31.960 INFO:tasks.workunit.client.1.smithi138.stdout:8/215: write d3/d8/d11/f26 [925036,116603] 0 2024-05-27T09:29:31.961 INFO:tasks.workunit.client.1.smithi138.stdout:8/216: dread - d3/d8/d32/f3b zero size 2024-05-27T09:29:31.961 INFO:tasks.workunit.client.0.smithi119.stdout:9/625: link d18/f30 d18/d19/fc2 0 2024-05-27T09:29:31.961 INFO:tasks.workunit.client.0.smithi119.stdout:9/626: readlink d18/d19/l8c 0 2024-05-27T09:29:31.961 INFO:tasks.workunit.client.0.smithi119.stdout:9/627: dread - d18/d19/d1b/f4c zero size 2024-05-27T09:29:31.969 INFO:tasks.workunit.client.0.smithi119.stdout:2/749: creat d2/da/d8a/dd4/d89/db0/ffb x:0 0 0 2024-05-27T09:29:31.994 INFO:tasks.workunit.client.0.smithi119.stdout:6/732: dwrite d0/d2/fb1 [0,4194304] 0 2024-05-27T09:29:31.994 INFO:tasks.workunit.client.0.smithi119.stdout:6/733: dread - d0/d2/d4e/feb zero size 2024-05-27T09:29:31.999 INFO:tasks.workunit.client.1.smithi138.stdout:2/181: dwrite d1/d2f/f33 [0,4194304] 0 2024-05-27T09:29:32.014 INFO:tasks.workunit.client.0.smithi119.stdout:7/795: dwrite d7/df/d19/d1d/f87 [0,4194304] 0 2024-05-27T09:29:32.016 INFO:tasks.workunit.client.1.smithi138.stdout:2/182: truncate d1/d15/d2d/d17/d25/f2a 619080 0 2024-05-27T09:29:32.017 INFO:tasks.workunit.client.1.smithi138.stdout:7/211: dwrite d0/f1a [0,4194304] 0 2024-05-27T09:29:32.019 INFO:tasks.workunit.client.0.smithi119.stdout:5/817: mknod d6/dd/d46/d47/d4b/dd8/d124/c126 0 2024-05-27T09:29:32.019 INFO:tasks.workunit.client.0.smithi119.stdout:5/818: fdatasync d6/f22 0 2024-05-27T09:29:32.019 INFO:tasks.workunit.client.0.smithi119.stdout:5/819: write d6/dd/d46/d47/d4b/f120 [36820,41274] 0 2024-05-27T09:29:32.020 INFO:tasks.workunit.client.1.smithi138.stdout:7/212: creat d0/d23/d3c/f42 x:0 0 0 2024-05-27T09:29:32.020 INFO:tasks.workunit.client.1.smithi138.stdout:7/213: fsync d0/d2b/f35 0 2024-05-27T09:29:32.026 INFO:tasks.workunit.client.0.smithi119.stdout:9/628: truncate d18/d19/d61/fa5 6235302 0 2024-05-27T09:29:32.027 INFO:tasks.workunit.client.0.smithi119.stdout:9/629: fdatasync d18/d19/d1b/d29/fa8 0 2024-05-27T09:29:32.034 INFO:tasks.workunit.client.1.smithi138.stdout:2/183: dread d1/d15/f19 [0,4194304] 0 2024-05-27T09:29:32.035 INFO:tasks.workunit.client.0.smithi119.stdout:3/872: sync 2024-05-27T09:29:32.038 INFO:tasks.workunit.client.1.smithi138.stdout:2/184: rename d1/d15/f19 to d1/d15/d2d/d31/f3b 0 2024-05-27T09:29:32.043 INFO:tasks.workunit.client.1.smithi138.stdout:4/214: sync 2024-05-27T09:29:32.043 INFO:tasks.workunit.client.1.smithi138.stdout:5/205: sync 2024-05-27T09:29:32.043 INFO:tasks.workunit.client.1.smithi138.stdout:5/206: chown d3/d12/f25 3856 1 2024-05-27T09:29:32.046 INFO:tasks.workunit.client.1.smithi138.stdout:2/185: rename d1/d15/d2d/d17/d25/d34 to d1/d10/d3c 0 2024-05-27T09:29:32.047 INFO:tasks.workunit.client.0.smithi119.stdout:4/775: truncate d4/de/d21/d40/fc6 3223513 0 2024-05-27T09:29:32.047 INFO:tasks.workunit.client.0.smithi119.stdout:4/776: write d4/d6/d28/d2f/db8/fcd [832762,28731] 0 2024-05-27T09:29:32.052 INFO:tasks.workunit.client.1.smithi138.stdout:0/250: dwrite da/d17/d1d/d1f/f21 [0,4194304] 0 2024-05-27T09:29:32.052 INFO:tasks.workunit.client.1.smithi138.stdout:0/251: chown da/d2a/d2b/l46 818812 1 2024-05-27T09:29:32.055 INFO:tasks.workunit.client.0.smithi119.stdout:1/782: dwrite d7/db/d52/fac [0,4194304] 0 2024-05-27T09:29:32.056 INFO:tasks.workunit.client.0.smithi119.stdout:1/783: fsync d7/db6/fc6 0 2024-05-27T09:29:32.056 INFO:tasks.workunit.client.0.smithi119.stdout:1/784: fdatasync d7/dd/d44/fa7 0 2024-05-27T09:29:32.058 INFO:tasks.workunit.client.0.smithi119.stdout:2/750: dwrite d2/da/d8a/dd4/f57 [0,4194304] 0 2024-05-27T09:29:32.058 INFO:tasks.workunit.client.0.smithi119.stdout:2/751: chown l0 24 1 2024-05-27T09:29:32.058 INFO:tasks.workunit.client.1.smithi138.stdout:9/193: dwrite d6/f2e [0,4194304] 0 2024-05-27T09:29:32.058 INFO:tasks.workunit.client.1.smithi138.stdout:9/194: stat d6/da/d26 0 2024-05-27T09:29:32.065 INFO:tasks.workunit.client.1.smithi138.stdout:6/299: dwrite d3/d11/d2d/d37/f4a [0,4194304] 0 2024-05-27T09:29:32.080 INFO:tasks.workunit.client.1.smithi138.stdout:9/195: rename d6/db/dc/c33 to d6/da/d26/c3d 0 2024-05-27T09:29:32.080 INFO:tasks.workunit.client.1.smithi138.stdout:0/252: dread da/f15 [0,4194304] 0 2024-05-27T09:29:32.081 INFO:tasks.workunit.client.0.smithi119.stdout:2/752: dread d2/da/f28 [0,4194304] 0 2024-05-27T09:29:32.082 INFO:tasks.workunit.client.0.smithi119.stdout:9/630: dwrite d18/d19/d1b/d29/fac [0,4194304] 0 2024-05-27T09:29:32.082 INFO:tasks.workunit.client.0.smithi119.stdout:9/631: write d18/d19/d1b/d29/f57 [719150,74334] 0 2024-05-27T09:29:32.085 INFO:tasks.workunit.client.1.smithi138.stdout:3/130: dwrite d0/d2/f22 [0,4194304] 0 2024-05-27T09:29:32.085 INFO:tasks.workunit.client.1.smithi138.stdout:3/131: fsync d0/d2/f24 0 2024-05-27T09:29:32.091 INFO:tasks.workunit.client.1.smithi138.stdout:2/186: write d1/d10/f6 [1126573,54686] 0 2024-05-27T09:29:32.091 INFO:tasks.workunit.client.1.smithi138.stdout:2/187: write d1/f1b [3090496,118448] 0 2024-05-27T09:29:32.093 INFO:tasks.workunit.client.1.smithi138.stdout:2/188: write d1/d10/f1a [2058622,67137] 0 2024-05-27T09:29:32.093 INFO:tasks.workunit.client.1.smithi138.stdout:2/189: chown d1/d15/d2d/d17/f11 1 1 2024-05-27T09:29:32.109 INFO:tasks.workunit.client.1.smithi138.stdout:5/207: dwrite d3/f1b [0,4194304] 0 2024-05-27T09:29:32.111 INFO:tasks.workunit.client.1.smithi138.stdout:2/190: creat d1/d15/d2d/d31/f3d x:0 0 0 2024-05-27T09:29:32.122 INFO:tasks.workunit.client.1.smithi138.stdout:4/215: fdatasync d7/d27/f34 0 2024-05-27T09:29:32.130 INFO:tasks.workunit.client.1.smithi138.stdout:1/255: dread d4/d48/d22/f2c [0,4194304] 0 2024-05-27T09:29:32.132 INFO:tasks.workunit.client.1.smithi138.stdout:6/300: write d3/d19/f1e [3689032,23877] 0 2024-05-27T09:29:32.132 INFO:tasks.workunit.client.1.smithi138.stdout:6/301: dread - d3/d35/f60 zero size 2024-05-27T09:29:32.135 INFO:tasks.workunit.client.0.smithi119.stdout:1/785: dwrite d7/f5b [0,4194304] 0 2024-05-27T09:29:32.143 INFO:tasks.workunit.client.1.smithi138.stdout:1/256: write d4/f24 [2333878,40668] 0 2024-05-27T09:29:32.144 INFO:tasks.workunit.client.1.smithi138.stdout:4/216: getdents d7/d27 0 2024-05-27T09:29:32.144 INFO:tasks.workunit.client.1.smithi138.stdout:4/217: write d7/d27/f2c [1012698,128935] 0 2024-05-27T09:29:32.144 INFO:tasks.workunit.client.1.smithi138.stdout:4/218: truncate d7/df/f29 1929647 0 2024-05-27T09:29:32.150 INFO:tasks.workunit.client.1.smithi138.stdout:0/253: dwrite da/d17/f44 [0,4194304] 0 2024-05-27T09:29:32.150 INFO:tasks.workunit.client.1.smithi138.stdout:0/254: write da/d2a/d2b/f4b [494877,22727] 0 2024-05-27T09:29:32.155 INFO:tasks.workunit.client.1.smithi138.stdout:6/302: unlink d3/d19/c1f 0 2024-05-27T09:29:32.158 INFO:tasks.workunit.client.1.smithi138.stdout:4/219: mkdir d7/d27/d47 0 2024-05-27T09:29:32.173 INFO:tasks.workunit.client.1.smithi138.stdout:0/255: creat da/d2a/d55/f5c x:0 0 0 2024-05-27T09:29:32.177 INFO:tasks.workunit.client.1.smithi138.stdout:3/132: dwrite d0/d2/da/f2d [0,4194304] 0 2024-05-27T09:29:32.177 INFO:tasks.workunit.client.0.smithi119.stdout:1/786: dwrite d7/dd/d25/f86 [0,4194304] 0 2024-05-27T09:29:32.178 INFO:tasks.workunit.client.1.smithi138.stdout:6/303: rename d3/d19/l46 to d3/d11/d2d/l6e 0 2024-05-27T09:29:32.179 INFO:tasks.workunit.client.1.smithi138.stdout:3/133: symlink d0/d2/da/l2f 0 2024-05-27T09:29:32.184 INFO:tasks.workunit.client.1.smithi138.stdout:4/220: creat d7/d27/f48 x:0 0 0 2024-05-27T09:29:32.191 INFO:tasks.workunit.client.1.smithi138.stdout:0/256: link da/d17/d1d/d43/l47 da/d17/d1d/d1f/l5d 0 2024-05-27T09:29:32.199 INFO:tasks.workunit.client.1.smithi138.stdout:6/304: creat d3/d35/d5a/d6a/f6f x:0 0 0 2024-05-27T09:29:32.211 INFO:tasks.workunit.client.1.smithi138.stdout:1/257: dwrite d4/da/f2f [0,4194304] 0 2024-05-27T09:29:32.219 INFO:tasks.workunit.client.1.smithi138.stdout:6/305: symlink d3/d35/d43/d1c/l70 0 2024-05-27T09:29:32.220 INFO:tasks.workunit.client.1.smithi138.stdout:1/258: dread d4/da/d4d/f3f [0,4194304] 0 2024-05-27T09:29:32.246 INFO:tasks.workunit.client.1.smithi138.stdout:3/134: dwrite d0/d2/f24 [0,4194304] 0 2024-05-27T09:29:32.246 INFO:tasks.workunit.client.1.smithi138.stdout:3/135: dread - d0/d8/f29 zero size 2024-05-27T09:29:32.246 INFO:tasks.workunit.client.1.smithi138.stdout:3/136: dread - d0/d8/f29 zero size 2024-05-27T09:29:32.246 INFO:tasks.workunit.client.1.smithi138.stdout:3/137: write d0/d2/f27 [232558,96771] 0 2024-05-27T09:29:32.246 INFO:tasks.workunit.client.1.smithi138.stdout:3/138: fdatasync d0/d2/f22 0 2024-05-27T09:29:32.247 INFO:tasks.workunit.client.1.smithi138.stdout:4/221: dwrite d7/f3d [0,4194304] 0 2024-05-27T09:29:32.247 INFO:tasks.workunit.client.1.smithi138.stdout:4/222: write d7/f42 [227153,108916] 0 2024-05-27T09:29:32.249 INFO:tasks.workunit.client.1.smithi138.stdout:1/259: dread d4/d48/f14 [0,4194304] 0 2024-05-27T09:29:32.250 INFO:tasks.workunit.client.1.smithi138.stdout:3/139: rename d0/d2/f24 to d0/f30 0 2024-05-27T09:29:32.251 INFO:tasks.workunit.client.1.smithi138.stdout:3/140: stat d0/d2/da/cb 0 2024-05-27T09:29:32.252 INFO:tasks.workunit.client.1.smithi138.stdout:3/141: mknod d0/d8/c31 0 2024-05-27T09:29:32.252 INFO:tasks.workunit.client.1.smithi138.stdout:3/142: chown d0/d2/c20 18 1 2024-05-27T09:29:32.252 INFO:tasks.workunit.client.1.smithi138.stdout:3/143: truncate d0/d8/f29 746481 0 2024-05-27T09:29:32.268 INFO:tasks.workunit.client.0.smithi119.stdout:5/820: truncate d6/dd/d46/d47/d4b/dd8/ddb/fe5 4171705 0 2024-05-27T09:29:32.268 INFO:tasks.workunit.client.0.smithi119.stdout:5/821: chown d6/dd/d46/d47/d105 56228977 1 2024-05-27T09:29:32.271 INFO:tasks.workunit.client.1.smithi138.stdout:6/306: dwrite d3/d11/d33/f66 [0,4194304] 0 2024-05-27T09:29:32.271 INFO:tasks.workunit.client.1.smithi138.stdout:6/307: chown d3/d35/d43/d1c/c57 9644455 1 2024-05-27T09:29:32.273 INFO:tasks.workunit.client.1.smithi138.stdout:4/223: mknod d7/d27/c49 0 2024-05-27T09:29:32.273 INFO:tasks.workunit.client.1.smithi138.stdout:4/224: fsync d7/f25 0 2024-05-27T09:29:32.282 INFO:tasks.workunit.client.1.smithi138.stdout:1/260: dwrite d4/da/f1f [4194304,4194304] 0 2024-05-27T09:29:32.284 INFO:tasks.workunit.client.1.smithi138.stdout:3/144: rename d0/d8 to d0/d2/da/df/d10/d32 0 2024-05-27T09:29:32.284 INFO:tasks.workunit.client.1.smithi138.stdout:3/145: fdatasync d0/d2/da/df/d10/d32/f1d 0 2024-05-27T09:29:32.288 INFO:tasks.workunit.client.1.smithi138.stdout:4/225: symlink d7/d27/d47/l4a 0 2024-05-27T09:29:32.297 INFO:tasks.workunit.client.1.smithi138.stdout:3/146: dread d0/d2/da/df/d10/d32/f9 [0,4194304] 0 2024-05-27T09:29:32.297 INFO:tasks.workunit.client.1.smithi138.stdout:3/147: chown d0/l26 86 1 2024-05-27T09:29:32.297 INFO:tasks.workunit.client.1.smithi138.stdout:3/148: fdatasync d0/f30 0 2024-05-27T09:29:32.297 INFO:tasks.workunit.client.1.smithi138.stdout:3/149: creat d0/d2/da/f33 x:0 0 0 2024-05-27T09:29:32.308 INFO:tasks.workunit.client.0.smithi119.stdout:9/632: stat d18/d19/d1b/d40/d44/l8d 0 2024-05-27T09:29:32.309 INFO:tasks.workunit.client.1.smithi138.stdout:3/150: creat d0/d2/da/df/d10/d32/f34 x:0 0 0 2024-05-27T09:29:32.309 INFO:tasks.workunit.client.1.smithi138.stdout:3/151: fdatasync d0/d2/da/df/d10/d32/f34 0 2024-05-27T09:29:32.310 INFO:tasks.workunit.client.0.smithi119.stdout:9/633: write d18/f23 [1895691,9969] 0 2024-05-27T09:29:32.312 INFO:tasks.workunit.client.1.smithi138.stdout:3/152: link d0/c18 d0/d2/c35 0 2024-05-27T09:29:32.312 INFO:tasks.workunit.client.1.smithi138.stdout:3/153: chown d0/c18 4 1 2024-05-27T09:29:32.312 INFO:tasks.workunit.client.1.smithi138.stdout:3/154: truncate d0/d2/fc 4276367 0 2024-05-27T09:29:32.314 INFO:tasks.workunit.client.1.smithi138.stdout:4/226: symlink d7/df/l4b 0 2024-05-27T09:29:32.321 INFO:tasks.workunit.client.1.smithi138.stdout:5/208: creat d3/d12/d16/d1d/f44 x:0 0 0 2024-05-27T09:29:32.321 INFO:tasks.workunit.client.1.smithi138.stdout:4/227: rmdir d7/df/d35 39 2024-05-27T09:29:32.321 INFO:tasks.workunit.client.1.smithi138.stdout:4/228: write d7/d27/f31 [3852591,44176] 0 2024-05-27T09:29:32.321 INFO:tasks.workunit.client.1.smithi138.stdout:3/155: rename d0/d2/da/df/d10/d32/c31 to d0/d2/da/c36 0 2024-05-27T09:29:32.322 INFO:tasks.workunit.client.1.smithi138.stdout:3/156: truncate d0/d2/da/df/d10/d32/f28 733891 0 2024-05-27T09:29:32.324 INFO:tasks.workunit.client.1.smithi138.stdout:3/157: unlink d0/d2/da/df/d10/d32/f34 0 2024-05-27T09:29:32.324 INFO:tasks.workunit.client.1.smithi138.stdout:3/158: fsync d0/d2/da/f1b 0 2024-05-27T09:29:32.325 INFO:tasks.workunit.client.1.smithi138.stdout:4/229: mkdir d7/df/d35/d4c 0 2024-05-27T09:29:32.326 INFO:tasks.workunit.client.0.smithi119.stdout:2/753: write d2/da/d2c/f38 [3276,17943] 0 2024-05-27T09:29:32.326 INFO:tasks.workunit.client.1.smithi138.stdout:5/209: dread d3/f21 [0,4194304] 0 2024-05-27T09:29:32.326 INFO:tasks.workunit.client.1.smithi138.stdout:4/230: write d7/d27/f48 [72406,69496] 0 2024-05-27T09:29:32.326 INFO:tasks.workunit.client.1.smithi138.stdout:4/231: stat d7/d27/c38 0 2024-05-27T09:29:32.326 INFO:tasks.workunit.client.1.smithi138.stdout:4/232: write d7/f26 [367682,37707] 0 2024-05-27T09:29:32.326 INFO:tasks.workunit.client.1.smithi138.stdout:4/233: fdatasync d7/f23 0 2024-05-27T09:29:32.327 INFO:tasks.workunit.client.1.smithi138.stdout:3/159: mkdir d0/d2/da/df/d10/d32/d37 0 2024-05-27T09:29:32.330 INFO:tasks.workunit.client.0.smithi119.stdout:5/822: dwrite d6/db/de/d27/f108 [0,4194304] 0 2024-05-27T09:29:32.331 INFO:tasks.workunit.client.0.smithi119.stdout:4/777: unlink d4/de/d21/f85 0 2024-05-27T09:29:32.331 INFO:tasks.workunit.client.1.smithi138.stdout:1/261: mkdir d4/d3c/d51 0 2024-05-27T09:29:32.334 INFO:tasks.workunit.client.1.smithi138.stdout:5/210: creat d3/d12/d26/f45 x:0 0 0 2024-05-27T09:29:32.334 INFO:tasks.workunit.client.1.smithi138.stdout:5/211: rename d3/d12/d16/d1d/d22 to d3/d12/d16/d1d/d22/d2b/d46 22 2024-05-27T09:29:32.342 INFO:tasks.workunit.client.0.smithi119.stdout:1/787: mkdir d7/dd/db0/dcd/df2 0 2024-05-27T09:29:32.348 INFO:tasks.workunit.client.0.smithi119.stdout:9/634: symlink d18/d19/d1b/d2a/d63/lc3 0 2024-05-27T09:29:32.351 INFO:tasks.workunit.client.0.smithi119.stdout:0/720: link d2/d6/d9/d24/d30/d4a/fc5 d2/d6/d9/d2f/fe9 0 2024-05-27T09:29:32.354 INFO:tasks.workunit.client.1.smithi138.stdout:4/234: write d7/f3d [3214195,76385] 0 2024-05-27T09:29:32.354 INFO:tasks.workunit.client.1.smithi138.stdout:6/308: dwrite d3/d35/d43/f29 [0,4194304] 0 2024-05-27T09:29:32.359 INFO:tasks.workunit.client.0.smithi119.stdout:6/734: rename d0/d85/caf to d0/d2/d57/d37/dad/c103 0 2024-05-27T09:29:32.359 INFO:tasks.workunit.client.0.smithi119.stdout:6/735: fsync d0/d19/fb0 0 2024-05-27T09:29:32.360 INFO:tasks.workunit.client.0.smithi119.stdout:2/754: truncate d2/da/d15/f18 3470167 0 2024-05-27T09:29:32.365 INFO:tasks.workunit.client.0.smithi119.stdout:5/823: mkdir d6/db/de/d27/dc9/d127 0 2024-05-27T09:29:32.370 INFO:tasks.workunit.client.1.smithi138.stdout:6/309: mkdir d3/d35/d43/d71 0 2024-05-27T09:29:32.371 INFO:tasks.workunit.client.0.smithi119.stdout:4/778: truncate d4/d6/fd 65172 0 2024-05-27T09:29:32.378 INFO:tasks.workunit.client.0.smithi119.stdout:1/788: creat d7/db/d12/ddb/d38/d3c/ff3 x:0 0 0 2024-05-27T09:29:32.378 INFO:tasks.workunit.client.0.smithi119.stdout:9/635: mkdir d18/d19/d1b/d29/dc4 0 2024-05-27T09:29:32.384 INFO:tasks.workunit.client.1.smithi138.stdout:6/310: rename d3/d11/d2d/d37/c3b to d3/d35/d43/d1c/d32/d47/c72 0 2024-05-27T09:29:32.385 INFO:tasks.workunit.client.0.smithi119.stdout:3/873: rename db/d36/d45/d4c/d62/d13/d33/d9b/de3/c113 to db/dc9/d98/d105/c11a 0 2024-05-27T09:29:32.386 INFO:tasks.workunit.client.0.smithi119.stdout:6/736: link d0/d2/d1c/d21/d58/fc1 d0/d2/d4e/f104 0 2024-05-27T09:29:32.387 INFO:tasks.workunit.client.0.smithi119.stdout:9/636: dread d18/d19/d1b/d29/f57 [0,4194304] 0 2024-05-27T09:29:32.393 INFO:tasks.workunit.client.0.smithi119.stdout:2/755: symlink d2/da/d8a/dd4/d89/d94/d87/dcf/de5/lfc 0 2024-05-27T09:29:32.402 INFO:tasks.workunit.client.1.smithi138.stdout:6/311: rename d3/d11/c4e to d3/d35/d43/c73 0 2024-05-27T09:29:32.404 INFO:tasks.workunit.client.0.smithi119.stdout:5/824: mknod d6/dd/d46/dba/c128 0 2024-05-27T09:29:32.404 INFO:tasks.workunit.client.0.smithi119.stdout:5/825: write d6/f63 [109140,69472] 0 2024-05-27T09:29:32.407 INFO:tasks.workunit.client.1.smithi138.stdout:6/312: mkdir d3/d35/d74 0 2024-05-27T09:29:32.410 INFO:tasks.workunit.client.1.smithi138.stdout:3/160: dwrite d0/d2/fc [0,4194304] 0 2024-05-27T09:29:32.410 INFO:tasks.workunit.client.0.smithi119.stdout:4/779: mknod d4/de/de8/cf2 0 2024-05-27T09:29:32.413 INFO:tasks.workunit.client.1.smithi138.stdout:6/313: creat d3/d11/d2d/f75 x:0 0 0 2024-05-27T09:29:32.413 INFO:tasks.workunit.client.1.smithi138.stdout:3/161: rename d0/f30 to d0/d2/da/df/d10/f38 0 2024-05-27T09:29:32.417 INFO:tasks.workunit.client.1.smithi138.stdout:3/162: link d0/d2/da/df/l1a d0/l39 0 2024-05-27T09:29:32.420 INFO:tasks.workunit.client.0.smithi119.stdout:1/789: creat d7/db/d52/d65/ff4 x:0 0 0 2024-05-27T09:29:32.424 INFO:tasks.workunit.client.0.smithi119.stdout:0/721: rename d2/d6/d9/f1c to d2/d6/d23/db8/fea 0 2024-05-27T09:29:32.424 INFO:tasks.workunit.client.0.smithi119.stdout:0/722: chown d2/d6/d9/d24/d30/d4a 227 1 2024-05-27T09:29:32.424 INFO:tasks.workunit.client.0.smithi119.stdout:0/723: dread - d2/d6/d75/fb3 zero size 2024-05-27T09:29:32.425 INFO:tasks.workunit.client.0.smithi119.stdout:3/874: mknod db/d7f/dd2/c11b 0 2024-05-27T09:29:32.429 INFO:tasks.workunit.client.0.smithi119.stdout:6/737: symlink d0/d2/d57/d37/d74/l105 0 2024-05-27T09:29:32.429 INFO:tasks.workunit.client.0.smithi119.stdout:6/738: chown d0/d2/d1c/d21/lb8 47084 1 2024-05-27T09:29:32.430 INFO:tasks.workunit.client.0.smithi119.stdout:9/637: getdents d18 0 2024-05-27T09:29:32.430 INFO:tasks.workunit.client.0.smithi119.stdout:9/638: chown d18/d19/d1b/f96 0 1 2024-05-27T09:29:32.430 INFO:tasks.workunit.client.0.smithi119.stdout:9/639: fdatasync d18/d19/f74 0 2024-05-27T09:29:32.431 INFO:tasks.workunit.client.0.smithi119.stdout:2/756: rename d2/da/d8a/dd4/d89/d94/f68 to d2/da/d8a/dd4/d89/d94/ffd 0 2024-05-27T09:29:32.436 INFO:tasks.workunit.client.1.smithi138.stdout:4/235: dwrite d7/f44 [0,4194304] 0 2024-05-27T09:29:32.436 INFO:tasks.workunit.client.0.smithi119.stdout:5/826: truncate d6/db/de/d27/d77/fbc 145602 0 2024-05-27T09:29:32.436 INFO:tasks.workunit.client.0.smithi119.stdout:4/780: truncate d4/d6/fd1 2283376 0 2024-05-27T09:29:32.440 INFO:tasks.workunit.client.0.smithi119.stdout:1/790: creat d7/dd/dcb/ff5 x:0 0 0 2024-05-27T09:29:32.442 INFO:tasks.workunit.client.1.smithi138.stdout:3/163: dread d0/d2/da/df/d10/d32/f1d [0,4194304] 0 2024-05-27T09:29:32.444 INFO:tasks.workunit.client.1.smithi138.stdout:3/164: symlink d0/l3a 0 2024-05-27T09:29:32.444 INFO:tasks.workunit.client.0.smithi119.stdout:0/724: mkdir d2/d6/d9/d12/d4b/deb 0 2024-05-27T09:29:32.449 INFO:tasks.workunit.client.1.smithi138.stdout:1/262: dwrite d4/d48/f14 [4194304,4194304] 0 2024-05-27T09:29:32.449 INFO:tasks.workunit.client.1.smithi138.stdout:1/263: chown d4/c6 101 1 2024-05-27T09:29:32.449 INFO:tasks.workunit.client.1.smithi138.stdout:1/264: chown d4/d48/d22/f35 7731096 1 2024-05-27T09:29:32.451 INFO:tasks.workunit.client.0.smithi119.stdout:3/875: unlink db/d36/d45/d4c/dbe/f115 0 2024-05-27T09:29:32.452 INFO:tasks.workunit.client.0.smithi119.stdout:3/876: readlink db/d7f/da9/le0 0 2024-05-27T09:29:32.462 INFO:tasks.workunit.client.0.smithi119.stdout:1/791: write d7/db/d35/fa0 [1935650,38574] 0 2024-05-27T09:29:32.462 INFO:tasks.workunit.client.0.smithi119.stdout:1/792: read - d7/db/d12/d15/dd2/fdf zero size 2024-05-27T09:29:32.466 INFO:tasks.workunit.client.1.smithi138.stdout:4/236: link d7/d27/f34 d7/df/f4d 0 2024-05-27T09:29:32.469 INFO:tasks.workunit.client.0.smithi119.stdout:9/640: getdents d18/d19/d61 0 2024-05-27T09:29:32.471 INFO:tasks.workunit.client.1.smithi138.stdout:6/314: dwrite d3/d11/d2d/d37/f4a [0,4194304] 0 2024-05-27T09:29:32.471 INFO:tasks.workunit.client.1.smithi138.stdout:6/315: dread - d3/d19/f38 zero size 2024-05-27T09:29:32.472 INFO:tasks.workunit.client.1.smithi138.stdout:3/165: dread d0/d2/da/f2d [0,4194304] 0 2024-05-27T09:29:32.475 INFO:tasks.workunit.client.0.smithi119.stdout:4/781: mknod d4/d6/d3e/cf3 0 2024-05-27T09:29:32.476 INFO:tasks.workunit.client.1.smithi138.stdout:3/166: rename d0/d2/l23 to d0/d2/da/df/d10/d32/d37/l3b 0 2024-05-27T09:29:32.480 INFO:tasks.workunit.client.1.smithi138.stdout:4/237: link d7/d27/c49 d7/c4e 0 2024-05-27T09:29:32.483 INFO:tasks.workunit.client.0.smithi119.stdout:8/710: sync 2024-05-27T09:29:32.483 INFO:tasks.workunit.client.0.smithi119.stdout:7/796: sync 2024-05-27T09:29:32.483 INFO:tasks.workunit.client.0.smithi119.stdout:3/877: mkdir db/d7f/dd2/d11c 0 2024-05-27T09:29:32.483 INFO:tasks.workunit.client.0.smithi119.stdout:3/878: stat db 0 2024-05-27T09:29:32.483 INFO:tasks.workunit.client.0.smithi119.stdout:3/879: dread - db/d36/d45/fe7 zero size 2024-05-27T09:29:32.486 INFO:tasks.workunit.client.1.smithi138.stdout:4/238: symlink d7/d27/d47/l4f 0 2024-05-27T09:29:32.499 INFO:tasks.workunit.client.1.smithi138.stdout:5/212: dwrite f0 [4194304,4194304] 0 2024-05-27T09:29:32.500 INFO:tasks.workunit.client.1.smithi138.stdout:6/316: dread d3/fa [0,4194304] 0 2024-05-27T09:29:32.501 INFO:tasks.workunit.client.0.smithi119.stdout:8/711: dread d3/d8/d4d/d5d/d60/f71 [4194304,4194304] 0 2024-05-27T09:29:32.508 INFO:tasks.workunit.client.0.smithi119.stdout:9/641: creat d18/d19/d1b/d40/fc5 x:0 0 0 2024-05-27T09:29:32.510 INFO:tasks.workunit.client.1.smithi138.stdout:4/239: dread d7/f26 [0,4194304] 0 2024-05-27T09:29:32.518 INFO:tasks.workunit.client.1.smithi138.stdout:5/213: creat d3/d12/d32/f47 x:0 0 0 2024-05-27T09:29:32.523 INFO:tasks.workunit.client.0.smithi119.stdout:0/725: symlink d2/d6/d23/d7a/lec 0 2024-05-27T09:29:32.523 INFO:tasks.workunit.client.0.smithi119.stdout:0/726: creat d2/d6/d9/d12/fed x:0 0 0 2024-05-27T09:29:32.524 INFO:tasks.workunit.client.0.smithi119.stdout:0/727: dread - d2/d6/d9/d12/da3/fd3 zero size 2024-05-27T09:29:32.526 INFO:tasks.workunit.client.0.smithi119.stdout:3/880: link db/d36/d45/d4d/d108/f109 db/d36/d45/dab/f11d 0 2024-05-27T09:29:32.528 INFO:tasks.workunit.client.1.smithi138.stdout:1/265: dwrite d4/d38/f44 [0,4194304] 0 2024-05-27T09:29:32.530 INFO:tasks.workunit.client.0.smithi119.stdout:2/757: getdents d2/da/d8a/dd4/d89/d94/d87/dcf/de5 0 2024-05-27T09:29:32.530 INFO:tasks.workunit.client.0.smithi119.stdout:2/758: fsync d2/d1a/f3d 0 2024-05-27T09:29:32.530 INFO:tasks.workunit.client.0.smithi119.stdout:2/759: write d2/da/d15/d53/fb6 [331126,20664] 0 2024-05-27T09:29:32.530 INFO:tasks.workunit.client.0.smithi119.stdout:2/760: write d2/f66 [6361617,23645] 0 2024-05-27T09:29:32.535 INFO:tasks.workunit.client.0.smithi119.stdout:4/782: dwrite d4/de/d21/d40/d47/d5c/d9e/fcb [0,4194304] 0 2024-05-27T09:29:32.535 INFO:tasks.workunit.client.0.smithi119.stdout:4/783: dread - d4/de/d21/fec zero size 2024-05-27T09:29:32.537 INFO:tasks.workunit.client.0.smithi119.stdout:3/881: fsync db/d36/d45/d4c/d62/f38 0 2024-05-27T09:29:32.541 INFO:tasks.workunit.client.1.smithi138.stdout:6/317: symlink d3/d35/d43/d1c/d32/d47/l76 0 2024-05-27T09:29:32.541 INFO:tasks.workunit.client.1.smithi138.stdout:6/318: fdatasync d3/f5 0 2024-05-27T09:29:32.545 INFO:tasks.workunit.client.0.smithi119.stdout:4/784: rename d4/d6/f86 to d4/db1/ff4 0 2024-05-27T09:29:32.548 INFO:tasks.workunit.client.0.smithi119.stdout:2/761: write d2/da/d6f/f7f [2122410,2712] 0 2024-05-27T09:29:32.551 INFO:tasks.workunit.client.0.smithi119.stdout:3/882: creat db/d36/d45/d4c/d62/f11e x:0 0 0 2024-05-27T09:29:32.555 INFO:tasks.workunit.client.0.smithi119.stdout:2/762: symlink d2/da/lfe 0 2024-05-27T09:29:32.557 INFO:tasks.workunit.client.0.smithi119.stdout:2/763: fdatasync d2/da/d8a/dd4/fb5 0 2024-05-27T09:29:32.559 INFO:tasks.workunit.client.1.smithi138.stdout:1/266: mknod d4/d1a/c52 0 2024-05-27T09:29:32.565 INFO:tasks.workunit.client.0.smithi119.stdout:2/764: creat d2/da/d15/d53/fff x:0 0 0 2024-05-27T09:29:32.568 INFO:tasks.workunit.client.1.smithi138.stdout:6/319: mkdir d3/d19/d77 0 2024-05-27T09:29:32.568 INFO:tasks.workunit.client.1.smithi138.stdout:6/320: dread - d3/d11/d2d/f39 zero size 2024-05-27T09:29:32.570 INFO:tasks.workunit.client.1.smithi138.stdout:4/240: truncate d7/f44 1982055 0 2024-05-27T09:29:32.574 INFO:tasks.workunit.client.1.smithi138.stdout:6/321: creat d3/d11/d33/f78 x:0 0 0 2024-05-27T09:29:32.575 INFO:tasks.workunit.client.1.smithi138.stdout:4/241: mknod d7/df/d35/d4c/c50 0 2024-05-27T09:29:32.576 INFO:tasks.workunit.client.0.smithi119.stdout:2/765: symlink d2/deb/l100 0 2024-05-27T09:29:32.580 INFO:tasks.workunit.client.1.smithi138.stdout:6/322: symlink d3/d35/d5a/l79 0 2024-05-27T09:29:32.581 INFO:tasks.workunit.client.1.smithi138.stdout:6/323: stat d3/d19/c2f 0 2024-05-27T09:29:32.581 INFO:tasks.workunit.client.1.smithi138.stdout:6/324: write d3/d19/f38 [767943,35114] 0 2024-05-27T09:29:32.581 INFO:tasks.workunit.client.1.smithi138.stdout:6/325: write d3/d35/d5a/d6a/f6b [152943,83484] 0 2024-05-27T09:29:32.581 INFO:tasks.workunit.client.1.smithi138.stdout:4/242: mknod d7/c51 0 2024-05-27T09:29:32.589 INFO:tasks.workunit.client.1.smithi138.stdout:4/243: rmdir d7/df/d35 39 2024-05-27T09:29:32.600 INFO:tasks.workunit.client.1.smithi138.stdout:4/244: write d7/df/f1a [1854152,116841] 0 2024-05-27T09:29:32.605 INFO:tasks.workunit.client.1.smithi138.stdout:9/196: sync 2024-05-27T09:29:32.605 INFO:tasks.workunit.client.1.smithi138.stdout:2/191: sync 2024-05-27T09:29:32.605 INFO:tasks.workunit.client.1.smithi138.stdout:9/197: dread - d6/f3b zero size 2024-05-27T09:29:32.605 INFO:tasks.workunit.client.1.smithi138.stdout:7/214: sync 2024-05-27T09:29:32.605 INFO:tasks.workunit.client.1.smithi138.stdout:7/215: readlink d0/d23/d3a/l3e 0 2024-05-27T09:29:32.605 INFO:tasks.workunit.client.1.smithi138.stdout:8/217: sync 2024-05-27T09:29:32.606 INFO:tasks.workunit.client.0.smithi119.stdout:2/766: dread d2/da/d8a/dd4/d89/d94/f56 [0,4194304] 0 2024-05-27T09:29:32.606 INFO:tasks.workunit.client.1.smithi138.stdout:0/257: sync 2024-05-27T09:29:32.606 INFO:tasks.workunit.client.1.smithi138.stdout:0/258: fdatasync da/d2a/d2b/f50 0 2024-05-27T09:29:32.607 INFO:tasks.workunit.client.1.smithi138.stdout:0/259: readlink da/d17/d1d/d1f/l5d 0 2024-05-27T09:29:32.607 INFO:tasks.workunit.client.1.smithi138.stdout:2/192: stat d1/d15/d2d/d17/d25/f2b 0 2024-05-27T09:29:32.607 INFO:tasks.workunit.client.1.smithi138.stdout:0/260: write da/d17/d1d/d43/f48 [4245900,28186] 0 2024-05-27T09:29:32.607 INFO:tasks.workunit.client.1.smithi138.stdout:9/198: unlink d6/da/d26/d28/f38 0 2024-05-27T09:29:32.615 INFO:tasks.workunit.client.1.smithi138.stdout:7/216: mkdir d0/d7/d43 0 2024-05-27T09:29:32.616 INFO:tasks.workunit.client.1.smithi138.stdout:7/217: dread - d0/d2b/f40 zero size 2024-05-27T09:29:32.618 INFO:tasks.workunit.client.0.smithi119.stdout:7/797: dwrite d7/df/d19/d1d/f87 [0,4194304] 0 2024-05-27T09:29:32.619 INFO:tasks.workunit.client.1.smithi138.stdout:5/214: dwrite d3/d12/d26/f2f [0,4194304] 0 2024-05-27T09:29:32.620 INFO:tasks.workunit.client.1.smithi138.stdout:5/215: chown d3/d12/d26 119371 1 2024-05-27T09:29:32.620 INFO:tasks.workunit.client.1.smithi138.stdout:5/216: write d3/f10 [5178892,89029] 0 2024-05-27T09:29:32.621 INFO:tasks.workunit.client.1.smithi138.stdout:5/217: fdatasync d3/fd 0 2024-05-27T09:29:32.624 INFO:tasks.workunit.client.1.smithi138.stdout:8/218: rename d3/d8/d11/l39 to d3/l3d 0 2024-05-27T09:29:32.627 INFO:tasks.workunit.client.0.smithi119.stdout:2/767: rename d2/f83 to d2/da/d8a/dd4/d89/db0/f101 0 2024-05-27T09:29:32.627 INFO:tasks.workunit.client.0.smithi119.stdout:2/768: fdatasync d2/da/d2c/f55 0 2024-05-27T09:29:32.627 INFO:tasks.workunit.client.0.smithi119.stdout:2/769: fdatasync d2/da/d8a/dd4/fdb 0 2024-05-27T09:29:32.636 INFO:tasks.workunit.client.0.smithi119.stdout:4/785: read d4/d1c/d39/f4b [1609203,58172] 0 2024-05-27T09:29:32.636 INFO:tasks.workunit.client.0.smithi119.stdout:4/786: fdatasync d4/d6/f7d 0 2024-05-27T09:29:32.637 INFO:tasks.workunit.client.1.smithi138.stdout:1/267: dwrite d4/d3c/f4c [4194304,4194304] 0 2024-05-27T09:29:32.637 INFO:tasks.workunit.client.1.smithi138.stdout:2/193: dread d1/d10/f1a [0,4194304] 0 2024-05-27T09:29:32.641 INFO:tasks.workunit.client.1.smithi138.stdout:2/194: fsync d1/d15/d2d/d17/d25/f2a 0 2024-05-27T09:29:32.641 INFO:tasks.workunit.client.1.smithi138.stdout:1/268: write d4/da/f50 [248187,95447] 0 2024-05-27T09:29:32.641 INFO:tasks.workunit.client.1.smithi138.stdout:1/269: chown d4/d3c/f4e 3112 1 2024-05-27T09:29:32.641 INFO:tasks.workunit.client.1.smithi138.stdout:1/270: chown d4/d1a/c52 2325114 1 2024-05-27T09:29:32.641 INFO:tasks.workunit.client.1.smithi138.stdout:7/218: truncate d0/d7/fe 493090 0 2024-05-27T09:29:32.641 INFO:tasks.workunit.client.1.smithi138.stdout:2/195: creat d1/d15/d2d/f3e x:0 0 0 2024-05-27T09:29:32.642 INFO:tasks.workunit.client.0.smithi119.stdout:2/770: symlink d2/deb/l102 0 2024-05-27T09:29:32.644 INFO:tasks.workunit.client.1.smithi138.stdout:2/196: write d1/d15/d2d/d31/f3b [740940,122094] 0 2024-05-27T09:29:32.644 INFO:tasks.workunit.client.1.smithi138.stdout:4/245: fsync d7/f39 0 2024-05-27T09:29:32.646 INFO:tasks.workunit.client.0.smithi119.stdout:4/787: mkdir d4/d6/d3e/df5 0 2024-05-27T09:29:32.646 INFO:tasks.workunit.client.0.smithi119.stdout:4/788: stat d4/d6/d3e/ca7 0 2024-05-27T09:29:32.649 INFO:tasks.workunit.client.1.smithi138.stdout:5/218: symlink d3/d12/d26/l48 0 2024-05-27T09:29:32.653 INFO:tasks.workunit.client.0.smithi119.stdout:2/771: symlink d2/da/d6f/l103 0 2024-05-27T09:29:32.655 INFO:tasks.workunit.client.0.smithi119.stdout:0/728: dwrite d2/d6/d9/d2f/d95/faf [0,4194304] 0 2024-05-27T09:29:32.659 INFO:tasks.workunit.client.0.smithi119.stdout:4/789: rename d4/de/d9b/l9d to d4/d6/d28/lf6 0 2024-05-27T09:29:32.661 INFO:tasks.workunit.client.1.smithi138.stdout:4/246: rmdir d7/df 39 2024-05-27T09:29:32.665 INFO:tasks.workunit.client.0.smithi119.stdout:4/790: creat d4/d23/d24/d5b/da8/dda/ddb/ff7 x:0 0 0 2024-05-27T09:29:32.665 INFO:tasks.workunit.client.0.smithi119.stdout:4/791: readlink d4/de/d21/d40/d47/d5c/d9e/lc7 0 2024-05-27T09:29:32.667 INFO:tasks.workunit.client.0.smithi119.stdout:2/772: rename d2/da/d2c/l78 to d2/da/d8a/dd4/d89/de8/l104 0 2024-05-27T09:29:32.667 INFO:tasks.workunit.client.0.smithi119.stdout:2/773: creat d2/da/d8a/d8f/f105 x:0 0 0 2024-05-27T09:29:32.668 INFO:tasks.workunit.client.0.smithi119.stdout:2/774: chown d2/d1a/f46 728 1 2024-05-27T09:29:32.668 INFO:tasks.workunit.client.0.smithi119.stdout:2/775: creat d2/deb/f106 x:0 0 0 2024-05-27T09:29:32.668 INFO:tasks.workunit.client.0.smithi119.stdout:9/642: dwrite d18/d19/d1b/d29/f71 [0,4194304] 0 2024-05-27T09:29:32.668 INFO:tasks.workunit.client.0.smithi119.stdout:9/643: fdatasync d18/d19/d1b/d40/db7/fb8 0 2024-05-27T09:29:32.669 INFO:tasks.workunit.client.0.smithi119.stdout:7/798: dwrite d7/df/d19/d97/d92/db7/dba/f61 [0,4194304] 0 2024-05-27T09:29:32.671 INFO:tasks.workunit.client.0.smithi119.stdout:4/792: creat d4/d6/d28/ff8 x:0 0 0 2024-05-27T09:29:32.671 INFO:tasks.workunit.client.1.smithi138.stdout:4/247: rename d7/d27/f31 to d7/df/f52 0 2024-05-27T09:29:32.673 INFO:tasks.workunit.client.1.smithi138.stdout:6/326: dwrite d3/d11/f2e [0,4194304] 0 2024-05-27T09:29:32.673 INFO:tasks.workunit.client.1.smithi138.stdout:6/327: chown d3/d19/f1e 28 1 2024-05-27T09:29:32.677 INFO:tasks.workunit.client.0.smithi119.stdout:7/799: dread d7/df/d19/d97/d92/db7/dba/d42/d76/f82 [0,4194304] 0 2024-05-27T09:29:32.678 INFO:tasks.workunit.client.1.smithi138.stdout:4/248: symlink d7/d27/d47/l53 0 2024-05-27T09:29:32.682 INFO:tasks.workunit.client.0.smithi119.stdout:4/793: dread d4/d1c/d39/f4b [0,4194304] 0 2024-05-27T09:29:32.683 INFO:tasks.workunit.client.1.smithi138.stdout:6/328: link d3/d35/d43/f48 d3/d35/d43/d1c/d32/d47/f7a 0 2024-05-27T09:29:32.689 INFO:tasks.workunit.client.1.smithi138.stdout:6/329: rmdir d3/d11 39 2024-05-27T09:29:32.695 INFO:tasks.workunit.client.0.smithi119.stdout:4/794: rename d4/d6/d81/cb9 to d4/d1c/d39/cf9 0 2024-05-27T09:29:32.696 INFO:tasks.workunit.client.1.smithi138.stdout:6/330: rmdir d3/d11/d2d/d37 39 2024-05-27T09:29:32.698 INFO:tasks.workunit.client.0.smithi119.stdout:9/644: rmdir d18/d19/d9b 39 2024-05-27T09:29:32.787 INFO:tasks.workunit.client.0.smithi119.stdout:3/883: dwrite db/f23 [0,4194304] 0 2024-05-27T09:29:32.796 INFO:tasks.workunit.client.1.smithi138.stdout:5/219: truncate d3/f10 4041793 0 2024-05-27T09:29:32.796 INFO:tasks.workunit.client.1.smithi138.stdout:5/220: symlink d3/d12/d32/l49 0 2024-05-27T09:29:32.799 INFO:tasks.workunit.client.1.smithi138.stdout:7/219: dwrite d0/f36 [0,4194304] 0 2024-05-27T09:29:32.820 INFO:tasks.workunit.client.0.smithi119.stdout:7/800: dwrite d7/df/d19/d46/d93/ff4 [0,4194304] 0 2024-05-27T09:29:32.825 INFO:tasks.workunit.client.0.smithi119.stdout:0/729: dwrite d2/d6/d75/fa2 [0,4194304] 0 2024-05-27T09:29:32.827 INFO:tasks.workunit.client.0.smithi119.stdout:7/801: mkdir d7/d1a/d99/dbb/dc2/d102 0 2024-05-27T09:29:32.831 INFO:tasks.workunit.client.0.smithi119.stdout:0/730: symlink d2/lee 0 2024-05-27T09:29:32.833 INFO:tasks.workunit.client.0.smithi119.stdout:4/795: dwrite d4/d6/d3e/d89/db2/fbd [0,4194304] 0 2024-05-27T09:29:32.833 INFO:tasks.workunit.client.0.smithi119.stdout:4/796: write d4/de/d21/d40/d47/fed [38331,78146] 0 2024-05-27T09:29:32.834 INFO:tasks.workunit.client.1.smithi138.stdout:0/261: dwrite da/d2a/d3f/f4c [0,4194304] 0 2024-05-27T09:29:32.834 INFO:tasks.workunit.client.1.smithi138.stdout:6/331: dwrite d3/d11/f4f [0,4194304] 0 2024-05-27T09:29:32.835 INFO:tasks.workunit.client.0.smithi119.stdout:0/731: creat d2/d6/d9/d12/d4b/deb/fef x:0 0 0 2024-05-27T09:29:32.842 INFO:tasks.workunit.client.1.smithi138.stdout:9/199: dwrite d6/db/de/f2b [4194304,4194304] 0 2024-05-27T09:29:32.842 INFO:tasks.workunit.client.1.smithi138.stdout:5/221: write d3/d12/d16/f3c [4087911,117828] 0 2024-05-27T09:29:32.842 INFO:tasks.workunit.client.0.smithi119.stdout:0/732: dread d2/d6/d9/d24/fc8 [0,4194304] 0 2024-05-27T09:29:32.844 INFO:tasks.workunit.client.1.smithi138.stdout:9/200: mknod d6/da/d29/c3e 0 2024-05-27T09:29:32.845 INFO:tasks.workunit.client.1.smithi138.stdout:9/201: creat d6/f3f x:0 0 0 2024-05-27T09:29:32.848 INFO:tasks.workunit.client.1.smithi138.stdout:0/262: mknod da/d17/d1d/d1f/d42/c5e 0 2024-05-27T09:29:32.848 INFO:tasks.workunit.client.1.smithi138.stdout:9/202: creat d6/da/d2f/f40 x:0 0 0 2024-05-27T09:29:32.848 INFO:tasks.workunit.client.1.smithi138.stdout:9/203: dread - d6/da/d2f/f40 zero size 2024-05-27T09:29:32.850 INFO:tasks.workunit.client.1.smithi138.stdout:9/204: mknod d6/da/d2f/c41 0 2024-05-27T09:29:32.851 INFO:tasks.workunit.client.0.smithi119.stdout:4/797: rename d4/d6/d28/d2f/d36/f94 to d4/d23/d24/d5b/ffa 0 2024-05-27T09:29:32.855 INFO:tasks.workunit.client.0.smithi119.stdout:3/884: dwrite db/f69 [0,4194304] 0 2024-05-27T09:29:32.856 INFO:tasks.workunit.client.0.smithi119.stdout:4/798: mkdir d4/de/dfb 0 2024-05-27T09:29:32.857 INFO:tasks.workunit.client.0.smithi119.stdout:4/799: truncate d4/db1/ff4 1327189 0 2024-05-27T09:29:32.859 INFO:tasks.workunit.client.0.smithi119.stdout:0/733: rename d2/d6/d9/d24/d30/d4a/c65 to d2/d6/d9/d24/d30/dcc/de3/cf0 0 2024-05-27T09:29:32.864 INFO:tasks.workunit.client.1.smithi138.stdout:9/205: write d6/db/dc/f2c [904152,51755] 0 2024-05-27T09:29:32.866 INFO:tasks.workunit.client.1.smithi138.stdout:0/263: creat da/d2a/d55/f5f x:0 0 0 2024-05-27T09:29:32.866 INFO:tasks.workunit.client.0.smithi119.stdout:3/885: truncate db/d36/d45/dab/fcd 130347 0 2024-05-27T09:29:32.868 INFO:tasks.workunit.client.0.smithi119.stdout:0/734: getdents d2/d6/d9/d24 0 2024-05-27T09:29:32.868 INFO:tasks.workunit.client.0.smithi119.stdout:4/800: write d4/de/d66/f68 [811114,10129] 0 2024-05-27T09:29:32.868 INFO:tasks.workunit.client.0.smithi119.stdout:0/735: creat d2/d6/d9/d12/d4f/ff1 x:0 0 0 2024-05-27T09:29:32.869 INFO:tasks.workunit.client.0.smithi119.stdout:0/736: write d2/d6/d9/f6d [16590,63541] 0 2024-05-27T09:29:32.869 INFO:tasks.workunit.client.1.smithi138.stdout:9/206: symlink d6/da/d2f/l42 0 2024-05-27T09:29:32.870 INFO:tasks.workunit.client.1.smithi138.stdout:9/207: chown d6/da/d2f/l42 8 1 2024-05-27T09:29:32.872 INFO:tasks.workunit.client.1.smithi138.stdout:9/208: rmdir d6/da/d26 39 2024-05-27T09:29:32.872 INFO:tasks.workunit.client.1.smithi138.stdout:7/220: dwrite d0/d7/f3b [0,4194304] 0 2024-05-27T09:29:32.876 INFO:tasks.workunit.client.1.smithi138.stdout:0/264: mkdir da/d17/d1d/d1f/d60 0 2024-05-27T09:29:32.877 INFO:tasks.workunit.client.0.smithi119.stdout:3/886: mkdir db/d5b/dce/d11f 0 2024-05-27T09:29:32.877 INFO:tasks.workunit.client.1.smithi138.stdout:7/221: rename d0/c22 to d0/d2b/c44 0 2024-05-27T09:29:32.877 INFO:tasks.workunit.client.1.smithi138.stdout:7/222: chown d0/d7/d43 920412 1 2024-05-27T09:29:32.878 INFO:tasks.workunit.client.1.smithi138.stdout:7/223: creat d0/f45 x:0 0 0 2024-05-27T09:29:32.879 INFO:tasks.workunit.client.0.smithi119.stdout:0/737: symlink d2/d6/d9/d24/d30/dcc/lf2 0 2024-05-27T09:29:32.881 INFO:tasks.workunit.client.1.smithi138.stdout:3/167: sync 2024-05-27T09:29:32.881 INFO:tasks.workunit.client.1.smithi138.stdout:3/168: stat d0/d2/da/df 0 2024-05-27T09:29:32.884 INFO:tasks.workunit.client.1.smithi138.stdout:3/169: getdents d0/d2/da 0 2024-05-27T09:29:32.884 INFO:tasks.workunit.client.1.smithi138.stdout:3/170: read - d0/d2/da/f33 zero size 2024-05-27T09:29:32.885 INFO:tasks.workunit.client.1.smithi138.stdout:6/332: dwrite d3/f53 [0,4194304] 0 2024-05-27T09:29:32.885 INFO:tasks.workunit.client.0.smithi119.stdout:3/887: mkdir db/d36/d45/d4c/d120 0 2024-05-27T09:29:32.886 INFO:tasks.workunit.client.0.smithi119.stdout:4/801: dread d4/de/d21/d40/fc6 [0,4194304] 0 2024-05-27T09:29:32.888 INFO:tasks.workunit.client.1.smithi138.stdout:3/171: truncate d0/d2/da/f2d 1201813 0 2024-05-27T09:29:32.888 INFO:tasks.workunit.client.1.smithi138.stdout:3/172: fsync d0/d2/da/df/d10/d32/f28 0 2024-05-27T09:29:32.889 INFO:tasks.workunit.client.0.smithi119.stdout:7/802: dwrite d7/df/d19/d4d/d89/deb/ff6 [0,4194304] 0 2024-05-27T09:29:32.890 INFO:tasks.workunit.client.0.smithi119.stdout:0/738: dread d2/d6/d9/d2f/fe9 [0,4194304] 0 2024-05-27T09:29:32.891 INFO:tasks.workunit.client.0.smithi119.stdout:3/888: read db/d36/d45/dab/f103 [476773,68924] 0 2024-05-27T09:29:32.891 INFO:tasks.workunit.client.0.smithi119.stdout:3/889: stat db/d36/d45/d4c/dbe/c10b 0 2024-05-27T09:29:32.893 INFO:tasks.workunit.client.0.smithi119.stdout:4/802: rmdir d4/d6/d3e/d89/dcf 39 2024-05-27T09:29:32.899 INFO:tasks.workunit.client.1.smithi138.stdout:6/333: unlink f2 0 2024-05-27T09:29:32.910 INFO:tasks.workunit.client.0.smithi119.stdout:0/739: symlink d2/d6/d9/d12/d4b/d96/lf3 0 2024-05-27T09:29:32.910 INFO:tasks.workunit.client.0.smithi119.stdout:3/890: truncate db/d36/d45/d4c/d62/d13/d33/d9b/fca 3957727 0 2024-05-27T09:29:32.913 INFO:tasks.workunit.client.0.smithi119.stdout:3/891: link db/d7f/c48 db/d7f/dd2/c121 0 2024-05-27T09:29:32.916 INFO:tasks.workunit.client.0.smithi119.stdout:0/740: creat d2/d6/d9/d24/d30/dcc/ff4 x:0 0 0 2024-05-27T09:29:32.916 INFO:tasks.workunit.client.0.smithi119.stdout:0/741: chown d2/d6/d9/d12/d4b/la1 2603 1 2024-05-27T09:29:32.919 INFO:tasks.workunit.client.1.smithi138.stdout:6/334: dread d3/d11/d2d/d37/f4a [0,4194304] 0 2024-05-27T09:29:32.926 INFO:tasks.workunit.client.0.smithi119.stdout:8/712: sync 2024-05-27T09:29:32.927 INFO:tasks.workunit.client.0.smithi119.stdout:2/776: sync 2024-05-27T09:29:32.927 INFO:tasks.workunit.client.0.smithi119.stdout:2/777: truncate d2/da/d8a/dd4/d89/db0/ffb 952222 0 2024-05-27T09:29:32.931 INFO:tasks.workunit.client.0.smithi119.stdout:5/827: sync 2024-05-27T09:29:32.931 INFO:tasks.workunit.client.0.smithi119.stdout:5/828: chown d6/dd/l65 486492 1 2024-05-27T09:29:32.931 INFO:tasks.workunit.client.0.smithi119.stdout:1/793: sync 2024-05-27T09:29:32.931 INFO:tasks.workunit.client.0.smithi119.stdout:9/645: sync 2024-05-27T09:29:32.931 INFO:tasks.workunit.client.0.smithi119.stdout:6/739: sync 2024-05-27T09:29:32.931 INFO:tasks.workunit.client.0.smithi119.stdout:6/740: fdatasync d0/d2/d1c/d21/d58/fcc 0 2024-05-27T09:29:32.932 INFO:tasks.workunit.client.0.smithi119.stdout:5/829: mkdir d6/db/de/d27/d77/d129 0 2024-05-27T09:29:32.933 INFO:tasks.workunit.client.0.smithi119.stdout:1/794: mknod d7/dd/d25/d60/cf6 0 2024-05-27T09:29:32.946 INFO:tasks.workunit.client.1.smithi138.stdout:7/224: dread d0/d7/f1b [0,4194304] 0 2024-05-27T09:29:32.950 INFO:tasks.workunit.client.1.smithi138.stdout:7/225: creat d0/d23/d3a/f46 x:0 0 0 2024-05-27T09:29:32.953 INFO:tasks.workunit.client.1.smithi138.stdout:7/226: write d0/d7/f39 [1219096,130500] 0 2024-05-27T09:29:32.953 INFO:tasks.workunit.client.0.smithi119.stdout:2/778: write d2/da/d8a/dd4/f7e [957346,63557] 0 2024-05-27T09:29:32.954 INFO:tasks.workunit.client.0.smithi119.stdout:2/779: creat d2/da/d8a/dd4/d89/db0/f107 x:0 0 0 2024-05-27T09:29:32.957 INFO:tasks.workunit.client.1.smithi138.stdout:3/173: dread d0/d2/da/f1b [0,4194304] 0 2024-05-27T09:29:32.957 INFO:tasks.workunit.client.1.smithi138.stdout:3/174: creat d0/d2/da/df/d10/d32/f3c x:0 0 0 2024-05-27T09:29:32.957 INFO:tasks.workunit.client.1.smithi138.stdout:7/227: creat d0/d23/f47 x:0 0 0 2024-05-27T09:29:32.958 INFO:tasks.workunit.client.0.smithi119.stdout:6/741: mknod d0/d2/d1c/c106 0 2024-05-27T09:29:32.959 INFO:tasks.workunit.client.0.smithi119.stdout:5/830: mkdir d6/dd/d46/d47/d4b/dd8/d124/d100/d12a 0 2024-05-27T09:29:32.959 INFO:tasks.workunit.client.0.smithi119.stdout:5/831: write d6/dd/f106 [166682,130911] 0 2024-05-27T09:29:32.959 INFO:tasks.workunit.client.0.smithi119.stdout:5/832: fdatasync d6/dd/d6e/fdf 0 2024-05-27T09:29:32.960 INFO:tasks.workunit.client.0.smithi119.stdout:8/713: write d3/d8/d4d/d58/f89 [1651654,37581] 0 2024-05-27T09:29:32.961 INFO:tasks.workunit.client.0.smithi119.stdout:1/795: symlink d7/db/d12/d26/d39/lf7 0 2024-05-27T09:29:32.962 INFO:tasks.workunit.client.1.smithi138.stdout:3/175: rename d0/d2/da/df/d10/d32/f29 to d0/d2/f3d 0 2024-05-27T09:29:32.963 INFO:tasks.workunit.client.0.smithi119.stdout:2/780: mkdir d2/da/d8a/dd4/d69/df5/d108 0 2024-05-27T09:29:32.963 INFO:tasks.workunit.client.0.smithi119.stdout:5/833: rmdir d6/db/dc 39 2024-05-27T09:29:32.964 INFO:tasks.workunit.client.0.smithi119.stdout:8/714: symlink d3/de/dad/lef 0 2024-05-27T09:29:32.964 INFO:tasks.workunit.client.0.smithi119.stdout:8/715: fdatasync d3/de/dad/fc0 0 2024-05-27T09:29:32.965 INFO:tasks.workunit.client.0.smithi119.stdout:5/834: write d6/db/d9a/f9e [220660,13464] 0 2024-05-27T09:29:32.966 INFO:tasks.workunit.client.1.smithi138.stdout:3/176: symlink d0/d2/da/df/d10/d32/d37/l3e 0 2024-05-27T09:29:32.968 INFO:tasks.workunit.client.0.smithi119.stdout:9/646: write fc [2431736,88350] 0 2024-05-27T09:29:32.968 INFO:tasks.workunit.client.0.smithi119.stdout:9/647: truncate d18/f76 253626 0 2024-05-27T09:29:32.968 INFO:tasks.workunit.client.0.smithi119.stdout:1/796: unlink d7/db/d12/ddb/d38/d3c/c5f 0 2024-05-27T09:29:32.968 INFO:tasks.workunit.client.0.smithi119.stdout:9/648: write d18/d19/d1b/f1e [1949799,97755] 0 2024-05-27T09:29:32.968 INFO:tasks.workunit.client.0.smithi119.stdout:5/835: symlink d6/db/dc/l12b 0 2024-05-27T09:29:32.968 INFO:tasks.workunit.client.0.smithi119.stdout:5/836: read - d6/db/de/d27/f76 zero size 2024-05-27T09:29:32.969 INFO:tasks.workunit.client.0.smithi119.stdout:1/797: readlink d7/db/d12/d15/d1f/l79 0 2024-05-27T09:29:32.969 INFO:tasks.workunit.client.0.smithi119.stdout:5/837: dread - d6/dd/d46/d47/d4b/dd8/d124/d100/f10c zero size 2024-05-27T09:29:32.969 INFO:tasks.workunit.client.1.smithi138.stdout:3/177: unlink d0/d2/da/df/d10/d2a/c2e 0 2024-05-27T09:29:32.970 INFO:tasks.workunit.client.0.smithi119.stdout:2/781: symlink d2/da/d8a/dd4/d89/d94/d87/l109 0 2024-05-27T09:29:32.970 INFO:tasks.workunit.client.0.smithi119.stdout:2/782: chown d2/da/d15/ca4 200063155 1 2024-05-27T09:29:32.970 INFO:tasks.workunit.client.1.smithi138.stdout:6/335: dwrite d3/f31 [4194304,4194304] 0 2024-05-27T09:29:32.970 INFO:tasks.workunit.client.1.smithi138.stdout:9/209: dwrite d6/db/f39 [0,4194304] 0 2024-05-27T09:29:32.970 INFO:tasks.workunit.client.1.smithi138.stdout:9/210: dread - d6/da/d2f/f40 zero size 2024-05-27T09:29:32.971 INFO:tasks.workunit.client.0.smithi119.stdout:0/742: dread d2/d6/d9/fbd [0,4194304] 0 2024-05-27T09:29:32.975 INFO:tasks.workunit.client.0.smithi119.stdout:1/798: rmdir d7/db/d12/d26/d39 39 2024-05-27T09:29:32.975 INFO:tasks.workunit.client.0.smithi119.stdout:1/799: read - d7/dd/d44/fa9 zero size 2024-05-27T09:29:32.975 INFO:tasks.workunit.client.0.smithi119.stdout:1/800: chown d7/c74 232 1 2024-05-27T09:29:32.975 INFO:tasks.workunit.client.0.smithi119.stdout:1/801: dread - d7/db/d35/fd4 zero size 2024-05-27T09:29:32.981 INFO:tasks.workunit.client.1.smithi138.stdout:6/336: read d3/d19/f3d [150052,40500] 0 2024-05-27T09:29:32.982 INFO:tasks.workunit.client.1.smithi138.stdout:6/337: fsync d3/d35/d5a/f68 0 2024-05-27T09:29:32.982 INFO:tasks.workunit.client.0.smithi119.stdout:0/743: rename d2/d6/d9/d24/f9f to d2/d6/d23/db8/ff5 0 2024-05-27T09:29:32.982 INFO:tasks.workunit.client.0.smithi119.stdout:1/802: truncate d7/dd/d44/f56 1097268 0 2024-05-27T09:29:32.982 INFO:tasks.workunit.client.0.smithi119.stdout:1/803: write d7/db/d12/feb [306496,100536] 0 2024-05-27T09:29:32.985 INFO:tasks.workunit.client.0.smithi119.stdout:0/744: rename d2/d6/d75/f86 to d2/d6/d9/d12/de5/ff6 0 2024-05-27T09:29:32.992 INFO:tasks.workunit.client.0.smithi119.stdout:8/716: dread d3/de/f62 [0,4194304] 0 2024-05-27T09:29:32.992 INFO:tasks.workunit.client.0.smithi119.stdout:0/745: unlink d2/d6/d9/d12/c91 0 2024-05-27T09:29:32.996 INFO:tasks.workunit.client.0.smithi119.stdout:7/803: dwrite d7/d1a/fa1 [0,4194304] 0 2024-05-27T09:29:32.996 INFO:tasks.workunit.client.0.smithi119.stdout:5/838: dread d6/db/de/d27/dc9/fc6 [0,4194304] 0 2024-05-27T09:29:32.999 INFO:tasks.workunit.client.0.smithi119.stdout:5/839: rmdir d6/dd/d6e 39 2024-05-27T09:29:32.999 INFO:tasks.workunit.client.0.smithi119.stdout:5/840: dread - d6/dd/d46/d47/d4b/f122 zero size 2024-05-27T09:29:32.999 INFO:tasks.workunit.client.0.smithi119.stdout:5/841: read d6/dd/d46/d47/d4b/f120 [40636,120744] 0 2024-05-27T09:29:32.999 INFO:tasks.workunit.client.1.smithi138.stdout:6/338: write d3/d11/d2d/d37/f4a [2342763,109751] 0 2024-05-27T09:29:32.999 INFO:tasks.workunit.client.0.smithi119.stdout:5/842: write d6/dd/d46/d47/d4b/dd8/d124/fb1 [1364385,125825] 0 2024-05-27T09:29:33.000 INFO:tasks.workunit.client.1.smithi138.stdout:6/339: chown d3/d11/d2d/d37/c58 1312 1 2024-05-27T09:29:33.000 INFO:tasks.workunit.client.1.smithi138.stdout:6/340: chown d3/c13 9528856 1 2024-05-27T09:29:33.011 INFO:tasks.workunit.client.1.smithi138.stdout:3/178: dread d0/d2/fc [0,4194304] 0 2024-05-27T09:29:33.018 INFO:tasks.workunit.client.0.smithi119.stdout:4/803: dwrite d4/d23/f52 [4194304,4194304] 0 2024-05-27T09:29:33.018 INFO:tasks.workunit.client.0.smithi119.stdout:4/804: chown d4/d6/d28/d2f/d36/dc8 136491695 1 2024-05-27T09:29:33.023 INFO:tasks.workunit.client.0.smithi119.stdout:4/805: creat d4/d6/d3e/dd2/ffc x:0 0 0 2024-05-27T09:29:33.030 INFO:tasks.workunit.client.0.smithi119.stdout:0/746: write d2/d1a/d20/f64 [4712562,129034] 0 2024-05-27T09:29:33.033 INFO:tasks.workunit.client.1.smithi138.stdout:3/179: rmdir d0 39 2024-05-27T09:29:33.034 INFO:tasks.workunit.client.0.smithi119.stdout:0/747: rmdir d2/d1a/d20 39 2024-05-27T09:29:33.034 INFO:tasks.workunit.client.0.smithi119.stdout:0/748: chown d2/d6/d9/d24/d69 0 1 2024-05-27T09:29:33.035 INFO:tasks.workunit.client.1.smithi138.stdout:3/180: unlink d0/d2/da/df/d10/c1f 0 2024-05-27T09:29:33.039 INFO:tasks.workunit.client.1.smithi138.stdout:7/228: dwrite d0/d7/f39 [0,4194304] 0 2024-05-27T09:29:33.043 INFO:tasks.workunit.client.0.smithi119.stdout:0/749: mknod d2/d6/d9/d24/d69/cf7 0 2024-05-27T09:29:33.043 INFO:tasks.workunit.client.1.smithi138.stdout:1/271: sync 2024-05-27T09:29:33.043 INFO:tasks.workunit.client.1.smithi138.stdout:8/219: sync 2024-05-27T09:29:33.047 INFO:tasks.workunit.client.1.smithi138.stdout:7/229: creat d0/d23/f48 x:0 0 0 2024-05-27T09:29:33.048 INFO:tasks.workunit.client.0.smithi119.stdout:0/750: mknod d2/d6/d9/d24/d30/dcc/cf8 0 2024-05-27T09:29:33.048 INFO:tasks.workunit.client.0.smithi119.stdout:0/751: truncate d2/d6/d9/d12/f47 4621802 0 2024-05-27T09:29:33.050 INFO:tasks.workunit.client.1.smithi138.stdout:4/249: sync 2024-05-27T09:29:33.050 INFO:tasks.workunit.client.1.smithi138.stdout:8/220: mknod d3/d8/d32/c3e 0 2024-05-27T09:29:33.055 INFO:tasks.workunit.client.0.smithi119.stdout:0/752: truncate d2/d6/d9/f66 2759674 0 2024-05-27T09:29:33.058 INFO:tasks.workunit.client.1.smithi138.stdout:4/250: mknod d7/df/d35/c54 0 2024-05-27T09:29:33.061 INFO:tasks.workunit.client.1.smithi138.stdout:4/251: link d7/f23 d7/d27/d3a/f55 0 2024-05-27T09:29:33.076 INFO:tasks.workunit.client.1.smithi138.stdout:7/230: write d0/d7/f1b [3115350,128882] 0 2024-05-27T09:29:33.077 INFO:tasks.workunit.client.0.smithi119.stdout:6/742: dwrite d0/d2/d1c/f4c [4194304,4194304] 0 2024-05-27T09:29:33.079 INFO:tasks.workunit.client.1.smithi138.stdout:6/341: dwrite d3/d11/f2e [0,4194304] 0 2024-05-27T09:29:33.079 INFO:tasks.workunit.client.1.smithi138.stdout:7/231: mkdir d0/d23/d49 0 2024-05-27T09:29:33.081 INFO:tasks.workunit.client.0.smithi119.stdout:6/743: symlink d0/d11/l107 0 2024-05-27T09:29:33.084 INFO:tasks.workunit.client.1.smithi138.stdout:7/232: rename d0/d7/f1b to d0/d23/d49/f4a 0 2024-05-27T09:29:33.085 INFO:tasks.workunit.client.0.smithi119.stdout:4/806: dread d4/de/d21/f2b [0,4194304] 0 2024-05-27T09:29:33.085 INFO:tasks.workunit.client.0.smithi119.stdout:5/843: dwrite d6/dd/d39/d19/d3c/d52/f99 [0,4194304] 0 2024-05-27T09:29:33.085 INFO:tasks.workunit.client.0.smithi119.stdout:4/807: fdatasync d4/f7e 0 2024-05-27T09:29:33.086 INFO:tasks.workunit.client.0.smithi119.stdout:1/804: dwrite d7/dd/d25/d60/d61/f7c [0,4194304] 0 2024-05-27T09:29:33.088 INFO:tasks.workunit.client.0.smithi119.stdout:7/804: dwrite d7/f66 [0,4194304] 0 2024-05-27T09:29:33.092 INFO:tasks.workunit.client.0.smithi119.stdout:6/744: rename d0/d2/f30 to d0/d2/d1c/d21/f108 0 2024-05-27T09:29:33.092 INFO:tasks.workunit.client.0.smithi119.stdout:2/783: dwrite d2/da/d2c/f3c [0,4194304] 0 2024-05-27T09:29:33.094 INFO:tasks.workunit.client.0.smithi119.stdout:5/844: getdents d6/dd/d46 0 2024-05-27T09:29:33.094 INFO:tasks.workunit.client.0.smithi119.stdout:4/808: unlink d4/d48/f5e 0 2024-05-27T09:29:33.094 INFO:tasks.workunit.client.0.smithi119.stdout:5/845: fdatasync d6/d2f/f3a 0 2024-05-27T09:29:33.094 INFO:tasks.workunit.client.0.smithi119.stdout:4/809: chown d4/d1c/d39/f4b 83267456 1 2024-05-27T09:29:33.094 INFO:tasks.workunit.client.0.smithi119.stdout:1/805: read d7/dd/d44/f56 [353194,100366] 0 2024-05-27T09:29:33.097 INFO:tasks.workunit.client.0.smithi119.stdout:2/784: mknod d2/da/d8a/dd4/d89/d94/de2/c10a 0 2024-05-27T09:29:33.100 INFO:tasks.workunit.client.1.smithi138.stdout:9/211: dwrite d6/da/d2f/f32 [0,4194304] 0 2024-05-27T09:29:33.100 INFO:tasks.workunit.client.1.smithi138.stdout:1/272: dread d4/f37 [0,4194304] 0 2024-05-27T09:29:33.100 INFO:tasks.workunit.client.1.smithi138.stdout:1/273: rename d4/d48 to d4/d48/d22/d53 22 2024-05-27T09:29:33.101 INFO:tasks.workunit.client.0.smithi119.stdout:9/649: dread d18/d19/d1b/f1e [0,4194304] 0 2024-05-27T09:29:33.102 INFO:tasks.workunit.client.0.smithi119.stdout:9/650: write d18/f8b [540460,119476] 0 2024-05-27T09:29:33.102 INFO:tasks.workunit.client.0.smithi119.stdout:9/651: write d18/d19/d1b/f64 [1311467,34609] 0 2024-05-27T09:29:33.103 INFO:tasks.workunit.client.0.smithi119.stdout:4/810: truncate d4/d6/d3e/faf 1001060 0 2024-05-27T09:29:33.103 INFO:tasks.workunit.client.0.smithi119.stdout:1/806: unlink d7/db/c4f 0 2024-05-27T09:29:33.103 INFO:tasks.workunit.client.0.smithi119.stdout:5/846: mkdir d6/db/de/d27/dc9/d12c 0 2024-05-27T09:29:33.103 INFO:tasks.workunit.client.0.smithi119.stdout:4/811: creat d4/d23/d24/d5b/da8/dda/ffd x:0 0 0 2024-05-27T09:29:33.103 INFO:tasks.workunit.client.1.smithi138.stdout:1/274: creat d4/d48/d39/f54 x:0 0 0 2024-05-27T09:29:33.106 INFO:tasks.workunit.client.0.smithi119.stdout:9/652: truncate d18/d19/d1b/d29/fac 2501204 0 2024-05-27T09:29:33.108 INFO:tasks.workunit.client.1.smithi138.stdout:4/252: write d7/df/f4d [141526,120698] 0 2024-05-27T09:29:33.109 INFO:tasks.workunit.client.1.smithi138.stdout:4/253: write d7/df/f24 [1172875,80983] 0 2024-05-27T09:29:33.111 INFO:tasks.workunit.client.0.smithi119.stdout:5/847: mknod d6/db/d9a/c12d 0 2024-05-27T09:29:33.114 INFO:tasks.workunit.client.0.smithi119.stdout:9/653: mkdir d18/d19/d1b/d40/d7b/dc6 0 2024-05-27T09:29:33.114 INFO:tasks.workunit.client.0.smithi119.stdout:9/654: chown d18/d19/d1b/d40/l84 2 1 2024-05-27T09:29:33.114 INFO:tasks.workunit.client.0.smithi119.stdout:9/655: creat d18/d19/d1b/d40/fc7 x:0 0 0 2024-05-27T09:29:33.115 INFO:tasks.workunit.client.0.smithi119.stdout:5/848: creat d6/dd/d39/d19/d7d/db7/f12e x:0 0 0 2024-05-27T09:29:33.115 INFO:tasks.workunit.client.1.smithi138.stdout:7/233: read d0/f36 [2839656,38730] 0 2024-05-27T09:29:33.119 INFO:tasks.workunit.client.0.smithi119.stdout:2/785: read d2/da/f2e [4151119,73694] 0 2024-05-27T09:29:33.120 INFO:tasks.workunit.client.1.smithi138.stdout:4/254: rename d7/df/d35/d4c/c50 to d7/df/c56 0 2024-05-27T09:29:33.125 INFO:tasks.workunit.client.0.smithi119.stdout:7/805: dread d7/df/d19/f1f [0,4194304] 0 2024-05-27T09:29:33.126 INFO:tasks.workunit.client.0.smithi119.stdout:9/656: truncate d18/d19/f59 3236950 0 2024-05-27T09:29:33.129 INFO:tasks.workunit.client.0.smithi119.stdout:7/806: mknod d7/d1a/d99/c103 0 2024-05-27T09:29:33.129 INFO:tasks.workunit.client.0.smithi119.stdout:7/807: fsync d7/df/f29 0 2024-05-27T09:29:33.129 INFO:tasks.workunit.client.0.smithi119.stdout:7/808: getdents d7/d1a/d99/dcf 0 2024-05-27T09:29:33.130 INFO:tasks.workunit.client.1.smithi138.stdout:9/212: write d6/db/f39 [1128140,111187] 0 2024-05-27T09:29:33.130 INFO:tasks.workunit.client.0.smithi119.stdout:7/809: creat d7/df/d19/d97/d92/db7/dba/d42/d64/dad/f104 x:0 0 0 2024-05-27T09:29:33.131 INFO:tasks.workunit.client.1.smithi138.stdout:9/213: getdents d6/da/d26 0 2024-05-27T09:29:33.131 INFO:tasks.workunit.client.1.smithi138.stdout:9/214: creat d6/da/d26/d28/f43 x:0 0 0 2024-05-27T09:29:33.132 INFO:tasks.workunit.client.1.smithi138.stdout:9/215: chown d6/cd 190 1 2024-05-27T09:29:33.132 INFO:tasks.workunit.client.1.smithi138.stdout:4/255: dread d7/f42 [0,4194304] 0 2024-05-27T09:29:33.134 INFO:tasks.workunit.client.1.smithi138.stdout:9/216: rename d6/db/f14 to d6/da/f44 0 2024-05-27T09:29:33.134 INFO:tasks.workunit.client.1.smithi138.stdout:9/217: fsync d6/db/dc/f12 0 2024-05-27T09:29:33.138 INFO:tasks.workunit.client.0.smithi119.stdout:5/849: dread d6/f9 [0,4194304] 0 2024-05-27T09:29:33.140 INFO:tasks.workunit.client.1.smithi138.stdout:9/218: symlink d6/da/d2f/l45 0 2024-05-27T09:29:33.140 INFO:tasks.workunit.client.1.smithi138.stdout:9/219: read d6/db/dc/f2c [645526,21319] 0 2024-05-27T09:29:33.140 INFO:tasks.workunit.client.1.smithi138.stdout:9/220: truncate d6/db/dc/f1e 1501363 0 2024-05-27T09:29:33.141 INFO:tasks.workunit.client.1.smithi138.stdout:4/256: rename d7/df/l4b to d7/df/d35/d4c/l57 0 2024-05-27T09:29:33.145 INFO:tasks.workunit.client.1.smithi138.stdout:4/257: creat d7/d27/d47/f58 x:0 0 0 2024-05-27T09:29:33.150 INFO:tasks.workunit.client.1.smithi138.stdout:9/221: read d6/da/f44 [331912,74997] 0 2024-05-27T09:29:33.150 INFO:tasks.workunit.client.1.smithi138.stdout:9/222: fdatasync f1 0 2024-05-27T09:29:33.158 INFO:tasks.workunit.client.0.smithi119.stdout:5/850: symlink d6/dd/d46/d47/d105/l12f 0 2024-05-27T09:29:33.162 INFO:tasks.workunit.client.1.smithi138.stdout:5/222: sync 2024-05-27T09:29:33.162 INFO:tasks.workunit.client.1.smithi138.stdout:2/197: sync 2024-05-27T09:29:33.162 INFO:tasks.workunit.client.1.smithi138.stdout:0/265: sync 2024-05-27T09:29:33.162 INFO:tasks.workunit.client.1.smithi138.stdout:2/198: readlink d1/d15/d2d/l20 0 2024-05-27T09:29:33.163 INFO:tasks.workunit.client.0.smithi119.stdout:0/753: dwrite d2/d6/d9/fbd [0,4194304] 0 2024-05-27T09:29:33.163 INFO:tasks.workunit.client.0.smithi119.stdout:5/851: creat d6/dd/d46/d47/d4b/dd8/ddb/f130 x:0 0 0 2024-05-27T09:29:33.163 INFO:tasks.workunit.client.0.smithi119.stdout:5/852: dread - d6/dd/d39/d74/feb zero size 2024-05-27T09:29:33.168 INFO:tasks.workunit.client.1.smithi138.stdout:1/275: dread d4/f12 [0,4194304] 0 2024-05-27T09:29:33.168 INFO:tasks.workunit.client.1.smithi138.stdout:1/276: chown d4/d1a/f27 0 1 2024-05-27T09:29:33.168 INFO:tasks.workunit.client.1.smithi138.stdout:2/199: symlink d1/d15/d2d/d31/l3f 0 2024-05-27T09:29:33.169 INFO:tasks.workunit.client.1.smithi138.stdout:5/223: rename d3/d12/f25 to d3/d12/d16/d1d/d30/f4a 0 2024-05-27T09:29:33.172 INFO:tasks.workunit.client.0.smithi119.stdout:2/786: read d2/f90 [3019255,82950] 0 2024-05-27T09:29:33.174 INFO:tasks.workunit.client.0.smithi119.stdout:0/754: rename d2/d6/d23/d7a/cdb to d2/d6/d9/dab/cf9 0 2024-05-27T09:29:33.175 INFO:tasks.workunit.client.1.smithi138.stdout:1/277: creat d4/d3c/d51/f55 x:0 0 0 2024-05-27T09:29:33.175 INFO:tasks.workunit.client.1.smithi138.stdout:8/221: dwrite d3/d8/d32/f3b [0,4194304] 0 2024-05-27T09:29:33.178 INFO:tasks.workunit.client.1.smithi138.stdout:1/278: mknod d4/d1a/d2e/d32/c56 0 2024-05-27T09:29:33.181 INFO:tasks.workunit.client.0.smithi119.stdout:2/787: rmdir d2/da/d8a/dd4 39 2024-05-27T09:29:33.183 INFO:tasks.workunit.client.1.smithi138.stdout:1/279: symlink d4/d48/l57 0 2024-05-27T09:29:33.184 INFO:tasks.workunit.client.0.smithi119.stdout:6/745: dwrite d0/ff5 [0,4194304] 0 2024-05-27T09:29:33.190 INFO:tasks.workunit.client.1.smithi138.stdout:0/266: write da/d2a/f35 [2458970,18284] 0 2024-05-27T09:29:33.190 INFO:tasks.workunit.client.1.smithi138.stdout:4/258: dread d7/df/f52 [0,4194304] 0 2024-05-27T09:29:33.192 INFO:tasks.workunit.client.0.smithi119.stdout:2/788: symlink d2/da/d8a/dd4/l10b 0 2024-05-27T09:29:33.192 INFO:tasks.workunit.client.0.smithi119.stdout:2/789: write d2/da/d15/d53/fff [357823,34376] 0 2024-05-27T09:29:33.192 INFO:tasks.workunit.client.1.smithi138.stdout:4/259: read - d7/df/f2e zero size 2024-05-27T09:29:33.192 INFO:tasks.workunit.client.1.smithi138.stdout:5/224: rmdir d3/d12/d16/d1d/d30 39 2024-05-27T09:29:33.192 INFO:tasks.workunit.client.1.smithi138.stdout:5/225: creat d3/d12/d26/f4b x:0 0 0 2024-05-27T09:29:33.194 INFO:tasks.workunit.client.1.smithi138.stdout:0/267: truncate da/d17/d1d/d1f/f38 2693905 0 2024-05-27T09:29:33.199 INFO:tasks.workunit.client.0.smithi119.stdout:6/746: rename d0/d19 to d0/d2/d57/d37/d74/df1/d109 0 2024-05-27T09:29:33.207 INFO:tasks.workunit.client.1.smithi138.stdout:4/260: truncate d7/f3d 2968597 0 2024-05-27T09:29:33.207 INFO:tasks.workunit.client.1.smithi138.stdout:4/261: truncate d7/d27/f2c 1651384 0 2024-05-27T09:29:33.207 INFO:tasks.workunit.client.1.smithi138.stdout:5/226: creat d3/f4c x:0 0 0 2024-05-27T09:29:33.207 INFO:tasks.workunit.client.1.smithi138.stdout:7/234: dwrite d0/f21 [0,4194304] 0 2024-05-27T09:29:33.207 INFO:tasks.workunit.client.1.smithi138.stdout:7/235: fdatasync d0/f32 0 2024-05-27T09:29:33.208 INFO:tasks.workunit.client.1.smithi138.stdout:5/227: write d3/d12/d16/f2e [5103265,124270] 0 2024-05-27T09:29:33.208 INFO:tasks.workunit.client.1.smithi138.stdout:6/342: dwrite d3/d11/d2d/f61 [0,4194304] 0 2024-05-27T09:29:33.208 INFO:tasks.workunit.client.1.smithi138.stdout:6/343: readlink d3/d35/d43/d1c/l70 0 2024-05-27T09:29:33.208 INFO:tasks.workunit.client.1.smithi138.stdout:8/222: read d3/d8/d28/f2f [2662774,125473] 0 2024-05-27T09:29:33.209 INFO:tasks.workunit.client.0.smithi119.stdout:4/812: dwrite d4/d6/fd1 [0,4194304] 0 2024-05-27T09:29:33.210 INFO:tasks.workunit.client.1.smithi138.stdout:7/236: getdents d0/d2b 0 2024-05-27T09:29:33.213 INFO:tasks.workunit.client.1.smithi138.stdout:0/268: link f1 da/d2a/d55/f61 0 2024-05-27T09:29:33.214 INFO:tasks.workunit.client.0.smithi119.stdout:6/747: mkdir d0/d2/d57/d37/d74/dfe/d10a 0 2024-05-27T09:29:33.216 INFO:tasks.workunit.client.1.smithi138.stdout:7/237: symlink d0/d23/d49/l4b 0 2024-05-27T09:29:33.217 INFO:tasks.workunit.client.1.smithi138.stdout:0/269: creat da/d17/d1d/d43/f62 x:0 0 0 2024-05-27T09:29:33.218 INFO:tasks.workunit.client.0.smithi119.stdout:6/748: getdents d0/d85 0 2024-05-27T09:29:33.227 INFO:tasks.workunit.client.0.smithi119.stdout:9/657: dread d18/d19/f59 [0,4194304] 0 2024-05-27T09:29:33.228 INFO:tasks.workunit.client.1.smithi138.stdout:0/270: mkdir da/d17/d1d/d1f/d42/d63 0 2024-05-27T09:29:33.233 INFO:tasks.workunit.client.1.smithi138.stdout:7/238: creat d0/d23/f4c x:0 0 0 2024-05-27T09:29:33.233 INFO:tasks.workunit.client.0.smithi119.stdout:9/658: creat d18/d19/d61/d95/fc8 x:0 0 0 2024-05-27T09:29:33.233 INFO:tasks.workunit.client.0.smithi119.stdout:9/659: fsync d18/d19/d61/f9d 0 2024-05-27T09:29:33.233 INFO:tasks.workunit.client.0.smithi119.stdout:0/755: dwrite d2/d6/d9/d24/d69/fe1 [0,4194304] 0 2024-05-27T09:29:33.233 INFO:tasks.workunit.client.0.smithi119.stdout:0/756: write d2/f8c [7997212,5806] 0 2024-05-27T09:29:33.234 INFO:tasks.workunit.client.1.smithi138.stdout:1/280: dread d4/d3c/f4c [0,4194304] 0 2024-05-27T09:29:33.235 INFO:tasks.workunit.client.0.smithi119.stdout:1/807: dwrite d7/db/d12/ddb/d38/d3c/fd3 [0,4194304] 0 2024-05-27T09:29:33.254 INFO:tasks.workunit.client.0.smithi119.stdout:9/660: dread d18/d19/d1b/d29/f57 [0,4194304] 0 2024-05-27T09:29:33.258 INFO:tasks.workunit.client.0.smithi119.stdout:6/749: dread d0/d2/d1c/d21/ff9 [0,4194304] 0 2024-05-27T09:29:33.259 INFO:tasks.workunit.client.0.smithi119.stdout:7/810: dwrite d7/df/d19/d97/d92/db7/dba/d42/d64/fd4 [0,4194304] 0 2024-05-27T09:29:33.259 INFO:tasks.workunit.client.0.smithi119.stdout:7/811: fdatasync d7/df/d19/d97/d92/fab 0 2024-05-27T09:29:33.261 INFO:tasks.workunit.client.1.smithi138.stdout:6/344: dread d3/d35/d43/d1c/d32/d47/f5e [0,4194304] 0 2024-05-27T09:29:33.267 INFO:tasks.workunit.client.0.smithi119.stdout:0/757: dread d2/d6/d9/fbb [0,4194304] 0 2024-05-27T09:29:33.267 INFO:tasks.workunit.client.0.smithi119.stdout:0/758: readlink d2/d6/d9/l43 0 2024-05-27T09:29:33.267 INFO:tasks.workunit.client.0.smithi119.stdout:0/759: chown d2/c35 320390 1 2024-05-27T09:29:33.276 INFO:tasks.workunit.client.1.smithi138.stdout:9/223: dwrite d6/f3b [0,4194304] 0 2024-05-27T09:29:33.276 INFO:tasks.workunit.client.1.smithi138.stdout:9/224: truncate d6/da/d26/d28/f43 150286 0 2024-05-27T09:29:33.279 INFO:tasks.workunit.client.0.smithi119.stdout:7/812: mknod d7/df/d19/d97/d92/db7/dba/d42/d64/dad/dd3/c105 0 2024-05-27T09:29:33.282 INFO:tasks.workunit.client.0.smithi119.stdout:7/813: getdents d7/df/d19/d97/d92/db7/dba/d42/d76 0 2024-05-27T09:29:33.284 INFO:tasks.workunit.client.1.smithi138.stdout:9/225: rmdir d6/db/dc 39 2024-05-27T09:29:33.290 INFO:tasks.workunit.client.0.smithi119.stdout:9/661: read d18/d19/d1b/d29/f71 [2843548,77750] 0 2024-05-27T09:29:33.290 INFO:tasks.workunit.client.0.smithi119.stdout:9/662: write d18/d19/d1b/d29/f9e [1011730,73855] 0 2024-05-27T09:29:33.293 INFO:tasks.workunit.client.1.smithi138.stdout:9/226: getdents d6/da/d26/d28 0 2024-05-27T09:29:33.293 INFO:tasks.workunit.client.1.smithi138.stdout:4/262: dwrite d7/d27/f2c [0,4194304] 0 2024-05-27T09:29:33.295 INFO:tasks.workunit.client.1.smithi138.stdout:3/181: sync 2024-05-27T09:29:33.295 INFO:tasks.workunit.client.1.smithi138.stdout:3/182: write d0/d2/da/f1b [419082,24121] 0 2024-05-27T09:29:33.296 INFO:tasks.workunit.client.0.smithi119.stdout:4/813: dwrite d4/d23/f35 [0,4194304] 0 2024-05-27T09:29:33.296 INFO:tasks.workunit.client.1.smithi138.stdout:4/263: rmdir d7/df/d35 39 2024-05-27T09:29:33.299 INFO:tasks.workunit.client.0.smithi119.stdout:4/814: mknod d4/d6/d3e/dd2/cfe 0 2024-05-27T09:29:33.300 INFO:tasks.workunit.client.0.smithi119.stdout:4/815: chown d4/d6/d3e/d89/dcf 8707140 1 2024-05-27T09:29:33.301 INFO:tasks.workunit.client.1.smithi138.stdout:4/264: rename d7/d27/d47 to d7/df/d35/d4c/d59 0 2024-05-27T09:29:33.301 INFO:tasks.workunit.client.1.smithi138.stdout:4/265: chown d7/df/f1a 536117308 1 2024-05-27T09:29:33.303 INFO:tasks.workunit.client.0.smithi119.stdout:1/808: dwrite d7/db/d12/d26/f43 [0,4194304] 0 2024-05-27T09:29:33.303 INFO:tasks.workunit.client.0.smithi119.stdout:4/816: creat d4/d6/d3e/d89/fff x:0 0 0 2024-05-27T09:29:33.307 INFO:tasks.workunit.client.0.smithi119.stdout:4/817: mkdir d4/d6/d3e/d89/db2/def/d100 0 2024-05-27T09:29:33.307 INFO:tasks.workunit.client.1.smithi138.stdout:0/271: dwrite da/d2a/d55/f5f [0,4194304] 0 2024-05-27T09:29:33.308 INFO:tasks.workunit.client.0.smithi119.stdout:4/818: getdents d4/d6/d3e/d89/dcf 0 2024-05-27T09:29:33.309 INFO:tasks.workunit.client.0.smithi119.stdout:4/819: fsync d4/d1c/d39/f64 0 2024-05-27T09:29:33.309 INFO:tasks.workunit.client.0.smithi119.stdout:4/820: mknod d4/de/dfb/c101 0 2024-05-27T09:29:33.311 INFO:tasks.workunit.client.1.smithi138.stdout:0/272: unlink da/d2a/f51 0 2024-05-27T09:29:33.311 INFO:tasks.workunit.client.1.smithi138.stdout:0/273: write da/d2a/d3f/f59 [209928,87540] 0 2024-05-27T09:29:33.317 INFO:tasks.workunit.client.1.smithi138.stdout:0/274: truncate da/d17/d1d/d1f/f38 1824129 0 2024-05-27T09:29:33.317 INFO:tasks.workunit.client.1.smithi138.stdout:0/275: truncate da/d17/f44 4451109 0 2024-05-27T09:29:33.318 INFO:tasks.workunit.client.0.smithi119.stdout:7/814: dread d7/d1a/d99/dcf/fd2 [0,4194304] 0 2024-05-27T09:29:33.318 INFO:tasks.workunit.client.0.smithi119.stdout:7/815: stat d7/df/d19/d46/db5 0 2024-05-27T09:29:33.318 INFO:tasks.workunit.client.0.smithi119.stdout:7/816: fsync d7/df/d19/d97/d92/db7/dba/d42/d64/fc9 0 2024-05-27T09:29:33.318 INFO:tasks.workunit.client.0.smithi119.stdout:4/821: truncate d4/d4e/fb7 2330018 0 2024-05-27T09:29:33.321 INFO:tasks.workunit.client.0.smithi119.stdout:7/817: rename d7/df/d19/d9a/da6/dda/d44/d67/laf to d7/d1a/d99/l106 0 2024-05-27T09:29:33.322 INFO:tasks.workunit.client.0.smithi119.stdout:7/818: write d7/df/d19/d97/d92/fd5 [5120032,113508] 0 2024-05-27T09:29:33.325 INFO:tasks.workunit.client.0.smithi119.stdout:6/750: dread d0/d2/d57/d37/d74/fa1 [0,4194304] 0 2024-05-27T09:29:33.326 INFO:tasks.workunit.client.1.smithi138.stdout:4/266: write d7/fb [2941910,47739] 0 2024-05-27T09:29:33.328 INFO:tasks.workunit.client.1.smithi138.stdout:4/267: getdents d7/d27/d3a 0 2024-05-27T09:29:33.330 INFO:tasks.workunit.client.0.smithi119.stdout:4/822: dread d4/de/d21/d40/d47/d5c/fd5 [0,4194304] 0 2024-05-27T09:29:33.330 INFO:tasks.workunit.client.1.smithi138.stdout:4/268: link d7/d27/f48 d7/df/d35/f5a 0 2024-05-27T09:29:33.330 INFO:tasks.workunit.client.1.smithi138.stdout:4/269: chown d7/f23 650 1 2024-05-27T09:29:33.331 INFO:tasks.workunit.client.0.smithi119.stdout:4/823: symlink d4/d23/d24/d5b/da8/l102 0 2024-05-27T09:29:33.332 INFO:tasks.workunit.client.1.smithi138.stdout:4/270: mkdir d7/df/d5b 0 2024-05-27T09:29:33.333 INFO:tasks.workunit.client.1.smithi138.stdout:4/271: write d7/d27/f48 [32729,77406] 0 2024-05-27T09:29:33.340 INFO:tasks.workunit.client.0.smithi119.stdout:2/790: dwrite d2/da/d2c/f38 [0,4194304] 0 2024-05-27T09:29:33.340 INFO:tasks.workunit.client.0.smithi119.stdout:8/717: sync 2024-05-27T09:29:33.340 INFO:tasks.workunit.client.0.smithi119.stdout:1/809: write d7/dd/d44/f4e [2663546,74738] 0 2024-05-27T09:29:33.340 INFO:tasks.workunit.client.0.smithi119.stdout:3/892: sync 2024-05-27T09:29:33.340 INFO:tasks.workunit.client.0.smithi119.stdout:2/791: symlink d2/da/d8a/dd4/d69/df5/d108/l10c 0 2024-05-27T09:29:33.340 INFO:tasks.workunit.client.0.smithi119.stdout:1/810: symlink d7/db/d12/d15/lf8 0 2024-05-27T09:29:33.341 INFO:tasks.workunit.client.1.smithi138.stdout:9/227: mkdir d6/db/de/d18/d27/d46 0 2024-05-27T09:29:33.341 INFO:tasks.workunit.client.1.smithi138.stdout:9/228: chown d6/f3f 1325383 1 2024-05-27T09:29:33.341 INFO:tasks.workunit.client.1.smithi138.stdout:9/229: stat d6/f9 0 2024-05-27T09:29:33.341 INFO:tasks.workunit.client.1.smithi138.stdout:3/183: mkdir d0/d3f 0 2024-05-27T09:29:33.341 INFO:tasks.workunit.client.1.smithi138.stdout:3/184: truncate d0/f12 1026163 0 2024-05-27T09:29:33.341 INFO:tasks.workunit.client.1.smithi138.stdout:3/185: chown d0/d2/da/cb 2806 1 2024-05-27T09:29:33.342 INFO:tasks.workunit.client.0.smithi119.stdout:3/893: truncate db/dc9/d98/faf 905341 0 2024-05-27T09:29:33.345 INFO:tasks.workunit.client.1.smithi138.stdout:9/230: creat d6/db/de/d18/d27/d46/f47 x:0 0 0 2024-05-27T09:29:33.345 INFO:tasks.workunit.client.1.smithi138.stdout:9/231: write d6/db/dc/f2c [909098,86895] 0 2024-05-27T09:29:33.345 INFO:tasks.workunit.client.1.smithi138.stdout:9/232: creat d6/f48 x:0 0 0 2024-05-27T09:29:33.347 INFO:tasks.workunit.client.0.smithi119.stdout:3/894: link db/d36/fbc db/d7f/dd2/d11c/f122 0 2024-05-27T09:29:33.347 INFO:tasks.workunit.client.0.smithi119.stdout:3/895: chown db/d36/d45/d4c/d62/d13/c42 0 1 2024-05-27T09:29:33.349 INFO:tasks.workunit.client.1.smithi138.stdout:3/186: symlink d0/d2/da/df/d10/d32/d37/l40 0 2024-05-27T09:29:33.362 INFO:tasks.workunit.client.1.smithi138.stdout:6/345: dwrite d3/d19/f63 [0,4194304] 0 2024-05-27T09:29:33.364 INFO:tasks.workunit.client.0.smithi119.stdout:0/760: dwrite d2/d6/d75/fa2 [0,4194304] 0 2024-05-27T09:29:33.366 INFO:tasks.workunit.client.0.smithi119.stdout:2/792: dread d2/da/d6f/f7f [0,4194304] 0 2024-05-27T09:29:33.366 INFO:tasks.workunit.client.0.smithi119.stdout:4/824: dread d4/d23/d24/d5b/f78 [0,4194304] 0 2024-05-27T09:29:33.368 INFO:tasks.workunit.client.0.smithi119.stdout:0/761: mkdir d2/d6/d9/d12/de5/dfa 0 2024-05-27T09:29:33.371 INFO:tasks.workunit.client.0.smithi119.stdout:2/793: creat d2/da/d15/d53/dac/f10d x:0 0 0 2024-05-27T09:29:33.372 INFO:tasks.workunit.client.0.smithi119.stdout:0/762: mknod d2/d6/d9/d12/cfb 0 2024-05-27T09:29:33.373 INFO:tasks.workunit.client.1.smithi138.stdout:4/272: dread d7/d27/f48 [0,4194304] 0 2024-05-27T09:29:33.374 INFO:tasks.workunit.client.0.smithi119.stdout:1/811: dread d7/dd/f77 [0,4194304] 0 2024-05-27T09:29:33.374 INFO:tasks.workunit.client.0.smithi119.stdout:1/812: fdatasync d7/db/d52/d65/ff4 0 2024-05-27T09:29:33.380 INFO:tasks.workunit.client.0.smithi119.stdout:4/825: rmdir d4/de/d21/d40/d47/da3 39 2024-05-27T09:29:33.382 INFO:tasks.workunit.client.0.smithi119.stdout:1/813: stat d7/db/d12/ddb/d38/d49/c4b 0 2024-05-27T09:29:33.382 INFO:tasks.workunit.client.0.smithi119.stdout:1/814: dread - d7/db/d8f/d9b/fb9 zero size 2024-05-27T09:29:33.384 INFO:tasks.workunit.client.1.smithi138.stdout:4/273: unlink d7/df/c15 0 2024-05-27T09:29:33.386 INFO:tasks.workunit.client.0.smithi119.stdout:1/815: mkdir d7/d40/df9 0 2024-05-27T09:29:33.390 INFO:tasks.workunit.client.1.smithi138.stdout:4/274: creat d7/df/d35/f5c x:0 0 0 2024-05-27T09:29:33.390 INFO:tasks.workunit.client.0.smithi119.stdout:4/826: unlink d4/d6/d3e/d89/cd8 0 2024-05-27T09:29:33.390 INFO:tasks.workunit.client.0.smithi119.stdout:4/827: chown d4/d6/d3e/d89/lbc 235 1 2024-05-27T09:29:33.392 INFO:tasks.workunit.client.0.smithi119.stdout:4/828: creat d4/de/dfb/f103 x:0 0 0 2024-05-27T09:29:33.397 INFO:tasks.workunit.client.0.smithi119.stdout:9/663: dwrite d18/d19/fab [0,4194304] 0 2024-05-27T09:29:33.397 INFO:tasks.workunit.client.0.smithi119.stdout:9/664: chown d18/d19 22846 1 2024-05-27T09:29:33.397 INFO:tasks.workunit.client.0.smithi119.stdout:4/829: mkdir d4/d6/d3e/df5/d104 0 2024-05-27T09:29:33.406 INFO:tasks.workunit.client.1.smithi138.stdout:1/281: dwrite d4/d3c/f42 [0,4194304] 0 2024-05-27T09:29:33.412 INFO:tasks.workunit.client.0.smithi119.stdout:9/665: rename f15 to d18/d19/d1b/d29/fc9 0 2024-05-27T09:29:33.422 INFO:tasks.workunit.client.0.smithi119.stdout:9/666: mknod d18/db4/dbe/cca 0 2024-05-27T09:29:33.423 INFO:tasks.workunit.client.0.smithi119.stdout:9/667: write d18/d19/d1b/d40/db7/fbb [49640,53331] 0 2024-05-27T09:29:33.428 INFO:tasks.workunit.client.0.smithi119.stdout:9/668: mkdir d18/d19/d1b/d40/d44/dcb 0 2024-05-27T09:29:33.433 INFO:tasks.workunit.client.0.smithi119.stdout:7/819: dwrite d7/df/d19/d9a/da6/dda/d44/ffd [0,4194304] 0 2024-05-27T09:29:33.433 INFO:tasks.workunit.client.0.smithi119.stdout:7/820: fsync d7/df/d19/d97/d92/db7/dba/d42/d64/de3/fe7 0 2024-05-27T09:29:33.433 INFO:tasks.workunit.client.0.smithi119.stdout:7/821: write d7/df/d19/d9a/da6/dda/d44/f5d [3828987,101535] 0 2024-05-27T09:29:33.448 INFO:tasks.workunit.client.1.smithi138.stdout:9/233: dwrite d6/f48 [0,4194304] 0 2024-05-27T09:29:33.448 INFO:tasks.workunit.client.1.smithi138.stdout:9/234: getdents d6/da/d26/d28 0 2024-05-27T09:29:33.448 INFO:tasks.workunit.client.1.smithi138.stdout:9/235: write d6/da/d2f/f32 [5120399,87387] 0 2024-05-27T09:29:33.455 INFO:tasks.workunit.client.0.smithi119.stdout:4/830: dread d4/de/d21/d40/d47/d5c/d9e/fe1 [0,4194304] 0 2024-05-27T09:29:33.455 INFO:tasks.workunit.client.0.smithi119.stdout:4/831: creat d4/d23/d24/d5b/f105 x:0 0 0 2024-05-27T09:29:33.456 INFO:tasks.workunit.client.0.smithi119.stdout:8/718: dwrite d3/d8/f7e [0,4194304] 0 2024-05-27T09:29:33.457 INFO:tasks.workunit.client.0.smithi119.stdout:8/719: creat d3/d8/d1b/db1/ff0 x:0 0 0 2024-05-27T09:29:33.461 INFO:tasks.workunit.client.0.smithi119.stdout:4/832: mknod d4/d6/d3e/df5/c106 0 2024-05-27T09:29:33.470 INFO:tasks.workunit.client.1.smithi138.stdout:3/187: dwrite d0/d2/f22 [0,4194304] 0 2024-05-27T09:29:33.470 INFO:tasks.workunit.client.1.smithi138.stdout:9/236: creat d6/da/d26/d28/f49 x:0 0 0 2024-05-27T09:29:33.470 INFO:tasks.workunit.client.1.smithi138.stdout:9/237: readlink d6/da/d2f/l42 0 2024-05-27T09:29:33.471 INFO:tasks.workunit.client.0.smithi119.stdout:8/720: symlink d3/d8/d4d/d5d/db7/de0/lf1 0 2024-05-27T09:29:33.475 INFO:tasks.workunit.client.0.smithi119.stdout:0/763: dwrite d2/d6/d9/d24/d69/fe1 [4194304,4194304] 0 2024-05-27T09:29:33.477 INFO:tasks.workunit.client.0.smithi119.stdout:8/721: symlink d3/d6e/d8e/lf2 0 2024-05-27T09:29:33.478 INFO:tasks.workunit.client.0.smithi119.stdout:8/722: chown d3/d8/d4d/d58/f63 204873 1 2024-05-27T09:29:33.478 INFO:tasks.workunit.client.0.smithi119.stdout:8/723: readlink d3/d6e/d80/la1 0 2024-05-27T09:29:33.478 INFO:tasks.workunit.client.0.smithi119.stdout:8/724: stat d3/faf 0 2024-05-27T09:29:33.478 INFO:tasks.workunit.client.1.smithi138.stdout:3/188: rename d0/c18 to d0/d2/da/df/d10/d32/d37/c41 0 2024-05-27T09:29:33.481 INFO:tasks.workunit.client.1.smithi138.stdout:9/238: unlink d6/db/f22 0 2024-05-27T09:29:33.486 INFO:tasks.workunit.client.0.smithi119.stdout:3/896: dwrite db/d36/d45/d4c/dbe/fd7 [0,4194304] 0 2024-05-27T09:29:33.487 INFO:tasks.workunit.client.0.smithi119.stdout:7/822: dread d7/df/d19/d46/f68 [0,4194304] 0 2024-05-27T09:29:33.488 INFO:tasks.workunit.client.1.smithi138.stdout:3/189: truncate d0/d2/f13 1777062 0 2024-05-27T09:29:33.488 INFO:tasks.workunit.client.1.smithi138.stdout:3/190: chown d0/l5 75 1 2024-05-27T09:29:33.489 INFO:tasks.workunit.client.0.smithi119.stdout:1/816: dwrite d7/db/d12/d15/d1f/fbb [0,4194304] 0 2024-05-27T09:29:33.489 INFO:tasks.workunit.client.0.smithi119.stdout:8/725: rename d3/d8/d4d/d5d/d60/fda to d3/ff3 0 2024-05-27T09:29:33.489 INFO:tasks.workunit.client.0.smithi119.stdout:1/817: fdatasync f6 0 2024-05-27T09:29:33.491 INFO:tasks.workunit.client.1.smithi138.stdout:9/239: rename d6/c7 to d6/d25/c4a 0 2024-05-27T09:29:33.492 INFO:tasks.workunit.client.0.smithi119.stdout:7/823: mkdir d7/d1a/d107 0 2024-05-27T09:29:33.495 INFO:tasks.workunit.client.1.smithi138.stdout:9/240: link d6/db/dc/f1c d6/db/de/d18/d24/f4b 0 2024-05-27T09:29:33.495 INFO:tasks.workunit.client.1.smithi138.stdout:9/241: fdatasync d6/db/f39 0 2024-05-27T09:29:33.498 INFO:tasks.workunit.client.1.smithi138.stdout:8/223: sync 2024-05-27T09:29:33.498 INFO:tasks.workunit.client.1.smithi138.stdout:5/228: sync 2024-05-27T09:29:33.499 INFO:tasks.workunit.client.1.smithi138.stdout:5/229: write d3/d12/d16/d1d/d22/d2b/f40 [653811,8350] 0 2024-05-27T09:29:33.499 INFO:tasks.workunit.client.1.smithi138.stdout:5/230: fdatasync d3/f18 0 2024-05-27T09:29:33.499 INFO:tasks.workunit.client.1.smithi138.stdout:2/200: sync 2024-05-27T09:29:33.499 INFO:tasks.workunit.client.1.smithi138.stdout:7/239: sync 2024-05-27T09:29:33.499 INFO:tasks.workunit.client.0.smithi119.stdout:7/824: mknod d7/d5f/d79/c108 0 2024-05-27T09:29:33.499 INFO:tasks.workunit.client.0.smithi119.stdout:1/818: symlink d7/db/d35/lfa 0 2024-05-27T09:29:33.499 INFO:tasks.workunit.client.0.smithi119.stdout:1/819: fdatasync d7/db6/fbc 0 2024-05-27T09:29:33.500 INFO:tasks.workunit.client.0.smithi119.stdout:8/726: unlink d3/de/f1c 0 2024-05-27T09:29:33.502 INFO:tasks.workunit.client.0.smithi119.stdout:1/820: unlink d7/db/d12/d15/dd2/fdf 0 2024-05-27T09:29:33.506 INFO:tasks.workunit.client.0.smithi119.stdout:8/727: mknod d3/d8/d4d/d5d/d74/cf4 0 2024-05-27T09:29:33.506 INFO:tasks.workunit.client.0.smithi119.stdout:8/728: chown d3/d8/d4d/d5d/d66/d85/f9e 1013 1 2024-05-27T09:29:33.508 INFO:tasks.workunit.client.0.smithi119.stdout:1/821: truncate d7/db/d12/ddb/d38/d3c/fbf 2652569 0 2024-05-27T09:29:33.513 INFO:tasks.workunit.client.1.smithi138.stdout:9/242: mkdir d6/db/de/d18/d27/d4c 0 2024-05-27T09:29:33.515 INFO:tasks.workunit.client.0.smithi119.stdout:8/729: mknod d3/d6e/d80/cf5 0 2024-05-27T09:29:33.515 INFO:tasks.workunit.client.0.smithi119.stdout:1/822: rename d7/db/d12/d15/f8e to d7/dd/d36/ffb 0 2024-05-27T09:29:33.517 INFO:tasks.workunit.client.1.smithi138.stdout:8/224: truncate d3/d8/d28/f2f 120686 0 2024-05-27T09:29:33.517 INFO:tasks.workunit.client.1.smithi138.stdout:8/225: chown d3/d8/d24 100340 1 2024-05-27T09:29:33.519 INFO:tasks.workunit.client.0.smithi119.stdout:1/823: symlink d7/db/d35/lfc 0 2024-05-27T09:29:33.523 INFO:tasks.workunit.client.0.smithi119.stdout:8/730: write d3/d8/d1b/db1/fc5 [440860,11368] 0 2024-05-27T09:29:33.524 INFO:tasks.workunit.client.0.smithi119.stdout:1/824: mkdir d7/db/d12/ddb/d38/d3c/dfd 0 2024-05-27T09:29:33.524 INFO:tasks.workunit.client.1.smithi138.stdout:2/201: truncate d1/d15/d2d/d17/d25/f2a 81117 0 2024-05-27T09:29:33.525 INFO:tasks.workunit.client.1.smithi138.stdout:2/202: write d1/d15/d2d/d17/d25/f32 [121027,97801] 0 2024-05-27T09:29:33.525 INFO:tasks.workunit.client.1.smithi138.stdout:4/275: dwrite d7/df/f19 [4194304,4194304] 0 2024-05-27T09:29:33.525 INFO:tasks.workunit.client.1.smithi138.stdout:4/276: readlink d7/df/d35/d4c/l57 0 2024-05-27T09:29:33.525 INFO:tasks.workunit.client.1.smithi138.stdout:4/277: stat d7/df/f52 0 2024-05-27T09:29:33.526 INFO:tasks.workunit.client.1.smithi138.stdout:5/231: write d3/f19 [1584078,25198] 0 2024-05-27T09:29:33.527 INFO:tasks.workunit.client.0.smithi119.stdout:3/897: write db/d36/d45/d4c/d62/f17 [3473121,15423] 0 2024-05-27T09:29:33.529 INFO:tasks.workunit.client.0.smithi119.stdout:1/825: unlink d7/dd/d25/d60/d61/c66 0 2024-05-27T09:29:33.529 INFO:tasks.workunit.client.1.smithi138.stdout:4/278: write f3 [2876635,23733] 0 2024-05-27T09:29:33.530 INFO:tasks.workunit.client.0.smithi119.stdout:1/826: write d7/db/d12/d26/d4a/fd6 [372300,124867] 0 2024-05-27T09:29:33.531 INFO:tasks.workunit.client.1.smithi138.stdout:1/282: dwrite d4/d1a/f27 [0,4194304] 0 2024-05-27T09:29:33.531 INFO:tasks.workunit.client.1.smithi138.stdout:1/283: stat d4/d3c/f4e 0 2024-05-27T09:29:33.532 INFO:tasks.workunit.client.0.smithi119.stdout:0/764: dread d2/d6/f8 [4194304,4194304] 0 2024-05-27T09:29:33.532 INFO:tasks.workunit.client.0.smithi119.stdout:0/765: chown d2/d6/d23/d7a/d53/dd1 6970 1 2024-05-27T09:29:33.533 INFO:tasks.workunit.client.1.smithi138.stdout:8/226: mkdir d3/d8/d32/d3f 0 2024-05-27T09:29:33.533 INFO:tasks.workunit.client.1.smithi138.stdout:8/227: dread - d3/f29 zero size 2024-05-27T09:29:33.533 INFO:tasks.workunit.client.1.smithi138.stdout:8/228: chown d3/d8/cb 22921119 1 2024-05-27T09:29:33.533 INFO:tasks.workunit.client.1.smithi138.stdout:9/243: getdents d6/db/de 0 2024-05-27T09:29:33.536 INFO:tasks.workunit.client.1.smithi138.stdout:5/232: creat d3/d12/f4d x:0 0 0 2024-05-27T09:29:33.536 INFO:tasks.workunit.client.1.smithi138.stdout:2/203: symlink d1/d10/d3c/l40 0 2024-05-27T09:29:33.539 INFO:tasks.workunit.client.1.smithi138.stdout:4/279: dread d7/d27/f48 [0,4194304] 0 2024-05-27T09:29:33.540 INFO:tasks.workunit.client.1.smithi138.stdout:4/280: write d7/df/f1a [2707502,78502] 0 2024-05-27T09:29:33.543 INFO:tasks.workunit.client.1.smithi138.stdout:8/229: unlink d3/d8/ca 0 2024-05-27T09:29:33.543 INFO:tasks.workunit.client.1.smithi138.stdout:9/244: creat d6/db/dc/f4d x:0 0 0 2024-05-27T09:29:33.544 INFO:tasks.workunit.client.1.smithi138.stdout:9/245: chown f1 65213 1 2024-05-27T09:29:33.544 INFO:tasks.workunit.client.1.smithi138.stdout:8/230: creat d3/d8/d28/f40 x:0 0 0 2024-05-27T09:29:33.544 INFO:tasks.workunit.client.0.smithi119.stdout:9/669: dwrite d18/f32 [4194304,4194304] 0 2024-05-27T09:29:33.544 INFO:tasks.workunit.client.0.smithi119.stdout:9/670: dread - d18/fa6 zero size 2024-05-27T09:29:33.544 INFO:tasks.workunit.client.0.smithi119.stdout:9/671: fsync d18/d19/d9b/fb3 0 2024-05-27T09:29:33.545 INFO:tasks.workunit.client.1.smithi138.stdout:2/204: mknod d1/d10/c41 0 2024-05-27T09:29:33.545 INFO:tasks.workunit.client.1.smithi138.stdout:5/233: rename d3/d12/d16/d1d/f1e to d3/d12/d32/f4e 0 2024-05-27T09:29:33.546 INFO:tasks.workunit.client.1.smithi138.stdout:5/234: creat d3/d12/d32/f4f x:0 0 0 2024-05-27T09:29:33.551 INFO:tasks.workunit.client.1.smithi138.stdout:9/246: symlink d6/da/d2f/l4e 0 2024-05-27T09:29:33.551 INFO:tasks.workunit.client.1.smithi138.stdout:2/205: symlink d1/l42 0 2024-05-27T09:29:33.551 INFO:tasks.workunit.client.1.smithi138.stdout:2/206: fsync d1/d10/f1a 0 2024-05-27T09:29:33.554 INFO:tasks.workunit.client.1.smithi138.stdout:3/191: dwrite d0/d2/da/f33 [0,4194304] 0 2024-05-27T09:29:33.554 INFO:tasks.workunit.client.1.smithi138.stdout:3/192: getdents d0/d3f 0 2024-05-27T09:29:33.555 INFO:tasks.workunit.client.1.smithi138.stdout:3/193: write d0/d2/f13 [2691067,89407] 0 2024-05-27T09:29:33.555 INFO:tasks.workunit.client.1.smithi138.stdout:3/194: chown d0/d2/c6 4675214 1 2024-05-27T09:29:33.555 INFO:tasks.workunit.client.1.smithi138.stdout:3/195: chown d0/d2 13 1 2024-05-27T09:29:33.558 INFO:tasks.workunit.client.0.smithi119.stdout:8/731: read d3/d6e/f75 [3664378,54947] 0 2024-05-27T09:29:33.560 INFO:tasks.workunit.client.0.smithi119.stdout:1/827: read d7/db/d12/d15/d1f/f8b [3477224,114584] 0 2024-05-27T09:29:33.561 INFO:tasks.workunit.client.0.smithi119.stdout:4/833: dwrite d4/d23/d24/d5b/da8/dda/fe2 [0,4194304] 0 2024-05-27T09:29:33.561 INFO:tasks.workunit.client.0.smithi119.stdout:4/834: fsync d4/d23/d24/d5b/da8/fe7 0 2024-05-27T09:29:33.561 INFO:tasks.workunit.client.0.smithi119.stdout:4/835: write d4/d23/fdf [1032953,38260] 0 2024-05-27T09:29:33.563 INFO:tasks.workunit.client.1.smithi138.stdout:2/207: rename d1/d15/d2d/d17/d25/f32 to d1/d15/d2d/d1d/f43 0 2024-05-27T09:29:33.563 INFO:tasks.workunit.client.0.smithi119.stdout:3/898: write db/d36/d45/d4c/d62/d13/f97 [1927244,43187] 0 2024-05-27T09:29:33.564 INFO:tasks.workunit.client.1.smithi138.stdout:2/208: creat d1/d15/d2d/d31/f44 x:0 0 0 2024-05-27T09:29:33.567 INFO:tasks.workunit.client.1.smithi138.stdout:2/209: read - d1/d15/d2d/d31/f44 zero size 2024-05-27T09:29:33.567 INFO:tasks.workunit.client.1.smithi138.stdout:2/210: readlink d1/d15/l1e 0 2024-05-27T09:29:33.569 INFO:tasks.workunit.client.1.smithi138.stdout:2/211: unlink d1/d15/d2d/d31/l3f 0 2024-05-27T09:29:33.570 INFO:tasks.workunit.client.0.smithi119.stdout:8/732: symlink d3/d8/d4d/d5d/lf6 0 2024-05-27T09:29:33.572 INFO:tasks.workunit.client.1.smithi138.stdout:5/235: rmdir d3/d12/d16/d1d/d30 39 2024-05-27T09:29:33.574 INFO:tasks.workunit.client.0.smithi119.stdout:0/766: dread d2/d6/d75/fa4 [0,4194304] 0 2024-05-27T09:29:33.574 INFO:tasks.workunit.client.0.smithi119.stdout:0/767: dread - d2/f94 zero size 2024-05-27T09:29:33.575 INFO:tasks.workunit.client.0.smithi119.stdout:1/828: mknod d7/dd/db0/dcd/cfe 0 2024-05-27T09:29:33.575 INFO:tasks.workunit.client.0.smithi119.stdout:1/829: stat d7/dd/d44 0 2024-05-27T09:29:33.576 INFO:tasks.workunit.client.1.smithi138.stdout:5/236: write d3/d12/d16/d1d/d22/d2b/f40 [216114,15925] 0 2024-05-27T09:29:33.576 INFO:tasks.workunit.client.1.smithi138.stdout:4/281: dread d7/d27/f2c [0,4194304] 0 2024-05-27T09:29:33.579 INFO:tasks.workunit.client.0.smithi119.stdout:4/836: creat d4/de/f107 x:0 0 0 2024-05-27T09:29:33.584 INFO:tasks.workunit.client.0.smithi119.stdout:3/899: rmdir db/d36/d45/d4c/d62/d13/d33/dfc/de2 0 2024-05-27T09:29:33.589 INFO:tasks.workunit.client.0.smithi119.stdout:7/825: dwrite d7/d1a/d99/fee [0,4194304] 0 2024-05-27T09:29:33.591 INFO:tasks.workunit.client.0.smithi119.stdout:8/733: getdents d3/d6e/d80/d84 0 2024-05-27T09:29:33.591 INFO:tasks.workunit.client.0.smithi119.stdout:8/734: write d3/d8/d1b/db1/ff0 [653328,94434] 0 2024-05-27T09:29:33.591 INFO:tasks.workunit.client.0.smithi119.stdout:8/735: write d3/d8/d1b/d77/fdb [563933,27842] 0 2024-05-27T09:29:33.594 INFO:tasks.workunit.client.0.smithi119.stdout:5/853: sync 2024-05-27T09:29:33.594 INFO:tasks.workunit.client.0.smithi119.stdout:6/751: sync 2024-05-27T09:29:33.594 INFO:tasks.workunit.client.0.smithi119.stdout:5/854: fdatasync d6/dd/d39/f69 0 2024-05-27T09:29:33.594 INFO:tasks.workunit.client.0.smithi119.stdout:5/855: truncate d6/f63 4848062 0 2024-05-27T09:29:33.595 INFO:tasks.workunit.client.0.smithi119.stdout:0/768: creat d2/d6/ffc x:0 0 0 2024-05-27T09:29:33.600 INFO:tasks.workunit.client.0.smithi119.stdout:4/837: truncate d4/d6/d28/f6b 209530 0 2024-05-27T09:29:33.600 INFO:tasks.workunit.client.0.smithi119.stdout:4/838: write d4/de/d9b/ff1 [795844,79106] 0 2024-05-27T09:29:33.604 INFO:tasks.workunit.client.0.smithi119.stdout:3/900: creat db/d7f/f123 x:0 0 0 2024-05-27T09:29:33.615 INFO:tasks.workunit.client.1.smithi138.stdout:2/212: dread d1/d15/d2d/d31/f3b [0,4194304] 0 2024-05-27T09:29:33.615 INFO:tasks.workunit.client.1.smithi138.stdout:2/213: truncate d1/d15/d2d/d31/f44 910331 0 2024-05-27T09:29:33.615 INFO:tasks.workunit.client.0.smithi119.stdout:7/826: truncate d7/df/d19/d9a/da6/dda/d44/fde 1861455 0 2024-05-27T09:29:33.618 INFO:tasks.workunit.client.0.smithi119.stdout:8/736: rename d3/d8/d1b/d46/ca8 to d3/d8/d4d/d79/cf7 0 2024-05-27T09:29:33.618 INFO:tasks.workunit.client.0.smithi119.stdout:8/737: stat d3/d8/d4d/d5d/d66/d85/lde 0 2024-05-27T09:29:33.628 INFO:tasks.workunit.client.0.smithi119.stdout:6/752: dread d0/d2/d57/d37/f5d [0,4194304] 0 2024-05-27T09:29:33.630 INFO:tasks.workunit.client.1.smithi138.stdout:0/276: sync 2024-05-27T09:29:33.630 INFO:tasks.workunit.client.1.smithi138.stdout:6/346: sync 2024-05-27T09:29:33.631 INFO:tasks.workunit.client.1.smithi138.stdout:0/277: truncate da/d2a/d2b/f50 926066 0 2024-05-27T09:29:33.638 INFO:tasks.workunit.client.1.smithi138.stdout:8/231: dwrite d3/d8/d11/f26 [0,4194304] 0 2024-05-27T09:29:33.641 INFO:tasks.workunit.client.1.smithi138.stdout:6/347: mknod d3/d35/d43/c7b 0 2024-05-27T09:29:33.651 INFO:tasks.workunit.client.0.smithi119.stdout:7/827: truncate d7/fa 3980100 0 2024-05-27T09:29:33.652 INFO:tasks.workunit.client.0.smithi119.stdout:2/794: sync 2024-05-27T09:29:33.652 INFO:tasks.workunit.client.0.smithi119.stdout:3/901: mknod db/d36/d45/d4c/d62/d13/d33/dfc/c124 0 2024-05-27T09:29:33.652 INFO:tasks.workunit.client.0.smithi119.stdout:2/795: readlink d2/da/d15/d53/dac/lb8 0 2024-05-27T09:29:33.652 INFO:tasks.workunit.client.0.smithi119.stdout:3/902: stat db/d36/d45/d4c/d62/d13/f2e 0 2024-05-27T09:29:33.653 INFO:tasks.workunit.client.1.smithi138.stdout:8/232: mknod d3/d8/d32/d3f/c41 0 2024-05-27T09:29:33.653 INFO:tasks.workunit.client.0.smithi119.stdout:8/738: write d3/d8/d4d/f32 [13060462,11269] 0 2024-05-27T09:29:33.653 INFO:tasks.workunit.client.0.smithi119.stdout:8/739: truncate d3/d8/d4d/d5d/db7/fee 521532 0 2024-05-27T09:29:33.653 INFO:tasks.workunit.client.0.smithi119.stdout:8/740: fdatasync d3/d8/d1b/f7a 0 2024-05-27T09:29:33.653 INFO:tasks.workunit.client.0.smithi119.stdout:8/741: chown d3/d6e/d80/d84/lcb 449697 1 2024-05-27T09:29:33.653 INFO:tasks.workunit.client.0.smithi119.stdout:8/742: dread - d3/d8/d4d/d79/fbd zero size 2024-05-27T09:29:33.654 INFO:tasks.workunit.client.1.smithi138.stdout:7/240: dwrite d0/f10 [0,4194304] 0 2024-05-27T09:29:33.654 INFO:tasks.workunit.client.1.smithi138.stdout:7/241: write d0/f36 [2101431,48726] 0 2024-05-27T09:29:33.657 INFO:tasks.workunit.client.1.smithi138.stdout:7/242: mknod d0/d23/d3c/c4d 0 2024-05-27T09:29:33.657 INFO:tasks.workunit.client.1.smithi138.stdout:7/243: write d0/d2b/f40 [795423,50711] 0 2024-05-27T09:29:33.660 INFO:tasks.workunit.client.1.smithi138.stdout:7/244: symlink d0/d23/d3c/d3f/l4e 0 2024-05-27T09:29:33.661 INFO:tasks.workunit.client.0.smithi119.stdout:1/830: dread d7/f5b [0,4194304] 0 2024-05-27T09:29:33.662 INFO:tasks.workunit.client.1.smithi138.stdout:5/237: dwrite d3/d12/d32/f47 [0,4194304] 0 2024-05-27T09:29:33.663 INFO:tasks.workunit.client.1.smithi138.stdout:7/245: creat d0/d23/f4f x:0 0 0 2024-05-27T09:29:33.664 INFO:tasks.workunit.client.1.smithi138.stdout:9/247: dread d6/da/d2f/f32 [0,4194304] 0 2024-05-27T09:29:33.665 INFO:tasks.workunit.client.1.smithi138.stdout:9/248: chown d6/db/de/d18/c2a 0 1 2024-05-27T09:29:33.665 INFO:tasks.workunit.client.1.smithi138.stdout:9/249: read d6/da/f44 [189134,72530] 0 2024-05-27T09:29:33.665 INFO:tasks.workunit.client.1.smithi138.stdout:9/250: chown d6/da/d29/c3e 0 1 2024-05-27T09:29:33.665 INFO:tasks.workunit.client.1.smithi138.stdout:7/246: creat d0/d23/f50 x:0 0 0 2024-05-27T09:29:33.665 INFO:tasks.workunit.client.1.smithi138.stdout:7/247: truncate d0/f1a 4483702 0 2024-05-27T09:29:33.669 INFO:tasks.workunit.client.1.smithi138.stdout:9/251: creat d6/da/d26/f4f x:0 0 0 2024-05-27T09:29:33.669 INFO:tasks.workunit.client.1.smithi138.stdout:7/248: getdents d0 0 2024-05-27T09:29:33.671 INFO:tasks.workunit.client.1.smithi138.stdout:7/249: mknod d0/d7/d43/c51 0 2024-05-27T09:29:33.674 INFO:tasks.workunit.client.1.smithi138.stdout:4/282: dwrite d7/d27/d3a/f55 [0,4194304] 0 2024-05-27T09:29:33.676 INFO:tasks.workunit.client.0.smithi119.stdout:7/828: link d7/df/d19/d97/d92/db7/dba/f52 d7/df/d19/d78/f109 0 2024-05-27T09:29:33.682 INFO:tasks.workunit.client.0.smithi119.stdout:2/796: rename d2/da/d8a/dd4/d89/d94/d81/fa2 to d2/da/d8a/d8f/f10e 0 2024-05-27T09:29:33.682 INFO:tasks.workunit.client.0.smithi119.stdout:2/797: readlink d2/da/d2c/lab 0 2024-05-27T09:29:33.687 INFO:tasks.workunit.client.1.smithi138.stdout:8/233: creat d3/d8/f42 x:0 0 0 2024-05-27T09:29:33.689 INFO:tasks.workunit.client.1.smithi138.stdout:5/238: rename d3/l2d to d3/d12/d32/l50 0 2024-05-27T09:29:33.700 INFO:tasks.workunit.client.1.smithi138.stdout:9/252: dread d6/f48 [0,4194304] 0 2024-05-27T09:29:33.700 INFO:tasks.workunit.client.1.smithi138.stdout:9/253: fdatasync d6/da/d2f/f32 0 2024-05-27T09:29:33.700 INFO:tasks.workunit.client.1.smithi138.stdout:9/254: chown d6/db/de/d18/d27/d46/f47 5612 1 2024-05-27T09:29:33.700 INFO:tasks.workunit.client.1.smithi138.stdout:9/255: chown d6/f9 5 1 2024-05-27T09:29:33.701 INFO:tasks.workunit.client.1.smithi138.stdout:2/214: dwrite d1/d2f/f33 [0,4194304] 0 2024-05-27T09:29:33.704 INFO:tasks.workunit.client.1.smithi138.stdout:2/215: creat d1/d15/d2d/d1d/f45 x:0 0 0 2024-05-27T09:29:33.704 INFO:tasks.workunit.client.1.smithi138.stdout:2/216: write d1/d15/d2d/d17/d25/f2a [1073680,106353] 0 2024-05-27T09:29:33.705 INFO:tasks.workunit.client.1.smithi138.stdout:2/217: creat d1/d2f/f46 x:0 0 0 2024-05-27T09:29:33.705 INFO:tasks.workunit.client.1.smithi138.stdout:2/218: dread - d1/d15/d2d/f3e zero size 2024-05-27T09:29:33.706 INFO:tasks.workunit.client.1.smithi138.stdout:2/219: mknod d1/d15/d2d/d17/c47 0 2024-05-27T09:29:33.708 INFO:tasks.workunit.client.0.smithi119.stdout:0/769: dwrite d2/d6/d9/d12/d4f/f89 [0,4194304] 0 2024-05-27T09:29:33.709 INFO:tasks.workunit.client.0.smithi119.stdout:0/770: creat d2/d6/d9/d12/d4b/d59/dd7/ffd x:0 0 0 2024-05-27T09:29:33.711 INFO:tasks.workunit.client.1.smithi138.stdout:2/220: write d1/d15/d2d/d17/d25/f2b [760618,31436] 0 2024-05-27T09:29:33.711 INFO:tasks.workunit.client.1.smithi138.stdout:2/221: write d1/d15/d2d/d17/d25/f3a [372342,124733] 0 2024-05-27T09:29:33.711 INFO:tasks.workunit.client.1.smithi138.stdout:2/222: creat d1/d15/f48 x:0 0 0 2024-05-27T09:29:33.711 INFO:tasks.workunit.client.1.smithi138.stdout:2/223: chown d1/d15/d2d/c2c 2621 1 2024-05-27T09:29:33.711 INFO:tasks.workunit.client.1.smithi138.stdout:2/224: readlink d1/d15/d2d/l20 0 2024-05-27T09:29:33.711 INFO:tasks.workunit.client.1.smithi138.stdout:4/283: mkdir d7/df/d35/d4c/d59/d5d 0 2024-05-27T09:29:33.711 INFO:tasks.workunit.client.1.smithi138.stdout:4/284: fsync d7/df/f41 0 2024-05-27T09:29:33.716 INFO:tasks.workunit.client.1.smithi138.stdout:4/285: truncate d7/f44 597598 0 2024-05-27T09:29:33.721 INFO:tasks.workunit.client.0.smithi119.stdout:1/831: mkdir d7/dd/d44/dff 0 2024-05-27T09:29:33.721 INFO:tasks.workunit.client.0.smithi119.stdout:2/798: mkdir d2/deb/d10f 0 2024-05-27T09:29:33.721 INFO:tasks.workunit.client.0.smithi119.stdout:2/799: chown d2/da/d8a/dd4/d89/d94/d81/f9e 36007 1 2024-05-27T09:29:33.721 INFO:tasks.workunit.client.0.smithi119.stdout:7/829: write d7/df/d19/d97/d92/db7/dba/f61 [188393,55154] 0 2024-05-27T09:29:33.723 INFO:tasks.workunit.client.1.smithi138.stdout:4/286: mkdir d7/df/d5b/d5e 0 2024-05-27T09:29:33.724 INFO:tasks.workunit.client.0.smithi119.stdout:6/753: dwrite d0/d2/d1c/d21/ff9 [4194304,4194304] 0 2024-05-27T09:29:33.732 INFO:tasks.workunit.client.0.smithi119.stdout:2/800: creat d2/da/d8a/dd4/f110 x:0 0 0 2024-05-27T09:29:33.733 INFO:tasks.workunit.client.0.smithi119.stdout:9/672: dwrite d18/d19/d1b/d40/d44/fb0 [4194304,4194304] 0 2024-05-27T09:29:33.733 INFO:tasks.workunit.client.0.smithi119.stdout:9/673: chown d18/d19/d1b/d40/lae 113355 1 2024-05-27T09:29:33.744 INFO:tasks.workunit.client.0.smithi119.stdout:2/801: creat d2/da/d6f/f111 x:0 0 0 2024-05-27T09:29:33.746 INFO:tasks.workunit.client.0.smithi119.stdout:6/754: symlink d0/d2/d57/d37/d74/l10b 0 2024-05-27T09:29:33.746 INFO:tasks.workunit.client.0.smithi119.stdout:6/755: stat d0/d2/fb1 0 2024-05-27T09:29:33.749 INFO:tasks.workunit.client.0.smithi119.stdout:7/830: read d7/df/d19/d97/d92/fd5 [1141620,94852] 0 2024-05-27T09:29:33.749 INFO:tasks.workunit.client.0.smithi119.stdout:7/831: write d7/d1a/d99/dcf/fd2 [4666089,62533] 0 2024-05-27T09:29:33.749 INFO:tasks.workunit.client.0.smithi119.stdout:7/832: creat d7/df/d19/d97/d92/db7/dba/d42/d64/dad/f10a x:0 0 0 2024-05-27T09:29:33.750 INFO:tasks.workunit.client.1.smithi138.stdout:1/284: dwrite d4/d48/d39/f54 [0,4194304] 0 2024-05-27T09:29:33.751 INFO:tasks.workunit.client.1.smithi138.stdout:1/285: write d4/f24 [698789,74717] 0 2024-05-27T09:29:33.751 INFO:tasks.workunit.client.0.smithi119.stdout:2/802: creat d2/f112 x:0 0 0 2024-05-27T09:29:33.751 INFO:tasks.workunit.client.0.smithi119.stdout:2/803: truncate d2/da/d8a/dd4/d89/d94/d87/ff3 286405 0 2024-05-27T09:29:33.759 INFO:tasks.workunit.client.0.smithi119.stdout:6/756: symlink d0/d85/de7/l10c 0 2024-05-27T09:29:33.760 INFO:tasks.workunit.client.1.smithi138.stdout:8/234: dread d3/d8/d11/f26 [0,4194304] 0 2024-05-27T09:29:33.768 INFO:tasks.workunit.client.1.smithi138.stdout:1/286: creat d4/d48/f58 x:0 0 0 2024-05-27T09:29:33.773 INFO:tasks.workunit.client.1.smithi138.stdout:4/287: dread d7/fb [0,4194304] 0 2024-05-27T09:29:33.776 INFO:tasks.workunit.client.1.smithi138.stdout:8/235: symlink d3/d8/d32/d34/d38/l43 0 2024-05-27T09:29:33.778 INFO:tasks.workunit.client.1.smithi138.stdout:9/256: dread f1 [0,4194304] 0 2024-05-27T09:29:33.782 INFO:tasks.workunit.client.1.smithi138.stdout:1/287: dread d4/f37 [0,4194304] 0 2024-05-27T09:29:33.784 INFO:tasks.workunit.client.0.smithi119.stdout:6/757: dread d0/d2/d57/d37/d74/df1/d109/fb0 [0,4194304] 0 2024-05-27T09:29:33.786 INFO:tasks.workunit.client.1.smithi138.stdout:2/225: dwrite d1/d15/f48 [0,4194304] 0 2024-05-27T09:29:33.787 INFO:tasks.workunit.client.0.smithi119.stdout:8/743: dwrite d3/d8/d4d/d5d/d60/f8b [0,4194304] 0 2024-05-27T09:29:33.788 INFO:tasks.workunit.client.1.smithi138.stdout:8/236: creat d3/d8/d24/f44 x:0 0 0 2024-05-27T09:29:33.788 INFO:tasks.workunit.client.1.smithi138.stdout:9/257: rename d6/da/f44 to d6/db/dc/f50 0 2024-05-27T09:29:33.789 INFO:tasks.workunit.client.0.smithi119.stdout:8/744: creat d3/d8/d1b/d46/da4/dbb/ff8 x:0 0 0 2024-05-27T09:29:33.800 INFO:tasks.workunit.client.1.smithi138.stdout:2/226: rename d1/d15/d2d/d31/f44 to d1/d15/d2d/d31/f49 0 2024-05-27T09:29:33.800 INFO:tasks.workunit.client.0.smithi119.stdout:8/745: dread d3/d8/d1b/d46/f2b [0,4194304] 0 2024-05-27T09:29:33.801 INFO:tasks.workunit.client.0.smithi119.stdout:6/758: symlink d0/d2/d4e/d5a/d76/l10d 0 2024-05-27T09:29:33.801 INFO:tasks.workunit.client.1.smithi138.stdout:9/258: rename d6/cd to d6/db/de/c51 0 2024-05-27T09:29:33.804 INFO:tasks.workunit.client.1.smithi138.stdout:8/237: mkdir d3/d8/d32/d45 0 2024-05-27T09:29:33.804 INFO:tasks.workunit.client.1.smithi138.stdout:8/238: rmdir d3/d3a 0 2024-05-27T09:29:33.810 INFO:tasks.workunit.client.1.smithi138.stdout:3/196: dwrite d0/d2/f13 [0,4194304] 0 2024-05-27T09:29:33.822 INFO:tasks.workunit.client.0.smithi119.stdout:8/746: mkdir d3/d8/d1b/db1/df9 0 2024-05-27T09:29:33.822 INFO:tasks.workunit.client.1.smithi138.stdout:3/197: creat d0/d2/da/df/d10/d2a/f42 x:0 0 0 2024-05-27T09:29:33.822 INFO:tasks.workunit.client.0.smithi119.stdout:8/747: write d3/d8/d1b/fed [576281,99761] 0 2024-05-27T09:29:33.823 INFO:tasks.workunit.client.1.smithi138.stdout:3/198: write d0/d2/da/df/d10/d2a/f42 [1036259,115963] 0 2024-05-27T09:29:33.825 INFO:tasks.workunit.client.0.smithi119.stdout:8/748: write d3/de/dad/fc0 [4664215,90124] 0 2024-05-27T09:29:33.825 INFO:tasks.workunit.client.0.smithi119.stdout:6/759: mknod d0/d85/c10e 0 2024-05-27T09:29:33.827 INFO:tasks.workunit.client.1.smithi138.stdout:3/199: mknod d0/d3f/c43 0 2024-05-27T09:29:33.830 INFO:tasks.workunit.client.0.smithi119.stdout:6/760: creat d0/d2/d57/d37/d74/dfe/d10a/f10f x:0 0 0 2024-05-27T09:29:33.834 INFO:tasks.workunit.client.0.smithi119.stdout:9/674: write d18/d19/d1b/d29/fc9 [1915480,61960] 0 2024-05-27T09:29:33.836 INFO:tasks.workunit.client.0.smithi119.stdout:7/833: dwrite d7/df/d19/d46/f85 [0,4194304] 0 2024-05-27T09:29:33.836 INFO:tasks.workunit.client.0.smithi119.stdout:7/834: truncate d7/df/d19/d97/d92/db7/dba/d42/d64/f7a 1389218 0 2024-05-27T09:29:33.836 INFO:tasks.workunit.client.0.smithi119.stdout:7/835: chown d7/d1a/d99/dbb/cea 49640200 1 2024-05-27T09:29:33.837 INFO:tasks.workunit.client.1.smithi138.stdout:3/200: rmdir d0/d2 39 2024-05-27T09:29:33.837 INFO:tasks.workunit.client.1.smithi138.stdout:3/201: chown d0/d2/da/df/d10/d32/le 1574106 1 2024-05-27T09:29:33.837 INFO:tasks.workunit.client.1.smithi138.stdout:3/202: read d0/d2/f27 [273708,69214] 0 2024-05-27T09:29:33.841 INFO:tasks.workunit.client.0.smithi119.stdout:9/675: mkdir d18/d19/d1b/d40/d44/dcc 0 2024-05-27T09:29:33.841 INFO:tasks.workunit.client.0.smithi119.stdout:9/676: chown d18/d19/d1b/d40/d44/l68 13 1 2024-05-27T09:29:33.841 INFO:tasks.workunit.client.0.smithi119.stdout:7/836: rmdir d7/df/d19/d97/d92/db7/dba/d42/d64/de3/de5 0 2024-05-27T09:29:33.845 INFO:tasks.workunit.client.1.smithi138.stdout:3/203: dread d0/d2/f27 [0,4194304] 0 2024-05-27T09:29:33.845 INFO:tasks.workunit.client.1.smithi138.stdout:3/204: write d0/d2/da/df/d10/d32/f1d [643812,3864] 0 2024-05-27T09:29:33.845 INFO:tasks.workunit.client.1.smithi138.stdout:3/205: chown d0/d2/da/df/d10/d32/d37/c41 1186429 1 2024-05-27T09:29:33.846 INFO:tasks.workunit.client.0.smithi119.stdout:2/804: write d2/da/f1c [674102,16876] 0 2024-05-27T09:29:33.846 INFO:tasks.workunit.client.0.smithi119.stdout:2/805: dread - d2/fbe zero size 2024-05-27T09:29:33.846 INFO:tasks.workunit.client.0.smithi119.stdout:4/839: dwrite f3 [0,4194304] 0 2024-05-27T09:29:33.852 INFO:tasks.workunit.client.1.smithi138.stdout:8/239: dread d3/d8/f14 [4194304,4194304] 0 2024-05-27T09:29:33.852 INFO:tasks.workunit.client.1.smithi138.stdout:8/240: write d3/d8/d24/f44 [482435,17438] 0 2024-05-27T09:29:33.854 INFO:tasks.workunit.client.1.smithi138.stdout:2/227: read d1/f1b [6338668,92235] 0 2024-05-27T09:29:33.860 INFO:tasks.workunit.client.0.smithi119.stdout:7/837: dread d7/df/f10 [0,4194304] 0 2024-05-27T09:29:33.860 INFO:tasks.workunit.client.0.smithi119.stdout:7/838: read - d7/df/d19/d97/d92/db7/dba/d42/d64/dad/f10a zero size 2024-05-27T09:29:33.860 INFO:tasks.workunit.client.0.smithi119.stdout:7/839: creat d7/df/d19/d4d/d89/f10b x:0 0 0 2024-05-27T09:29:33.860 INFO:tasks.workunit.client.0.smithi119.stdout:0/771: dwrite d2/d6/d9/d24/d30/dcc/fd9 [0,4194304] 0 2024-05-27T09:29:33.860 INFO:tasks.workunit.client.0.smithi119.stdout:2/806: symlink d2/da/d8a/dd4/d69/df5/d108/l113 0 2024-05-27T09:29:33.860 INFO:tasks.workunit.client.0.smithi119.stdout:2/807: fsync d2/da/d4e/f72 0 2024-05-27T09:29:33.862 INFO:tasks.workunit.client.1.smithi138.stdout:8/241: dread - d3/d8/d28/f40 zero size 2024-05-27T09:29:33.878 INFO:tasks.workunit.client.0.smithi119.stdout:2/808: mkdir d2/da/d15/d53/d114 0 2024-05-27T09:29:33.880 INFO:tasks.workunit.client.0.smithi119.stdout:2/809: link d2/da/d8a/dd4/c97 d2/da/d8a/dd4/d89/d94/d87/c115 0 2024-05-27T09:29:33.881 INFO:tasks.workunit.client.0.smithi119.stdout:0/772: dread d2/d6/d9/d2f/fc1 [0,4194304] 0 2024-05-27T09:29:33.882 INFO:tasks.workunit.client.0.smithi119.stdout:2/810: mkdir d2/da/d15/d53/d114/d116 0 2024-05-27T09:29:33.882 INFO:tasks.workunit.client.0.smithi119.stdout:2/811: readlink d2/da/l20 0 2024-05-27T09:29:33.883 INFO:tasks.workunit.client.0.smithi119.stdout:2/812: symlink d2/da/d8a/dd4/d89/d94/d87/dcf/de5/l117 0 2024-05-27T09:29:33.883 INFO:tasks.workunit.client.0.smithi119.stdout:2/813: truncate d2/fa0 487667 0 2024-05-27T09:29:33.884 INFO:tasks.workunit.client.1.smithi138.stdout:9/259: dwrite d6/db/de/d18/d27/d46/f47 [0,4194304] 0 2024-05-27T09:29:33.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:33 smithi138 ceph-mon[34885]: pgmap v14: 65 pgs: 65 active+clean; 2.2 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 196 MiB/s rd, 238 MiB/s wr, 534 op/s 2024-05-27T09:29:33.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:33 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:33.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:33 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:33.886 INFO:tasks.workunit.client.1.smithi138.stdout:4/288: dwrite d7/df/f19 [4194304,4194304] 0 2024-05-27T09:29:33.887 INFO:tasks.workunit.client.1.smithi138.stdout:4/289: rename d7/d27/c32 to d7/df/d35/c5f 0 2024-05-27T09:29:33.890 INFO:tasks.workunit.client.1.smithi138.stdout:4/290: truncate d7/d27/d3a/f55 100467 0 2024-05-27T09:29:33.890 INFO:tasks.workunit.client.1.smithi138.stdout:4/291: creat d7/f60 x:0 0 0 2024-05-27T09:29:33.891 INFO:tasks.workunit.client.1.smithi138.stdout:4/292: rmdir d7/df/d5b/d5e 0 2024-05-27T09:29:33.891 INFO:tasks.workunit.client.1.smithi138.stdout:9/260: mknod d6/da/d2f/c52 0 2024-05-27T09:29:33.892 INFO:tasks.workunit.client.1.smithi138.stdout:4/293: mknod d7/df/d35/c61 0 2024-05-27T09:29:33.897 INFO:tasks.workunit.client.1.smithi138.stdout:9/261: rename d6/da/d26/d28/f43 to d6/f53 0 2024-05-27T09:29:33.903 INFO:tasks.workunit.client.1.smithi138.stdout:1/288: dwrite d4/d48/d39/f45 [0,4194304] 0 2024-05-27T09:29:33.909 INFO:tasks.workunit.client.1.smithi138.stdout:1/289: symlink d4/d1a/d2e/l59 0 2024-05-27T09:29:33.909 INFO:tasks.workunit.client.1.smithi138.stdout:6/348: sync 2024-05-27T09:29:33.909 INFO:tasks.workunit.client.1.smithi138.stdout:0/278: sync 2024-05-27T09:29:33.909 INFO:tasks.workunit.client.1.smithi138.stdout:0/279: chown da/d17/d1d/d1f/d42 17096088 1 2024-05-27T09:29:33.909 INFO:tasks.workunit.client.1.smithi138.stdout:6/349: chown d3/d35/d43/d1c/l2b 258425 1 2024-05-27T09:29:33.910 INFO:tasks.workunit.client.1.smithi138.stdout:0/280: creat da/d17/d1d/d43/f64 x:0 0 0 2024-05-27T09:29:33.910 INFO:tasks.workunit.client.0.smithi119.stdout:5/856: dwrite d6/f1e [4194304,4194304] 0 2024-05-27T09:29:33.910 INFO:tasks.workunit.client.0.smithi119.stdout:5/857: readlink d6/dd/d39/lca 0 2024-05-27T09:29:33.910 INFO:tasks.workunit.client.0.smithi119.stdout:3/903: dwrite db/d36/fbc [0,4194304] 0 2024-05-27T09:29:33.910 INFO:tasks.workunit.client.0.smithi119.stdout:5/858: dread - d6/dd/d46/dba/ffc zero size 2024-05-27T09:29:33.911 INFO:tasks.workunit.client.0.smithi119.stdout:5/859: write d6/db/de/d27/f7e [1153319,113427] 0 2024-05-27T09:29:33.912 INFO:tasks.workunit.client.0.smithi119.stdout:5/860: chown d6/dd/d46/d47/d4b/l86 36 1 2024-05-27T09:29:33.913 INFO:tasks.workunit.client.1.smithi138.stdout:9/262: symlink d6/db/de/d18/l54 0 2024-05-27T09:29:33.914 INFO:tasks.workunit.client.1.smithi138.stdout:1/290: unlink d4/da/l1b 0 2024-05-27T09:29:33.914 INFO:tasks.workunit.client.1.smithi138.stdout:6/350: rename d3/d11/d2d/f61 to d3/d35/f7c 0 2024-05-27T09:29:33.924 INFO:tasks.workunit.client.1.smithi138.stdout:0/281: getdents da/d2a/d2b 0 2024-05-27T09:29:33.924 INFO:tasks.workunit.client.1.smithi138.stdout:0/282: truncate da/d2a/d2b/f4b 892966 0 2024-05-27T09:29:33.925 INFO:tasks.workunit.client.0.smithi119.stdout:0/773: dread d2/d6/d9/fbd [0,4194304] 0 2024-05-27T09:29:33.925 INFO:tasks.workunit.client.0.smithi119.stdout:9/677: dwrite fe [4194304,4194304] 0 2024-05-27T09:29:33.925 INFO:tasks.workunit.client.0.smithi119.stdout:9/678: write d18/d19/d1b/d40/d7b/f7e [1820868,51605] 0 2024-05-27T09:29:33.925 INFO:tasks.workunit.client.1.smithi138.stdout:1/291: creat d4/d48/d22/f5a x:0 0 0 2024-05-27T09:29:33.925 INFO:tasks.workunit.client.1.smithi138.stdout:1/292: chown d4/d48/l31 177815 1 2024-05-27T09:29:33.926 INFO:tasks.workunit.client.0.smithi119.stdout:5/861: mknod d6/dd/d39/d74/c131 0 2024-05-27T09:29:33.926 INFO:tasks.workunit.client.0.smithi119.stdout:3/904: rename db/f23 to db/d7f/dd2/f125 0 2024-05-27T09:29:33.933 INFO:tasks.workunit.client.1.smithi138.stdout:1/293: truncate d4/d1a/f27 2180941 0 2024-05-27T09:29:33.933 INFO:tasks.workunit.client.1.smithi138.stdout:6/351: mkdir d3/d35/d7d 0 2024-05-27T09:29:33.933 INFO:tasks.workunit.client.1.smithi138.stdout:6/352: chown d3/d11/d33/f66 14661724 1 2024-05-27T09:29:33.936 INFO:tasks.workunit.client.0.smithi119.stdout:9/679: symlink d18/d19/d1b/d2a/d63/lcd 0 2024-05-27T09:29:33.937 INFO:tasks.workunit.client.1.smithi138.stdout:1/294: creat d4/da/f5b x:0 0 0 2024-05-27T09:29:33.939 INFO:tasks.workunit.client.1.smithi138.stdout:6/353: mkdir d3/d11/d2d/d37/d7e 0 2024-05-27T09:29:33.940 INFO:tasks.workunit.client.1.smithi138.stdout:6/354: creat d3/d35/d5a/d6a/f7f x:0 0 0 2024-05-27T09:29:33.941 INFO:tasks.workunit.client.0.smithi119.stdout:5/862: dread - d6/dd/d39/d19/d3c/fb2 zero size 2024-05-27T09:29:33.941 INFO:tasks.workunit.client.0.smithi119.stdout:5/863: creat d6/dd/d39/d19/d7d/db7/f132 x:0 0 0 2024-05-27T09:29:33.943 INFO:tasks.workunit.client.0.smithi119.stdout:5/864: chown d6/dd/d39/d19/d3c/d52/lf3 30 1 2024-05-27T09:29:33.943 INFO:tasks.workunit.client.0.smithi119.stdout:5/865: dread - d6/dd/d39/fd1 zero size 2024-05-27T09:29:33.943 INFO:tasks.workunit.client.0.smithi119.stdout:5/866: read - d6/dd/d46/d47/d4b/dd8/df2/f10a zero size 2024-05-27T09:29:33.946 INFO:tasks.workunit.client.0.smithi119.stdout:3/905: symlink db/d7f/da9/l126 0 2024-05-27T09:29:33.947 INFO:tasks.workunit.client.0.smithi119.stdout:3/906: fsync db/d36/d45/d4c/d62/f38 0 2024-05-27T09:29:33.952 INFO:tasks.workunit.client.0.smithi119.stdout:0/774: rename d2/d6/d9/d24/d30/c34 to d2/d6/d9/d24/cfe 0 2024-05-27T09:29:33.958 INFO:tasks.workunit.client.0.smithi119.stdout:6/761: dwrite d0/d2/dec/ffc [0,4194304] 0 2024-05-27T09:29:33.960 INFO:tasks.workunit.client.1.smithi138.stdout:6/355: creat d3/d35/f80 x:0 0 0 2024-05-27T09:29:33.967 INFO:tasks.workunit.client.0.smithi119.stdout:5/867: rename d6/db/de/fdc to d6/dd/d46/f133 0 2024-05-27T09:29:33.971 INFO:tasks.workunit.client.1.smithi138.stdout:6/356: symlink d3/d35/d5a/l81 0 2024-05-27T09:29:33.971 INFO:tasks.workunit.client.1.smithi138.stdout:6/357: fsync d3/d35/d43/f48 0 2024-05-27T09:29:33.972 INFO:tasks.workunit.client.1.smithi138.stdout:5/239: sync 2024-05-27T09:29:33.973 INFO:tasks.workunit.client.1.smithi138.stdout:8/242: dwrite d3/f29 [0,4194304] 0 2024-05-27T09:29:33.977 INFO:tasks.workunit.client.1.smithi138.stdout:7/250: sync 2024-05-27T09:29:33.978 INFO:tasks.workunit.client.0.smithi119.stdout:6/762: dread d0/f84 [0,4194304] 0 2024-05-27T09:29:33.980 INFO:tasks.workunit.client.0.smithi119.stdout:2/814: dwrite d2/da/d8a/dd4/f60 [0,4194304] 0 2024-05-27T09:29:33.981 INFO:tasks.workunit.client.0.smithi119.stdout:2/815: write d2/da/d8a/dd4/d89/db0/f107 [810434,4456] 0 2024-05-27T09:29:33.981 INFO:tasks.workunit.client.0.smithi119.stdout:2/816: write d2/da/d6f/fc1 [4156598,12106] 0 2024-05-27T09:29:33.981 INFO:tasks.workunit.client.0.smithi119.stdout:2/817: chown d2/da/f95 1 1 2024-05-27T09:29:33.984 INFO:tasks.workunit.client.1.smithi138.stdout:7/251: truncate d0/f10 2190193 0 2024-05-27T09:29:33.988 INFO:tasks.workunit.client.0.smithi119.stdout:0/775: dread d2/d6/d9/f2c [0,4194304] 0 2024-05-27T09:29:33.991 INFO:tasks.workunit.client.1.smithi138.stdout:7/252: symlink d0/d23/d3c/d3f/l52 0 2024-05-27T09:29:33.991 INFO:tasks.workunit.client.1.smithi138.stdout:6/358: dread d3/d19/f38 [0,4194304] 0 2024-05-27T09:29:33.991 INFO:tasks.workunit.client.1.smithi138.stdout:6/359: fdatasync d3/d19/f63 0 2024-05-27T09:29:33.991 INFO:tasks.workunit.client.1.smithi138.stdout:6/360: write d3/d35/d5a/d6a/f6f [80600,12815] 0 2024-05-27T09:29:33.991 INFO:tasks.workunit.client.1.smithi138.stdout:6/361: rename d3 to d3/d11/d33/d82 22 2024-05-27T09:29:33.992 INFO:tasks.workunit.client.1.smithi138.stdout:8/243: write d3/d8/d11/f26 [2157351,126998] 0 2024-05-27T09:29:33.993 INFO:tasks.workunit.client.0.smithi119.stdout:3/907: rename db/d5b/dce/l10d to db/dc9/l127 0 2024-05-27T09:29:33.993 INFO:tasks.workunit.client.0.smithi119.stdout:6/763: creat d0/d2/d1c/d21/d58/dd3/f110 x:0 0 0 2024-05-27T09:29:33.993 INFO:tasks.workunit.client.1.smithi138.stdout:6/362: creat d3/d19/f83 x:0 0 0 2024-05-27T09:29:33.994 INFO:tasks.workunit.client.1.smithi138.stdout:6/363: readlink d3/d11/l62 0 2024-05-27T09:29:33.994 INFO:tasks.workunit.client.0.smithi119.stdout:7/840: dwrite d7/df/d19/d9a/da6/dda/d44/f80 [4194304,4194304] 0 2024-05-27T09:29:34.000 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:33 smithi119 ceph-mon[29082]: pgmap v14: 65 pgs: 65 active+clean; 2.2 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 196 MiB/s rd, 238 MiB/s wr, 534 op/s 2024-05-27T09:29:34.001 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:33 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:34.001 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:33 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:34.005 INFO:tasks.workunit.client.0.smithi119.stdout:8/749: dwrite d3/d8/d4d/d58/f70 [0,4194304] 0 2024-05-27T09:29:34.005 INFO:tasks.workunit.client.0.smithi119.stdout:8/750: write d3/f8c [1295148,23664] 0 2024-05-27T09:29:34.005 INFO:tasks.workunit.client.0.smithi119.stdout:8/751: fsync d3/d8/d1b/db1/fbc 0 2024-05-27T09:29:34.008 INFO:tasks.workunit.client.0.smithi119.stdout:0/776: mknod d2/d6/d9/d12/d4b/d59/dd7/cff 0 2024-05-27T09:29:34.008 INFO:tasks.workunit.client.0.smithi119.stdout:3/908: rename db/d7f/f74 to db/d36/d45/d93/f128 0 2024-05-27T09:29:34.011 INFO:tasks.workunit.client.0.smithi119.stdout:4/840: dwrite d4/d6/d3e/dd2/ffc [0,4194304] 0 2024-05-27T09:29:34.011 INFO:tasks.workunit.client.0.smithi119.stdout:4/841: write d4/de/d21/d40/d47/da3/fb5 [557444,111393] 0 2024-05-27T09:29:34.013 INFO:tasks.workunit.client.0.smithi119.stdout:3/909: mkdir db/dc9/d98/d7a/d129 0 2024-05-27T09:29:34.013 INFO:tasks.workunit.client.0.smithi119.stdout:3/910: write db/d7f/da9/f110 [590760,20992] 0 2024-05-27T09:29:34.015 INFO:tasks.workunit.client.1.smithi138.stdout:3/206: dwrite d0/d2/da/df/d10/d2a/f42 [0,4194304] 0 2024-05-27T09:29:34.016 INFO:tasks.workunit.client.1.smithi138.stdout:4/294: dwrite d7/df/d35/f5c [0,4194304] 0 2024-05-27T09:29:34.016 INFO:tasks.workunit.client.0.smithi119.stdout:2/818: rename d2/da/f13 to d2/da/d8a/dd4/d89/d94/d87/f118 0 2024-05-27T09:29:34.018 INFO:tasks.workunit.client.1.smithi138.stdout:5/240: dread d3/fd [0,4194304] 0 2024-05-27T09:29:34.018 INFO:tasks.workunit.client.1.smithi138.stdout:5/241: write d3/d12/d16/f39 [2058792,34210] 0 2024-05-27T09:29:34.023 INFO:tasks.workunit.client.1.smithi138.stdout:3/207: rename d0/d2/da/df/d10/d32/d37/l40 to d0/d2/da/df/d10/l44 0 2024-05-27T09:29:34.023 INFO:tasks.workunit.client.1.smithi138.stdout:3/208: stat d0/c4 0 2024-05-27T09:29:34.023 INFO:tasks.workunit.client.1.smithi138.stdout:3/209: stat d0/fd 0 2024-05-27T09:29:34.032 INFO:tasks.workunit.client.1.smithi138.stdout:4/295: creat d7/d27/d3a/f62 x:0 0 0 2024-05-27T09:29:34.034 INFO:tasks.workunit.client.1.smithi138.stdout:5/242: getdents d3/d12/d16/d1d/d23 0 2024-05-27T09:29:34.039 INFO:tasks.workunit.client.0.smithi119.stdout:4/842: mkdir d4/de/dfb/d108 0 2024-05-27T09:29:34.039 INFO:tasks.workunit.client.0.smithi119.stdout:7/841: rename d7/df/d19/d97/d92/fd5 to d7/df/d19/d97/d92/db7/dba/d42/f10c 0 2024-05-27T09:29:34.043 INFO:tasks.workunit.client.1.smithi138.stdout:8/244: dread d3/d8/fc [0,4194304] 0 2024-05-27T09:29:34.046 INFO:tasks.workunit.client.0.smithi119.stdout:3/911: rmdir db/d36/d45/d4d/dc8 39 2024-05-27T09:29:34.051 INFO:tasks.workunit.client.1.smithi138.stdout:8/245: link f1 d3/d8/d32/d34/d38/f46 0 2024-05-27T09:29:34.052 INFO:tasks.workunit.client.1.smithi138.stdout:5/243: symlink d3/l51 0 2024-05-27T09:29:34.058 INFO:tasks.workunit.client.1.smithi138.stdout:3/210: dread d0/d2/f27 [0,4194304] 0 2024-05-27T09:29:34.058 INFO:tasks.workunit.client.1.smithi138.stdout:3/211: fsync d0/d2/da/f1b 0 2024-05-27T09:29:34.058 INFO:tasks.workunit.client.1.smithi138.stdout:5/244: rename d3/d12/d16/d1d/c35 to d3/d12/d16/d1d/d30/c52 0 2024-05-27T09:29:34.059 INFO:tasks.workunit.client.1.smithi138.stdout:3/212: write d0/d2/da/f1b [3236232,51116] 0 2024-05-27T09:29:34.059 INFO:tasks.workunit.client.1.smithi138.stdout:5/245: write d3/f21 [417342,17490] 0 2024-05-27T09:29:34.064 INFO:tasks.workunit.client.0.smithi119.stdout:4/843: write d4/d6/d3e/f6e [1119747,21242] 0 2024-05-27T09:29:34.065 INFO:tasks.workunit.client.0.smithi119.stdout:5/868: dwrite d6/dd/d46/dba/ffc [0,4194304] 0 2024-05-27T09:29:34.065 INFO:tasks.workunit.client.0.smithi119.stdout:5/869: dread - d6/dd/d46/fb8 zero size 2024-05-27T09:29:34.067 INFO:tasks.workunit.client.1.smithi138.stdout:0/283: dwrite da/d2a/d2b/f33 [0,4194304] 0 2024-05-27T09:29:34.067 INFO:tasks.workunit.client.1.smithi138.stdout:0/284: write da/f20 [2501707,118096] 0 2024-05-27T09:29:34.067 INFO:tasks.workunit.client.1.smithi138.stdout:0/285: write da/f20 [2276489,31198] 0 2024-05-27T09:29:34.067 INFO:tasks.workunit.client.1.smithi138.stdout:0/286: write da/d17/f31 [765679,123068] 0 2024-05-27T09:29:34.072 INFO:tasks.workunit.client.1.smithi138.stdout:5/246: unlink d3/d12/f17 0 2024-05-27T09:29:34.073 INFO:tasks.workunit.client.1.smithi138.stdout:5/247: chown d3/d12/d16/d1d/d30 8611 1 2024-05-27T09:29:34.073 INFO:tasks.workunit.client.1.smithi138.stdout:5/248: chown d3/d12/d32/f4f 528 1 2024-05-27T09:29:34.073 INFO:tasks.workunit.client.1.smithi138.stdout:5/249: truncate d3/f21 1206505 0 2024-05-27T09:29:34.078 INFO:tasks.workunit.client.0.smithi119.stdout:4/844: rename d4/d23/d24/d5b/ffa to d4/d1c/d39/f109 0 2024-05-27T09:29:34.079 INFO:tasks.workunit.client.0.smithi119.stdout:8/752: dread d3/d8/f7e [0,4194304] 0 2024-05-27T09:29:34.079 INFO:tasks.workunit.client.0.smithi119.stdout:8/753: creat d3/de/dad/ffa x:0 0 0 2024-05-27T09:29:34.079 INFO:tasks.workunit.client.0.smithi119.stdout:8/754: fdatasync d3/d6e/f75 0 2024-05-27T09:29:34.079 INFO:tasks.workunit.client.0.smithi119.stdout:8/755: write d3/d8/d1b/db1/fbc [4197758,2279] 0 2024-05-27T09:29:34.080 INFO:tasks.workunit.client.0.smithi119.stdout:8/756: write d3/d8/d4d/d58/f8f [786469,101919] 0 2024-05-27T09:29:34.080 INFO:tasks.workunit.client.1.smithi138.stdout:0/287: mknod da/d17/d1d/d1f/d42/d4a/c65 0 2024-05-27T09:29:34.080 INFO:tasks.workunit.client.1.smithi138.stdout:0/288: dread - da/d2a/d55/f5c zero size 2024-05-27T09:29:34.081 INFO:tasks.workunit.client.1.smithi138.stdout:0/289: write da/f20 [1387795,78818] 0 2024-05-27T09:29:34.084 INFO:tasks.workunit.client.0.smithi119.stdout:2/819: dwrite d2/d1a/f30 [4194304,4194304] 0 2024-05-27T09:29:34.084 INFO:tasks.workunit.client.0.smithi119.stdout:2/820: fsync d2/da/d8a/dd4/d89/fa3 0 2024-05-27T09:29:34.088 INFO:tasks.workunit.client.0.smithi119.stdout:3/912: dread db/d36/d45/d4c/d62/d13/f2c [0,4194304] 0 2024-05-27T09:29:34.088 INFO:tasks.workunit.client.1.smithi138.stdout:5/250: mkdir d3/d12/d26/d53 0 2024-05-27T09:29:34.088 INFO:tasks.workunit.client.1.smithi138.stdout:9/263: dwrite d6/da/d2f/f40 [0,4194304] 0 2024-05-27T09:29:34.088 INFO:tasks.workunit.client.1.smithi138.stdout:9/264: fdatasync d6/db/dc/f12 0 2024-05-27T09:29:34.088 INFO:tasks.workunit.client.1.smithi138.stdout:9/265: stat d6/f53 0 2024-05-27T09:29:34.089 INFO:tasks.workunit.client.1.smithi138.stdout:6/364: dwrite d3/d35/d43/f2a [0,4194304] 0 2024-05-27T09:29:34.096 INFO:tasks.workunit.client.0.smithi119.stdout:3/913: read db/dc9/d98/d7a/f9e [254632,114387] 0 2024-05-27T09:29:34.099 INFO:tasks.workunit.client.0.smithi119.stdout:7/842: write d7/df/d19/d78/ff9 [2188104,71869] 0 2024-05-27T09:29:34.100 INFO:tasks.workunit.client.1.smithi138.stdout:7/253: dwrite d0/d23/f4c [0,4194304] 0 2024-05-27T09:29:34.101 INFO:tasks.workunit.client.0.smithi119.stdout:6/764: dwrite d0/d2/fb1 [0,4194304] 0 2024-05-27T09:29:34.110 INFO:tasks.workunit.client.0.smithi119.stdout:5/870: mkdir d6/db/de/d27/dc9/d127/d134 0 2024-05-27T09:29:34.120 INFO:tasks.workunit.client.1.smithi138.stdout:3/213: dread d0/d2/da/f33 [0,4194304] 0 2024-05-27T09:29:34.126 INFO:tasks.workunit.client.1.smithi138.stdout:6/365: write d3/f53 [3720024,80146] 0 2024-05-27T09:29:34.127 INFO:tasks.workunit.client.1.smithi138.stdout:6/366: write d3/d35/d43/d1c/d32/d47/f56 [965678,55765] 0 2024-05-27T09:29:34.128 INFO:tasks.workunit.client.0.smithi119.stdout:2/821: dread d2/da/d8a/dd4/fb5 [0,4194304] 0 2024-05-27T09:29:34.128 INFO:tasks.workunit.client.0.smithi119.stdout:5/871: mkdir d6/dd/d39/d19/d3c/d52/d135 0 2024-05-27T09:29:34.131 INFO:tasks.workunit.client.0.smithi119.stdout:0/777: dwrite d2/d6/d9/d12/f16 [8388608,4194304] 0 2024-05-27T09:29:34.132 INFO:tasks.workunit.client.0.smithi119.stdout:2/822: creat d2/da/d6f/f119 x:0 0 0 2024-05-27T09:29:34.132 INFO:tasks.workunit.client.0.smithi119.stdout:2/823: readlink d2/d1a/l86 0 2024-05-27T09:29:34.132 INFO:tasks.workunit.client.0.smithi119.stdout:2/824: stat d2/da/d8a/dd4/d89/d94 0 2024-05-27T09:29:34.132 INFO:tasks.workunit.client.0.smithi119.stdout:0/778: creat d2/d6/d9/d12/f100 x:0 0 0 2024-05-27T09:29:34.136 INFO:tasks.workunit.client.1.smithi138.stdout:3/214: dread d0/d2/da/df/d10/d32/f28 [0,4194304] 0 2024-05-27T09:29:34.137 INFO:tasks.workunit.client.0.smithi119.stdout:6/765: rename d0/d2/d57/d37/c3d to d0/d2/de1/c111 0 2024-05-27T09:29:34.140 INFO:tasks.workunit.client.0.smithi119.stdout:5/872: link d6/dd/d46/d47/d4b/dd8/d124/cd3 d6/db/dd4/c136 0 2024-05-27T09:29:34.147 INFO:tasks.workunit.client.1.smithi138.stdout:6/367: fsync d3/f53 0 2024-05-27T09:29:34.148 INFO:tasks.workunit.client.0.smithi119.stdout:1/832: sync 2024-05-27T09:29:34.148 INFO:tasks.workunit.client.0.smithi119.stdout:1/833: fdatasync d7/dd/d44/fa7 0 2024-05-27T09:29:34.148 INFO:tasks.workunit.client.1.smithi138.stdout:6/368: write d3/d19/f1e [1115089,82923] 0 2024-05-27T09:29:34.148 INFO:tasks.workunit.client.1.smithi138.stdout:6/369: fsync d3/d35/d43/f2a 0 2024-05-27T09:29:34.148 INFO:tasks.workunit.client.1.smithi138.stdout:6/370: fdatasync d3/d19/f3d 0 2024-05-27T09:29:34.148 INFO:tasks.workunit.client.1.smithi138.stdout:6/371: getdents d3/d35/d43/d71 0 2024-05-27T09:29:34.155 INFO:tasks.workunit.client.1.smithi138.stdout:4/296: dwrite d7/f26 [0,4194304] 0 2024-05-27T09:29:34.156 INFO:tasks.workunit.client.1.smithi138.stdout:6/372: creat d3/f84 x:0 0 0 2024-05-27T09:29:34.156 INFO:tasks.workunit.client.1.smithi138.stdout:4/297: chown d7/f23 154 1 2024-05-27T09:29:34.156 INFO:tasks.workunit.client.1.smithi138.stdout:4/298: write d7/fb [219255,91130] 0 2024-05-27T09:29:34.161 INFO:tasks.workunit.client.1.smithi138.stdout:9/266: dwrite d6/f3b [0,4194304] 0 2024-05-27T09:29:34.163 INFO:tasks.workunit.client.1.smithi138.stdout:9/267: write d6/f3f [206655,30946] 0 2024-05-27T09:29:34.167 INFO:tasks.workunit.client.0.smithi119.stdout:8/757: dwrite d3/d6e/dcf/dce/fd8 [0,4194304] 0 2024-05-27T09:29:34.167 INFO:tasks.workunit.client.0.smithi119.stdout:8/758: write d3/d8/d1b/d46/f7c [1032216,108900] 0 2024-05-27T09:29:34.167 INFO:tasks.workunit.client.0.smithi119.stdout:8/759: fdatasync d3/d8/d1b/d46/da4/fb9 0 2024-05-27T09:29:34.167 INFO:tasks.workunit.client.0.smithi119.stdout:8/760: chown d3/d8/d4d/d5d/d74/f98 426893 1 2024-05-27T09:29:34.173 INFO:tasks.workunit.client.1.smithi138.stdout:9/268: rename d6/d25 to d6/da/d2f/d35/d55 0 2024-05-27T09:29:34.176 INFO:tasks.workunit.client.1.smithi138.stdout:3/215: dread d0/f12 [0,4194304] 0 2024-05-27T09:29:34.176 INFO:tasks.workunit.client.0.smithi119.stdout:2/825: truncate d2/da/d8a/dec/ff6 962385 0 2024-05-27T09:29:34.177 INFO:tasks.workunit.client.1.smithi138.stdout:4/299: symlink d7/df/d35/d4c/l63 0 2024-05-27T09:29:34.177 INFO:tasks.workunit.client.1.smithi138.stdout:4/300: fsync d7/df/f19 0 2024-05-27T09:29:34.178 INFO:tasks.workunit.client.0.smithi119.stdout:3/914: dwrite db/d36/d45/dab/fea [0,4194304] 0 2024-05-27T09:29:34.180 INFO:tasks.workunit.client.1.smithi138.stdout:7/254: dwrite d0/f24 [4194304,4194304] 0 2024-05-27T09:29:34.180 INFO:tasks.workunit.client.0.smithi119.stdout:4/845: dwrite f1 [0,4194304] 0 2024-05-27T09:29:34.183 INFO:tasks.workunit.client.1.smithi138.stdout:7/255: mknod d0/d23/d3c/c53 0 2024-05-27T09:29:34.183 INFO:tasks.workunit.client.1.smithi138.stdout:7/256: chown d0/d2b/f35 26 1 2024-05-27T09:29:34.183 INFO:tasks.workunit.client.1.smithi138.stdout:7/257: dread - d0/f45 zero size 2024-05-27T09:29:34.184 INFO:tasks.workunit.client.1.smithi138.stdout:7/258: readlink d0/d23/l26 0 2024-05-27T09:29:34.185 INFO:tasks.workunit.client.0.smithi119.stdout:0/779: symlink d2/d6/d9/d12/l101 0 2024-05-27T09:29:34.191 INFO:tasks.workunit.client.1.smithi138.stdout:9/269: dread d6/db/dc/f2c [0,4194304] 0 2024-05-27T09:29:34.192 INFO:tasks.workunit.client.0.smithi119.stdout:5/873: symlink d6/d2f/dfd/l137 0 2024-05-27T09:29:34.193 INFO:tasks.workunit.client.1.smithi138.stdout:5/251: dwrite f0 [4194304,4194304] 0 2024-05-27T09:29:34.194 INFO:tasks.workunit.client.1.smithi138.stdout:9/270: unlink d6/db/de/d18/c2a 0 2024-05-27T09:29:34.195 INFO:tasks.workunit.client.0.smithi119.stdout:1/834: creat d7/dd/d44/f100 x:0 0 0 2024-05-27T09:29:34.198 INFO:tasks.workunit.client.1.smithi138.stdout:7/259: rmdir d0/d23/d49 39 2024-05-27T09:29:34.202 INFO:tasks.workunit.client.0.smithi119.stdout:7/843: write d7/df/d19/d97/d37/f5e [3267703,6974] 0 2024-05-27T09:29:34.202 INFO:tasks.workunit.client.0.smithi119.stdout:7/844: readlink d7/df/d19/d97/d92/db7/led 0 2024-05-27T09:29:34.203 INFO:tasks.workunit.client.0.smithi119.stdout:7/845: chown d7/df/d19/d9a/da6/dda/d44/c54 2 1 2024-05-27T09:29:34.205 INFO:tasks.workunit.client.1.smithi138.stdout:9/271: rmdir d6 39 2024-05-27T09:29:34.206 INFO:tasks.workunit.client.1.smithi138.stdout:3/216: read d0/fd [3539857,68925] 0 2024-05-27T09:29:34.209 INFO:tasks.workunit.client.1.smithi138.stdout:9/272: symlink d6/db/l56 0 2024-05-27T09:29:34.211 INFO:tasks.workunit.client.0.smithi119.stdout:2/826: rename d2/da/d8a/laa to d2/da/d8a/l11a 0 2024-05-27T09:29:34.211 INFO:tasks.workunit.client.0.smithi119.stdout:2/827: fsync d2/da/d8a/dd4/d89/d94/ffd 0 2024-05-27T09:29:34.213 INFO:tasks.workunit.client.1.smithi138.stdout:3/217: dread d0/f12 [0,4194304] 0 2024-05-27T09:29:34.215 INFO:tasks.workunit.client.0.smithi119.stdout:5/874: dread d6/dd/d46/d47/d4b/dd8/ddb/fe5 [0,4194304] 0 2024-05-27T09:29:34.228 INFO:tasks.workunit.client.0.smithi119.stdout:3/915: rename db/d7f/c96 to db/d36/d45/d4c/d62/d13/d33/c12a 0 2024-05-27T09:29:34.228 INFO:tasks.workunit.client.0.smithi119.stdout:3/916: creat db/dc9/f12b x:0 0 0 2024-05-27T09:29:34.229 INFO:tasks.workunit.client.0.smithi119.stdout:9/680: sync 2024-05-27T09:29:34.229 INFO:tasks.workunit.client.0.smithi119.stdout:4/846: truncate d4/d6/d3e/dd2/ffc 3220323 0 2024-05-27T09:29:34.229 INFO:tasks.workunit.client.0.smithi119.stdout:4/847: chown d4/de/dfb/d108 105305 1 2024-05-27T09:29:34.229 INFO:tasks.workunit.client.0.smithi119.stdout:3/917: write db/d36/fc3 [4253103,42858] 0 2024-05-27T09:29:34.232 INFO:tasks.workunit.client.0.smithi119.stdout:0/780: creat d2/d6/d23/d7a/d53/dd1/f102 x:0 0 0 2024-05-27T09:29:34.232 INFO:tasks.workunit.client.0.smithi119.stdout:0/781: creat d2/d6/d23/db8/f103 x:0 0 0 2024-05-27T09:29:34.232 INFO:tasks.workunit.client.1.smithi138.stdout:4/301: rmdir d7/df 39 2024-05-27T09:29:34.233 INFO:tasks.workunit.client.1.smithi138.stdout:2/228: sync 2024-05-27T09:29:34.233 INFO:tasks.workunit.client.1.smithi138.stdout:1/295: sync 2024-05-27T09:29:34.233 INFO:tasks.workunit.client.1.smithi138.stdout:2/229: readlink d1/l18 0 2024-05-27T09:29:34.238 INFO:tasks.workunit.client.1.smithi138.stdout:9/273: write d6/da/d26/f37 [2377248,56096] 0 2024-05-27T09:29:34.239 INFO:tasks.workunit.client.1.smithi138.stdout:4/302: mkdir d7/df/d35/d4c/d59/d5d/d64 0 2024-05-27T09:29:34.387 INFO:tasks.workunit.client.1.smithi138.stdout:4/303: write d7/f44 [721329,119338] 0 2024-05-27T09:29:34.387 INFO:tasks.workunit.client.1.smithi138.stdout:4/304: write d7/fb [1704020,62748] 0 2024-05-27T09:29:34.387 INFO:tasks.workunit.client.1.smithi138.stdout:4/305: write d7/d27/d3a/f55 [611858,70908] 0 2024-05-27T09:29:34.396 INFO:tasks.workunit.client.1.smithi138.stdout:6/373: dwrite d3/f5 [0,4194304] 0 2024-05-27T09:29:34.396 INFO:tasks.workunit.client.1.smithi138.stdout:9/274: rmdir d6/db/de/d18 39 2024-05-27T09:29:34.396 INFO:tasks.workunit.client.1.smithi138.stdout:1/296: link d4/d48/l3b d4/d38/l5c 0 2024-05-27T09:29:34.396 INFO:tasks.workunit.client.1.smithi138.stdout:9/275: rename d6/f53 to d6/db/de/d18/f57 0 2024-05-27T09:29:34.397 INFO:tasks.workunit.client.1.smithi138.stdout:5/252: rmdir d3 39 2024-05-27T09:29:34.397 INFO:tasks.workunit.client.1.smithi138.stdout:7/260: dwrite d0/d23/d3a/f46 [0,4194304] 0 2024-05-27T09:29:34.397 INFO:tasks.workunit.client.1.smithi138.stdout:3/218: dread d0/d2/fc [0,4194304] 0 2024-05-27T09:29:34.397 INFO:tasks.workunit.client.1.smithi138.stdout:3/219: stat d0/c4 0 2024-05-27T09:29:34.397 INFO:tasks.workunit.client.1.smithi138.stdout:4/306: getdents d7/df/d5b 0 2024-05-27T09:29:34.397 INFO:tasks.workunit.client.1.smithi138.stdout:4/307: write d7/d27/d3a/f62 [436109,70525] 0 2024-05-27T09:29:34.397 INFO:tasks.workunit.client.1.smithi138.stdout:7/261: unlink d0/d23/d3c/c4d 0 2024-05-27T09:29:34.397 INFO:tasks.workunit.client.1.smithi138.stdout:6/374: rmdir d3/d19/d77 0 2024-05-27T09:29:34.397 INFO:tasks.workunit.client.1.smithi138.stdout:3/220: creat d0/d2/f45 x:0 0 0 2024-05-27T09:29:34.397 INFO:tasks.workunit.client.1.smithi138.stdout:3/221: dread - d0/d2/f45 zero size 2024-05-27T09:29:34.397 INFO:tasks.workunit.client.1.smithi138.stdout:5/253: dread - d3/d12/d16/d1d/d22/f3a zero size 2024-05-27T09:29:34.397 INFO:tasks.workunit.client.1.smithi138.stdout:7/262: dread d0/d2b/f35 [0,4194304] 0 2024-05-27T09:29:34.397 INFO:tasks.workunit.client.1.smithi138.stdout:4/308: rename d7/df/d35/c54 to d7/d27/c65 0 2024-05-27T09:29:34.398 INFO:tasks.workunit.client.1.smithi138.stdout:4/309: chown d7/df/d35/d4c/l57 177723 1 2024-05-27T09:29:34.398 INFO:tasks.workunit.client.1.smithi138.stdout:7/263: creat d0/d7/f54 x:0 0 0 2024-05-27T09:29:34.398 INFO:tasks.workunit.client.1.smithi138.stdout:7/264: chown d0/c25 165 1 2024-05-27T09:29:34.398 INFO:tasks.workunit.client.1.smithi138.stdout:7/265: creat d0/d23/d3a/f55 x:0 0 0 2024-05-27T09:29:34.398 INFO:tasks.workunit.client.1.smithi138.stdout:4/310: creat d7/df/d35/d4c/f66 x:0 0 0 2024-05-27T09:29:34.398 INFO:tasks.workunit.client.1.smithi138.stdout:1/297: dwrite d4/f12 [4194304,4194304] 0 2024-05-27T09:29:34.398 INFO:tasks.workunit.client.1.smithi138.stdout:1/298: symlink d4/d3c/l5d 0 2024-05-27T09:29:34.398 INFO:tasks.workunit.client.1.smithi138.stdout:1/299: write d4/f12 [8893599,118636] 0 2024-05-27T09:29:34.398 INFO:tasks.workunit.client.1.smithi138.stdout:9/276: dwrite d6/f3f [0,4194304] 0 2024-05-27T09:29:34.398 INFO:tasks.workunit.client.1.smithi138.stdout:9/277: write d6/da/d26/d28/f49 [286167,51333] 0 2024-05-27T09:29:34.398 INFO:tasks.workunit.client.1.smithi138.stdout:7/266: write d0/d7/f39 [1605888,69514] 0 2024-05-27T09:29:34.398 INFO:tasks.workunit.client.1.smithi138.stdout:0/290: sync 2024-05-27T09:29:34.398 INFO:tasks.workunit.client.1.smithi138.stdout:8/246: sync 2024-05-27T09:29:34.399 INFO:tasks.workunit.client.1.smithi138.stdout:0/291: fdatasync da/f29 0 2024-05-27T09:29:34.399 INFO:tasks.workunit.client.1.smithi138.stdout:8/247: truncate d3/d8/d11/f26 3148171 0 2024-05-27T09:29:34.399 INFO:tasks.workunit.client.1.smithi138.stdout:8/248: chown d3/d8/fc 25746 1 2024-05-27T09:29:34.399 INFO:tasks.workunit.client.1.smithi138.stdout:8/249: dread - d3/d8/f33 zero size 2024-05-27T09:29:34.399 INFO:tasks.workunit.client.1.smithi138.stdout:3/222: dwrite d0/d2/da/df/d10/d32/f9 [0,4194304] 0 2024-05-27T09:29:34.399 INFO:tasks.workunit.client.1.smithi138.stdout:3/223: stat d0/fd 0 2024-05-27T09:29:34.399 INFO:tasks.workunit.client.1.smithi138.stdout:3/224: mknod d0/d2/da/df/d10/d32/c46 0 2024-05-27T09:29:34.399 INFO:tasks.workunit.client.1.smithi138.stdout:3/225: truncate d0/d2/fc 3312654 0 2024-05-27T09:29:34.399 INFO:tasks.workunit.client.1.smithi138.stdout:3/226: fsync d0/d2/f45 0 2024-05-27T09:29:34.400 INFO:tasks.workunit.client.0.smithi119.stdout:5/875: symlink d6/db/de/d27/d77/l138 0 2024-05-27T09:29:34.400 INFO:tasks.workunit.client.0.smithi119.stdout:5/876: dread - d6/dd/d46/d47/d4b/f122 zero size 2024-05-27T09:29:34.400 INFO:tasks.workunit.client.0.smithi119.stdout:5/877: getdents d6/dd/d46/d47/d4b/dd8/d124/d100/d12a 0 2024-05-27T09:29:34.400 INFO:tasks.workunit.client.0.smithi119.stdout:5/878: write d6/dd/fd9 [826357,33101] 0 2024-05-27T09:29:34.400 INFO:tasks.workunit.client.0.smithi119.stdout:4/848: dread d4/d6/fd [0,4194304] 0 2024-05-27T09:29:34.400 INFO:tasks.workunit.client.0.smithi119.stdout:3/918: write db/d36/fbc [1292081,29052] 0 2024-05-27T09:29:34.400 INFO:tasks.workunit.client.0.smithi119.stdout:9/681: creat d18/d19/d1b/d29/fce x:0 0 0 2024-05-27T09:29:34.400 INFO:tasks.workunit.client.0.smithi119.stdout:7/846: dwrite d7/df/d19/d9a/da6/dda/d44/f80 [4194304,4194304] 0 2024-05-27T09:29:34.401 INFO:tasks.workunit.client.0.smithi119.stdout:0/782: getdents d2/d6/d9/d24/d30 0 2024-05-27T09:29:34.401 INFO:tasks.workunit.client.0.smithi119.stdout:0/783: chown d2/d6/d9/d12/da3 812595 1 2024-05-27T09:29:34.401 INFO:tasks.workunit.client.0.smithi119.stdout:1/835: symlink d7/db/d52/l101 0 2024-05-27T09:29:34.401 INFO:tasks.workunit.client.0.smithi119.stdout:1/836: write d7/db6/fbc [409951,114394] 0 2024-05-27T09:29:34.401 INFO:tasks.workunit.client.0.smithi119.stdout:1/837: chown d7/d40/cab 248 1 2024-05-27T09:29:34.401 INFO:tasks.workunit.client.0.smithi119.stdout:4/849: symlink d4/de/d21/d40/d47/l10a 0 2024-05-27T09:29:34.401 INFO:tasks.workunit.client.0.smithi119.stdout:3/919: dread db/fc [0,4194304] 0 2024-05-27T09:29:34.401 INFO:tasks.workunit.client.0.smithi119.stdout:7/847: rename d7/d1a/d99/dbb/ff1 to d7/df/d19/d97/d92/db7/dba/f10d 0 2024-05-27T09:29:34.401 INFO:tasks.workunit.client.0.smithi119.stdout:3/920: symlink db/d36/d45/d4d/d108/l12c 0 2024-05-27T09:29:34.401 INFO:tasks.workunit.client.0.smithi119.stdout:7/848: mknod d7/d1a/d99/dcf/c10e 0 2024-05-27T09:29:34.401 INFO:tasks.workunit.client.0.smithi119.stdout:7/849: fsync d7/fa 0 2024-05-27T09:29:34.401 INFO:tasks.workunit.client.0.smithi119.stdout:1/838: link d7/dd/d25/d60/d61/fb3 d7/db/d12/d15/dd2/de8/f102 0 2024-05-27T09:29:34.401 INFO:tasks.workunit.client.0.smithi119.stdout:1/839: unlink d7/f27 0 2024-05-27T09:29:34.401 INFO:tasks.workunit.client.0.smithi119.stdout:1/840: symlink d7/db/d12/d26/d39/l103 0 2024-05-27T09:29:34.402 INFO:tasks.workunit.client.0.smithi119.stdout:1/841: write d7/db/d52/d65/ff4 [531869,112269] 0 2024-05-27T09:29:34.402 INFO:tasks.workunit.client.0.smithi119.stdout:1/842: chown d7/db/d8f 1325620 1 2024-05-27T09:29:34.402 INFO:tasks.workunit.client.0.smithi119.stdout:1/843: link d7/db/d12/feb d7/db/d8f/d9b/f104 0 2024-05-27T09:29:34.402 INFO:tasks.workunit.client.0.smithi119.stdout:9/682: write d18/d19/fc2 [4103612,82307] 0 2024-05-27T09:29:34.402 INFO:tasks.workunit.client.0.smithi119.stdout:9/683: readlink d18/d19/d1b/d40/l84 0 2024-05-27T09:29:34.402 INFO:tasks.workunit.client.0.smithi119.stdout:9/684: readlink d18/d19/d1b/d40/d44/l5b 0 2024-05-27T09:29:34.402 INFO:tasks.workunit.client.0.smithi119.stdout:4/850: dread d4/d23/f35 [0,4194304] 0 2024-05-27T09:29:34.402 INFO:tasks.workunit.client.0.smithi119.stdout:9/685: write d18/f8b [268135,15876] 0 2024-05-27T09:29:34.402 INFO:tasks.workunit.client.0.smithi119.stdout:4/851: mkdir d4/d6/d81/d10b 0 2024-05-27T09:29:34.402 INFO:tasks.workunit.client.0.smithi119.stdout:9/686: creat d18/d19/d1b/d40/d44/dcc/fcf x:0 0 0 2024-05-27T09:29:34.402 INFO:tasks.workunit.client.0.smithi119.stdout:9/687: fdatasync d18/d19/d1b/d40/d7b/f89 0 2024-05-27T09:29:34.402 INFO:tasks.workunit.client.0.smithi119.stdout:9/688: fsync d18/d19/d1b/f64 0 2024-05-27T09:29:34.402 INFO:tasks.workunit.client.0.smithi119.stdout:5/879: dwrite d6/db/de/f89 [0,4194304] 0 2024-05-27T09:29:34.402 INFO:tasks.workunit.client.0.smithi119.stdout:9/689: creat d18/d19/d1b/d29/dc4/fd0 x:0 0 0 2024-05-27T09:29:34.403 INFO:tasks.workunit.client.0.smithi119.stdout:9/690: truncate d18/f8b 1501771 0 2024-05-27T09:29:34.403 INFO:tasks.workunit.client.0.smithi119.stdout:5/880: creat d6/dd/d46/d47/d4b/dd8/d124/f139 x:0 0 0 2024-05-27T09:29:34.403 INFO:tasks.workunit.client.0.smithi119.stdout:0/784: dwrite d2/d6/d9/d12/d4f/f89 [4194304,4194304] 0 2024-05-27T09:29:34.403 INFO:tasks.workunit.client.0.smithi119.stdout:5/881: unlink d6/dd/d39/d19/d70/dc1/f75 0 2024-05-27T09:29:34.403 INFO:tasks.workunit.client.0.smithi119.stdout:5/882: fdatasync d6/dd/fd9 0 2024-05-27T09:29:34.403 INFO:tasks.workunit.client.0.smithi119.stdout:5/883: chown d6/c28 5414441 1 2024-05-27T09:29:34.403 INFO:tasks.workunit.client.0.smithi119.stdout:5/884: creat d6/dd/d39/d19/d7d/dc2/f13a x:0 0 0 2024-05-27T09:29:34.403 INFO:tasks.workunit.client.0.smithi119.stdout:5/885: write d6/f63 [2703642,111529] 0 2024-05-27T09:29:34.403 INFO:tasks.workunit.client.0.smithi119.stdout:3/921: dread db/dc9/d98/f73 [0,4194304] 0 2024-05-27T09:29:34.403 INFO:tasks.workunit.client.0.smithi119.stdout:3/922: creat db/d36/d45/d49/d9d/f12d x:0 0 0 2024-05-27T09:29:34.403 INFO:tasks.workunit.client.0.smithi119.stdout:5/886: dread d6/d2f/f3a [0,4194304] 0 2024-05-27T09:29:34.403 INFO:tasks.workunit.client.0.smithi119.stdout:5/887: readlink d6/dd/d46/d47/d4b/dd8/lf9 0 2024-05-27T09:29:34.404 INFO:tasks.workunit.client.1.smithi138.stdout:6/375: dwrite d3/d11/d2d/f75 [0,4194304] 0 2024-05-27T09:29:34.404 INFO:tasks.workunit.client.1.smithi138.stdout:9/278: dread d6/db/de/d18/d24/f4b [0,4194304] 0 2024-05-27T09:29:34.405 INFO:tasks.workunit.client.1.smithi138.stdout:0/292: dread da/f10 [0,4194304] 0 2024-05-27T09:29:34.406 INFO:tasks.workunit.client.1.smithi138.stdout:9/279: link d6/db/de/f2b d6/da/d26/d28/f58 0 2024-05-27T09:29:34.414 INFO:tasks.workunit.client.0.smithi119.stdout:1/844: dread d7/db/d12/ddb/d38/d3c/fbf [0,4194304] 0 2024-05-27T09:29:34.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.421+0000 7faf55c00700 1 -- 172.21.15.119:0/3079860678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf50076950 msgr2=0x7faf50076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:34.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.421+0000 7faf55c00700 1 --2- 172.21.15.119:0/3079860678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf50076950 0x7faf50076dc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faf3c009910 tx=0x7faf3c009c20 comp rx=0 tx=0).stop 2024-05-27T09:29:34.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.421+0000 7faf55c00700 1 -- 172.21.15.119:0/3079860678 shutdown_connections 2024-05-27T09:29:34.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.421+0000 7faf55c00700 1 --2- 172.21.15.119:0/3079860678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf50076950 0x7faf50076dc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:34.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.421+0000 7faf55c00700 1 --2- 172.21.15.119:0/3079860678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf50075700 0x7faf50075b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:34.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.421+0000 7faf55c00700 1 -- 172.21.15.119:0/3079860678 >> 172.21.15.119:0/3079860678 conn(0x7faf500fd960 msgr2=0x7faf500ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:34.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.421+0000 7faf55c00700 1 -- 172.21.15.119:0/3079860678 shutdown_connections 2024-05-27T09:29:34.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.422+0000 7faf55c00700 1 -- 172.21.15.119:0/3079860678 wait complete. 2024-05-27T09:29:34.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.423+0000 7faf55c00700 1 Processor -- start 2024-05-27T09:29:34.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.424+0000 7faf55c00700 1 -- start start 2024-05-27T09:29:34.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.424+0000 7faf55c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf50075700 0x7faf5019bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:34.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.424+0000 7faf55c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf50076950 0x7faf5019c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:34.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.424+0000 7faf55c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5019cae0 con 0x7faf50076950 2024-05-27T09:29:34.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.424+0000 7faf55c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5019cc20 con 0x7faf50075700 2024-05-27T09:29:34.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.425+0000 7faf4e000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf50076950 0x7faf5019c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:34.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.425+0000 7faf4e000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf50076950 0x7faf5019c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33826/0 (socket says 172.21.15.119:33826) 2024-05-27T09:29:34.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.425+0000 7faf4e000700 1 -- 172.21.15.119:0/1345970228 learned_addr learned my addr 172.21.15.119:0/1345970228 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:29:34.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.425+0000 7faf4ea00700 1 --2- 172.21.15.119:0/1345970228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf50075700 0x7faf5019bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:34.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.425+0000 7faf4ea00700 1 -- 172.21.15.119:0/1345970228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf50076950 msgr2=0x7faf5019c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:34.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.425+0000 7faf4ea00700 1 --2- 172.21.15.119:0/1345970228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf50076950 0x7faf5019c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:34.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.426+0000 7faf4ea00700 1 -- 172.21.15.119:0/1345970228 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c0095f0 con 0x7faf50075700 2024-05-27T09:29:34.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.426+0000 7faf4ea00700 1 --2- 172.21.15.119:0/1345970228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf50075700 0x7faf5019bf80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faf4000d8d0 tx=0x7faf4000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:34.428 INFO:tasks.workunit.client.1.smithi138.stdout:6/376: creat d3/d11/d2d/d37/f85 x:0 0 0 2024-05-27T09:29:34.428 INFO:tasks.workunit.client.1.smithi138.stdout:6/377: creat d3/d35/d5a/d6a/f86 x:0 0 0 2024-05-27T09:29:34.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.427+0000 7faf47400700 1 -- 172.21.15.119:0/1345970228 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf40009900 con 0x7faf50075700 2024-05-27T09:29:34.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.427+0000 7faf47400700 1 -- 172.21.15.119:0/1345970228 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faf40010460 con 0x7faf50075700 2024-05-27T09:29:34.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.427+0000 7faf47400700 1 -- 172.21.15.119:0/1345970228 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf4000b5d0 con 0x7faf50075700 2024-05-27T09:29:34.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.427+0000 7faf55c00700 1 -- 172.21.15.119:0/1345970228 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf5006a910 con 0x7faf50075700 2024-05-27T09:29:34.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.427+0000 7faf55c00700 1 -- 172.21.15.119:0/1345970228 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf5006ae60 con 0x7faf50075700 2024-05-27T09:29:34.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.429+0000 7faf55c00700 1 -- 172.21.15.119:0/1345970228 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5010f300 con 0x7faf50075700 2024-05-27T09:29:34.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.431+0000 7faf47400700 1 -- 172.21.15.119:0/1345970228 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) v1 ==== 95024+0+0 (secure 0 0 0) 0x7faf400105d0 con 0x7faf50075700 2024-05-27T09:29:34.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.433+0000 7faf47400700 1 --2- 172.21.15.119:0/1345970228 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7faf340727d0 0x7faf34074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:34.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.433+0000 7faf47400700 1 -- 172.21.15.119:0/1345970228 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7faf40092ed0 con 0x7faf50075700 2024-05-27T09:29:34.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.433+0000 7faf4e000700 1 --2- 172.21.15.119:0/1345970228 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7faf340727d0 0x7faf34074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:34.434 INFO:tasks.workunit.client.1.smithi138.stdout:0/293: truncate da/d2a/d2b/f33 3422906 0 2024-05-27T09:29:34.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.435+0000 7faf4e000700 1 --2- 172.21.15.119:0/1345970228 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7faf340727d0 0x7faf34074c80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00b5c0 tx=0x7faf3c005a90 comp rx=0 tx=0).ready entity=mgr.14570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:34.440 INFO:tasks.workunit.client.1.smithi138.stdout:9/280: write d6/f3f [4162891,47937] 0 2024-05-27T09:29:34.440 INFO:tasks.workunit.client.1.smithi138.stdout:9/281: truncate d6/f48 4924692 0 2024-05-27T09:29:34.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.439+0000 7faf47400700 1 -- 172.21.15.119:0/1345970228 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7faf4005af80 con 0x7faf50075700 2024-05-27T09:29:34.442 INFO:tasks.workunit.client.1.smithi138.stdout:9/282: mkdir d6/da/d29/d59 0 2024-05-27T09:29:34.442 INFO:tasks.workunit.client.1.smithi138.stdout:9/283: fsync d6/db/de/d18/d27/d46/f47 0 2024-05-27T09:29:34.443 INFO:tasks.workunit.client.0.smithi119.stdout:3/923: unlink db/d36/d45/d4c/d62/d13/d33/f104 0 2024-05-27T09:29:34.445 INFO:tasks.workunit.client.0.smithi119.stdout:5/888: truncate d6/dd/d39/d19/d3c/d52/fbd 984067 0 2024-05-27T09:29:34.445 INFO:tasks.workunit.client.0.smithi119.stdout:5/889: read - d6/dd/fff zero size 2024-05-27T09:29:34.445 INFO:tasks.workunit.client.0.smithi119.stdout:5/890: fdatasync d6/dd/d39/d19/d7d/db7/f132 0 2024-05-27T09:29:34.445 INFO:tasks.workunit.client.0.smithi119.stdout:5/891: write d6/db/de/d27/d94/fa9 [1367605,36563] 0 2024-05-27T09:29:34.445 INFO:tasks.workunit.client.0.smithi119.stdout:5/892: truncate d6/db/dc/fd2 5156346 0 2024-05-27T09:29:34.448 INFO:tasks.workunit.client.1.smithi138.stdout:6/378: dread d3/d35/d43/f48 [0,4194304] 0 2024-05-27T09:29:34.448 INFO:tasks.workunit.client.0.smithi119.stdout:4/852: dwrite d4/d23/d24/d5b/da8/dda/fe2 [0,4194304] 0 2024-05-27T09:29:34.448 INFO:tasks.workunit.client.0.smithi119.stdout:4/853: fsync d4/de/d21/f2d 0 2024-05-27T09:29:34.449 INFO:tasks.workunit.client.1.smithi138.stdout:7/267: dwrite d0/d7/f54 [0,4194304] 0 2024-05-27T09:29:34.451 INFO:tasks.workunit.client.0.smithi119.stdout:1/845: unlink d7/dd/f5d 0 2024-05-27T09:29:34.455 INFO:tasks.workunit.client.1.smithi138.stdout:0/294: rename da/d2a/f35 to da/f66 0 2024-05-27T09:29:34.455 INFO:tasks.workunit.client.1.smithi138.stdout:0/295: fsync da/d17/d1d/d1f/f3e 0 2024-05-27T09:29:34.455 INFO:tasks.workunit.client.0.smithi119.stdout:0/785: dwrite d2/d1a/d20/f64 [4194304,4194304] 0 2024-05-27T09:29:34.455 INFO:tasks.workunit.client.0.smithi119.stdout:0/786: chown d2/d6/c5d 126965943 1 2024-05-27T09:29:34.462 INFO:tasks.workunit.client.0.smithi119.stdout:7/850: dwrite d7/df/d19/d9a/da6/dda/f48 [0,4194304] 0 2024-05-27T09:29:34.462 INFO:tasks.workunit.client.0.smithi119.stdout:7/851: fdatasync d7/d1a/f2b 0 2024-05-27T09:29:34.465 INFO:tasks.workunit.client.1.smithi138.stdout:6/379: mknod d3/d35/d5a/c87 0 2024-05-27T09:29:34.465 INFO:tasks.workunit.client.1.smithi138.stdout:6/380: write d3/fa [1433152,108331] 0 2024-05-27T09:29:34.470 INFO:tasks.workunit.client.0.smithi119.stdout:9/691: dwrite d18/f9a [0,4194304] 0 2024-05-27T09:29:34.470 INFO:tasks.workunit.client.0.smithi119.stdout:9/692: readlink d18/d19/d1b/l53 0 2024-05-27T09:29:34.470 INFO:tasks.workunit.client.1.smithi138.stdout:1/300: dwrite d4/f12 [0,4194304] 0 2024-05-27T09:29:34.471 INFO:tasks.workunit.client.1.smithi138.stdout:1/301: getdents d4/d1a/d2e/d4a 0 2024-05-27T09:29:34.471 INFO:tasks.workunit.client.0.smithi119.stdout:9/693: write d18/d19/d1b/d2a/f8a [4417820,94364] 0 2024-05-27T09:29:34.471 INFO:tasks.workunit.client.0.smithi119.stdout:9/694: chown d18/l3d 10645 1 2024-05-27T09:29:34.474 INFO:tasks.workunit.client.1.smithi138.stdout:0/296: mknod da/d17/d1d/d1f/d60/c67 0 2024-05-27T09:29:34.477 INFO:tasks.workunit.client.1.smithi138.stdout:6/381: creat d3/d11/f88 x:0 0 0 2024-05-27T09:29:34.480 INFO:tasks.workunit.client.1.smithi138.stdout:8/250: dread d3/d8/d11/f26 [0,4194304] 0 2024-05-27T09:29:34.481 INFO:tasks.workunit.client.1.smithi138.stdout:8/251: symlink d3/d8/d24/l47 0 2024-05-27T09:29:34.482 INFO:tasks.workunit.client.1.smithi138.stdout:8/252: creat d3/d8/d11/f48 x:0 0 0 2024-05-27T09:29:34.482 INFO:tasks.workunit.client.1.smithi138.stdout:8/253: creat d3/d8/d32/f49 x:0 0 0 2024-05-27T09:29:34.482 INFO:tasks.workunit.client.1.smithi138.stdout:1/302: mknod d4/da/d4d/d2b/c5e 0 2024-05-27T09:29:34.484 INFO:tasks.workunit.client.1.smithi138.stdout:8/254: getdents d3/d8 0 2024-05-27T09:29:34.486 INFO:tasks.workunit.client.1.smithi138.stdout:8/255: creat d3/f4a x:0 0 0 2024-05-27T09:29:34.487 INFO:tasks.workunit.client.1.smithi138.stdout:8/256: dread - d3/d8/f42 zero size 2024-05-27T09:29:34.487 INFO:tasks.workunit.client.1.smithi138.stdout:8/257: creat d3/d8/d28/f4b x:0 0 0 2024-05-27T09:29:34.489 INFO:tasks.workunit.client.0.smithi119.stdout:5/893: creat d6/dd/d46/d47/d4b/dd8/d124/d100/d12a/f13b x:0 0 0 2024-05-27T09:29:34.490 INFO:tasks.workunit.client.1.smithi138.stdout:8/258: creat d3/d8/f4c x:0 0 0 2024-05-27T09:29:34.490 INFO:tasks.workunit.client.1.smithi138.stdout:8/259: truncate d3/d8/d11/f48 619006 0 2024-05-27T09:29:34.492 INFO:tasks.workunit.client.0.smithi119.stdout:8/761: sync 2024-05-27T09:29:34.492 INFO:tasks.workunit.client.0.smithi119.stdout:6/766: sync 2024-05-27T09:29:34.492 INFO:tasks.workunit.client.0.smithi119.stdout:2/828: sync 2024-05-27T09:29:34.492 INFO:tasks.workunit.client.1.smithi138.stdout:0/297: mknod da/d17/d1d/d1f/d60/c68 0 2024-05-27T09:29:34.501 INFO:tasks.workunit.client.0.smithi119.stdout:4/854: mkdir d4/de/d21/d40/d47/d5c/d9e/dd3/d10c 0 2024-05-27T09:29:34.510 INFO:tasks.workunit.client.0.smithi119.stdout:0/787: truncate d2/d6/d9/d12/f79 8256650 0 2024-05-27T09:29:34.510 INFO:tasks.workunit.client.0.smithi119.stdout:0/788: write d2/d6/d9/d12/f16 [2690207,29045] 0 2024-05-27T09:29:34.511 INFO:tasks.workunit.client.0.smithi119.stdout:0/789: write d2/d6/d23/f33 [4193402,104456] 0 2024-05-27T09:29:34.513 INFO:tasks.workunit.client.0.smithi119.stdout:7/852: symlink d7/df/d19/d97/d92/db7/ddc/l10f 0 2024-05-27T09:29:34.513 INFO:tasks.workunit.client.1.smithi138.stdout:8/260: getdents d3/d8/d24 0 2024-05-27T09:29:34.514 INFO:tasks.workunit.client.1.smithi138.stdout:8/261: read - d3/d8/d28/f40 zero size 2024-05-27T09:29:34.515 INFO:tasks.workunit.client.0.smithi119.stdout:3/924: mknod db/d36/d45/d4c/d62/c12e 0 2024-05-27T09:29:34.515 INFO:tasks.workunit.client.0.smithi119.stdout:3/925: readlink db/d7f/la2 0 2024-05-27T09:29:34.516 INFO:tasks.workunit.client.0.smithi119.stdout:3/926: dread - db/d36/d45/d4c/dbe/ff1 zero size 2024-05-27T09:29:34.516 INFO:tasks.workunit.client.0.smithi119.stdout:3/927: creat db/d36/d45/d4c/dbe/f12f x:0 0 0 2024-05-27T09:29:34.518 INFO:tasks.workunit.client.1.smithi138.stdout:6/382: dread d3/d35/d43/d1c/d32/d47/f5e [0,4194304] 0 2024-05-27T09:29:34.519 INFO:tasks.workunit.client.0.smithi119.stdout:5/894: mkdir d6/db/dc/d10b/d13c 0 2024-05-27T09:29:34.519 INFO:tasks.workunit.client.1.smithi138.stdout:8/262: mkdir d3/d8/d24/d4d 0 2024-05-27T09:29:34.519 INFO:tasks.workunit.client.1.smithi138.stdout:8/263: write d3/d8/f4c [127978,24989] 0 2024-05-27T09:29:34.519 INFO:tasks.workunit.client.1.smithi138.stdout:6/383: creat d3/d19/f89 x:0 0 0 2024-05-27T09:29:34.520 INFO:tasks.workunit.client.1.smithi138.stdout:0/298: dread da/d17/d1d/d1f/f38 [0,4194304] 0 2024-05-27T09:29:34.520 INFO:tasks.workunit.client.1.smithi138.stdout:0/299: fsync da/d2a/d3f/f4c 0 2024-05-27T09:29:34.522 INFO:tasks.workunit.client.0.smithi119.stdout:8/762: rename d3/d8/d1b/f28 to d3/d8/d1b/d46/da4/dbb/ffb 0 2024-05-27T09:29:34.524 INFO:tasks.workunit.client.1.smithi138.stdout:8/264: creat d3/d8/d32/d45/f4e x:0 0 0 2024-05-27T09:29:34.524 INFO:tasks.workunit.client.1.smithi138.stdout:8/265: fsync d3/d8/f14 0 2024-05-27T09:29:34.524 INFO:tasks.workunit.client.1.smithi138.stdout:8/266: readlink d3/d8/d32/d34/d38/l43 0 2024-05-27T09:29:34.524 INFO:tasks.workunit.client.1.smithi138.stdout:8/267: readlink d3/l1d 0 2024-05-27T09:29:34.525 INFO:tasks.workunit.client.1.smithi138.stdout:3/227: dwrite d0/d2/f27 [0,4194304] 0 2024-05-27T09:29:34.528 INFO:tasks.workunit.client.0.smithi119.stdout:4/855: link d4/de/d9b/fba d4/d6/d28/d2f/db8/f10d 0 2024-05-27T09:29:34.528 INFO:tasks.workunit.client.0.smithi119.stdout:4/856: dread - d4/d6/d28/ff8 zero size 2024-05-27T09:29:34.528 INFO:tasks.workunit.client.0.smithi119.stdout:4/857: write d4/de/d21/d40/f7c [1171974,32929] 0 2024-05-27T09:29:34.529 INFO:tasks.workunit.client.0.smithi119.stdout:2/829: symlink d2/da/d8a/dd4/d89/d94/d87/dcf/l11b 0 2024-05-27T09:29:34.531 INFO:tasks.workunit.client.0.smithi119.stdout:1/846: getdents d7/db/d12/d26/d4a 0 2024-05-27T09:29:34.531 INFO:tasks.workunit.client.0.smithi119.stdout:0/790: dread d2/d6/d9/d12/d4b/d59/f7d [0,4194304] 0 2024-05-27T09:29:34.534 INFO:tasks.workunit.client.1.smithi138.stdout:3/228: getdents d0/d3f 0 2024-05-27T09:29:34.537 INFO:tasks.workunit.client.1.smithi138.stdout:3/229: mknod d0/d2/da/df/d10/d32/c47 0 2024-05-27T09:29:34.539 INFO:tasks.workunit.client.0.smithi119.stdout:5/895: link d6/db/de/d27/l3d d6/dd/d39/d19/d7d/dc2/l13d 0 2024-05-27T09:29:34.539 INFO:tasks.workunit.client.1.smithi138.stdout:3/230: mkdir d0/d2/da/df/d10/d2a/d48 0 2024-05-27T09:29:34.539 INFO:tasks.workunit.client.1.smithi138.stdout:3/231: getdents d0/d2/da/df/d10/d2a/d48 0 2024-05-27T09:29:34.540 INFO:tasks.workunit.client.0.smithi119.stdout:6/767: rmdir d0/d2/d1c/d21/d58 39 2024-05-27T09:29:34.542 INFO:tasks.workunit.client.1.smithi138.stdout:3/232: creat d0/d2/da/df/d10/d2a/d48/f49 x:0 0 0 2024-05-27T09:29:34.542 INFO:tasks.workunit.client.1.smithi138.stdout:9/284: dwrite d6/db/dc/f1e [0,4194304] 0 2024-05-27T09:29:34.543 INFO:tasks.workunit.client.1.smithi138.stdout:3/233: creat d0/d2/da/df/d10/f4a x:0 0 0 2024-05-27T09:29:34.543 INFO:tasks.workunit.client.1.smithi138.stdout:3/234: chown d0/l39 0 1 2024-05-27T09:29:34.545 INFO:tasks.workunit.client.1.smithi138.stdout:9/285: link d6/da/d29/c3e d6/da/d29/c5a 0 2024-05-27T09:29:34.545 INFO:tasks.workunit.client.1.smithi138.stdout:9/286: write d6/db/de/d18/d27/d46/f47 [4585120,97177] 0 2024-05-27T09:29:34.545 INFO:tasks.workunit.client.1.smithi138.stdout:9/287: readlink d6/da/d2f/l45 0 2024-05-27T09:29:34.546 INFO:tasks.workunit.client.1.smithi138.stdout:3/235: rename d0/d2/f27 to d0/d2/f4b 0 2024-05-27T09:29:34.546 INFO:tasks.workunit.client.1.smithi138.stdout:3/236: chown d0/d2/da/f1b 53843258 1 2024-05-27T09:29:34.546 INFO:tasks.workunit.client.1.smithi138.stdout:3/237: write d0/fd [205709,77258] 0 2024-05-27T09:29:34.547 INFO:tasks.workunit.client.0.smithi119.stdout:4/858: chown d4/d23/d31/f33 94 1 2024-05-27T09:29:34.547 INFO:tasks.workunit.client.0.smithi119.stdout:7/853: write d7/df/d19/d97/f9f [6677649,33417] 0 2024-05-27T09:29:34.547 INFO:tasks.workunit.client.1.smithi138.stdout:9/288: mknod d6/db/de/d18/d24/c5b 0 2024-05-27T09:29:34.548 INFO:tasks.workunit.client.1.smithi138.stdout:9/289: chown d6/f2e 267508 1 2024-05-27T09:29:34.548 INFO:tasks.workunit.client.1.smithi138.stdout:3/238: mknod d0/d2/da/df/d10/c4c 0 2024-05-27T09:29:34.548 INFO:tasks.workunit.client.1.smithi138.stdout:3/239: creat d0/d2/da/df/d10/d2a/d48/f4d x:0 0 0 2024-05-27T09:29:34.548 INFO:tasks.workunit.client.1.smithi138.stdout:3/240: fsync d0/f12 0 2024-05-27T09:29:34.554 INFO:tasks.workunit.client.1.smithi138.stdout:9/290: unlink d6/f1f 0 2024-05-27T09:29:34.556 INFO:tasks.workunit.client.0.smithi119.stdout:0/791: creat d2/d6/d9/d24/d30/dcc/de6/f104 x:0 0 0 2024-05-27T09:29:34.557 INFO:tasks.workunit.client.0.smithi119.stdout:5/896: rename d6/dd/d46/fb8 to d6/db/de/d27/dc9/f13e 0 2024-05-27T09:29:34.563 INFO:tasks.workunit.client.0.smithi119.stdout:2/830: rmdir d2/da/d8a/dec 39 2024-05-27T09:29:34.563 INFO:tasks.workunit.client.0.smithi119.stdout:2/831: fsync d2/da/d6f/f7f 0 2024-05-27T09:29:34.563 INFO:tasks.workunit.client.0.smithi119.stdout:2/832: readlink d2/da/d8a/dd4/d69/lbd 0 2024-05-27T09:29:34.563 INFO:tasks.workunit.client.1.smithi138.stdout:6/384: read d3/d11/d33/f66 [363675,27886] 0 2024-05-27T09:29:34.564 INFO:tasks.workunit.client.1.smithi138.stdout:6/385: creat d3/d11/f8a x:0 0 0 2024-05-27T09:29:34.564 INFO:tasks.workunit.client.1.smithi138.stdout:9/291: creat d6/db/de/d18/d24/f5c x:0 0 0 2024-05-27T09:29:34.564 INFO:tasks.workunit.client.1.smithi138.stdout:6/386: write d3/d11/d33/f66 [1327664,130229] 0 2024-05-27T09:29:34.570 INFO:tasks.workunit.client.0.smithi119.stdout:1/847: rmdir d7/dd/dcb 39 2024-05-27T09:29:34.580 INFO:tasks.workunit.client.0.smithi119.stdout:9/695: dwrite d18/d19/d1b/d29/f7c [0,4194304] 0 2024-05-27T09:29:34.580 INFO:tasks.workunit.client.0.smithi119.stdout:9/696: stat d18/d19/f74 0 2024-05-27T09:29:34.584 INFO:tasks.workunit.client.0.smithi119.stdout:5/897: link d6/db/l8c d6/d2f/l13f 0 2024-05-27T09:29:34.586 INFO:tasks.workunit.client.0.smithi119.stdout:1/848: mknod d7/db/d8f/c105 0 2024-05-27T09:29:34.595 INFO:tasks.workunit.client.1.smithi138.stdout:3/241: dread d0/d2/f13 [0,4194304] 0 2024-05-27T09:29:34.598 INFO:tasks.workunit.client.1.smithi138.stdout:3/242: dread - d0/d2/da/df/d10/d2a/d48/f49 zero size 2024-05-27T09:29:34.599 INFO:tasks.workunit.client.1.smithi138.stdout:6/387: mkdir d3/d11/d2d/d37/d8b 0 2024-05-27T09:29:34.599 INFO:tasks.workunit.client.0.smithi119.stdout:1/849: truncate d7/dd/d44/d47/f72 1441379 0 2024-05-27T09:29:34.599 INFO:tasks.workunit.client.0.smithi119.stdout:9/697: creat d18/fd1 x:0 0 0 2024-05-27T09:29:34.600 INFO:tasks.workunit.client.0.smithi119.stdout:1/850: write f3 [3674670,43907] 0 2024-05-27T09:29:34.607 INFO:tasks.workunit.client.0.smithi119.stdout:9/698: symlink d18/d19/d1b/d2a/d9f/ld2 0 2024-05-27T09:29:34.609 INFO:tasks.workunit.client.1.smithi138.stdout:1/303: dwrite d4/d48/d39/f54 [0,4194304] 0 2024-05-27T09:29:34.609 INFO:tasks.workunit.client.1.smithi138.stdout:9/292: dread d6/db/f39 [0,4194304] 0 2024-05-27T09:29:34.609 INFO:tasks.workunit.client.1.smithi138.stdout:9/293: readlink d6/db/de/d18/l54 0 2024-05-27T09:29:34.613 INFO:tasks.workunit.client.1.smithi138.stdout:9/294: write d6/da/d26/d28/f49 [246627,39165] 0 2024-05-27T09:29:34.614 INFO:tasks.workunit.client.1.smithi138.stdout:6/388: mkdir d3/d35/d43/d8c 0 2024-05-27T09:29:34.614 INFO:tasks.workunit.client.1.smithi138.stdout:6/389: stat d3/d11/l62 0 2024-05-27T09:29:34.614 INFO:tasks.workunit.client.1.smithi138.stdout:6/390: chown d3/f84 0 1 2024-05-27T09:29:34.616 INFO:tasks.workunit.client.1.smithi138.stdout:9/295: creat d6/da/f5d x:0 0 0 2024-05-27T09:29:34.616 INFO:tasks.workunit.client.1.smithi138.stdout:9/296: readlink d6/db/de/d18/l23 0 2024-05-27T09:29:34.616 INFO:tasks.workunit.client.1.smithi138.stdout:9/297: fdatasync d6/db/de/d18/d24/f4b 0 2024-05-27T09:29:34.638 INFO:tasks.workunit.client.0.smithi119.stdout:1/851: rmdir d7/db/d52 39 2024-05-27T09:29:34.640 INFO:tasks.workunit.client.1.smithi138.stdout:3/243: dread d0/d2/da/df/d10/d32/f9 [0,4194304] 0 2024-05-27T09:29:34.640 INFO:tasks.workunit.client.0.smithi119.stdout:3/928: dread db/f2f [0,4194304] 0 2024-05-27T09:29:34.640 INFO:tasks.workunit.client.0.smithi119.stdout:3/929: chown db/d36/d45/d4c/d62/d13 0 1 2024-05-27T09:29:34.641 INFO:tasks.workunit.client.1.smithi138.stdout:3/244: mknod d0/d2/da/df/d10/d32/c4e 0 2024-05-27T09:29:34.643 INFO:tasks.workunit.client.1.smithi138.stdout:3/245: rmdir d0/d2/da/df/d10/d2a/d48 39 2024-05-27T09:29:34.648 INFO:tasks.workunit.client.1.smithi138.stdout:4/311: sync 2024-05-27T09:29:34.648 INFO:tasks.workunit.client.1.smithi138.stdout:5/254: sync 2024-05-27T09:29:34.648 INFO:tasks.workunit.client.1.smithi138.stdout:5/255: write d3/d12/d16/f39 [5245108,70318] 0 2024-05-27T09:29:34.648 INFO:tasks.workunit.client.1.smithi138.stdout:2/230: sync 2024-05-27T09:29:34.648 INFO:tasks.workunit.client.0.smithi119.stdout:3/930: symlink db/d36/d45/d4c/d120/l130 0 2024-05-27T09:29:34.651 INFO:tasks.workunit.client.1.smithi138.stdout:2/231: mknod d1/d2f/c4a 0 2024-05-27T09:29:34.652 INFO:tasks.workunit.client.0.smithi119.stdout:9/699: read d18/d19/f74 [165235,91182] 0 2024-05-27T09:29:34.653 INFO:tasks.workunit.client.0.smithi119.stdout:9/700: chown d18/d19/f3f 341693 1 2024-05-27T09:29:34.655 INFO:tasks.workunit.client.0.smithi119.stdout:7/854: dwrite d7/d1a/f2d [0,4194304] 0 2024-05-27T09:29:34.655 INFO:tasks.workunit.client.0.smithi119.stdout:7/855: truncate d7/df/d19/d9a/da6/dda/f8b 9745 0 2024-05-27T09:29:34.655 INFO:tasks.workunit.client.0.smithi119.stdout:7/856: creat d7/df/d19/d97/f110 x:0 0 0 2024-05-27T09:29:34.656 INFO:tasks.workunit.client.1.smithi138.stdout:2/232: mkdir d1/d15/d2d/d31/d4b 0 2024-05-27T09:29:34.659 INFO:tasks.workunit.client.1.smithi138.stdout:2/233: creat d1/d15/d2d/d31/d4b/f4c x:0 0 0 2024-05-27T09:29:34.659 INFO:tasks.workunit.client.1.smithi138.stdout:1/304: rename d4/da to d4/d48/d39/d5f 0 2024-05-27T09:29:34.660 INFO:tasks.workunit.client.1.smithi138.stdout:5/256: mknod d3/d12/d26/d53/c54 0 2024-05-27T09:29:34.661 INFO:tasks.workunit.client.0.smithi119.stdout:4/859: dread d4/d6/d28/f6b [0,4194304] 0 2024-05-27T09:29:34.662 INFO:tasks.workunit.client.1.smithi138.stdout:2/234: unlink d1/d15/d2d/d17/f11 0 2024-05-27T09:29:34.665 INFO:tasks.workunit.client.1.smithi138.stdout:3/246: dread d0/d2/da/df/d10/d32/f28 [0,4194304] 0 2024-05-27T09:29:34.667 INFO:tasks.workunit.client.0.smithi119.stdout:2/833: dread d2/da/d2c/f58 [0,4194304] 0 2024-05-27T09:29:34.667 INFO:tasks.workunit.client.0.smithi119.stdout:3/931: creat db/d36/d45/d4c/d120/f131 x:0 0 0 2024-05-27T09:29:34.667 INFO:tasks.workunit.client.0.smithi119.stdout:3/932: fdatasync db/f83 0 2024-05-27T09:29:34.667 INFO:tasks.workunit.client.0.smithi119.stdout:2/834: dread - d2/deb/ff8 zero size 2024-05-27T09:29:34.670 INFO:tasks.workunit.client.1.smithi138.stdout:3/247: truncate d0/d2/da/f33 2476572 0 2024-05-27T09:29:34.674 INFO:tasks.workunit.client.1.smithi138.stdout:2/235: mkdir d1/d15/d2d/d17/d25/d4d 0 2024-05-27T09:29:34.674 INFO:tasks.workunit.client.1.smithi138.stdout:3/248: rmdir d0/d2/da 39 2024-05-27T09:29:34.677 INFO:tasks.workunit.client.0.smithi119.stdout:9/701: rename d18/d19/d61/f92 to d18/d19/d1b/d40/d44/fd3 0 2024-05-27T09:29:34.680 INFO:tasks.workunit.client.1.smithi138.stdout:1/305: link d4/d48/d39/d5f/d4d/f3f d4/d1a/d2e/d32/f60 0 2024-05-27T09:29:34.680 INFO:tasks.workunit.client.1.smithi138.stdout:1/306: stat d4/d1a/d2e/d32/f60 0 2024-05-27T09:29:34.682 INFO:tasks.workunit.client.1.smithi138.stdout:1/307: creat d4/d3c/d51/f61 x:0 0 0 2024-05-27T09:29:34.683 INFO:tasks.workunit.client.1.smithi138.stdout:1/308: stat d4/d48/d22 0 2024-05-27T09:29:34.691 INFO:tasks.workunit.client.0.smithi119.stdout:6/768: dwrite d0/d2/d57/f50 [0,4194304] 0 2024-05-27T09:29:34.691 INFO:tasks.workunit.client.0.smithi119.stdout:6/769: fsync d0/d2/d4e/dc5/fee 0 2024-05-27T09:29:34.691 INFO:tasks.workunit.client.1.smithi138.stdout:4/312: dread d7/f3d [0,4194304] 0 2024-05-27T09:29:34.691 INFO:tasks.workunit.client.1.smithi138.stdout:5/257: creat d3/d12/d16/d1d/f55 x:0 0 0 2024-05-27T09:29:34.692 INFO:tasks.workunit.client.1.smithi138.stdout:2/236: mkdir d1/d15/d2d/d17/d25/d4d/d4e 0 2024-05-27T09:29:34.692 INFO:tasks.workunit.client.1.smithi138.stdout:2/237: creat d1/d15/f4f x:0 0 0 2024-05-27T09:29:34.697 INFO:tasks.workunit.client.1.smithi138.stdout:2/238: creat d1/d15/d2d/d2e/f50 x:0 0 0 2024-05-27T09:29:34.703 INFO:tasks.workunit.client.1.smithi138.stdout:3/249: write d0/d2/fc [389542,119768] 0 2024-05-27T09:29:34.706 INFO:tasks.workunit.client.1.smithi138.stdout:2/239: dread d1/d10/fe [0,4194304] 0 2024-05-27T09:29:34.709 INFO:tasks.workunit.client.0.smithi119.stdout:5/898: dread d6/db/de/d27/dc9/fc6 [0,4194304] 0 2024-05-27T09:29:34.709 INFO:tasks.workunit.client.0.smithi119.stdout:8/763: dwrite d3/d8/d4d/d5d/d60/f71 [8388608,4194304] 0 2024-05-27T09:29:34.709 INFO:tasks.workunit.client.0.smithi119.stdout:7/857: mkdir d7/d1a/d107/d111 0 2024-05-27T09:29:34.711 INFO:tasks.workunit.client.1.smithi138.stdout:6/391: dwrite d3/d11/f2e [4194304,4194304] 0 2024-05-27T09:29:34.712 INFO:tasks.workunit.client.1.smithi138.stdout:8/268: dwrite d3/d8/d11/f22 [0,4194304] 0 2024-05-27T09:29:34.715 INFO:tasks.workunit.client.0.smithi119.stdout:3/933: mknod db/d5b/c132 0 2024-05-27T09:29:34.729 INFO:tasks.workunit.client.0.smithi119.stdout:7/858: mkdir d7/d1a/d99/d112 0 2024-05-27T09:29:34.729 INFO:tasks.workunit.client.0.smithi119.stdout:7/859: write d7/df/d19/d97/d37/f5e [728824,55761] 0 2024-05-27T09:29:34.731 INFO:tasks.workunit.client.0.smithi119.stdout:1/852: dwrite d7/db/d8f/d9b/f104 [0,4194304] 0 2024-05-27T09:29:34.731 INFO:tasks.workunit.client.0.smithi119.stdout:1/853: fdatasync d7/f1e 0 2024-05-27T09:29:34.731 INFO:tasks.workunit.client.0.smithi119.stdout:1/854: creat d7/db/d12/f106 x:0 0 0 2024-05-27T09:29:34.731 INFO:tasks.workunit.client.0.smithi119.stdout:6/770: dread d0/f68 [4194304,4194304] 0 2024-05-27T09:29:34.737 INFO:tasks.workunit.client.0.smithi119.stdout:7/860: creat d7/d1a/f113 x:0 0 0 2024-05-27T09:29:34.737 INFO:tasks.workunit.client.1.smithi138.stdout:9/298: dwrite d6/db/dc/f50 [0,4194304] 0 2024-05-27T09:29:34.742 INFO:tasks.workunit.client.1.smithi138.stdout:3/250: write d0/d2/da/f33 [2253659,21441] 0 2024-05-27T09:29:34.742 INFO:tasks.workunit.client.1.smithi138.stdout:3/251: creat d0/d2/da/df/d10/d2a/f4f x:0 0 0 2024-05-27T09:29:34.743 INFO:tasks.workunit.client.0.smithi119.stdout:1/855: truncate d7/db/d12/ddb/d38/d3c/fd3 128236 0 2024-05-27T09:29:34.745 INFO:tasks.workunit.client.0.smithi119.stdout:6/771: creat d0/d2/f112 x:0 0 0 2024-05-27T09:29:34.750 INFO:tasks.workunit.client.0.smithi119.stdout:6/772: creat d0/d2/d4e/dc5/f113 x:0 0 0 2024-05-27T09:29:34.750 INFO:tasks.workunit.client.0.smithi119.stdout:6/773: chown d0/d2/d1c/d21/lf3 1042436 1 2024-05-27T09:29:34.750 INFO:tasks.workunit.client.0.smithi119.stdout:0/792: dwrite d2/d6/d75/fb3 [0,4194304] 0 2024-05-27T09:29:34.750 INFO:tasks.workunit.client.0.smithi119.stdout:7/861: mknod d7/df/d19/d97/d92/db7/dba/d42/c114 0 2024-05-27T09:29:34.750 INFO:tasks.workunit.client.0.smithi119.stdout:7/862: dread - d7/df/d19/d97/d92/db7/dba/d42/d64/dad/f104 zero size 2024-05-27T09:29:34.754 INFO:tasks.workunit.client.1.smithi138.stdout:4/313: rename d7/c1f to d7/d27/c67 0 2024-05-27T09:29:34.754 INFO:tasks.workunit.client.1.smithi138.stdout:4/314: readlink d7/df/d35/d4c/l63 0 2024-05-27T09:29:34.754 INFO:tasks.workunit.client.1.smithi138.stdout:9/299: chown d6/da/d2f/f40 43151769 1 2024-05-27T09:29:34.755 INFO:tasks.workunit.client.1.smithi138.stdout:6/392: mkdir d3/d11/d2d/d37/d7e/d8d 0 2024-05-27T09:29:34.755 INFO:tasks.workunit.client.1.smithi138.stdout:3/252: mkdir d0/d3f/d50 0 2024-05-27T09:29:34.755 INFO:tasks.workunit.client.1.smithi138.stdout:3/253: chown d0/l39 208 1 2024-05-27T09:29:34.755 INFO:tasks.workunit.client.1.smithi138.stdout:3/254: chown d0/d2/da/l2f 31 1 2024-05-27T09:29:34.755 INFO:tasks.workunit.client.1.smithi138.stdout:3/255: fsync d0/d2/da/df/d10/d32/f9 0 2024-05-27T09:29:34.763 INFO:tasks.workunit.client.0.smithi119.stdout:7/863: read d7/df/d19/d97/d92/db7/dba/fb3 [173519,113460] 0 2024-05-27T09:29:34.763 INFO:tasks.workunit.client.0.smithi119.stdout:7/864: stat d7/df/d19/d97/d92 0 2024-05-27T09:29:34.764 INFO:tasks.workunit.client.0.smithi119.stdout:0/793: unlink d2/d6/d9/d12/d4b/d59/dd7/c9a 0 2024-05-27T09:29:34.764 INFO:tasks.workunit.client.0.smithi119.stdout:0/794: fdatasync d2/f49 0 2024-05-27T09:29:34.764 INFO:tasks.workunit.client.0.smithi119.stdout:0/795: read - d2/d6/d9/d12/d4b/d96/fad zero size 2024-05-27T09:29:34.766 INFO:tasks.workunit.client.1.smithi138.stdout:3/256: symlink d0/d2/da/df/d10/d2a/l51 0 2024-05-27T09:29:34.773 INFO:tasks.workunit.client.0.smithi119.stdout:3/934: write db/d36/d45/d4c/d62/f3a [2969129,53687] 0 2024-05-27T09:29:34.773 INFO:tasks.workunit.client.0.smithi119.stdout:3/935: chown db/dc9/d98/db9/dd1/l102 91575730 1 2024-05-27T09:29:34.774 INFO:tasks.workunit.client.1.smithi138.stdout:4/315: creat d7/df/f68 x:0 0 0 2024-05-27T09:29:34.774 INFO:tasks.workunit.client.1.smithi138.stdout:4/316: creat d7/df/d35/d4c/f69 x:0 0 0 2024-05-27T09:29:34.777 INFO:tasks.workunit.client.0.smithi119.stdout:5/899: dread d6/f63 [4194304,4194304] 0 2024-05-27T09:29:34.777 INFO:tasks.workunit.client.1.smithi138.stdout:9/300: link d6/db/dc/cf d6/da/d26/c5e 0 2024-05-27T09:29:34.777 INFO:tasks.workunit.client.1.smithi138.stdout:9/301: read d6/db/dc/f12 [651539,129951] 0 2024-05-27T09:29:34.780 INFO:tasks.workunit.client.1.smithi138.stdout:1/309: dread d4/d38/f44 [0,4194304] 0 2024-05-27T09:29:34.780 INFO:tasks.workunit.client.1.smithi138.stdout:4/317: mkdir d7/df/d35/d4c/d59/d5d/d64/d6a 0 2024-05-27T09:29:34.781 INFO:tasks.workunit.client.1.smithi138.stdout:3/257: creat d0/d3f/d50/f52 x:0 0 0 2024-05-27T09:29:34.781 INFO:tasks.workunit.client.1.smithi138.stdout:3/258: creat d0/d2/da/df/d10/d2a/f53 x:0 0 0 2024-05-27T09:29:34.783 INFO:tasks.workunit.client.1.smithi138.stdout:1/310: rename d4/f12 to d4/d1a/d2e/f62 0 2024-05-27T09:29:34.787 INFO:tasks.workunit.client.1.smithi138.stdout:1/311: mkdir d4/d48/d39/d5f/d63 0 2024-05-27T09:29:34.792 INFO:tasks.workunit.client.0.smithi119.stdout:0/796: truncate d2/d1a/d20/f74 914155 0 2024-05-27T09:29:34.792 INFO:tasks.workunit.client.0.smithi119.stdout:0/797: stat d2/d6/d9/d12/cfb 0 2024-05-27T09:29:34.793 INFO:tasks.workunit.client.1.smithi138.stdout:4/318: rename d7/df/f52 to d7/d27/d3a/f6b 0 2024-05-27T09:29:34.798 INFO:tasks.workunit.client.1.smithi138.stdout:4/319: creat d7/df/d35/f6c x:0 0 0 2024-05-27T09:29:34.798 INFO:tasks.workunit.client.1.smithi138.stdout:4/320: readlink d7/d27/d3a/l3e 0 2024-05-27T09:29:34.802 INFO:tasks.workunit.client.1.smithi138.stdout:4/321: creat d7/df/d35/d4c/d59/d5d/d64/d6a/f6d x:0 0 0 2024-05-27T09:29:34.811 INFO:tasks.workunit.client.1.smithi138.stdout:4/322: mkdir d7/d6e 0 2024-05-27T09:29:34.814 INFO:tasks.workunit.client.1.smithi138.stdout:4/323: creat d7/f6f x:0 0 0 2024-05-27T09:29:34.818 INFO:tasks.workunit.client.1.smithi138.stdout:4/324: mknod d7/df/c70 0 2024-05-27T09:29:34.822 INFO:tasks.workunit.client.1.smithi138.stdout:4/325: write d7/df/f68 [635886,3770] 0 2024-05-27T09:29:34.824 INFO:tasks.workunit.client.1.smithi138.stdout:4/326: truncate d7/f42 110776 0 2024-05-27T09:29:34.824 INFO:tasks.workunit.client.1.smithi138.stdout:4/327: write d7/df/f4d [138993,18452] 0 2024-05-27T09:29:34.824 INFO:tasks.workunit.client.1.smithi138.stdout:4/328: stat d7/df/f19 0 2024-05-27T09:29:34.835 INFO:tasks.workunit.client.0.smithi119.stdout:4/860: dwrite d4/d23/d31/f33 [0,4194304] 0 2024-05-27T09:29:34.835 INFO:tasks.workunit.client.0.smithi119.stdout:4/861: stat d4/d23/d24/f82 0 2024-05-27T09:29:34.844 INFO:tasks.workunit.client.0.smithi119.stdout:0/798: dread d2/d6/d75/fb3 [0,4194304] 0 2024-05-27T09:29:34.845 INFO:tasks.workunit.client.0.smithi119.stdout:9/702: dwrite d18/d19/d1b/d29/fc9 [0,4194304] 0 2024-05-27T09:29:34.846 INFO:tasks.workunit.client.0.smithi119.stdout:4/862: symlink d4/d6/d3e/d89/l10e 0 2024-05-27T09:29:34.846 INFO:tasks.workunit.client.0.smithi119.stdout:4/863: chown d4/d23/d24/d5b/f105 1 1 2024-05-27T09:29:34.847 INFO:tasks.workunit.client.0.smithi119.stdout:0/799: mkdir d2/d6/d9/d12/d4b/d59/dd7/d105 0 2024-05-27T09:29:34.851 INFO:tasks.workunit.client.0.smithi119.stdout:1/856: dwrite d7/db/d52/fe7 [0,4194304] 0 2024-05-27T09:29:34.851 INFO:tasks.workunit.client.0.smithi119.stdout:9/703: creat d18/d19/d9b/fd4 x:0 0 0 2024-05-27T09:29:34.851 INFO:tasks.workunit.client.0.smithi119.stdout:8/764: dwrite d3/d6e/dcf/fa5 [0,4194304] 0 2024-05-27T09:29:34.853 INFO:tasks.workunit.client.0.smithi119.stdout:4/864: creat d4/d1c/f10f x:0 0 0 2024-05-27T09:29:34.854 INFO:tasks.workunit.client.0.smithi119.stdout:7/865: write d7/d1a/d99/fee [2696331,101165] 0 2024-05-27T09:29:34.854 INFO:tasks.workunit.client.0.smithi119.stdout:7/866: chown d7/df/d19/d4d/d89/f9b 30756 1 2024-05-27T09:29:34.864 INFO:tasks.workunit.client.0.smithi119.stdout:3/936: dwrite db/f12 [0,4194304] 0 2024-05-27T09:29:34.864 INFO:tasks.workunit.client.0.smithi119.stdout:6/774: dwrite d0/d2/d4e/feb [0,4194304] 0 2024-05-27T09:29:34.865 INFO:tasks.workunit.client.1.smithi138.stdout:1/312: dread d4/d3c/f4e [0,4194304] 0 2024-05-27T09:29:34.868 INFO:tasks.workunit.client.1.smithi138.stdout:8/269: dwrite f1 [0,4194304] 0 2024-05-27T09:29:34.870 INFO:tasks.workunit.client.0.smithi119.stdout:0/800: write d2/d6/d9/f66 [564754,120930] 0 2024-05-27T09:29:34.875 INFO:tasks.workunit.client.0.smithi119.stdout:1/857: creat d7/db/d12/ddb/d38/f107 x:0 0 0 2024-05-27T09:29:34.875 INFO:tasks.workunit.client.0.smithi119.stdout:8/765: rename d3/d8/d1b/f7a to d3/d8/d4d/d58/ffc 0 2024-05-27T09:29:34.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.874+0000 7faf55c00700 1 -- 172.21.15.119:0/1345970228 --> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faf5010dd30 con 0x7faf340727d0 2024-05-27T09:29:34.881 INFO:tasks.workunit.client.1.smithi138.stdout:7/268: sync 2024-05-27T09:29:34.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.885+0000 7faf47400700 1 -- 172.21.15.119:0/1345970228 <== mgr.14570 v2:172.21.15.138:6828/1384904888 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7faf5010dd30 con 0x7faf340727d0 2024-05-27T09:29:34.886 INFO:tasks.workunit.client.0.smithi119.stdout:4/865: dread d4/d23/f35 [0,4194304] 0 2024-05-27T09:29:34.886 INFO:tasks.workunit.client.0.smithi119.stdout:1/858: creat d7/db/d8f/f108 x:0 0 0 2024-05-27T09:29:34.886 INFO:tasks.workunit.client.0.smithi119.stdout:1/859: chown d7/dd/d44/f100 8630132 1 2024-05-27T09:29:34.887 INFO:tasks.workunit.client.0.smithi119.stdout:7/867: rename d7/df/d19/d97/d37/c4c to d7/df/d19/d4d/d89/c115 0 2024-05-27T09:29:34.888 INFO:tasks.workunit.client.1.smithi138.stdout:6/393: dwrite d3/d35/d43/d1c/f4d [0,4194304] 0 2024-05-27T09:29:34.890 INFO:tasks.workunit.client.0.smithi119.stdout:5/900: dwrite d6/dd/d39/d19/d70/dc1/f5c [0,4194304] 0 2024-05-27T09:29:34.891 INFO:tasks.workunit.client.1.smithi138.stdout:9/302: dwrite d6/da/d26/d28/f58 [0,4194304] 0 2024-05-27T09:29:34.891 INFO:tasks.workunit.client.1.smithi138.stdout:9/303: chown d6/da/l36 316 1 2024-05-27T09:29:34.894 INFO:tasks.workunit.client.1.smithi138.stdout:8/270: mknod d3/d8/d28/c4f 0 2024-05-27T09:29:34.894 INFO:tasks.workunit.client.1.smithi138.stdout:8/271: readlink d3/d8/l2a 0 2024-05-27T09:29:34.895 INFO:tasks.workunit.client.1.smithi138.stdout:7/269: creat d0/d7/f56 x:0 0 0 2024-05-27T09:29:34.895 INFO:tasks.workunit.client.0.smithi119.stdout:4/866: mkdir d4/de/d21/d40/d110 0 2024-05-27T09:29:34.895 INFO:tasks.workunit.client.0.smithi119.stdout:4/867: dread - d4/d6/d28/ff8 zero size 2024-05-27T09:29:34.898 INFO:tasks.workunit.client.1.smithi138.stdout:1/313: write d4/d48/d39/d5f/f2f [4031171,86086] 0 2024-05-27T09:29:34.899 INFO:tasks.workunit.client.1.smithi138.stdout:8/272: creat d3/f50 x:0 0 0 2024-05-27T09:29:34.899 INFO:tasks.workunit.client.1.smithi138.stdout:4/329: rmdir d7/df/d35 39 2024-05-27T09:29:34.899 INFO:tasks.workunit.client.1.smithi138.stdout:8/273: creat d3/d8/d11/f51 x:0 0 0 2024-05-27T09:29:34.900 INFO:tasks.workunit.client.0.smithi119.stdout:1/860: symlink d7/db/d12/ddb/d38/d82/l109 0 2024-05-27T09:29:34.901 INFO:tasks.workunit.client.1.smithi138.stdout:7/270: mkdir d0/d7/d57 0 2024-05-27T09:29:34.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.901+0000 7faf55c00700 1 -- 172.21.15.119:0/1345970228 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7faf340727d0 msgr2=0x7faf34074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:34.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.901+0000 7faf55c00700 1 --2- 172.21.15.119:0/1345970228 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7faf340727d0 0x7faf34074c80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00b5c0 tx=0x7faf3c005a90 comp rx=0 tx=0).stop 2024-05-27T09:29:34.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.901+0000 7faf55c00700 1 -- 172.21.15.119:0/1345970228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf50075700 msgr2=0x7faf5019bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:34.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.901+0000 7faf55c00700 1 --2- 172.21.15.119:0/1345970228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf50075700 0x7faf5019bf80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faf4000d8d0 tx=0x7faf4000dc90 comp rx=0 tx=0).stop 2024-05-27T09:29:34.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.901+0000 7faf55c00700 1 -- 172.21.15.119:0/1345970228 shutdown_connections 2024-05-27T09:29:34.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.901+0000 7faf55c00700 1 --2- 172.21.15.119:0/1345970228 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7faf340727d0 0x7faf34074c80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:34.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.901+0000 7faf55c00700 1 --2- 172.21.15.119:0/1345970228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf50075700 0x7faf5019bf80 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faf4000d8d0 tx=0x7faf4000dc90 comp rx=0 tx=0).stop 2024-05-27T09:29:34.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.901+0000 7faf55c00700 1 --2- 172.21.15.119:0/1345970228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf50076950 0x7faf5019c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:34.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.901+0000 7faf55c00700 1 -- 172.21.15.119:0/1345970228 >> 172.21.15.119:0/1345970228 conn(0x7faf500fd960 msgr2=0x7faf5010c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:34.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.901+0000 7faf55c00700 1 -- 172.21.15.119:0/1345970228 shutdown_connections 2024-05-27T09:29:34.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:34.902+0000 7faf55c00700 1 -- 172.21.15.119:0/1345970228 wait complete. 2024-05-27T09:29:34.905 INFO:tasks.workunit.client.1.smithi138.stdout:8/274: rename d3/d8/d28/f4b to d3/f52 0 2024-05-27T09:29:34.905 INFO:tasks.workunit.client.1.smithi138.stdout:8/275: readlink d3/d8/d11/l35 0 2024-05-27T09:29:34.905 INFO:tasks.workunit.client.1.smithi138.stdout:8/276: chown d3/d8/c36 4 1 2024-05-27T09:29:34.908 INFO:tasks.workunit.client.1.smithi138.stdout:9/304: dread d6/db/f10 [0,4194304] 0 2024-05-27T09:29:34.908 INFO:tasks.workunit.client.1.smithi138.stdout:8/277: symlink d3/d8/d32/d45/l53 0 2024-05-27T09:29:34.910 INFO:tasks.workunit.client.0.smithi119.stdout:5/901: rmdir d6/d2f 39 2024-05-27T09:29:34.912 INFO:tasks.workunit.client.1.smithi138.stdout:4/330: truncate d7/df/f19 5387797 0 2024-05-27T09:29:34.912 INFO:tasks.workunit.client.0.smithi119.stdout:4/868: creat d4/d23/d24/d5b/da8/dda/dea/f111 x:0 0 0 2024-05-27T09:29:34.912 INFO:tasks.workunit.client.0.smithi119.stdout:3/937: write db/dc9/d98/d7a/f84 [611719,23776] 0 2024-05-27T09:29:34.914 INFO:tasks.workunit.client.1.smithi138.stdout:9/305: creat d6/da/d2f/f5f x:0 0 0 2024-05-27T09:29:34.915 INFO:tasks.workunit.client.1.smithi138.stdout:8/278: creat d3/d8/d24/f54 x:0 0 0 2024-05-27T09:29:34.917 INFO:tasks.workunit.client.1.smithi138.stdout:9/306: symlink d6/da/d29/l60 0 2024-05-27T09:29:34.917 INFO:tasks.workunit.client.0.smithi119.stdout:1/861: link d7/db/d12/d26/d4a/l70 d7/db/d8f/d9b/dc9/l10a 0 2024-05-27T09:29:34.918 INFO:tasks.workunit.client.1.smithi138.stdout:4/331: symlink d7/df/d5b/l71 0 2024-05-27T09:29:34.919 INFO:tasks.workunit.client.1.smithi138.stdout:4/332: dread - d7/df/f2e zero size 2024-05-27T09:29:34.922 INFO:tasks.workunit.client.0.smithi119.stdout:4/869: creat d4/d6/d81/dce/d88/f112 x:0 0 0 2024-05-27T09:29:34.922 INFO:tasks.workunit.client.0.smithi119.stdout:5/902: dread d6/db/dc/f3b [0,4194304] 0 2024-05-27T09:29:34.923 INFO:tasks.workunit.client.1.smithi138.stdout:8/279: unlink d3/l3d 0 2024-05-27T09:29:34.923 INFO:tasks.workunit.client.0.smithi119.stdout:4/870: chown d4/f8f 3535732 1 2024-05-27T09:29:34.924 INFO:tasks.workunit.client.1.smithi138.stdout:4/333: mknod d7/df/d35/d4c/c72 0 2024-05-27T09:29:34.924 INFO:tasks.workunit.client.1.smithi138.stdout:4/334: truncate d7/d27/d3a/f55 970746 0 2024-05-27T09:29:34.926 INFO:tasks.workunit.client.0.smithi119.stdout:5/903: mknod d6/db/de/d27/dc9/d12c/c140 0 2024-05-27T09:29:34.928 INFO:tasks.workunit.client.1.smithi138.stdout:9/307: mkdir d6/da/d29/d61 0 2024-05-27T09:29:34.929 INFO:tasks.workunit.client.1.smithi138.stdout:4/335: link d7/df/f1a d7/df/d35/d4c/d59/f73 0 2024-05-27T09:29:34.929 INFO:tasks.workunit.client.1.smithi138.stdout:4/336: chown d7/df/l43 26942758 1 2024-05-27T09:29:34.932 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:29:34.934 INFO:tasks.workunit.client.1.smithi138.stdout:8/280: mkdir d3/d8/d24/d55 0 2024-05-27T09:29:34.934 INFO:tasks.workunit.client.1.smithi138.stdout:4/337: link d7/d27/d3a/l3e d7/d6e/l74 0 2024-05-27T09:29:34.934 INFO:tasks.workunit.client.1.smithi138.stdout:4/338: dread - d7/df/f41 zero size 2024-05-27T09:29:34.934 INFO:tasks.workunit.client.1.smithi138.stdout:4/339: truncate d7/df/d35/d4c/f69 326262 0 2024-05-27T09:29:34.934 INFO:tasks.workunit.client.1.smithi138.stdout:6/394: dread d3/d19/f1e [0,4194304] 0 2024-05-27T09:29:34.936 INFO:tasks.workunit.client.1.smithi138.stdout:9/308: getdents d6/db 0 2024-05-27T09:29:34.939 INFO:tasks.workunit.client.1.smithi138.stdout:6/395: mknod d3/d11/d2d/d37/d8b/c8e 0 2024-05-27T09:29:34.939 INFO:tasks.workunit.client.0.smithi119.stdout:5/904: write d6/dd/d46/d47/d4b/dd8/d124/fb1 [1245656,116975] 0 2024-05-27T09:29:34.939 INFO:tasks.workunit.client.0.smithi119.stdout:5/905: write d6/dd/d39/d19/d70/dc1/d41/fda [155413,102235] 0 2024-05-27T09:29:34.942 INFO:tasks.workunit.client.1.smithi138.stdout:8/281: symlink d3/d8/d32/d45/l56 0 2024-05-27T09:29:34.942 INFO:tasks.workunit.client.1.smithi138.stdout:8/282: fdatasync d3/d8/d28/f2e 0 2024-05-27T09:29:34.943 INFO:tasks.workunit.client.1.smithi138.stdout:9/309: mknod d6/db/de/c62 0 2024-05-27T09:29:34.946 INFO:tasks.workunit.client.0.smithi119.stdout:4/871: rmdir d4/d6/d28/d2f/d36 39 2024-05-27T09:29:34.946 INFO:tasks.workunit.client.0.smithi119.stdout:7/868: dread d7/df/d19/d9a/da6/dda/d44/ffd [0,4194304] 0 2024-05-27T09:29:34.947 INFO:tasks.workunit.client.1.smithi138.stdout:8/283: rename d3/d8/d32/d45/l53 to d3/d8/d24/d55/l57 0 2024-05-27T09:29:34.953 INFO:tasks.workunit.client.1.smithi138.stdout:4/340: dread d7/f44 [0,4194304] 0 2024-05-27T09:29:34.954 INFO:tasks.workunit.client.1.smithi138.stdout:4/341: rmdir d7/df/d35 39 2024-05-27T09:29:34.955 INFO:tasks.workunit.client.1.smithi138.stdout:4/342: mknod d7/df/d35/d4c/c75 0 2024-05-27T09:29:34.955 INFO:tasks.workunit.client.0.smithi119.stdout:1/862: write d7/db/d8f/d9b/f104 [3783115,69047] 0 2024-05-27T09:29:34.956 INFO:tasks.workunit.client.0.smithi119.stdout:1/863: stat d7/db/d12/d26/d39/lf7 0 2024-05-27T09:29:34.957 INFO:tasks.workunit.client.1.smithi138.stdout:4/343: creat d7/df/d35/d4c/d59/d5d/d64/d6a/f76 x:0 0 0 2024-05-27T09:29:34.957 INFO:tasks.workunit.client.1.smithi138.stdout:4/344: truncate d7/f6f 266952 0 2024-05-27T09:29:34.957 INFO:tasks.workunit.client.1.smithi138.stdout:4/345: readlink d7/df/d35/d4c/l57 0 2024-05-27T09:29:34.957 INFO:tasks.workunit.client.1.smithi138.stdout:4/346: fsync d7/d27/f2c 0 2024-05-27T09:29:34.957 INFO:tasks.workunit.client.1.smithi138.stdout:4/347: fsync d7/df/f29 0 2024-05-27T09:29:34.958 INFO:tasks.workunit.client.0.smithi119.stdout:1/864: mknod d7/db/d12/d15/c10b 0 2024-05-27T09:29:34.959 INFO:tasks.workunit.client.0.smithi119.stdout:3/938: dread db/fbf [4194304,4194304] 0 2024-05-27T09:29:34.997 INFO:tasks.workunit.client.1.smithi138.stdout:6/396: rmdir d3/d11 39 2024-05-27T09:29:34.997 INFO:tasks.workunit.client.1.smithi138.stdout:6/397: rmdir d3/d11 39 2024-05-27T09:29:35.001 INFO:tasks.workunit.client.1.smithi138.stdout:6/398: symlink d3/l8f 0 2024-05-27T09:29:35.004 INFO:tasks.workunit.client.1.smithi138.stdout:6/399: mknod d3/d35/d74/c90 0 2024-05-27T09:29:35.004 INFO:tasks.workunit.client.1.smithi138.stdout:6/400: write d3/d11/d2d/d37/f85 [288373,86462] 0 2024-05-27T09:29:35.004 INFO:tasks.workunit.client.1.smithi138.stdout:6/401: chown d3/d35/d43/d1c/l2b 2 1 2024-05-27T09:29:35.014 INFO:tasks.workunit.client.1.smithi138.stdout:6/402: mkdir d3/d91 0 2024-05-27T09:29:35.014 INFO:tasks.workunit.client.0.smithi119.stdout:6/775: dwrite d0/d2/d1c/d21/ff9 [0,4194304] 0 2024-05-27T09:29:35.015 INFO:tasks.workunit.client.1.smithi138.stdout:6/403: write d3/d35/f60 [597715,23790] 0 2024-05-27T09:29:35.028 INFO:tasks.workunit.client.0.smithi119.stdout:5/906: dread d6/db/de/f5b [0,4194304] 0 2024-05-27T09:29:35.036 INFO:tasks.workunit.client.0.smithi119.stdout:6/776: read d0/d2/d57/f90 [749551,26075] 0 2024-05-27T09:29:35.039 INFO:tasks.workunit.client.0.smithi119.stdout:6/777: creat d0/d2/d1c/d21/d58/f114 x:0 0 0 2024-05-27T09:29:35.041 INFO:tasks.workunit.client.0.smithi119.stdout:6/778: mkdir d0/d2/d57/d37/d115 0 2024-05-27T09:29:35.047 INFO:tasks.workunit.client.0.smithi119.stdout:6/779: mknod d0/d2/d4e/d5a/d76/c116 0 2024-05-27T09:29:35.047 INFO:tasks.workunit.client.0.smithi119.stdout:6/780: dread - d0/f9a zero size 2024-05-27T09:29:35.047 INFO:tasks.workunit.client.0.smithi119.stdout:6/781: read - d0/d2/d1c/d21/d58/dd3/f110 zero size 2024-05-27T09:29:35.056 INFO:tasks.workunit.client.0.smithi119.stdout:9/704: dwrite d18/d19/d1b/d40/d44/f70 [4194304,4194304] 0 2024-05-27T09:29:35.077 INFO:tasks.workunit.client.0.smithi119.stdout:1/865: dwrite d7/db/fe1 [0,4194304] 0 2024-05-27T09:29:35.079 INFO:tasks.workunit.client.0.smithi119.stdout:1/866: dread - d7/db/d35/fd4 zero size 2024-05-27T09:29:35.082 INFO:tasks.workunit.client.0.smithi119.stdout:1/867: creat d7/db/d8f/d9b/dce/f10c x:0 0 0 2024-05-27T09:29:35.082 INFO:tasks.workunit.client.0.smithi119.stdout:1/868: stat d7/db/d12/d15/cb2 0 2024-05-27T09:29:35.082 INFO:tasks.workunit.client.0.smithi119.stdout:1/869: chown f6 25870112 1 2024-05-27T09:29:35.084 INFO:tasks.workunit.client.1.smithi138.stdout:6/404: dread d3/f17 [0,4194304] 0 2024-05-27T09:29:35.084 INFO:tasks.workunit.client.1.smithi138.stdout:7/271: dwrite d0/d7/f56 [0,4194304] 0 2024-05-27T09:29:35.084 INFO:tasks.workunit.client.1.smithi138.stdout:8/284: dwrite d3/d8/d32/f49 [0,4194304] 0 2024-05-27T09:29:35.084 INFO:tasks.workunit.client.1.smithi138.stdout:4/348: dwrite d7/df/d35/d4c/f69 [0,4194304] 0 2024-05-27T09:29:35.086 INFO:tasks.workunit.client.1.smithi138.stdout:4/349: creat d7/df/d35/f77 x:0 0 0 2024-05-27T09:29:35.087 INFO:tasks.workunit.client.0.smithi119.stdout:1/870: write d7/dd/d25/d60/d61/fb3 [713105,104920] 0 2024-05-27T09:29:35.087 INFO:tasks.workunit.client.0.smithi119.stdout:1/871: read d7/db/d12/d26/d4a/f81 [655848,66431] 0 2024-05-27T09:29:35.089 INFO:tasks.workunit.client.1.smithi138.stdout:9/310: dwrite d6/f48 [4194304,4194304] 0 2024-05-27T09:29:35.090 INFO:tasks.workunit.client.0.smithi119.stdout:7/869: dwrite d7/df/d19/d97/d92/fab [0,4194304] 0 2024-05-27T09:29:35.091 INFO:tasks.workunit.client.1.smithi138.stdout:4/350: write d7/df/f29 [395855,25323] 0 2024-05-27T09:29:35.092 INFO:tasks.workunit.client.0.smithi119.stdout:1/872: symlink d7/d40/l10d 0 2024-05-27T09:29:35.094 INFO:tasks.workunit.client.1.smithi138.stdout:7/272: creat d0/d23/f58 x:0 0 0 2024-05-27T09:29:35.097 INFO:tasks.workunit.client.1.smithi138.stdout:4/351: creat d7/f78 x:0 0 0 2024-05-27T09:29:35.097 INFO:tasks.workunit.client.1.smithi138.stdout:4/352: chown d7/df/d35/d4c/d59/d5d/d64/d6a 7 1 2024-05-27T09:29:35.098 INFO:tasks.workunit.client.0.smithi119.stdout:7/870: truncate d7/df/d19/d4d/f71 774420 0 2024-05-27T09:29:35.098 INFO:tasks.workunit.client.0.smithi119.stdout:1/873: mknod d7/db/d12/ddb/d38/d3c/c10e 0 2024-05-27T09:29:35.099 INFO:tasks.workunit.client.1.smithi138.stdout:4/353: symlink d7/df/d35/d4c/l79 0 2024-05-27T09:29:35.099 INFO:tasks.workunit.client.0.smithi119.stdout:0/801: dwrite d2/d6/d23/fa9 [0,4194304] 0 2024-05-27T09:29:35.102 INFO:tasks.workunit.client.1.smithi138.stdout:8/285: rmdir d3/d8/d32/d45 39 2024-05-27T09:29:35.102 INFO:tasks.workunit.client.1.smithi138.stdout:8/286: chown d3/d8/d28/l3c 0 1 2024-05-27T09:29:35.107 INFO:tasks.workunit.client.1.smithi138.stdout:1/314: dwrite d4/d1a/f27 [0,4194304] 0 2024-05-27T09:29:35.107 INFO:tasks.workunit.client.1.smithi138.stdout:1/315: readlink d4/d48/l3a 0 2024-05-27T09:29:35.112 INFO:tasks.workunit.client.0.smithi119.stdout:5/907: dwrite d6/dd/fff [0,4194304] 0 2024-05-27T09:29:35.112 INFO:tasks.workunit.client.0.smithi119.stdout:5/908: dread - d6/dd/d46/fe7 zero size 2024-05-27T09:29:35.112 INFO:tasks.workunit.client.0.smithi119.stdout:4/872: dwrite d4/d6/f7d [0,4194304] 0 2024-05-27T09:29:35.113 INFO:tasks.workunit.client.0.smithi119.stdout:4/873: chown d4/d6/d81/dce/lac 245597393 1 2024-05-27T09:29:35.113 INFO:tasks.workunit.client.0.smithi119.stdout:8/766: dwrite d3/de/dad/ffa [0,4194304] 0 2024-05-27T09:29:35.113 INFO:tasks.workunit.client.0.smithi119.stdout:8/767: chown d3/d8/d1b/d46/d92 319 1 2024-05-27T09:29:35.113 INFO:tasks.workunit.client.0.smithi119.stdout:8/768: write d3/fe5 [777871,28102] 0 2024-05-27T09:29:35.119 INFO:tasks.workunit.client.1.smithi138.stdout:7/273: dread d0/f36 [0,4194304] 0 2024-05-27T09:29:35.124 INFO:tasks.workunit.client.1.smithi138.stdout:9/311: dread d6/f2e [0,4194304] 0 2024-05-27T09:29:35.125 INFO:tasks.workunit.client.1.smithi138.stdout:9/312: write d6/db/dc/f1c [318320,67645] 0 2024-05-27T09:29:35.128 INFO:tasks.workunit.client.1.smithi138.stdout:9/313: symlink d6/db/l63 0 2024-05-27T09:29:35.131 INFO:tasks.workunit.client.0.smithi119.stdout:0/802: symlink d2/d6/d9/d12/da3/l106 0 2024-05-27T09:29:35.131 INFO:tasks.workunit.client.0.smithi119.stdout:0/803: write d2/d6/d9/d24/d30/dcc/ff4 [624476,127920] 0 2024-05-27T09:29:35.138 INFO:tasks.workunit.client.1.smithi138.stdout:1/316: dread d4/d1a/d2e/f62 [4194304,4194304] 0 2024-05-27T09:29:35.138 INFO:tasks.workunit.client.1.smithi138.stdout:1/317: read d4/f24 [1422962,55587] 0 2024-05-27T09:29:35.140 INFO:tasks.workunit.client.0.smithi119.stdout:7/871: getdents d7/df/d19 0 2024-05-27T09:29:35.140 INFO:tasks.workunit.client.0.smithi119.stdout:7/872: creat d7/df/d19/d9a/da6/f116 x:0 0 0 2024-05-27T09:29:35.140 INFO:tasks.workunit.client.0.smithi119.stdout:7/873: chown d7/df/d19/d97/d92/db7/dba/d42/d76/lce 13854 1 2024-05-27T09:29:35.144 INFO:tasks.workunit.client.0.smithi119.stdout:8/769: mkdir d3/d6e/dcf/dfd 0 2024-05-27T09:29:35.151 INFO:tasks.workunit.client.1.smithi138.stdout:7/274: dread d0/f21 [0,4194304] 0 2024-05-27T09:29:35.156 INFO:tasks.workunit.client.1.smithi138.stdout:7/275: mkdir d0/d23/d59 0 2024-05-27T09:29:35.158 INFO:tasks.workunit.client.1.smithi138.stdout:7/276: getdents d0/d23/d49 0 2024-05-27T09:29:35.161 INFO:tasks.workunit.client.1.smithi138.stdout:1/318: write d4/d3c/f42 [2236489,74803] 0 2024-05-27T09:29:35.163 INFO:tasks.workunit.client.1.smithi138.stdout:8/287: dread d3/d8/d32/f49 [0,4194304] 0 2024-05-27T09:29:35.165 INFO:tasks.workunit.client.1.smithi138.stdout:1/319: unlink d4/d48/l3b 0 2024-05-27T09:29:35.165 INFO:tasks.workunit.client.0.smithi119.stdout:3/939: dwrite db/d36/d45/d4c/d62/d13/f97 [0,4194304] 0 2024-05-27T09:29:35.167 INFO:tasks.workunit.client.1.smithi138.stdout:1/320: mknod d4/d48/c64 0 2024-05-27T09:29:35.170 INFO:tasks.workunit.client.1.smithi138.stdout:8/288: rename d3/d8/fc to d3/d8/f58 0 2024-05-27T09:29:35.170 INFO:tasks.workunit.client.0.smithi119.stdout:7/874: mkdir d7/df/d19/d97/d92/db7/dba/d42/d117 0 2024-05-27T09:29:35.170 INFO:tasks.workunit.client.0.smithi119.stdout:7/875: write d7/df/d19/d4d/d89/fd6 [356902,89241] 0 2024-05-27T09:29:35.171 INFO:tasks.workunit.client.0.smithi119.stdout:7/876: stat d7/d1a/d99/dbb 0 2024-05-27T09:29:35.171 INFO:tasks.workunit.client.0.smithi119.stdout:8/770: creat d3/d39/ffe x:0 0 0 2024-05-27T09:29:35.171 INFO:tasks.workunit.client.0.smithi119.stdout:8/771: write d3/d8/d4d/d58/f63 [953162,113571] 0 2024-05-27T09:29:35.172 INFO:tasks.workunit.client.1.smithi138.stdout:1/321: mkdir d4/d48/d22/d65 0 2024-05-27T09:29:35.172 INFO:tasks.workunit.client.1.smithi138.stdout:1/322: readlink d4/d38/l5c 0 2024-05-27T09:29:35.175 INFO:tasks.workunit.client.0.smithi119.stdout:9/705: dwrite d18/d19/d1b/f1e [0,4194304] 0 2024-05-27T09:29:35.175 INFO:tasks.workunit.client.0.smithi119.stdout:9/706: fdatasync d18/d19/d1b/d29/fac 0 2024-05-27T09:29:35.179 INFO:tasks.workunit.client.1.smithi138.stdout:1/323: symlink d4/d48/d39/d5f/l66 0 2024-05-27T09:29:35.185 INFO:tasks.workunit.client.1.smithi138.stdout:7/277: write d0/d7/f12 [1580441,27307] 0 2024-05-27T09:29:35.186 INFO:tasks.workunit.client.1.smithi138.stdout:7/278: truncate d0/d23/d49/f4a 4831871 0 2024-05-27T09:29:35.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.185+0000 7f35bd400700 1 -- 172.21.15.119:0/230756470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35b81035a0 msgr2=0x7f35b8105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:35.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.185+0000 7f35bd400700 1 --2- 172.21.15.119:0/230756470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35b81035a0 0x7f35b8105980 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009910 tx=0x7f35a4009c20 comp rx=0 tx=0).stop 2024-05-27T09:29:35.187 INFO:tasks.workunit.client.0.smithi119.stdout:4/874: dread d4/d6/d28/f6b [0,4194304] 0 2024-05-27T09:29:35.188 INFO:tasks.workunit.client.1.smithi138.stdout:7/279: chown d0/d2b/c44 206 1 2024-05-27T09:29:35.188 INFO:tasks.workunit.client.1.smithi138.stdout:7/280: chown d0/d2b/c44 28 1 2024-05-27T09:29:35.188 INFO:tasks.workunit.client.1.smithi138.stdout:7/281: fdatasync d0/d7/f39 0 2024-05-27T09:29:35.188 INFO:tasks.workunit.client.1.smithi138.stdout:1/324: rmdir d4/d48/d39/d5f/d4d 39 2024-05-27T09:29:35.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.187+0000 7f35bd400700 1 -- 172.21.15.119:0/230756470 shutdown_connections 2024-05-27T09:29:35.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.187+0000 7f35bd400700 1 --2- 172.21.15.119:0/230756470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35b81035a0 0x7f35b8105980 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:35.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.187+0000 7f35bd400700 1 --2- 172.21.15.119:0/230756470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b8100c80 0x7f35b8103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:35.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.187+0000 7f35bd400700 1 -- 172.21.15.119:0/230756470 >> 172.21.15.119:0/230756470 conn(0x7f35b80fa5e0 msgr2=0x7f35b80fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:35.189 INFO:tasks.workunit.client.0.smithi119.stdout:2/835: sync 2024-05-27T09:29:35.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.188+0000 7f35bd400700 1 -- 172.21.15.119:0/230756470 shutdown_connections 2024-05-27T09:29:35.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.189+0000 7f35bd400700 1 -- 172.21.15.119:0/230756470 wait complete. 2024-05-27T09:29:35.189 INFO:tasks.workunit.client.0.smithi119.stdout:8/772: rename d3/d6e/d80/d84/c8d to d3/d8/d4d/d5d/d66/d85/cff 0 2024-05-27T09:29:35.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.191+0000 7f35bd400700 1 Processor -- start 2024-05-27T09:29:35.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.191+0000 7f35bd400700 1 -- start start 2024-05-27T09:29:35.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.192+0000 7f35bd400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35b8100c80 0x7f35b81937c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:35.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.192+0000 7f35bd400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b81035a0 0x7f35b8193d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:35.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.192+0000 7f35bd400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b8194320 con 0x7f35b81035a0 2024-05-27T09:29:35.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.192+0000 7f35bd400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b8194460 con 0x7f35b8100c80 2024-05-27T09:29:35.193 INFO:tasks.workunit.client.0.smithi119.stdout:9/707: unlink d18/d19/d1b/d2a/d63/f9c 0 2024-05-27T09:29:35.194 INFO:tasks.workunit.client.0.smithi119.stdout:9/708: creat d18/d19/d9b/fd5 x:0 0 0 2024-05-27T09:29:35.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.193+0000 7f35b5e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b81035a0 0x7f35b8193d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:35.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.194+0000 7f35b5e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b81035a0 0x7f35b8193d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33842/0 (socket says 172.21.15.119:33842) 2024-05-27T09:29:35.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.194+0000 7f35b5e00700 1 -- 172.21.15.119:0/182346947 learned_addr learned my addr 172.21.15.119:0/182346947 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:29:35.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.194+0000 7f35b5e00700 1 -- 172.21.15.119:0/182346947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35b8100c80 msgr2=0x7f35b81937c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:35.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.194+0000 7f35b5e00700 1 --2- 172.21.15.119:0/182346947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35b8100c80 0x7f35b81937c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:35.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.194+0000 7f35b5e00700 1 -- 172.21.15.119:0/182346947 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a40095f0 con 0x7f35b81035a0 2024-05-27T09:29:35.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.195+0000 7f35b5e00700 1 --2- 172.21.15.119:0/182346947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b81035a0 0x7f35b8193d00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f35a4005b40 tx=0x7f35a400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:35.196 INFO:tasks.workunit.client.0.smithi119.stdout:4/875: creat d4/d6/d81/d10b/f113 x:0 0 0 2024-05-27T09:29:35.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.196+0000 7f35af400700 1 -- 172.21.15.119:0/182346947 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35a401d070 con 0x7f35b81035a0 2024-05-27T09:29:35.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.196+0000 7f35af400700 1 -- 172.21.15.119:0/182346947 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f35a4022470 con 0x7f35b81035a0 2024-05-27T09:29:35.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.196+0000 7f35af400700 1 -- 172.21.15.119:0/182346947 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35a400f690 con 0x7f35b81035a0 2024-05-27T09:29:35.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.197+0000 7f35bd400700 1 -- 172.21.15.119:0/182346947 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b8198eb0 con 0x7f35b81035a0 2024-05-27T09:29:35.198 INFO:tasks.workunit.client.1.smithi138.stdout:8/289: dread d3/d8/d28/f2e [0,4194304] 0 2024-05-27T09:29:35.198 INFO:tasks.workunit.client.1.smithi138.stdout:8/290: creat d3/d8/d32/f59 x:0 0 0 2024-05-27T09:29:35.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.197+0000 7f35bd400700 1 -- 172.21.15.119:0/182346947 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b81993f0 con 0x7f35b81035a0 2024-05-27T09:29:35.198 INFO:tasks.workunit.client.1.smithi138.stdout:6/405: dwrite d3/d11/f88 [0,4194304] 0 2024-05-27T09:29:35.199 INFO:tasks.workunit.client.1.smithi138.stdout:2/240: sync 2024-05-27T09:29:35.199 INFO:tasks.workunit.client.1.smithi138.stdout:0/300: sync 2024-05-27T09:29:35.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.199+0000 7f35bd400700 1 -- 172.21.15.119:0/182346947 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b818da00 con 0x7f35b81035a0 2024-05-27T09:29:35.201 INFO:tasks.workunit.client.1.smithi138.stdout:1/325: write d4/d1a/d2e/f41 [604570,60124] 0 2024-05-27T09:29:35.201 INFO:tasks.workunit.client.1.smithi138.stdout:5/258: sync 2024-05-27T09:29:35.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.201+0000 7f35af400700 1 -- 172.21.15.119:0/182346947 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) v1 ==== 95024+0+0 (secure 0 0 0) 0x7f35a40225e0 con 0x7f35b81035a0 2024-05-27T09:29:35.202 INFO:tasks.workunit.client.0.smithi119.stdout:2/836: rename d2/deb/l102 to d2/da/d15/l11c 0 2024-05-27T09:29:35.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.202+0000 7f35af400700 1 --2- 172.21.15.119:0/182346947 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f359c076be0 0x7f359c079090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:35.203 INFO:tasks.workunit.client.1.smithi138.stdout:8/291: creat d3/d8/d24/d4d/f5a x:0 0 0 2024-05-27T09:29:35.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.203+0000 7f35af400700 1 -- 172.21.15.119:0/182346947 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f35a40949a0 con 0x7f35b81035a0 2024-05-27T09:29:35.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.203+0000 7f35b6800700 1 --2- 172.21.15.119:0/182346947 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f359c076be0 0x7f359c079090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:35.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.204+0000 7f35b6800700 1 --2- 172.21.15.119:0/182346947 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f359c076be0 0x7f359c079090 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f35a80097b0 tx=0x7f35a8006d20 comp rx=0 tx=0).ready entity=mgr.14570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:35.206 INFO:tasks.workunit.client.0.smithi119.stdout:3/940: dread db/d36/d45/d93/fc6 [0,4194304] 0 2024-05-27T09:29:35.206 INFO:tasks.workunit.client.0.smithi119.stdout:9/709: symlink d18/d19/d1b/d40/d7b/dc6/ld6 0 2024-05-27T09:29:35.206 INFO:tasks.workunit.client.0.smithi119.stdout:3/941: creat db/d36/d45/d4c/dbe/f133 x:0 0 0 2024-05-27T09:29:35.206 INFO:tasks.workunit.client.0.smithi119.stdout:9/710: creat d18/d19/d61/d98/fd7 x:0 0 0 2024-05-27T09:29:35.210 INFO:tasks.workunit.client.0.smithi119.stdout:4/876: getdents d4/de/de8 0 2024-05-27T09:29:35.210 INFO:tasks.workunit.client.1.smithi138.stdout:9/314: dwrite d6/da/d26/d28/f49 [0,4194304] 0 2024-05-27T09:29:35.210 INFO:tasks.workunit.client.1.smithi138.stdout:9/315: truncate d6/db/dc/f12 952625 0 2024-05-27T09:29:35.210 INFO:tasks.workunit.client.1.smithi138.stdout:9/316: chown d6/db/de/d18 15856893 1 2024-05-27T09:29:35.210 INFO:tasks.workunit.client.1.smithi138.stdout:8/292: rmdir d3/d8/d24/d55 39 2024-05-27T09:29:35.211 INFO:tasks.workunit.client.1.smithi138.stdout:6/406: unlink d3/d35/d43/f2a 0 2024-05-27T09:29:35.213 INFO:tasks.workunit.client.1.smithi138.stdout:1/326: creat d4/d48/d39/d5f/d4d/d2b/f67 x:0 0 0 2024-05-27T09:29:35.217 INFO:tasks.workunit.client.1.smithi138.stdout:6/407: getdents d3/d35/d43/d1c/d32/d47 0 2024-05-27T09:29:35.218 INFO:tasks.workunit.client.1.smithi138.stdout:5/259: truncate d3/d12/d16/f2e 3251519 0 2024-05-27T09:29:35.220 INFO:tasks.workunit.client.0.smithi119.stdout:8/773: read d3/de/f62 [1349112,1102] 0 2024-05-27T09:29:35.220 INFO:tasks.workunit.client.1.smithi138.stdout:6/408: creat d3/d35/d7d/f92 x:0 0 0 2024-05-27T09:29:35.220 INFO:tasks.workunit.client.1.smithi138.stdout:6/409: dread - d3/d35/d43/d1c/d32/d47/f51 zero size 2024-05-27T09:29:35.221 INFO:tasks.workunit.client.1.smithi138.stdout:5/260: symlink d3/d12/d16/d1d/d23/l56 0 2024-05-27T09:29:35.225 INFO:tasks.workunit.client.0.smithi119.stdout:8/774: read d3/fe5 [259051,21838] 0 2024-05-27T09:29:35.226 INFO:tasks.workunit.client.1.smithi138.stdout:6/410: link d3/c13 d3/d11/c93 0 2024-05-27T09:29:35.226 INFO:tasks.workunit.client.1.smithi138.stdout:1/327: dread d4/f37 [0,4194304] 0 2024-05-27T09:29:35.227 INFO:tasks.workunit.client.0.smithi119.stdout:9/711: creat d18/d19/d1b/d2a/d9f/fd8 x:0 0 0 2024-05-27T09:29:35.227 INFO:tasks.workunit.client.0.smithi119.stdout:4/877: creat d4/de/de8/f114 x:0 0 0 2024-05-27T09:29:35.229 INFO:tasks.workunit.client.0.smithi119.stdout:3/942: creat db/dc9/d98/d105/f134 x:0 0 0 2024-05-27T09:29:35.231 INFO:tasks.workunit.client.0.smithi119.stdout:1/874: dwrite d7/d6e/fe2 [0,4194304] 0 2024-05-27T09:29:35.233 INFO:tasks.workunit.client.1.smithi138.stdout:4/354: dwrite d7/df/d35/d4c/f66 [0,4194304] 0 2024-05-27T09:29:35.233 INFO:tasks.workunit.client.1.smithi138.stdout:1/328: creat d4/d1a/d2e/d32/f68 x:0 0 0 2024-05-27T09:29:35.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.232+0000 7f35af400700 1 -- 172.21.15.119:0/182346947 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f35a405caf0 con 0x7f35b81035a0 2024-05-27T09:29:35.234 INFO:tasks.workunit.client.0.smithi119.stdout:8/775: symlink d3/d6e/d80/d84/l100 0 2024-05-27T09:29:35.235 INFO:tasks.workunit.client.0.smithi119.stdout:4/878: symlink d4/db1/l115 0 2024-05-27T09:29:35.236 INFO:tasks.workunit.client.1.smithi138.stdout:2/241: rmdir d1/d15/d2d 39 2024-05-27T09:29:35.240 INFO:tasks.workunit.client.0.smithi119.stdout:3/943: symlink db/d36/d45/d4c/d62/d13/d33/d9b/l135 0 2024-05-27T09:29:35.247 INFO:tasks.workunit.client.1.smithi138.stdout:8/293: mknod d3/d8/d28/c5b 0 2024-05-27T09:29:35.247 INFO:tasks.workunit.client.1.smithi138.stdout:4/355: symlink d7/df/d35/d4c/d59/d5d/d64/l7a 0 2024-05-27T09:29:35.247 INFO:tasks.workunit.client.1.smithi138.stdout:4/356: creat d7/df/f7b x:0 0 0 2024-05-27T09:29:35.248 INFO:tasks.workunit.client.1.smithi138.stdout:4/357: stat d7/df/f19 0 2024-05-27T09:29:35.250 INFO:tasks.workunit.client.1.smithi138.stdout:2/242: creat d1/d15/d2d/d31/d4b/f51 x:0 0 0 2024-05-27T09:29:35.252 INFO:tasks.workunit.client.1.smithi138.stdout:8/294: unlink d3/d8/l2a 0 2024-05-27T09:29:35.253 INFO:tasks.workunit.client.1.smithi138.stdout:2/243: mkdir d1/d10/d52 0 2024-05-27T09:29:35.253 INFO:tasks.workunit.client.0.smithi119.stdout:1/875: mknod d7/db/d12/ddb/d38/d3c/dfd/c10f 0 2024-05-27T09:29:35.253 INFO:tasks.workunit.client.0.smithi119.stdout:1/876: write d7/dd/dcb/ff5 [998111,91805] 0 2024-05-27T09:29:35.253 INFO:tasks.workunit.client.0.smithi119.stdout:1/877: chown d7/dd/db0 240395708 1 2024-05-27T09:29:35.254 INFO:tasks.workunit.client.1.smithi138.stdout:8/295: unlink d3/d8/f14 0 2024-05-27T09:29:35.255 INFO:tasks.workunit.client.1.smithi138.stdout:8/296: chown d3/f29 2027 1 2024-05-27T09:29:35.256 INFO:tasks.workunit.client.1.smithi138.stdout:2/244: creat d1/d15/d2d/f53 x:0 0 0 2024-05-27T09:29:35.257 INFO:tasks.workunit.client.1.smithi138.stdout:2/245: dread - d1/d15/d2d/d2e/f50 zero size 2024-05-27T09:29:35.257 INFO:tasks.workunit.client.1.smithi138.stdout:2/246: chown d1/d15/d2d/d31/f3b 4892 1 2024-05-27T09:29:35.257 INFO:tasks.workunit.client.1.smithi138.stdout:2/247: chown d1/d15/d2d/d17/d25/f28 4010 1 2024-05-27T09:29:35.259 INFO:tasks.workunit.client.0.smithi119.stdout:8/776: mkdir d3/d8/d4d/d5d/d66/d101 0 2024-05-27T09:29:35.259 INFO:tasks.workunit.client.0.smithi119.stdout:8/777: readlink d3/d8/d1b/d46/da4/la6 0 2024-05-27T09:29:35.261 INFO:tasks.workunit.client.1.smithi138.stdout:4/358: creat d7/df/d5b/f7c x:0 0 0 2024-05-27T09:29:35.261 INFO:tasks.workunit.client.1.smithi138.stdout:4/359: chown d7/d27/l2b 98 1 2024-05-27T09:29:35.261 INFO:tasks.workunit.client.1.smithi138.stdout:4/360: write d7/f6f [418438,15666] 0 2024-05-27T09:29:35.263 INFO:tasks.workunit.client.1.smithi138.stdout:2/248: mknod d1/d15/d2d/d2e/c54 0 2024-05-27T09:29:35.269 INFO:tasks.workunit.client.0.smithi119.stdout:4/879: mknod d4/de/d21/c116 0 2024-05-27T09:29:35.270 INFO:tasks.workunit.client.0.smithi119.stdout:4/880: write d4/de/d21/d40/d47/d5c/fd5 [1405573,32133] 0 2024-05-27T09:29:35.270 INFO:tasks.workunit.client.0.smithi119.stdout:3/944: rename db/d36/d45/d4c/d62/d13/d33/dfc/fb7 to db/dc9/d98/d105/f136 0 2024-05-27T09:29:35.270 INFO:tasks.workunit.client.0.smithi119.stdout:1/878: creat d7/db/f110 x:0 0 0 2024-05-27T09:29:35.276 INFO:tasks.workunit.client.0.smithi119.stdout:8/778: creat d3/d8/d4d/d5d/db7/f102 x:0 0 0 2024-05-27T09:29:35.276 INFO:tasks.workunit.client.0.smithi119.stdout:0/804: write d2/d6/d9/d12/f79 [7249426,94062] 0 2024-05-27T09:29:35.276 INFO:tasks.workunit.client.0.smithi119.stdout:3/945: symlink db/dc9/d98/d105/l137 0 2024-05-27T09:29:35.277 INFO:tasks.workunit.client.1.smithi138.stdout:6/411: dread d3/d11/d2d/d37/f4a [0,4194304] 0 2024-05-27T09:29:35.277 INFO:tasks.workunit.client.1.smithi138.stdout:1/329: dread d4/d3c/f42 [0,4194304] 0 2024-05-27T09:29:35.277 INFO:tasks.workunit.client.1.smithi138.stdout:1/330: stat d4/d38/c47 0 2024-05-27T09:29:35.277 INFO:tasks.workunit.client.1.smithi138.stdout:1/331: chown d4/d48/d39/d5f/d4d/d2b/c5e 60872 1 2024-05-27T09:29:35.278 INFO:tasks.workunit.client.1.smithi138.stdout:1/332: write d4/d3c/f4c [8413971,55593] 0 2024-05-27T09:29:35.278 INFO:tasks.workunit.client.1.smithi138.stdout:1/333: unlink d4/d48/l33 0 2024-05-27T09:29:35.280 INFO:tasks.workunit.client.1.smithi138.stdout:1/334: mknod d4/d48/d39/d5f/d63/c69 0 2024-05-27T09:29:35.280 INFO:tasks.workunit.client.1.smithi138.stdout:1/335: fdatasync d4/d48/d39/d5f/f2f 0 2024-05-27T09:29:35.282 INFO:tasks.workunit.client.0.smithi119.stdout:8/779: mknod d3/d8/d1b/c103 0 2024-05-27T09:29:35.282 INFO:tasks.workunit.client.0.smithi119.stdout:8/780: read d3/d6e/f75 [2827285,44163] 0 2024-05-27T09:29:35.291 INFO:tasks.workunit.client.1.smithi138.stdout:2/249: write d1/d15/d2d/d17/d25/f2b [526496,123554] 0 2024-05-27T09:29:35.295 INFO:tasks.workunit.client.1.smithi138.stdout:5/261: dread d3/d12/d16/d1d/d30/f4a [0,4194304] 0 2024-05-27T09:29:35.296 INFO:tasks.workunit.client.1.smithi138.stdout:0/301: dwrite da/f15 [0,4194304] 0 2024-05-27T09:29:35.298 INFO:tasks.workunit.client.0.smithi119.stdout:2/837: dwrite d2/da/d6f/f111 [0,4194304] 0 2024-05-27T09:29:35.298 INFO:tasks.workunit.client.0.smithi119.stdout:9/712: dwrite d18/d19/fb1 [0,4194304] 0 2024-05-27T09:29:35.300 INFO:tasks.workunit.client.1.smithi138.stdout:4/361: dread d7/df/d35/f5a [0,4194304] 0 2024-05-27T09:29:35.303 INFO:tasks.workunit.client.0.smithi119.stdout:7/877: dwrite d7/d1a/d99/dcf/fd2 [4194304,4194304] 0 2024-05-27T09:29:35.308 INFO:tasks.workunit.client.1.smithi138.stdout:9/317: dwrite d6/db/de/d18/f57 [0,4194304] 0 2024-05-27T09:29:35.308 INFO:tasks.workunit.client.1.smithi138.stdout:2/250: symlink d1/d15/l55 0 2024-05-27T09:29:35.308 INFO:tasks.workunit.client.1.smithi138.stdout:2/251: readlink d1/d15/l55 0 2024-05-27T09:29:35.308 INFO:tasks.workunit.client.1.smithi138.stdout:5/262: unlink f0 0 2024-05-27T09:29:35.308 INFO:tasks.workunit.client.1.smithi138.stdout:0/302: mknod da/d2a/d2b/c69 0 2024-05-27T09:29:35.308 INFO:tasks.workunit.client.1.smithi138.stdout:0/303: read da/f29 [1599704,78265] 0 2024-05-27T09:29:35.311 INFO:tasks.workunit.client.1.smithi138.stdout:5/263: getdents d3/d12/d16/d1d/d30 0 2024-05-27T09:29:35.313 INFO:tasks.workunit.client.0.smithi119.stdout:2/838: symlink d2/da/d8a/dd4/d89/db0/l11d 0 2024-05-27T09:29:35.315 INFO:tasks.workunit.client.1.smithi138.stdout:3/259: sync 2024-05-27T09:29:35.315 INFO:tasks.workunit.client.1.smithi138.stdout:3/260: stat d0/d2/da/l16 0 2024-05-27T09:29:35.315 INFO:tasks.workunit.client.1.smithi138.stdout:3/261: truncate d0/d2/da/df/d10/f4a 896861 0 2024-05-27T09:29:35.321 INFO:tasks.workunit.client.0.smithi119.stdout:7/878: rename d7/df/f10 to d7/d5f/d79/f118 0 2024-05-27T09:29:35.321 INFO:tasks.workunit.client.0.smithi119.stdout:7/879: write d7/df/d19/d78/ff9 [315431,21897] 0 2024-05-27T09:29:35.322 INFO:tasks.workunit.client.0.smithi119.stdout:7/880: dread - d7/df/d19/d97/d92/db7/dba/d42/d76/f9d zero size 2024-05-27T09:29:35.322 INFO:tasks.workunit.client.0.smithi119.stdout:7/881: creat d7/df/d19/d97/d37/f119 x:0 0 0 2024-05-27T09:29:35.336 INFO:tasks.workunit.client.1.smithi138.stdout:4/362: dread d7/fb [0,4194304] 0 2024-05-27T09:29:35.337 INFO:tasks.workunit.client.1.smithi138.stdout:5/264: dread d3/d12/d16/d1d/d22/d2b/d36/f41 [0,4194304] 0 2024-05-27T09:29:35.337 INFO:tasks.workunit.client.1.smithi138.stdout:4/363: unlink d7/cd 0 2024-05-27T09:29:35.341 INFO:tasks.workunit.client.1.smithi138.stdout:5/265: creat d3/d12/d26/f57 x:0 0 0 2024-05-27T09:29:35.341 INFO:tasks.workunit.client.1.smithi138.stdout:5/266: stat d3/d12/d16/d1d 0 2024-05-27T09:29:35.353 INFO:tasks.workunit.client.1.smithi138.stdout:0/304: rmdir da/d2a 39 2024-05-27T09:29:35.353 INFO:tasks.workunit.client.1.smithi138.stdout:0/305: write da/f10 [4246097,104226] 0 2024-05-27T09:29:35.370 INFO:tasks.workunit.client.1.smithi138.stdout:9/318: read d6/da/d26/d28/f49 [58071,62727] 0 2024-05-27T09:29:35.373 INFO:tasks.workunit.client.1.smithi138.stdout:9/319: creat d6/db/de/d18/f64 x:0 0 0 2024-05-27T09:29:35.375 INFO:tasks.workunit.client.1.smithi138.stdout:6/412: dwrite d3/d19/f22 [0,4194304] 0 2024-05-27T09:29:35.379 INFO:tasks.workunit.client.1.smithi138.stdout:4/364: dread d7/df/d35/f5c [0,4194304] 0 2024-05-27T09:29:35.382 INFO:tasks.workunit.client.0.smithi119.stdout:3/946: dwrite db/fbf [0,4194304] 0 2024-05-27T09:29:35.383 INFO:tasks.workunit.client.1.smithi138.stdout:3/262: dwrite d0/d2/da/df/d10/d2a/d48/f4d [0,4194304] 0 2024-05-27T09:29:35.384 INFO:tasks.workunit.client.0.smithi119.stdout:8/781: dwrite d3/f87 [0,4194304] 0 2024-05-27T09:29:35.384 INFO:tasks.workunit.client.1.smithi138.stdout:1/336: truncate d4/d3c/f4c 7206625 0 2024-05-27T09:29:35.385 INFO:tasks.workunit.client.1.smithi138.stdout:1/337: fdatasync d4/d48/d22/f35 0 2024-05-27T09:29:35.387 INFO:tasks.workunit.client.1.smithi138.stdout:6/413: symlink d3/d35/d5a/d6a/l94 0 2024-05-27T09:29:35.393 INFO:tasks.workunit.client.1.smithi138.stdout:4/365: link d7/df/l14 d7/df/d5b/l7d 0 2024-05-27T09:29:35.393 INFO:tasks.workunit.client.1.smithi138.stdout:4/366: readlink d7/df/d35/l3f 0 2024-05-27T09:29:35.401 INFO:tasks.workunit.client.0.smithi119.stdout:7/882: dwrite d7/df/d19/d8d/fa3 [0,4194304] 0 2024-05-27T09:29:35.401 INFO:tasks.workunit.client.0.smithi119.stdout:3/947: mkdir db/d36/d45/d49/d9d/d138 0 2024-05-27T09:29:35.402 INFO:tasks.workunit.client.1.smithi138.stdout:6/414: unlink d3/d11/d33/f66 0 2024-05-27T09:29:35.402 INFO:tasks.workunit.client.1.smithi138.stdout:6/415: fsync d3/d35/d43/d1c/d32/d47/f7a 0 2024-05-27T09:29:35.406 INFO:tasks.workunit.client.0.smithi119.stdout:6/782: sync 2024-05-27T09:29:35.406 INFO:tasks.workunit.client.0.smithi119.stdout:5/909: sync 2024-05-27T09:29:35.406 INFO:tasks.workunit.client.0.smithi119.stdout:5/910: fdatasync d6/dd/d39/d19/d70/dc1/d41/fda 0 2024-05-27T09:29:35.406 INFO:tasks.workunit.client.1.smithi138.stdout:3/263: dread d0/fd [0,4194304] 0 2024-05-27T09:29:35.407 INFO:tasks.workunit.client.0.smithi119.stdout:8/782: rmdir d3/d8/d4d/d5d/d66 39 2024-05-27T09:29:35.407 INFO:tasks.workunit.client.1.smithi138.stdout:4/367: symlink d7/df/d35/d4c/d59/d5d/d64/d6a/l7e 0 2024-05-27T09:29:35.407 INFO:tasks.workunit.client.1.smithi138.stdout:1/338: unlink d4/d3c/f4c 0 2024-05-27T09:29:35.408 INFO:tasks.workunit.client.0.smithi119.stdout:6/783: creat d0/d85/f117 x:0 0 0 2024-05-27T09:29:35.408 INFO:tasks.workunit.client.0.smithi119.stdout:6/784: chown d0/d2/d4e/feb 30704 1 2024-05-27T09:29:35.409 INFO:tasks.workunit.client.1.smithi138.stdout:4/368: getdents d7/df/d35/d4c/d59 0 2024-05-27T09:29:35.409 INFO:tasks.workunit.client.1.smithi138.stdout:4/369: creat d7/df/d5b/f7f x:0 0 0 2024-05-27T09:29:35.409 INFO:tasks.workunit.client.1.smithi138.stdout:9/320: dread d6/db/f39 [0,4194304] 0 2024-05-27T09:29:35.409 INFO:tasks.workunit.client.1.smithi138.stdout:1/339: creat d4/d48/d39/d5f/f6a x:0 0 0 2024-05-27T09:29:35.411 INFO:tasks.workunit.client.1.smithi138.stdout:1/340: write d4/d48/d39/d5f/d4d/f3f [2132429,125479] 0 2024-05-27T09:29:35.411 INFO:tasks.workunit.client.0.smithi119.stdout:6/785: mkdir d0/d2/d57/d37/d74/df1/d109/d118 0 2024-05-27T09:29:35.411 INFO:tasks.workunit.client.0.smithi119.stdout:6/786: truncate d0/f9a 462845 0 2024-05-27T09:29:35.414 INFO:tasks.workunit.client.1.smithi138.stdout:3/264: rename d0/d2/f22 to d0/f54 0 2024-05-27T09:29:35.416 INFO:tasks.workunit.client.1.smithi138.stdout:0/306: dread da/f10 [0,4194304] 0 2024-05-27T09:29:35.416 INFO:tasks.workunit.client.1.smithi138.stdout:4/370: truncate d7/fb 2209696 0 2024-05-27T09:29:35.418 INFO:tasks.workunit.client.0.smithi119.stdout:6/787: link d0/d85/de7/l10c d0/d2/d57/d37/d115/l119 0 2024-05-27T09:29:35.420 INFO:tasks.workunit.client.1.smithi138.stdout:3/265: write d0/d2/da/f2d [182197,1074] 0 2024-05-27T09:29:35.421 INFO:tasks.workunit.client.1.smithi138.stdout:0/307: mkdir da/d17/d1d/d6a 0 2024-05-27T09:29:35.422 INFO:tasks.workunit.client.0.smithi119.stdout:1/879: dread d7/db/d12/d15/d1f/fbb [0,4194304] 0 2024-05-27T09:29:35.422 INFO:tasks.workunit.client.0.smithi119.stdout:4/881: dwrite d4/f8f [0,4194304] 0 2024-05-27T09:29:35.422 INFO:tasks.workunit.client.0.smithi119.stdout:4/882: write d4/d6/d28/ff8 [921928,85659] 0 2024-05-27T09:29:35.422 INFO:tasks.workunit.client.1.smithi138.stdout:5/267: dwrite d3/d12/d32/f42 [0,4194304] 0 2024-05-27T09:29:35.424 INFO:tasks.workunit.client.0.smithi119.stdout:1/880: mknod d7/db/d8f/d9b/dc9/ddc/c111 0 2024-05-27T09:29:35.430 INFO:tasks.workunit.client.1.smithi138.stdout:0/308: unlink da/d17/d1d/d43/c45 0 2024-05-27T09:29:35.430 INFO:tasks.workunit.client.1.smithi138.stdout:5/268: creat d3/d12/d26/d53/f58 x:0 0 0 2024-05-27T09:29:35.431 INFO:tasks.workunit.client.1.smithi138.stdout:5/269: chown d3/d12/f31 629 1 2024-05-27T09:29:35.434 INFO:tasks.workunit.client.0.smithi119.stdout:4/883: getdents d4/de/d21/d40/d47/da3 0 2024-05-27T09:29:35.435 INFO:tasks.workunit.client.1.smithi138.stdout:6/416: write d3/d11/f2e [3863712,54234] 0 2024-05-27T09:29:35.438 INFO:tasks.workunit.client.1.smithi138.stdout:5/270: mknod d3/d12/d16/c59 0 2024-05-27T09:29:35.446 INFO:tasks.workunit.client.1.smithi138.stdout:6/417: mknod d3/d11/d2d/c95 0 2024-05-27T09:29:35.447 INFO:tasks.workunit.client.1.smithi138.stdout:0/309: mknod da/d17/d1d/d1f/d42/d4a/d53/c6b 0 2024-05-27T09:29:35.448 INFO:tasks.workunit.client.1.smithi138.stdout:0/310: creat da/d17/d1d/d43/f6c x:0 0 0 2024-05-27T09:29:35.451 INFO:tasks.workunit.client.1.smithi138.stdout:4/371: dread d7/d27/f2c [0,4194304] 0 2024-05-27T09:29:35.452 INFO:tasks.workunit.client.1.smithi138.stdout:6/418: creat d3/d35/d43/d8c/f96 x:0 0 0 2024-05-27T09:29:35.452 INFO:tasks.workunit.client.1.smithi138.stdout:6/419: stat d3/d35/d43/l3e 0 2024-05-27T09:29:35.453 INFO:tasks.workunit.client.1.smithi138.stdout:6/420: creat d3/d35/f97 x:0 0 0 2024-05-27T09:29:35.454 INFO:tasks.workunit.client.1.smithi138.stdout:6/421: chown d3/d11/d2d/d37/f6d 10318287 1 2024-05-27T09:29:35.458 INFO:tasks.workunit.client.1.smithi138.stdout:0/311: symlink da/d17/d1d/d43/l6d 0 2024-05-27T09:29:35.462 INFO:tasks.workunit.client.1.smithi138.stdout:4/372: link d7/d27/f2c d7/d27/f80 0 2024-05-27T09:29:35.462 INFO:tasks.workunit.client.1.smithi138.stdout:4/373: chown d7/df/d5b/l71 0 1 2024-05-27T09:29:35.462 INFO:tasks.workunit.client.1.smithi138.stdout:4/374: write d7/df/d35/d4c/d59/f73 [3665072,114692] 0 2024-05-27T09:29:35.472 INFO:tasks.workunit.client.1.smithi138.stdout:9/321: read d6/db/de/d18/f57 [2329764,56343] 0 2024-05-27T09:29:35.473 INFO:tasks.workunit.client.1.smithi138.stdout:9/322: readlink d6/db/l56 0 2024-05-27T09:29:35.473 INFO:tasks.workunit.client.1.smithi138.stdout:9/323: creat d6/db/de/d18/d24/f65 x:0 0 0 2024-05-27T09:29:35.474 INFO:tasks.workunit.client.1.smithi138.stdout:3/266: dread d0/d2/da/df/d10/d2a/f42 [0,4194304] 0 2024-05-27T09:29:35.479 INFO:tasks.workunit.client.1.smithi138.stdout:0/312: creat da/d17/d1d/d1f/d42/d4a/d53/f6e x:0 0 0 2024-05-27T09:29:35.479 INFO:tasks.workunit.client.1.smithi138.stdout:0/313: dread - da/d17/d1d/d43/f64 zero size 2024-05-27T09:29:35.484 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:35 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:35.484 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:35 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:35.484 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:35 smithi138 ceph-mon[34885]: pgmap v15: 65 pgs: 65 active+clean; 2.2 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 153 MiB/s rd, 183 MiB/s wr, 272 op/s 2024-05-27T09:29:35.496 INFO:tasks.workunit.client.0.smithi119.stdout:8/783: dread d3/d8/d4d/d5d/db7/fee [0,4194304] 0 2024-05-27T09:29:35.496 INFO:tasks.workunit.client.0.smithi119.stdout:8/784: chown d3/d8/d1b/fa7 321 1 2024-05-27T09:29:35.497 INFO:tasks.workunit.client.0.smithi119.stdout:8/785: chown d3/d6e/d80/la1 777349 1 2024-05-27T09:29:35.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:35 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:35.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:35 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:35.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:35 smithi119 ceph-mon[29082]: pgmap v15: 65 pgs: 65 active+clean; 2.2 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 153 MiB/s rd, 183 MiB/s wr, 272 op/s 2024-05-27T09:29:35.498 INFO:tasks.workunit.client.0.smithi119.stdout:6/788: getdents d0/d2/d57/d37/d115 0 2024-05-27T09:29:35.498 INFO:tasks.workunit.client.0.smithi119.stdout:6/789: write d0/d2/d4e/f1f [3649187,130636] 0 2024-05-27T09:29:35.498 INFO:tasks.workunit.client.0.smithi119.stdout:6/790: dread - d0/d2/d1c/d21/d58/f114 zero size 2024-05-27T09:29:35.498 INFO:tasks.workunit.client.0.smithi119.stdout:8/786: write d3/d8/f47 [167712,45571] 0 2024-05-27T09:29:35.508 INFO:tasks.workunit.client.0.smithi119.stdout:6/791: write d0/d85/fae [24408,39807] 0 2024-05-27T09:29:35.510 INFO:tasks.workunit.client.0.smithi119.stdout:6/792: getdents d0/d2 0 2024-05-27T09:29:35.510 INFO:tasks.workunit.client.0.smithi119.stdout:0/805: dwrite d2/d6/d9/d24/d30/dcc/fd9 [0,4194304] 0 2024-05-27T09:29:35.511 INFO:tasks.workunit.client.0.smithi119.stdout:0/806: fsync d2/d6/d23/d7a/fd4 0 2024-05-27T09:29:35.511 INFO:tasks.workunit.client.0.smithi119.stdout:0/807: creat d2/d6/d9/d12/d4b/deb/f107 x:0 0 0 2024-05-27T09:29:35.513 INFO:tasks.workunit.client.0.smithi119.stdout:6/793: creat d0/d2/d57/d37/d74/dfe/f11a x:0 0 0 2024-05-27T09:29:35.525 INFO:tasks.workunit.client.1.smithi138.stdout:3/267: dread d0/d2/f3d [0,4194304] 0 2024-05-27T09:29:35.525 INFO:tasks.workunit.client.1.smithi138.stdout:9/324: dread f1 [0,4194304] 0 2024-05-27T09:29:35.527 INFO:tasks.workunit.client.1.smithi138.stdout:3/268: creat d0/d3f/f55 x:0 0 0 2024-05-27T09:29:35.527 INFO:tasks.workunit.client.1.smithi138.stdout:3/269: creat d0/d2/da/df/d10/d2a/f56 x:0 0 0 2024-05-27T09:29:35.530 INFO:tasks.workunit.client.1.smithi138.stdout:9/325: symlink d6/da/d29/d61/l66 0 2024-05-27T09:29:35.531 INFO:tasks.workunit.client.0.smithi119.stdout:4/884: read d4/d23/f35 [4041775,4147] 0 2024-05-27T09:29:35.532 INFO:tasks.workunit.client.0.smithi119.stdout:4/885: creat d4/d1c/f117 x:0 0 0 2024-05-27T09:29:35.532 INFO:tasks.workunit.client.0.smithi119.stdout:0/808: dread d2/d6/d23/db8/ff5 [0,4194304] 0 2024-05-27T09:29:35.533 INFO:tasks.workunit.client.0.smithi119.stdout:4/886: truncate d4/de/d21/f2d 2695142 0 2024-05-27T09:29:35.533 INFO:tasks.workunit.client.0.smithi119.stdout:8/787: dread d3/d8/d1b/d46/da4/fb9 [0,4194304] 0 2024-05-27T09:29:35.533 INFO:tasks.workunit.client.0.smithi119.stdout:8/788: stat d3 0 2024-05-27T09:29:35.533 INFO:tasks.workunit.client.0.smithi119.stdout:8/789: creat d3/d8/d1b/d46/da4/dbb/f104 x:0 0 0 2024-05-27T09:29:35.537 INFO:tasks.workunit.client.0.smithi119.stdout:4/887: link d4/d1c/d39/l46 d4/d23/d24/d5b/da8/l118 0 2024-05-27T09:29:35.537 INFO:tasks.workunit.client.0.smithi119.stdout:2/839: dwrite d2/da/d2c/f38 [0,4194304] 0 2024-05-27T09:29:35.537 INFO:tasks.workunit.client.0.smithi119.stdout:2/840: fdatasync d2/da/d15/fd7 0 2024-05-27T09:29:35.538 INFO:tasks.workunit.client.1.smithi138.stdout:9/326: symlink d6/da/d2f/l67 0 2024-05-27T09:29:35.539 INFO:tasks.workunit.client.0.smithi119.stdout:9/713: dwrite d18/d19/d1b/d40/d44/f93 [0,4194304] 0 2024-05-27T09:29:35.540 INFO:tasks.workunit.client.0.smithi119.stdout:8/790: creat d3/d8/d4d/d5d/d66/f105 x:0 0 0 2024-05-27T09:29:35.541 INFO:tasks.workunit.client.0.smithi119.stdout:1/881: dwrite d7/db/f110 [0,4194304] 0 2024-05-27T09:29:35.544 INFO:tasks.workunit.client.0.smithi119.stdout:3/948: dwrite db/d36/d45/dab/f103 [0,4194304] 0 2024-05-27T09:29:35.544 INFO:tasks.workunit.client.0.smithi119.stdout:6/794: dread d0/d2/d57/d37/f86 [4194304,4194304] 0 2024-05-27T09:29:35.544 INFO:tasks.workunit.client.1.smithi138.stdout:4/375: dwrite d7/d27/f80 [0,4194304] 0 2024-05-27T09:29:35.544 INFO:tasks.workunit.client.0.smithi119.stdout:7/883: dwrite d7/df/d19/d97/d92/db7/dba/d42/fdf [0,4194304] 0 2024-05-27T09:29:35.546 INFO:tasks.workunit.client.0.smithi119.stdout:2/841: mkdir d2/da/d8a/d8f/d11e 0 2024-05-27T09:29:35.547 INFO:tasks.workunit.client.1.smithi138.stdout:4/376: creat d7/d6e/f81 x:0 0 0 2024-05-27T09:29:35.548 INFO:tasks.workunit.client.0.smithi119.stdout:4/888: creat d4/d6/d3e/d89/dcf/f119 x:0 0 0 2024-05-27T09:29:35.548 INFO:tasks.workunit.client.1.smithi138.stdout:4/377: stat d7/df/d35/d4c/d59/d5d 0 2024-05-27T09:29:35.548 INFO:tasks.workunit.client.1.smithi138.stdout:4/378: dread - d7/df/d35/f77 zero size 2024-05-27T09:29:35.548 INFO:tasks.workunit.client.1.smithi138.stdout:4/379: chown d7/df/d35/d4c/f66 4341 1 2024-05-27T09:29:35.549 INFO:tasks.workunit.client.0.smithi119.stdout:8/791: mkdir d3/d8/d1b/d77/d95/d106 0 2024-05-27T09:29:35.551 INFO:tasks.workunit.client.0.smithi119.stdout:1/882: mkdir d7/dd/d25/d60/d61/d112 0 2024-05-27T09:29:35.552 INFO:tasks.workunit.client.1.smithi138.stdout:6/422: dwrite d3/d35/d43/d1c/f59 [0,4194304] 0 2024-05-27T09:29:35.553 INFO:tasks.workunit.client.1.smithi138.stdout:0/314: dread da/f3c [0,4194304] 0 2024-05-27T09:29:35.559 INFO:tasks.workunit.client.1.smithi138.stdout:6/423: symlink d3/d35/d43/l98 0 2024-05-27T09:29:35.559 INFO:tasks.workunit.client.1.smithi138.stdout:6/424: fsync d3/d11/d2d/f75 0 2024-05-27T09:29:35.562 INFO:tasks.workunit.client.1.smithi138.stdout:0/315: creat da/d17/d1d/d1f/d4f/f6f x:0 0 0 2024-05-27T09:29:35.566 INFO:tasks.workunit.client.1.smithi138.stdout:9/327: dread d6/f3b [0,4194304] 0 2024-05-27T09:29:35.572 INFO:tasks.workunit.client.0.smithi119.stdout:6/795: mkdir d0/d2/d11b 0 2024-05-27T09:29:35.573 INFO:tasks.workunit.client.0.smithi119.stdout:7/884: rmdir d7/df 39 2024-05-27T09:29:35.573 INFO:tasks.workunit.client.0.smithi119.stdout:2/842: getdents d2/da/d8a/dd4/d89/de8 0 2024-05-27T09:29:35.579 INFO:tasks.workunit.client.0.smithi119.stdout:8/792: creat d3/d8/d4d/d5d/f107 x:0 0 0 2024-05-27T09:29:35.579 INFO:tasks.workunit.client.0.smithi119.stdout:8/793: write d3/fe5 [514454,121706] 0 2024-05-27T09:29:35.579 INFO:tasks.workunit.client.0.smithi119.stdout:4/889: creat d4/de/d66/f11a x:0 0 0 2024-05-27T09:29:35.579 INFO:tasks.workunit.client.0.smithi119.stdout:4/890: chown d4/d6/d3e/c8b 453612 1 2024-05-27T09:29:35.579 INFO:tasks.workunit.client.0.smithi119.stdout:4/891: creat d4/d6/d81/dce/d88/f11b x:0 0 0 2024-05-27T09:29:35.580 INFO:tasks.workunit.client.1.smithi138.stdout:3/270: link d0/d2/da/l2f d0/d2/l57 0 2024-05-27T09:29:35.580 INFO:tasks.workunit.client.1.smithi138.stdout:3/271: fdatasync d0/d3f/f55 0 2024-05-27T09:29:35.580 INFO:tasks.workunit.client.1.smithi138.stdout:9/328: creat d6/da/d29/d61/f68 x:0 0 0 2024-05-27T09:29:35.581 INFO:tasks.workunit.client.0.smithi119.stdout:6/796: symlink d0/d2/d57/d37/dad/db6/l11c 0 2024-05-27T09:29:35.583 INFO:tasks.workunit.client.0.smithi119.stdout:2/843: mknod d2/da/d15/d53/d114/c11f 0 2024-05-27T09:29:35.584 INFO:tasks.workunit.client.0.smithi119.stdout:2/844: creat d2/deb/f120 x:0 0 0 2024-05-27T09:29:35.584 INFO:tasks.workunit.client.0.smithi119.stdout:6/797: write d0/d2/d57/d37/d74/df1/d109/fb0 [514128,108496] 0 2024-05-27T09:29:35.586 INFO:tasks.workunit.client.0.smithi119.stdout:8/794: symlink d3/d8/d1b/l108 0 2024-05-27T09:29:35.586 INFO:tasks.workunit.client.0.smithi119.stdout:8/795: chown d3/d8/d4d/d5d/d66/f94 1 1 2024-05-27T09:29:35.587 INFO:tasks.workunit.client.1.smithi138.stdout:3/272: symlink d0/d2/l58 0 2024-05-27T09:29:35.589 INFO:tasks.workunit.client.1.smithi138.stdout:3/273: symlink d0/d2/da/df/d10/l59 0 2024-05-27T09:29:35.591 INFO:tasks.workunit.client.0.smithi119.stdout:4/892: rename d4/d1c/d39/fee to d4/de/d21/d40/d47/d5c/f11c 0 2024-05-27T09:29:35.597 INFO:tasks.workunit.client.1.smithi138.stdout:9/329: dread d6/db/dc/f1e [0,4194304] 0 2024-05-27T09:29:35.597 INFO:tasks.workunit.client.1.smithi138.stdout:9/330: dread - d6/db/dc/f4d zero size 2024-05-27T09:29:35.600 INFO:tasks.workunit.client.1.smithi138.stdout:9/331: mknod d6/da/d26/d28/c69 0 2024-05-27T09:29:35.600 INFO:tasks.workunit.client.0.smithi119.stdout:1/883: dread d7/db/d12/d26/f97 [0,4194304] 0 2024-05-27T09:29:35.601 INFO:tasks.workunit.client.0.smithi119.stdout:2/845: mkdir d2/da/d15/d121 0 2024-05-27T09:29:35.601 INFO:tasks.workunit.client.0.smithi119.stdout:2/846: write d2/da/d4e/f72 [1165044,98471] 0 2024-05-27T09:29:35.603 INFO:tasks.workunit.client.1.smithi138.stdout:4/380: dread d7/df/d35/d4c/d59/f73 [0,4194304] 0 2024-05-27T09:29:35.605 INFO:tasks.workunit.client.1.smithi138.stdout:4/381: fdatasync d7/f25 0 2024-05-27T09:29:35.608 INFO:tasks.workunit.client.0.smithi119.stdout:6/798: unlink d0/d2/f31 0 2024-05-27T09:29:35.612 INFO:tasks.workunit.client.0.smithi119.stdout:1/884: creat d7/db6/dc2/f113 x:0 0 0 2024-05-27T09:29:35.612 INFO:tasks.workunit.client.0.smithi119.stdout:1/885: write d7/db/d12/d15/fa1 [195984,17996] 0 2024-05-27T09:29:35.613 INFO:tasks.workunit.client.0.smithi119.stdout:1/886: fdatasync d7/f5b 0 2024-05-27T09:29:35.617 INFO:tasks.workunit.client.0.smithi119.stdout:4/893: link d4/d23/f26 d4/de/f11d 0 2024-05-27T09:29:35.618 INFO:tasks.workunit.client.0.smithi119.stdout:2/847: truncate d2/da/f28 940671 0 2024-05-27T09:29:35.623 INFO:tasks.workunit.client.0.smithi119.stdout:4/894: link d4/d48/f51 d4/d6/d28/d2f/f11e 0 2024-05-27T09:29:35.625 INFO:tasks.workunit.client.0.smithi119.stdout:8/796: dread d3/f25 [0,4194304] 0 2024-05-27T09:29:35.625 INFO:tasks.workunit.client.0.smithi119.stdout:8/797: creat d3/de/f109 x:0 0 0 2024-05-27T09:29:35.628 INFO:tasks.workunit.client.1.smithi138.stdout:6/425: dwrite d3/d35/d5a/f68 [0,4194304] 0 2024-05-27T09:29:35.628 INFO:tasks.workunit.client.1.smithi138.stdout:6/426: chown d3/d35/d5a/d6a/f6f 29353 1 2024-05-27T09:29:35.629 INFO:tasks.workunit.client.1.smithi138.stdout:6/427: creat d3/d35/d7d/f99 x:0 0 0 2024-05-27T09:29:35.629 INFO:tasks.workunit.client.1.smithi138.stdout:6/428: getdents d3/d91 0 2024-05-27T09:29:35.629 INFO:tasks.workunit.client.0.smithi119.stdout:4/895: creat d4/d23/d24/d5b/da8/dda/ddb/f11f x:0 0 0 2024-05-27T09:29:35.633 INFO:tasks.workunit.client.1.smithi138.stdout:9/332: dread d6/db/dc/f50 [0,4194304] 0 2024-05-27T09:29:35.635 INFO:tasks.workunit.client.0.smithi119.stdout:9/714: dwrite d18/d19/d1b/d40/d44/f45 [0,4194304] 0 2024-05-27T09:29:35.635 INFO:tasks.workunit.client.0.smithi119.stdout:9/715: chown d18/d19/d1b/d2a/c66 1 1 2024-05-27T09:29:35.636 INFO:tasks.workunit.client.0.smithi119.stdout:4/896: creat d4/d6/d81/dce/f120 x:0 0 0 2024-05-27T09:29:35.646 INFO:tasks.workunit.client.1.smithi138.stdout:3/274: dread d0/d2/f13 [0,4194304] 0 2024-05-27T09:29:35.648 INFO:tasks.workunit.client.1.smithi138.stdout:3/275: symlink d0/d2/da/l5a 0 2024-05-27T09:29:35.654 INFO:tasks.workunit.client.1.smithi138.stdout:3/276: creat d0/d2/da/df/d10/d32/f5b x:0 0 0 2024-05-27T09:29:35.654 INFO:tasks.workunit.client.1.smithi138.stdout:3/277: fdatasync d0/fd 0 2024-05-27T09:29:35.656 INFO:tasks.workunit.client.1.smithi138.stdout:3/278: truncate d0/d2/da/df/d10/d32/f1d 3366341 0 2024-05-27T09:29:35.658 INFO:tasks.workunit.client.0.smithi119.stdout:4/897: rmdir d4/d6/d28 39 2024-05-27T09:29:35.659 INFO:tasks.workunit.client.1.smithi138.stdout:3/279: rename d0/d3f/d50/f52 to d0/d3f/d50/f5c 0 2024-05-27T09:29:35.660 INFO:tasks.workunit.client.1.smithi138.stdout:3/280: creat d0/d2/da/df/d10/f5d x:0 0 0 2024-05-27T09:29:35.661 INFO:tasks.workunit.client.1.smithi138.stdout:3/281: readlink d0/d2/da/df/d10/l44 0 2024-05-27T09:29:35.670 INFO:tasks.workunit.client.1.smithi138.stdout:3/282: dread d0/d2/da/df/d10/d32/f28 [0,4194304] 0 2024-05-27T09:29:35.670 INFO:tasks.workunit.client.1.smithi138.stdout:6/429: write d3/d11/d2d/d37/f3c [3998357,120420] 0 2024-05-27T09:29:35.671 INFO:tasks.workunit.client.1.smithi138.stdout:3/283: mkdir d0/d2/da/df/d5e 0 2024-05-27T09:29:35.675 INFO:tasks.workunit.client.1.smithi138.stdout:3/284: creat d0/d2/da/df/d10/d2a/d48/f5f x:0 0 0 2024-05-27T09:29:35.675 INFO:tasks.workunit.client.1.smithi138.stdout:3/285: chown d0/d2/da/df/d10/d32/d37/l3b 1797295507 1 2024-05-27T09:29:35.675 INFO:tasks.workunit.client.1.smithi138.stdout:6/430: mknod d3/d35/d43/d1c/d32/c9a 0 2024-05-27T09:29:35.676 INFO:tasks.workunit.client.1.smithi138.stdout:3/286: mkdir d0/d3f/d60 0 2024-05-27T09:29:35.677 INFO:tasks.workunit.client.1.smithi138.stdout:6/431: mkdir d3/d35/d43/d1c/d32/d9b 0 2024-05-27T09:29:35.679 INFO:tasks.workunit.client.1.smithi138.stdout:4/382: dwrite d7/f23 [0,4194304] 0 2024-05-27T09:29:35.679 INFO:tasks.workunit.client.1.smithi138.stdout:4/383: readlink d7/d6e/l74 0 2024-05-27T09:29:35.680 INFO:tasks.workunit.client.0.smithi119.stdout:7/885: dwrite d7/fd [0,4194304] 0 2024-05-27T09:29:35.680 INFO:tasks.workunit.client.0.smithi119.stdout:3/949: dwrite db/d36/d45/d4c/d120/f131 [0,4194304] 0 2024-05-27T09:29:35.680 INFO:tasks.workunit.client.0.smithi119.stdout:7/886: dread - d7/df/d19/d97/d37/fac zero size 2024-05-27T09:29:35.681 INFO:tasks.workunit.client.1.smithi138.stdout:3/287: mkdir d0/d2/da/df/d10/d2a/d61 0 2024-05-27T09:29:35.681 INFO:tasks.workunit.client.1.smithi138.stdout:9/333: read d6/f48 [3071950,105115] 0 2024-05-27T09:29:35.682 INFO:tasks.workunit.client.1.smithi138.stdout:0/316: dwrite da/d2a/d3f/f4c [4194304,4194304] 0 2024-05-27T09:29:35.682 INFO:tasks.workunit.client.1.smithi138.stdout:0/317: rename da to da/d17/d1d/d1f/d60/d70 22 2024-05-27T09:29:35.685 INFO:tasks.workunit.client.1.smithi138.stdout:3/288: creat d0/f62 x:0 0 0 2024-05-27T09:29:35.686 INFO:tasks.workunit.client.1.smithi138.stdout:3/289: creat d0/d3f/d50/f63 x:0 0 0 2024-05-27T09:29:35.686 INFO:tasks.workunit.client.1.smithi138.stdout:3/290: dread - d0/d2/da/df/d10/d2a/f56 zero size 2024-05-27T09:29:35.686 INFO:tasks.workunit.client.0.smithi119.stdout:2/848: dread d2/d1a/f46 [0,4194304] 0 2024-05-27T09:29:35.689 INFO:tasks.workunit.client.1.smithi138.stdout:9/334: truncate d6/f9 3826157 0 2024-05-27T09:29:35.689 INFO:tasks.workunit.client.0.smithi119.stdout:3/950: mkdir db/dc9/d98/d7a/d139 0 2024-05-27T09:29:35.690 INFO:tasks.workunit.client.1.smithi138.stdout:3/291: creat d0/d2/da/df/d5e/f64 x:0 0 0 2024-05-27T09:29:35.690 INFO:tasks.workunit.client.1.smithi138.stdout:3/292: truncate d0/d2/da/df/d10/d2a/d48/f5f 939039 0 2024-05-27T09:29:35.691 INFO:tasks.workunit.client.0.smithi119.stdout:7/887: creat d7/d1a/d99/d112/f11a x:0 0 0 2024-05-27T09:29:35.691 INFO:tasks.workunit.client.0.smithi119.stdout:7/888: readlink d7/df/d19/l17 0 2024-05-27T09:29:35.695 INFO:tasks.workunit.client.1.smithi138.stdout:3/293: write d0/d2/da/df/d10/d32/f5b [1036098,113270] 0 2024-05-27T09:29:35.695 INFO:tasks.workunit.client.1.smithi138.stdout:3/294: dread - d0/d3f/d50/f63 zero size 2024-05-27T09:29:35.695 INFO:tasks.workunit.client.1.smithi138.stdout:3/295: dread - d0/f62 zero size 2024-05-27T09:29:35.695 INFO:tasks.workunit.client.1.smithi138.stdout:3/296: chown d0/d2/f4b 467120 1 2024-05-27T09:29:35.695 INFO:tasks.workunit.client.1.smithi138.stdout:3/297: read - d0/d2/da/df/d10/d2a/f4f zero size 2024-05-27T09:29:35.696 INFO:tasks.workunit.client.0.smithi119.stdout:2/849: chown d2/da/d2c/c45 3 1 2024-05-27T09:29:35.701 INFO:tasks.workunit.client.1.smithi138.stdout:3/298: dread d0/d2/f3d [0,4194304] 0 2024-05-27T09:29:35.701 INFO:tasks.workunit.client.1.smithi138.stdout:3/299: fsync d0/d2/f45 0 2024-05-27T09:29:35.704 INFO:tasks.workunit.client.0.smithi119.stdout:4/898: dread d4/de/d66/f68 [0,4194304] 0 2024-05-27T09:29:35.704 INFO:tasks.workunit.client.0.smithi119.stdout:2/850: getdents d2/da/d8a 0 2024-05-27T09:29:35.708 INFO:tasks.workunit.client.0.smithi119.stdout:6/799: dwrite d0/d2/fca [0,4194304] 0 2024-05-27T09:29:35.710 INFO:tasks.workunit.client.0.smithi119.stdout:7/889: dread d7/df/d19/d46/d93/ff4 [0,4194304] 0 2024-05-27T09:29:35.711 INFO:tasks.workunit.client.0.smithi119.stdout:7/890: write d7/df/d19/d97/f9f [7820857,4856] 0 2024-05-27T09:29:35.712 INFO:tasks.workunit.client.1.smithi138.stdout:6/432: read d3/d11/f88 [2634144,12633] 0 2024-05-27T09:29:35.719 INFO:tasks.workunit.client.0.smithi119.stdout:3/951: write f6 [1391799,71761] 0 2024-05-27T09:29:35.722 INFO:tasks.workunit.client.0.smithi119.stdout:2/851: mkdir d2/da/d15/d53/dac/d122 0 2024-05-27T09:29:35.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.723+0000 7f35bd400700 1 -- 172.21.15.119:0/182346947 --> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f35b8061190 con 0x7f359c076be0 2024-05-27T09:29:35.726 INFO:tasks.workunit.client.1.smithi138.stdout:6/433: creat d3/d35/d43/d1c/f9c x:0 0 0 2024-05-27T09:29:35.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.730+0000 7f35af400700 1 -- 172.21.15.119:0/182346947 <== mgr.14570 v2:172.21.15.138:6828/1384904888 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f35b8061190 con 0x7f359c076be0 2024-05-27T09:29:35.732 INFO:tasks.workunit.client.1.smithi138.stdout:9/335: write d6/da/d26/d28/f58 [6788686,84531] 0 2024-05-27T09:29:35.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.734+0000 7f35bd400700 1 -- 172.21.15.119:0/182346947 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f359c076be0 msgr2=0x7f359c079090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:35.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.734+0000 7f35bd400700 1 --2- 172.21.15.119:0/182346947 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f359c076be0 0x7f359c079090 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f35a80097b0 tx=0x7f35a8006d20 comp rx=0 tx=0).stop 2024-05-27T09:29:35.735 INFO:tasks.workunit.client.0.smithi119.stdout:1/887: dwrite d7/dd/d25/fc5 [4194304,4194304] 0 2024-05-27T09:29:35.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.734+0000 7f35bd400700 1 -- 172.21.15.119:0/182346947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b81035a0 msgr2=0x7f35b8193d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:35.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.734+0000 7f35bd400700 1 --2- 172.21.15.119:0/182346947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b81035a0 0x7f35b8193d00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f35a4005b40 tx=0x7f35a400be20 comp rx=0 tx=0).stop 2024-05-27T09:29:35.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.735+0000 7f35bd400700 1 -- 172.21.15.119:0/182346947 shutdown_connections 2024-05-27T09:29:35.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.735+0000 7f35bd400700 1 --2- 172.21.15.119:0/182346947 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f359c076be0 0x7f359c079090 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:35.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.735+0000 7f35bd400700 1 --2- 172.21.15.119:0/182346947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35b8100c80 0x7f35b81937c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:35.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.735+0000 7f35bd400700 1 --2- 172.21.15.119:0/182346947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b81035a0 0x7f35b8193d00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:35.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.735+0000 7f35bd400700 1 -- 172.21.15.119:0/182346947 >> 172.21.15.119:0/182346947 conn(0x7f35b80fa5e0 msgr2=0x7f35b80fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:35.737 INFO:tasks.workunit.client.1.smithi138.stdout:6/434: mknod d3/d35/d43/d1c/d32/d9b/c9d 0 2024-05-27T09:29:35.737 INFO:tasks.workunit.client.1.smithi138.stdout:6/435: write d3/f84 [782116,72289] 0 2024-05-27T09:29:35.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.735+0000 7f35bd400700 1 -- 172.21.15.119:0/182346947 shutdown_connections 2024-05-27T09:29:35.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.735+0000 7f35bd400700 1 -- 172.21.15.119:0/182346947 wait complete. 2024-05-27T09:29:35.739 INFO:tasks.workunit.client.1.smithi138.stdout:5/271: sync 2024-05-27T09:29:35.740 INFO:tasks.workunit.client.1.smithi138.stdout:2/252: sync 2024-05-27T09:29:35.740 INFO:tasks.workunit.client.1.smithi138.stdout:9/336: creat d6/db/f6a x:0 0 0 2024-05-27T09:29:35.740 INFO:tasks.workunit.client.1.smithi138.stdout:1/341: sync 2024-05-27T09:29:35.740 INFO:tasks.workunit.client.1.smithi138.stdout:7/282: sync 2024-05-27T09:29:35.740 INFO:tasks.workunit.client.1.smithi138.stdout:8/297: sync 2024-05-27T09:29:35.741 INFO:tasks.workunit.client.1.smithi138.stdout:8/298: write d3/d8/d11/f51 [580193,121721] 0 2024-05-27T09:29:35.741 INFO:tasks.workunit.client.0.smithi119.stdout:6/800: mknod d0/d2/d57/d37/de5/c11d 0 2024-05-27T09:29:35.741 INFO:tasks.workunit.client.0.smithi119.stdout:4/899: mkdir d4/d23/d24/d121 0 2024-05-27T09:29:35.741 INFO:tasks.workunit.client.0.smithi119.stdout:6/801: dread - d0/d2/d1c/d21/ffd zero size 2024-05-27T09:29:35.744 INFO:tasks.workunit.client.1.smithi138.stdout:0/318: dwrite da/d17/d1d/d43/f64 [0,4194304] 0 2024-05-27T09:29:35.744 INFO:tasks.workunit.client.1.smithi138.stdout:0/319: dread - da/d17/d1d/d1f/d42/d4a/d53/f6e zero size 2024-05-27T09:29:35.744 INFO:tasks.workunit.client.1.smithi138.stdout:6/436: mknod d3/d11/d2d/d37/d7e/c9e 0 2024-05-27T09:29:35.745 INFO:tasks.workunit.client.0.smithi119.stdout:2/852: unlink d2/da/d8a/dd4/d89/cc2 0 2024-05-27T09:29:35.745 INFO:tasks.workunit.client.0.smithi119.stdout:9/716: dwrite d18/d19/d1b/d29/f43 [0,4194304] 0 2024-05-27T09:29:35.746 INFO:tasks.workunit.client.1.smithi138.stdout:1/342: creat d4/d3c/f6b x:0 0 0 2024-05-27T09:29:35.746 INFO:tasks.workunit.client.1.smithi138.stdout:1/343: truncate d4/d3c/f4e 941512 0 2024-05-27T09:29:35.746 INFO:tasks.workunit.client.1.smithi138.stdout:1/344: fdatasync d4/d48/d39/d5f/f5b 0 2024-05-27T09:29:35.748 INFO:tasks.workunit.client.1.smithi138.stdout:1/345: truncate d4/d3c/f6b 1011135 0 2024-05-27T09:29:35.748 INFO:tasks.workunit.client.1.smithi138.stdout:1/346: chown d4/fd 11960520 1 2024-05-27T09:29:35.752 INFO:tasks.workunit.client.0.smithi119.stdout:7/891: getdents d7/d1a 0 2024-05-27T09:29:35.752 INFO:tasks.workunit.client.1.smithi138.stdout:9/337: unlink d6/da/d29/c3e 0 2024-05-27T09:29:35.753 INFO:tasks.workunit.client.1.smithi138.stdout:0/320: mkdir da/d17/d1d/d1f/d71 0 2024-05-27T09:29:35.756 INFO:tasks.workunit.client.1.smithi138.stdout:2/253: mknod d1/d15/d2d/c56 0 2024-05-27T09:29:35.757 INFO:tasks.workunit.client.1.smithi138.stdout:7/283: creat d0/d23/f5a x:0 0 0 2024-05-27T09:29:35.759 INFO:tasks.workunit.client.1.smithi138.stdout:6/437: creat d3/d11/d2d/d37/d7e/f9f x:0 0 0 2024-05-27T09:29:35.762 INFO:tasks.workunit.client.0.smithi119.stdout:2/853: rename d2/da/cdd to d2/c123 0 2024-05-27T09:29:35.762 INFO:tasks.workunit.client.1.smithi138.stdout:1/347: readlink d4/d38/l4f 0 2024-05-27T09:29:35.762 INFO:tasks.workunit.client.1.smithi138.stdout:1/348: read d4/d48/d39/d5f/f50 [337618,75681] 0 2024-05-27T09:29:35.765 INFO:tasks.workunit.client.0.smithi119.stdout:2/854: mkdir d2/da/d15/d53/dac/d124 0 2024-05-27T09:29:35.765 INFO:tasks.workunit.client.0.smithi119.stdout:2/855: chown d2/da/d8a/dd4/d89/d94 7683611 1 2024-05-27T09:29:35.766 INFO:tasks.workunit.client.1.smithi138.stdout:2/254: mknod d1/d15/d2d/d31/d4b/c57 0 2024-05-27T09:29:35.768 INFO:tasks.workunit.client.0.smithi119.stdout:2/856: link d2/da/d8a/dd4/d69/f6b d2/da/d8a/d8f/d11e/f125 0 2024-05-27T09:29:35.770 INFO:tasks.workunit.client.1.smithi138.stdout:1/349: rename d4/d48/d22/f2c to d4/d48/d39/d5f/d4d/d2b/f6c 0 2024-05-27T09:29:35.774 INFO:tasks.workunit.client.1.smithi138.stdout:7/284: link d0/d2b/c31 d0/d23/c5b 0 2024-05-27T09:29:35.774 INFO:tasks.workunit.client.1.smithi138.stdout:1/350: link d4/f18 d4/d48/d39/d5f/d4d/f6d 0 2024-05-27T09:29:35.775 INFO:tasks.workunit.client.1.smithi138.stdout:1/351: write d4/d48/d39/d5f/d4d/d2b/f6c [1477259,20579] 0 2024-05-27T09:29:35.776 INFO:tasks.workunit.client.1.smithi138.stdout:1/352: symlink d4/d1a/d2e/d32/l6e 0 2024-05-27T09:29:35.778 INFO:tasks.workunit.client.1.smithi138.stdout:1/353: creat d4/d48/d22/d65/f6f x:0 0 0 2024-05-27T09:29:35.779 INFO:tasks.workunit.client.1.smithi138.stdout:1/354: mknod d4/d48/d39/d5f/d4d/d2b/c70 0 2024-05-27T09:29:35.779 INFO:tasks.workunit.client.1.smithi138.stdout:1/355: write d4/f18 [4436433,26385] 0 2024-05-27T09:29:35.781 INFO:tasks.workunit.client.1.smithi138.stdout:9/338: dread d6/db/f39 [0,4194304] 0 2024-05-27T09:29:35.784 INFO:tasks.workunit.client.1.smithi138.stdout:1/356: write d4/d1a/d2e/d32/f60 [1264578,83109] 0 2024-05-27T09:29:35.785 INFO:tasks.workunit.client.1.smithi138.stdout:1/357: symlink d4/d48/d22/l71 0 2024-05-27T09:29:35.786 INFO:tasks.workunit.client.1.smithi138.stdout:1/358: fdatasync d4/d48/d22/d65/f6f 0 2024-05-27T09:29:35.786 INFO:tasks.workunit.client.1.smithi138.stdout:1/359: write d4/d48/d22/f5a [427991,52040] 0 2024-05-27T09:29:35.798 INFO:tasks.workunit.client.0.smithi119.stdout:0/809: sync 2024-05-27T09:29:35.798 INFO:tasks.workunit.client.0.smithi119.stdout:5/911: sync 2024-05-27T09:29:35.798 INFO:tasks.workunit.client.0.smithi119.stdout:5/912: truncate d6/db/de/d27/f108 5175475 0 2024-05-27T09:29:35.802 INFO:tasks.workunit.client.0.smithi119.stdout:5/913: write d6/db/de/d27/dc9/fc6 [4947327,14046] 0 2024-05-27T09:29:35.802 INFO:tasks.workunit.client.0.smithi119.stdout:0/810: mkdir d2/d6/d9/d12/d4b/d59/d108 0 2024-05-27T09:29:35.802 INFO:tasks.workunit.client.1.smithi138.stdout:9/339: mkdir d6/db/de/d6b 0 2024-05-27T09:29:35.805 INFO:tasks.workunit.client.0.smithi119.stdout:0/811: mkdir d2/d6/d9/d24/d30/dcc/d109 0 2024-05-27T09:29:35.805 INFO:tasks.workunit.client.0.smithi119.stdout:0/812: fsync d2/d6/d9/d12/d4f/f89 0 2024-05-27T09:29:35.807 INFO:tasks.workunit.client.0.smithi119.stdout:5/914: getdents d6/db/de 0 2024-05-27T09:29:35.810 INFO:tasks.workunit.client.1.smithi138.stdout:9/340: rmdir d6/da 39 2024-05-27T09:29:35.811 INFO:tasks.workunit.client.1.smithi138.stdout:9/341: truncate d6/da/d26/f4f 84470 0 2024-05-27T09:29:35.811 INFO:tasks.workunit.client.1.smithi138.stdout:9/342: write d6/db/de/d18/f64 [814932,19152] 0 2024-05-27T09:29:35.811 INFO:tasks.workunit.client.1.smithi138.stdout:9/343: chown f1 322 1 2024-05-27T09:29:35.811 INFO:tasks.workunit.client.1.smithi138.stdout:9/344: readlink d6/db/dc/l1d 0 2024-05-27T09:29:35.814 INFO:tasks.workunit.client.0.smithi119.stdout:0/813: mkdir d2/d6/d9/d24/d30/dcc/de3/d10a 0 2024-05-27T09:29:35.814 INFO:tasks.workunit.client.0.smithi119.stdout:0/814: creat d2/d6/d9/d12/f10b x:0 0 0 2024-05-27T09:29:35.814 INFO:tasks.workunit.client.0.smithi119.stdout:0/815: dread - d2/d6/ffc zero size 2024-05-27T09:29:35.815 INFO:tasks.workunit.client.0.smithi119.stdout:0/816: write d2/d6/d9/d24/d30/d4a/fc5 [1108526,94330] 0 2024-05-27T09:29:35.815 INFO:tasks.workunit.client.0.smithi119.stdout:0/817: getdents d2/d6/d9/d24/d30/dcc/d109 0 2024-05-27T09:29:35.816 INFO:tasks.workunit.client.1.smithi138.stdout:7/285: write d0/f32 [1559942,78090] 0 2024-05-27T09:29:35.817 INFO:tasks.workunit.client.1.smithi138.stdout:3/300: dwrite d0/d2/f3d [0,4194304] 0 2024-05-27T09:29:35.817 INFO:tasks.workunit.client.1.smithi138.stdout:3/301: truncate d0/d2/da/f2d 1926027 0 2024-05-27T09:29:35.817 INFO:tasks.workunit.client.1.smithi138.stdout:3/302: fsync d0/d2/da/df/d10/f38 0 2024-05-27T09:29:35.818 INFO:tasks.workunit.client.0.smithi119.stdout:7/892: dread d7/df/d19/d8d/fa3 [0,4194304] 0 2024-05-27T09:29:35.827 INFO:tasks.workunit.client.1.smithi138.stdout:4/384: dwrite d7/d6e/f81 [0,4194304] 0 2024-05-27T09:29:35.827 INFO:tasks.workunit.client.1.smithi138.stdout:4/385: fdatasync d7/f42 0 2024-05-27T09:29:35.827 INFO:tasks.workunit.client.1.smithi138.stdout:7/286: rmdir d0/d7/d43 39 2024-05-27T09:29:35.827 INFO:tasks.workunit.client.1.smithi138.stdout:7/287: chown d0/d23/f47 7 1 2024-05-27T09:29:35.828 INFO:tasks.workunit.client.0.smithi119.stdout:0/818: creat d2/d6/d23/f10c x:0 0 0 2024-05-27T09:29:35.828 INFO:tasks.workunit.client.0.smithi119.stdout:0/819: dread - d2/d6/d9/d12/d4b/fa7 zero size 2024-05-27T09:29:35.828 INFO:tasks.workunit.client.0.smithi119.stdout:0/820: readlink d2/d6/d9/d2f/la6 0 2024-05-27T09:29:35.828 INFO:tasks.workunit.client.0.smithi119.stdout:0/821: creat d2/d6/d75/f10d x:0 0 0 2024-05-27T09:29:35.834 INFO:tasks.workunit.client.0.smithi119.stdout:4/900: dwrite d4/d6/d28/d2f/db8/f10d [0,4194304] 0 2024-05-27T09:29:35.841 INFO:tasks.workunit.client.0.smithi119.stdout:0/822: rename d2/d6/d9/cc7 to d2/d6/d9/d12/d4b/d59/dd7/d105/c10e 0 2024-05-27T09:29:35.842 INFO:tasks.workunit.client.0.smithi119.stdout:0/823: fsync d2/d6/d23/d7a/fd4 0 2024-05-27T09:29:35.842 INFO:tasks.workunit.client.0.smithi119.stdout:0/824: chown d2/d6/d9/d24/d69/l7b 4500849 1 2024-05-27T09:29:35.843 INFO:tasks.workunit.client.1.smithi138.stdout:3/303: unlink d0/d2/da/df/d5e/f64 0 2024-05-27T09:29:35.843 INFO:tasks.workunit.client.1.smithi138.stdout:3/304: dread - d0/f62 zero size 2024-05-27T09:29:35.843 INFO:tasks.workunit.client.0.smithi119.stdout:0/825: symlink d2/d1a/l10f 0 2024-05-27T09:29:35.845 INFO:tasks.workunit.client.0.smithi119.stdout:4/901: rename d4/l44 to d4/d23/d24/d5b/da8/l122 0 2024-05-27T09:29:35.847 INFO:tasks.workunit.client.1.smithi138.stdout:7/288: rename d0/d23/d3a/l3e to d0/d23/d59/l5c 0 2024-05-27T09:29:35.849 INFO:tasks.workunit.client.0.smithi119.stdout:4/902: getdents d4/d6/d3e/d89/db2/def 0 2024-05-27T09:29:35.851 INFO:tasks.workunit.client.1.smithi138.stdout:2/255: dwrite d1/d15/f4f [0,4194304] 0 2024-05-27T09:29:35.851 INFO:tasks.workunit.client.1.smithi138.stdout:2/256: dread - d1/d15/d2d/f3e zero size 2024-05-27T09:29:35.853 INFO:tasks.workunit.client.1.smithi138.stdout:3/305: dread d0/d2/da/df/d10/f4a [0,4194304] 0 2024-05-27T09:29:35.855 INFO:tasks.workunit.client.0.smithi119.stdout:3/952: dwrite db/d36/d45/f118 [0,4194304] 0 2024-05-27T09:29:35.855 INFO:tasks.workunit.client.1.smithi138.stdout:7/289: rename d0/d7/f3b to d0/d23/f5d 0 2024-05-27T09:29:35.859 INFO:tasks.workunit.client.1.smithi138.stdout:4/386: write d7/f18 [848778,91564] 0 2024-05-27T09:29:35.860 INFO:tasks.workunit.client.1.smithi138.stdout:6/438: dwrite d3/d11/f2e [4194304,4194304] 0 2024-05-27T09:29:35.861 INFO:tasks.workunit.client.1.smithi138.stdout:3/306: creat d0/d2/da/df/d10/d32/d37/f65 x:0 0 0 2024-05-27T09:29:35.861 INFO:tasks.workunit.client.1.smithi138.stdout:3/307: fdatasync d0/d2/f4b 0 2024-05-27T09:29:35.861 INFO:tasks.workunit.client.0.smithi119.stdout:9/717: dwrite d18/d19/d1b/f4c [0,4194304] 0 2024-05-27T09:29:35.865 INFO:tasks.workunit.client.1.smithi138.stdout:7/290: creat d0/d2b/f5e x:0 0 0 2024-05-27T09:29:35.866 INFO:tasks.workunit.client.1.smithi138.stdout:7/291: creat d0/d2b/f5f x:0 0 0 2024-05-27T09:29:35.872 INFO:tasks.workunit.client.1.smithi138.stdout:7/292: mkdir d0/d7/d60 0 2024-05-27T09:29:35.874 INFO:tasks.workunit.client.0.smithi119.stdout:1/888: dwrite d7/db/d52/fac [0,4194304] 0 2024-05-27T09:29:35.874 INFO:tasks.workunit.client.0.smithi119.stdout:1/889: truncate d7/dd/d44/d47/f72 1915335 0 2024-05-27T09:29:35.874 INFO:tasks.workunit.client.0.smithi119.stdout:1/890: dread - d7/db/d8f/d9b/dce/f10c zero size 2024-05-27T09:29:35.874 INFO:tasks.workunit.client.0.smithi119.stdout:1/891: read - d7/dd/d44/f100 zero size 2024-05-27T09:29:35.875 INFO:tasks.workunit.client.1.smithi138.stdout:7/293: symlink d0/l61 0 2024-05-27T09:29:35.875 INFO:tasks.workunit.client.1.smithi138.stdout:7/294: write d0/d2b/f5f [416230,116666] 0 2024-05-27T09:29:35.880 INFO:tasks.workunit.client.1.smithi138.stdout:7/295: mknod d0/d7/c62 0 2024-05-27T09:29:35.880 INFO:tasks.workunit.client.1.smithi138.stdout:7/296: truncate d0/d23/f4f 122736 0 2024-05-27T09:29:35.881 INFO:tasks.workunit.client.1.smithi138.stdout:8/299: dwrite d3/d8/d32/d45/f4e [0,4194304] 0 2024-05-27T09:29:35.881 INFO:tasks.workunit.client.1.smithi138.stdout:8/300: read - d3/d8/f33 zero size 2024-05-27T09:29:35.883 INFO:tasks.workunit.client.1.smithi138.stdout:4/387: rename d7/d27/c65 to d7/d27/d3a/c82 0 2024-05-27T09:29:35.884 INFO:tasks.workunit.client.1.smithi138.stdout:4/388: fsync d7/d27/d3a/f55 0 2024-05-27T09:29:35.885 INFO:tasks.workunit.client.1.smithi138.stdout:7/297: unlink d0/d23/c41 0 2024-05-27T09:29:35.885 INFO:tasks.workunit.client.1.smithi138.stdout:7/298: write d0/d23/f4f [762600,65141] 0 2024-05-27T09:29:35.889 INFO:tasks.workunit.client.1.smithi138.stdout:6/439: creat d3/d91/fa0 x:0 0 0 2024-05-27T09:29:35.891 INFO:tasks.workunit.client.1.smithi138.stdout:8/301: mkdir d3/d8/d32/d45/d5c 0 2024-05-27T09:29:35.892 INFO:tasks.workunit.client.1.smithi138.stdout:3/308: dread d0/d2/da/df/d10/d2a/f42 [0,4194304] 0 2024-05-27T09:29:35.892 INFO:tasks.workunit.client.1.smithi138.stdout:3/309: truncate d0/d3f/d50/f63 608076 0 2024-05-27T09:29:35.893 INFO:tasks.workunit.client.1.smithi138.stdout:3/310: read - d0/d2/da/df/d10/d2a/f56 zero size 2024-05-27T09:29:35.893 INFO:tasks.workunit.client.0.smithi119.stdout:1/892: truncate d7/db/d12/d15/d1f/f8b 20751 0 2024-05-27T09:29:35.896 INFO:tasks.workunit.client.0.smithi119.stdout:3/953: dread db/dc9/f79 [0,4194304] 0 2024-05-27T09:29:35.896 INFO:tasks.workunit.client.0.smithi119.stdout:9/718: dread d18/d19/d1b/f5d [0,4194304] 0 2024-05-27T09:29:35.896 INFO:tasks.workunit.client.0.smithi119.stdout:3/954: creat db/d36/d45/d4c/d120/f13a x:0 0 0 2024-05-27T09:29:35.898 INFO:tasks.workunit.client.1.smithi138.stdout:9/345: dwrite d6/db/dc/f50 [0,4194304] 0 2024-05-27T09:29:35.899 INFO:tasks.workunit.client.0.smithi119.stdout:1/893: mknod d7/db/d52/c114 0 2024-05-27T09:29:35.901 INFO:tasks.workunit.client.1.smithi138.stdout:6/440: link d3/d35/d43/d1c/d32/d47/f7a d3/d35/d43/d1c/d32/fa1 0 2024-05-27T09:29:35.901 INFO:tasks.workunit.client.1.smithi138.stdout:6/441: write d3/d11/f2e [710767,114942] 0 2024-05-27T09:29:35.901 INFO:tasks.workunit.client.1.smithi138.stdout:6/442: write d3/d35/d5a/d6a/f7f [302816,105955] 0 2024-05-27T09:29:35.906 INFO:tasks.workunit.client.0.smithi119.stdout:7/893: write d7/df/d19/d4d/f71 [1638553,105297] 0 2024-05-27T09:29:35.911 INFO:tasks.workunit.client.1.smithi138.stdout:6/443: symlink d3/d35/d43/d8c/la2 0 2024-05-27T09:29:35.911 INFO:tasks.workunit.client.1.smithi138.stdout:6/444: dread - d3/d11/d33/f78 zero size 2024-05-27T09:29:35.914 INFO:tasks.workunit.client.1.smithi138.stdout:0/321: dwrite f5 [0,4194304] 0 2024-05-27T09:29:35.914 INFO:tasks.workunit.client.1.smithi138.stdout:0/322: fsync da/d17/d1d/d1f/f21 0 2024-05-27T09:29:35.918 INFO:tasks.workunit.client.0.smithi119.stdout:9/719: getdents d18/d19/d1b/d29 0 2024-05-27T09:29:35.918 INFO:tasks.workunit.client.0.smithi119.stdout:9/720: chown d18/d19/d1b/d40/l52 62877066 1 2024-05-27T09:29:35.918 INFO:tasks.workunit.client.0.smithi119.stdout:7/894: symlink d7/d1a/d107/d111/l11b 0 2024-05-27T09:29:35.918 INFO:tasks.workunit.client.0.smithi119.stdout:7/895: write d7/df/d19/f14 [1313962,17120] 0 2024-05-27T09:29:35.919 INFO:tasks.workunit.client.1.smithi138.stdout:8/302: rename d3/d8/d11/f51 to d3/d8/d32/d34/d38/f5d 0 2024-05-27T09:29:35.919 INFO:tasks.workunit.client.1.smithi138.stdout:8/303: creat d3/d8/d28/f5e x:0 0 0 2024-05-27T09:29:35.919 INFO:tasks.workunit.client.1.smithi138.stdout:8/304: stat d3/d8/d32/d45/f4e 0 2024-05-27T09:29:35.919 INFO:tasks.workunit.client.1.smithi138.stdout:8/305: stat d3 0 2024-05-27T09:29:35.919 INFO:tasks.workunit.client.1.smithi138.stdout:8/306: fdatasync d3/d8/d28/f2f 0 2024-05-27T09:29:35.925 INFO:tasks.workunit.client.0.smithi119.stdout:0/826: rename d2/d6/d9/d12/d4b to d2/d110 0 2024-05-27T09:29:35.925 INFO:tasks.workunit.client.0.smithi119.stdout:0/827: creat d2/d6/d23/d7a/d53/dd1/f111 x:0 0 0 2024-05-27T09:29:35.928 INFO:tasks.workunit.client.1.smithi138.stdout:9/346: creat d6/da/d2f/d35/d55/f6c x:0 0 0 2024-05-27T09:29:35.929 INFO:tasks.workunit.client.1.smithi138.stdout:3/311: dread d0/d2/da/df/d10/d32/f28 [0,4194304] 0 2024-05-27T09:29:35.930 INFO:tasks.workunit.client.1.smithi138.stdout:3/312: dread - d0/d3f/d50/f5c zero size 2024-05-27T09:29:35.930 INFO:tasks.workunit.client.1.smithi138.stdout:7/299: dread d0/f19 [0,4194304] 0 2024-05-27T09:29:35.930 INFO:tasks.workunit.client.0.smithi119.stdout:7/896: mkdir d7/d6a/d7b/d11c 0 2024-05-27T09:29:35.930 INFO:tasks.workunit.client.0.smithi119.stdout:5/915: dwrite d6/dd/d6e/f118 [0,4194304] 0 2024-05-27T09:29:35.932 INFO:tasks.workunit.client.1.smithi138.stdout:8/307: mknod d3/d8/d32/d34/d38/c5f 0 2024-05-27T09:29:35.933 INFO:tasks.workunit.client.0.smithi119.stdout:4/903: dwrite d4/d6/fe0 [0,4194304] 0 2024-05-27T09:29:35.933 INFO:tasks.workunit.client.0.smithi119.stdout:4/904: creat d4/d6/d81/d10b/f123 x:0 0 0 2024-05-27T09:29:35.935 INFO:tasks.workunit.client.1.smithi138.stdout:6/445: dread d3/d19/f34 [0,4194304] 0 2024-05-27T09:29:35.936 INFO:tasks.workunit.client.1.smithi138.stdout:0/323: dread da/d17/d1d/d1f/f3b [0,4194304] 0 2024-05-27T09:29:35.936 INFO:tasks.workunit.client.1.smithi138.stdout:0/324: truncate da/d2a/d3f/f59 438940 0 2024-05-27T09:29:35.939 INFO:tasks.workunit.client.1.smithi138.stdout:0/325: rename da/d2a/d55/f5f to da/d17/d1d/d1f/d42/d4a/d53/f72 0 2024-05-27T09:29:35.940 INFO:tasks.workunit.client.1.smithi138.stdout:9/347: chown d6/da/d29/l60 114762 1 2024-05-27T09:29:35.941 INFO:tasks.workunit.client.1.smithi138.stdout:9/348: dread - d6/db/f6a zero size 2024-05-27T09:29:35.941 INFO:tasks.workunit.client.1.smithi138.stdout:9/349: dread - d6/da/d2f/f5f zero size 2024-05-27T09:29:35.941 INFO:tasks.workunit.client.1.smithi138.stdout:9/350: chown d6/da/d2f/l67 0 1 2024-05-27T09:29:35.941 INFO:tasks.workunit.client.1.smithi138.stdout:9/351: chown d6/da/d2f/d35/d55/f6c 1108245 1 2024-05-27T09:29:35.941 INFO:tasks.workunit.client.1.smithi138.stdout:6/446: write d3/d19/f38 [91179,89088] 0 2024-05-27T09:29:35.946 INFO:tasks.workunit.client.1.smithi138.stdout:8/308: mkdir d3/d60 0 2024-05-27T09:29:35.949 INFO:tasks.workunit.client.0.smithi119.stdout:5/916: truncate d6/dd/d6e/fa4 1194488 0 2024-05-27T09:29:35.949 INFO:tasks.workunit.client.0.smithi119.stdout:4/905: creat d4/d6/d28/d2f/d36/f124 x:0 0 0 2024-05-27T09:29:35.949 INFO:tasks.workunit.client.0.smithi119.stdout:4/906: chown d4/db1/cd7 6421719 1 2024-05-27T09:29:35.949 INFO:tasks.workunit.client.0.smithi119.stdout:5/917: truncate d6/dd/d39/fd1 53194 0 2024-05-27T09:29:35.952 INFO:tasks.workunit.client.1.smithi138.stdout:2/257: dwrite d1/d15/d2d/d17/d25/f28 [0,4194304] 0 2024-05-27T09:29:35.952 INFO:tasks.workunit.client.1.smithi138.stdout:8/309: rename d3/d8/d11/f48 to d3/d8/d24/d4d/f61 0 2024-05-27T09:29:35.952 INFO:tasks.workunit.client.1.smithi138.stdout:2/258: dread - d1/d15/d2d/d31/d4b/f4c zero size 2024-05-27T09:29:35.955 INFO:tasks.workunit.client.1.smithi138.stdout:9/352: link d6/f2e d6/da/d29/f6d 0 2024-05-27T09:29:35.955 INFO:tasks.workunit.client.0.smithi119.stdout:2/857: dwrite d2/da/d8a/dd4/d89/d94/f73 [0,4194304] 0 2024-05-27T09:29:35.957 INFO:tasks.workunit.client.1.smithi138.stdout:2/259: creat d1/d15/d2d/d17/f58 x:0 0 0 2024-05-27T09:29:35.957 INFO:tasks.workunit.client.0.smithi119.stdout:0/828: getdents d2/d6/d23/d7a/d53/dd1 0 2024-05-27T09:29:35.962 INFO:tasks.workunit.client.1.smithi138.stdout:9/353: creat d6/da/d29/d59/f6e x:0 0 0 2024-05-27T09:29:35.965 INFO:tasks.workunit.client.1.smithi138.stdout:6/447: dread d3/f17 [0,4194304] 0 2024-05-27T09:29:35.966 INFO:tasks.workunit.client.1.smithi138.stdout:9/354: mknod d6/db/de/d18/d24/c6f 0 2024-05-27T09:29:35.966 INFO:tasks.workunit.client.1.smithi138.stdout:9/355: readlink d6/db/de/d18/l23 0 2024-05-27T09:29:35.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.969+0000 7fd295c00700 1 -- 172.21.15.119:0/1175122328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd290074d80 msgr2=0x7fd2900731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:35.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.969+0000 7fd295c00700 1 --2- 172.21.15.119:0/1175122328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd290074d80 0x7fd2900731e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd280009910 tx=0x7fd280009c20 comp rx=0 tx=0).stop 2024-05-27T09:29:35.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.969+0000 7fd295c00700 1 -- 172.21.15.119:0/1175122328 shutdown_connections 2024-05-27T09:29:35.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.969+0000 7fd295c00700 1 --2- 172.21.15.119:0/1175122328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2900737b0 0x7fd290073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:35.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.969+0000 7fd295c00700 1 --2- 172.21.15.119:0/1175122328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd290074d80 0x7fd2900731e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:35.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.969+0000 7fd295c00700 1 -- 172.21.15.119:0/1175122328 >> 172.21.15.119:0/1175122328 conn(0x7fd2900fb6e0 msgr2=0x7fd2900fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:35.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.970+0000 7fd295c00700 1 -- 172.21.15.119:0/1175122328 shutdown_connections 2024-05-27T09:29:35.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.970+0000 7fd295c00700 1 -- 172.21.15.119:0/1175122328 wait complete. 2024-05-27T09:29:35.971 INFO:tasks.workunit.client.1.smithi138.stdout:9/356: rename d6/f3b to d6/da/d29/d59/f70 0 2024-05-27T09:29:35.971 INFO:tasks.workunit.client.1.smithi138.stdout:9/357: dread - d6/db/dc/f4d zero size 2024-05-27T09:29:35.971 INFO:tasks.workunit.client.1.smithi138.stdout:9/358: readlink d6/da/d2f/l67 0 2024-05-27T09:29:35.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.972+0000 7fd295c00700 1 Processor -- start 2024-05-27T09:29:35.974 INFO:tasks.workunit.client.0.smithi119.stdout:5/918: symlink d6/dd/d46/l141 0 2024-05-27T09:29:35.974 INFO:tasks.workunit.client.0.smithi119.stdout:5/919: truncate d6/dd/d39/d19/d70/dc1/d41/f98 873645 0 2024-05-27T09:29:35.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.973+0000 7fd295c00700 1 -- start start 2024-05-27T09:29:35.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.973+0000 7fd295c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2900737b0 0x7fd29019bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:35.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.973+0000 7fd295c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd290074d80 0x7fd29019c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:35.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.973+0000 7fd295c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29019ca90 con 0x7fd2900737b0 2024-05-27T09:29:35.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.973+0000 7fd295c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29019cbd0 con 0x7fd290074d80 2024-05-27T09:29:35.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.974+0000 7fd28e000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd290074d80 0x7fd29019c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:35.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.974+0000 7fd28e000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd290074d80 0x7fd29019c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.119:45836/0 (socket says 172.21.15.119:45836) 2024-05-27T09:29:35.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.974+0000 7fd28e000700 1 -- 172.21.15.119:0/3626118503 learned_addr learned my addr 172.21.15.119:0/3626118503 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:29:35.975 INFO:tasks.workunit.client.1.smithi138.stdout:7/300: dread d0/d23/d3a/f46 [0,4194304] 0 2024-05-27T09:29:35.976 INFO:tasks.workunit.client.0.smithi119.stdout:4/907: dread d4/d6/d3e/faf [0,4194304] 0 2024-05-27T09:29:35.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.975+0000 7fd28ea00700 1 --2- 172.21.15.119:0/3626118503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2900737b0 0x7fd29019bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:35.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.976+0000 7fd28ea00700 1 -- 172.21.15.119:0/3626118503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd290074d80 msgr2=0x7fd29019c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:35.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.976+0000 7fd28ea00700 1 --2- 172.21.15.119:0/3626118503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd290074d80 0x7fd29019c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:35.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.976+0000 7fd28ea00700 1 -- 172.21.15.119:0/3626118503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2800095f0 con 0x7fd2900737b0 2024-05-27T09:29:35.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.977+0000 7fd28ea00700 1 --2- 172.21.15.119:0/3626118503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2900737b0 0x7fd29019bf30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd280009910 tx=0x7fd28000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:35.980 INFO:tasks.workunit.client.0.smithi119.stdout:4/908: creat d4/d23/d24/d5b/da8/dda/f125 x:0 0 0 2024-05-27T09:29:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.980+0000 7fd287400700 1 -- 172.21.15.119:0/3626118503 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd28001d070 con 0x7fd2900737b0 2024-05-27T09:29:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.980+0000 7fd287400700 1 -- 172.21.15.119:0/3626118503 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd280005dc0 con 0x7fd2900737b0 2024-05-27T09:29:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.980+0000 7fd287400700 1 -- 172.21.15.119:0/3626118503 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd280021410 con 0x7fd2900737b0 2024-05-27T09:29:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.980+0000 7fd295c00700 1 -- 172.21.15.119:0/3626118503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd290104060 con 0x7fd2900737b0 2024-05-27T09:29:35.982 INFO:tasks.workunit.client.1.smithi138.stdout:7/301: creat d0/d7/d60/f63 x:0 0 0 2024-05-27T09:29:35.982 INFO:tasks.workunit.client.1.smithi138.stdout:7/302: chown d0/l2d 546 1 2024-05-27T09:29:35.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.980+0000 7fd295c00700 1 -- 172.21.15.119:0/3626118503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd290104550 con 0x7fd2900737b0 2024-05-27T09:29:35.982 INFO:tasks.workunit.client.0.smithi119.stdout:2/858: creat d2/da/d8a/dd4/d69/df5/d108/f126 x:0 0 0 2024-05-27T09:29:35.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.983+0000 7fd295c00700 1 -- 172.21.15.119:0/3626118503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd29010f280 con 0x7fd2900737b0 2024-05-27T09:29:35.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.985+0000 7fd287400700 1 -- 172.21.15.119:0/3626118503 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) v1 ==== 95024+0+0 (secure 0 0 0) 0x7fd28000f7d0 con 0x7fd2900737b0 2024-05-27T09:29:35.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.987+0000 7fd287400700 1 --2- 172.21.15.119:0/3626118503 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7fd274072810 0x7fd274074cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:35.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.987+0000 7fd287400700 1 -- 172.21.15.119:0/3626118503 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fd280094460 con 0x7fd2900737b0 2024-05-27T09:29:35.989 INFO:tasks.workunit.client.1.smithi138.stdout:6/448: dread d3/d11/d2d/d37/f4a [0,4194304] 0 2024-05-27T09:29:35.989 INFO:tasks.workunit.client.0.smithi119.stdout:5/920: mknod d6/d2f/dfd/c142 0 2024-05-27T09:29:35.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.991+0000 7fd28e000700 1 --2- 172.21.15.119:0/3626118503 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7fd274072810 0x7fd274074cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:35.994 INFO:tasks.workunit.client.0.smithi119.stdout:3/955: dwrite db/f69 [4194304,4194304] 0 2024-05-27T09:29:35.996 INFO:tasks.workunit.client.0.smithi119.stdout:2/859: symlink d2/da/d15/d53/dac/d122/l127 0 2024-05-27T09:29:35.998 INFO:tasks.workunit.client.1.smithi138.stdout:9/359: write d6/f2e [1627554,6358] 0 2024-05-27T09:29:35.999 INFO:tasks.workunit.client.0.smithi119.stdout:5/921: rename d6/db/de/l34 to d6/dd/d46/l143 0 2024-05-27T09:29:36.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:35.999+0000 7fd28e000700 1 --2- 172.21.15.119:0/3626118503 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7fd274072810 0x7fd274074cc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd2880094f0 tx=0x7fd288009040 comp rx=0 tx=0).ready entity=mgr.14570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:36.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.000+0000 7fd287400700 1 -- 172.21.15.119:0/3626118503 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7fd28005c4f0 con 0x7fd2900737b0 2024-05-27T09:29:36.001 INFO:tasks.workunit.client.0.smithi119.stdout:7/897: dwrite d7/d1a/d99/fee [4194304,4194304] 0 2024-05-27T09:29:36.001 INFO:tasks.workunit.client.0.smithi119.stdout:7/898: fsync d7/d1a/f2d 0 2024-05-27T09:29:36.007 INFO:tasks.workunit.client.1.smithi138.stdout:7/303: dread d0/f1d [0,4194304] 0 2024-05-27T09:29:36.012 INFO:tasks.workunit.client.0.smithi119.stdout:3/956: symlink db/dc9/d98/d105/l13b 0 2024-05-27T09:29:36.015 INFO:tasks.workunit.client.1.smithi138.stdout:9/360: symlink d6/db/de/d18/d24/l71 0 2024-05-27T09:29:36.015 INFO:tasks.workunit.client.0.smithi119.stdout:6/802: sync 2024-05-27T09:29:36.015 INFO:tasks.workunit.client.0.smithi119.stdout:8/798: sync 2024-05-27T09:29:36.021 INFO:tasks.workunit.client.1.smithi138.stdout:9/361: mkdir d6/db/de/d18/d24/d72 0 2024-05-27T09:29:36.021 INFO:tasks.workunit.client.1.smithi138.stdout:9/362: write d6/da/d2f/d35/d55/f6c [764986,31514] 0 2024-05-27T09:29:36.022 INFO:tasks.workunit.client.1.smithi138.stdout:7/304: mkdir d0/d7/d64 0 2024-05-27T09:29:36.022 INFO:tasks.workunit.client.1.smithi138.stdout:7/305: write d0/f24 [9222215,12011] 0 2024-05-27T09:29:36.022 INFO:tasks.workunit.client.1.smithi138.stdout:7/306: fdatasync d0/d23/f48 0 2024-05-27T09:29:36.022 INFO:tasks.workunit.client.1.smithi138.stdout:7/307: dread - d0/d7/d60/f63 zero size 2024-05-27T09:29:36.022 INFO:tasks.workunit.client.1.smithi138.stdout:9/363: creat d6/db/de/d18/d24/f73 x:0 0 0 2024-05-27T09:29:36.026 INFO:tasks.workunit.client.0.smithi119.stdout:5/922: dread d6/db/de/d27/f6b [0,4194304] 0 2024-05-27T09:29:36.026 INFO:tasks.workunit.client.0.smithi119.stdout:5/923: dread - d6/dd/d46/d47/d4b/dd8/d124/d100/f10c zero size 2024-05-27T09:29:36.031 INFO:tasks.workunit.client.1.smithi138.stdout:8/310: dwrite d3/d8/d11/f26 [0,4194304] 0 2024-05-27T09:29:36.036 INFO:tasks.workunit.client.1.smithi138.stdout:9/364: unlink d6/db/de/l20 0 2024-05-27T09:29:36.041 INFO:tasks.workunit.client.0.smithi119.stdout:3/957: read db/d36/d45/d49/d9d/fbb [2143131,106892] 0 2024-05-27T09:29:36.041 INFO:tasks.workunit.client.0.smithi119.stdout:7/899: dread d7/d1a/f59 [0,4194304] 0 2024-05-27T09:29:36.041 INFO:tasks.workunit.client.0.smithi119.stdout:7/900: write d7/f66 [4484996,46790] 0 2024-05-27T09:29:36.042 INFO:tasks.workunit.client.0.smithi119.stdout:7/901: fdatasync d7/d1a/f20 0 2024-05-27T09:29:36.042 INFO:tasks.workunit.client.1.smithi138.stdout:9/365: unlink d6/db/f39 0 2024-05-27T09:29:36.042 INFO:tasks.workunit.client.1.smithi138.stdout:9/366: fsync d6/da/d2f/d35/d55/f3a 0 2024-05-27T09:29:36.045 INFO:tasks.workunit.client.0.smithi119.stdout:3/958: rename db/dc9/d98/f73 to db/d36/d45/d4c/d62/d13/f13c 0 2024-05-27T09:29:36.045 INFO:tasks.workunit.client.0.smithi119.stdout:3/959: readlink db/d5b/dce/de5/lfa 0 2024-05-27T09:29:36.048 INFO:tasks.workunit.client.0.smithi119.stdout:7/902: rmdir d7/df/d19/d4d/d89/deb 39 2024-05-27T09:29:36.049 INFO:tasks.workunit.client.0.smithi119.stdout:7/903: read d7/df/d19/d4d/d89/fe8 [2222976,119742] 0 2024-05-27T09:29:36.049 INFO:tasks.workunit.client.0.smithi119.stdout:7/904: write d7/d1a/fa1 [4291628,116250] 0 2024-05-27T09:29:36.049 INFO:tasks.workunit.client.0.smithi119.stdout:7/905: stat d7/f66 0 2024-05-27T09:29:36.051 INFO:tasks.workunit.client.0.smithi119.stdout:3/960: rmdir db/d36/d45/d4c/d62/d13/d33/d9b/de3 39 2024-05-27T09:29:36.055 INFO:tasks.workunit.client.0.smithi119.stdout:7/906: unlink d7/df/d19/d97/d92/fd1 0 2024-05-27T09:29:36.055 INFO:tasks.workunit.client.0.smithi119.stdout:7/907: stat d7/df/d19/d97/d92/db7/dba/f61 0 2024-05-27T09:29:36.056 INFO:tasks.workunit.client.1.smithi138.stdout:8/311: dread d3/d8/d32/f49 [0,4194304] 0 2024-05-27T09:29:36.056 INFO:tasks.workunit.client.1.smithi138.stdout:3/313: dwrite d0/d2/da/f2d [0,4194304] 0 2024-05-27T09:29:36.058 INFO:tasks.workunit.client.1.smithi138.stdout:2/260: dwrite d1/d15/d2d/f39 [0,4194304] 0 2024-05-27T09:29:36.058 INFO:tasks.workunit.client.1.smithi138.stdout:2/261: write d1/d15/d2d/d1d/f45 [424421,128926] 0 2024-05-27T09:29:36.058 INFO:tasks.workunit.client.1.smithi138.stdout:2/262: chown d1/d15/d2d/l20 79 1 2024-05-27T09:29:36.059 INFO:tasks.workunit.client.0.smithi119.stdout:7/908: creat d7/d1a/d99/dbb/dc2/d102/f11d x:0 0 0 2024-05-27T09:29:36.064 INFO:tasks.workunit.client.0.smithi119.stdout:5/924: write d6/dd/d39/d19/fb5 [4013364,52602] 0 2024-05-27T09:29:36.064 INFO:tasks.workunit.client.0.smithi119.stdout:5/925: chown d6/d2f/ce3 27 1 2024-05-27T09:29:36.066 INFO:tasks.workunit.client.0.smithi119.stdout:5/926: creat d6/dd/d46/d47/d4b/dd8/f144 x:0 0 0 2024-05-27T09:29:36.067 INFO:tasks.workunit.client.1.smithi138.stdout:3/314: mknod d0/c66 0 2024-05-27T09:29:36.069 INFO:tasks.workunit.client.0.smithi119.stdout:5/927: unlink d6/dd/d46/d47/d4b/c60 0 2024-05-27T09:29:36.069 INFO:tasks.workunit.client.0.smithi119.stdout:5/928: read - d6/dd/d46/d47/d4b/dd8/d124/d100/f10c zero size 2024-05-27T09:29:36.071 INFO:tasks.workunit.client.0.smithi119.stdout:5/929: stat d6/db/de/d27/d77/fbc 0 2024-05-27T09:29:36.074 INFO:tasks.workunit.client.1.smithi138.stdout:3/315: symlink d0/d2/da/l67 0 2024-05-27T09:29:36.081 INFO:tasks.workunit.client.1.smithi138.stdout:3/316: getdents d0/d2/da/df/d10 0 2024-05-27T09:29:36.086 INFO:tasks.workunit.client.1.smithi138.stdout:6/449: dwrite d3/d35/d5a/d6a/f6f [0,4194304] 0 2024-05-27T09:29:36.086 INFO:tasks.workunit.client.1.smithi138.stdout:6/450: write d3/d35/f60 [1102424,42190] 0 2024-05-27T09:29:36.089 INFO:tasks.workunit.client.1.smithi138.stdout:6/451: mknod d3/d35/d43/d1c/d32/ca3 0 2024-05-27T09:29:36.090 INFO:tasks.workunit.client.1.smithi138.stdout:3/317: dread d0/d2/da/df/d10/d32/f28 [0,4194304] 0 2024-05-27T09:29:36.090 INFO:tasks.workunit.client.1.smithi138.stdout:3/318: chown d0/d2/da/df/c2c 139778 1 2024-05-27T09:29:36.090 INFO:tasks.workunit.client.1.smithi138.stdout:3/319: readlink d0/d2/da/l2f 0 2024-05-27T09:29:36.091 INFO:tasks.workunit.client.1.smithi138.stdout:6/452: truncate d3/d11/f88 108740 0 2024-05-27T09:29:36.092 INFO:tasks.workunit.client.1.smithi138.stdout:6/453: mknod d3/d11/d2d/d37/ca4 0 2024-05-27T09:29:36.093 INFO:tasks.workunit.client.1.smithi138.stdout:6/454: mkdir d3/d35/d7d/da5 0 2024-05-27T09:29:36.098 INFO:tasks.workunit.client.1.smithi138.stdout:0/326: dwrite da/d17/d1d/d1f/f26 [4194304,4194304] 0 2024-05-27T09:29:36.102 INFO:tasks.workunit.client.1.smithi138.stdout:3/320: rmdir d0/d3f 39 2024-05-27T09:29:36.102 INFO:tasks.workunit.client.1.smithi138.stdout:3/321: chown d0/d2/da/df/c2c 7 1 2024-05-27T09:29:36.103 INFO:tasks.workunit.client.1.smithi138.stdout:6/455: dread d3/f84 [0,4194304] 0 2024-05-27T09:29:36.104 INFO:tasks.workunit.client.1.smithi138.stdout:6/456: creat d3/d35/d43/d8c/fa6 x:0 0 0 2024-05-27T09:29:36.104 INFO:tasks.workunit.client.1.smithi138.stdout:6/457: creat d3/d19/fa7 x:0 0 0 2024-05-27T09:29:36.104 INFO:tasks.workunit.client.1.smithi138.stdout:6/458: write d3/d19/f38 [724153,42488] 0 2024-05-27T09:29:36.104 INFO:tasks.workunit.client.1.smithi138.stdout:6/459: readlink d3/d11/d2d/d37/l40 0 2024-05-27T09:29:36.107 INFO:tasks.workunit.client.1.smithi138.stdout:6/460: truncate d3/d19/f41 2609198 0 2024-05-27T09:29:36.109 INFO:tasks.workunit.client.1.smithi138.stdout:6/461: mkdir d3/d11/d2d/d37/d7e/d8d/da8 0 2024-05-27T09:29:36.110 INFO:tasks.workunit.client.1.smithi138.stdout:3/322: mknod d0/d2/da/c68 0 2024-05-27T09:29:36.122 INFO:tasks.workunit.client.1.smithi138.stdout:3/323: link d0/d2/da/df/d10/d32/d37/l3e d0/d3f/l69 0 2024-05-27T09:29:36.123 INFO:tasks.workunit.client.1.smithi138.stdout:3/324: read - d0/d2/da/df/d10/f5d zero size 2024-05-27T09:29:36.123 INFO:tasks.workunit.client.1.smithi138.stdout:3/325: truncate d0/d2/da/df/d10/f5d 979376 0 2024-05-27T09:29:36.126 INFO:tasks.workunit.client.0.smithi119.stdout:7/909: dread d7/d1a/d99/fee [4194304,4194304] 0 2024-05-27T09:29:36.127 INFO:tasks.workunit.client.1.smithi138.stdout:3/326: link d0/d2/c20 d0/d2/c6a 0 2024-05-27T09:29:36.128 INFO:tasks.workunit.client.1.smithi138.stdout:3/327: stat d0/d2/da/df/d10/d2a/f4f 0 2024-05-27T09:29:36.128 INFO:tasks.workunit.client.1.smithi138.stdout:3/328: fdatasync d0/d2/da/df/d10/d2a/d48/f4d 0 2024-05-27T09:29:36.131 INFO:tasks.workunit.client.0.smithi119.stdout:8/799: dwrite d3/d8/d4d/d58/f89 [0,4194304] 0 2024-05-27T09:29:36.131 INFO:tasks.workunit.client.0.smithi119.stdout:8/800: write d3/d8/d4d/d5d/fa3 [2017,76421] 0 2024-05-27T09:29:36.131 INFO:tasks.workunit.client.0.smithi119.stdout:7/910: readlink d7/df/d19/d9a/da6/dda/d44/d67/l6d 0 2024-05-27T09:29:36.133 INFO:tasks.workunit.client.0.smithi119.stdout:1/894: dwrite d7/db/d12/ddb/d38/f90 [0,4194304] 0 2024-05-27T09:29:36.133 INFO:tasks.workunit.client.0.smithi119.stdout:7/911: mkdir d7/d1a/d99/d112/d11e 0 2024-05-27T09:29:36.135 INFO:tasks.workunit.client.1.smithi138.stdout:3/329: unlink d0/d2/da/df/d10/d32/d37/l3b 0 2024-05-27T09:29:36.135 INFO:tasks.workunit.client.0.smithi119.stdout:7/912: mkdir d7/df/d19/d9a/da6/dda/d11f 0 2024-05-27T09:29:36.135 INFO:tasks.workunit.client.0.smithi119.stdout:3/961: dread db/d7f/dd2/f125 [0,4194304] 0 2024-05-27T09:29:36.137 INFO:tasks.workunit.client.1.smithi138.stdout:2/263: dwrite d1/d15/f48 [4194304,4194304] 0 2024-05-27T09:29:36.139 INFO:tasks.workunit.client.0.smithi119.stdout:1/895: rename d7/db/c10 to d7/db/d8f/d9b/c115 0 2024-05-27T09:29:36.139 INFO:tasks.workunit.client.0.smithi119.stdout:1/896: truncate d7/db/d12/ddb/d38/d3c/ff3 772904 0 2024-05-27T09:29:36.139 INFO:tasks.workunit.client.0.smithi119.stdout:7/913: symlink d7/d1a/d99/dbb/dc2/dfe/l120 0 2024-05-27T09:29:36.142 INFO:tasks.workunit.client.0.smithi119.stdout:1/897: rmdir d7/db/d12/d15/d1f 39 2024-05-27T09:29:36.144 INFO:tasks.workunit.client.1.smithi138.stdout:2/264: unlink d1/d15/d2d/d17/d25/f2b 0 2024-05-27T09:29:36.146 INFO:tasks.workunit.client.0.smithi119.stdout:8/801: dread d3/d8/d4d/f67 [0,4194304] 0 2024-05-27T09:29:36.148 INFO:tasks.workunit.client.0.smithi119.stdout:0/829: dwrite d2/d6/d9/d24/d30/dcc/fd9 [0,4194304] 0 2024-05-27T09:29:36.149 INFO:tasks.workunit.client.0.smithi119.stdout:8/802: mknod d3/d6e/d80/c10a 0 2024-05-27T09:29:36.149 INFO:tasks.workunit.client.0.smithi119.stdout:8/803: chown d3/d8/d4d/d5d/d60/f8b 1645544 1 2024-05-27T09:29:36.149 INFO:tasks.workunit.client.0.smithi119.stdout:8/804: write d3/d8/d4d/d5d/f8a [4350772,120871] 0 2024-05-27T09:29:36.150 INFO:tasks.workunit.client.1.smithi138.stdout:3/330: read d0/d2/f13 [159854,22784] 0 2024-05-27T09:29:36.152 INFO:tasks.workunit.client.0.smithi119.stdout:9/721: dwrite d18/d19/d1b/d29/f71 [0,4194304] 0 2024-05-27T09:29:36.153 INFO:tasks.workunit.client.0.smithi119.stdout:8/805: unlink d3/c6 0 2024-05-27T09:29:36.153 INFO:tasks.workunit.client.0.smithi119.stdout:8/806: truncate d3/d8/d4d/d79/fc7 786662 0 2024-05-27T09:29:36.153 INFO:tasks.workunit.client.0.smithi119.stdout:8/807: dread - d3/d6e/fa0 zero size 2024-05-27T09:29:36.154 INFO:tasks.workunit.client.0.smithi119.stdout:4/909: dwrite d4/d4e/fb7 [0,4194304] 0 2024-05-27T09:29:36.156 INFO:tasks.workunit.client.0.smithi119.stdout:9/722: symlink d18/d19/d1b/d29/ld9 0 2024-05-27T09:29:36.156 INFO:tasks.workunit.client.0.smithi119.stdout:9/723: readlink d18/d19/d1b/d40/d44/l78 0 2024-05-27T09:29:36.161 INFO:tasks.workunit.client.0.smithi119.stdout:4/910: mkdir d4/d23/d24/d5b/da8/d126 0 2024-05-27T09:29:36.162 INFO:tasks.workunit.client.0.smithi119.stdout:4/911: creat d4/d1c/d39/f127 x:0 0 0 2024-05-27T09:29:36.168 INFO:tasks.workunit.client.0.smithi119.stdout:6/803: dwrite d0/d2/d57/f90 [0,4194304] 0 2024-05-27T09:29:36.172 INFO:tasks.workunit.client.0.smithi119.stdout:4/912: rename d4/d6/d81/dce/fa2 to d4/d6/d28/d2f/d36/f128 0 2024-05-27T09:29:36.172 INFO:tasks.workunit.client.0.smithi119.stdout:9/724: symlink d18/d19/d1b/d2a/lda 0 2024-05-27T09:29:36.172 INFO:tasks.workunit.client.0.smithi119.stdout:4/913: creat d4/de/d21/d40/d47/d5c/f129 x:0 0 0 2024-05-27T09:29:36.172 INFO:tasks.workunit.client.0.smithi119.stdout:4/914: chown d4/d6/d28/f6b 449 1 2024-05-27T09:29:36.173 INFO:tasks.workunit.client.1.smithi138.stdout:7/308: dwrite d0/f36 [0,4194304] 0 2024-05-27T09:29:36.175 INFO:tasks.workunit.client.1.smithi138.stdout:8/312: dwrite d3/d8/d32/d34/d38/f5d [0,4194304] 0 2024-05-27T09:29:36.175 INFO:tasks.workunit.client.1.smithi138.stdout:8/313: chown d3/f29 6440 1 2024-05-27T09:29:36.175 INFO:tasks.workunit.client.1.smithi138.stdout:8/314: chown d3/d8/d11 76 1 2024-05-27T09:29:36.176 INFO:tasks.workunit.client.0.smithi119.stdout:6/804: truncate d0/d11/f62 6748411 0 2024-05-27T09:29:36.177 INFO:tasks.workunit.client.1.smithi138.stdout:2/265: read d1/d15/d2d/d2e/f37 [5794719,107064] 0 2024-05-27T09:29:36.177 INFO:tasks.workunit.client.1.smithi138.stdout:2/266: rename d1/d15/d2d/d17 to d1/d15/d2d/d17/d59 22 2024-05-27T09:29:36.178 INFO:tasks.workunit.client.1.smithi138.stdout:7/309: mknod d0/d7/d60/c65 0 2024-05-27T09:29:36.178 INFO:tasks.workunit.client.1.smithi138.stdout:7/310: fsync d0/d7/f56 0 2024-05-27T09:29:36.178 INFO:tasks.workunit.client.1.smithi138.stdout:6/462: dwrite d3/d35/f97 [0,4194304] 0 2024-05-27T09:29:36.182 INFO:tasks.workunit.client.1.smithi138.stdout:2/267: symlink d1/d15/d2d/d31/l5a 0 2024-05-27T09:29:36.182 INFO:tasks.workunit.client.1.smithi138.stdout:8/315: truncate d3/d8/d24/f44 1018189 0 2024-05-27T09:29:36.182 INFO:tasks.workunit.client.1.smithi138.stdout:7/311: symlink d0/d7/d57/l66 0 2024-05-27T09:29:36.182 INFO:tasks.workunit.client.0.smithi119.stdout:4/915: rename d4/d6/d3e/dd2/ffc to d4/d6/d3e/df5/d104/f12a 0 2024-05-27T09:29:36.182 INFO:tasks.workunit.client.0.smithi119.stdout:4/916: write d4/f22 [3515906,126001] 0 2024-05-27T09:29:36.186 INFO:tasks.workunit.client.0.smithi119.stdout:7/914: write d7/fd [378537,17260] 0 2024-05-27T09:29:36.186 INFO:tasks.workunit.client.1.smithi138.stdout:2/268: creat d1/d15/d2d/d31/f5b x:0 0 0 2024-05-27T09:29:36.188 INFO:tasks.workunit.client.1.smithi138.stdout:8/316: truncate d3/d8/f58 1909550 0 2024-05-27T09:29:36.188 INFO:tasks.workunit.client.1.smithi138.stdout:8/317: chown d3/d8/d32 115 1 2024-05-27T09:29:36.192 INFO:tasks.workunit.client.1.smithi138.stdout:7/312: link d0/d23/f48 d0/d7/d43/f67 0 2024-05-27T09:29:36.193 INFO:tasks.workunit.client.0.smithi119.stdout:3/962: dread db/f69 [4194304,4194304] 0 2024-05-27T09:29:36.195 INFO:tasks.workunit.client.1.smithi138.stdout:3/331: truncate d0/d3f/d50/f63 278314 0 2024-05-27T09:29:36.196 INFO:tasks.workunit.client.1.smithi138.stdout:6/463: rename d3/d11/f8a to d3/d35/d43/d1c/fa9 0 2024-05-27T09:29:36.196 INFO:tasks.workunit.client.1.smithi138.stdout:6/464: fsync d3/d19/f63 0 2024-05-27T09:29:36.198 INFO:tasks.workunit.client.0.smithi119.stdout:5/930: dwrite d6/db/de/d27/dc9/fc6 [0,4194304] 0 2024-05-27T09:29:36.198 INFO:tasks.workunit.client.0.smithi119.stdout:5/931: fsync d6/dd/f73 0 2024-05-27T09:29:36.198 INFO:tasks.workunit.client.0.smithi119.stdout:1/898: dread d7/db/d12/feb [0,4194304] 0 2024-05-27T09:29:36.218 INFO:tasks.workunit.client.1.smithi138.stdout:2/269: dread d1/d2f/f33 [0,4194304] 0 2024-05-27T09:29:36.218 INFO:tasks.workunit.client.1.smithi138.stdout:2/270: write d1/d15/d2d/d31/f3b [935401,46476] 0 2024-05-27T09:29:36.221 INFO:tasks.workunit.client.1.smithi138.stdout:2/271: creat d1/d15/d2d/d17/d25/d4d/f5c x:0 0 0 2024-05-27T09:29:36.222 INFO:tasks.workunit.client.0.smithi119.stdout:6/805: rmdir d0/d85/d100 0 2024-05-27T09:29:36.223 INFO:tasks.workunit.client.0.smithi119.stdout:8/808: dwrite d3/d8/d4d/d5d/feb [0,4194304] 0 2024-05-27T09:29:36.223 INFO:tasks.workunit.client.0.smithi119.stdout:8/809: creat d3/d8/d4d/d5d/d66/f10b x:0 0 0 2024-05-27T09:29:36.231 INFO:tasks.workunit.client.0.smithi119.stdout:9/725: write d18/f9a [1376174,109705] 0 2024-05-27T09:29:36.233 INFO:tasks.workunit.client.1.smithi138.stdout:2/272: rmdir d1/d15/d2d/d31 39 2024-05-27T09:29:36.234 INFO:tasks.workunit.client.0.smithi119.stdout:4/917: truncate d4/d6/d3e/faf 285366 0 2024-05-27T09:29:36.234 INFO:tasks.workunit.client.0.smithi119.stdout:7/915: rmdir d7/df/d19/d46 39 2024-05-27T09:29:36.234 INFO:tasks.workunit.client.0.smithi119.stdout:7/916: chown d7/df/d19/d97/d92/db7/dba/d42/d64/c77 1974280 1 2024-05-27T09:29:36.237 INFO:tasks.workunit.client.1.smithi138.stdout:2/273: truncate d1/d15/d2d/d2e/f37 809845 0 2024-05-27T09:29:36.239 INFO:tasks.workunit.client.0.smithi119.stdout:5/932: truncate d6/dd/f67 605655 0 2024-05-27T09:29:36.242 INFO:tasks.workunit.client.1.smithi138.stdout:2/274: unlink d1/d2f/f46 0 2024-05-27T09:29:36.242 INFO:tasks.workunit.client.1.smithi138.stdout:2/275: truncate d1/d15/d2d/d17/d25/f2a 1593819 0 2024-05-27T09:29:36.242 INFO:tasks.workunit.client.0.smithi119.stdout:1/899: symlink d7/db/d12/d26/d4a/l116 0 2024-05-27T09:29:36.253 INFO:tasks.workunit.client.0.smithi119.stdout:9/726: mknod d18/d19/d1b/d2a/d9f/cdb 0 2024-05-27T09:29:36.253 INFO:tasks.workunit.client.0.smithi119.stdout:8/810: creat d3/d8/d4d/d5d/d74/f10c x:0 0 0 2024-05-27T09:29:36.254 INFO:tasks.workunit.client.0.smithi119.stdout:8/811: write d3/d6e/fa0 [1011259,59855] 0 2024-05-27T09:29:36.254 INFO:tasks.workunit.client.0.smithi119.stdout:8/812: creat d3/d39/d5a/f10d x:0 0 0 2024-05-27T09:29:36.254 INFO:tasks.workunit.client.0.smithi119.stdout:8/813: read d3/de/f62 [2303676,21640] 0 2024-05-27T09:29:36.254 INFO:tasks.workunit.client.0.smithi119.stdout:8/814: readlink d3/de/l17 0 2024-05-27T09:29:36.254 INFO:tasks.workunit.client.0.smithi119.stdout:7/917: chown d7/df/d19/d1d/lc7 65 1 2024-05-27T09:29:36.255 INFO:tasks.workunit.client.1.smithi138.stdout:6/465: dread d3/d35/d43/d1c/d32/fa1 [0,4194304] 0 2024-05-27T09:29:36.255 INFO:tasks.workunit.client.1.smithi138.stdout:0/327: dwrite da/d17/d1d/d1f/f3b [0,4194304] 0 2024-05-27T09:29:36.261 INFO:tasks.workunit.client.1.smithi138.stdout:8/318: dread d3/d8/f58 [0,4194304] 0 2024-05-27T09:29:36.269 INFO:tasks.workunit.client.0.smithi119.stdout:0/830: dwrite d2/d110/d59/dd7/f70 [0,4194304] 0 2024-05-27T09:29:36.275 INFO:tasks.workunit.client.0.smithi119.stdout:5/933: truncate d6/dd/d46/d47/d4b/dd8/ddb/fe5 902989 0 2024-05-27T09:29:36.279 INFO:tasks.workunit.client.0.smithi119.stdout:9/727: symlink d18/d19/d9b/ldc 0 2024-05-27T09:29:36.280 INFO:tasks.workunit.client.0.smithi119.stdout:9/728: creat d18/d19/d1b/d40/d44/fdd x:0 0 0 2024-05-27T09:29:36.280 INFO:tasks.workunit.client.1.smithi138.stdout:6/466: mkdir d3/d11/d2d/d37/daa 0 2024-05-27T09:29:36.284 INFO:tasks.workunit.client.1.smithi138.stdout:7/313: dwrite d0/d7/fe [0,4194304] 0 2024-05-27T09:29:36.286 INFO:tasks.workunit.client.1.smithi138.stdout:3/332: dwrite d0/d3f/f55 [0,4194304] 0 2024-05-27T09:29:36.286 INFO:tasks.workunit.client.1.smithi138.stdout:3/333: chown d0/d2/da/df/d10/l44 990835 1 2024-05-27T09:29:36.286 INFO:tasks.workunit.client.1.smithi138.stdout:3/334: chown d0/c4 3128483 1 2024-05-27T09:29:36.287 INFO:tasks.workunit.client.1.smithi138.stdout:7/314: readlink d0/d7/l20 0 2024-05-27T09:29:36.288 INFO:tasks.workunit.client.0.smithi119.stdout:3/963: dwrite db/d36/d45/d93/f128 [0,4194304] 0 2024-05-27T09:29:36.288 INFO:tasks.workunit.client.0.smithi119.stdout:3/964: chown db/d36/d45/fe7 1 1 2024-05-27T09:29:36.288 INFO:tasks.workunit.client.0.smithi119.stdout:3/965: write db/d36/d45/d4d/dc8/ff9 [3418949,99280] 0 2024-05-27T09:29:36.288 INFO:tasks.workunit.client.0.smithi119.stdout:3/966: fsync db/fbf 0 2024-05-27T09:29:36.288 INFO:tasks.workunit.client.0.smithi119.stdout:6/806: dread d0/f68 [4194304,4194304] 0 2024-05-27T09:29:36.288 INFO:tasks.workunit.client.0.smithi119.stdout:6/807: chown d0/d2/d1c/f3e 545696297 1 2024-05-27T09:29:36.289 INFO:tasks.workunit.client.0.smithi119.stdout:6/808: creat d0/d2/d57/d37/d74/d65/f11e x:0 0 0 2024-05-27T09:29:36.289 INFO:tasks.workunit.client.0.smithi119.stdout:6/809: truncate d0/d2/d1c/d21/ffd 241524 0 2024-05-27T09:29:36.289 INFO:tasks.workunit.client.1.smithi138.stdout:0/328: rename da/d17/d1d/d43/c58 to da/d2a/d3f/c73 0 2024-05-27T09:29:36.289 INFO:tasks.workunit.client.1.smithi138.stdout:0/329: chown da/d17/d1d/d1f/c56 2 1 2024-05-27T09:29:36.289 INFO:tasks.workunit.client.1.smithi138.stdout:0/330: chown da/d17/d1d/d1f/d4f 274333598 1 2024-05-27T09:29:36.292 INFO:tasks.workunit.client.1.smithi138.stdout:3/335: creat d0/d2/da/f6b x:0 0 0 2024-05-27T09:29:36.295 INFO:tasks.workunit.client.1.smithi138.stdout:3/336: mknod d0/d2/da/df/d10/d32/c6c 0 2024-05-27T09:29:36.296 INFO:tasks.workunit.client.0.smithi119.stdout:8/815: unlink d3/d8/d4d/d5d/cd4 0 2024-05-27T09:29:36.296 INFO:tasks.workunit.client.0.smithi119.stdout:8/816: stat d3/de 0 2024-05-27T09:29:36.299 INFO:tasks.workunit.client.1.smithi138.stdout:3/337: rename d0/d2/da/c36 to d0/d2/da/df/d10/c6d 0 2024-05-27T09:29:36.300 INFO:tasks.workunit.client.1.smithi138.stdout:0/331: mkdir da/d17/d1d/d43/d74 0 2024-05-27T09:29:36.300 INFO:tasks.workunit.client.1.smithi138.stdout:0/332: chown da/d2a/d2b/c2f 4 1 2024-05-27T09:29:36.300 INFO:tasks.workunit.client.1.smithi138.stdout:0/333: chown da/c13 216 1 2024-05-27T09:29:36.301 INFO:tasks.workunit.client.0.smithi119.stdout:7/918: mkdir d7/d1a/d99/dbb/dc2/d102/d121 0 2024-05-27T09:29:36.303 INFO:tasks.workunit.client.1.smithi138.stdout:3/338: mknod d0/d3f/d60/c6e 0 2024-05-27T09:29:36.305 INFO:tasks.workunit.client.1.smithi138.stdout:6/467: write d3/d19/f22 [1329274,129898] 0 2024-05-27T09:29:36.305 INFO:tasks.workunit.client.1.smithi138.stdout:6/468: creat d3/d19/fab x:0 0 0 2024-05-27T09:29:36.305 INFO:tasks.workunit.client.1.smithi138.stdout:6/469: write d3/d35/d43/d1c/d32/d47/f51 [404912,71577] 0 2024-05-27T09:29:36.313 INFO:tasks.workunit.client.0.smithi119.stdout:5/934: mkdir d6/db/dc/dc3/d145 0 2024-05-27T09:29:36.321 INFO:tasks.workunit.client.0.smithi119.stdout:9/729: rename d18/d19/l6a to d18/d19/d1b/d29/lde 0 2024-05-27T09:29:36.324 INFO:tasks.workunit.client.1.smithi138.stdout:3/339: write d0/d2/f3d [2592711,19503] 0 2024-05-27T09:29:36.325 INFO:tasks.workunit.client.0.smithi119.stdout:1/900: dwrite d7/dd/d25/d60/d61/f7c [0,4194304] 0 2024-05-27T09:29:36.333 INFO:tasks.workunit.client.1.smithi138.stdout:2/276: dwrite d1/d15/d2d/d31/d4b/f51 [0,4194304] 0 2024-05-27T09:29:36.333 INFO:tasks.workunit.client.1.smithi138.stdout:2/277: fdatasync d1/d10/f6 0 2024-05-27T09:29:36.335 INFO:tasks.workunit.client.1.smithi138.stdout:2/278: rmdir d1/d15/d2d/d31 39 2024-05-27T09:29:36.335 INFO:tasks.workunit.client.1.smithi138.stdout:2/279: fsync d1/d15/d2d/f39 0 2024-05-27T09:29:36.339 INFO:tasks.workunit.client.1.smithi138.stdout:6/470: link d3/d11/f88 d3/d35/d5a/d6a/fac 0 2024-05-27T09:29:36.343 INFO:tasks.workunit.client.0.smithi119.stdout:1/901: read d7/dd/f77 [161961,90602] 0 2024-05-27T09:29:36.346 INFO:tasks.workunit.client.1.smithi138.stdout:3/340: dread d0/d2/da/f33 [0,4194304] 0 2024-05-27T09:29:36.346 INFO:tasks.workunit.client.1.smithi138.stdout:0/334: truncate da/d17/d1d/d1f/f3b 115216 0 2024-05-27T09:29:36.346 INFO:tasks.workunit.client.1.smithi138.stdout:5/272: sync 2024-05-27T09:29:36.347 INFO:tasks.workunit.client.1.smithi138.stdout:8/319: dread d3/d8/d32/f49 [0,4194304] 0 2024-05-27T09:29:36.349 INFO:tasks.workunit.client.1.smithi138.stdout:3/341: rename d0/f62 to d0/d2/da/df/f6f 0 2024-05-27T09:29:36.350 INFO:tasks.workunit.client.1.smithi138.stdout:3/342: truncate d0/f12 1178883 0 2024-05-27T09:29:36.350 INFO:tasks.workunit.client.0.smithi119.stdout:4/918: dwrite d4/d6/d81/dce/d88/f112 [0,4194304] 0 2024-05-27T09:29:36.350 INFO:tasks.workunit.client.0.smithi119.stdout:4/919: dread - d4/d6/d28/d2f/d36/dc8/fd0 zero size 2024-05-27T09:29:36.350 INFO:tasks.workunit.client.0.smithi119.stdout:4/920: dread - d4/d6/d81/dce/d88/f11b zero size 2024-05-27T09:29:36.351 INFO:tasks.workunit.client.1.smithi138.stdout:8/320: getdents d3/d8/d32/d34 0 2024-05-27T09:29:36.351 INFO:tasks.workunit.client.1.smithi138.stdout:8/321: write d3/d8/d24/d4d/f61 [1421067,2362] 0 2024-05-27T09:29:36.351 INFO:tasks.workunit.client.1.smithi138.stdout:8/322: dread - d3/d8/d24/d4d/f5a zero size 2024-05-27T09:29:36.353 INFO:tasks.workunit.client.1.smithi138.stdout:3/343: symlink d0/d2/da/df/d10/d2a/d48/l70 0 2024-05-27T09:29:36.356 INFO:tasks.workunit.client.1.smithi138.stdout:6/471: mknod d3/d35/d7d/cad 0 2024-05-27T09:29:36.360 INFO:tasks.workunit.client.1.smithi138.stdout:2/280: read d1/d15/d2d/d17/d25/f28 [1029219,35216] 0 2024-05-27T09:29:36.361 INFO:tasks.workunit.client.1.smithi138.stdout:3/344: rmdir d0/d2/da/df/d10 39 2024-05-27T09:29:36.362 INFO:tasks.workunit.client.1.smithi138.stdout:8/323: symlink d3/d8/d24/l62 0 2024-05-27T09:29:36.363 INFO:tasks.workunit.client.0.smithi119.stdout:8/817: mknod d3/d6e/dcf/dce/c10e 0 2024-05-27T09:29:36.363 INFO:tasks.workunit.client.0.smithi119.stdout:8/818: dread - d3/ff3 zero size 2024-05-27T09:29:36.364 INFO:tasks.workunit.client.1.smithi138.stdout:2/281: mkdir d1/d10/d52/d5d 0 2024-05-27T09:29:36.366 INFO:tasks.workunit.client.1.smithi138.stdout:2/282: mknod d1/d15/d2d/d1d/c5e 0 2024-05-27T09:29:36.367 INFO:tasks.workunit.client.1.smithi138.stdout:2/283: fdatasync d1/f1b 0 2024-05-27T09:29:36.371 INFO:tasks.workunit.client.1.smithi138.stdout:3/345: rmdir d0/d2/da/df/d5e 0 2024-05-27T09:29:36.372 INFO:tasks.workunit.client.1.smithi138.stdout:7/315: dwrite d0/d7/f12 [0,4194304] 0 2024-05-27T09:29:36.373 INFO:tasks.workunit.client.0.smithi119.stdout:4/921: dread d4/d23/d31/f33 [0,4194304] 0 2024-05-27T09:29:36.373 INFO:tasks.workunit.client.0.smithi119.stdout:6/810: dwrite d0/d2/d1c/d21/ff9 [0,4194304] 0 2024-05-27T09:29:36.374 INFO:tasks.workunit.client.1.smithi138.stdout:3/346: truncate d0/d2/da/df/d10/d2a/d48/f5f 708143 0 2024-05-27T09:29:36.375 INFO:tasks.workunit.client.0.smithi119.stdout:5/935: truncate d6/d2f/da5/f111 1903719 0 2024-05-27T09:29:36.376 INFO:tasks.workunit.client.1.smithi138.stdout:3/347: read d0/d2/da/df/d10/f5d [840952,58986] 0 2024-05-27T09:29:36.376 INFO:tasks.workunit.client.1.smithi138.stdout:3/348: creat d0/d2/da/df/d10/d32/f71 x:0 0 0 2024-05-27T09:29:36.377 INFO:tasks.workunit.client.0.smithi119.stdout:7/919: dwrite d7/df/d19/d97/d92/db7/dba/d42/d64/f7a [0,4194304] 0 2024-05-27T09:29:36.377 INFO:tasks.workunit.client.0.smithi119.stdout:7/920: truncate d7/df/d19/d97/d92/db7/dba/d42/fe9 1132672 0 2024-05-27T09:29:36.378 INFO:tasks.workunit.client.1.smithi138.stdout:7/316: symlink d0/l68 0 2024-05-27T09:29:36.378 INFO:tasks.workunit.client.1.smithi138.stdout:7/317: truncate d0/d23/d3c/f42 963842 0 2024-05-27T09:29:36.378 INFO:tasks.workunit.client.1.smithi138.stdout:7/318: readlink d0/d2b/l2c 0 2024-05-27T09:29:36.378 INFO:tasks.workunit.client.1.smithi138.stdout:7/319: chown d0/d23/d3a/f46 1 1 2024-05-27T09:29:36.379 INFO:tasks.workunit.client.1.smithi138.stdout:2/284: dread d1/d15/d2d/d1d/f43 [0,4194304] 0 2024-05-27T09:29:36.381 INFO:tasks.workunit.client.1.smithi138.stdout:0/335: symlink da/d17/d1d/d43/l75 0 2024-05-27T09:29:36.381 INFO:tasks.workunit.client.1.smithi138.stdout:0/336: write da/f10 [4533959,71551] 0 2024-05-27T09:29:36.381 INFO:tasks.workunit.client.1.smithi138.stdout:0/337: fdatasync f1 0 2024-05-27T09:29:36.381 INFO:tasks.workunit.client.1.smithi138.stdout:0/338: write da/f3c [4032163,27222] 0 2024-05-27T09:29:36.381 INFO:tasks.workunit.client.1.smithi138.stdout:3/349: rmdir d0/d2 39 2024-05-27T09:29:36.385 INFO:tasks.workunit.client.1.smithi138.stdout:7/320: rmdir d0 39 2024-05-27T09:29:36.385 INFO:tasks.workunit.client.1.smithi138.stdout:2/285: symlink d1/d15/d2d/d31/l5f 0 2024-05-27T09:29:36.386 INFO:tasks.workunit.client.1.smithi138.stdout:3/350: symlink d0/d2/da/df/d10/d2a/l72 0 2024-05-27T09:29:36.386 INFO:tasks.workunit.client.1.smithi138.stdout:3/351: stat d0/d2/da/c21 0 2024-05-27T09:29:36.386 INFO:tasks.workunit.client.1.smithi138.stdout:3/352: truncate d0/d2/f45 981926 0 2024-05-27T09:29:36.386 INFO:tasks.workunit.client.1.smithi138.stdout:3/353: mkdir d0/d2/da/df/d10/d2a/d61/d73 0 2024-05-27T09:29:36.392 INFO:tasks.workunit.client.1.smithi138.stdout:3/354: rmdir d0/d2 39 2024-05-27T09:29:36.392 INFO:tasks.workunit.client.1.smithi138.stdout:3/355: dread - d0/d2/da/df/f6f zero size 2024-05-27T09:29:36.392 INFO:tasks.workunit.client.1.smithi138.stdout:3/356: fdatasync d0/d2/da/df/d10/d2a/d48/f4d 0 2024-05-27T09:29:36.394 INFO:tasks.workunit.client.0.smithi119.stdout:2/860: sync 2024-05-27T09:29:36.399 INFO:tasks.workunit.client.0.smithi119.stdout:3/967: getdents db/d36/d45/d4c/d62/d13/d33/d9b 0 2024-05-27T09:29:36.399 INFO:tasks.workunit.client.1.smithi138.stdout:5/273: rename d3/d12/d16/d1d/f44 to d3/d12/d26/d53/f5a 0 2024-05-27T09:29:36.399 INFO:tasks.workunit.client.0.smithi119.stdout:6/811: dread d0/ff5 [0,4194304] 0 2024-05-27T09:29:36.401 INFO:tasks.workunit.client.1.smithi138.stdout:6/472: creat d3/fae x:0 0 0 2024-05-27T09:29:36.415 INFO:tasks.workunit.client.1.smithi138.stdout:6/473: symlink d3/d11/d2d/d37/d8b/laf 0 2024-05-27T09:29:36.419 INFO:tasks.workunit.client.0.smithi119.stdout:8/819: creat d3/d6e/dcf/dfd/f10f x:0 0 0 2024-05-27T09:29:36.422 INFO:tasks.workunit.client.1.smithi138.stdout:4/389: sync 2024-05-27T09:29:36.423 INFO:tasks.workunit.client.1.smithi138.stdout:1/360: sync 2024-05-27T09:29:36.423 INFO:tasks.workunit.client.1.smithi138.stdout:1/361: fsync d4/d48/d22/f5a 0 2024-05-27T09:29:36.424 INFO:tasks.workunit.client.0.smithi119.stdout:3/968: write db/d36/d45/ff0 [1103145,115200] 0 2024-05-27T09:29:36.434 INFO:tasks.workunit.client.1.smithi138.stdout:6/474: unlink d3/l25 0 2024-05-27T09:29:36.434 INFO:tasks.workunit.client.1.smithi138.stdout:6/475: creat d3/d11/d2d/d37/d7e/fb0 x:0 0 0 2024-05-27T09:29:36.437 INFO:tasks.workunit.client.1.smithi138.stdout:4/390: mknod d7/df/d35/d4c/d59/c83 0 2024-05-27T09:29:36.440 INFO:tasks.workunit.client.1.smithi138.stdout:2/286: dwrite d1/d15/f4f [4194304,4194304] 0 2024-05-27T09:29:36.442 INFO:tasks.workunit.client.0.smithi119.stdout:5/936: mkdir d6/d2f/d146 0 2024-05-27T09:29:36.443 INFO:tasks.workunit.client.1.smithi138.stdout:3/357: dwrite d0/d2/da/df/d10/d32/f28 [0,4194304] 0 2024-05-27T09:29:36.443 INFO:tasks.workunit.client.1.smithi138.stdout:6/476: symlink d3/d35/d43/d71/lb1 0 2024-05-27T09:29:36.443 INFO:tasks.workunit.client.1.smithi138.stdout:6/477: write d3/d19/f28 [393437,71156] 0 2024-05-27T09:29:36.443 INFO:tasks.workunit.client.1.smithi138.stdout:6/478: getdents d3/d35/d43/d1c/d32/d47 0 2024-05-27T09:29:36.443 INFO:tasks.workunit.client.1.smithi138.stdout:2/287: mkdir d1/d15/d2d/d17/d25/d4d/d4e/d60 0 2024-05-27T09:29:36.444 INFO:tasks.workunit.client.1.smithi138.stdout:4/391: symlink d7/df/l84 0 2024-05-27T09:29:36.444 INFO:tasks.workunit.client.1.smithi138.stdout:4/392: chown d7/d27/f80 0 1 2024-05-27T09:29:36.445 INFO:tasks.workunit.client.1.smithi138.stdout:6/479: creat d3/d11/d2d/d37/daa/fb2 x:0 0 0 2024-05-27T09:29:36.450 INFO:tasks.workunit.client.1.smithi138.stdout:4/393: symlink d7/d27/l85 0 2024-05-27T09:29:36.450 INFO:tasks.workunit.client.1.smithi138.stdout:4/394: creat d7/df/d5b/f86 x:0 0 0 2024-05-27T09:29:36.450 INFO:tasks.workunit.client.1.smithi138.stdout:6/480: creat d3/d35/d5a/fb3 x:0 0 0 2024-05-27T09:29:36.450 INFO:tasks.workunit.client.1.smithi138.stdout:1/362: write d4/d48/d22/f35 [7402360,27083] 0 2024-05-27T09:29:36.451 INFO:tasks.workunit.client.1.smithi138.stdout:0/339: dwrite da/d2a/d3f/f4c [8388608,4194304] 0 2024-05-27T09:29:36.451 INFO:tasks.workunit.client.0.smithi119.stdout:7/921: symlink d7/d1a/l122 0 2024-05-27T09:29:36.451 INFO:tasks.workunit.client.0.smithi119.stdout:9/730: getdents d18/d19/d1b/d40/d44 0 2024-05-27T09:29:36.451 INFO:tasks.workunit.client.0.smithi119.stdout:9/731: write d18/d19/d1b/d29/f43 [4450630,41154] 0 2024-05-27T09:29:36.453 INFO:tasks.workunit.client.1.smithi138.stdout:3/358: dread d0/d2/da/f33 [0,4194304] 0 2024-05-27T09:29:36.455 INFO:tasks.workunit.client.0.smithi119.stdout:0/831: dwrite d2/d6/d9/f2c [0,4194304] 0 2024-05-27T09:29:36.455 INFO:tasks.workunit.client.0.smithi119.stdout:0/832: chown d2/lee 13501133 1 2024-05-27T09:29:36.455 INFO:tasks.workunit.client.0.smithi119.stdout:5/937: dread d6/dd/d46/d47/d4b/dd8/d124/fb1 [0,4194304] 0 2024-05-27T09:29:36.458 INFO:tasks.workunit.client.1.smithi138.stdout:4/395: creat d7/df/d35/d4c/d59/d5d/f87 x:0 0 0 2024-05-27T09:29:36.458 INFO:tasks.workunit.client.1.smithi138.stdout:4/396: chown d7/f44 68 1 2024-05-27T09:29:36.458 INFO:tasks.workunit.client.0.smithi119.stdout:1/902: rmdir d7/db/d8f/d9b 39 2024-05-27T09:29:36.459 INFO:tasks.workunit.client.1.smithi138.stdout:1/363: chown d4/d38/c46 0 1 2024-05-27T09:29:36.459 INFO:tasks.workunit.client.1.smithi138.stdout:1/364: chown d4/f18 17 1 2024-05-27T09:29:36.459 INFO:tasks.workunit.client.1.smithi138.stdout:1/365: chown d4/d1a 65664 1 2024-05-27T09:29:36.461 INFO:tasks.workunit.client.0.smithi119.stdout:2/861: creat d2/da/d8a/d8f/d11e/f128 x:0 0 0 2024-05-27T09:29:36.462 INFO:tasks.workunit.client.1.smithi138.stdout:6/481: mkdir d3/d35/d5a/db4 0 2024-05-27T09:29:36.462 INFO:tasks.workunit.client.1.smithi138.stdout:3/359: mknod d0/d2/da/df/c74 0 2024-05-27T09:29:36.464 INFO:tasks.workunit.client.0.smithi119.stdout:6/812: creat d0/d2/d57/d37/d74/df1/f11f x:0 0 0 2024-05-27T09:29:36.464 INFO:tasks.workunit.client.1.smithi138.stdout:3/360: unlink d0/d2/da/df/d10/c6d 0 2024-05-27T09:29:36.464 INFO:tasks.workunit.client.1.smithi138.stdout:3/361: fsync d0/d3f/f55 0 2024-05-27T09:29:36.466 INFO:tasks.workunit.client.0.smithi119.stdout:7/922: dread d7/df/d19/d78/f109 [0,4194304] 0 2024-05-27T09:29:36.467 INFO:tasks.workunit.client.1.smithi138.stdout:1/366: mknod d4/d1a/d2e/d4a/c72 0 2024-05-27T09:29:36.476 INFO:tasks.workunit.client.0.smithi119.stdout:3/969: truncate db/d36/d45/dab/fea 4393283 0 2024-05-27T09:29:36.476 INFO:tasks.workunit.client.0.smithi119.stdout:3/970: write db/dc9/d98/db9/fbd [5627940,127163] 0 2024-05-27T09:29:36.478 INFO:tasks.workunit.client.1.smithi138.stdout:1/367: mknod d4/d48/d39/d5f/d63/c73 0 2024-05-27T09:29:36.482 INFO:tasks.workunit.client.0.smithi119.stdout:4/922: getdents d4/d23/d24/d5b/da8/dda/ddb 0 2024-05-27T09:29:36.482 INFO:tasks.workunit.client.1.smithi138.stdout:6/482: mknod d3/cb5 0 2024-05-27T09:29:36.485 INFO:tasks.workunit.client.1.smithi138.stdout:4/397: mknod d7/c88 0 2024-05-27T09:29:36.486 INFO:tasks.workunit.client.1.smithi138.stdout:4/398: write d7/df/d5b/f7c [886050,63952] 0 2024-05-27T09:29:36.486 INFO:tasks.workunit.client.1.smithi138.stdout:4/399: creat d7/df/d5b/f89 x:0 0 0 2024-05-27T09:29:36.495 INFO:tasks.workunit.client.0.smithi119.stdout:0/833: mknod d2/d1a/c112 0 2024-05-27T09:29:36.495 INFO:tasks.workunit.client.0.smithi119.stdout:0/834: creat d2/d6/d9/d24/d30/dcc/f113 x:0 0 0 2024-05-27T09:29:36.495 INFO:tasks.workunit.client.0.smithi119.stdout:0/835: dread - d2/d110/fa7 zero size 2024-05-27T09:29:36.495 INFO:tasks.workunit.client.0.smithi119.stdout:0/836: write d2/d110/d59/f58 [1340924,79682] 0 2024-05-27T09:29:36.497 INFO:tasks.workunit.client.1.smithi138.stdout:4/400: link d7/df/d35/d4c/d59/f58 d7/df/d35/f8a 0 2024-05-27T09:29:36.497 INFO:tasks.workunit.client.1.smithi138.stdout:4/401: creat d7/df/d35/f8b x:0 0 0 2024-05-27T09:29:36.497 INFO:tasks.workunit.client.1.smithi138.stdout:4/402: fdatasync d7/d27/f2c 0 2024-05-27T09:29:36.498 INFO:tasks.workunit.client.0.smithi119.stdout:3/971: dread db/d36/d45/d93/fb0 [0,4194304] 0 2024-05-27T09:29:36.498 INFO:tasks.workunit.client.0.smithi119.stdout:3/972: stat db/d36/d45/d93/fb0 0 2024-05-27T09:29:36.499 INFO:tasks.workunit.client.0.smithi119.stdout:5/938: creat d6/dd/d39/d19/d3c/d52/d9c/f147 x:0 0 0 2024-05-27T09:29:36.499 INFO:tasks.workunit.client.0.smithi119.stdout:4/923: write d4/d6/d28/d2f/f50 [3843357,108328] 0 2024-05-27T09:29:36.504 INFO:tasks.workunit.client.0.smithi119.stdout:1/903: rmdir d7/db/d12/d15/dd2 39 2024-05-27T09:29:36.504 INFO:tasks.workunit.client.0.smithi119.stdout:7/923: chown d7/df/d19/d9a/da6/dda/d44/d67/cca 2 1 2024-05-27T09:29:36.504 INFO:tasks.workunit.client.0.smithi119.stdout:6/813: mknod d0/d2/d57/dd0/c120 0 2024-05-27T09:29:36.505 INFO:tasks.workunit.client.0.smithi119.stdout:6/814: creat d0/d2/d1c/d21/f121 x:0 0 0 2024-05-27T09:29:36.505 INFO:tasks.workunit.client.0.smithi119.stdout:8/820: rmdir d3/d39 39 2024-05-27T09:29:36.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.506+0000 7fd295c00700 1 -- 172.21.15.119:0/3626118503 --> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd290103200 con 0x7fd274072810 2024-05-27T09:29:36.511 INFO:tasks.workunit.client.0.smithi119.stdout:3/973: mknod db/d7f/c13d 0 2024-05-27T09:29:36.520 INFO:tasks.workunit.client.0.smithi119.stdout:0/837: dread d2/d1a/f21 [0,4194304] 0 2024-05-27T09:29:36.521 INFO:tasks.workunit.client.1.smithi138.stdout:4/403: mknod d7/df/c8c 0 2024-05-27T09:29:36.524 INFO:tasks.workunit.client.1.smithi138.stdout:4/404: rename d7/df/d35/d4c/d59/l53 to d7/df/d35/d4c/d59/d5d/d64/d6a/l8d 0 2024-05-27T09:29:36.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.523+0000 7fd287400700 1 -- 172.21.15.119:0/3626118503 <== mgr.14570 v2:172.21.15.138:6828/1384904888 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd290103200 con 0x7fd274072810 2024-05-27T09:29:36.531 INFO:tasks.workunit.client.0.smithi119.stdout:4/924: mknod d4/d6/d28/d2f/de6/c12b 0 2024-05-27T09:29:36.531 INFO:tasks.workunit.client.0.smithi119.stdout:6/815: mkdir d0/d2/d57/d37/de5/d122 0 2024-05-27T09:29:36.531 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:29:36.532 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (8m) 2s ago 10m 13.9M - 0.25.0 c8568f914cd2 bc280230ab28 2024-05-27T09:29:36.532 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (11m) 2s ago 11m 7772k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:29:36.532 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (9m) 6s ago 9m 7629k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:29:36.532 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (11m) 2s ago 11m 6665k - 18.2.0 dc2bc1663786 e5fafde44183 2024-05-27T09:29:36.532 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (9m) 6s ago 9m 6660k - 18.2.0 dc2bc1663786 46956ab99bab 2024-05-27T09:29:36.532 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (8m) 2s ago 10m 74.5M - 9.4.7 954c08fa6188 16eeaded4295 2024-05-27T09:29:36.532 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (3m) 2s ago 3m 13.5M - 18.2.0 dc2bc1663786 93b21713fb1e 2024-05-27T09:29:36.532 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (3m) 2s ago 3m 132M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:29:36.532 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (3m) 6s ago 3m 13.5M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:29:36.532 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (3m) 6s ago 3m 127M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:29:36.532 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:9283,8765,8443 running (12m) 2s ago 12m 323M - 18.2.0 dc2bc1663786 abe5e83463f2 2024-05-27T09:29:36.532 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (41s) 6s ago 9m 572M - 19.0.0-3868-ga6507388 850a06a84d28 be5c0668306a 2024-05-27T09:29:36.532 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (12m) 2s ago 12m 51.5M 2048M 18.2.0 dc2bc1663786 ba998b789a1c 2024-05-27T09:29:36.532 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (9m) 6s ago 9m 41.4M 2048M 18.2.0 dc2bc1663786 60216fd67078 2024-05-27T09:29:36.533 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (10m) 2s ago 10m 8828k - 1.5.0 0da6a335fe13 79c36d4e5b04 2024-05-27T09:29:36.533 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (9m) 6s ago 9m 8086k - 1.5.0 0da6a335fe13 80ee890e8830 2024-05-27T09:29:36.533 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (8m) 2s ago 8m 175M 3947M 18.2.0 dc2bc1663786 825bd1392508 2024-05-27T09:29:36.533 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (7m) 2s ago 7m 227M 3947M 18.2.0 dc2bc1663786 fce9819903d9 2024-05-27T09:29:36.533 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (7m) 2s ago 7m 168M 3947M 18.2.0 dc2bc1663786 3955a6112c1c 2024-05-27T09:29:36.533 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (6m) 6s ago 6m 215M 4971M 18.2.0 dc2bc1663786 693c92220622 2024-05-27T09:29:36.533 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (5m) 6s ago 5m 186M 4971M 18.2.0 dc2bc1663786 7559e658e917 2024-05-27T09:29:36.533 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (5m) 6s ago 5m 213M 4971M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:29:36.533 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (8m) 2s ago 10m 60.7M - 2.43.0 a07b618ecd1d 7f3d89d1f425 2024-05-27T09:29:36.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.530+0000 7fd295c00700 1 -- 172.21.15.119:0/3626118503 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7fd274072810 msgr2=0x7fd274074cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:36.534 INFO:tasks.workunit.client.0.smithi119.stdout:0/838: creat d2/d6/d23/d7a/f114 x:0 0 0 2024-05-27T09:29:36.534 INFO:tasks.workunit.client.0.smithi119.stdout:0/839: chown d2/d6/d9/d24/d30/dcc/cf8 3059 1 2024-05-27T09:29:36.534 INFO:tasks.workunit.client.0.smithi119.stdout:0/840: read d2/d6/f8 [3058132,57582] 0 2024-05-27T09:29:36.534 INFO:tasks.workunit.client.0.smithi119.stdout:9/732: rename d18/d19/d61/d98 to d18/ddf 0 2024-05-27T09:29:36.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.530+0000 7fd295c00700 1 --2- 172.21.15.119:0/3626118503 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7fd274072810 0x7fd274074cc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd2880094f0 tx=0x7fd288009040 comp rx=0 tx=0).stop 2024-05-27T09:29:36.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.530+0000 7fd295c00700 1 -- 172.21.15.119:0/3626118503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2900737b0 msgr2=0x7fd29019bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:36.535 INFO:tasks.workunit.client.0.smithi119.stdout:9/733: creat d18/d19/d1b/d40/fe0 x:0 0 0 2024-05-27T09:29:36.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.530+0000 7fd295c00700 1 --2- 172.21.15.119:0/3626118503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2900737b0 0x7fd29019bf30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd280009910 tx=0x7fd28000bd70 comp rx=0 tx=0).stop 2024-05-27T09:29:36.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.531+0000 7fd295c00700 1 -- 172.21.15.119:0/3626118503 shutdown_connections 2024-05-27T09:29:36.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.531+0000 7fd295c00700 1 --2- 172.21.15.119:0/3626118503 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7fd274072810 0x7fd274074cc0 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd2880094f0 tx=0x7fd288009040 comp rx=0 tx=0).stop 2024-05-27T09:29:36.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.531+0000 7fd295c00700 1 --2- 172.21.15.119:0/3626118503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2900737b0 0x7fd29019bf30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:36.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.531+0000 7fd295c00700 1 --2- 172.21.15.119:0/3626118503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd290074d80 0x7fd29019c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:36.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.531+0000 7fd295c00700 1 -- 172.21.15.119:0/3626118503 >> 172.21.15.119:0/3626118503 conn(0x7fd2900fb6e0 msgr2=0x7fd290101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:36.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.532+0000 7fd295c00700 1 -- 172.21.15.119:0/3626118503 shutdown_connections 2024-05-27T09:29:36.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.532+0000 7fd295c00700 1 -- 172.21.15.119:0/3626118503 wait complete. 2024-05-27T09:29:36.536 INFO:tasks.workunit.client.0.smithi119.stdout:9/734: write d18/f30 [1342745,53724] 0 2024-05-27T09:29:36.536 INFO:tasks.workunit.client.0.smithi119.stdout:9/735: chown d18/d19/d1b/d29/fc9 53641 1 2024-05-27T09:29:36.536 INFO:tasks.workunit.client.0.smithi119.stdout:7/924: mknod d7/d5f/c123 0 2024-05-27T09:29:36.536 INFO:tasks.workunit.client.0.smithi119.stdout:0/841: write d2/d6/d9/d12/da3/fe0 [630917,106396] 0 2024-05-27T09:29:36.536 INFO:tasks.workunit.client.0.smithi119.stdout:6/816: mkdir d0/d2/d57/d37/de5/d123 0 2024-05-27T09:29:36.536 INFO:tasks.workunit.client.0.smithi119.stdout:0/842: creat d2/d6/d9/d24/d69/f115 x:0 0 0 2024-05-27T09:29:36.536 INFO:tasks.workunit.client.0.smithi119.stdout:4/925: mkdir d4/d6/d81/dce/d12c 0 2024-05-27T09:29:36.537 INFO:tasks.workunit.client.0.smithi119.stdout:3/974: rename db/d36/d45/d4d/dc8 to db/dc9/d98/db9/dd1/d13e 0 2024-05-27T09:29:36.539 INFO:tasks.workunit.client.0.smithi119.stdout:9/736: mkdir d18/d19/d1b/d29/dc4/de1 0 2024-05-27T09:29:36.542 INFO:tasks.workunit.client.0.smithi119.stdout:7/925: unlink d7/c18 0 2024-05-27T09:29:36.542 INFO:tasks.workunit.client.0.smithi119.stdout:7/926: write d7/df/d19/d9a/da6/dda/d44/fde [817509,85660] 0 2024-05-27T09:29:36.544 INFO:tasks.workunit.client.0.smithi119.stdout:3/975: creat db/d36/d45/d4c/d62/d13/f13f x:0 0 0 2024-05-27T09:29:36.546 INFO:tasks.workunit.client.0.smithi119.stdout:0/843: mkdir d2/d6/d23/d7a/d116 0 2024-05-27T09:29:36.548 INFO:tasks.workunit.client.0.smithi119.stdout:7/927: getdents d7/df/d19/d9a/da6 0 2024-05-27T09:29:36.548 INFO:tasks.workunit.client.0.smithi119.stdout:3/976: truncate db/d36/d45/d93/fc6 363158 0 2024-05-27T09:29:36.551 INFO:tasks.workunit.client.0.smithi119.stdout:7/928: rename d7/df/d19/d46/lf8 to d7/d6a/d7b/d11c/l124 0 2024-05-27T09:29:36.553 INFO:tasks.workunit.client.0.smithi119.stdout:3/977: creat db/d5b/d112/f140 x:0 0 0 2024-05-27T09:29:36.553 INFO:tasks.workunit.client.0.smithi119.stdout:3/978: write db/dc9/fd5 [794645,119880] 0 2024-05-27T09:29:36.556 INFO:tasks.workunit.client.0.smithi119.stdout:7/929: mknod d7/d1a/d99/d112/c125 0 2024-05-27T09:29:36.556 INFO:tasks.workunit.client.0.smithi119.stdout:6/817: write d0/d2/d1c/f2d [4831883,51490] 0 2024-05-27T09:29:36.557 INFO:tasks.workunit.client.0.smithi119.stdout:7/930: mknod d7/df/d19/d97/c126 0 2024-05-27T09:29:36.558 INFO:tasks.workunit.client.0.smithi119.stdout:3/979: mkdir db/d36/d45/d4c/d62/d141 0 2024-05-27T09:29:36.558 INFO:tasks.workunit.client.0.smithi119.stdout:3/980: readlink db/l26 0 2024-05-27T09:29:36.558 INFO:tasks.workunit.client.0.smithi119.stdout:7/931: symlink d7/df/d19/d97/l127 0 2024-05-27T09:29:36.558 INFO:tasks.workunit.client.0.smithi119.stdout:7/932: fdatasync d7/df/d19/d97/d92/db7/dba/d42/fe9 0 2024-05-27T09:29:36.560 INFO:tasks.workunit.client.0.smithi119.stdout:7/933: rename d7/d1a/d99/dcf/fd2 to d7/d5f/d79/f128 0 2024-05-27T09:29:36.562 INFO:tasks.workunit.client.0.smithi119.stdout:3/981: unlink db/d36/d45/d4c/d62/l27 0 2024-05-27T09:29:36.569 INFO:tasks.workunit.client.0.smithi119.stdout:9/737: write d18/d19/d1b/d29/fc9 [3538927,1338] 0 2024-05-27T09:29:36.589 INFO:tasks.workunit.client.1.smithi138.stdout:6/483: rmdir d3 39 2024-05-27T09:29:36.594 INFO:tasks.workunit.client.1.smithi138.stdout:4/405: getdents d7/df/d5b 0 2024-05-27T09:29:36.594 INFO:tasks.workunit.client.1.smithi138.stdout:4/406: creat d7/d27/d3a/f8e x:0 0 0 2024-05-27T09:29:36.594 INFO:tasks.workunit.client.1.smithi138.stdout:4/407: write d7/df/d35/f8a [864158,91298] 0 2024-05-27T09:29:36.598 INFO:tasks.workunit.client.1.smithi138.stdout:4/408: unlink d7/df/d35/d4c/d59/l4f 0 2024-05-27T09:29:36.601 INFO:tasks.workunit.client.0.smithi119.stdout:0/844: rmdir d2/d6/d9/d24/d30/dcc 39 2024-05-27T09:29:36.601 INFO:tasks.workunit.client.0.smithi119.stdout:9/738: dread d18/d19/d1b/d29/f9e [0,4194304] 0 2024-05-27T09:29:36.604 INFO:tasks.workunit.client.0.smithi119.stdout:9/739: rmdir d18/d19/d61/d95 39 2024-05-27T09:29:36.607 INFO:tasks.workunit.client.1.smithi138.stdout:2/288: dwrite d1/d15/f48 [0,4194304] 0 2024-05-27T09:29:36.607 INFO:tasks.workunit.client.1.smithi138.stdout:0/340: dwrite da/f3c [0,4194304] 0 2024-05-27T09:29:36.607 INFO:tasks.workunit.client.1.smithi138.stdout:3/362: dwrite d0/d2/da/df/d10/d2a/f4f [0,4194304] 0 2024-05-27T09:29:36.607 INFO:tasks.workunit.client.1.smithi138.stdout:3/363: chown d0/d2/da/df/d10/l44 40 1 2024-05-27T09:29:36.608 INFO:tasks.workunit.client.1.smithi138.stdout:6/484: dread d3/d11/d2d/d37/f85 [0,4194304] 0 2024-05-27T09:29:36.609 INFO:tasks.workunit.client.1.smithi138.stdout:3/364: mknod d0/d2/da/df/d10/c75 0 2024-05-27T09:29:36.610 INFO:tasks.workunit.client.0.smithi119.stdout:5/939: dwrite d6/db/f125 [0,4194304] 0 2024-05-27T09:29:36.610 INFO:tasks.workunit.client.0.smithi119.stdout:9/740: mknod d18/d19/d1b/d40/d7b/dc6/ce2 0 2024-05-27T09:29:36.610 INFO:tasks.workunit.client.0.smithi119.stdout:9/741: readlink d18/d19/d1b/d2a/lda 0 2024-05-27T09:29:36.612 INFO:tasks.workunit.client.1.smithi138.stdout:3/365: rename d0/d2/f3d to d0/d3f/d50/f76 0 2024-05-27T09:29:36.612 INFO:tasks.workunit.client.1.smithi138.stdout:6/485: creat d3/d35/d5a/db4/fb6 x:0 0 0 2024-05-27T09:29:36.614 INFO:tasks.workunit.client.1.smithi138.stdout:1/368: dwrite d4/d1a/d2e/d32/f68 [0,4194304] 0 2024-05-27T09:29:36.615 INFO:tasks.workunit.client.1.smithi138.stdout:6/486: creat d3/d35/d43/d1c/d32/d9b/fb7 x:0 0 0 2024-05-27T09:29:36.617 INFO:tasks.workunit.client.0.smithi119.stdout:2/862: dwrite d2/da/d8a/dd4/d89/d94/d87/fba [0,4194304] 0 2024-05-27T09:29:36.623 INFO:tasks.workunit.client.0.smithi119.stdout:5/940: symlink d6/dd/d39/d19/d7d/db7/l148 0 2024-05-27T09:29:36.623 INFO:tasks.workunit.client.0.smithi119.stdout:8/821: dwrite d3/d8/d1b/d46/f43 [0,4194304] 0 2024-05-27T09:29:36.629 INFO:tasks.workunit.client.0.smithi119.stdout:1/904: dwrite d7/db/d12/d15/fa1 [0,4194304] 0 2024-05-27T09:29:36.629 INFO:tasks.workunit.client.0.smithi119.stdout:1/905: creat d7/db/d12/ddb/d38/d3c/f117 x:0 0 0 2024-05-27T09:29:36.629 INFO:tasks.workunit.client.0.smithi119.stdout:1/906: truncate d7/db/d12/ddb/d38/d49/f87 731273 0 2024-05-27T09:29:36.629 INFO:tasks.workunit.client.0.smithi119.stdout:1/907: chown d7/dd/cf 4558820 1 2024-05-27T09:29:36.629 INFO:tasks.workunit.client.0.smithi119.stdout:1/908: chown d7/db/d8f/fc4 54 1 2024-05-27T09:29:36.629 INFO:tasks.workunit.client.0.smithi119.stdout:1/909: chown d7/f5b 524257542 1 2024-05-27T09:29:36.631 INFO:tasks.workunit.client.0.smithi119.stdout:9/742: rename d18/d19/d1b/d29/lde to d18/d19/d1b/d40/d7b/dc6/le3 0 2024-05-27T09:29:36.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:36 smithi138 ceph-mon[34885]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:29:36.650 INFO:tasks.workunit.client.0.smithi119.stdout:8/822: symlink d3/d8/d1b/l110 0 2024-05-27T09:29:36.651 INFO:tasks.workunit.client.0.smithi119.stdout:8/823: chown d3/d6e/dcf/dce 197 1 2024-05-27T09:29:36.651 INFO:tasks.workunit.client.0.smithi119.stdout:8/824: write d3/d6e/dcf/dfd/f10f [801714,999] 0 2024-05-27T09:29:36.652 INFO:tasks.workunit.client.1.smithi138.stdout:6/487: symlink d3/d11/lb8 0 2024-05-27T09:29:36.655 INFO:tasks.workunit.client.0.smithi119.stdout:1/910: mknod d7/dd/dcb/c118 0 2024-05-27T09:29:36.655 INFO:tasks.workunit.client.0.smithi119.stdout:1/911: chown d7/dd/d25/d60/d61 1290874 1 2024-05-27T09:29:36.658 INFO:tasks.workunit.client.0.smithi119.stdout:8/825: mkdir d3/d8/d1b/d46/d92/d111 0 2024-05-27T09:29:36.659 INFO:tasks.workunit.client.0.smithi119.stdout:1/912: mknod d7/dd/d25/d60/c119 0 2024-05-27T09:29:36.664 INFO:tasks.workunit.client.1.smithi138.stdout:1/369: dread d4/d48/d39/d5f/f1f [4194304,4194304] 0 2024-05-27T09:29:36.667 INFO:tasks.workunit.client.0.smithi119.stdout:4/926: dwrite d4/d6/d28/d2f/d36/f128 [0,4194304] 0 2024-05-27T09:29:36.667 INFO:tasks.workunit.client.0.smithi119.stdout:4/927: stat d4/d6/d3e 0 2024-05-27T09:29:36.674 INFO:tasks.workunit.client.1.smithi138.stdout:1/370: link d4/d48/c64 d4/d38/c74 0 2024-05-27T09:29:36.674 INFO:tasks.workunit.client.1.smithi138.stdout:1/371: chown d4/d48/f58 3839 1 2024-05-27T09:29:36.674 INFO:tasks.workunit.client.1.smithi138.stdout:1/372: write d4/d48/d22/d65/f6f [660372,28177] 0 2024-05-27T09:29:36.674 INFO:tasks.workunit.client.1.smithi138.stdout:1/373: truncate d4/d1a/d2e/d32/f49 254185 0 2024-05-27T09:29:36.689 INFO:tasks.workunit.client.0.smithi119.stdout:9/743: dread d18/d19/fab [0,4194304] 0 2024-05-27T09:29:36.691 INFO:tasks.workunit.client.1.smithi138.stdout:3/366: dwrite d0/d2/da/df/d10/d2a/d48/f49 [0,4194304] 0 2024-05-27T09:29:36.692 INFO:tasks.workunit.client.0.smithi119.stdout:3/982: dwrite db/d36/d45/d4c/d62/d13/f13f [0,4194304] 0 2024-05-27T09:29:36.694 INFO:tasks.workunit.client.0.smithi119.stdout:8/826: read d3/d8/d4d/d58/f82 [2013493,105186] 0 2024-05-27T09:29:36.694 INFO:tasks.workunit.client.0.smithi119.stdout:9/744: mkdir d18/d19/d61/d95/de4 0 2024-05-27T09:29:36.694 INFO:tasks.workunit.client.0.smithi119.stdout:9/745: fdatasync d18/fd1 0 2024-05-27T09:29:36.696 INFO:tasks.workunit.client.1.smithi138.stdout:3/367: stat d0/d2/c20 0 2024-05-27T09:29:36.696 INFO:tasks.workunit.client.1.smithi138.stdout:3/368: chown d0 14 1 2024-05-27T09:29:36.696 INFO:tasks.workunit.client.1.smithi138.stdout:3/369: stat d0/d2/da/df/f6f 0 2024-05-27T09:29:36.698 INFO:tasks.workunit.client.1.smithi138.stdout:3/370: creat d0/d2/da/df/d10/f77 x:0 0 0 2024-05-27T09:29:36.700 INFO:tasks.workunit.client.1.smithi138.stdout:3/371: link d0/d2/da/l2f d0/d2/da/df/d10/d32/l78 0 2024-05-27T09:29:36.700 INFO:tasks.workunit.client.0.smithi119.stdout:3/983: rmdir db/d36/d45 39 2024-05-27T09:29:36.703 INFO:tasks.workunit.client.0.smithi119.stdout:8/827: creat d3/d8/d1b/d77/f112 x:0 0 0 2024-05-27T09:29:36.703 INFO:tasks.workunit.client.0.smithi119.stdout:9/746: creat d18/d19/d1b/d29/dc4/de1/fe5 x:0 0 0 2024-05-27T09:29:36.706 INFO:tasks.workunit.client.0.smithi119.stdout:2/863: dwrite d2/da/d8a/dd4/d89/d94/d87/f9a [0,4194304] 0 2024-05-27T09:29:36.706 INFO:tasks.workunit.client.0.smithi119.stdout:2/864: readlink d2/da/d15/d53/dac/ld0 0 2024-05-27T09:29:36.716 INFO:tasks.workunit.client.1.smithi138.stdout:2/289: dwrite d1/d15/d2d/d17/d25/d4d/f5c [0,4194304] 0 2024-05-27T09:29:36.724 INFO:tasks.workunit.client.1.smithi138.stdout:3/372: read d0/d2/da/df/d10/d32/f28 [2447779,131070] 0 2024-05-27T09:29:36.724 INFO:tasks.workunit.client.1.smithi138.stdout:3/373: creat d0/d2/da/df/d10/d2a/d48/f79 x:0 0 0 2024-05-27T09:29:36.724 INFO:tasks.workunit.client.1.smithi138.stdout:2/290: truncate d1/f1b 6181769 0 2024-05-27T09:29:36.727 INFO:tasks.workunit.client.0.smithi119.stdout:9/747: unlink d18/f30 0 2024-05-27T09:29:36.728 INFO:tasks.workunit.client.0.smithi119.stdout:9/748: chown d18/d19/d1b/l2b 1646 1 2024-05-27T09:29:36.728 INFO:tasks.workunit.client.0.smithi119.stdout:9/749: creat d18/d19/d1b/d2a/d9f/fe6 x:0 0 0 2024-05-27T09:29:36.728 INFO:tasks.workunit.client.0.smithi119.stdout:4/928: read d4/d6/d28/d2f/d36/f72 [2845913,14333] 0 2024-05-27T09:29:36.728 INFO:tasks.workunit.client.0.smithi119.stdout:2/865: dread d2/da/d2c/f3c [0,4194304] 0 2024-05-27T09:29:36.729 INFO:tasks.workunit.client.0.smithi119.stdout:2/866: read - d2/da/d8a/dd4/d89/d94/f8b zero size 2024-05-27T09:29:36.729 INFO:tasks.workunit.client.1.smithi138.stdout:3/374: symlink d0/d3f/d60/l7a 0 2024-05-27T09:29:36.730 INFO:tasks.workunit.client.1.smithi138.stdout:6/488: dwrite d3/d35/d5a/d6a/f86 [0,4194304] 0 2024-05-27T09:29:36.734 INFO:tasks.workunit.client.0.smithi119.stdout:0/845: dwrite d2/d6/d9/d2f/d95/faf [0,4194304] 0 2024-05-27T09:29:36.734 INFO:tasks.workunit.client.0.smithi119.stdout:9/750: creat d18/d19/d1b/d40/d44/dcb/fe7 x:0 0 0 2024-05-27T09:29:36.734 INFO:tasks.workunit.client.0.smithi119.stdout:9/751: creat d18/d19/d9b/fe8 x:0 0 0 2024-05-27T09:29:36.735 INFO:tasks.workunit.client.0.smithi119.stdout:9/752: creat d18/d19/d1b/d40/fe9 x:0 0 0 2024-05-27T09:29:36.736 INFO:tasks.workunit.client.1.smithi138.stdout:3/375: creat d0/d3f/f7b x:0 0 0 2024-05-27T09:29:36.738 INFO:tasks.workunit.client.0.smithi119.stdout:8/828: rename d3/d8/d4d/d5d to d3/d8/d1b/d46/da4/d113 0 2024-05-27T09:29:36.739 INFO:tasks.workunit.client.1.smithi138.stdout:6/489: mkdir d3/d35/d7d/db9 0 2024-05-27T09:29:36.739 INFO:tasks.workunit.client.1.smithi138.stdout:9/367: sync 2024-05-27T09:29:36.739 INFO:tasks.workunit.client.1.smithi138.stdout:6/490: write d3/d11/f2e [699144,35546] 0 2024-05-27T09:29:36.741 INFO:tasks.workunit.client.1.smithi138.stdout:3/376: creat d0/d3f/d50/f7c x:0 0 0 2024-05-27T09:29:36.741 INFO:tasks.workunit.client.1.smithi138.stdout:3/377: chown d0/d2/da/df/d10/d32/c4e 917 1 2024-05-27T09:29:36.741 INFO:tasks.workunit.client.1.smithi138.stdout:3/378: fdatasync d0/d2/da/df/d10/d32/f1d 0 2024-05-27T09:29:36.743 INFO:tasks.workunit.client.0.smithi119.stdout:5/941: dwrite d6/dd/f32 [0,4194304] 0 2024-05-27T09:29:36.745 INFO:tasks.workunit.client.0.smithi119.stdout:5/942: creat d6/dd/d39/d19/d3c/dd5/f149 x:0 0 0 2024-05-27T09:29:36.747 INFO:tasks.workunit.client.1.smithi138.stdout:6/491: rename d3/d11/d2d/d37/f85 to d3/d11/d2d/d37/d8b/fba 0 2024-05-27T09:29:36.747 INFO:tasks.workunit.client.1.smithi138.stdout:3/379: rename d0/d2 to d0/d2/da/df/d10/d7d 22 2024-05-27T09:29:36.747 INFO:tasks.workunit.client.1.smithi138.stdout:3/380: write d0/f12 [2023006,80932] 0 2024-05-27T09:29:36.748 INFO:tasks.workunit.client.1.smithi138.stdout:3/381: truncate d0/d2/da/df/d10/d2a/f4f 4737116 0 2024-05-27T09:29:36.748 INFO:tasks.workunit.client.1.smithi138.stdout:1/374: dwrite d4/d48/f14 [0,4194304] 0 2024-05-27T09:29:36.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:36 smithi119 ceph-mon[29082]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:29:36.753 INFO:tasks.workunit.client.1.smithi138.stdout:1/375: unlink d4/d1a/d2e/l4b 0 2024-05-27T09:29:36.753 INFO:tasks.workunit.client.1.smithi138.stdout:1/376: creat d4/d48/d22/d65/f75 x:0 0 0 2024-05-27T09:29:36.760 INFO:tasks.workunit.client.0.smithi119.stdout:8/829: creat d3/d8/d1b/d46/da4/d113/d66/d85/f114 x:0 0 0 2024-05-27T09:29:36.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.777+0000 7f426b200700 1 -- 172.21.15.119:0/1703356589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4264074d80 msgr2=0x7f42640731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:36.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.778+0000 7f426b200700 1 --2- 172.21.15.119:0/1703356589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4264074d80 0x7f42640731e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4254009940 tx=0x7f4254009c50 comp rx=0 tx=0).stop 2024-05-27T09:29:36.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.778+0000 7f426b200700 1 -- 172.21.15.119:0/1703356589 shutdown_connections 2024-05-27T09:29:36.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.778+0000 7f426b200700 1 --2- 172.21.15.119:0/1703356589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42640737b0 0x7f4264073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:36.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.778+0000 7f426b200700 1 --2- 172.21.15.119:0/1703356589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4264074d80 0x7f42640731e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:36.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.778+0000 7f426b200700 1 -- 172.21.15.119:0/1703356589 >> 172.21.15.119:0/1703356589 conn(0x7f42640fb6e0 msgr2=0x7f42640fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:36.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.778+0000 7f426b200700 1 -- 172.21.15.119:0/1703356589 shutdown_connections 2024-05-27T09:29:36.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.778+0000 7f426b200700 1 -- 172.21.15.119:0/1703356589 wait complete. 2024-05-27T09:29:36.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.779+0000 7f426b200700 1 Processor -- start 2024-05-27T09:29:36.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.779+0000 7f426b200700 1 -- start start 2024-05-27T09:29:36.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.779+0000 7f426b200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42640737b0 0x7f4264197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:36.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.779+0000 7f426b200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4264198320 0x7f426419d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:36.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.779+0000 7f426b200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4264198820 con 0x7f4264198320 2024-05-27T09:29:36.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.779+0000 7f426b200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4264198990 con 0x7f42640737b0 2024-05-27T09:29:36.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.780+0000 7f4263e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42640737b0 0x7f4264197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:36.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.780+0000 7f4263e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42640737b0 0x7f4264197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.119:45856/0 (socket says 172.21.15.119:45856) 2024-05-27T09:29:36.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.780+0000 7f4263e00700 1 -- 172.21.15.119:0/2099298037 learned_addr learned my addr 172.21.15.119:0/2099298037 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:29:36.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.780+0000 7f4263e00700 1 -- 172.21.15.119:0/2099298037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4264198320 msgr2=0x7f426419d390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T09:29:36.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.781+0000 7f4263e00700 1 --2- 172.21.15.119:0/2099298037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4264198320 0x7f426419d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:36.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.781+0000 7f4263e00700 1 -- 172.21.15.119:0/2099298037 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42540095f0 con 0x7f42640737b0 2024-05-27T09:29:36.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.781+0000 7f4263e00700 1 --2- 172.21.15.119:0/2099298037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42640737b0 0x7f4264197de0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4254006010 tx=0x7f425400bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:36.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.781+0000 7f4260c00700 1 -- 172.21.15.119:0/2099298037 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f425401d070 con 0x7f42640737b0 2024-05-27T09:29:36.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.781+0000 7f4260c00700 1 -- 172.21.15.119:0/2099298037 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f425400f460 con 0x7f42640737b0 2024-05-27T09:29:36.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.781+0000 7f426b200700 1 -- 172.21.15.119:0/2099298037 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f426419d8d0 con 0x7f42640737b0 2024-05-27T09:29:36.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.782+0000 7f4260c00700 1 -- 172.21.15.119:0/2099298037 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4254017610 con 0x7f42640737b0 2024-05-27T09:29:36.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.782+0000 7f426b200700 1 -- 172.21.15.119:0/2099298037 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426419dd30 con 0x7f42640737b0 2024-05-27T09:29:36.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.783+0000 7f426b200700 1 -- 172.21.15.119:0/2099298037 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f426410aeb0 con 0x7f42640737b0 2024-05-27T09:29:36.784 INFO:tasks.workunit.client.0.smithi119.stdout:2/867: rename d2/da/d8a/dd4/d69/f6b to d2/da/d15/d53/d114/f129 0 2024-05-27T09:29:36.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.786+0000 7f4260c00700 1 -- 172.21.15.119:0/2099298037 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) v1 ==== 95024+0+0 (secure 0 0 0) 0x7f425400f5d0 con 0x7f42640737b0 2024-05-27T09:29:36.786 INFO:tasks.workunit.client.1.smithi138.stdout:2/291: dwrite d1/d15/d2d/f39 [4194304,4194304] 0 2024-05-27T09:29:36.787 INFO:tasks.workunit.client.0.smithi119.stdout:0/846: getdents d2/d6/d9/d24/d69 0 2024-05-27T09:29:36.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.787+0000 7f4260c00700 1 --2- 172.21.15.119:0/2099298037 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f4248072820 0x7f4248074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:36.788 INFO:tasks.workunit.client.0.smithi119.stdout:9/753: dread d18/d19/d1b/d40/d44/f4a [0,4194304] 0 2024-05-27T09:29:36.788 INFO:tasks.workunit.client.0.smithi119.stdout:3/984: dwrite db/d5b/dce/f100 [0,4194304] 0 2024-05-27T09:29:36.788 INFO:tasks.workunit.client.0.smithi119.stdout:8/830: read d3/d8/d1b/db1/ff0 [223460,8508] 0 2024-05-27T09:29:36.788 INFO:tasks.workunit.client.0.smithi119.stdout:8/831: read - d3/d8/d1b/d77/f112 zero size 2024-05-27T09:29:36.788 INFO:tasks.workunit.client.0.smithi119.stdout:8/832: chown d3/d39/d5a 1 1 2024-05-27T09:29:36.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.787+0000 7f4260c00700 1 -- 172.21.15.119:0/2099298037 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f4254094630 con 0x7f42640737b0 2024-05-27T09:29:36.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.787+0000 7f4263400700 1 --2- 172.21.15.119:0/2099298037 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f4248072820 0x7f4248074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:36.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.788+0000 7f4263400700 1 --2- 172.21.15.119:0/2099298037 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f4248072820 0x7f4248074cd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4250005950 tx=0x7f42500058e0 comp rx=0 tx=0).ready entity=mgr.14570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:36.791 INFO:tasks.workunit.client.0.smithi119.stdout:5/943: mkdir d6/dd/d46/d47/d4b/dd8/df2/d14a 0 2024-05-27T09:29:36.795 INFO:tasks.workunit.client.0.smithi119.stdout:2/868: mkdir d2/da/d15/d53/d114/d116/d12a 0 2024-05-27T09:29:36.795 INFO:tasks.workunit.client.0.smithi119.stdout:2/869: chown d2/da/d8a/dd4/l34 79341084 1 2024-05-27T09:29:36.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:36.794+0000 7f4260c00700 1 -- 172.21.15.119:0/2099298037 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f425405c6c0 con 0x7f42640737b0 2024-05-27T09:29:36.797 INFO:tasks.workunit.client.1.smithi138.stdout:2/292: symlink d1/d15/d2d/d17/d25/d4d/d4e/d60/l61 0 2024-05-27T09:29:36.804 INFO:tasks.workunit.client.1.smithi138.stdout:6/492: write d3/d35/d5a/d6a/f6f [1248633,26684] 0 2024-05-27T09:29:36.804 INFO:tasks.workunit.client.1.smithi138.stdout:6/493: chown d3/d19/f89 288879 1 2024-05-27T09:29:36.805 INFO:tasks.workunit.client.0.smithi119.stdout:9/754: link d18/d19/d1b/d40/db7/fbb d18/ddf/fea 0 2024-05-27T09:29:36.806 INFO:tasks.workunit.client.1.smithi138.stdout:2/293: rename d1/d15/d2d/d1d/f45 to d1/d10/d3c/f62 0 2024-05-27T09:29:36.806 INFO:tasks.workunit.client.0.smithi119.stdout:5/944: write d6/fc8 [1148854,127155] 0 2024-05-27T09:29:36.806 INFO:tasks.workunit.client.0.smithi119.stdout:5/945: write d6/dd/d46/d47/d4b/dd8/ddb/fe5 [816783,14870] 0 2024-05-27T09:29:36.808 INFO:tasks.workunit.client.1.smithi138.stdout:2/294: truncate d1/d15/d2d/d31/f49 498321 0 2024-05-27T09:29:36.808 INFO:tasks.workunit.client.1.smithi138.stdout:6/494: mknod d3/d11/d2d/cbb 0 2024-05-27T09:29:36.809 INFO:tasks.workunit.client.1.smithi138.stdout:2/295: mknod d1/d10/d3c/c63 0 2024-05-27T09:29:36.809 INFO:tasks.workunit.client.1.smithi138.stdout:2/296: write d1/d15/d2d/d31/f3d [972357,74124] 0 2024-05-27T09:29:36.810 INFO:tasks.workunit.client.0.smithi119.stdout:3/985: mknod db/d36/d45/d4c/d62/d13/c142 0 2024-05-27T09:29:36.810 INFO:tasks.workunit.client.0.smithi119.stdout:3/986: chown db/d36/d45/fcb 0 1 2024-05-27T09:29:36.812 INFO:tasks.workunit.client.1.smithi138.stdout:6/495: creat d3/d35/d43/d1c/fbc x:0 0 0 2024-05-27T09:29:36.816 INFO:tasks.workunit.client.1.smithi138.stdout:9/368: dwrite f1 [0,4194304] 0 2024-05-27T09:29:36.819 INFO:tasks.workunit.client.1.smithi138.stdout:6/496: dread d3/d35/d5a/d6a/fac [0,4194304] 0 2024-05-27T09:29:36.820 INFO:tasks.workunit.client.1.smithi138.stdout:9/369: dread - d6/da/d29/d61/f68 zero size 2024-05-27T09:29:36.821 INFO:tasks.workunit.client.1.smithi138.stdout:9/370: write d6/db/de/f2b [6759520,69341] 0 2024-05-27T09:29:36.822 INFO:tasks.workunit.client.1.smithi138.stdout:9/371: dread - d6/db/de/d18/d24/f5c zero size 2024-05-27T09:29:36.823 INFO:tasks.workunit.client.0.smithi119.stdout:2/870: rename d2/da/d8a/dd4/d89/db0/ffb to d2/da/d8a/d8f/d11e/f12b 0 2024-05-27T09:29:36.823 INFO:tasks.workunit.client.0.smithi119.stdout:0/847: mkdir d2/d6/d23/d7a/d117 0 2024-05-27T09:29:36.823 INFO:tasks.workunit.client.0.smithi119.stdout:8/833: write d3/f87 [2941506,99279] 0 2024-05-27T09:29:36.826 INFO:tasks.workunit.client.0.smithi119.stdout:9/755: creat d18/db4/dbe/feb x:0 0 0 2024-05-27T09:29:36.826 INFO:tasks.workunit.client.0.smithi119.stdout:9/756: fdatasync d18/f8b 0 2024-05-27T09:29:36.826 INFO:tasks.workunit.client.1.smithi138.stdout:2/297: dread d1/d15/d2d/d17/d25/f3a [0,4194304] 0 2024-05-27T09:29:36.826 INFO:tasks.workunit.client.1.smithi138.stdout:2/298: chown d1/d15/d2d/d1d/f43 3708 1 2024-05-27T09:29:36.827 INFO:tasks.workunit.client.1.smithi138.stdout:1/377: dwrite d4/d48/d22/f5a [0,4194304] 0 2024-05-27T09:29:36.828 INFO:tasks.workunit.client.1.smithi138.stdout:2/299: creat d1/d10/f64 x:0 0 0 2024-05-27T09:29:36.829 INFO:tasks.workunit.client.1.smithi138.stdout:2/300: getdents d1/d15/d2d/d31 0 2024-05-27T09:29:36.833 INFO:tasks.workunit.client.1.smithi138.stdout:2/301: rmdir d1/d15/d2d/d17 39 2024-05-27T09:29:36.834 INFO:tasks.workunit.client.0.smithi119.stdout:0/848: rename d2/d6/d9/d12/de5 to d2/d6/d23/db8/d118 0 2024-05-27T09:29:36.835 INFO:tasks.workunit.client.1.smithi138.stdout:2/302: symlink d1/d15/d2d/l65 0 2024-05-27T09:29:36.836 INFO:tasks.workunit.client.1.smithi138.stdout:2/303: creat d1/d15/d2d/d17/d25/d4d/d4e/d60/f66 x:0 0 0 2024-05-27T09:29:36.837 INFO:tasks.workunit.client.0.smithi119.stdout:8/834: write d3/d6e/d8e/f9f [620146,63846] 0 2024-05-27T09:29:36.837 INFO:tasks.workunit.client.0.smithi119.stdout:8/835: fdatasync d3/d8/d4d/fe9 0 2024-05-27T09:29:36.838 INFO:tasks.workunit.client.1.smithi138.stdout:2/304: creat d1/d15/d2d/d1d/f67 x:0 0 0 2024-05-27T09:29:36.841 INFO:tasks.workunit.client.1.smithi138.stdout:2/305: mknod d1/d15/d2d/c68 0 2024-05-27T09:29:36.843 INFO:tasks.workunit.client.1.smithi138.stdout:2/306: symlink d1/d15/d2d/d31/d4b/l69 0 2024-05-27T09:29:36.844 INFO:tasks.workunit.client.0.smithi119.stdout:3/987: rmdir db/d5b/dce 39 2024-05-27T09:29:36.844 INFO:tasks.workunit.client.0.smithi119.stdout:3/988: creat db/d36/d45/d4d/d108/f143 x:0 0 0 2024-05-27T09:29:36.845 INFO:tasks.workunit.client.0.smithi119.stdout:0/849: rename d2/d6/d9/d12/fed to d2/d6/d9/dab/f119 0 2024-05-27T09:29:36.846 INFO:tasks.workunit.client.0.smithi119.stdout:0/850: chown d2/d6/d23 6347 1 2024-05-27T09:29:36.846 INFO:tasks.workunit.client.0.smithi119.stdout:0/851: write d2/d6/d9/d12/d4f/ff1 [851879,43902] 0 2024-05-27T09:29:36.846 INFO:tasks.workunit.client.0.smithi119.stdout:2/871: write d2/f2d [1793475,35132] 0 2024-05-27T09:29:36.846 INFO:tasks.workunit.client.0.smithi119.stdout:0/852: creat d2/d6/d75/f11a x:0 0 0 2024-05-27T09:29:36.846 INFO:tasks.workunit.client.0.smithi119.stdout:0/853: readlink d2/d6/d9/l43 0 2024-05-27T09:29:36.846 INFO:tasks.workunit.client.0.smithi119.stdout:8/836: symlink d3/d8/d1b/d46/da4/d113/d60/l115 0 2024-05-27T09:29:36.853 INFO:tasks.workunit.client.0.smithi119.stdout:0/854: symlink d2/d6/d9/d12/l11b 0 2024-05-27T09:29:36.855 INFO:tasks.workunit.client.0.smithi119.stdout:8/837: creat d3/d39/d5a/f116 x:0 0 0 2024-05-27T09:29:36.856 INFO:tasks.workunit.client.0.smithi119.stdout:6/818: sync 2024-05-27T09:29:36.856 INFO:tasks.workunit.client.0.smithi119.stdout:6/819: chown d0/d2/d1c/d21/c44 38 1 2024-05-27T09:29:36.858 INFO:tasks.workunit.client.0.smithi119.stdout:2/872: symlink d2/l12c 0 2024-05-27T09:29:36.858 INFO:tasks.workunit.client.0.smithi119.stdout:4/929: sync 2024-05-27T09:29:36.860 INFO:tasks.workunit.client.0.smithi119.stdout:7/934: sync 2024-05-27T09:29:36.864 INFO:tasks.workunit.client.1.smithi138.stdout:2/307: dread d1/d15/d2d/d17/d25/d4d/f5c [0,4194304] 0 2024-05-27T09:29:36.864 INFO:tasks.workunit.client.1.smithi138.stdout:2/308: chown d1/d15/d2d/c68 33 1 2024-05-27T09:29:36.866 INFO:tasks.workunit.client.1.smithi138.stdout:2/309: rename d1/cc to d1/c6a 0 2024-05-27T09:29:36.869 INFO:tasks.workunit.client.1.smithi138.stdout:2/310: creat d1/d15/d2d/d17/f6b x:0 0 0 2024-05-27T09:29:36.869 INFO:tasks.workunit.client.1.smithi138.stdout:2/311: dread - d1/d15/d2d/d17/f58 zero size 2024-05-27T09:29:36.870 INFO:tasks.workunit.client.0.smithi119.stdout:3/989: dread db/d36/d45/d4c/d62/d13/f13f [0,4194304] 0 2024-05-27T09:29:36.870 INFO:tasks.workunit.client.0.smithi119.stdout:9/757: dread d18/d19/d1b/d40/d44/f69 [0,4194304] 0 2024-05-27T09:29:36.870 INFO:tasks.workunit.client.0.smithi119.stdout:0/855: creat d2/d6/d23/d7a/f11c x:0 0 0 2024-05-27T09:29:36.870 INFO:tasks.workunit.client.0.smithi119.stdout:8/838: mkdir d3/d8/d1b/d46/da4/d117 0 2024-05-27T09:29:36.871 INFO:tasks.workunit.client.0.smithi119.stdout:1/913: sync 2024-05-27T09:29:36.871 INFO:tasks.workunit.client.1.smithi138.stdout:2/312: creat d1/d15/d2d/d17/d25/f6c x:0 0 0 2024-05-27T09:29:36.873 INFO:tasks.workunit.client.1.smithi138.stdout:6/497: dwrite d3/d11/d2d/d37/f4a [0,4194304] 0 2024-05-27T09:29:36.873 INFO:tasks.workunit.client.1.smithi138.stdout:6/498: dread - d3/fae zero size 2024-05-27T09:29:36.880 INFO:tasks.workunit.client.0.smithi119.stdout:3/990: rename db/d7f/dd2/c121 to db/d5b/dce/de5/c144 0 2024-05-27T09:29:36.880 INFO:tasks.workunit.client.0.smithi119.stdout:9/758: getdents d18/d19/d9b 0 2024-05-27T09:29:36.882 INFO:tasks.workunit.client.0.smithi119.stdout:0/856: rename d2/d6/d75/l82 to d2/d6/d9/l11d 0 2024-05-27T09:29:36.888 INFO:tasks.workunit.client.0.smithi119.stdout:5/946: write d6/d2f/da5/f111 [73332,24988] 0 2024-05-27T09:29:36.888 INFO:tasks.workunit.client.0.smithi119.stdout:0/857: creat d2/f11e x:0 0 0 2024-05-27T09:29:36.890 INFO:tasks.workunit.client.0.smithi119.stdout:8/839: dread d3/d8/d1b/d46/f2f [0,4194304] 0 2024-05-27T09:29:36.892 INFO:tasks.workunit.client.1.smithi138.stdout:2/313: dread d1/d15/d2d/d17/d25/d4d/f5c [0,4194304] 0 2024-05-27T09:29:36.892 INFO:tasks.workunit.client.1.smithi138.stdout:2/314: read - d1/d15/d2d/d17/d25/f6c zero size 2024-05-27T09:29:36.892 INFO:tasks.workunit.client.1.smithi138.stdout:2/315: truncate d1/d15/d2d/f3e 185575 0 2024-05-27T09:29:36.894 INFO:tasks.workunit.client.0.smithi119.stdout:7/935: write d7/df/d19/d97/d92/db7/dba/f3f [595102,64482] 0 2024-05-27T09:29:36.894 INFO:tasks.workunit.client.0.smithi119.stdout:2/873: dread d2/da/d6f/f7f [0,4194304] 0 2024-05-27T09:29:36.895 INFO:tasks.workunit.client.0.smithi119.stdout:2/874: getdents d2/da/d15/d53/dac/d122 0 2024-05-27T09:29:36.896 INFO:tasks.workunit.client.0.smithi119.stdout:5/947: dread d6/dd/d39/fd1 [0,4194304] 0 2024-05-27T09:29:36.896 INFO:tasks.workunit.client.0.smithi119.stdout:8/840: rename d3/d8/l35 to d3/d6e/l118 0 2024-05-27T09:29:36.898 INFO:tasks.workunit.client.0.smithi119.stdout:5/948: rename d6/dd/d6e/fdf to d6/f14b 0 2024-05-27T09:29:36.900 INFO:tasks.workunit.client.0.smithi119.stdout:5/949: mknod d6/dd/d39/d19/d3c/d52/c14c 0 2024-05-27T09:29:36.903 INFO:tasks.workunit.client.1.smithi138.stdout:2/316: dread d1/d2f/f33 [0,4194304] 0 2024-05-27T09:29:36.903 INFO:tasks.workunit.client.1.smithi138.stdout:2/317: stat d1/d15/d2d/d31/d4b/c57 0 2024-05-27T09:29:36.906 INFO:tasks.workunit.client.0.smithi119.stdout:5/950: mkdir d6/db/dc/dc3/d145/d14d 0 2024-05-27T09:29:36.906 INFO:tasks.workunit.client.0.smithi119.stdout:8/841: rmdir d3/d8/d4d/d79 39 2024-05-27T09:29:36.906 INFO:tasks.workunit.client.0.smithi119.stdout:1/914: dread d7/dd/f77 [0,4194304] 0 2024-05-27T09:29:36.906 INFO:tasks.workunit.client.0.smithi119.stdout:1/915: fdatasync d7/db/d12/f9a 0 2024-05-27T09:29:36.908 INFO:tasks.workunit.client.0.smithi119.stdout:0/858: dread d2/d6/d23/d7a/f62 [4194304,4194304] 0 2024-05-27T09:29:36.908 INFO:tasks.workunit.client.0.smithi119.stdout:0/859: truncate d2/d110/deb/fef 693394 0 2024-05-27T09:29:36.908 INFO:tasks.workunit.client.0.smithi119.stdout:0/860: truncate d2/d6/d9/dab/f119 403519 0 2024-05-27T09:29:36.911 INFO:tasks.workunit.client.0.smithi119.stdout:7/936: write d7/df/d19/d46/d93/ff4 [393160,120656] 0 2024-05-27T09:29:36.911 INFO:tasks.workunit.client.0.smithi119.stdout:1/916: getdents d7/dd7 0 2024-05-27T09:29:36.917 INFO:tasks.workunit.client.0.smithi119.stdout:0/861: creat d2/d6/d9/d2f/f11f x:0 0 0 2024-05-27T09:29:36.917 INFO:tasks.workunit.client.0.smithi119.stdout:7/937: chown d7/df/d19/d1d/c38 33894 1 2024-05-27T09:29:36.917 INFO:tasks.workunit.client.0.smithi119.stdout:7/938: creat d7/d1a/d99/f129 x:0 0 0 2024-05-27T09:29:36.919 INFO:tasks.workunit.client.0.smithi119.stdout:7/939: creat d7/df/d19/d4d/f12a x:0 0 0 2024-05-27T09:29:36.922 INFO:tasks.workunit.client.0.smithi119.stdout:7/940: write d7/df/d19/d97/fcc [834018,106386] 0 2024-05-27T09:29:36.922 INFO:tasks.workunit.client.0.smithi119.stdout:7/941: truncate d7/df/d19/d97/d92/db7/dba/d42/f73 4372754 0 2024-05-27T09:29:36.925 INFO:tasks.workunit.client.0.smithi119.stdout:9/759: dread d18/d19/d1b/d2a/f7a [0,4194304] 0 2024-05-27T09:29:36.966 INFO:tasks.workunit.client.0.smithi119.stdout:7/942: dread d7/df/d19/d46/f85 [0,4194304] 0 2024-05-27T09:29:36.967 INFO:tasks.workunit.client.0.smithi119.stdout:2/875: fsync d2/f2d 0 2024-05-27T09:29:36.967 INFO:tasks.workunit.client.0.smithi119.stdout:3/991: dread db/dc9/d98/d7a/f9e [0,4194304] 0 2024-05-27T09:29:36.967 INFO:tasks.workunit.client.0.smithi119.stdout:6/820: dread d0/f1e [0,4194304] 0 2024-05-27T09:29:36.967 INFO:tasks.workunit.client.0.smithi119.stdout:4/930: dwrite d4/d6/d3e/d89/db2/fbd [4194304,4194304] 0 2024-05-27T09:29:36.967 INFO:tasks.workunit.client.0.smithi119.stdout:4/931: readlink d4/d1c/l83 0 2024-05-27T09:29:36.969 INFO:tasks.workunit.client.0.smithi119.stdout:7/943: rename d7/df/d19/d78/fb4 to d7/df/d19/d97/d92/db7/dba/d42/d64/de3/f12b 0 2024-05-27T09:29:36.970 INFO:tasks.workunit.client.0.smithi119.stdout:7/944: write d7/df/d19/d97/d37/f119 [782903,69949] 0 2024-05-27T09:29:36.997 INFO:tasks.workunit.client.0.smithi119.stdout:4/932: mknod d4/d6/d28/c12d 0 2024-05-27T09:29:36.997 INFO:tasks.workunit.client.0.smithi119.stdout:3/992: mknod db/d36/d45/d4c/d62/d141/c145 0 2024-05-27T09:29:37.003 INFO:tasks.workunit.client.0.smithi119.stdout:8/842: dwrite d3/d6e/d8e/fb0 [0,4194304] 0 2024-05-27T09:29:37.010 INFO:tasks.workunit.client.1.smithi138.stdout:2/318: dwrite d1/d15/d2d/d17/f58 [0,4194304] 0 2024-05-27T09:29:37.034 INFO:tasks.workunit.client.0.smithi119.stdout:5/951: dwrite d6/dd/d39/d19/fac [0,4194304] 0 2024-05-27T09:29:37.055 INFO:tasks.workunit.client.0.smithi119.stdout:5/952: write d6/dd/d39/d19/d3c/d52/f99 [3515441,98437] 0 2024-05-27T09:29:37.055 INFO:tasks.workunit.client.0.smithi119.stdout:2/876: dwrite d2/da/d2c/fe1 [0,4194304] 0 2024-05-27T09:29:37.055 INFO:tasks.workunit.client.0.smithi119.stdout:2/877: fsync d2/da/d8a/dd4/d89/d94/d87/ff3 0 2024-05-27T09:29:37.055 INFO:tasks.workunit.client.0.smithi119.stdout:5/953: write d6/db/de/d27/f7e [1861796,103572] 0 2024-05-27T09:29:37.055 INFO:tasks.workunit.client.0.smithi119.stdout:2/878: truncate d2/da/d8a/dd4/d89/d94/d87/dcf/de5/fea 763685 0 2024-05-27T09:29:37.061 INFO:tasks.workunit.client.1.smithi138.stdout:2/319: dwrite d1/d15/f4f [4194304,4194304] 0 2024-05-27T09:29:37.063 INFO:tasks.workunit.client.1.smithi138.stdout:2/320: write d1/d10/d3c/f62 [256895,76623] 0 2024-05-27T09:29:37.069 INFO:tasks.workunit.client.1.smithi138.stdout:2/321: mkdir d1/d15/d2d/d17/d25/d4d/d6d 0 2024-05-27T09:29:37.070 INFO:tasks.workunit.client.1.smithi138.stdout:2/322: chown d1/c36 7 1 2024-05-27T09:29:37.070 INFO:tasks.workunit.client.1.smithi138.stdout:2/323: write d1/d15/d2d/f3e [893617,104279] 0 2024-05-27T09:29:37.070 INFO:tasks.workunit.client.1.smithi138.stdout:2/324: truncate d1/d10/f64 558073 0 2024-05-27T09:29:37.091 INFO:tasks.workunit.client.0.smithi119.stdout:0/862: dwrite d2/d110/d96/fad [0,4194304] 0 2024-05-27T09:29:37.115 INFO:tasks.workunit.client.0.smithi119.stdout:4/933: symlink d4/de/d21/l12e 0 2024-05-27T09:29:37.115 INFO:tasks.workunit.client.0.smithi119.stdout:9/760: dwrite d18/d19/d1b/d40/d44/f70 [4194304,4194304] 0 2024-05-27T09:29:37.115 INFO:tasks.workunit.client.0.smithi119.stdout:3/993: symlink db/d36/d45/d49/d9d/d138/l146 0 2024-05-27T09:29:37.122 INFO:tasks.workunit.client.0.smithi119.stdout:6/821: dwrite d0/d2/d4e/feb [0,4194304] 0 2024-05-27T09:29:37.124 INFO:tasks.workunit.client.0.smithi119.stdout:8/843: creat d3/d8/d1b/d46/da4/d113/d66/f119 x:0 0 0 2024-05-27T09:29:37.135 INFO:tasks.workunit.client.1.smithi138.stdout:8/324: sync 2024-05-27T09:29:37.136 INFO:tasks.workunit.client.0.smithi119.stdout:2/879: readlink d2/l17 0 2024-05-27T09:29:37.147 INFO:tasks.workunit.client.0.smithi119.stdout:0/863: truncate d2/d6/d75/fa4 2682934 0 2024-05-27T09:29:37.150 INFO:tasks.workunit.client.0.smithi119.stdout:3/994: creat db/d36/d45/f147 x:0 0 0 2024-05-27T09:29:37.151 INFO:tasks.workunit.client.0.smithi119.stdout:6/822: mknod d0/d2/d57/d37/d74/df1/c124 0 2024-05-27T09:29:37.151 INFO:tasks.workunit.client.1.smithi138.stdout:2/325: dwrite d1/d15/d2d/d31/d4b/f51 [0,4194304] 0 2024-05-27T09:29:37.157 INFO:tasks.workunit.client.1.smithi138.stdout:2/326: getdents d1/d15/d2d/d1d 0 2024-05-27T09:29:37.162 INFO:tasks.workunit.client.0.smithi119.stdout:8/844: creat d3/d8/d1b/d46/da4/d113/f11a x:0 0 0 2024-05-27T09:29:37.163 INFO:tasks.workunit.client.0.smithi119.stdout:8/845: creat d3/d8/d1b/d46/da4/dbb/f11b x:0 0 0 2024-05-27T09:29:37.163 INFO:tasks.workunit.client.0.smithi119.stdout:8/846: readlink d3/de/l5e 0 2024-05-27T09:29:37.163 INFO:tasks.workunit.client.1.smithi138.stdout:8/325: read d3/d8/d32/d34/d38/f5d [3637409,81387] 0 2024-05-27T09:29:37.163 INFO:tasks.workunit.client.1.smithi138.stdout:8/326: fsync d3/f50 0 2024-05-27T09:29:37.163 INFO:tasks.workunit.client.0.smithi119.stdout:2/880: chown d2/l23 7 1 2024-05-27T09:29:37.165 INFO:tasks.workunit.client.0.smithi119.stdout:3/995: rename db/d7f/dd2/c11b to db/d7f/dd2/c148 0 2024-05-27T09:29:37.165 INFO:tasks.workunit.client.0.smithi119.stdout:3/996: fdatasync db/d36/d45/d4d/d108/f109 0 2024-05-27T09:29:37.166 INFO:tasks.workunit.client.0.smithi119.stdout:3/997: creat db/d7f/f149 x:0 0 0 2024-05-27T09:29:37.166 INFO:tasks.workunit.client.0.smithi119.stdout:6/823: symlink d0/d2/d57/d37/d74/df1/l125 0 2024-05-27T09:29:37.170 INFO:tasks.workunit.client.0.smithi119.stdout:9/761: rmdir d18/d19/d9b 39 2024-05-27T09:29:37.170 INFO:tasks.workunit.client.0.smithi119.stdout:9/762: chown d18/d19/d1b/d40/d44/dcb 23995504 1 2024-05-27T09:29:37.171 INFO:tasks.workunit.client.0.smithi119.stdout:5/954: dwrite d6/dd/d46/d47/d4b/dd8/df2/f10a [0,4194304] 0 2024-05-27T09:29:37.173 INFO:tasks.workunit.client.0.smithi119.stdout:8/847: creat d3/d8/d1b/f11c x:0 0 0 2024-05-27T09:29:37.177 INFO:tasks.workunit.client.0.smithi119.stdout:9/763: truncate d18/d19/d1b/f5d 3192710 0 2024-05-27T09:29:37.179 INFO:tasks.workunit.client.1.smithi138.stdout:4/409: sync 2024-05-27T09:29:37.179 INFO:tasks.workunit.client.1.smithi138.stdout:3/382: sync 2024-05-27T09:29:37.181 INFO:tasks.workunit.client.1.smithi138.stdout:5/274: sync 2024-05-27T09:29:37.181 INFO:tasks.workunit.client.1.smithi138.stdout:0/341: sync 2024-05-27T09:29:37.181 INFO:tasks.workunit.client.1.smithi138.stdout:7/321: sync 2024-05-27T09:29:37.182 INFO:tasks.workunit.client.1.smithi138.stdout:3/383: creat d0/d2/da/df/d10/d2a/f7e x:0 0 0 2024-05-27T09:29:37.185 INFO:tasks.workunit.client.0.smithi119.stdout:5/955: creat d6/f14e x:0 0 0 2024-05-27T09:29:37.188 INFO:tasks.workunit.client.1.smithi138.stdout:4/410: mknod d7/df/d35/d4c/d59/d5d/d64/d6a/c8f 0 2024-05-27T09:29:37.192 INFO:tasks.workunit.client.1.smithi138.stdout:0/342: symlink da/d17/d1d/d1f/d42/d4a/d53/l76 0 2024-05-27T09:29:37.193 INFO:tasks.workunit.client.1.smithi138.stdout:0/343: write da/d2a/d55/f5c [333704,61596] 0 2024-05-27T09:29:37.193 INFO:tasks.workunit.client.1.smithi138.stdout:7/322: mknod d0/d7/c69 0 2024-05-27T09:29:37.194 INFO:tasks.workunit.client.0.smithi119.stdout:8/848: getdents d3/d8/d1b/d46/da4/d113/db7/de0 0 2024-05-27T09:29:37.195 INFO:tasks.workunit.client.1.smithi138.stdout:7/323: mknod d0/d23/d3a/c6a 0 2024-05-27T09:29:37.196 INFO:tasks.workunit.client.1.smithi138.stdout:7/324: creat d0/d7/f6b x:0 0 0 2024-05-27T09:29:37.196 INFO:tasks.workunit.client.1.smithi138.stdout:7/325: chown d0/f45 1871 1 2024-05-27T09:29:37.197 INFO:tasks.workunit.client.1.smithi138.stdout:7/326: dread - d0/d2b/f5e zero size 2024-05-27T09:29:37.198 INFO:tasks.workunit.client.1.smithi138.stdout:7/327: symlink d0/d2b/l6c 0 2024-05-27T09:29:37.201 INFO:tasks.workunit.client.1.smithi138.stdout:3/384: rmdir d0/d2/da/df/d10/d2a 39 2024-05-27T09:29:37.201 INFO:tasks.workunit.client.1.smithi138.stdout:3/385: read - d0/d2/da/df/f6f zero size 2024-05-27T09:29:37.201 INFO:tasks.workunit.client.1.smithi138.stdout:3/386: fdatasync d0/d2/da/df/d10/d32/f5b 0 2024-05-27T09:29:37.201 INFO:tasks.workunit.client.1.smithi138.stdout:3/387: creat d0/d3f/f7f x:0 0 0 2024-05-27T09:29:37.202 INFO:tasks.workunit.client.1.smithi138.stdout:3/388: write d0/d3f/f7b [629973,87758] 0 2024-05-27T09:29:37.202 INFO:tasks.workunit.client.1.smithi138.stdout:3/389: write d0/d2/da/f6b [55573,16047] 0 2024-05-27T09:29:37.202 INFO:tasks.workunit.client.1.smithi138.stdout:7/328: symlink d0/d23/d59/l6d 0 2024-05-27T09:29:37.202 INFO:tasks.workunit.client.1.smithi138.stdout:7/329: truncate d0/d2b/f35 1013526 0 2024-05-27T09:29:37.202 INFO:tasks.workunit.client.1.smithi138.stdout:5/275: rename d3/d12/d16/d1d/d22/d2b/f40 to d3/f5b 0 2024-05-27T09:29:37.204 INFO:tasks.workunit.client.1.smithi138.stdout:3/390: mkdir d0/d80 0 2024-05-27T09:29:37.204 INFO:tasks.workunit.client.1.smithi138.stdout:3/391: creat d0/d2/da/df/d10/f81 x:0 0 0 2024-05-27T09:29:37.206 INFO:tasks.workunit.client.1.smithi138.stdout:7/330: creat d0/d7/d64/f6e x:0 0 0 2024-05-27T09:29:37.206 INFO:tasks.workunit.client.1.smithi138.stdout:7/331: fdatasync d0/d7/d64/f6e 0 2024-05-27T09:29:37.206 INFO:tasks.workunit.client.0.smithi119.stdout:5/956: unlink d6/db/dc/fde 0 2024-05-27T09:29:37.207 INFO:tasks.workunit.client.1.smithi138.stdout:3/392: symlink d0/d2/da/df/d10/d2a/l82 0 2024-05-27T09:29:37.208 INFO:tasks.workunit.client.1.smithi138.stdout:7/332: mknod d0/d23/d3a/c6f 0 2024-05-27T09:29:37.208 INFO:tasks.workunit.client.0.smithi119.stdout:4/934: dwrite d4/de/d66/f11a [0,4194304] 0 2024-05-27T09:29:37.208 INFO:tasks.workunit.client.0.smithi119.stdout:4/935: chown d4/d6/d81/dce/d88/c91 3 1 2024-05-27T09:29:37.209 INFO:tasks.workunit.client.1.smithi138.stdout:4/411: rename d7/f25 to d7/df/d35/d4c/d59/f90 0 2024-05-27T09:29:37.216 INFO:tasks.workunit.client.1.smithi138.stdout:5/276: unlink d3/d12/d32/f47 0 2024-05-27T09:29:37.219 INFO:tasks.workunit.client.0.smithi119.stdout:7/945: rename d7/df/d19/d97/d92/db7/dba/d42/d64/de3 to d7/d1a/d107/d12c 0 2024-05-27T09:29:37.226 INFO:tasks.workunit.client.0.smithi119.stdout:9/764: unlink d18/d19/f59 0 2024-05-27T09:29:37.227 INFO:tasks.workunit.client.1.smithi138.stdout:4/412: creat d7/df/d35/d4c/f91 x:0 0 0 2024-05-27T09:29:37.228 INFO:tasks.workunit.client.1.smithi138.stdout:4/413: chown d7/c2f 9 1 2024-05-27T09:29:37.228 INFO:tasks.workunit.client.1.smithi138.stdout:4/414: write d7/df/d5b/f86 [810177,121093] 0 2024-05-27T09:29:37.228 INFO:tasks.workunit.client.1.smithi138.stdout:4/415: creat d7/df/d35/f92 x:0 0 0 2024-05-27T09:29:37.249 INFO:tasks.workunit.client.0.smithi119.stdout:0/864: rename d2/d6/d23/db8/d118/dfa to d2/d6/d9/d24/d30/dcc/d120 0 2024-05-27T09:29:37.257 INFO:tasks.workunit.client.0.smithi119.stdout:0/865: write d2/d1a/f21 [2502651,52522] 0 2024-05-27T09:29:37.257 INFO:tasks.workunit.client.0.smithi119.stdout:5/957: mkdir d6/db/de/d27/d77/d14f 0 2024-05-27T09:29:37.257 INFO:tasks.workunit.client.0.smithi119.stdout:2/881: mkdir d2/da/d8a/d12d 0 2024-05-27T09:29:37.257 INFO:tasks.workunit.client.0.smithi119.stdout:8/849: dread d3/d8/d4d/f59 [0,4194304] 0 2024-05-27T09:29:37.259 INFO:tasks.workunit.client.1.smithi138.stdout:7/333: write d0/f24 [5950940,61772] 0 2024-05-27T09:29:37.265 INFO:tasks.workunit.client.1.smithi138.stdout:7/334: mknod d0/d23/d59/c70 0 2024-05-27T09:29:37.269 INFO:tasks.workunit.client.1.smithi138.stdout:8/327: dwrite d3/d8/d32/f59 [0,4194304] 0 2024-05-27T09:29:37.270 INFO:tasks.workunit.client.1.smithi138.stdout:2/327: dwrite d1/d15/d2d/d2e/f50 [0,4194304] 0 2024-05-27T09:29:37.278 INFO:tasks.workunit.client.0.smithi119.stdout:3/998: rename db/dc9/d98/db9/dd1/l102 to db/d36/d45/d4c/d62/d13/d33/dfc/l14a 0 2024-05-27T09:29:37.278 INFO:tasks.workunit.client.0.smithi119.stdout:9/765: rename d18/d19/d1b/d29/dc4/fd0 to d18/d19/d1b/d2a/d9f/fec 0 2024-05-27T09:29:37.279 INFO:tasks.workunit.client.0.smithi119.stdout:6/824: dwrite d0/f9a [0,4194304] 0 2024-05-27T09:29:37.282 INFO:tasks.workunit.client.1.smithi138.stdout:8/328: creat d3/d8/d32/d34/d38/f63 x:0 0 0 2024-05-27T09:29:37.285 INFO:tasks.workunit.client.0.smithi119.stdout:2/882: dread d2/da/d6f/f111 [0,4194304] 0 2024-05-27T09:29:37.285 INFO:tasks.workunit.client.0.smithi119.stdout:8/850: rmdir d3/d39 39 2024-05-27T09:29:37.285 INFO:tasks.workunit.client.0.smithi119.stdout:8/851: truncate d3/d8/d1b/d46/da4/d113/d66/f105 781948 0 2024-05-27T09:29:37.286 INFO:tasks.workunit.client.0.smithi119.stdout:8/852: truncate d3/d8/d1b/d46/da4/d113/fd6 4877849 0 2024-05-27T09:29:37.286 INFO:tasks.workunit.client.1.smithi138.stdout:2/328: unlink d1/d15/d2d/d31/d4b/l69 0 2024-05-27T09:29:37.293 INFO:tasks.workunit.client.0.smithi119.stdout:1/917: sync 2024-05-27T09:29:37.293 INFO:tasks.workunit.client.1.smithi138.stdout:3/393: dwrite d0/fd [4194304,4194304] 0 2024-05-27T09:29:37.295 INFO:tasks.workunit.client.1.smithi138.stdout:3/394: creat d0/d2/da/df/d10/d2a/f83 x:0 0 0 2024-05-27T09:29:37.295 INFO:tasks.workunit.client.1.smithi138.stdout:3/395: fdatasync d0/d3f/d50/f63 0 2024-05-27T09:29:37.295 INFO:tasks.workunit.client.1.smithi138.stdout:3/396: readlink d0/l5 0 2024-05-27T09:29:37.298 INFO:tasks.workunit.client.1.smithi138.stdout:3/397: link d0/f12 d0/d3f/f84 0 2024-05-27T09:29:37.298 INFO:tasks.workunit.client.0.smithi119.stdout:6/825: unlink d0/d2/d57/d37/d74/df1/d109/ff6 0 2024-05-27T09:29:37.301 INFO:tasks.workunit.client.1.smithi138.stdout:3/398: creat d0/d2/da/df/f85 x:0 0 0 2024-05-27T09:29:37.304 INFO:tasks.workunit.client.1.smithi138.stdout:3/399: chown d0/d3f/f7f 29 1 2024-05-27T09:29:37.304 INFO:tasks.workunit.client.1.smithi138.stdout:0/344: dwrite da/d17/d1d/d1f/f21 [0,4194304] 0 2024-05-27T09:29:37.309 INFO:tasks.workunit.client.1.smithi138.stdout:7/335: dread d0/d23/d49/f4a [0,4194304] 0 2024-05-27T09:29:37.310 INFO:tasks.workunit.client.1.smithi138.stdout:8/329: dread d3/d8/d28/f2e [0,4194304] 0 2024-05-27T09:29:37.311 INFO:tasks.workunit.client.0.smithi119.stdout:9/766: creat d18/d19/d61/fed x:0 0 0 2024-05-27T09:29:37.311 INFO:tasks.workunit.client.0.smithi119.stdout:7/946: dwrite d7/df/d19/d97/fbf [0,4194304] 0 2024-05-27T09:29:37.311 INFO:tasks.workunit.client.0.smithi119.stdout:3/999: dread db/d5b/f82 [0,4194304] 0 2024-05-27T09:29:37.312 INFO:tasks.workunit.client.1.smithi138.stdout:7/336: link d0/l61 d0/d7/d43/l71 0 2024-05-27T09:29:37.314 INFO:tasks.workunit.client.0.smithi119.stdout:2/883: creat d2/da/d8a/dd4/d89/d94/d87/f12e x:0 0 0 2024-05-27T09:29:37.314 INFO:tasks.workunit.client.1.smithi138.stdout:8/330: unlink d3/d8/d11/l35 0 2024-05-27T09:29:37.315 INFO:tasks.workunit.client.0.smithi119.stdout:8/853: mknod d3/d6e/dcf/c11d 0 2024-05-27T09:29:37.315 INFO:tasks.workunit.client.0.smithi119.stdout:6/826: truncate d0/d2/d57/d37/f43 383756 0 2024-05-27T09:29:37.316 INFO:tasks.workunit.client.1.smithi138.stdout:8/331: readlink d3/d8/d24/l47 0 2024-05-27T09:29:37.316 INFO:tasks.workunit.client.1.smithi138.stdout:8/332: dread - d3/d8/f33 zero size 2024-05-27T09:29:37.320 INFO:tasks.workunit.client.0.smithi119.stdout:4/936: dwrite d4/de/d66/f68 [0,4194304] 0 2024-05-27T09:29:37.320 INFO:tasks.workunit.client.0.smithi119.stdout:4/937: dread - d4/de/d21/d40/d47/d5c/f129 zero size 2024-05-27T09:29:37.320 INFO:tasks.workunit.client.0.smithi119.stdout:7/947: truncate d7/df/d19/d9a/da6/dda/f48 1455708 0 2024-05-27T09:29:37.320 INFO:tasks.workunit.client.0.smithi119.stdout:7/948: readlink d7/df/d19/d97/d92/db7/dba/l6c 0 2024-05-27T09:29:37.320 INFO:tasks.workunit.client.0.smithi119.stdout:7/949: chown d7/df/d19/d9a/da6/dda/l8e 706584406 1 2024-05-27T09:29:37.323 INFO:tasks.workunit.client.0.smithi119.stdout:2/884: mkdir d2/da/d8a/dd4/d89/de8/d12f 0 2024-05-27T09:29:37.323 INFO:tasks.workunit.client.1.smithi138.stdout:4/416: dwrite d7/df/f1a [0,4194304] 0 2024-05-27T09:29:37.327 INFO:tasks.workunit.client.1.smithi138.stdout:4/417: rename d7/d27/d3a/l3e to d7/df/l93 0 2024-05-27T09:29:37.330 INFO:tasks.workunit.client.0.smithi119.stdout:8/854: rmdir d3/d8/d1b/d46/d92 39 2024-05-27T09:29:37.330 INFO:tasks.workunit.client.0.smithi119.stdout:8/855: chown d3/d8/d1b/d77/d95/d106 2017 1 2024-05-27T09:29:37.332 INFO:tasks.workunit.client.0.smithi119.stdout:5/958: dwrite d6/dd/d46/d47/d4b/f122 [0,4194304] 0 2024-05-27T09:29:37.334 INFO:tasks.workunit.client.1.smithi138.stdout:0/345: dread da/d2a/f30 [0,4194304] 0 2024-05-27T09:29:37.335 INFO:tasks.workunit.client.1.smithi138.stdout:2/329: rmdir d1/d15/d2d/d31/d4b 39 2024-05-27T09:29:37.338 INFO:tasks.workunit.client.0.smithi119.stdout:6/827: link d0/d2/f7f d0/d2/f126 0 2024-05-27T09:29:37.340 INFO:tasks.workunit.client.1.smithi138.stdout:2/330: write d1/d15/d2d/f3e [31307,4691] 0 2024-05-27T09:29:37.340 INFO:tasks.workunit.client.1.smithi138.stdout:4/418: link d7/df/d5b/f7c d7/df/d35/d4c/d59/d5d/d64/d6a/f94 0 2024-05-27T09:29:37.340 INFO:tasks.workunit.client.1.smithi138.stdout:4/419: write d7/df/d35/f5a [945829,102724] 0 2024-05-27T09:29:37.340 INFO:tasks.workunit.client.1.smithi138.stdout:4/420: dread - d7/df/f2e zero size 2024-05-27T09:29:37.340 INFO:tasks.workunit.client.1.smithi138.stdout:4/421: dread - d7/df/d35/d4c/d59/d5d/d64/d6a/f6d zero size 2024-05-27T09:29:37.341 INFO:tasks.workunit.client.0.smithi119.stdout:9/767: getdents d18/d19/d1b/d2a/d63 0 2024-05-27T09:29:37.342 INFO:tasks.workunit.client.1.smithi138.stdout:2/331: rename d1/c36 to d1/d10/d52/c6e 0 2024-05-27T09:29:37.343 INFO:tasks.workunit.client.1.smithi138.stdout:0/346: symlink da/d2a/d3f/l77 0 2024-05-27T09:29:37.346 INFO:tasks.workunit.client.1.smithi138.stdout:7/337: dread d0/d23/d49/f4a [0,4194304] 0 2024-05-27T09:29:37.346 INFO:tasks.workunit.client.1.smithi138.stdout:3/400: write d0/d2/da/df/d10/d2a/f42 [3745473,108516] 0 2024-05-27T09:29:37.346 INFO:tasks.workunit.client.1.smithi138.stdout:3/401: fdatasync d0/d2/da/df/d10/d32/f1d 0 2024-05-27T09:29:37.346 INFO:tasks.workunit.client.1.smithi138.stdout:3/402: readlink d0/d2/da/df/d10/d32/d37/l3e 0 2024-05-27T09:29:37.347 INFO:tasks.workunit.client.0.smithi119.stdout:4/938: link d4/de/d21/d40/f6a d4/d6/d3e/df5/d104/f12f 0 2024-05-27T09:29:37.349 INFO:tasks.workunit.client.1.smithi138.stdout:7/338: write d0/d2b/f5f [20633,73475] 0 2024-05-27T09:29:37.349 INFO:tasks.workunit.client.1.smithi138.stdout:4/422: rename d7/f42 to d7/df/d5b/f95 0 2024-05-27T09:29:37.353 INFO:tasks.workunit.client.0.smithi119.stdout:2/885: symlink d2/da/d8a/dd4/d89/d94/de2/l130 0 2024-05-27T09:29:37.353 INFO:tasks.workunit.client.1.smithi138.stdout:3/403: mknod d0/d2/da/df/d10/d2a/d61/d73/c86 0 2024-05-27T09:29:37.353 INFO:tasks.workunit.client.1.smithi138.stdout:3/404: creat d0/d2/da/df/d10/d2a/f87 x:0 0 0 2024-05-27T09:29:37.353 INFO:tasks.workunit.client.1.smithi138.stdout:3/405: creat d0/d3f/f88 x:0 0 0 2024-05-27T09:29:37.358 INFO:tasks.workunit.client.0.smithi119.stdout:8/856: truncate d3/d8/d1b/d77/fdb 377191 0 2024-05-27T09:29:37.365 INFO:tasks.workunit.client.1.smithi138.stdout:4/423: creat d7/df/d35/d4c/d59/d5d/d64/f96 x:0 0 0 2024-05-27T09:29:37.369 INFO:tasks.workunit.client.1.smithi138.stdout:2/332: dread d1/d2f/f33 [0,4194304] 0 2024-05-27T09:29:37.369 INFO:tasks.workunit.client.1.smithi138.stdout:2/333: chown d1/d15/d2d/d17/d25/d4d/d4e/d60 867 1 2024-05-27T09:29:37.371 INFO:tasks.workunit.client.1.smithi138.stdout:4/424: unlink d7/df/d35/c5f 0 2024-05-27T09:29:37.374 INFO:tasks.workunit.client.0.smithi119.stdout:6/828: creat d0/d2/d57/d37/f127 x:0 0 0 2024-05-27T09:29:37.374 INFO:tasks.workunit.client.0.smithi119.stdout:9/768: mknod d18/db4/cee 0 2024-05-27T09:29:37.380 INFO:tasks.workunit.client.0.smithi119.stdout:4/939: symlink d4/d23/d24/d5b/da8/dda/l130 0 2024-05-27T09:29:37.383 INFO:tasks.workunit.client.1.smithi138.stdout:1/378: sync 2024-05-27T09:29:37.384 INFO:tasks.workunit.client.1.smithi138.stdout:9/372: sync 2024-05-27T09:29:37.385 INFO:tasks.workunit.client.0.smithi119.stdout:2/886: truncate d2/da/d2c/fe1 3199286 0 2024-05-27T09:29:37.387 INFO:tasks.workunit.client.0.smithi119.stdout:5/959: unlink d6/db/de/d27/d77/fb9 0 2024-05-27T09:29:37.390 INFO:tasks.workunit.client.0.smithi119.stdout:2/887: link d2/da/d8a/d8f/d11e/f125 d2/da/d8a/dd4/d89/de8/d12f/f131 0 2024-05-27T09:29:37.391 INFO:tasks.workunit.client.0.smithi119.stdout:2/888: chown d2/da/d8a/dd4/d69/f9d 747341 1 2024-05-27T09:29:37.391 INFO:tasks.workunit.client.0.smithi119.stdout:2/889: stat d2/da/d8a/d8f/f105 0 2024-05-27T09:29:37.394 INFO:tasks.workunit.client.0.smithi119.stdout:9/769: link d18/d19/f3f d18/d19/d1b/d29/dc4/fef 0 2024-05-27T09:29:37.397 INFO:tasks.workunit.client.0.smithi119.stdout:2/890: mkdir d2/d1a/d132 0 2024-05-27T09:29:37.404 INFO:tasks.workunit.client.0.smithi119.stdout:9/770: rename d18/d19/d1b/d29/fc9 to d18/d19/d1b/d40/d7b/dc6/ff0 0 2024-05-27T09:29:37.406 INFO:tasks.workunit.client.0.smithi119.stdout:5/960: dread d6/dd/d39/d19/d3c/f43 [0,4194304] 0 2024-05-27T09:29:37.406 INFO:tasks.workunit.client.0.smithi119.stdout:5/961: write d6/dd/f9b [365903,62675] 0 2024-05-27T09:29:37.406 INFO:tasks.workunit.client.1.smithi138.stdout:1/379: write d4/d1a/d2e/f41 [3802997,8063] 0 2024-05-27T09:29:37.408 INFO:tasks.workunit.client.0.smithi119.stdout:8/857: dread d3/d39/f52 [0,4194304] 0 2024-05-27T09:29:37.410 INFO:tasks.workunit.client.1.smithi138.stdout:1/380: mknod d4/d1a/d2e/d4a/c76 0 2024-05-27T09:29:37.412 INFO:tasks.workunit.client.0.smithi119.stdout:2/891: rename d2/da/d8a/dd4/d89/db0/f101 to d2/da/d8a/dd4/d89/d94/f133 0 2024-05-27T09:29:37.412 INFO:tasks.workunit.client.1.smithi138.stdout:4/425: dread d7/d27/f34 [0,4194304] 0 2024-05-27T09:29:37.413 INFO:tasks.workunit.client.1.smithi138.stdout:9/373: write d6/da/d26/d28/f58 [2498053,67408] 0 2024-05-27T09:29:37.415 INFO:tasks.workunit.client.0.smithi119.stdout:8/858: creat d3/d8/d1b/d46/da4/d113/d60/f11e x:0 0 0 2024-05-27T09:29:37.417 INFO:tasks.workunit.client.0.smithi119.stdout:8/859: rename d3/c44 to d3/d8/d1b/d46/da4/d113/d66/c11f 0 2024-05-27T09:29:37.419 INFO:tasks.workunit.client.0.smithi119.stdout:8/860: mkdir d3/d8/d1b/d46/da4/d113/d74/d120 0 2024-05-27T09:29:37.419 INFO:tasks.workunit.client.0.smithi119.stdout:5/962: dread d6/dd/f33 [0,4194304] 0 2024-05-27T09:29:37.419 INFO:tasks.workunit.client.0.smithi119.stdout:5/963: read - d6/db/dd4/f123 zero size 2024-05-27T09:29:37.419 INFO:tasks.workunit.client.0.smithi119.stdout:5/964: write d6/dd/d39/d74/feb [326061,73806] 0 2024-05-27T09:29:37.419 INFO:tasks.workunit.client.1.smithi138.stdout:4/426: symlink d7/df/d35/d4c/l97 0 2024-05-27T09:29:37.420 INFO:tasks.workunit.client.1.smithi138.stdout:4/427: chown d7/df/d35/d4c/c75 480173969 1 2024-05-27T09:29:37.420 INFO:tasks.workunit.client.1.smithi138.stdout:4/428: fsync d7/df/d35/f92 0 2024-05-27T09:29:37.420 INFO:tasks.workunit.client.1.smithi138.stdout:4/429: write d7/df/d35/f6c [633971,85926] 0 2024-05-27T09:29:37.420 INFO:tasks.workunit.client.1.smithi138.stdout:9/374: creat d6/da/d29/f74 x:0 0 0 2024-05-27T09:29:37.423 INFO:tasks.workunit.client.0.smithi119.stdout:5/965: symlink d6/dd/d39/d74/l150 0 2024-05-27T09:29:37.423 INFO:tasks.workunit.client.0.smithi119.stdout:5/966: truncate d6/dd/d39/d19/d3c/fb2 461020 0 2024-05-27T09:29:37.423 INFO:tasks.workunit.client.0.smithi119.stdout:5/967: fdatasync d6/db/de/d27/f3f 0 2024-05-27T09:29:37.424 INFO:tasks.workunit.client.1.smithi138.stdout:9/375: mkdir d6/da/d2f/d35/d55/d75 0 2024-05-27T09:29:37.433 INFO:tasks.workunit.client.0.smithi119.stdout:9/771: dread d18/d19/d1b/d40/f62 [0,4194304] 0 2024-05-27T09:29:37.439 INFO:tasks.workunit.client.1.smithi138.stdout:3/406: dwrite d0/d2/da/df/d10/d32/f5b [0,4194304] 0 2024-05-27T09:29:37.444 INFO:tasks.workunit.client.1.smithi138.stdout:0/347: dwrite da/d17/d1d/d43/f48 [0,4194304] 0 2024-05-27T09:29:37.444 INFO:tasks.workunit.client.1.smithi138.stdout:0/348: chown da/d17/d1d/d1f/d60/c68 6283 1 2024-05-27T09:29:37.445 INFO:tasks.workunit.client.1.smithi138.stdout:7/339: dwrite d0/d23/f58 [0,4194304] 0 2024-05-27T09:29:37.450 INFO:tasks.workunit.client.1.smithi138.stdout:9/376: rmdir d6/db 39 2024-05-27T09:29:37.451 INFO:tasks.workunit.client.1.smithi138.stdout:0/349: truncate da/f29 2829876 0 2024-05-27T09:29:37.451 INFO:tasks.workunit.client.1.smithi138.stdout:0/350: truncate da/d2a/d55/f5c 559126 0 2024-05-27T09:29:37.451 INFO:tasks.workunit.client.1.smithi138.stdout:9/377: creat d6/da/d2f/d35/d55/f76 x:0 0 0 2024-05-27T09:29:37.458 INFO:tasks.workunit.client.1.smithi138.stdout:3/407: mknod d0/d2/da/df/d10/d2a/d48/c89 0 2024-05-27T09:29:37.459 INFO:tasks.workunit.client.1.smithi138.stdout:3/408: dread - d0/d2/da/df/d10/d2a/f87 zero size 2024-05-27T09:29:37.459 INFO:tasks.workunit.client.1.smithi138.stdout:3/409: creat d0/d2/da/f8a x:0 0 0 2024-05-27T09:29:37.460 INFO:tasks.workunit.client.0.smithi119.stdout:5/968: write d6/db/f36 [7684512,88716] 0 2024-05-27T09:29:37.460 INFO:tasks.workunit.client.0.smithi119.stdout:5/969: fsync d6/db/de/d27/d77/fbc 0 2024-05-27T09:29:37.462 INFO:tasks.workunit.client.1.smithi138.stdout:7/340: creat d0/d23/d49/f72 x:0 0 0 2024-05-27T09:29:37.466 INFO:tasks.workunit.client.0.smithi119.stdout:5/970: unlink d6/dd/d39/d19/l66 0 2024-05-27T09:29:37.471 INFO:tasks.workunit.client.1.smithi138.stdout:1/381: dread d4/d48/d39/f45 [0,4194304] 0 2024-05-27T09:29:37.471 INFO:tasks.workunit.client.1.smithi138.stdout:3/410: unlink d0/d2/da/df/d10/d2a/d48/f49 0 2024-05-27T09:29:37.471 INFO:tasks.workunit.client.1.smithi138.stdout:1/382: dread - d4/d48/d22/d65/f75 zero size 2024-05-27T09:29:37.471 INFO:tasks.workunit.client.1.smithi138.stdout:3/411: dread - d0/d2/da/df/d10/d32/f3c zero size 2024-05-27T09:29:37.471 INFO:tasks.workunit.client.1.smithi138.stdout:3/412: write d0/d2/da/df/d10/d2a/d48/f5f [983515,11043] 0 2024-05-27T09:29:37.471 INFO:tasks.workunit.client.1.smithi138.stdout:3/413: truncate d0/d2/da/df/d10/d2a/d48/f79 404396 0 2024-05-27T09:29:37.472 INFO:tasks.workunit.client.1.smithi138.stdout:7/341: mknod d0/d23/d3c/d3f/c73 0 2024-05-27T09:29:37.472 INFO:tasks.workunit.client.1.smithi138.stdout:7/342: fsync d0/f10 0 2024-05-27T09:29:37.472 INFO:tasks.workunit.client.1.smithi138.stdout:7/343: fsync d0/d23/d3c/f42 0 2024-05-27T09:29:37.474 INFO:tasks.workunit.client.1.smithi138.stdout:0/351: read da/f29 [1893625,36277] 0 2024-05-27T09:29:37.478 INFO:tasks.workunit.client.1.smithi138.stdout:4/430: dwrite f3 [4194304,4194304] 0 2024-05-27T09:29:37.478 INFO:tasks.workunit.client.1.smithi138.stdout:4/431: dread - d7/df/d35/d4c/d59/d5d/d64/f96 zero size 2024-05-27T09:29:37.478 INFO:tasks.workunit.client.1.smithi138.stdout:4/432: truncate d7/df/d35/d4c/d59/d5d/f87 894679 0 2024-05-27T09:29:37.478 INFO:tasks.workunit.client.1.smithi138.stdout:4/433: fsync d7/df/d35/d4c/d59/f90 0 2024-05-27T09:29:37.478 INFO:tasks.workunit.client.1.smithi138.stdout:4/434: creat d7/d27/f98 x:0 0 0 2024-05-27T09:29:37.478 INFO:tasks.workunit.client.1.smithi138.stdout:4/435: stat d7/df/d35/d4c/d59/d5d/d64/d6a/f76 0 2024-05-27T09:29:37.478 INFO:tasks.workunit.client.1.smithi138.stdout:0/352: mknod da/d17/d1d/d1f/d4f/c78 0 2024-05-27T09:29:37.479 INFO:tasks.workunit.client.0.smithi119.stdout:9/772: write d18/d19/d1b/d40/d44/f93 [777206,48835] 0 2024-05-27T09:29:37.480 INFO:tasks.workunit.client.1.smithi138.stdout:7/344: mknod d0/d23/c74 0 2024-05-27T09:29:37.481 INFO:tasks.workunit.client.1.smithi138.stdout:0/353: rmdir da/d17/d1d 39 2024-05-27T09:29:37.481 INFO:tasks.workunit.client.1.smithi138.stdout:0/354: chown da/c39 53523 1 2024-05-27T09:29:37.482 INFO:tasks.workunit.client.1.smithi138.stdout:7/345: symlink d0/l75 0 2024-05-27T09:29:37.485 INFO:tasks.workunit.client.1.smithi138.stdout:1/383: symlink d4/l77 0 2024-05-27T09:29:37.485 INFO:tasks.workunit.client.1.smithi138.stdout:1/384: readlink d4/d3c/l5d 0 2024-05-27T09:29:37.485 INFO:tasks.workunit.client.1.smithi138.stdout:1/385: write d4/d48/d22/d65/f6f [781396,57520] 0 2024-05-27T09:29:37.486 INFO:tasks.workunit.client.0.smithi119.stdout:9/773: creat d18/d19/d1b/d40/d7b/ff1 x:0 0 0 2024-05-27T09:29:37.486 INFO:tasks.workunit.client.1.smithi138.stdout:9/378: dread d6/da/d29/f6d [0,4194304] 0 2024-05-27T09:29:37.486 INFO:tasks.workunit.client.1.smithi138.stdout:9/379: dread - d6/da/d2f/d35/d55/f76 zero size 2024-05-27T09:29:37.487 INFO:tasks.workunit.client.1.smithi138.stdout:0/355: creat da/d17/d1d/d1f/d71/f79 x:0 0 0 2024-05-27T09:29:37.487 INFO:tasks.workunit.client.1.smithi138.stdout:0/356: chown da/d17/d1d/d1f/d4f 2561 1 2024-05-27T09:29:37.489 INFO:tasks.workunit.client.1.smithi138.stdout:0/357: rename da/d17/l49 to da/d17/d1d/d1f/l7a 0 2024-05-27T09:29:37.491 INFO:tasks.workunit.client.1.smithi138.stdout:0/358: creat da/d17/d1d/d1f/f7b x:0 0 0 2024-05-27T09:29:37.493 INFO:tasks.workunit.client.1.smithi138.stdout:0/359: creat da/d2a/d3f/f7c x:0 0 0 2024-05-27T09:29:37.493 INFO:tasks.workunit.client.1.smithi138.stdout:0/360: fsync da/d17/d1d/d1f/f21 0 2024-05-27T09:29:37.493 INFO:tasks.workunit.client.1.smithi138.stdout:0/361: write da/d17/d1d/d1f/f3b [505516,115972] 0 2024-05-27T09:29:37.495 INFO:tasks.workunit.client.1.smithi138.stdout:0/362: mknod da/d17/d1d/d6a/c7d 0 2024-05-27T09:29:37.496 INFO:tasks.workunit.client.1.smithi138.stdout:0/363: fdatasync da/d2a/d2b/f4d 0 2024-05-27T09:29:37.499 INFO:tasks.workunit.client.1.smithi138.stdout:4/436: dread d7/f18 [0,4194304] 0 2024-05-27T09:29:37.501 INFO:tasks.workunit.client.1.smithi138.stdout:4/437: write d7/df/d35/d4c/d59/f58 [1190951,58646] 0 2024-05-27T09:29:37.501 INFO:tasks.workunit.client.1.smithi138.stdout:4/438: write d7/d27/d3a/f62 [769741,111203] 0 2024-05-27T09:29:37.502 INFO:tasks.workunit.client.1.smithi138.stdout:4/439: rename d7/c51 to d7/df/d35/c99 0 2024-05-27T09:29:37.502 INFO:tasks.workunit.client.1.smithi138.stdout:4/440: chown d7/df/d35/f5c 158 1 2024-05-27T09:29:37.502 INFO:tasks.workunit.client.1.smithi138.stdout:4/441: chown d7/df/d5b 191 1 2024-05-27T09:29:37.503 INFO:tasks.workunit.client.1.smithi138.stdout:4/442: creat d7/d27/f9a x:0 0 0 2024-05-27T09:29:37.503 INFO:tasks.workunit.client.1.smithi138.stdout:9/380: truncate d6/db/f10 2022944 0 2024-05-27T09:29:37.504 INFO:tasks.workunit.client.1.smithi138.stdout:0/364: read da/d2a/d2b/f50 [565968,128219] 0 2024-05-27T09:29:37.505 INFO:tasks.workunit.client.0.smithi119.stdout:5/971: dread d6/dd/d46/dba/ffc [0,4194304] 0 2024-05-27T09:29:37.507 INFO:tasks.workunit.client.1.smithi138.stdout:4/443: rename d7/d27/f80 to d7/d6e/f9b 0 2024-05-27T09:29:37.508 INFO:tasks.workunit.client.1.smithi138.stdout:4/444: write d7/df/d35/d4c/f91 [72529,92158] 0 2024-05-27T09:29:37.513 INFO:tasks.workunit.client.1.smithi138.stdout:0/365: mknod da/c7e 0 2024-05-27T09:29:37.513 INFO:tasks.workunit.client.1.smithi138.stdout:0/366: chown da/d17/d1d/d1f/f3e 12376726 1 2024-05-27T09:29:37.515 INFO:tasks.workunit.client.0.smithi119.stdout:8/861: write d3/d8/d4d/d58/f8f [341577,70558] 0 2024-05-27T09:29:37.515 INFO:tasks.workunit.client.0.smithi119.stdout:9/774: write d18/d19/d1b/d40/db7/fb8 [1759796,61248] 0 2024-05-27T09:29:37.515 INFO:tasks.workunit.client.0.smithi119.stdout:9/775: fdatasync d18/d19/d1b/d40/f62 0 2024-05-27T09:29:37.516 INFO:tasks.workunit.client.1.smithi138.stdout:7/346: dread d0/f36 [0,4194304] 0 2024-05-27T09:29:37.516 INFO:tasks.workunit.client.0.smithi119.stdout:9/776: creat d18/db4/dbe/ff2 x:0 0 0 2024-05-27T09:29:37.516 INFO:tasks.workunit.client.0.smithi119.stdout:9/777: chown d18/d19/d1b/d40/cc0 101084178 1 2024-05-27T09:29:37.516 INFO:tasks.workunit.client.0.smithi119.stdout:9/778: readlink d18/d19/d1b/l31 0 2024-05-27T09:29:37.521 INFO:tasks.workunit.client.0.smithi119.stdout:7/950: dwrite d7/df/d19/d97/d37/fac [0,4194304] 0 2024-05-27T09:29:37.522 INFO:tasks.workunit.client.0.smithi119.stdout:9/779: rename d18/d19/d61/d95/fc8 to d18/db4/dbe/ff3 0 2024-05-27T09:29:37.524 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:29:37.524 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:29:37.524 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-05-27T09:29:37.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:29:37.524 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:29:37.524 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-05-27T09:29:37.524 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 1 2024-05-27T09:29:37.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:29:37.524 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:29:37.524 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-05-27T09:29:37.524 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:29:37.524 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:29:37.525 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-05-27T09:29:37.525 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:29:37.525 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:29:37.525 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 13, 2024-05-27T09:29:37.525 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 1 2024-05-27T09:29:37.525 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:29:37.525 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:29:37.526 INFO:tasks.workunit.client.1.smithi138.stdout:7/347: symlink d0/d23/d49/l76 0 2024-05-27T09:29:37.526 INFO:tasks.workunit.client.1.smithi138.stdout:7/348: stat d0/d23/f48 0 2024-05-27T09:29:37.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.518+0000 7f426b200700 1 -- 172.21.15.119:0/2099298037 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f426419e430 con 0x7f42640737b0 2024-05-27T09:29:37.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.520+0000 7f4260c00700 1 -- 172.21.15.119:0/2099298037 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7f425405c060 con 0x7f42640737b0 2024-05-27T09:29:37.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.524+0000 7f426b200700 1 -- 172.21.15.119:0/2099298037 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f4248072820 msgr2=0x7f4248074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:37.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.524+0000 7f426b200700 1 --2- 172.21.15.119:0/2099298037 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f4248072820 0x7f4248074cd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4250005950 tx=0x7f42500058e0 comp rx=0 tx=0).stop 2024-05-27T09:29:37.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.524+0000 7f426b200700 1 -- 172.21.15.119:0/2099298037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42640737b0 msgr2=0x7f4264197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:37.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.524+0000 7f426b200700 1 --2- 172.21.15.119:0/2099298037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42640737b0 0x7f4264197de0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4254006010 tx=0x7f425400bf30 comp rx=0 tx=0).stop 2024-05-27T09:29:37.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.524+0000 7f426b200700 1 -- 172.21.15.119:0/2099298037 shutdown_connections 2024-05-27T09:29:37.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.524+0000 7f426b200700 1 --2- 172.21.15.119:0/2099298037 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f4248072820 0x7f4248074cd0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:37.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.524+0000 7f426b200700 1 --2- 172.21.15.119:0/2099298037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42640737b0 0x7f4264197de0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:37.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.524+0000 7f426b200700 1 --2- 172.21.15.119:0/2099298037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4264198320 0x7f426419d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:37.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.524+0000 7f426b200700 1 -- 172.21.15.119:0/2099298037 >> 172.21.15.119:0/2099298037 conn(0x7f42640fb6e0 msgr2=0x7f4264105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:37.527 INFO:tasks.workunit.client.0.smithi119.stdout:9/780: rmdir d18/d19/d61/d95 39 2024-05-27T09:29:37.529 INFO:tasks.workunit.client.0.smithi119.stdout:9/781: mknod d18/d19/d1b/d40/d44/dcb/cf4 0 2024-05-27T09:29:37.529 INFO:tasks.workunit.client.0.smithi119.stdout:9/782: chown d18/d19/d61/f6d 161770 1 2024-05-27T09:29:37.529 INFO:tasks.workunit.client.1.smithi138.stdout:7/349: creat d0/f77 x:0 0 0 2024-05-27T09:29:37.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.529+0000 7f426b200700 1 -- 172.21.15.119:0/2099298037 shutdown_connections 2024-05-27T09:29:37.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.529+0000 7f426b200700 1 -- 172.21.15.119:0/2099298037 wait complete. 2024-05-27T09:29:37.531 INFO:tasks.workunit.client.0.smithi119.stdout:9/783: mknod d18/db4/dbe/cf5 0 2024-05-27T09:29:37.534 INFO:tasks.workunit.client.1.smithi138.stdout:0/367: dread da/d17/d1d/d43/f48 [0,4194304] 0 2024-05-27T09:29:37.535 INFO:tasks.workunit.client.0.smithi119.stdout:6/829: dwrite d0/d2/d1c/d21/f121 [0,4194304] 0 2024-05-27T09:29:37.535 INFO:tasks.workunit.client.0.smithi119.stdout:9/784: mknod d18/db4/dbe/cf6 0 2024-05-27T09:29:37.537 INFO:tasks.workunit.client.0.smithi119.stdout:9/785: link d18/d19/d1b/d40/d7b/fa9 d18/d19/d61/ff7 0 2024-05-27T09:29:37.537 INFO:tasks.workunit.client.0.smithi119.stdout:9/786: read d18/d19/d1b/d2a/f8a [2527827,107682] 0 2024-05-27T09:29:37.538 INFO:tasks.workunit.client.0.smithi119.stdout:9/787: write d18/d19/d1b/d29/fa7 [11103,94696] 0 2024-05-27T09:29:37.538 INFO:tasks.workunit.client.0.smithi119.stdout:9/788: creat d18/db4/dbe/ff8 x:0 0 0 2024-05-27T09:29:37.542 INFO:tasks.workunit.client.0.smithi119.stdout:6/830: link d0/d2/d4e/f6a d0/d2/d1c/f128 0 2024-05-27T09:29:37.542 INFO:tasks.workunit.client.0.smithi119.stdout:6/831: write d0/d85/f8a [1403655,21660] 0 2024-05-27T09:29:37.542 INFO:tasks.workunit.client.0.smithi119.stdout:6/832: getdents d0/d2/d57/d37/de5/d122 0 2024-05-27T09:29:37.542 INFO:tasks.workunit.client.0.smithi119.stdout:6/833: write d0/d85/de7/ffa [748610,101204] 0 2024-05-27T09:29:37.544 INFO:tasks.workunit.client.0.smithi119.stdout:9/789: write d18/d19/f74 [756450,2309] 0 2024-05-27T09:29:37.546 INFO:tasks.workunit.client.0.smithi119.stdout:8/862: write d3/d8/d1b/d46/f4b [1340817,42257] 0 2024-05-27T09:29:37.550 INFO:tasks.workunit.client.0.smithi119.stdout:6/834: creat d0/d2/d57/d37/dad/f129 x:0 0 0 2024-05-27T09:29:37.550 INFO:tasks.workunit.client.0.smithi119.stdout:9/790: rename f16 to d18/d19/d1b/d2a/d63/ff9 0 2024-05-27T09:29:37.551 INFO:tasks.workunit.client.0.smithi119.stdout:9/791: rmdir d18/d19/d1b/d2a 39 2024-05-27T09:29:37.554 INFO:tasks.workunit.client.0.smithi119.stdout:5/972: dread d6/dd/fd9 [0,4194304] 0 2024-05-27T09:29:37.554 INFO:tasks.workunit.client.1.smithi138.stdout:7/350: rmdir d0/d23/d49 39 2024-05-27T09:29:37.555 INFO:tasks.workunit.client.1.smithi138.stdout:7/351: write d0/d23/f47 [154532,37056] 0 2024-05-27T09:29:37.555 INFO:tasks.workunit.client.1.smithi138.stdout:7/352: fsync d0/d7/d64/f6e 0 2024-05-27T09:29:37.558 INFO:tasks.workunit.client.1.smithi138.stdout:9/381: dread d6/f3f [0,4194304] 0 2024-05-27T09:29:37.558 INFO:tasks.workunit.client.1.smithi138.stdout:9/382: stat f1 0 2024-05-27T09:29:37.558 INFO:tasks.workunit.client.0.smithi119.stdout:7/951: dread d7/df/d19/d4d/d89/fe8 [4194304,4194304] 0 2024-05-27T09:29:37.558 INFO:tasks.workunit.client.0.smithi119.stdout:5/973: symlink d6/dd/d46/d47/d4b/l151 0 2024-05-27T09:29:37.562 INFO:tasks.workunit.client.0.smithi119.stdout:2/892: dwrite d2/da/d8a/dd4/d89/d94/d81/f9e [0,4194304] 0 2024-05-27T09:29:37.562 INFO:tasks.workunit.client.0.smithi119.stdout:7/952: mkdir d7/d5f/d79/d12d 0 2024-05-27T09:29:37.562 INFO:tasks.workunit.client.0.smithi119.stdout:7/953: stat d7/df/d19/d97/d92/db7/dba/f52 0 2024-05-27T09:29:37.562 INFO:tasks.workunit.client.0.smithi119.stdout:7/954: write d7/df/d19/d97/d37/fa4 [899712,108648] 0 2024-05-27T09:29:37.562 INFO:tasks.workunit.client.0.smithi119.stdout:7/955: dread - d7/df/d19/d97/f110 zero size 2024-05-27T09:29:37.563 INFO:tasks.workunit.client.1.smithi138.stdout:7/353: mkdir d0/d23/d49/d78 0 2024-05-27T09:29:37.563 INFO:tasks.workunit.client.0.smithi119.stdout:5/974: mkdir d6/db/dd4/d152 0 2024-05-27T09:29:37.566 INFO:tasks.workunit.client.1.smithi138.stdout:9/383: creat d6/db/de/d18/d27/d46/f77 x:0 0 0 2024-05-27T09:29:37.567 INFO:tasks.workunit.client.1.smithi138.stdout:9/384: chown d6/db/de/d18/d24/f73 3473 1 2024-05-27T09:29:37.568 INFO:tasks.workunit.client.1.smithi138.stdout:9/385: read d6/da/d26/f4f [30312,1302] 0 2024-05-27T09:29:37.571 INFO:tasks.workunit.client.0.smithi119.stdout:2/893: mkdir d2/da/d8a/dd4/d69/d134 0 2024-05-27T09:29:37.574 INFO:tasks.workunit.client.0.smithi119.stdout:5/975: mknod d6/db/de/d27/d77/d14f/c153 0 2024-05-27T09:29:37.574 INFO:tasks.workunit.client.0.smithi119.stdout:5/976: write d6/db/dd4/f123 [248260,45860] 0 2024-05-27T09:29:37.576 INFO:tasks.workunit.client.1.smithi138.stdout:9/386: rename d6/db/de/d18/d27/d46/f77 to d6/da/d2f/d35/d55/d75/f78 0 2024-05-27T09:29:37.576 INFO:tasks.workunit.client.1.smithi138.stdout:7/354: mknod d0/d2b/c79 0 2024-05-27T09:29:37.576 INFO:tasks.workunit.client.1.smithi138.stdout:0/368: dread f5 [0,4194304] 0 2024-05-27T09:29:37.577 INFO:tasks.workunit.client.1.smithi138.stdout:0/369: readlink da/d17/l3d 0 2024-05-27T09:29:37.577 INFO:tasks.workunit.client.0.smithi119.stdout:2/894: getdents d2/da 0 2024-05-27T09:29:37.579 INFO:tasks.workunit.client.1.smithi138.stdout:7/355: read d0/d23/f4f [5056,69010] 0 2024-05-27T09:29:37.579 INFO:tasks.workunit.client.1.smithi138.stdout:7/356: chown d0/d23/d3c/d3f/c73 15448 1 2024-05-27T09:29:37.581 INFO:tasks.workunit.client.1.smithi138.stdout:0/370: link da/c3a da/d2a/d2b/c7f 0 2024-05-27T09:29:37.583 INFO:tasks.workunit.client.1.smithi138.stdout:0/371: getdents da/d17/d1d/d1f/d71 0 2024-05-27T09:29:37.583 INFO:tasks.workunit.client.0.smithi119.stdout:6/835: dread d0/d2/d57/d37/f5d [0,4194304] 0 2024-05-27T09:29:37.584 INFO:tasks.workunit.client.1.smithi138.stdout:1/386: dwrite d4/d1a/d2e/d32/f68 [0,4194304] 0 2024-05-27T09:29:37.585 INFO:tasks.workunit.client.1.smithi138.stdout:3/414: dwrite d0/d3f/d50/f63 [0,4194304] 0 2024-05-27T09:29:37.586 INFO:tasks.workunit.client.1.smithi138.stdout:0/372: getdents da/d17/d1d/d1f/d60 0 2024-05-27T09:29:37.588 INFO:tasks.workunit.client.1.smithi138.stdout:9/387: mknod d6/db/de/d18/d27/d46/c79 0 2024-05-27T09:29:37.588 INFO:tasks.workunit.client.1.smithi138.stdout:1/387: creat d4/f78 x:0 0 0 2024-05-27T09:29:37.591 INFO:tasks.workunit.client.1.smithi138.stdout:1/388: creat d4/d1a/d2e/f79 x:0 0 0 2024-05-27T09:29:37.593 INFO:tasks.workunit.client.1.smithi138.stdout:7/357: creat d0/d7/d43/f7a x:0 0 0 2024-05-27T09:29:37.601 INFO:tasks.workunit.client.0.smithi119.stdout:5/977: write d6/dd/f32 [4111147,69164] 0 2024-05-27T09:29:37.603 INFO:tasks.workunit.client.0.smithi119.stdout:4/940: dwrite d4/d23/f26 [0,4194304] 0 2024-05-27T09:29:37.604 INFO:tasks.workunit.client.1.smithi138.stdout:0/373: write da/d17/d1d/d1f/f21 [3055868,93648] 0 2024-05-27T09:29:37.605 INFO:tasks.workunit.client.1.smithi138.stdout:9/388: rmdir d6/db/de/d18/d24 39 2024-05-27T09:29:37.605 INFO:tasks.workunit.client.1.smithi138.stdout:7/358: symlink d0/d2b/l7b 0 2024-05-27T09:29:37.607 INFO:tasks.workunit.client.0.smithi119.stdout:4/941: symlink d4/d23/d24/d121/l131 0 2024-05-27T09:29:37.607 INFO:tasks.workunit.client.0.smithi119.stdout:4/942: fsync d4/d6/d81/dce/d88/f112 0 2024-05-27T09:29:37.615 INFO:tasks.workunit.client.1.smithi138.stdout:9/389: unlink d6/da/d2f/d35/d55/d75/f78 0 2024-05-27T09:29:37.619 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:37 smithi138 ceph-mon[34885]: from='client.14608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:29:37.619 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:37 smithi138 ceph-mon[34885]: pgmap v16: 65 pgs: 65 active+clean; 2.2 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 153 MiB/s rd, 183 MiB/s wr, 272 op/s 2024-05-27T09:29:37.619 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:37 smithi138 ceph-mon[34885]: from='client.14612 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:29:37.627 INFO:tasks.workunit.client.1.smithi138.stdout:0/374: write da/d17/d1d/d1f/d42/d4a/d53/f72 [3818708,90439] 0 2024-05-27T09:29:37.627 INFO:tasks.workunit.client.1.smithi138.stdout:0/375: write da/d17/f31 [992469,123273] 0 2024-05-27T09:29:37.627 INFO:tasks.workunit.client.1.smithi138.stdout:0/376: readlink da/d2a/d2b/l36 0 2024-05-27T09:29:37.627 INFO:tasks.workunit.client.1.smithi138.stdout:0/377: chown da/d17/d1d/d1f/d60/c67 4135 1 2024-05-27T09:29:37.627 INFO:tasks.workunit.client.1.smithi138.stdout:0/378: readlink da/d17/d1d/d1f/d42/d4a/l54 0 2024-05-27T09:29:37.627 INFO:tasks.workunit.client.1.smithi138.stdout:0/379: chown da/d17/d1d/d6a/c7d 5 1 2024-05-27T09:29:37.628 INFO:tasks.workunit.client.0.smithi119.stdout:5/978: write d6/db/de/d27/f108 [3701562,92134] 0 2024-05-27T09:29:37.628 INFO:tasks.workunit.client.0.smithi119.stdout:6/836: write d0/d11/f62 [4487002,89268] 0 2024-05-27T09:29:37.629 INFO:tasks.workunit.client.0.smithi119.stdout:0/866: sync 2024-05-27T09:29:37.629 INFO:tasks.workunit.client.0.smithi119.stdout:1/918: sync 2024-05-27T09:29:37.629 INFO:tasks.workunit.client.0.smithi119.stdout:0/867: fsync d2/f49 0 2024-05-27T09:29:37.630 INFO:tasks.workunit.client.0.smithi119.stdout:0/868: creat d2/d6/d9/d2f/d95/f121 x:0 0 0 2024-05-27T09:29:37.630 INFO:tasks.workunit.client.0.smithi119.stdout:0/869: readlink d2/d6/d9/d12/d4f/l80 0 2024-05-27T09:29:37.633 INFO:tasks.workunit.client.0.smithi119.stdout:5/979: getdents d6/dd/d39/d19/d3c/d52 0 2024-05-27T09:29:37.633 INFO:tasks.workunit.client.0.smithi119.stdout:4/943: dread d4/de/d21/d40/f6a [0,4194304] 0 2024-05-27T09:29:37.633 INFO:tasks.workunit.client.0.smithi119.stdout:6/837: creat d0/d2/d57/d37/f12a x:0 0 0 2024-05-27T09:29:37.634 INFO:tasks.workunit.client.0.smithi119.stdout:4/944: write d4/d23/f35 [4917763,36058] 0 2024-05-27T09:29:37.638 INFO:tasks.workunit.client.0.smithi119.stdout:0/870: rename d2/d110/d59/dd7/cff to d2/d6/d23/c122 0 2024-05-27T09:29:37.640 INFO:tasks.workunit.client.0.smithi119.stdout:6/838: mkdir d0/d2/d57/dd0/d12b 0 2024-05-27T09:29:37.640 INFO:tasks.workunit.client.0.smithi119.stdout:6/839: chown d0/d2/d57/d37/d74/d65/lb7 175957443 1 2024-05-27T09:29:37.645 INFO:tasks.workunit.client.1.smithi138.stdout:0/380: dread da/d17/d1d/d1f/f3b [0,4194304] 0 2024-05-27T09:29:37.645 INFO:tasks.workunit.client.0.smithi119.stdout:8/863: dwrite d3/d8/f47 [0,4194304] 0 2024-05-27T09:29:37.646 INFO:tasks.workunit.client.1.smithi138.stdout:7/359: dread d0/d7/f39 [0,4194304] 0 2024-05-27T09:29:37.648 INFO:tasks.workunit.client.0.smithi119.stdout:8/864: chown d3/d8/fba 209850 1 2024-05-27T09:29:37.652 INFO:tasks.workunit.client.1.smithi138.stdout:1/389: dwrite d4/d3c/f6b [0,4194304] 0 2024-05-27T09:29:37.654 INFO:tasks.workunit.client.1.smithi138.stdout:7/360: dread d0/f32 [4194304,4194304] 0 2024-05-27T09:29:37.655 INFO:tasks.workunit.client.1.smithi138.stdout:6/499: sync 2024-05-27T09:29:37.664 INFO:tasks.workunit.client.0.smithi119.stdout:5/980: write d6/dd/f64 [3713293,93319] 0 2024-05-27T09:29:37.664 INFO:tasks.workunit.client.0.smithi119.stdout:5/981: stat d6/dd/d39/d19/d3c/d52/c14c 0 2024-05-27T09:29:37.664 INFO:tasks.workunit.client.0.smithi119.stdout:5/982: chown d6/db/de/d27/f108 10 1 2024-05-27T09:29:37.679 INFO:tasks.workunit.client.1.smithi138.stdout:7/361: rmdir d0/d23/d3a 39 2024-05-27T09:29:37.684 INFO:tasks.workunit.client.1.smithi138.stdout:7/362: creat d0/d7/f7c x:0 0 0 2024-05-27T09:29:37.684 INFO:tasks.workunit.client.1.smithi138.stdout:7/363: creat d0/d7/d64/f7d x:0 0 0 2024-05-27T09:29:37.689 INFO:tasks.workunit.client.1.smithi138.stdout:7/364: mknod d0/d7/d64/c7e 0 2024-05-27T09:29:37.699 INFO:tasks.workunit.client.1.smithi138.stdout:3/415: dwrite d0/d2/da/df/d10/d32/f28 [0,4194304] 0 2024-05-27T09:29:37.706 INFO:tasks.workunit.client.0.smithi119.stdout:0/871: dread d2/d6/d23/db8/fdc [0,4194304] 0 2024-05-27T09:29:37.710 INFO:tasks.workunit.client.0.smithi119.stdout:0/872: creat d2/d6/d23/db8/d118/f123 x:0 0 0 2024-05-27T09:29:37.715 INFO:tasks.workunit.client.0.smithi119.stdout:0/873: mkdir d2/d6/d9/d24/d30/dcc/d124 0 2024-05-27T09:29:37.722 INFO:tasks.workunit.client.0.smithi119.stdout:0/874: truncate d2/d6/d9/f27 2690978 0 2024-05-27T09:29:37.722 INFO:tasks.workunit.client.0.smithi119.stdout:5/983: dread d6/dd/d46/fbf [0,4194304] 0 2024-05-27T09:29:37.741 INFO:tasks.workunit.client.1.smithi138.stdout:9/390: dwrite d6/da/d2f/f5f [0,4194304] 0 2024-05-27T09:29:37.743 INFO:tasks.workunit.client.1.smithi138.stdout:0/381: dwrite da/d17/d1d/d1f/f26 [0,4194304] 0 2024-05-27T09:29:37.746 INFO:tasks.workunit.client.1.smithi138.stdout:0/382: mknod da/d2a/d3f/c80 0 2024-05-27T09:29:37.747 INFO:tasks.workunit.client.1.smithi138.stdout:0/383: fdatasync da/d2a/d3f/f4c 0 2024-05-27T09:29:37.747 INFO:tasks.workunit.client.1.smithi138.stdout:0/384: truncate da/d2a/d3f/f7c 520319 0 2024-05-27T09:29:37.747 INFO:tasks.workunit.client.1.smithi138.stdout:0/385: readlink da/d17/d1d/d1f/l7a 0 2024-05-27T09:29:37.747 INFO:tasks.workunit.client.1.smithi138.stdout:0/386: stat da/d17/d1d/l2c 0 2024-05-27T09:29:37.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:37 smithi119 ceph-mon[29082]: from='client.14608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:29:37.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:37 smithi119 ceph-mon[29082]: pgmap v16: 65 pgs: 65 active+clean; 2.2 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 153 MiB/s rd, 183 MiB/s wr, 272 op/s 2024-05-27T09:29:37.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:37 smithi119 ceph-mon[29082]: from='client.14612 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:29:37.751 INFO:tasks.workunit.client.1.smithi138.stdout:9/391: dread d6/db/de/d18/d24/f4b [0,4194304] 0 2024-05-27T09:29:37.758 INFO:tasks.workunit.client.0.smithi119.stdout:0/875: dread d2/d1a/d20/f64 [0,4194304] 0 2024-05-27T09:29:37.758 INFO:tasks.workunit.client.0.smithi119.stdout:0/876: chown d2/l14 6 1 2024-05-27T09:29:37.761 INFO:tasks.workunit.client.0.smithi119.stdout:0/877: symlink d2/d6/d9/d24/d30/dcc/de6/l125 0 2024-05-27T09:29:37.762 INFO:tasks.workunit.client.0.smithi119.stdout:0/878: mknod d2/d6/d9/d24/c126 0 2024-05-27T09:29:37.763 INFO:tasks.workunit.client.0.smithi119.stdout:0/879: mknod d2/d6/d9/d24/d30/dcc/de6/c127 0 2024-05-27T09:29:37.764 INFO:tasks.workunit.client.0.smithi119.stdout:0/880: symlink d2/d110/d96/l128 0 2024-05-27T09:29:37.764 INFO:tasks.workunit.client.0.smithi119.stdout:0/881: truncate d2/d6/d23/f10c 332008 0 2024-05-27T09:29:37.766 INFO:tasks.workunit.client.1.smithi138.stdout:0/387: write da/d17/d1d/d43/f64 [2432573,113607] 0 2024-05-27T09:29:37.770 INFO:tasks.workunit.client.1.smithi138.stdout:1/390: dwrite d4/d48/d39/d5f/f2f [4194304,4194304] 0 2024-05-27T09:29:37.770 INFO:tasks.workunit.client.1.smithi138.stdout:1/391: read - d4/d48/d39/d5f/f6a zero size 2024-05-27T09:29:37.770 INFO:tasks.workunit.client.1.smithi138.stdout:5/277: sync 2024-05-27T09:29:37.770 INFO:tasks.workunit.client.1.smithi138.stdout:3/416: dwrite d0/d2/da/df/d10/d2a/f83 [0,4194304] 0 2024-05-27T09:29:37.771 INFO:tasks.workunit.client.1.smithi138.stdout:3/417: stat d0/d3f/d50/f76 0 2024-05-27T09:29:37.771 INFO:tasks.workunit.client.0.smithi119.stdout:6/840: dwrite d0/d2/d1c/f128 [0,4194304] 0 2024-05-27T09:29:37.771 INFO:tasks.workunit.client.0.smithi119.stdout:6/841: chown d0/d2/d57/dd0 88841 1 2024-05-27T09:29:37.771 INFO:tasks.workunit.client.0.smithi119.stdout:6/842: truncate d0/d2/d57/d37/d74/df1/f11f 711926 0 2024-05-27T09:29:37.771 INFO:tasks.workunit.client.0.smithi119.stdout:6/843: chown d0/d2/d1c/d21 115 1 2024-05-27T09:29:37.772 INFO:tasks.workunit.client.0.smithi119.stdout:8/865: dwrite d3/d6e/dcf/dfd/f10f [0,4194304] 0 2024-05-27T09:29:37.773 INFO:tasks.workunit.client.1.smithi138.stdout:7/365: dwrite d0/d7/d43/f7a [0,4194304] 0 2024-05-27T09:29:37.777 INFO:tasks.workunit.client.1.smithi138.stdout:1/392: getdents d4/d1a/d2e/d32 0 2024-05-27T09:29:37.777 INFO:tasks.workunit.client.1.smithi138.stdout:1/393: stat d4/d1a/d2e/d32/f49 0 2024-05-27T09:29:37.778 INFO:tasks.workunit.client.0.smithi119.stdout:1/919: dwrite d7/db/d52/d65/fa4 [0,4194304] 0 2024-05-27T09:29:37.780 INFO:tasks.workunit.client.0.smithi119.stdout:6/844: mknod d0/d2/d57/d37/d74/dfe/d10a/c12c 0 2024-05-27T09:29:37.780 INFO:tasks.workunit.client.0.smithi119.stdout:6/845: fsync d0/d85/de7/ffa 0 2024-05-27T09:29:37.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.784+0000 7ff767400700 1 -- 172.21.15.119:0/142564632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff760100150 msgr2=0x7ff760100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:37.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.784+0000 7ff767400700 1 --2- 172.21.15.119:0/142564632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff760100150 0x7ff760100560 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff754009910 tx=0x7ff754009c20 comp rx=0 tx=0).stop 2024-05-27T09:29:37.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.784+0000 7ff767400700 1 -- 172.21.15.119:0/142564632 shutdown_connections 2024-05-27T09:29:37.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.784+0000 7ff767400700 1 --2- 172.21.15.119:0/142564632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff760101350 0x7ff7601017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:37.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.784+0000 7ff767400700 1 --2- 172.21.15.119:0/142564632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff760100150 0x7ff760100560 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:37.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.784+0000 7ff767400700 1 -- 172.21.15.119:0/142564632 >> 172.21.15.119:0/142564632 conn(0x7ff7600fb6e0 msgr2=0x7ff7600fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:37.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.784+0000 7ff767400700 1 -- 172.21.15.119:0/142564632 shutdown_connections 2024-05-27T09:29:37.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.784+0000 7ff767400700 1 -- 172.21.15.119:0/142564632 wait complete. 2024-05-27T09:29:37.786 INFO:tasks.workunit.client.0.smithi119.stdout:8/866: creat d3/d6e/d80/d84/f121 x:0 0 0 2024-05-27T09:29:37.786 INFO:tasks.workunit.client.0.smithi119.stdout:6/846: mknod d0/d2/d57/d37/dad/c12d 0 2024-05-27T09:29:37.786 INFO:tasks.workunit.client.0.smithi119.stdout:6/847: chown d0/d2/d57/d37/dad/c12d 162 1 2024-05-27T09:29:37.787 INFO:tasks.workunit.client.1.smithi138.stdout:7/366: symlink d0/d23/d49/d78/l7f 0 2024-05-27T09:29:37.787 INFO:tasks.workunit.client.1.smithi138.stdout:9/392: dread d6/da/d29/d59/f70 [0,4194304] 0 2024-05-27T09:29:37.790 INFO:tasks.workunit.client.1.smithi138.stdout:6/500: dwrite d3/d35/d5a/d6a/f6b [0,4194304] 0 2024-05-27T09:29:37.790 INFO:tasks.workunit.client.1.smithi138.stdout:6/501: fdatasync d3/d11/d2d/d37/f4a 0 2024-05-27T09:29:37.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.790+0000 7ff767400700 1 Processor -- start 2024-05-27T09:29:37.791 INFO:tasks.workunit.client.1.smithi138.stdout:9/393: symlink d6/da/d2f/d35/d55/d75/l7a 0 2024-05-27T09:29:37.791 INFO:tasks.workunit.client.1.smithi138.stdout:7/367: unlink d0/d2b/l7b 0 2024-05-27T09:29:37.791 INFO:tasks.workunit.client.1.smithi138.stdout:7/368: dread - d0/d23/f50 zero size 2024-05-27T09:29:37.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.790+0000 7ff767400700 1 -- start start 2024-05-27T09:29:37.793 INFO:tasks.workunit.client.1.smithi138.stdout:9/394: creat d6/db/de/d18/f7b x:0 0 0 2024-05-27T09:29:37.793 INFO:tasks.workunit.client.0.smithi119.stdout:1/920: dread d7/db/d12/ddb/d38/d3c/ff3 [0,4194304] 0 2024-05-27T09:29:37.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.795+0000 7ff767400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff760100150 0x7ff760197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:37.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.795+0000 7ff767400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff760101350 0x7ff760198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:37.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.796+0000 7ff767400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7601987a0 con 0x7ff760101350 2024-05-27T09:29:37.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.796+0000 7ff767400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76019d1b0 con 0x7ff760100150 2024-05-27T09:29:37.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.797+0000 7ff764a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff760100150 0x7ff760197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:37.798 INFO:tasks.workunit.client.0.smithi119.stdout:6/848: creat d0/d2/d1c/d21/d58/dd3/f12e x:0 0 0 2024-05-27T09:29:37.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.797+0000 7ff764a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff760100150 0x7ff760197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.119:45870/0 (socket says 172.21.15.119:45870) 2024-05-27T09:29:37.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.797+0000 7ff764a00700 1 -- 172.21.15.119:0/348631982 learned_addr learned my addr 172.21.15.119:0/348631982 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:29:37.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.798+0000 7ff764a00700 1 -- 172.21.15.119:0/348631982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff760101350 msgr2=0x7ff760198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:37.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.798+0000 7ff764a00700 1 --2- 172.21.15.119:0/348631982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff760101350 0x7ff760198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:37.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.798+0000 7ff764a00700 1 -- 172.21.15.119:0/348631982 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7540095f0 con 0x7ff760100150 2024-05-27T09:29:37.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.798+0000 7ff764a00700 1 --2- 172.21.15.119:0/348631982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff760100150 0x7ff760197c40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff75400bd70 tx=0x7ff75400be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:37.800 INFO:tasks.workunit.client.1.smithi138.stdout:6/502: creat d3/d35/fbd x:0 0 0 2024-05-27T09:29:37.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.800+0000 7ff75d600700 1 -- 172.21.15.119:0/348631982 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff75401d070 con 0x7ff760100150 2024-05-27T09:29:37.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.800+0000 7ff75d600700 1 -- 172.21.15.119:0/348631982 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff754004be0 con 0x7ff760100150 2024-05-27T09:29:37.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.801+0000 7ff767400700 1 -- 172.21.15.119:0/348631982 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff76019d350 con 0x7ff760100150 2024-05-27T09:29:37.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.801+0000 7ff767400700 1 -- 172.21.15.119:0/348631982 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff76019d890 con 0x7ff760100150 2024-05-27T09:29:37.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.801+0000 7ff75d600700 1 -- 172.21.15.119:0/348631982 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff754021620 con 0x7ff760100150 2024-05-27T09:29:37.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.804+0000 7ff767400700 1 -- 172.21.15.119:0/348631982 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff740005320 con 0x7ff760100150 2024-05-27T09:29:37.807 INFO:tasks.workunit.client.0.smithi119.stdout:5/984: dwrite d6/db/de/d27/d94/fa9 [0,4194304] 0 2024-05-27T09:29:37.807 INFO:tasks.workunit.client.0.smithi119.stdout:5/985: readlink d6/dd/d46/d47/d4b/l86 0 2024-05-27T09:29:37.807 INFO:tasks.workunit.client.0.smithi119.stdout:5/986: chown d6/db/de/d27 55 1 2024-05-27T09:29:37.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.808+0000 7ff75d600700 1 -- 172.21.15.119:0/348631982 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) v1 ==== 95024+0+0 (secure 0 0 0) 0x7ff754004d50 con 0x7ff760100150 2024-05-27T09:29:37.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.810+0000 7ff75d600700 1 --2- 172.21.15.119:0/348631982 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7ff748072810 0x7ff748074cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:29:37.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.810+0000 7ff75d600700 1 -- 172.21.15.119:0/348631982 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7ff754094720 con 0x7ff760100150 2024-05-27T09:29:37.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.811+0000 7ff75fe00700 1 --2- 172.21.15.119:0/348631982 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7ff748072810 0x7ff748074cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:29:37.812 INFO:tasks.workunit.client.1.smithi138.stdout:7/369: rmdir d0/d2b 39 2024-05-27T09:29:37.812 INFO:tasks.workunit.client.1.smithi138.stdout:7/370: chown d0/d23/d3c/d3f/l52 1418847 1 2024-05-27T09:29:37.812 INFO:tasks.workunit.client.1.smithi138.stdout:7/371: write d0/f19 [2653560,18665] 0 2024-05-27T09:29:37.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.814+0000 7ff75fe00700 1 --2- 172.21.15.119:0/348631982 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7ff748072810 0x7ff748074cc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0097b0 tx=0x7ff74c006d20 comp rx=0 tx=0).ready entity=mgr.14570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:29:37.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:37.815+0000 7ff75d600700 1 -- 172.21.15.119:0/348631982 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7ff75405c860 con 0x7ff760100150 2024-05-27T09:29:37.825 INFO:tasks.workunit.client.1.smithi138.stdout:7/372: rmdir d0/d23/d3c/d3f 39 2024-05-27T09:29:37.828 INFO:tasks.workunit.client.1.smithi138.stdout:7/373: mkdir d0/d2b/d80 0 2024-05-27T09:29:37.831 INFO:tasks.workunit.client.1.smithi138.stdout:7/374: mknod d0/d23/d49/d78/c81 0 2024-05-27T09:29:37.831 INFO:tasks.workunit.client.1.smithi138.stdout:7/375: write d0/d7/f7c [971805,107907] 0 2024-05-27T09:29:37.835 INFO:tasks.workunit.client.1.smithi138.stdout:7/376: unlink d0/d2b/c79 0 2024-05-27T09:29:37.837 INFO:tasks.workunit.client.0.smithi119.stdout:5/987: write d6/f22 [4440855,74607] 0 2024-05-27T09:29:37.837 INFO:tasks.workunit.client.0.smithi119.stdout:5/988: chown d6/db/dc/l12b 117883 1 2024-05-27T09:29:37.838 INFO:tasks.workunit.client.1.smithi138.stdout:3/418: write d0/d2/da/df/d10/d2a/f4f [1948496,79448] 0 2024-05-27T09:29:37.844 INFO:tasks.workunit.client.0.smithi119.stdout:5/989: rename d6/dd/d39/d19/d3c to d6/db/de/d154 0 2024-05-27T09:29:37.845 INFO:tasks.workunit.client.0.smithi119.stdout:5/990: getdents d6/dd/d46/d47 0 2024-05-27T09:29:37.846 INFO:tasks.workunit.client.0.smithi119.stdout:5/991: creat d6/db/dc/d10b/d13c/f155 x:0 0 0 2024-05-27T09:29:37.847 INFO:tasks.workunit.client.0.smithi119.stdout:5/992: mknod d6/dd/d39/d19/d7d/db7/c156 0 2024-05-27T09:29:37.847 INFO:tasks.workunit.client.0.smithi119.stdout:5/993: dread - d6/db/dc/d10b/d13c/f155 zero size 2024-05-27T09:29:37.849 INFO:tasks.workunit.client.0.smithi119.stdout:5/994: mkdir d6/db/dc/dc3/d145/d14d/d157 0 2024-05-27T09:29:37.849 INFO:tasks.workunit.client.0.smithi119.stdout:5/995: stat d6/db/dc/c1f 0 2024-05-27T09:29:37.849 INFO:tasks.workunit.client.0.smithi119.stdout:5/996: readlink d6/db/de/led 0 2024-05-27T09:29:37.852 INFO:tasks.workunit.client.1.smithi138.stdout:7/377: dread d0/d7/d43/f7a [0,4194304] 0 2024-05-27T09:29:37.852 INFO:tasks.workunit.client.1.smithi138.stdout:7/378: truncate d0/d23/f5a 846656 0 2024-05-27T09:29:37.855 INFO:tasks.workunit.client.1.smithi138.stdout:7/379: truncate d0/f36 1763769 0 2024-05-27T09:29:37.855 INFO:tasks.workunit.client.1.smithi138.stdout:7/380: chown d0/d7/d64/f6e 0 1 2024-05-27T09:29:37.855 INFO:tasks.workunit.client.1.smithi138.stdout:7/381: fsync d0/f1d 0 2024-05-27T09:29:37.864 INFO:tasks.workunit.client.1.smithi138.stdout:0/388: dwrite da/d17/d1d/d1f/f3e [0,4194304] 0 2024-05-27T09:29:37.868 INFO:tasks.workunit.client.1.smithi138.stdout:0/389: mknod da/d17/d1d/d1f/d42/d4a/c81 0 2024-05-27T09:29:37.870 INFO:tasks.workunit.client.1.smithi138.stdout:0/390: truncate da/f3c 244596 0 2024-05-27T09:29:37.871 INFO:tasks.workunit.client.1.smithi138.stdout:0/391: symlink da/d17/d1d/d1f/d42/d4a/l82 0 2024-05-27T09:29:37.885 INFO:tasks.workunit.client.1.smithi138.stdout:8/333: sync 2024-05-27T09:29:37.885 INFO:tasks.workunit.client.1.smithi138.stdout:2/334: sync 2024-05-27T09:29:37.885 INFO:tasks.workunit.client.1.smithi138.stdout:2/335: write d1/d10/f64 [1476704,47121] 0 2024-05-27T09:29:37.889 INFO:tasks.workunit.client.1.smithi138.stdout:4/445: sync 2024-05-27T09:29:37.889 INFO:tasks.workunit.client.1.smithi138.stdout:7/382: dread d0/d7/f56 [0,4194304] 0 2024-05-27T09:29:37.891 INFO:tasks.workunit.client.1.smithi138.stdout:4/446: mknod d7/df/d5b/c9c 0 2024-05-27T09:29:37.891 INFO:tasks.workunit.client.1.smithi138.stdout:4/447: truncate d7/df/d5b/f7c 1652160 0 2024-05-27T09:29:37.891 INFO:tasks.workunit.client.1.smithi138.stdout:4/448: dread - d7/f78 zero size 2024-05-27T09:29:37.891 INFO:tasks.workunit.client.1.smithi138.stdout:7/383: getdents d0/d23/d3c 0 2024-05-27T09:29:37.898 INFO:tasks.workunit.client.1.smithi138.stdout:4/449: creat d7/df/d35/d4c/d59/f9d x:0 0 0 2024-05-27T09:29:37.898 INFO:tasks.workunit.client.1.smithi138.stdout:5/278: dwrite d3/d12/d26/d53/f58 [0,4194304] 0 2024-05-27T09:29:37.898 INFO:tasks.workunit.client.1.smithi138.stdout:5/279: read - d3/d12/d16/d1d/f55 zero size 2024-05-27T09:29:37.913 INFO:tasks.workunit.client.0.smithi119.stdout:6/849: dwrite d0/d2/f67 [0,4194304] 0 2024-05-27T09:29:37.913 INFO:tasks.workunit.client.0.smithi119.stdout:6/850: write d0/d2/d4e/dc5/f113 [699477,28241] 0 2024-05-27T09:29:37.914 INFO:tasks.workunit.client.0.smithi119.stdout:6/851: mknod d0/d2/d57/dd0/c12f 0 2024-05-27T09:29:37.915 INFO:tasks.workunit.client.1.smithi138.stdout:8/334: write d3/d8/d32/d34/d38/f46 [1053229,49764] 0 2024-05-27T09:29:37.916 INFO:tasks.workunit.client.0.smithi119.stdout:8/867: dwrite d3/d8/d1b/d46/da4/d113/fd6 [0,4194304] 0 2024-05-27T09:29:37.920 INFO:tasks.workunit.client.0.smithi119.stdout:5/997: dwrite d6/db/de/d154/dd5/f104 [0,4194304] 0 2024-05-27T09:29:37.926 INFO:tasks.workunit.client.1.smithi138.stdout:9/395: dwrite d6/da/d2f/d35/d55/f6c [0,4194304] 0 2024-05-27T09:29:37.928 INFO:tasks.workunit.client.1.smithi138.stdout:6/503: dwrite d3/d19/f22 [0,4194304] 0 2024-05-27T09:29:37.932 INFO:tasks.workunit.client.1.smithi138.stdout:8/335: rename d3/d8/d28 to d3/d8/d64 0 2024-05-27T09:29:37.934 INFO:tasks.workunit.client.1.smithi138.stdout:5/280: write d3/f1b [1157873,105248] 0 2024-05-27T09:29:37.934 INFO:tasks.workunit.client.1.smithi138.stdout:6/504: mkdir d3/d35/d43/dbe 0 2024-05-27T09:29:37.934 INFO:tasks.workunit.client.0.smithi119.stdout:0/882: dwrite d2/d6/d9/d12/da3/fd3 [0,4194304] 0 2024-05-27T09:29:37.936 INFO:tasks.workunit.client.1.smithi138.stdout:8/336: getdents d3 0 2024-05-27T09:29:37.938 INFO:tasks.workunit.client.1.smithi138.stdout:5/281: mkdir d3/d12/d16/d1d/d30/d5c 0 2024-05-27T09:29:37.940 INFO:tasks.workunit.client.1.smithi138.stdout:8/337: link d3/d8/c10 d3/d8/c65 0 2024-05-27T09:29:37.940 INFO:tasks.workunit.client.0.smithi119.stdout:6/852: write d0/d2/fb1 [3819150,65271] 0 2024-05-27T09:29:37.946 INFO:tasks.workunit.client.0.smithi119.stdout:6/853: rmdir d0/d2/d57/d37/de5/d123 0 2024-05-27T09:29:37.952 INFO:tasks.workunit.client.1.smithi138.stdout:0/392: dwrite da/d2a/d2b/f50 [0,4194304] 0 2024-05-27T09:29:37.952 INFO:tasks.workunit.client.1.smithi138.stdout:7/384: dwrite d0/d7/d64/f7d [0,4194304] 0 2024-05-27T09:29:37.957 INFO:tasks.workunit.client.1.smithi138.stdout:9/396: dread f3 [0,4194304] 0 2024-05-27T09:29:37.957 INFO:tasks.workunit.client.1.smithi138.stdout:9/397: write d6/db/dc/f12 [409145,18120] 0 2024-05-27T09:29:37.965 INFO:tasks.workunit.client.1.smithi138.stdout:0/393: symlink da/d17/d1d/d1f/l83 0 2024-05-27T09:29:37.965 INFO:tasks.workunit.client.1.smithi138.stdout:0/394: fdatasync da/d2a/d2b/f50 0 2024-05-27T09:29:37.968 INFO:tasks.workunit.client.1.smithi138.stdout:0/395: link da/c13 da/d17/d1d/d1f/c84 0 2024-05-27T09:29:37.968 INFO:tasks.workunit.client.1.smithi138.stdout:0/396: dread - da/d2a/d2b/f4d zero size 2024-05-27T09:29:37.968 INFO:tasks.workunit.client.1.smithi138.stdout:0/397: fsync da/d17/d1d/d43/f64 0 2024-05-27T09:29:37.975 INFO:tasks.workunit.client.1.smithi138.stdout:7/385: mknod d0/d23/d49/c82 0 2024-05-27T09:29:37.990 INFO:tasks.workunit.client.0.smithi119.stdout:7/956: sync 2024-05-27T09:29:37.990 INFO:tasks.workunit.client.0.smithi119.stdout:1/921: sync 2024-05-27T09:29:37.990 INFO:tasks.workunit.client.0.smithi119.stdout:4/945: sync 2024-05-27T09:29:37.991 INFO:tasks.workunit.client.0.smithi119.stdout:2/895: sync 2024-05-27T09:29:37.991 INFO:tasks.workunit.client.0.smithi119.stdout:4/946: chown d4/d6/d28/f69 1 1 2024-05-27T09:29:37.991 INFO:tasks.workunit.client.0.smithi119.stdout:4/947: chown d4/d6/d28 111526906 1 2024-05-27T09:29:37.991 INFO:tasks.workunit.client.0.smithi119.stdout:9/792: sync 2024-05-27T09:29:37.991 INFO:tasks.workunit.client.0.smithi119.stdout:9/793: stat d18/d19/d1b/d40/d7b/caf 0 2024-05-27T09:29:37.992 INFO:tasks.workunit.client.1.smithi138.stdout:4/450: dwrite d7/df/d35/d4c/d59/f58 [0,4194304] 0 2024-05-27T09:29:37.997 INFO:tasks.workunit.client.0.smithi119.stdout:1/922: rename d7/db/d12/d26/d39/fec to d7/d40/df9/f11a 0 2024-05-27T09:29:38.003 INFO:tasks.workunit.client.1.smithi138.stdout:4/451: symlink d7/df/d35/d4c/d59/d5d/d64/l9e 0 2024-05-27T09:29:38.006 INFO:tasks.workunit.client.0.smithi119.stdout:2/896: rename d2/da/f65 to d2/da/d8a/dd4/d89/db0/f135 0 2024-05-27T09:29:38.006 INFO:tasks.workunit.client.0.smithi119.stdout:2/897: chown d2/da/d8a/dd4/d89/d94/f92 1921 1 2024-05-27T09:29:38.008 INFO:tasks.workunit.client.1.smithi138.stdout:4/452: unlink d7/d27/c30 0 2024-05-27T09:29:38.012 INFO:tasks.workunit.client.1.smithi138.stdout:7/386: write d0/d7/fe [3713652,75116] 0 2024-05-27T09:29:38.013 INFO:tasks.workunit.client.0.smithi119.stdout:5/998: dwrite d6/db/d9a/f115 [0,4194304] 0 2024-05-27T09:29:38.023 INFO:tasks.workunit.client.0.smithi119.stdout:7/957: dread d7/d1a/f59 [0,4194304] 0 2024-05-27T09:29:38.023 INFO:tasks.workunit.client.0.smithi119.stdout:7/958: truncate d7/df/d19/d9a/da6/dda/d44/ff7 28666 0 2024-05-27T09:29:38.024 INFO:tasks.workunit.client.1.smithi138.stdout:8/338: dwrite d3/d8/d11/f26 [0,4194304] 0 2024-05-27T09:29:38.024 INFO:tasks.workunit.client.1.smithi138.stdout:8/339: getdents d3/d60 0 2024-05-27T09:29:38.025 INFO:tasks.workunit.client.1.smithi138.stdout:8/340: fsync d3/d8/d32/f3b 0 2024-05-27T09:29:38.025 INFO:tasks.workunit.client.1.smithi138.stdout:8/341: chown d3/d8/c65 1798 1 2024-05-27T09:29:38.025 INFO:tasks.workunit.client.0.smithi119.stdout:7/959: rmdir d7/d6a/d7b/d11c 39 2024-05-27T09:29:38.025 INFO:tasks.workunit.client.1.smithi138.stdout:8/342: write d3/d8/d32/d34/d38/f63 [525383,70018] 0 2024-05-27T09:29:38.027 INFO:tasks.workunit.client.0.smithi119.stdout:7/960: mknod d7/df/d19/d4d/d89/dc4/c12e 0 2024-05-27T09:29:38.030 INFO:tasks.workunit.client.0.smithi119.stdout:7/961: link d7/df/d19/d9a/da6/dda/d44/fde d7/df/d19/d8d/f12f 0 2024-05-27T09:29:38.034 INFO:tasks.workunit.client.0.smithi119.stdout:1/923: dread f3 [0,4194304] 0 2024-05-27T09:29:38.036 INFO:tasks.workunit.client.1.smithi138.stdout:5/282: dwrite d3/d12/d16/f2e [0,4194304] 0 2024-05-27T09:29:38.036 INFO:tasks.workunit.client.1.smithi138.stdout:5/283: dread - d3/d12/d26/f4b zero size 2024-05-27T09:29:38.055 INFO:tasks.workunit.client.1.smithi138.stdout:4/453: rmdir d7/df/d35/d4c 39 2024-05-27T09:29:38.055 INFO:tasks.workunit.client.1.smithi138.stdout:4/454: write d7/df/d35/d4c/d59/d5d/d64/d6a/f6d [427714,101501] 0 2024-05-27T09:29:38.057 INFO:tasks.workunit.client.0.smithi119.stdout:1/924: write d7/db/d12/d15/f3f [893265,77388] 0 2024-05-27T09:29:38.057 INFO:tasks.workunit.client.0.smithi119.stdout:5/999: write d6/db/f125 [198945,34938] 0 2024-05-27T09:29:38.058 INFO:tasks.workunit.client.0.smithi119.stdout:1/925: symlink d7/db/d12/d15/d1f/dbe/l11b 0 2024-05-27T09:29:38.058 INFO:tasks.workunit.client.0.smithi119.stdout:1/926: chown d7/dd/db0/fc3 454021 1 2024-05-27T09:29:38.060 INFO:tasks.workunit.client.0.smithi119.stdout:1/927: creat d7/dd/dcb/f11c x:0 0 0 2024-05-27T09:29:38.067 INFO:tasks.workunit.client.1.smithi138.stdout:8/343: dread d3/d8/d11/f26 [0,4194304] 0 2024-05-27T09:29:38.073 INFO:tasks.workunit.client.1.smithi138.stdout:8/344: mkdir d3/d8/d24/d55/d66 0 2024-05-27T09:29:38.079 INFO:tasks.workunit.client.1.smithi138.stdout:6/505: dwrite d3/d35/f97 [0,4194304] 0 2024-05-27T09:29:38.079 INFO:tasks.workunit.client.1.smithi138.stdout:6/506: chown d3/d35/d5a 1 1 2024-05-27T09:29:38.079 INFO:tasks.workunit.client.1.smithi138.stdout:6/507: truncate d3/d35/d43/d1c/f9c 333215 0 2024-05-27T09:29:38.079 INFO:tasks.workunit.client.1.smithi138.stdout:6/508: truncate d3/fae 649921 0 2024-05-27T09:29:38.079 INFO:tasks.workunit.client.1.smithi138.stdout:6/509: write d3/d35/d43/d1c/d32/f6c [466617,104745] 0 2024-05-27T09:29:38.080 INFO:tasks.workunit.client.1.smithi138.stdout:6/510: rename d3/d11/l21 to d3/d35/lbf 0 2024-05-27T09:29:38.081 INFO:tasks.workunit.client.1.smithi138.stdout:6/511: write d3/d11/d33/f78 [444339,77753] 0 2024-05-27T09:29:38.081 INFO:tasks.workunit.client.1.smithi138.stdout:6/512: chown d3/d35/d5a/c87 8 1 2024-05-27T09:29:38.086 INFO:tasks.workunit.client.1.smithi138.stdout:8/345: fsync d3/d8/d32/d34/d38/f63 0 2024-05-27T09:29:38.086 INFO:tasks.workunit.client.1.smithi138.stdout:8/346: read - d3/d8/d64/f40 zero size 2024-05-27T09:29:38.087 INFO:tasks.workunit.client.1.smithi138.stdout:0/398: dwrite da/d17/d1d/d43/f64 [0,4194304] 0 2024-05-27T09:29:38.087 INFO:tasks.workunit.client.0.smithi119.stdout:0/883: dwrite d2/d6/d9/d12/fce [0,4194304] 0 2024-05-27T09:29:38.089 INFO:tasks.workunit.client.1.smithi138.stdout:9/398: dwrite d6/db/de/d18/f57 [0,4194304] 0 2024-05-27T09:29:38.089 INFO:tasks.workunit.client.1.smithi138.stdout:8/347: creat d3/d8/d24/d55/f67 x:0 0 0 2024-05-27T09:29:38.089 INFO:tasks.workunit.client.1.smithi138.stdout:8/348: write d3/d8/d32/d34/d38/f46 [866632,61563] 0 2024-05-27T09:29:38.094 INFO:tasks.workunit.client.1.smithi138.stdout:0/399: truncate da/d17/d1d/d1f/f26 2345804 0 2024-05-27T09:29:38.094 INFO:tasks.workunit.client.1.smithi138.stdout:0/400: creat da/d2a/f85 x:0 0 0 2024-05-27T09:29:38.095 INFO:tasks.workunit.client.0.smithi119.stdout:6/854: dwrite d0/d2/f10 [0,4194304] 0 2024-05-27T09:29:38.095 INFO:tasks.workunit.client.0.smithi119.stdout:6/855: chown d0/d2/d57/d37/d74/df1/d109/fa6 3013097 1 2024-05-27T09:29:38.096 INFO:tasks.workunit.client.1.smithi138.stdout:7/387: dwrite d0/d23/f48 [0,4194304] 0 2024-05-27T09:29:38.096 INFO:tasks.workunit.client.1.smithi138.stdout:7/388: creat d0/f83 x:0 0 0 2024-05-27T09:29:38.097 INFO:tasks.workunit.client.0.smithi119.stdout:6/856: mkdir d0/d2/dec/d130 0 2024-05-27T09:29:38.097 INFO:tasks.workunit.client.0.smithi119.stdout:6/857: write d0/d2/d57/d37/d74/df1/d109/f69 [259271,127621] 0 2024-05-27T09:29:38.100 INFO:tasks.workunit.client.0.smithi119.stdout:6/858: mknod d0/d2/d57/dd0/c131 0 2024-05-27T09:29:38.101 INFO:tasks.workunit.client.1.smithi138.stdout:8/349: symlink d3/d8/d32/d3f/l68 0 2024-05-27T09:29:38.101 INFO:tasks.workunit.client.1.smithi138.stdout:5/284: dwrite d3/d12/d32/f42 [0,4194304] 0 2024-05-27T09:29:38.101 INFO:tasks.workunit.client.1.smithi138.stdout:5/285: dread - d3/d12/d26/f45 zero size 2024-05-27T09:29:38.101 INFO:tasks.workunit.client.1.smithi138.stdout:5/286: write d3/fd [4921389,54684] 0 2024-05-27T09:29:38.104 INFO:tasks.workunit.client.1.smithi138.stdout:5/287: getdents d3/d12/d16 0 2024-05-27T09:29:38.104 INFO:tasks.workunit.client.1.smithi138.stdout:5/288: creat d3/d12/d26/f5d x:0 0 0 2024-05-27T09:29:38.104 INFO:tasks.workunit.client.1.smithi138.stdout:5/289: write d3/d12/d26/f4b [830269,123153] 0 2024-05-27T09:29:38.106 INFO:tasks.workunit.client.0.smithi119.stdout:6/859: symlink d0/d2/d57/d37/d74/d65/l132 0 2024-05-27T09:29:38.114 INFO:tasks.workunit.client.1.smithi138.stdout:5/290: getdents d3/d12/d26/d53 0 2024-05-27T09:29:38.114 INFO:tasks.workunit.client.1.smithi138.stdout:5/291: fsync d3/f5b 0 2024-05-27T09:29:38.114 INFO:tasks.workunit.client.1.smithi138.stdout:5/292: fdatasync d3/d12/d16/f2e 0 2024-05-27T09:29:38.118 INFO:tasks.workunit.client.1.smithi138.stdout:5/293: rmdir d3/d12/d16/d1d/d30/d5c 0 2024-05-27T09:29:38.118 INFO:tasks.workunit.client.1.smithi138.stdout:5/294: fsync d3/d12/d26/d53/f5a 0 2024-05-27T09:29:38.122 INFO:tasks.workunit.client.1.smithi138.stdout:5/295: creat d3/d12/d16/d1d/d22/f5e x:0 0 0 2024-05-27T09:29:38.122 INFO:tasks.workunit.client.1.smithi138.stdout:7/389: write d0/d23/f4c [75549,110252] 0 2024-05-27T09:29:38.124 INFO:tasks.workunit.client.1.smithi138.stdout:5/296: symlink d3/d12/d16/d1d/l5f 0 2024-05-27T09:29:38.124 INFO:tasks.workunit.client.1.smithi138.stdout:5/297: write d3/f21 [1366604,95487] 0 2024-05-27T09:29:38.125 INFO:tasks.workunit.client.1.smithi138.stdout:7/390: mkdir d0/d23/d3c/d84 0 2024-05-27T09:29:38.126 INFO:tasks.workunit.client.1.smithi138.stdout:6/513: rename d3/d35 to d3/d11/dc0 0 2024-05-27T09:29:38.126 INFO:tasks.workunit.client.1.smithi138.stdout:6/514: write d3/d19/f22 [4964395,81257] 0 2024-05-27T09:29:38.128 INFO:tasks.workunit.client.1.smithi138.stdout:7/391: symlink d0/d2b/d80/l85 0 2024-05-27T09:29:38.128 INFO:tasks.workunit.client.1.smithi138.stdout:7/392: write d0/d23/f5a [1851875,45195] 0 2024-05-27T09:29:38.130 INFO:tasks.workunit.client.1.smithi138.stdout:7/393: write d0/d23/d3c/f42 [593186,125240] 0 2024-05-27T09:29:38.131 INFO:tasks.workunit.client.1.smithi138.stdout:5/298: truncate d3/d12/d32/f4e 114240 0 2024-05-27T09:29:38.132 INFO:tasks.workunit.client.1.smithi138.stdout:4/455: dwrite d7/df/d35/d4c/d59/d5d/d64/d6a/f94 [0,4194304] 0 2024-05-27T09:29:38.132 INFO:tasks.workunit.client.1.smithi138.stdout:4/456: creat d7/d27/d3a/f9f x:0 0 0 2024-05-27T09:29:38.133 INFO:tasks.workunit.client.1.smithi138.stdout:7/394: unlink d0/l68 0 2024-05-27T09:29:38.133 INFO:tasks.workunit.client.1.smithi138.stdout:7/395: chown d0/d23/d3c/d3f 0 1 2024-05-27T09:29:38.134 INFO:tasks.workunit.client.1.smithi138.stdout:7/396: fdatasync d0/d23/f48 0 2024-05-27T09:29:38.134 INFO:tasks.workunit.client.1.smithi138.stdout:7/397: chown d0/d2b 10272557 1 2024-05-27T09:29:38.137 INFO:tasks.workunit.client.1.smithi138.stdout:7/398: mknod d0/d23/d3a/c86 0 2024-05-27T09:29:38.137 INFO:tasks.workunit.client.1.smithi138.stdout:7/399: write d0/f10 [2298565,25629] 0 2024-05-27T09:29:38.137 INFO:tasks.workunit.client.1.smithi138.stdout:7/400: mkdir d0/d7/d64/d87 0 2024-05-27T09:29:38.138 INFO:tasks.workunit.client.1.smithi138.stdout:7/401: write d0/d2b/f5f [670234,16824] 0 2024-05-27T09:29:38.139 INFO:tasks.workunit.client.0.smithi119.stdout:1/928: dwrite d7/dd/d44/f4e [0,4194304] 0 2024-05-27T09:29:38.139 INFO:tasks.workunit.client.0.smithi119.stdout:1/929: truncate d7/db/d12/ddb/d38/f107 623665 0 2024-05-27T09:29:38.139 INFO:tasks.workunit.client.0.smithi119.stdout:1/930: readlink d7/db/d12/d26/d39/lf7 0 2024-05-27T09:29:38.141 INFO:tasks.workunit.client.1.smithi138.stdout:5/299: mkdir d3/d12/d32/d60 0 2024-05-27T09:29:38.141 INFO:tasks.workunit.client.0.smithi119.stdout:7/962: dwrite d7/df/d19/d97/d37/fac [0,4194304] 0 2024-05-27T09:29:38.144 INFO:tasks.workunit.client.1.smithi138.stdout:4/457: creat d7/df/d5b/fa0 x:0 0 0 2024-05-27T09:29:38.144 INFO:tasks.workunit.client.1.smithi138.stdout:4/458: truncate d7/df/d35/d4c/f91 494070 0 2024-05-27T09:29:38.147 INFO:tasks.workunit.client.1.smithi138.stdout:7/402: dread d0/d23/f47 [0,4194304] 0 2024-05-27T09:29:38.148 INFO:tasks.workunit.client.0.smithi119.stdout:1/931: mkdir d7/db/d12/d26/d11d 0 2024-05-27T09:29:38.148 INFO:tasks.workunit.client.1.smithi138.stdout:5/300: creat d3/d12/d32/d60/f61 x:0 0 0 2024-05-27T09:29:38.148 INFO:tasks.workunit.client.1.smithi138.stdout:5/301: chown d3/d12/d16/c59 4723 1 2024-05-27T09:29:38.152 INFO:tasks.workunit.client.1.smithi138.stdout:7/403: dread d0/d2b/f35 [0,4194304] 0 2024-05-27T09:29:38.154 INFO:tasks.workunit.client.0.smithi119.stdout:0/884: dwrite d2/d6/d23/db8/fda [0,4194304] 0 2024-05-27T09:29:38.156 INFO:tasks.workunit.client.1.smithi138.stdout:4/459: mkdir d7/df/d5b/da1 0 2024-05-27T09:29:38.156 INFO:tasks.workunit.client.1.smithi138.stdout:6/515: write d3/d11/dc0/f60 [129120,60993] 0 2024-05-27T09:29:38.156 INFO:tasks.workunit.client.1.smithi138.stdout:6/516: chown d3/d11/dc0/d5a/d6a/l94 449 1 2024-05-27T09:29:38.159 INFO:tasks.workunit.client.1.smithi138.stdout:5/302: creat d3/d12/d16/d1d/d23/f62 x:0 0 0 2024-05-27T09:29:38.161 INFO:tasks.workunit.client.0.smithi119.stdout:6/860: dwrite d0/d2/d4e/f4b [4194304,4194304] 0 2024-05-27T09:29:38.164 INFO:tasks.workunit.client.1.smithi138.stdout:7/404: dread d0/d7/fe [0,4194304] 0 2024-05-27T09:29:38.165 INFO:tasks.workunit.client.0.smithi119.stdout:7/963: mknod d7/d1a/d107/c130 0 2024-05-27T09:29:38.166 INFO:tasks.workunit.client.0.smithi119.stdout:7/964: fdatasync d7/df/d19/d9a/da6/dda/d44/ff7 0 2024-05-27T09:29:38.167 INFO:tasks.workunit.client.0.smithi119.stdout:1/932: symlink d7/db/d12/ddb/d38/d3c/l11e 0 2024-05-27T09:29:38.168 INFO:tasks.workunit.client.1.smithi138.stdout:5/303: dread d3/f5b [0,4194304] 0 2024-05-27T09:29:38.170 INFO:tasks.workunit.client.0.smithi119.stdout:0/885: symlink d2/l129 0 2024-05-27T09:29:38.171 INFO:tasks.workunit.client.1.smithi138.stdout:7/405: rmdir d0/d7/d64 39 2024-05-27T09:29:38.171 INFO:tasks.workunit.client.1.smithi138.stdout:6/517: mkdir d3/d11/d2d/d37/d8b/dc1 0 2024-05-27T09:29:38.172 INFO:tasks.workunit.client.1.smithi138.stdout:7/406: mknod d0/d7/d64/c88 0 2024-05-27T09:29:38.173 INFO:tasks.workunit.client.0.smithi119.stdout:6/861: creat d0/d2/f133 x:0 0 0 2024-05-27T09:29:38.175 INFO:tasks.workunit.client.0.smithi119.stdout:7/965: symlink d7/df/d19/d78/l131 0 2024-05-27T09:29:38.177 INFO:tasks.workunit.client.1.smithi138.stdout:4/460: dread d7/df/f1a [0,4194304] 0 2024-05-27T09:29:38.177 INFO:tasks.workunit.client.1.smithi138.stdout:7/407: link d0/d23/d59/c70 d0/d23/d49/d78/c89 0 2024-05-27T09:29:38.177 INFO:tasks.workunit.client.1.smithi138.stdout:8/350: dwrite d3/d8/d32/f49 [0,4194304] 0 2024-05-27T09:29:38.177 INFO:tasks.workunit.client.1.smithi138.stdout:8/351: dread - d3/f52 zero size 2024-05-27T09:29:38.177 INFO:tasks.workunit.client.1.smithi138.stdout:8/352: write d3/f52 [909303,52633] 0 2024-05-27T09:29:38.177 INFO:tasks.workunit.client.1.smithi138.stdout:8/353: chown d3/d8/d32/d34/d38/f46 956881440 1 2024-05-27T09:29:38.177 INFO:tasks.workunit.client.1.smithi138.stdout:7/408: symlink d0/l8a 0 2024-05-27T09:29:38.178 INFO:tasks.workunit.client.1.smithi138.stdout:7/409: symlink d0/d23/d3a/l8b 0 2024-05-27T09:29:38.181 INFO:tasks.workunit.client.1.smithi138.stdout:1/394: sync 2024-05-27T09:29:38.181 INFO:tasks.workunit.client.1.smithi138.stdout:1/395: write d4/d48/d39/d5f/f50 [1378629,75895] 0 2024-05-27T09:29:38.182 INFO:tasks.workunit.client.1.smithi138.stdout:1/396: truncate d4/d48/d22/d65/f75 191179 0 2024-05-27T09:29:38.182 INFO:tasks.workunit.client.0.smithi119.stdout:8/868: sync 2024-05-27T09:29:38.184 INFO:tasks.workunit.client.0.smithi119.stdout:8/869: rmdir d3/d8/d1b/d46/da4/d113/db7/de0 39 2024-05-27T09:29:38.184 INFO:tasks.workunit.client.0.smithi119.stdout:8/870: read - d3/d39/d5a/f116 zero size 2024-05-27T09:29:38.184 INFO:tasks.workunit.client.0.smithi119.stdout:8/871: fdatasync d3/de/f68 0 2024-05-27T09:29:38.184 INFO:tasks.workunit.client.1.smithi138.stdout:1/397: creat d4/d1a/d2e/d32/f7a x:0 0 0 2024-05-27T09:29:38.184 INFO:tasks.workunit.client.1.smithi138.stdout:1/398: creat d4/d3c/d51/f7b x:0 0 0 2024-05-27T09:29:38.188 INFO:tasks.workunit.client.1.smithi138.stdout:1/399: rename d4/d1a/f27 to d4/d1a/d2e/f7c 0 2024-05-27T09:29:38.190 INFO:tasks.workunit.client.0.smithi119.stdout:8/872: rename d3/d8/d1b/d46/da4/d113/d66/c11f to d3/d8/d1b/d46/da4/d113/db7/de0/c122 0 2024-05-27T09:29:38.190 INFO:tasks.workunit.client.0.smithi119.stdout:1/933: dread d7/dd/d36/ffb [0,4194304] 0 2024-05-27T09:29:38.190 INFO:tasks.workunit.client.0.smithi119.stdout:1/934: stat d7/db/d12/d26/d4a/fd6 0 2024-05-27T09:29:38.197 INFO:tasks.workunit.client.0.smithi119.stdout:4/948: sync 2024-05-27T09:29:38.197 INFO:tasks.workunit.client.0.smithi119.stdout:2/898: sync 2024-05-27T09:29:38.197 INFO:tasks.workunit.client.0.smithi119.stdout:9/794: sync 2024-05-27T09:29:38.200 INFO:tasks.workunit.client.0.smithi119.stdout:4/949: rename d4/de/d21/d40/f7c to d4/d48/f132 0 2024-05-27T09:29:38.200 INFO:tasks.workunit.client.0.smithi119.stdout:4/950: chown d4/d6/d81 1556654 1 2024-05-27T09:29:38.201 INFO:tasks.workunit.client.1.smithi138.stdout:5/304: read d3/d12/d32/f4e [7025,44993] 0 2024-05-27T09:29:38.203 INFO:tasks.workunit.client.1.smithi138.stdout:5/305: truncate d3/d12/d32/f4e 631365 0 2024-05-27T09:29:38.206 INFO:tasks.workunit.client.0.smithi119.stdout:4/951: creat d4/de/d21/d40/d47/d5c/d9e/dd3/f133 x:0 0 0 2024-05-27T09:29:38.210 INFO:tasks.workunit.client.0.smithi119.stdout:0/886: write d2/d6/d75/f8a [1574300,80205] 0 2024-05-27T09:29:38.213 INFO:tasks.workunit.client.1.smithi138.stdout:1/400: dread d4/d48/d39/d5f/f50 [0,4194304] 0 2024-05-27T09:29:38.214 INFO:tasks.workunit.client.0.smithi119.stdout:8/873: write d3/d6e/dcf/dce/fd8 [384785,2045] 0 2024-05-27T09:29:38.215 INFO:tasks.workunit.client.1.smithi138.stdout:1/401: mknod d4/d1a/c7d 0 2024-05-27T09:29:38.215 INFO:tasks.workunit.client.0.smithi119.stdout:8/874: rmdir d3/d8/d1b/d46 39 2024-05-27T09:29:38.217 INFO:tasks.workunit.client.1.smithi138.stdout:1/402: mkdir d4/d1a/d2e/d4a/d7e 0 2024-05-27T09:29:38.217 INFO:tasks.workunit.client.1.smithi138.stdout:1/403: write d4/d48/d39/d5f/f50 [2288682,70939] 0 2024-05-27T09:29:38.220 INFO:tasks.workunit.client.1.smithi138.stdout:1/404: mknod d4/c7f 0 2024-05-27T09:29:38.229 INFO:tasks.workunit.client.0.smithi119.stdout:2/899: dread d2/da/d8a/dd4/d89/d94/d87/f9a [0,4194304] 0 2024-05-27T09:29:38.236 INFO:tasks.workunit.client.0.smithi119.stdout:6/862: dwrite d0/d2/d1c/d21/d58/dd3/f12e [0,4194304] 0 2024-05-27T09:29:38.237 INFO:tasks.workunit.client.0.smithi119.stdout:8/875: write d3/de/dad/ffa [2059620,12621] 0 2024-05-27T09:29:38.237 INFO:tasks.workunit.client.0.smithi119.stdout:8/876: dread - d3/faf zero size 2024-05-27T09:29:38.239 INFO:tasks.workunit.client.1.smithi138.stdout:1/405: fdatasync d4/d48/d39/d5f/f50 0 2024-05-27T09:29:38.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:38.238+0000 7ff767400700 1 -- 172.21.15.119:0/348631982 --> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff740000bf0 con 0x7ff748072810 2024-05-27T09:29:38.240 INFO:tasks.workunit.client.0.smithi119.stdout:6/863: unlink d0/d2/d1c/d21/lf3 0 2024-05-27T09:29:38.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:38.240+0000 7ff75d600700 1 -- 172.21.15.119:0/348631982 <== mgr.14570 v2:172.21.15.138:6828/1384904888 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7ff740000bf0 con 0x7ff748072810 2024-05-27T09:29:38.241 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:29:38.241 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:29:38.241 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:29:38.241 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-27T09:29:38.241 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [], 2024-05-27T09:29:38.241 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "1/2 daemons upgraded", 2024-05-27T09:29:38.241 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "", 2024-05-27T09:29:38.241 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:29:38.241 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:29:38.242 INFO:tasks.workunit.client.0.smithi119.stdout:4/952: dread d4/de/d66/f68 [0,4194304] 0 2024-05-27T09:29:38.243 INFO:tasks.workunit.client.0.smithi119.stdout:6/864: symlink d0/d2/d57/d37/d115/l134 0 2024-05-27T09:29:38.243 INFO:tasks.workunit.client.0.smithi119.stdout:6/865: chown d0/d2/d57/d37/d74/f70 8 1 2024-05-27T09:29:38.243 INFO:tasks.workunit.client.0.smithi119.stdout:6/866: readlink d0/d2/d4e/l2f 0 2024-05-27T09:29:38.246 INFO:tasks.workunit.client.0.smithi119.stdout:9/795: dread d18/d19/d1b/d40/d7b/dc6/ff0 [0,4194304] 0 2024-05-27T09:29:38.246 INFO:tasks.workunit.client.0.smithi119.stdout:9/796: write f6 [4388796,55846] 0 2024-05-27T09:29:38.246 INFO:tasks.workunit.client.0.smithi119.stdout:9/797: stat d18/d19/d61/fed 0 2024-05-27T09:29:38.246 INFO:tasks.workunit.client.0.smithi119.stdout:9/798: chown d18/d19/d1b/d29/dc4/de1/fe5 15448541 1 2024-05-27T09:29:38.246 INFO:tasks.workunit.client.0.smithi119.stdout:9/799: dread - d18/d19/d61/fed zero size 2024-05-27T09:29:38.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:38.245+0000 7ff767400700 1 -- 172.21.15.119:0/348631982 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7ff748072810 msgr2=0x7ff748074cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:38.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:38.245+0000 7ff767400700 1 --2- 172.21.15.119:0/348631982 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7ff748072810 0x7ff748074cc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0097b0 tx=0x7ff74c006d20 comp rx=0 tx=0).stop 2024-05-27T09:29:38.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:38.245+0000 7ff767400700 1 -- 172.21.15.119:0/348631982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff760100150 msgr2=0x7ff760197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:29:38.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:38.245+0000 7ff767400700 1 --2- 172.21.15.119:0/348631982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff760100150 0x7ff760197c40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff75400bd70 tx=0x7ff75400be50 comp rx=0 tx=0).stop 2024-05-27T09:29:38.247 INFO:tasks.workunit.client.0.smithi119.stdout:4/953: truncate d4/d4e/fb7 3864993 0 2024-05-27T09:29:38.247 INFO:tasks.workunit.client.0.smithi119.stdout:4/954: fdatasync d4/d6/d3e/f6e 0 2024-05-27T09:29:38.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:38.246+0000 7ff767400700 1 -- 172.21.15.119:0/348631982 shutdown_connections 2024-05-27T09:29:38.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:38.246+0000 7ff767400700 1 --2- 172.21.15.119:0/348631982 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7ff748072810 0x7ff748074cc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:38.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:38.246+0000 7ff767400700 1 --2- 172.21.15.119:0/348631982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff760100150 0x7ff760197c40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:38.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:38.246+0000 7ff767400700 1 --2- 172.21.15.119:0/348631982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff760101350 0x7ff760198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:29:38.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:38.246+0000 7ff767400700 1 -- 172.21.15.119:0/348631982 >> 172.21.15.119:0/348631982 conn(0x7ff7600fb6e0 msgr2=0x7ff760102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:29:38.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:38.246+0000 7ff767400700 1 -- 172.21.15.119:0/348631982 shutdown_connections 2024-05-27T09:29:38.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:29:38.246+0000 7ff767400700 1 -- 172.21.15.119:0/348631982 wait complete. 2024-05-27T09:29:38.249 INFO:tasks.workunit.client.0.smithi119.stdout:2/900: dread d2/da/d8a/dd4/d89/d94/d87/f118 [0,4194304] 0 2024-05-27T09:29:38.249 INFO:tasks.workunit.client.0.smithi119.stdout:2/901: fsync d2/f2d 0 2024-05-27T09:29:38.249 INFO:tasks.workunit.client.0.smithi119.stdout:6/867: creat d0/d2/d11b/f135 x:0 0 0 2024-05-27T09:29:38.251 INFO:tasks.workunit.client.1.smithi138.stdout:1/406: creat d4/d1a/d2e/d4a/f80 x:0 0 0 2024-05-27T09:29:38.253 INFO:tasks.workunit.client.1.smithi138.stdout:5/306: dread d3/f10 [0,4194304] 0 2024-05-27T09:29:38.254 INFO:tasks.workunit.client.1.smithi138.stdout:5/307: write d3/d12/f4d [308255,126721] 0 2024-05-27T09:29:38.254 INFO:tasks.workunit.client.1.smithi138.stdout:5/308: truncate d3/d12/d16/d1d/d22/f2a 991039 0 2024-05-27T09:29:38.258 INFO:tasks.workunit.client.1.smithi138.stdout:1/407: link d4/d48/l3a d4/d48/l81 0 2024-05-27T09:29:38.259 INFO:tasks.workunit.client.1.smithi138.stdout:4/461: read d7/df/d5b/f7c [1304379,121815] 0 2024-05-27T09:29:38.259 INFO:tasks.workunit.client.1.smithi138.stdout:4/462: fsync d7/f60 0 2024-05-27T09:29:38.260 INFO:tasks.workunit.client.1.smithi138.stdout:4/463: chown d7/df/f29 179 1 2024-05-27T09:29:38.260 INFO:tasks.workunit.client.1.smithi138.stdout:5/309: rename d3/c20 to d3/d12/d16/d1d/d22/d2b/c63 0 2024-05-27T09:29:38.260 INFO:tasks.workunit.client.0.smithi119.stdout:1/935: dread d7/dd/d25/d60/d61/fb3 [0,4194304] 0 2024-05-27T09:29:38.260 INFO:tasks.workunit.client.0.smithi119.stdout:8/877: read d3/d8/d1b/db1/fbc [3061589,23424] 0 2024-05-27T09:29:38.260 INFO:tasks.workunit.client.0.smithi119.stdout:4/955: mknod d4/db1/c134 0 2024-05-27T09:29:38.260 INFO:tasks.workunit.client.0.smithi119.stdout:2/902: creat d2/da/d8a/dd4/d89/de8/d12f/f136 x:0 0 0 2024-05-27T09:29:38.260 INFO:tasks.workunit.client.0.smithi119.stdout:2/903: write d2/da/d4e/fe3 [440401,46621] 0 2024-05-27T09:29:38.260 INFO:tasks.workunit.client.0.smithi119.stdout:6/868: rename d0/d2/d1c/d21/d58/c9e to d0/d2/d1c/c136 0 2024-05-27T09:29:38.260 INFO:tasks.workunit.client.0.smithi119.stdout:6/869: fsync d0/d2/d57/d37/d74/f70 0 2024-05-27T09:29:38.261 INFO:tasks.workunit.client.1.smithi138.stdout:4/464: write d7/d27/f48 [1148633,53005] 0 2024-05-27T09:29:38.261 INFO:tasks.workunit.client.1.smithi138.stdout:5/310: write d3/d12/d16/d1d/d22/d2b/d36/f41 [1645715,63609] 0 2024-05-27T09:29:38.261 INFO:tasks.workunit.client.1.smithi138.stdout:4/465: creat d7/df/d35/fa2 x:0 0 0 2024-05-27T09:29:38.261 INFO:tasks.workunit.client.1.smithi138.stdout:4/466: chown d7/df/d5b/f7c 204852 1 2024-05-27T09:29:38.261 INFO:tasks.workunit.client.0.smithi119.stdout:4/956: mknod d4/d6/d28/c135 0 2024-05-27T09:29:38.265 INFO:tasks.workunit.client.0.smithi119.stdout:2/904: write d2/da/d4e/f72 [259360,115869] 0 2024-05-27T09:29:38.266 INFO:tasks.workunit.client.0.smithi119.stdout:9/800: mkdir d18/d19/dfa 0 2024-05-27T09:29:38.266 INFO:tasks.workunit.client.0.smithi119.stdout:6/870: unlink d0/d2/d57/d37/cc3 0 2024-05-27T09:29:38.266 INFO:tasks.workunit.client.0.smithi119.stdout:6/871: readlink d0/d2/d1c/d21/d58/ld7 0 2024-05-27T09:29:38.266 INFO:tasks.workunit.client.0.smithi119.stdout:9/801: write d18/d19/d1b/d40/d7b/ff1 [379317,86597] 0 2024-05-27T09:29:38.268 INFO:tasks.workunit.client.1.smithi138.stdout:5/311: rmdir d3/d12/d32/d60 39 2024-05-27T09:29:38.269 INFO:tasks.workunit.client.0.smithi119.stdout:0/887: dwrite d2/d6/d9/d12/d4f/ff1 [0,4194304] 0 2024-05-27T09:29:38.269 INFO:tasks.workunit.client.0.smithi119.stdout:0/888: chown d2/d6/d9/d12/l101 11080 1 2024-05-27T09:29:38.270 INFO:tasks.workunit.client.0.smithi119.stdout:2/905: creat d2/da/d8a/dd4/d89/f137 x:0 0 0 2024-05-27T09:29:38.270 INFO:tasks.workunit.client.0.smithi119.stdout:6/872: symlink d0/d2/d57/d37/de5/d122/l137 0 2024-05-27T09:29:38.270 INFO:tasks.workunit.client.0.smithi119.stdout:2/906: stat d2/da/d8a/dd4/d89/d94/f8b 0 2024-05-27T09:29:38.270 INFO:tasks.workunit.client.0.smithi119.stdout:6/873: getdents d0/d2/dec/d130 0 2024-05-27T09:29:38.270 INFO:tasks.workunit.client.0.smithi119.stdout:6/874: fdatasync d0/d2/d11b/f135 0 2024-05-27T09:29:38.271 INFO:tasks.workunit.client.1.smithi138.stdout:5/312: creat d3/d12/f64 x:0 0 0 2024-05-27T09:29:38.273 INFO:tasks.workunit.client.0.smithi119.stdout:9/802: creat d18/d19/d1b/d40/d44/ffb x:0 0 0 2024-05-27T09:29:38.273 INFO:tasks.workunit.client.0.smithi119.stdout:9/803: creat d18/d19/d1b/d29/dc4/de1/ffc x:0 0 0 2024-05-27T09:29:38.273 INFO:tasks.workunit.client.0.smithi119.stdout:9/804: fsync d18/d19/f3f 0 2024-05-27T09:29:38.273 INFO:tasks.workunit.client.0.smithi119.stdout:9/805: chown d18/d19/d1b/d29/f43 109375609 1 2024-05-27T09:29:38.282 INFO:tasks.workunit.client.0.smithi119.stdout:0/889: creat d2/d6/d9/d24/d30/dcc/d120/f12a x:0 0 0 2024-05-27T09:29:38.283 INFO:tasks.workunit.client.0.smithi119.stdout:2/907: mknod d2/da/d8a/dd4/d69/c138 0 2024-05-27T09:29:38.285 INFO:tasks.workunit.client.0.smithi119.stdout:9/806: symlink d18/d19/d61/lfd 0 2024-05-27T09:29:38.286 INFO:tasks.workunit.client.1.smithi138.stdout:6/518: dwrite d3/f84 [0,4194304] 0 2024-05-27T09:29:38.287 INFO:tasks.workunit.client.0.smithi119.stdout:0/890: symlink d2/d6/d9/d24/d30/dcc/de6/l12b 0 2024-05-27T09:29:38.289 INFO:tasks.workunit.client.1.smithi138.stdout:5/313: rmdir d3/d12/d26 39 2024-05-27T09:29:38.289 INFO:tasks.workunit.client.0.smithi119.stdout:2/908: link d2/da/d8a/dd4/d89/d94/c6a d2/da/d15/d53/d114/d116/d12a/c139 0 2024-05-27T09:29:38.290 INFO:tasks.workunit.client.0.smithi119.stdout:2/909: chown d2/da/d8a/dd4/d89/d94/de2/df9 15996 1 2024-05-27T09:29:38.293 INFO:tasks.workunit.client.1.smithi138.stdout:5/314: symlink d3/d12/d16/d1d/d22/l65 0 2024-05-27T09:29:38.298 INFO:tasks.workunit.client.1.smithi138.stdout:5/315: rename d3/d12/d32/f42 to d3/d12/d16/f66 0 2024-05-27T09:29:38.298 INFO:tasks.workunit.client.1.smithi138.stdout:1/408: dread d4/d38/f44 [0,4194304] 0 2024-05-27T09:29:38.298 INFO:tasks.workunit.client.1.smithi138.stdout:5/316: getdents d3/d12/d32/d60 0 2024-05-27T09:29:38.298 INFO:tasks.workunit.client.1.smithi138.stdout:5/317: write d3/f19 [1032210,68928] 0 2024-05-27T09:29:38.299 INFO:tasks.workunit.client.0.smithi119.stdout:2/910: rename d2/da/d8a/d8f/d11e/f128 to d2/da/d8a/dd4/d89/db0/f13a 0 2024-05-27T09:29:38.299 INFO:tasks.workunit.client.0.smithi119.stdout:9/807: getdents d18/d19/d1b/d40/d44/dcc 0 2024-05-27T09:29:38.299 INFO:tasks.workunit.client.0.smithi119.stdout:9/808: write d18/d19/d61/fed [148158,41349] 0 2024-05-27T09:29:38.299 INFO:tasks.workunit.client.0.smithi119.stdout:8/878: read d3/d8/d1b/f41 [1317885,103525] 0 2024-05-27T09:29:38.299 INFO:tasks.workunit.client.0.smithi119.stdout:8/879: chown d3/d8/d1b/d46/da4/dbb/f11b 11 1 2024-05-27T09:29:38.299 INFO:tasks.workunit.client.0.smithi119.stdout:8/880: chown d3/d6e/dcf/dce 29284 1 2024-05-27T09:29:38.300 INFO:tasks.workunit.client.1.smithi138.stdout:1/409: symlink d4/d1a/l82 0 2024-05-27T09:29:38.300 INFO:tasks.workunit.client.1.smithi138.stdout:1/410: rmdir d4/d1a/d2e/d32 39 2024-05-27T09:29:38.306 INFO:tasks.workunit.client.0.smithi119.stdout:7/966: dwrite d7/df/d19/d4d/d89/fe8 [0,4194304] 0 2024-05-27T09:29:38.310 INFO:tasks.workunit.client.1.smithi138.stdout:5/318: write d3/d12/d16/f39 [5234571,3119] 0 2024-05-27T09:29:38.310 INFO:tasks.workunit.client.1.smithi138.stdout:4/467: dread d7/df/d35/f8a [0,4194304] 0 2024-05-27T09:29:38.311 INFO:tasks.workunit.client.1.smithi138.stdout:6/519: dread d3/d19/f1e [0,4194304] 0 2024-05-27T09:29:38.311 INFO:tasks.workunit.client.1.smithi138.stdout:6/520: chown d3/d11/d2d/d37/d8b/laf 860396 1 2024-05-27T09:29:38.311 INFO:tasks.workunit.client.1.smithi138.stdout:6/521: dread - d3/d11/d2d/d37/d7e/f9f zero size 2024-05-27T09:29:38.313 INFO:tasks.workunit.client.0.smithi119.stdout:7/967: unlink d7/df/d19/d9a/da6/dda/f48 0 2024-05-27T09:29:38.314 INFO:tasks.workunit.client.1.smithi138.stdout:8/354: dwrite d3/d8/d32/d34/d38/f5d [0,4194304] 0 2024-05-27T09:29:38.315 INFO:tasks.workunit.client.1.smithi138.stdout:4/468: link d7/c1e d7/df/d35/d4c/ca3 0 2024-05-27T09:29:38.317 INFO:tasks.workunit.client.0.smithi119.stdout:8/881: read d3/f25 [4080694,123056] 0 2024-05-27T09:29:38.320 INFO:tasks.workunit.client.1.smithi138.stdout:8/355: rename d3/d8/c16 to d3/d8/d24/d4d/c69 0 2024-05-27T09:29:38.321 INFO:tasks.workunit.client.0.smithi119.stdout:7/968: creat d7/df/d19/d9a/da6/dda/f132 x:0 0 0 2024-05-27T09:29:38.322 INFO:tasks.workunit.client.1.smithi138.stdout:8/356: mknod d3/d8/d32/d45/d5c/c6a 0 2024-05-27T09:29:38.328 INFO:tasks.workunit.client.1.smithi138.stdout:6/522: truncate d3/d11/dc0/f60 794872 0 2024-05-27T09:29:38.328 INFO:tasks.workunit.client.1.smithi138.stdout:5/319: write d3/f10 [3400503,86422] 0 2024-05-27T09:29:38.328 INFO:tasks.workunit.client.1.smithi138.stdout:5/320: creat d3/d12/d16/d1d/d23/f67 x:0 0 0 2024-05-27T09:29:38.329 INFO:tasks.workunit.client.1.smithi138.stdout:5/321: write d3/d12/d16/f39 [6190854,48853] 0 2024-05-27T09:29:38.329 INFO:tasks.workunit.client.1.smithi138.stdout:8/357: creat d3/d8/d32/d3f/f6b x:0 0 0 2024-05-27T09:29:38.330 INFO:tasks.workunit.client.0.smithi119.stdout:8/882: creat d3/d8/d1b/db1/df9/f123 x:0 0 0 2024-05-27T09:29:38.331 INFO:tasks.workunit.client.1.smithi138.stdout:4/469: link d7/df/d35/d4c/d59/d5d/f87 d7/df/d35/d4c/d59/fa4 0 2024-05-27T09:29:38.334 INFO:tasks.workunit.client.0.smithi119.stdout:7/969: unlink d7/df/d19/d97/d92/db7/dba/d42/dc5/cfb 0 2024-05-27T09:29:38.335 INFO:tasks.workunit.client.1.smithi138.stdout:8/358: mkdir d3/d8/d6c 0 2024-05-27T09:29:38.335 INFO:tasks.workunit.client.1.smithi138.stdout:1/411: write d4/d38/f44 [283441,30896] 0 2024-05-27T09:29:38.335 INFO:tasks.workunit.client.1.smithi138.stdout:1/412: chown d4/l77 12263 1 2024-05-27T09:29:38.335 INFO:tasks.workunit.client.1.smithi138.stdout:1/413: fdatasync d4/d1a/d2e/f79 0 2024-05-27T09:29:38.341 INFO:tasks.workunit.client.1.smithi138.stdout:8/359: rename d3/d8/d11/l20 to d3/d8/d24/d55/d66/l6d 0 2024-05-27T09:29:38.354 INFO:tasks.workunit.client.0.smithi119.stdout:4/957: dwrite d4/d1c/f29 [0,4194304] 0 2024-05-27T09:29:38.355 INFO:tasks.workunit.client.1.smithi138.stdout:6/523: write d3/d11/dc0/d43/f29 [3579925,116652] 0 2024-05-27T09:29:38.355 INFO:tasks.workunit.client.1.smithi138.stdout:6/524: write d3/d19/fab [796604,52137] 0 2024-05-27T09:29:38.358 INFO:tasks.workunit.client.0.smithi119.stdout:4/958: truncate d4/d23/d31/f93 185866 0 2024-05-27T09:29:38.359 INFO:tasks.workunit.client.0.smithi119.stdout:4/959: truncate d4/de/d21/d40/f42 5085671 0 2024-05-27T09:29:38.359 INFO:tasks.workunit.client.0.smithi119.stdout:4/960: read - d4/d23/d24/d5b/da8/fe7 zero size 2024-05-27T09:29:38.359 INFO:tasks.workunit.client.0.smithi119.stdout:4/961: readlink d4/d6/d81/l9c 0 2024-05-27T09:29:38.363 INFO:tasks.workunit.client.1.smithi138.stdout:6/525: truncate d3/d11/d2d/d37/f4a 2061190 0 2024-05-27T09:29:38.363 INFO:tasks.workunit.client.1.smithi138.stdout:1/414: getdents d4/d1a 0 2024-05-27T09:29:38.363 INFO:tasks.workunit.client.1.smithi138.stdout:5/322: write d3/d12/d16/f2e [495899,73571] 0 2024-05-27T09:29:38.364 INFO:tasks.workunit.client.1.smithi138.stdout:5/323: write d3/d12/d16/d1d/d30/f4a [3458820,112343] 0 2024-05-27T09:29:38.364 INFO:tasks.workunit.client.1.smithi138.stdout:5/324: dread - d3/d12/d16/d1d/d23/f62 zero size 2024-05-27T09:29:38.364 INFO:tasks.workunit.client.1.smithi138.stdout:5/325: chown d3/d12/d16/d1d/d22/d2b/d36/f43 724243155 1 2024-05-27T09:29:38.365 INFO:tasks.workunit.client.0.smithi119.stdout:4/962: mknod d4/d6/d3e/dd2/c136 0 2024-05-27T09:29:38.369 INFO:tasks.workunit.client.0.smithi119.stdout:4/963: getdents d4/d23/d24/d121 0 2024-05-27T09:29:38.370 INFO:tasks.workunit.client.0.smithi119.stdout:4/964: chown d4/d6/d28/d2f/d36/f72 30559738 1 2024-05-27T09:29:38.370 INFO:tasks.workunit.client.0.smithi119.stdout:2/911: dread d2/da/d8a/dd4/d89/d94/d87/fba [0,4194304] 0 2024-05-27T09:29:38.370 INFO:tasks.workunit.client.0.smithi119.stdout:4/965: fdatasync d4/de/d21/d40/d47/f90 0 2024-05-27T09:29:38.371 INFO:tasks.workunit.client.1.smithi138.stdout:6/526: mkdir d3/d11/dc2 0 2024-05-27T09:29:38.371 INFO:tasks.workunit.client.1.smithi138.stdout:6/527: fsync d3/d11/dc0/fbd 0 2024-05-27T09:29:38.380 INFO:tasks.workunit.client.0.smithi119.stdout:4/966: creat d4/de/d21/d40/d110/f137 x:0 0 0 2024-05-27T09:29:38.382 INFO:tasks.workunit.client.1.smithi138.stdout:8/360: write d3/d8/d11/f22 [2170585,88642] 0 2024-05-27T09:29:38.382 INFO:tasks.workunit.client.1.smithi138.stdout:8/361: chown d3/d8/d32/d45/d5c 861787 1 2024-05-27T09:29:38.384 INFO:tasks.workunit.client.0.smithi119.stdout:8/883: read d3/f8c [570530,38907] 0 2024-05-27T09:29:38.385 INFO:tasks.workunit.client.1.smithi138.stdout:5/326: mkdir d3/d12/d16/d1d/d23/d68 0 2024-05-27T09:29:38.387 INFO:tasks.workunit.client.1.smithi138.stdout:1/415: link d4/d48/d39/d5f/l66 d4/d3c/d51/l83 0 2024-05-27T09:29:38.390 INFO:tasks.workunit.client.0.smithi119.stdout:6/875: dwrite d0/d2/d11b/f135 [0,4194304] 0 2024-05-27T09:29:38.394 INFO:tasks.workunit.client.0.smithi119.stdout:4/967: mknod d4/de/d21/d40/d47/c138 0 2024-05-27T09:29:38.394 INFO:tasks.workunit.client.1.smithi138.stdout:8/362: dread d3/f52 [0,4194304] 0 2024-05-27T09:29:38.395 INFO:tasks.workunit.client.1.smithi138.stdout:8/363: read f1 [2551818,58429] 0 2024-05-27T09:29:38.396 INFO:tasks.workunit.client.1.smithi138.stdout:8/364: link d3/l1d d3/d8/d32/d45/d5c/l6e 0 2024-05-27T09:29:38.400 INFO:tasks.workunit.client.1.smithi138.stdout:8/365: stat d3/d8/d32/d45/l56 0 2024-05-27T09:29:38.404 INFO:tasks.workunit.client.0.smithi119.stdout:8/884: rename d3/d6e/d80/d84/ldc to d3/d8/d1b/d46/da4/d117/l124 0 2024-05-27T09:29:38.404 INFO:tasks.workunit.client.0.smithi119.stdout:7/970: dwrite d7/df/d19/d97/d92/db7/dba/d42/d76/f9d [0,4194304] 0 2024-05-27T09:29:38.406 INFO:tasks.workunit.client.1.smithi138.stdout:6/528: getdents d3/d19 0 2024-05-27T09:29:38.408 INFO:tasks.workunit.client.0.smithi119.stdout:1/936: dwrite d7/db/d12/ddb/d38/d3c/ff3 [0,4194304] 0 2024-05-27T09:29:38.412 INFO:tasks.workunit.client.1.smithi138.stdout:1/416: mknod d4/d48/d39/d5f/d4d/c84 0 2024-05-27T09:29:38.414 INFO:tasks.workunit.client.0.smithi119.stdout:4/968: mkdir d4/d1c/d139 0 2024-05-27T09:29:38.415 INFO:tasks.workunit.client.0.smithi119.stdout:4/969: readlink d4/d6/d28/l76 0 2024-05-27T09:29:38.415 INFO:tasks.workunit.client.0.smithi119.stdout:4/970: chown d4/d23/d24/d5b/da8/dda/dea 226 1 2024-05-27T09:29:38.415 INFO:tasks.workunit.client.0.smithi119.stdout:4/971: write d4/de/d9b/ff1 [1604344,100100] 0 2024-05-27T09:29:38.415 INFO:tasks.workunit.client.0.smithi119.stdout:4/972: write d4/d6/fd [94404,52080] 0 2024-05-27T09:29:38.415 INFO:tasks.workunit.client.0.smithi119.stdout:4/973: fdatasync d4/d23/d24/d5b/da8/dda/dea/f111 0 2024-05-27T09:29:38.425 INFO:tasks.workunit.client.0.smithi119.stdout:7/971: rename d7/df/d19/d4d/d89/f10b to d7/d5f/d79/d12d/f133 0 2024-05-27T09:29:38.429 INFO:tasks.workunit.client.0.smithi119.stdout:0/891: dwrite d2/d6/d9/d24/d69/f115 [0,4194304] 0 2024-05-27T09:29:38.429 INFO:tasks.workunit.client.0.smithi119.stdout:0/892: chown d2/d6/d9/d24/d30/c71 14 1 2024-05-27T09:29:38.429 INFO:tasks.workunit.client.0.smithi119.stdout:0/893: truncate d2/d6/d9/d24/d30/dcc/de6/f104 454106 0 2024-05-27T09:29:38.429 INFO:tasks.workunit.client.0.smithi119.stdout:0/894: stat d2/d6/d9/f6d 0 2024-05-27T09:29:38.429 INFO:tasks.workunit.client.0.smithi119.stdout:4/974: symlink d4/d6/d3e/dd2/l13a 0 2024-05-27T09:29:38.429 INFO:tasks.workunit.client.0.smithi119.stdout:4/975: chown d4/d1c/d39/cf9 169758603 1 2024-05-27T09:29:38.430 INFO:tasks.workunit.client.1.smithi138.stdout:4/470: dwrite d7/d6e/f9b [0,4194304] 0 2024-05-27T09:29:38.430 INFO:tasks.workunit.client.1.smithi138.stdout:4/471: fsync d7/f39 0 2024-05-27T09:29:38.430 INFO:tasks.workunit.client.1.smithi138.stdout:4/472: write d7/df/f68 [1050376,37089] 0 2024-05-27T09:29:38.430 INFO:tasks.workunit.client.1.smithi138.stdout:4/473: fsync f3 0 2024-05-27T09:29:38.430 INFO:tasks.workunit.client.1.smithi138.stdout:4/474: write d7/d27/f98 [569730,45451] 0 2024-05-27T09:29:38.431 INFO:tasks.workunit.client.0.smithi119.stdout:0/895: creat d2/d6/d9/d24/d30/dcc/d124/f12c x:0 0 0 2024-05-27T09:29:38.431 INFO:tasks.workunit.client.0.smithi119.stdout:0/896: dread - d2/d6/d9/d24/fcb zero size 2024-05-27T09:29:38.431 INFO:tasks.workunit.client.0.smithi119.stdout:0/897: readlink d2/l129 0 2024-05-27T09:29:38.432 INFO:tasks.workunit.client.0.smithi119.stdout:4/976: mkdir d4/d23/d24/d121/d13b 0 2024-05-27T09:29:38.432 INFO:tasks.workunit.client.0.smithi119.stdout:4/977: chown d4/d6/d28/d2f/f11e 221014739 1 2024-05-27T09:29:38.434 INFO:tasks.workunit.client.0.smithi119.stdout:9/809: dwrite d18/d19/d1b/d40/fe9 [0,4194304] 0 2024-05-27T09:29:38.434 INFO:tasks.workunit.client.0.smithi119.stdout:0/898: link d2/d6/d9/d24/d30/c99 d2/d6/d9/dab/c12d 0 2024-05-27T09:29:38.443 INFO:tasks.workunit.client.1.smithi138.stdout:4/475: rename d7/d27/d3a/f55 to d7/df/d35/d4c/d59/fa5 0 2024-05-27T09:29:38.447 INFO:tasks.workunit.client.0.smithi119.stdout:1/937: dread d7/dd/d44/f4e [0,4194304] 0 2024-05-27T09:29:38.452 INFO:tasks.workunit.client.0.smithi119.stdout:7/972: write d7/d5f/d79/f118 [84259,102901] 0 2024-05-27T09:29:38.452 INFO:tasks.workunit.client.1.smithi138.stdout:4/476: unlink d7/d27/d3a/f9f 0 2024-05-27T09:29:38.456 INFO:tasks.workunit.client.0.smithi119.stdout:0/899: mknod d2/d6/d9/c12e 0 2024-05-27T09:29:38.459 INFO:tasks.workunit.client.0.smithi119.stdout:1/938: rmdir d7/db 39 2024-05-27T09:29:38.463 INFO:tasks.workunit.client.0.smithi119.stdout:7/973: rename f1 to d7/df/d19/d9a/da6/dda/d44/f134 0 2024-05-27T09:29:38.466 INFO:tasks.workunit.client.1.smithi138.stdout:4/477: mkdir d7/df/d35/da6 0 2024-05-27T09:29:38.466 INFO:tasks.workunit.client.1.smithi138.stdout:4/478: write d7/f78 [1042596,516] 0 2024-05-27T09:29:38.472 INFO:tasks.workunit.client.0.smithi119.stdout:4/978: write d4/d6/fd1 [367065,11880] 0 2024-05-27T09:29:38.475 INFO:tasks.workunit.client.0.smithi119.stdout:7/974: creat d7/d1a/d107/d111/f135 x:0 0 0 2024-05-27T09:29:38.477 INFO:tasks.workunit.client.1.smithi138.stdout:3/419: sync 2024-05-27T09:29:38.477 INFO:tasks.workunit.client.1.smithi138.stdout:2/336: sync 2024-05-27T09:29:38.480 INFO:tasks.workunit.client.1.smithi138.stdout:2/337: mknod d1/d15/d2d/d17/d25/d4d/c6f 0 2024-05-27T09:29:38.483 INFO:tasks.workunit.client.1.smithi138.stdout:2/338: mknod d1/d15/d2d/d17/c70 0 2024-05-27T09:29:38.490 INFO:tasks.workunit.client.0.smithi119.stdout:7/975: mknod d7/df/d19/d46/d93/c136 0 2024-05-27T09:29:38.490 INFO:tasks.workunit.client.0.smithi119.stdout:4/979: dread d4/d6/d28/d2f/f50 [0,4194304] 0 2024-05-27T09:29:38.491 INFO:tasks.workunit.client.1.smithi138.stdout:2/339: rename d1/d15/d2d/d31/d4b/c57 to d1/d15/d2d/d17/d25/d4d/c71 0 2024-05-27T09:29:38.494 INFO:tasks.workunit.client.1.smithi138.stdout:2/340: symlink d1/d15/d2d/d17/d25/d4d/d4e/l72 0 2024-05-27T09:29:38.497 INFO:tasks.workunit.client.1.smithi138.stdout:2/341: creat d1/d10/d52/f73 x:0 0 0 2024-05-27T09:29:38.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:38 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/2099298037' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:29:38.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:38 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:38.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:38 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:38.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:38 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:38.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:38 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:38.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:38 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:38.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:38 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:38.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:38 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:38.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:38 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:38.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:38 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:38.500 INFO:tasks.workunit.client.1.smithi138.stdout:5/327: getdents d3/d12/d16 0 2024-05-27T09:29:38.501 INFO:tasks.workunit.client.0.smithi119.stdout:8/885: dwrite d3/d39/f61 [0,4194304] 0 2024-05-27T09:29:38.502 INFO:tasks.workunit.client.0.smithi119.stdout:8/886: chown d3/d8/d4d/d79/fc7 2918153 1 2024-05-27T09:29:38.502 INFO:tasks.workunit.client.0.smithi119.stdout:8/887: chown d3/d6e/d80/c10a 223 1 2024-05-27T09:29:38.504 INFO:tasks.workunit.client.1.smithi138.stdout:3/420: write d0/d3f/d50/f63 [612364,18585] 0 2024-05-27T09:29:38.505 INFO:tasks.workunit.client.1.smithi138.stdout:6/529: dwrite d3/d11/dc0/d43/d1c/d32/fa1 [0,4194304] 0 2024-05-27T09:29:38.505 INFO:tasks.workunit.client.1.smithi138.stdout:8/366: dwrite d3/d8/f4c [0,4194304] 0 2024-05-27T09:29:38.505 INFO:tasks.workunit.client.1.smithi138.stdout:8/367: chown d3/d8/d32/d34/d38 0 1 2024-05-27T09:29:38.505 INFO:tasks.workunit.client.1.smithi138.stdout:8/368: truncate d3/d8/d32/d3f/f6b 106058 0 2024-05-27T09:29:38.507 INFO:tasks.workunit.client.0.smithi119.stdout:2/912: dwrite d2/da/d8a/dd4/d89/d94/f92 [0,4194304] 0 2024-05-27T09:29:38.507 INFO:tasks.workunit.client.0.smithi119.stdout:6/876: dwrite d0/d2/d57/d37/d74/df1/d109/fa6 [0,4194304] 0 2024-05-27T09:29:38.507 INFO:tasks.workunit.client.1.smithi138.stdout:3/421: symlink d0/d2/da/df/d10/d2a/d61/d73/l8b 0 2024-05-27T09:29:38.507 INFO:tasks.workunit.client.1.smithi138.stdout:5/328: creat d3/d12/d16/f69 x:0 0 0 2024-05-27T09:29:38.508 INFO:tasks.workunit.client.0.smithi119.stdout:8/888: truncate d3/d6e/dcf/fa5 1983506 0 2024-05-27T09:29:38.508 INFO:tasks.workunit.client.1.smithi138.stdout:9/399: sync 2024-05-27T09:29:38.508 INFO:tasks.workunit.client.1.smithi138.stdout:3/422: creat d0/d2/da/df/d10/d2a/d48/f8c x:0 0 0 2024-05-27T09:29:38.509 INFO:tasks.workunit.client.1.smithi138.stdout:8/369: symlink d3/d8/d32/d45/d5c/l6f 0 2024-05-27T09:29:38.510 INFO:tasks.workunit.client.1.smithi138.stdout:8/370: rmdir d3/d8/d32/d34 39 2024-05-27T09:29:38.510 INFO:tasks.workunit.client.1.smithi138.stdout:9/400: creat d6/f7c x:0 0 0 2024-05-27T09:29:38.512 INFO:tasks.workunit.client.0.smithi119.stdout:4/980: dread d4/d23/d31/f33 [0,4194304] 0 2024-05-27T09:29:38.512 INFO:tasks.workunit.client.0.smithi119.stdout:4/981: creat d4/de/d66/f13c x:0 0 0 2024-05-27T09:29:38.512 INFO:tasks.workunit.client.0.smithi119.stdout:4/982: read d4/d6/d28/f7a [3288372,71228] 0 2024-05-27T09:29:38.514 INFO:tasks.workunit.client.1.smithi138.stdout:8/371: symlink d3/d8/d32/l70 0 2024-05-27T09:29:38.514 INFO:tasks.workunit.client.1.smithi138.stdout:8/372: creat d3/d8/d32/d3f/f71 x:0 0 0 2024-05-27T09:29:38.514 INFO:tasks.workunit.client.1.smithi138.stdout:9/401: dread - d6/db/de/d18/d24/f5c zero size 2024-05-27T09:29:38.517 INFO:tasks.workunit.client.1.smithi138.stdout:9/402: truncate d6/f2e 30102 0 2024-05-27T09:29:38.518 INFO:tasks.workunit.client.1.smithi138.stdout:9/403: chown d6/da/c16 160 1 2024-05-27T09:29:38.521 INFO:tasks.workunit.client.0.smithi119.stdout:9/810: truncate f6 2656598 0 2024-05-27T09:29:38.522 INFO:tasks.workunit.client.1.smithi138.stdout:9/404: symlink d6/db/dc/l7d 0 2024-05-27T09:29:38.528 INFO:tasks.workunit.client.0.smithi119.stdout:6/877: creat d0/d2/d57/d37/d74/df1/d109/db2/f138 x:0 0 0 2024-05-27T09:29:38.529 INFO:tasks.workunit.client.0.smithi119.stdout:8/889: mknod d3/d8/d1b/d46/da4/d117/c125 0 2024-05-27T09:29:38.529 INFO:tasks.workunit.client.0.smithi119.stdout:8/890: truncate d3/d8/fe7 795982 0 2024-05-27T09:29:38.529 INFO:tasks.workunit.client.1.smithi138.stdout:6/530: getdents d3/d11/dc0/d7d 0 2024-05-27T09:29:38.533 INFO:tasks.workunit.client.0.smithi119.stdout:9/811: mkdir d18/d19/d1b/d2a/dfe 0 2024-05-27T09:29:38.539 INFO:tasks.workunit.client.0.smithi119.stdout:6/878: creat d0/d11/f139 x:0 0 0 2024-05-27T09:29:38.540 INFO:tasks.workunit.client.1.smithi138.stdout:5/329: dread d3/d12/d16/d1d/d22/f2a [0,4194304] 0 2024-05-27T09:29:38.540 INFO:tasks.workunit.client.1.smithi138.stdout:5/330: chown d3/d12/d26/f4b 4049993 1 2024-05-27T09:29:38.540 INFO:tasks.workunit.client.0.smithi119.stdout:8/891: symlink d3/d8/d4d/d58/l126 0 2024-05-27T09:29:38.543 INFO:tasks.workunit.client.0.smithi119.stdout:9/812: unlink d18/d19/d1b/d40/cc0 0 2024-05-27T09:29:38.546 INFO:tasks.workunit.client.1.smithi138.stdout:5/331: unlink d3/d12/d16/d1d/d22/f5e 0 2024-05-27T09:29:38.549 INFO:tasks.workunit.client.0.smithi119.stdout:1/939: dwrite d7/db/d12/ddb/f55 [4194304,4194304] 0 2024-05-27T09:29:38.549 INFO:tasks.workunit.client.0.smithi119.stdout:8/892: creat d3/d39/d5a/dc9/f127 x:0 0 0 2024-05-27T09:29:38.558 INFO:tasks.workunit.client.0.smithi119.stdout:9/813: creat d18/d19/d1b/d2a/da2/fff x:0 0 0 2024-05-27T09:29:38.561 INFO:tasks.workunit.client.0.smithi119.stdout:7/976: dread d7/df/d19/d97/d92/db7/dba/d42/d64/f7a [0,4194304] 0 2024-05-27T09:29:38.562 INFO:tasks.workunit.client.0.smithi119.stdout:6/879: unlink d0/d2/d4e/d5a/d76/fc6 0 2024-05-27T09:29:38.562 INFO:tasks.workunit.client.0.smithi119.stdout:6/880: dread - d0/d11/f7d zero size 2024-05-27T09:29:38.564 INFO:tasks.workunit.client.0.smithi119.stdout:1/940: mkdir d7/dd/d36/d11f 0 2024-05-27T09:29:38.564 INFO:tasks.workunit.client.0.smithi119.stdout:1/941: truncate d7/db/d52/d65/ff4 1434803 0 2024-05-27T09:29:38.566 INFO:tasks.workunit.client.0.smithi119.stdout:8/893: symlink d3/d6e/d8e/l128 0 2024-05-27T09:29:38.567 INFO:tasks.workunit.client.0.smithi119.stdout:9/814: mknod d18/d19/d1b/d29/c100 0 2024-05-27T09:29:38.567 INFO:tasks.workunit.client.0.smithi119.stdout:7/977: rmdir d7/df/d19/d4d/d89 39 2024-05-27T09:29:38.567 INFO:tasks.workunit.client.0.smithi119.stdout:7/978: dread - d7/df/d19/d97/d92/db7/dba/d42/d64/dad/f104 zero size 2024-05-27T09:29:38.569 INFO:tasks.workunit.client.0.smithi119.stdout:6/881: creat d0/d2/de1/f13a x:0 0 0 2024-05-27T09:29:38.575 INFO:tasks.workunit.client.0.smithi119.stdout:1/942: rename d7/d40/l6a to d7/dd/d44/l120 0 2024-05-27T09:29:38.575 INFO:tasks.workunit.client.0.smithi119.stdout:1/943: readlink d7/l46 0 2024-05-27T09:29:38.575 INFO:tasks.workunit.client.0.smithi119.stdout:1/944: fsync f3 0 2024-05-27T09:29:38.575 INFO:tasks.workunit.client.0.smithi119.stdout:9/815: rmdir d18/d19/d61/d95 39 2024-05-27T09:29:38.575 INFO:tasks.workunit.client.0.smithi119.stdout:8/894: symlink d3/d39/d5a/l129 0 2024-05-27T09:29:38.575 INFO:tasks.workunit.client.0.smithi119.stdout:8/895: fdatasync d3/d8/d1b/d46/da4/d113/d66/f105 0 2024-05-27T09:29:38.576 INFO:tasks.workunit.client.0.smithi119.stdout:8/896: read d3/d8/d4d/d58/f6a [59775,1899] 0 2024-05-27T09:29:38.576 INFO:tasks.workunit.client.0.smithi119.stdout:6/882: mknod d0/d2/d57/d37/dad/c13b 0 2024-05-27T09:29:38.577 INFO:tasks.workunit.client.0.smithi119.stdout:9/816: mknod d18/d19/d1b/d2a/d63/c101 0 2024-05-27T09:29:38.578 INFO:tasks.workunit.client.0.smithi119.stdout:8/897: rmdir d3/d8/d1b/d46/da4/d117 39 2024-05-27T09:29:38.578 INFO:tasks.workunit.client.0.smithi119.stdout:8/898: creat d3/d6e/d80/d84/f12a x:0 0 0 2024-05-27T09:29:38.582 INFO:tasks.workunit.client.0.smithi119.stdout:6/883: mknod d0/d2/d1c/d21/d58/c13c 0 2024-05-27T09:29:38.585 INFO:tasks.workunit.client.0.smithi119.stdout:9/817: rmdir d18/d19/d61/d95/de4 0 2024-05-27T09:29:38.588 INFO:tasks.workunit.client.0.smithi119.stdout:6/884: creat d0/d2/d1c/f13d x:0 0 0 2024-05-27T09:29:38.597 INFO:tasks.workunit.client.1.smithi138.stdout:4/479: dwrite d7/df/d35/d4c/f69 [0,4194304] 0 2024-05-27T09:29:38.599 INFO:tasks.workunit.client.1.smithi138.stdout:6/531: dread d3/d11/dc0/d43/d1c/f4c [0,4194304] 0 2024-05-27T09:29:38.600 INFO:tasks.workunit.client.1.smithi138.stdout:6/532: dread - d3/d11/dc0/d43/d8c/f96 zero size 2024-05-27T09:29:38.600 INFO:tasks.workunit.client.1.smithi138.stdout:6/533: write d3/d11/dc0/d43/d1c/d32/d47/f7a [5209952,58392] 0 2024-05-27T09:29:38.600 INFO:tasks.workunit.client.1.smithi138.stdout:6/534: write d3/d91/fa0 [890155,62907] 0 2024-05-27T09:29:38.603 INFO:tasks.workunit.client.0.smithi119.stdout:7/979: dread d7/df/d19/f1f [0,4194304] 0 2024-05-27T09:29:38.603 INFO:tasks.workunit.client.0.smithi119.stdout:1/945: write d7/db/d12/ddb/f55 [5948261,49818] 0 2024-05-27T09:29:38.603 INFO:tasks.workunit.client.0.smithi119.stdout:1/946: fsync d7/db/d8f/d9b/dce/f10c 0 2024-05-27T09:29:38.604 INFO:tasks.workunit.client.0.smithi119.stdout:7/980: creat d7/df/d19/d97/d81/f137 x:0 0 0 2024-05-27T09:29:38.605 INFO:tasks.workunit.client.0.smithi119.stdout:1/947: truncate d7/db/f7d 1014710 0 2024-05-27T09:29:38.607 INFO:tasks.workunit.client.0.smithi119.stdout:9/818: dread d18/d19/d1b/d2a/d63/f97 [0,4194304] 0 2024-05-27T09:29:38.608 INFO:tasks.workunit.client.1.smithi138.stdout:3/423: dwrite d0/d2/da/f6b [0,4194304] 0 2024-05-27T09:29:38.609 INFO:tasks.workunit.client.0.smithi119.stdout:2/913: dwrite d2/d1a/f46 [4194304,4194304] 0 2024-05-27T09:29:38.609 INFO:tasks.workunit.client.0.smithi119.stdout:2/914: fsync d2/da/d8a/dd4/d89/d94/d81/fc9 0 2024-05-27T09:29:38.609 INFO:tasks.workunit.client.0.smithi119.stdout:7/981: mknod d7/df/d19/d46/db5/c138 0 2024-05-27T09:29:38.609 INFO:tasks.workunit.client.0.smithi119.stdout:7/982: stat d7/d6a/d7b/d11c 0 2024-05-27T09:29:38.610 INFO:tasks.workunit.client.0.smithi119.stdout:6/885: dread d0/d2/d4e/dc5/fee [0,4194304] 0 2024-05-27T09:29:38.611 INFO:tasks.workunit.client.1.smithi138.stdout:3/424: unlink d0/d3f/d60/c6e 0 2024-05-27T09:29:38.611 INFO:tasks.workunit.client.1.smithi138.stdout:3/425: truncate d0/d3f/d50/f7c 919811 0 2024-05-27T09:29:38.612 INFO:tasks.workunit.client.1.smithi138.stdout:9/405: dwrite d6/f48 [0,4194304] 0 2024-05-27T09:29:38.612 INFO:tasks.workunit.client.1.smithi138.stdout:9/406: chown d6/db/de/d18/d24/c5b 1227649128 1 2024-05-27T09:29:38.612 INFO:tasks.workunit.client.1.smithi138.stdout:3/426: mknod d0/d2/da/df/d10/d32/d37/c8d 0 2024-05-27T09:29:38.613 INFO:tasks.workunit.client.0.smithi119.stdout:8/899: dread d3/d6e/d8e/f9f [0,4194304] 0 2024-05-27T09:29:38.613 INFO:tasks.workunit.client.0.smithi119.stdout:9/819: rename d18/d19/d1b/d40/db7/fbb to d18/db4/f102 0 2024-05-27T09:29:38.613 INFO:tasks.workunit.client.0.smithi119.stdout:9/820: chown d18/d19/d1b/d29/fa7 0 1 2024-05-27T09:29:38.613 INFO:tasks.workunit.client.0.smithi119.stdout:8/900: write d3/d8/d1b/d46/da4/d113/f107 [618987,40497] 0 2024-05-27T09:29:38.614 INFO:tasks.workunit.client.0.smithi119.stdout:7/983: creat d7/df/d19/d8d/f139 x:0 0 0 2024-05-27T09:29:38.614 INFO:tasks.workunit.client.0.smithi119.stdout:7/984: dread - d7/d1a/d99/d112/f11a zero size 2024-05-27T09:29:38.617 INFO:tasks.workunit.client.1.smithi138.stdout:3/427: mknod d0/d80/c8e 0 2024-05-27T09:29:38.622 INFO:tasks.workunit.client.1.smithi138.stdout:8/373: dwrite d3/d8/d32/d3f/f71 [0,4194304] 0 2024-05-27T09:29:38.625 INFO:tasks.workunit.client.0.smithi119.stdout:6/886: unlink d0/d2/d4e/fb9 0 2024-05-27T09:29:38.625 INFO:tasks.workunit.client.0.smithi119.stdout:2/915: rename d2/da/d8a/dd4/d69/df5/d108/l10c to d2/da/d8a/d8f/l13b 0 2024-05-27T09:29:38.626 INFO:tasks.workunit.client.0.smithi119.stdout:2/916: write d2/da/d4e/ff2 [180191,30424] 0 2024-05-27T09:29:38.626 INFO:tasks.workunit.client.0.smithi119.stdout:2/917: fsync d2/da/d4e/f4f 0 2024-05-27T09:29:38.626 INFO:tasks.workunit.client.0.smithi119.stdout:2/918: stat d2/da/d8a/dd4/d89/fa3 0 2024-05-27T09:29:38.629 INFO:tasks.workunit.client.0.smithi119.stdout:8/901: creat d3/d8/d4d/d79/f12b x:0 0 0 2024-05-27T09:29:38.629 INFO:tasks.workunit.client.0.smithi119.stdout:8/902: dread - d3/d39/d5a/f116 zero size 2024-05-27T09:29:38.629 INFO:tasks.workunit.client.0.smithi119.stdout:8/903: write d3/d8/d1b/db1/ff0 [1385822,34023] 0 2024-05-27T09:29:38.629 INFO:tasks.workunit.client.0.smithi119.stdout:7/985: mknod d7/df/d19/d97/d92/db7/dba/d42/d117/c13a 0 2024-05-27T09:29:38.630 INFO:tasks.workunit.client.1.smithi138.stdout:9/407: dread f1 [0,4194304] 0 2024-05-27T09:29:38.630 INFO:tasks.workunit.client.1.smithi138.stdout:9/408: chown d6/db/de/d18 11778 1 2024-05-27T09:29:38.635 INFO:tasks.workunit.client.0.smithi119.stdout:6/887: unlink d0/d2/d57/d37/f8d 0 2024-05-27T09:29:38.635 INFO:tasks.workunit.client.0.smithi119.stdout:6/888: fdatasync d0/d2/d4e/dc5/f113 0 2024-05-27T09:29:38.635 INFO:tasks.workunit.client.1.smithi138.stdout:9/409: symlink d6/da/d2f/d35/d55/l7e 0 2024-05-27T09:29:38.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:38 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/2099298037' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:29:38.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:38 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:38.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:38 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:38.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:38 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:38.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:38 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:38.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:38 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:38.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:38 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:38.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:38 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:38.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:38 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:38.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:38 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:38.637 INFO:tasks.workunit.client.1.smithi138.stdout:9/410: link d6/da/d2f/f40 d6/db/de/d18/d24/f7f 0 2024-05-27T09:29:38.639 INFO:tasks.workunit.client.0.smithi119.stdout:8/904: mkdir d3/de/d12c 0 2024-05-27T09:29:38.639 INFO:tasks.workunit.client.0.smithi119.stdout:8/905: creat d3/d8/d1b/f12d x:0 0 0 2024-05-27T09:29:38.643 INFO:tasks.workunit.client.1.smithi138.stdout:4/480: dread d7/df/d35/d4c/d59/fa5 [0,4194304] 0 2024-05-27T09:29:38.643 INFO:tasks.workunit.client.1.smithi138.stdout:5/332: dwrite d3/f21 [0,4194304] 0 2024-05-27T09:29:38.643 INFO:tasks.workunit.client.1.smithi138.stdout:5/333: read d3/fd [2837516,103998] 0 2024-05-27T09:29:38.643 INFO:tasks.workunit.client.1.smithi138.stdout:5/334: chown d3/d12/c1f 1 1 2024-05-27T09:29:38.643 INFO:tasks.workunit.client.1.smithi138.stdout:5/335: chown d3/d12/d26 112 1 2024-05-27T09:29:38.643 INFO:tasks.workunit.client.1.smithi138.stdout:5/336: fdatasync d3/d12/d16/d1d/d22/d2b/d36/f41 0 2024-05-27T09:29:38.643 INFO:tasks.workunit.client.1.smithi138.stdout:6/535: dwrite d3/d11/d2d/d37/f4a [0,4194304] 0 2024-05-27T09:29:38.643 INFO:tasks.workunit.client.1.smithi138.stdout:6/536: truncate d3/d11/dc0/d43/d1c/fbc 202160 0 2024-05-27T09:29:38.643 INFO:tasks.workunit.client.1.smithi138.stdout:6/537: chown d3/f84 650160 1 2024-05-27T09:29:38.646 INFO:tasks.workunit.client.0.smithi119.stdout:7/986: symlink d7/df/d19/d46/db5/l13b 0 2024-05-27T09:29:38.650 INFO:tasks.workunit.client.0.smithi119.stdout:6/889: truncate d0/d2/d1c/f2d 8910683 0 2024-05-27T09:29:38.660 INFO:tasks.workunit.client.0.smithi119.stdout:2/919: unlink d2/l24 0 2024-05-27T09:29:38.660 INFO:tasks.workunit.client.0.smithi119.stdout:2/920: chown d2/da/d8a/dd4/d89/d94/d87/ff3 94 1 2024-05-27T09:29:38.660 INFO:tasks.workunit.client.0.smithi119.stdout:2/921: readlink d2/da/d8a/dd4/d89/d94/lbf 0 2024-05-27T09:29:38.660 INFO:tasks.workunit.client.0.smithi119.stdout:1/948: dwrite d7/db/d12/feb [0,4194304] 0 2024-05-27T09:29:38.661 INFO:tasks.workunit.client.0.smithi119.stdout:1/949: readlink d7/db/d35/lfa 0 2024-05-27T09:29:38.661 INFO:tasks.workunit.client.0.smithi119.stdout:1/950: write d7/db6/dc2/f113 [485081,14355] 0 2024-05-27T09:29:38.661 INFO:tasks.workunit.client.0.smithi119.stdout:1/951: dread - d7/d6e/fe4 zero size 2024-05-27T09:29:38.661 INFO:tasks.workunit.client.0.smithi119.stdout:1/952: truncate d7/f3d 8732820 0 2024-05-27T09:29:38.661 INFO:tasks.workunit.client.0.smithi119.stdout:6/890: dread d0/d85/f99 [4194304,4194304] 0 2024-05-27T09:29:38.663 INFO:tasks.workunit.client.0.smithi119.stdout:8/906: creat d3/d8/d1b/d46/da4/d113/db7/de0/f12e x:0 0 0 2024-05-27T09:29:38.663 INFO:tasks.workunit.client.0.smithi119.stdout:8/907: dread - d3/d8/d1b/d46/da4/dbb/f11b zero size 2024-05-27T09:29:38.663 INFO:tasks.workunit.client.0.smithi119.stdout:8/908: dread - d3/d8/d1b/f12d zero size 2024-05-27T09:29:38.663 INFO:tasks.workunit.client.0.smithi119.stdout:8/909: chown d3/d8/d4d/d58/f63 22901661 1 2024-05-27T09:29:38.672 INFO:tasks.workunit.client.1.smithi138.stdout:4/481: dread d7/f26 [0,4194304] 0 2024-05-27T09:29:38.679 INFO:tasks.workunit.client.0.smithi119.stdout:9/821: dwrite d18/d19/fb1 [0,4194304] 0 2024-05-27T09:29:38.679 INFO:tasks.workunit.client.0.smithi119.stdout:9/822: chown d18/d19/d1b/d40/d44/dcc/fcf 796 1 2024-05-27T09:29:38.679 INFO:tasks.workunit.client.1.smithi138.stdout:3/428: dwrite d0/d2/da/df/d10/d32/f9 [4194304,4194304] 0 2024-05-27T09:29:38.680 INFO:tasks.workunit.client.0.smithi119.stdout:9/823: write d18/d19/d1b/d29/dc4/de1/ffc [755989,47922] 0 2024-05-27T09:29:38.681 INFO:tasks.workunit.client.1.smithi138.stdout:5/337: dread d3/d12/d16/f39 [0,4194304] 0 2024-05-27T09:29:38.683 INFO:tasks.workunit.client.1.smithi138.stdout:3/429: getdents d0/d3f 0 2024-05-27T09:29:38.683 INFO:tasks.workunit.client.1.smithi138.stdout:4/482: dread d7/df/d35/f6c [0,4194304] 0 2024-05-27T09:29:38.683 INFO:tasks.workunit.client.0.smithi119.stdout:2/922: dread d2/da/d8a/dd4/d89/d94/d81/f9e [0,4194304] 0 2024-05-27T09:29:38.684 INFO:tasks.workunit.client.1.smithi138.stdout:3/430: creat d0/d2/da/df/d10/d2a/f8f x:0 0 0 2024-05-27T09:29:38.684 INFO:tasks.workunit.client.1.smithi138.stdout:4/483: write d7/df/d35/d4c/d59/d5d/d64/f96 [478884,45362] 0 2024-05-27T09:29:38.684 INFO:tasks.workunit.client.1.smithi138.stdout:4/484: fsync d7/df/d35/fa2 0 2024-05-27T09:29:38.688 INFO:tasks.workunit.client.0.smithi119.stdout:1/953: symlink d7/dd/d25/d60/d61/da5/l121 0 2024-05-27T09:29:38.688 INFO:tasks.workunit.client.1.smithi138.stdout:3/431: mkdir d0/d2/da/df/d10/d2a/d48/d90 0 2024-05-27T09:29:38.695 INFO:tasks.workunit.client.1.smithi138.stdout:3/432: mknod d0/d2/da/df/d10/d2a/d48/d90/c91 0 2024-05-27T09:29:38.695 INFO:tasks.workunit.client.1.smithi138.stdout:4/485: creat d7/df/d35/d4c/d59/d5d/d64/fa7 x:0 0 0 2024-05-27T09:29:38.695 INFO:tasks.workunit.client.0.smithi119.stdout:1/954: dread d7/db/d12/ddb/d38/d49/fd9 [0,4194304] 0 2024-05-27T09:29:38.698 INFO:tasks.workunit.client.0.smithi119.stdout:8/910: unlink d3/de/c51 0 2024-05-27T09:29:38.700 INFO:tasks.workunit.client.1.smithi138.stdout:5/338: dread d3/d12/d16/d1d/d22/d2b/d36/f41 [0,4194304] 0 2024-05-27T09:29:38.702 INFO:tasks.workunit.client.0.smithi119.stdout:6/891: link d0/d2/d4e/d5a/lf4 d0/l13e 0 2024-05-27T09:29:38.705 INFO:tasks.workunit.client.1.smithi138.stdout:4/486: rmdir d7/df/d35/da6 0 2024-05-27T09:29:38.709 INFO:tasks.workunit.client.0.smithi119.stdout:1/955: mkdir d7/dd/d44/d122 0 2024-05-27T09:29:38.710 INFO:tasks.workunit.client.1.smithi138.stdout:9/411: dread d6/db/dc/f50 [0,4194304] 0 2024-05-27T09:29:38.710 INFO:tasks.workunit.client.0.smithi119.stdout:8/911: creat d3/d8/d1b/d46/d92/f12f x:0 0 0 2024-05-27T09:29:38.711 INFO:tasks.workunit.client.0.smithi119.stdout:7/987: dwrite d7/df/d19/d97/fbf [0,4194304] 0 2024-05-27T09:29:38.714 INFO:tasks.workunit.client.1.smithi138.stdout:4/487: symlink d7/df/d35/d4c/d59/d5d/d64/d6a/la8 0 2024-05-27T09:29:38.715 INFO:tasks.workunit.client.1.smithi138.stdout:6/538: dread d3/d11/dc0/d5a/d6a/f6f [0,4194304] 0 2024-05-27T09:29:38.716 INFO:tasks.workunit.client.1.smithi138.stdout:8/374: dwrite d3/d8/d24/d55/f67 [0,4194304] 0 2024-05-27T09:29:38.716 INFO:tasks.workunit.client.1.smithi138.stdout:8/375: readlink d3/d8/d32/d34/d38/l43 0 2024-05-27T09:29:38.716 INFO:tasks.workunit.client.1.smithi138.stdout:8/376: stat d3/d8/d32/f49 0 2024-05-27T09:29:38.716 INFO:tasks.workunit.client.1.smithi138.stdout:8/377: dread - d3/d8/f42 zero size 2024-05-27T09:29:38.717 INFO:tasks.workunit.client.1.smithi138.stdout:9/412: dread d6/da/d2f/d35/d55/f3a [0,4194304] 0 2024-05-27T09:29:38.719 INFO:tasks.workunit.client.0.smithi119.stdout:1/956: creat d7/dd/d25/d60/f123 x:0 0 0 2024-05-27T09:29:38.719 INFO:tasks.workunit.client.0.smithi119.stdout:8/912: dread d3/d8/d1b/d46/da4/d113/fa3 [0,4194304] 0 2024-05-27T09:29:38.721 INFO:tasks.workunit.client.1.smithi138.stdout:4/488: mknod d7/df/d35/d4c/d59/d5d/d64/d6a/ca9 0 2024-05-27T09:29:38.721 INFO:tasks.workunit.client.1.smithi138.stdout:4/489: fsync d7/f3d 0 2024-05-27T09:29:38.721 INFO:tasks.workunit.client.1.smithi138.stdout:4/490: write d7/df/d35/f8b [3115,75619] 0 2024-05-27T09:29:38.722 INFO:tasks.workunit.client.1.smithi138.stdout:6/539: mkdir d3/d11/dc0/d43/dbe/dc3 0 2024-05-27T09:29:38.722 INFO:tasks.workunit.client.1.smithi138.stdout:6/540: write d3/d11/dc0/f60 [1642454,27796] 0 2024-05-27T09:29:38.724 INFO:tasks.workunit.client.0.smithi119.stdout:1/957: creat d7/dd/d44/dff/f124 x:0 0 0 2024-05-27T09:29:38.727 INFO:tasks.workunit.client.1.smithi138.stdout:9/413: stat d6/da/d26/c5e 0 2024-05-27T09:29:38.730 INFO:tasks.workunit.client.1.smithi138.stdout:9/414: unlink d6/da/d26/f4f 0 2024-05-27T09:29:38.730 INFO:tasks.workunit.client.1.smithi138.stdout:9/415: write d6/f3f [1166656,76008] 0 2024-05-27T09:29:38.730 INFO:tasks.workunit.client.0.smithi119.stdout:8/913: mknod d3/d8/d1b/d46/da4/c130 0 2024-05-27T09:29:38.730 INFO:tasks.workunit.client.0.smithi119.stdout:8/914: fdatasync d3/d6e/dcf/fc2 0 2024-05-27T09:29:38.731 INFO:tasks.workunit.client.1.smithi138.stdout:4/491: mkdir d7/d27/daa 0 2024-05-27T09:29:38.733 INFO:tasks.workunit.client.0.smithi119.stdout:7/988: mkdir d7/df/d13c 0 2024-05-27T09:29:38.733 INFO:tasks.workunit.client.0.smithi119.stdout:8/915: rmdir d3/d8/d1b/d46/da4/d113/d74 39 2024-05-27T09:29:38.734 INFO:tasks.workunit.client.0.smithi119.stdout:4/983: sync 2024-05-27T09:29:38.734 INFO:tasks.workunit.client.0.smithi119.stdout:0/900: sync 2024-05-27T09:29:38.738 INFO:tasks.workunit.client.0.smithi119.stdout:0/901: write d2/f94 [941611,10466] 0 2024-05-27T09:29:38.738 INFO:tasks.workunit.client.0.smithi119.stdout:0/902: write d2/d110/d59/dd7/f70 [4492514,39760] 0 2024-05-27T09:29:38.741 INFO:tasks.workunit.client.0.smithi119.stdout:4/984: symlink d4/d6/l13d 0 2024-05-27T09:29:38.743 INFO:tasks.workunit.client.0.smithi119.stdout:4/985: mkdir d4/d6/d28/d13e 0 2024-05-27T09:29:38.759 INFO:tasks.workunit.client.0.smithi119.stdout:4/986: write d4/d23/d31/f33 [1840447,16907] 0 2024-05-27T09:29:38.765 INFO:tasks.workunit.client.1.smithi138.stdout:8/378: write d3/d8/d32/f59 [3394053,62998] 0 2024-05-27T09:29:38.765 INFO:tasks.workunit.client.1.smithi138.stdout:8/379: fdatasync d3/d8/d64/f2e 0 2024-05-27T09:29:38.767 INFO:tasks.workunit.client.1.smithi138.stdout:5/339: dwrite d3/d12/d16/d1d/d30/f4a [0,4194304] 0 2024-05-27T09:29:38.768 INFO:tasks.workunit.client.0.smithi119.stdout:8/916: dread d3/d8/d1b/d77/d95/fab [0,4194304] 0 2024-05-27T09:29:38.768 INFO:tasks.workunit.client.0.smithi119.stdout:8/917: readlink d3/d8/d1b/d46/ld1 0 2024-05-27T09:29:38.769 INFO:tasks.workunit.client.0.smithi119.stdout:7/989: dread d7/df/d19/d78/ff9 [0,4194304] 0 2024-05-27T09:29:38.770 INFO:tasks.workunit.client.1.smithi138.stdout:8/380: unlink d3/d8/d32/d3f/f71 0 2024-05-27T09:29:38.770 INFO:tasks.workunit.client.1.smithi138.stdout:8/381: chown d3/d8/c65 2905719 1 2024-05-27T09:29:38.774 INFO:tasks.workunit.client.0.smithi119.stdout:9/824: dwrite d18/d19/d1b/d40/d7b/ff1 [0,4194304] 0 2024-05-27T09:29:38.774 INFO:tasks.workunit.client.0.smithi119.stdout:9/825: chown d18/d19/d1b/f64 41481216 1 2024-05-27T09:29:38.784 INFO:tasks.workunit.client.0.smithi119.stdout:1/958: dread d7/d6e/fe2 [0,4194304] 0 2024-05-27T09:29:38.784 INFO:tasks.workunit.client.0.smithi119.stdout:1/959: truncate d7/dd/db0/dcd/fda 595202 0 2024-05-27T09:29:38.788 INFO:tasks.workunit.client.1.smithi138.stdout:8/382: dread d3/d8/d32/d34/d38/f63 [0,4194304] 0 2024-05-27T09:29:38.789 INFO:tasks.workunit.client.0.smithi119.stdout:1/960: mkdir d7/d125 0 2024-05-27T09:29:38.789 INFO:tasks.workunit.client.0.smithi119.stdout:1/961: write d7/fef [163525,97551] 0 2024-05-27T09:29:38.789 INFO:tasks.workunit.client.0.smithi119.stdout:1/962: chown d7/db/d12/d26/d4a/fd6 33962715 1 2024-05-27T09:29:38.789 INFO:tasks.workunit.client.0.smithi119.stdout:1/963: write d7/dd/d44/f88 [1267739,99726] 0 2024-05-27T09:29:38.792 INFO:tasks.workunit.client.0.smithi119.stdout:1/964: symlink d7/db/d12/d26/d39/l126 0 2024-05-27T09:29:38.794 INFO:tasks.workunit.client.0.smithi119.stdout:1/965: link d7/db/d12/ddb/d38/d3c/c6f d7/d125/c127 0 2024-05-27T09:29:38.797 INFO:tasks.workunit.client.0.smithi119.stdout:1/966: getdents d7/db/d52 0 2024-05-27T09:29:38.798 INFO:tasks.workunit.client.0.smithi119.stdout:1/967: symlink d7/db/d12/d15/dd2/l128 0 2024-05-27T09:29:38.800 INFO:tasks.workunit.client.1.smithi138.stdout:9/416: dread d6/da/d29/f6d [0,4194304] 0 2024-05-27T09:29:38.802 INFO:tasks.workunit.client.1.smithi138.stdout:9/417: unlink d6/da/d26/f37 0 2024-05-27T09:29:38.804 INFO:tasks.workunit.client.0.smithi119.stdout:7/990: dread d7/df/d19/d9a/da6/dda/d44/f134 [4194304,4194304] 0 2024-05-27T09:29:38.805 INFO:tasks.workunit.client.1.smithi138.stdout:9/418: creat d6/da/d2f/d35/f80 x:0 0 0 2024-05-27T09:29:38.809 INFO:tasks.workunit.client.1.smithi138.stdout:9/419: rename d6/da/d26/c5e to d6/da/d29/d61/c81 0 2024-05-27T09:29:38.811 INFO:tasks.workunit.client.1.smithi138.stdout:9/420: mknod d6/da/d26/d28/c82 0 2024-05-27T09:29:38.813 INFO:tasks.workunit.client.1.smithi138.stdout:8/383: dread d3/d8/d32/f49 [0,4194304] 0 2024-05-27T09:29:38.813 INFO:tasks.workunit.client.1.smithi138.stdout:3/433: dwrite d0/d2/da/df/d10/d2a/f4f [0,4194304] 0 2024-05-27T09:29:38.814 INFO:tasks.workunit.client.0.smithi119.stdout:2/923: dwrite d2/da/d2c/fe1 [0,4194304] 0 2024-05-27T09:29:38.821 INFO:tasks.workunit.client.0.smithi119.stdout:4/987: write d4/de/d21/d40/d47/d5c/d9e/fcb [3592501,24362] 0 2024-05-27T09:29:38.822 INFO:tasks.workunit.client.1.smithi138.stdout:9/421: mknod d6/da/d2f/c83 0 2024-05-27T09:29:38.823 INFO:tasks.workunit.client.1.smithi138.stdout:9/422: fdatasync d6/db/f10 0 2024-05-27T09:29:38.823 INFO:tasks.workunit.client.1.smithi138.stdout:9/423: truncate f3 4708117 0 2024-05-27T09:29:38.823 INFO:tasks.workunit.client.1.smithi138.stdout:8/384: rmdir d3/d8/d32/d45 39 2024-05-27T09:29:38.823 INFO:tasks.workunit.client.1.smithi138.stdout:8/385: readlink d3/d8/l2d 0 2024-05-27T09:29:38.823 INFO:tasks.workunit.client.1.smithi138.stdout:7/410: sync 2024-05-27T09:29:38.824 INFO:tasks.workunit.client.1.smithi138.stdout:2/342: sync 2024-05-27T09:29:38.824 INFO:tasks.workunit.client.1.smithi138.stdout:2/343: dread - d1/d15/d2d/d1d/f67 zero size 2024-05-27T09:29:38.824 INFO:tasks.workunit.client.0.smithi119.stdout:0/903: dwrite d2/d6/d9/d12/f77 [0,4194304] 0 2024-05-27T09:29:38.824 INFO:tasks.workunit.client.0.smithi119.stdout:0/904: fsync d2/d1a/d20/f74 0 2024-05-27T09:29:38.825 INFO:tasks.workunit.client.1.smithi138.stdout:0/401: sync 2024-05-27T09:29:38.826 INFO:tasks.workunit.client.1.smithi138.stdout:1/417: sync 2024-05-27T09:29:38.827 INFO:tasks.workunit.client.1.smithi138.stdout:1/418: dread - d4/d1a/d2e/f79 zero size 2024-05-27T09:29:38.827 INFO:tasks.workunit.client.1.smithi138.stdout:3/434: creat d0/d80/f92 x:0 0 0 2024-05-27T09:29:38.827 INFO:tasks.workunit.client.1.smithi138.stdout:3/435: truncate d0/d2/f45 1953380 0 2024-05-27T09:29:38.827 INFO:tasks.workunit.client.0.smithi119.stdout:4/988: rmdir d4/d6/d3e/dd2 39 2024-05-27T09:29:38.828 INFO:tasks.workunit.client.0.smithi119.stdout:4/989: stat d4/db1/ff4 0 2024-05-27T09:29:38.830 INFO:tasks.workunit.client.0.smithi119.stdout:0/905: rename d2/d6/d9/d12/l1d to d2/d6/d9/d24/d30/d4a/l12f 0 2024-05-27T09:29:38.830 INFO:tasks.workunit.client.0.smithi119.stdout:0/906: write d2/d1a/d20/f8b [2365869,26485] 0 2024-05-27T09:29:38.832 INFO:tasks.workunit.client.1.smithi138.stdout:9/424: rename d6/db/dc/l1d to d6/da/d2f/d35/l84 0 2024-05-27T09:29:38.832 INFO:tasks.workunit.client.1.smithi138.stdout:9/425: dread - d6/f7c zero size 2024-05-27T09:29:38.832 INFO:tasks.workunit.client.1.smithi138.stdout:9/426: chown d6/da/d29 0 1 2024-05-27T09:29:38.836 INFO:tasks.workunit.client.0.smithi119.stdout:4/990: link d4/f7e d4/d23/d24/d121/f13f 0 2024-05-27T09:29:38.840 INFO:tasks.workunit.client.0.smithi119.stdout:2/924: dread d2/f90 [0,4194304] 0 2024-05-27T09:29:38.840 INFO:tasks.workunit.client.0.smithi119.stdout:6/892: dwrite d0/d2/d57/d37/f63 [4194304,4194304] 0 2024-05-27T09:29:38.841 INFO:tasks.workunit.client.0.smithi119.stdout:2/925: read - d2/da/d15/d53/dac/f10d zero size 2024-05-27T09:29:38.842 INFO:tasks.workunit.client.1.smithi138.stdout:0/402: mkdir da/d17/d1d/d1f/d42/d4a/d53/d86 0 2024-05-27T09:29:38.842 INFO:tasks.workunit.client.1.smithi138.stdout:0/403: truncate da/d17/d1d/d1f/d4f/f6f 697109 0 2024-05-27T09:29:38.842 INFO:tasks.workunit.client.1.smithi138.stdout:7/411: write d0/d2b/f35 [282528,78741] 0 2024-05-27T09:29:38.842 INFO:tasks.workunit.client.1.smithi138.stdout:7/412: fdatasync d0/f10 0 2024-05-27T09:29:38.842 INFO:tasks.workunit.client.1.smithi138.stdout:8/386: mkdir d3/d8/d72 0 2024-05-27T09:29:38.842 INFO:tasks.workunit.client.1.smithi138.stdout:9/427: unlink d6/db/l21 0 2024-05-27T09:29:38.842 INFO:tasks.workunit.client.1.smithi138.stdout:8/387: write d3/d8/d32/d45/f4e [5073535,89230] 0 2024-05-27T09:29:38.842 INFO:tasks.workunit.client.1.smithi138.stdout:9/428: chown d6/da/d2f/d35/d55/l34 11 1 2024-05-27T09:29:38.843 INFO:tasks.workunit.client.1.smithi138.stdout:4/492: dwrite d7/f26 [0,4194304] 0 2024-05-27T09:29:38.843 INFO:tasks.workunit.client.1.smithi138.stdout:1/419: unlink d4/c7f 0 2024-05-27T09:29:38.843 INFO:tasks.workunit.client.1.smithi138.stdout:1/420: chown d4/d48 8099102 1 2024-05-27T09:29:38.843 INFO:tasks.workunit.client.1.smithi138.stdout:1/421: chown d4/d1a/d2e/d4a/d7e 19 1 2024-05-27T09:29:38.844 INFO:tasks.workunit.client.0.smithi119.stdout:0/907: getdents d2/d6/d9/d24/d30/dcc/de6 0 2024-05-27T09:29:38.844 INFO:tasks.workunit.client.0.smithi119.stdout:4/991: mknod d4/d6/d81/d10b/c140 0 2024-05-27T09:29:38.844 INFO:tasks.workunit.client.1.smithi138.stdout:9/429: mkdir d6/db/de/d85 0 2024-05-27T09:29:38.844 INFO:tasks.workunit.client.1.smithi138.stdout:5/340: write d3/d12/d26/d53/f58 [2687326,104424] 0 2024-05-27T09:29:38.844 INFO:tasks.workunit.client.1.smithi138.stdout:6/541: dwrite d3/f17 [0,4194304] 0 2024-05-27T09:29:38.845 INFO:tasks.workunit.client.1.smithi138.stdout:9/430: mknod d6/db/de/d18/d27/c86 0 2024-05-27T09:29:38.847 INFO:tasks.workunit.client.1.smithi138.stdout:9/431: mknod d6/db/dc/c87 0 2024-05-27T09:29:38.848 INFO:tasks.workunit.client.1.smithi138.stdout:9/432: fdatasync d6/db/de/d18/d24/f65 0 2024-05-27T09:29:38.848 INFO:tasks.workunit.client.1.smithi138.stdout:9/433: creat d6/f88 x:0 0 0 2024-05-27T09:29:38.851 INFO:tasks.workunit.client.0.smithi119.stdout:2/926: creat d2/da/d8a/dec/f13c x:0 0 0 2024-05-27T09:29:38.853 INFO:tasks.workunit.client.0.smithi119.stdout:6/893: mkdir d0/d2/d57/d37/d74/d13f 0 2024-05-27T09:29:38.856 INFO:tasks.workunit.client.1.smithi138.stdout:5/341: creat d3/d12/d16/d1d/d23/d68/f6a x:0 0 0 2024-05-27T09:29:38.856 INFO:tasks.workunit.client.1.smithi138.stdout:2/344: dread d1/d15/f4f [4194304,4194304] 0 2024-05-27T09:29:38.857 INFO:tasks.workunit.client.0.smithi119.stdout:0/908: mkdir d2/d6/d9/d12/da3/d130 0 2024-05-27T09:29:38.857 INFO:tasks.workunit.client.1.smithi138.stdout:6/542: symlink d3/d11/dc0/d7d/da5/lc4 0 2024-05-27T09:29:38.860 INFO:tasks.workunit.client.0.smithi119.stdout:2/927: stat d2/da/d15/c54 0 2024-05-27T09:29:38.862 INFO:tasks.workunit.client.1.smithi138.stdout:2/345: unlink d1/d15/d2d/d17/c47 0 2024-05-27T09:29:38.862 INFO:tasks.workunit.client.1.smithi138.stdout:2/346: chown d1/d15/d2d/d31/d4b 527 1 2024-05-27T09:29:38.863 INFO:tasks.workunit.client.0.smithi119.stdout:8/918: dwrite d3/d8/d4d/f36 [0,4194304] 0 2024-05-27T09:29:38.863 INFO:tasks.workunit.client.0.smithi119.stdout:6/894: mknod d0/d2/d57/d37/d74/df1/d109/d118/c140 0 2024-05-27T09:29:38.864 INFO:tasks.workunit.client.1.smithi138.stdout:6/543: creat d3/d11/dc0/d7d/fc5 x:0 0 0 2024-05-27T09:29:38.864 INFO:tasks.workunit.client.1.smithi138.stdout:6/544: chown d3/d11/d2d/d37/d7e/d8d/da8 545185294 1 2024-05-27T09:29:38.864 INFO:tasks.workunit.client.1.smithi138.stdout:2/347: getdents d1/d15/d2d/d17/d25/d4d 0 2024-05-27T09:29:38.866 INFO:tasks.workunit.client.0.smithi119.stdout:0/909: symlink d2/d110/d59/dd7/d105/l131 0 2024-05-27T09:29:38.867 INFO:tasks.workunit.client.0.smithi119.stdout:4/992: unlink d4/d23/d31/l4d 0 2024-05-27T09:29:38.867 INFO:tasks.workunit.client.0.smithi119.stdout:4/993: dread - d4/d6/d3e/d89/dcf/f119 zero size 2024-05-27T09:29:38.867 INFO:tasks.workunit.client.0.smithi119.stdout:4/994: write d4/d6/fd1 [4993725,15520] 0 2024-05-27T09:29:38.869 INFO:tasks.workunit.client.0.smithi119.stdout:2/928: unlink d2/da/d6f/f119 0 2024-05-27T09:29:38.873 INFO:tasks.workunit.client.1.smithi138.stdout:6/545: symlink d3/d11/dc0/d43/d8c/lc6 0 2024-05-27T09:29:38.875 INFO:tasks.workunit.client.0.smithi119.stdout:9/826: dwrite d18/d19/d1b/d29/f7c [4194304,4194304] 0 2024-05-27T09:29:38.881 INFO:tasks.workunit.client.0.smithi119.stdout:8/919: mkdir d3/d8/d1b/d46/da4/d113/db7/d131 0 2024-05-27T09:29:38.883 INFO:tasks.workunit.client.1.smithi138.stdout:4/493: dread d7/fb [0,4194304] 0 2024-05-27T09:29:38.883 INFO:tasks.workunit.client.1.smithi138.stdout:4/494: dread - d7/df/d35/d4c/d59/f9d zero size 2024-05-27T09:29:38.884 INFO:tasks.workunit.client.0.smithi119.stdout:4/995: mkdir d4/d6/d3e/df5/d141 0 2024-05-27T09:29:38.884 INFO:tasks.workunit.client.0.smithi119.stdout:2/929: getdents d2/d1a 0 2024-05-27T09:29:38.888 INFO:tasks.workunit.client.0.smithi119.stdout:7/991: dwrite d7/d1a/d99/fee [0,4194304] 0 2024-05-27T09:29:38.888 INFO:tasks.workunit.client.0.smithi119.stdout:2/930: mknod d2/deb/c13d 0 2024-05-27T09:29:38.889 INFO:tasks.workunit.client.0.smithi119.stdout:8/920: dread d3/d8/f34 [0,4194304] 0 2024-05-27T09:29:38.891 INFO:tasks.workunit.client.0.smithi119.stdout:1/968: dwrite d7/db/d12/d26/d4a/f81 [0,4194304] 0 2024-05-27T09:29:38.891 INFO:tasks.workunit.client.0.smithi119.stdout:1/969: write d7/dd/d25/d60/d61/fb3 [863282,5737] 0 2024-05-27T09:29:38.892 INFO:tasks.workunit.client.1.smithi138.stdout:4/495: rename c4 to d7/df/d5b/cab 0 2024-05-27T09:29:38.892 INFO:tasks.workunit.client.1.smithi138.stdout:4/496: chown d7/df/d5b/f89 211022759 1 2024-05-27T09:29:38.892 INFO:tasks.workunit.client.0.smithi119.stdout:4/996: creat d4/de/d21/f142 x:0 0 0 2024-05-27T09:29:38.893 INFO:tasks.workunit.client.1.smithi138.stdout:2/348: read d1/d15/f48 [1221142,38393] 0 2024-05-27T09:29:38.894 INFO:tasks.workunit.client.1.smithi138.stdout:2/349: getdents d1/d15/d2d 0 2024-05-27T09:29:38.896 INFO:tasks.workunit.client.1.smithi138.stdout:4/497: symlink d7/df/d35/lac 0 2024-05-27T09:29:38.900 INFO:tasks.workunit.client.1.smithi138.stdout:1/422: getdents d4 0 2024-05-27T09:29:38.901 INFO:tasks.workunit.client.1.smithi138.stdout:1/423: chown d4/d38/c46 377513518 1 2024-05-27T09:29:38.903 INFO:tasks.workunit.client.1.smithi138.stdout:8/388: dwrite d3/d8/d11/f26 [0,4194304] 0 2024-05-27T09:29:38.903 INFO:tasks.workunit.client.1.smithi138.stdout:8/389: truncate d3/d8/d32/d34/d38/f5d 4870506 0 2024-05-27T09:29:38.903 INFO:tasks.workunit.client.1.smithi138.stdout:8/390: chown f1 14619 1 2024-05-27T09:29:38.908 INFO:tasks.workunit.client.1.smithi138.stdout:4/498: unlink d7/df/f24 0 2024-05-27T09:29:38.908 INFO:tasks.workunit.client.1.smithi138.stdout:4/499: write d7/df/f41 [630945,94322] 0 2024-05-27T09:29:38.909 INFO:tasks.workunit.client.0.smithi119.stdout:8/921: truncate d3/d8/d1b/d77/d95/fab 2383858 0 2024-05-27T09:29:38.909 INFO:tasks.workunit.client.0.smithi119.stdout:8/922: dread - d3/d8/d1b/d46/da4/d113/f11a zero size 2024-05-27T09:29:38.909 INFO:tasks.workunit.client.1.smithi138.stdout:8/391: unlink d3/d8/c36 0 2024-05-27T09:29:38.912 INFO:tasks.workunit.client.1.smithi138.stdout:8/392: rmdir d3/d8/d11 39 2024-05-27T09:29:38.914 INFO:tasks.workunit.client.1.smithi138.stdout:1/424: rename d4/d1a/d2e/d32/f68 to d4/d48/d39/f85 0 2024-05-27T09:29:38.915 INFO:tasks.workunit.client.1.smithi138.stdout:1/425: write d4/d3c/d51/f61 [176575,21148] 0 2024-05-27T09:29:38.920 INFO:tasks.workunit.client.0.smithi119.stdout:1/970: link d7/dd/d44/c51 d7/db/d8f/c129 0 2024-05-27T09:29:38.921 INFO:tasks.workunit.client.1.smithi138.stdout:5/342: dread d3/f15 [0,4194304] 0 2024-05-27T09:29:38.921 INFO:tasks.workunit.client.1.smithi138.stdout:3/436: dwrite d0/d2/da/f8a [0,4194304] 0 2024-05-27T09:29:38.922 INFO:tasks.workunit.client.0.smithi119.stdout:8/923: symlink d3/de/l132 0 2024-05-27T09:29:38.923 INFO:tasks.workunit.client.1.smithi138.stdout:8/393: rename d3/d8/d11/c1c to d3/d60/c73 0 2024-05-27T09:29:38.923 INFO:tasks.workunit.client.1.smithi138.stdout:8/394: fdatasync d3/d8/d32/d34/d38/f46 0 2024-05-27T09:29:38.923 INFO:tasks.workunit.client.0.smithi119.stdout:4/997: symlink d4/de/d21/d40/d47/d5c/l143 0 2024-05-27T09:29:38.923 INFO:tasks.workunit.client.0.smithi119.stdout:4/998: write d4/d6/d81/dce/d88/f11b [974728,36495] 0 2024-05-27T09:29:38.926 INFO:tasks.workunit.client.1.smithi138.stdout:4/500: dread d7/df/d35/d4c/d59/d5d/d64/d6a/f6d [0,4194304] 0 2024-05-27T09:29:38.926 INFO:tasks.workunit.client.1.smithi138.stdout:1/426: read d4/d48/d39/d5f/d4d/d2b/f6c [1105954,21048] 0 2024-05-27T09:29:38.927 INFO:tasks.workunit.client.1.smithi138.stdout:0/404: write da/d17/d1d/d1f/f3e [1409010,126399] 0 2024-05-27T09:29:38.928 INFO:tasks.workunit.client.1.smithi138.stdout:0/405: write da/d17/d1d/d1f/d42/d4a/d53/f57 [974332,29068] 0 2024-05-27T09:29:38.928 INFO:tasks.workunit.client.1.smithi138.stdout:0/406: fdatasync da/d17/f31 0 2024-05-27T09:29:38.928 INFO:tasks.workunit.client.1.smithi138.stdout:0/407: creat da/d17/d1d/d1f/d42/d4a/d53/f87 x:0 0 0 2024-05-27T09:29:38.929 INFO:tasks.workunit.client.1.smithi138.stdout:0/408: fsync da/f29 0 2024-05-27T09:29:38.929 INFO:tasks.workunit.client.0.smithi119.stdout:0/910: rename d2/d110/d59 to d2/d110/d96/d132 0 2024-05-27T09:29:38.929 INFO:tasks.workunit.client.0.smithi119.stdout:0/911: chown d2/d6/d23/d7a/lec 4489357 1 2024-05-27T09:29:38.929 INFO:tasks.workunit.client.0.smithi119.stdout:4/999: write d4/d6/d28/d2f/db8/fcd [505256,15874] 0 2024-05-27T09:29:38.930 INFO:tasks.workunit.client.1.smithi138.stdout:1/427: truncate d4/d38/f44 973406 0 2024-05-27T09:29:38.930 INFO:tasks.workunit.client.1.smithi138.stdout:5/343: mkdir d3/d12/d32/d6b 0 2024-05-27T09:29:38.932 INFO:tasks.workunit.client.1.smithi138.stdout:8/395: mknod d3/d60/c74 0 2024-05-27T09:29:38.932 INFO:tasks.workunit.client.1.smithi138.stdout:8/396: read - d3/d8/f33 zero size 2024-05-27T09:29:38.937 INFO:tasks.workunit.client.1.smithi138.stdout:8/397: symlink d3/d8/d32/d45/d5c/l75 0 2024-05-27T09:29:38.938 INFO:tasks.workunit.client.0.smithi119.stdout:0/912: creat d2/d6/d23/d7a/f133 x:0 0 0 2024-05-27T09:29:38.939 INFO:tasks.workunit.client.1.smithi138.stdout:6/546: dread d3/f31 [4194304,4194304] 0 2024-05-27T09:29:38.940 INFO:tasks.workunit.client.0.smithi119.stdout:8/924: dread d3/f25 [0,4194304] 0 2024-05-27T09:29:38.940 INFO:tasks.workunit.client.1.smithi138.stdout:0/409: symlink da/d17/d1d/d1f/d4f/l88 0 2024-05-27T09:29:38.940 INFO:tasks.workunit.client.1.smithi138.stdout:0/410: truncate da/d2a/f85 247021 0 2024-05-27T09:29:38.941 INFO:tasks.workunit.client.1.smithi138.stdout:8/398: creat d3/d8/d64/f76 x:0 0 0 2024-05-27T09:29:38.942 INFO:tasks.workunit.client.1.smithi138.stdout:7/413: dwrite d0/d23/f50 [0,4194304] 0 2024-05-27T09:29:38.942 INFO:tasks.workunit.client.1.smithi138.stdout:7/414: creat d0/f8c x:0 0 0 2024-05-27T09:29:38.944 INFO:tasks.workunit.client.1.smithi138.stdout:1/428: truncate d4/d48/d39/d5f/d4d/f6d 2516323 0 2024-05-27T09:29:38.945 INFO:tasks.workunit.client.1.smithi138.stdout:3/437: dread d0/f12 [0,4194304] 0 2024-05-27T09:29:38.945 INFO:tasks.workunit.client.0.smithi119.stdout:7/992: rename d7/df/d19/d9a/da6/dda/c96 to d7/df/d19/d46/db5/c13d 0 2024-05-27T09:29:38.946 INFO:tasks.workunit.client.1.smithi138.stdout:3/438: fdatasync d0/d2/da/df/f85 0 2024-05-27T09:29:38.946 INFO:tasks.workunit.client.1.smithi138.stdout:3/439: chown d0/d2/da/f6b 90 1 2024-05-27T09:29:38.948 INFO:tasks.workunit.client.1.smithi138.stdout:0/411: link da/d17/d1d/d1f/d4f/c78 da/d17/d1d/d1f/d4f/c89 0 2024-05-27T09:29:38.948 INFO:tasks.workunit.client.1.smithi138.stdout:0/412: chown da/f20 342695 1 2024-05-27T09:29:38.948 INFO:tasks.workunit.client.1.smithi138.stdout:5/344: mknod d3/d12/d32/c6c 0 2024-05-27T09:29:38.951 INFO:tasks.workunit.client.1.smithi138.stdout:2/350: dwrite d1/d15/d2d/d17/d25/d4d/f5c [0,4194304] 0 2024-05-27T09:29:38.951 INFO:tasks.workunit.client.1.smithi138.stdout:7/415: mknod d0/d23/d3c/d84/c8d 0 2024-05-27T09:29:38.952 INFO:tasks.workunit.client.1.smithi138.stdout:1/429: mkdir d4/d48/d39/d5f/d86 0 2024-05-27T09:29:38.952 INFO:tasks.workunit.client.1.smithi138.stdout:1/430: fdatasync d4/d48/d39/d5f/d4d/d2b/f67 0 2024-05-27T09:29:38.952 INFO:tasks.workunit.client.1.smithi138.stdout:1/431: truncate d4/d1a/d2e/d4a/f80 507864 0 2024-05-27T09:29:38.952 INFO:tasks.workunit.client.1.smithi138.stdout:1/432: read - d4/d48/d39/d5f/d4d/d2b/f67 zero size 2024-05-27T09:29:38.952 INFO:tasks.workunit.client.1.smithi138.stdout:7/416: mkdir d0/d23/d59/d8e 0 2024-05-27T09:29:38.953 INFO:tasks.workunit.client.1.smithi138.stdout:7/417: getdents d0/d23/d49/d78 0 2024-05-27T09:29:38.954 INFO:tasks.workunit.client.1.smithi138.stdout:0/413: link da/d17/d1d/l23 da/d2a/l8a 0 2024-05-27T09:29:38.954 INFO:tasks.workunit.client.1.smithi138.stdout:0/414: write da/f66 [4676968,102980] 0 2024-05-27T09:29:38.955 INFO:tasks.workunit.client.0.smithi119.stdout:1/971: rename d7/d6e/fe4 to d7/db/f12a 0 2024-05-27T09:29:38.955 INFO:tasks.workunit.client.0.smithi119.stdout:1/972: write d7/db/d8f/f108 [155160,23651] 0 2024-05-27T09:29:38.956 INFO:tasks.workunit.client.0.smithi119.stdout:1/973: creat d7/dd/d36/f12b x:0 0 0 2024-05-27T09:29:38.958 INFO:tasks.workunit.client.0.smithi119.stdout:9/827: write d18/d19/d1b/d29/f71 [420591,96601] 0 2024-05-27T09:29:38.959 INFO:tasks.workunit.client.1.smithi138.stdout:6/547: dread d3/d11/dc0/d5a/d6a/f7f [0,4194304] 0 2024-05-27T09:29:38.963 INFO:tasks.workunit.client.1.smithi138.stdout:4/501: read d7/df/d35/d4c/d59/f73 [3579490,64966] 0 2024-05-27T09:29:38.966 INFO:tasks.workunit.client.1.smithi138.stdout:1/433: symlink d4/d48/d39/d5f/l87 0 2024-05-27T09:29:38.966 INFO:tasks.workunit.client.1.smithi138.stdout:2/351: write d1/d10/f1a [1002641,2987] 0 2024-05-27T09:29:38.975 INFO:tasks.workunit.client.0.smithi119.stdout:7/993: write d7/d1a/f20 [985919,63786] 0 2024-05-27T09:29:38.975 INFO:tasks.workunit.client.0.smithi119.stdout:2/931: dwrite d2/da/d8a/dd4/d89/d94/d87/f118 [0,4194304] 0 2024-05-27T09:29:38.975 INFO:tasks.workunit.client.0.smithi119.stdout:0/913: dread d2/d6/d75/fa2 [0,4194304] 0 2024-05-27T09:29:38.976 INFO:tasks.workunit.client.0.smithi119.stdout:6/895: dwrite d0/d2/d57/f6d [0,4194304] 0 2024-05-27T09:29:38.982 INFO:tasks.workunit.client.1.smithi138.stdout:7/418: mknod d0/d7/c8f 0 2024-05-27T09:29:38.982 INFO:tasks.workunit.client.1.smithi138.stdout:7/419: dread - d0/d7/d60/f63 zero size 2024-05-27T09:29:38.982 INFO:tasks.workunit.client.1.smithi138.stdout:6/548: creat d3/d11/d2d/d37/d8b/fc7 x:0 0 0 2024-05-27T09:29:38.983 INFO:tasks.workunit.client.1.smithi138.stdout:5/345: dread d3/d12/d32/f4e [0,4194304] 0 2024-05-27T09:29:38.984 INFO:tasks.workunit.client.1.smithi138.stdout:8/399: dread d3/d8/d32/d45/f4e [0,4194304] 0 2024-05-27T09:29:38.989 INFO:tasks.workunit.client.0.smithi119.stdout:9/828: write d18/d19/d1b/f4c [33834,14615] 0 2024-05-27T09:29:38.989 INFO:tasks.workunit.client.0.smithi119.stdout:1/974: chown d7/dd/d25/la3 2395452 1 2024-05-27T09:29:38.989 INFO:tasks.workunit.client.0.smithi119.stdout:1/975: stat d7/dd/db0/dcd/df2 0 2024-05-27T09:29:38.989 INFO:tasks.workunit.client.0.smithi119.stdout:9/829: stat d18/d19/d1b/d40/d44/dcb 0 2024-05-27T09:29:38.989 INFO:tasks.workunit.client.0.smithi119.stdout:1/976: readlink d7/db/d12/ddb/lba 0 2024-05-27T09:29:38.989 INFO:tasks.workunit.client.0.smithi119.stdout:9/830: dread - d18/d19/d9b/fb3 zero size 2024-05-27T09:29:38.989 INFO:tasks.workunit.client.0.smithi119.stdout:1/977: chown d7/db/d12/d26/d4a/l70 475824650 1 2024-05-27T09:29:38.989 INFO:tasks.workunit.client.0.smithi119.stdout:1/978: write d7/db/d35/fd4 [223719,120180] 0 2024-05-27T09:29:38.992 INFO:tasks.workunit.client.1.smithi138.stdout:2/352: dread d1/d15/d2d/f39 [0,4194304] 0 2024-05-27T09:29:38.994 INFO:tasks.workunit.client.1.smithi138.stdout:1/434: mkdir d4/d88 0 2024-05-27T09:29:38.994 INFO:tasks.workunit.client.1.smithi138.stdout:1/435: chown d4/d48/d39/d5f/l87 0 1 2024-05-27T09:29:38.994 INFO:tasks.workunit.client.1.smithi138.stdout:1/436: fdatasync d4/d1a/d2e/f62 0 2024-05-27T09:29:38.996 INFO:tasks.workunit.client.0.smithi119.stdout:7/994: dread d7/df/d19/d9a/da6/dda/d44/f134 [0,4194304] 0 2024-05-27T09:29:39.000 INFO:tasks.workunit.client.1.smithi138.stdout:6/549: creat d3/d11/dc0/d74/fc8 x:0 0 0 2024-05-27T09:29:39.000 INFO:tasks.workunit.client.0.smithi119.stdout:8/925: dwrite d3/ff3 [0,4194304] 0 2024-05-27T09:29:39.004 INFO:tasks.workunit.client.0.smithi119.stdout:0/914: mknod d2/d6/d9/d24/d30/dcc/de3/c134 0 2024-05-27T09:29:39.007 INFO:tasks.workunit.client.1.smithi138.stdout:2/353: dread d1/d15/d2d/f3e [0,4194304] 0 2024-05-27T09:29:39.009 INFO:tasks.workunit.client.1.smithi138.stdout:3/440: dwrite d0/d2/da/df/d10/d2a/d48/f4d [0,4194304] 0 2024-05-27T09:29:39.009 INFO:tasks.workunit.client.1.smithi138.stdout:2/354: creat d1/d15/d2d/d17/d25/f74 x:0 0 0 2024-05-27T09:29:39.010 INFO:tasks.workunit.client.0.smithi119.stdout:6/896: creat d0/d2/d57/d37/d74/df1/d109/d118/f141 x:0 0 0 2024-05-27T09:29:39.011 INFO:tasks.workunit.client.1.smithi138.stdout:8/400: read d3/d8/d32/f3b [2469252,12722] 0 2024-05-27T09:29:39.014 INFO:tasks.workunit.client.1.smithi138.stdout:3/441: mkdir d0/d3f/d60/d93 0 2024-05-27T09:29:39.014 INFO:tasks.workunit.client.1.smithi138.stdout:3/442: truncate d0/d2/da/df/d10/f4a 1591744 0 2024-05-27T09:29:39.014 INFO:tasks.workunit.client.1.smithi138.stdout:3/443: creat d0/d2/da/f94 x:0 0 0 2024-05-27T09:29:39.014 INFO:tasks.workunit.client.1.smithi138.stdout:3/444: write d0/d2/da/f33 [884128,109591] 0 2024-05-27T09:29:39.014 INFO:tasks.workunit.client.1.smithi138.stdout:3/445: dread - d0/d2/da/df/d10/f77 zero size 2024-05-27T09:29:39.016 INFO:tasks.workunit.client.0.smithi119.stdout:9/831: mknod d18/db4/c103 0 2024-05-27T09:29:39.017 INFO:tasks.workunit.client.0.smithi119.stdout:9/832: creat d18/d19/d1b/d40/db7/f104 x:0 0 0 2024-05-27T09:29:39.017 INFO:tasks.workunit.client.0.smithi119.stdout:9/833: write fc [1192614,27863] 0 2024-05-27T09:29:39.021 INFO:tasks.workunit.client.1.smithi138.stdout:2/355: rename d1/d15/d2d/d17/c1f to d1/d15/d2d/d31/c75 0 2024-05-27T09:29:39.021 INFO:tasks.workunit.client.1.smithi138.stdout:2/356: creat d1/d10/f76 x:0 0 0 2024-05-27T09:29:39.021 INFO:tasks.workunit.client.1.smithi138.stdout:2/357: write d1/d10/f76 [331353,63620] 0 2024-05-27T09:29:39.023 INFO:tasks.workunit.client.1.smithi138.stdout:8/401: rename d3/f52 to d3/f77 0 2024-05-27T09:29:39.025 INFO:tasks.workunit.client.1.smithi138.stdout:5/346: dread d3/d12/d16/d1d/d22/d2b/d36/f41 [0,4194304] 0 2024-05-27T09:29:39.025 INFO:tasks.workunit.client.1.smithi138.stdout:5/347: read - d3/d12/d16/d1d/d22/f3a zero size 2024-05-27T09:29:39.028 INFO:tasks.workunit.client.1.smithi138.stdout:4/502: rmdir d7/d6e 39 2024-05-27T09:29:39.031 INFO:tasks.workunit.client.0.smithi119.stdout:7/995: truncate d7/df/d19/d9a/da6/dda/d44/ffd 3468829 0 2024-05-27T09:29:39.031 INFO:tasks.workunit.client.1.smithi138.stdout:2/358: link d1/d10/d3c/c63 d1/d15/d2d/d31/c77 0 2024-05-27T09:29:39.033 INFO:tasks.workunit.client.1.smithi138.stdout:8/402: mkdir d3/d8/d32/d34/d78 0 2024-05-27T09:29:39.033 INFO:tasks.workunit.client.1.smithi138.stdout:8/403: creat d3/d8/d32/d34/d38/f79 x:0 0 0 2024-05-27T09:29:39.034 INFO:tasks.workunit.client.1.smithi138.stdout:2/359: symlink d1/d15/d2d/l78 0 2024-05-27T09:29:39.037 INFO:tasks.workunit.client.1.smithi138.stdout:8/404: link d3/d8/d32/d34/d38/f63 d3/d8/d32/d45/d5c/f7a 0 2024-05-27T09:29:39.037 INFO:tasks.workunit.client.0.smithi119.stdout:6/897: dread d0/d2/d1c/f2a [4194304,4194304] 0 2024-05-27T09:29:39.038 INFO:tasks.workunit.client.1.smithi138.stdout:1/437: getdents d4/d38 0 2024-05-27T09:29:39.038 INFO:tasks.workunit.client.1.smithi138.stdout:1/438: chown d4/d48/d39/d5f/d4d 9526 1 2024-05-27T09:29:39.038 INFO:tasks.workunit.client.1.smithi138.stdout:1/439: creat d4/d48/d22/f89 x:0 0 0 2024-05-27T09:29:39.041 INFO:tasks.workunit.client.1.smithi138.stdout:7/420: dwrite d0/d2b/f40 [0,4194304] 0 2024-05-27T09:29:39.046 INFO:tasks.workunit.client.1.smithi138.stdout:8/405: mknod d3/d8/d32/d45/c7b 0 2024-05-27T09:29:39.048 INFO:tasks.workunit.client.1.smithi138.stdout:2/360: mknod d1/d15/d2d/d31/d4b/c79 0 2024-05-27T09:29:39.048 INFO:tasks.workunit.client.1.smithi138.stdout:2/361: readlink d1/d15/d2d/d17/d25/d4d/d4e/l72 0 2024-05-27T09:29:39.049 INFO:tasks.workunit.client.0.smithi119.stdout:0/915: mkdir d2/d1a/d20/d135 0 2024-05-27T09:29:39.049 INFO:tasks.workunit.client.0.smithi119.stdout:0/916: creat d2/d6/d9/d2f/f136 x:0 0 0 2024-05-27T09:29:39.050 INFO:tasks.workunit.client.1.smithi138.stdout:8/406: unlink d3/d8/d24/l62 0 2024-05-27T09:29:39.050 INFO:tasks.workunit.client.1.smithi138.stdout:8/407: stat d3/d8/l2d 0 2024-05-27T09:29:39.053 INFO:tasks.workunit.client.1.smithi138.stdout:5/348: rmdir d3/d12/d16/d1d/d23/d3b 39 2024-05-27T09:29:39.053 INFO:tasks.workunit.client.1.smithi138.stdout:2/362: link d1/d15/f4f d1/f7a 0 2024-05-27T09:29:39.061 INFO:tasks.workunit.client.1.smithi138.stdout:5/349: dread d3/d12/f4d [0,4194304] 0 2024-05-27T09:29:39.064 INFO:tasks.workunit.client.0.smithi119.stdout:2/932: dwrite d2/f26 [0,4194304] 0 2024-05-27T09:29:39.067 INFO:tasks.workunit.client.1.smithi138.stdout:5/350: write d3/d12/d16/d1d/d22/d2b/d36/f41 [1429075,83931] 0 2024-05-27T09:29:39.071 INFO:tasks.workunit.client.1.smithi138.stdout:8/408: read d3/f29 [1964133,90155] 0 2024-05-27T09:29:39.071 INFO:tasks.workunit.client.1.smithi138.stdout:8/409: creat d3/d8/d64/f7c x:0 0 0 2024-05-27T09:29:39.071 INFO:tasks.workunit.client.1.smithi138.stdout:8/410: read - d3/d8/d64/f7c zero size 2024-05-27T09:29:39.072 INFO:tasks.workunit.client.1.smithi138.stdout:4/503: rename d7/df/l93 to d7/lad 0 2024-05-27T09:29:39.075 INFO:tasks.workunit.client.1.smithi138.stdout:2/363: read d1/d10/f64 [182739,48402] 0 2024-05-27T09:29:39.075 INFO:tasks.workunit.client.1.smithi138.stdout:8/411: read d3/d8/d24/f44 [715543,117490] 0 2024-05-27T09:29:39.085 INFO:tasks.workunit.client.0.smithi119.stdout:8/926: dwrite d3/d8/d1b/d46/f81 [0,4194304] 0 2024-05-27T09:29:39.085 INFO:tasks.workunit.client.0.smithi119.stdout:8/927: readlink d3/de/lc8 0 2024-05-27T09:29:39.085 INFO:tasks.workunit.client.0.smithi119.stdout:8/928: dread - d3/f9d zero size 2024-05-27T09:29:39.087 INFO:tasks.workunit.client.1.smithi138.stdout:4/504: rename d7/f23 to d7/df/d35/d4c/d59/d5d/d64/fae 0 2024-05-27T09:29:39.087 INFO:tasks.workunit.client.1.smithi138.stdout:4/505: write d7/d27/f9a [354678,117714] 0 2024-05-27T09:29:39.087 INFO:tasks.workunit.client.1.smithi138.stdout:4/506: getdents d7/df/d5b/da1 0 2024-05-27T09:29:39.087 INFO:tasks.workunit.client.1.smithi138.stdout:4/507: readlink d7/df/d35/d4c/d59/d5d/d64/d6a/la8 0 2024-05-27T09:29:39.088 INFO:tasks.workunit.client.0.smithi119.stdout:7/996: mknod d7/df/d19/d4d/c13e 0 2024-05-27T09:29:39.089 INFO:tasks.workunit.client.0.smithi119.stdout:7/997: write d7/df/d19/d97/f110 [284949,23206] 0 2024-05-27T09:29:39.089 INFO:tasks.workunit.client.0.smithi119.stdout:7/998: read d7/df/d19/d9a/da6/dda/d44/fde [1010778,112509] 0 2024-05-27T09:29:39.089 INFO:tasks.workunit.client.0.smithi119.stdout:7/999: write d7/df/d19/d9a/da6/dda/d44/ff7 [186012,57765] 0 2024-05-27T09:29:39.089 INFO:tasks.workunit.client.0.smithi119.stdout:6/898: rename d0/d2/d4e/d5a/d76/l10d to d0/d2/d1c/d21/d58/l142 0 2024-05-27T09:29:39.089 INFO:tasks.workunit.client.0.smithi119.stdout:6/899: stat d0/d2/d1c/d21/c9c 0 2024-05-27T09:29:39.093 INFO:tasks.workunit.client.0.smithi119.stdout:6/900: read d0/d2/f80 [787240,90688] 0 2024-05-27T09:29:39.093 INFO:tasks.workunit.client.0.smithi119.stdout:6/901: readlink d0/d2/d57/d37/d115/l134 0 2024-05-27T09:29:39.095 INFO:tasks.workunit.client.1.smithi138.stdout:3/446: dread d0/d2/da/f6b [0,4194304] 0 2024-05-27T09:29:39.095 INFO:tasks.workunit.client.1.smithi138.stdout:6/550: dwrite d3/d11/dc0/d43/d1c/fa9 [0,4194304] 0 2024-05-27T09:29:39.095 INFO:tasks.workunit.client.1.smithi138.stdout:3/447: chown d0/d2/da/df/d10/d2a/f42 99 1 2024-05-27T09:29:39.095 INFO:tasks.workunit.client.1.smithi138.stdout:7/421: rename d0/d2b/l6c to d0/d23/d3a/l90 0 2024-05-27T09:29:39.095 INFO:tasks.workunit.client.1.smithi138.stdout:7/422: fdatasync d0/d23/d3a/f46 0 2024-05-27T09:29:39.097 INFO:tasks.workunit.client.1.smithi138.stdout:3/448: rename d0/d2/da/df/d10/f81 to d0/d3f/d60/f95 0 2024-05-27T09:29:39.097 INFO:tasks.workunit.client.1.smithi138.stdout:3/449: write d0/d2/da/df/d10/f77 [181627,7706] 0 2024-05-27T09:29:39.097 INFO:tasks.workunit.client.1.smithi138.stdout:3/450: write d0/d3f/f7b [693716,279] 0 2024-05-27T09:29:39.097 INFO:tasks.workunit.client.1.smithi138.stdout:3/451: chown d0/d2/da/df/d10/l59 13166717 1 2024-05-27T09:29:39.105 INFO:tasks.workunit.client.1.smithi138.stdout:5/351: rename d3 to d3/d12/d26/d6d 22 2024-05-27T09:29:39.105 INFO:tasks.workunit.client.0.smithi119.stdout:0/917: unlink d2/d6/d9/f6d 0 2024-05-27T09:29:39.109 INFO:tasks.workunit.client.1.smithi138.stdout:8/412: dread d3/d8/d32/f49 [0,4194304] 0 2024-05-27T09:29:39.109 INFO:tasks.workunit.client.1.smithi138.stdout:8/413: read f1 [1698444,99521] 0 2024-05-27T09:29:39.111 INFO:tasks.workunit.client.0.smithi119.stdout:9/834: dwrite d18/d19/d1b/d40/f62 [0,4194304] 0 2024-05-27T09:29:39.112 INFO:tasks.workunit.client.1.smithi138.stdout:6/551: creat d3/d11/dc0/d5a/db4/fc9 x:0 0 0 2024-05-27T09:29:39.112 INFO:tasks.workunit.client.1.smithi138.stdout:6/552: truncate d3/d11/dc0/d43/d1c/f9c 533660 0 2024-05-27T09:29:39.112 INFO:tasks.workunit.client.1.smithi138.stdout:6/553: chown d3/d19 0 1 2024-05-27T09:29:39.112 INFO:tasks.workunit.client.1.smithi138.stdout:6/554: dread - d3/d11/dc0/d43/d1c/d32/d9b/fb7 zero size 2024-05-27T09:29:39.112 INFO:tasks.workunit.client.1.smithi138.stdout:6/555: fdatasync d3/d11/dc0/d43/d1c/f4d 0 2024-05-27T09:29:39.119 INFO:tasks.workunit.client.1.smithi138.stdout:5/352: unlink d3/d12/d16/d1d/d23/l28 0 2024-05-27T09:29:39.129 INFO:tasks.workunit.client.0.smithi119.stdout:2/933: creat d2/da/d8a/dd4/d89/d94/de2/df9/f13e x:0 0 0 2024-05-27T09:29:39.130 INFO:tasks.workunit.client.0.smithi119.stdout:0/918: dread d2/d1a/f21 [0,4194304] 0 2024-05-27T09:29:39.137 INFO:tasks.workunit.client.1.smithi138.stdout:5/353: mkdir d3/d12/d32/d6b/d6e 0 2024-05-27T09:29:39.150 INFO:tasks.workunit.client.1.smithi138.stdout:5/354: mkdir d3/d12/d16/d1d/d23/d3b/d6f 0 2024-05-27T09:29:39.150 INFO:tasks.workunit.client.1.smithi138.stdout:7/423: dread d0/f24 [0,4194304] 0 2024-05-27T09:29:39.152 INFO:tasks.workunit.client.1.smithi138.stdout:7/424: rmdir d0/d7/d64/d87 0 2024-05-27T09:29:39.158 INFO:tasks.workunit.client.0.smithi119.stdout:8/929: mkdir d3/d8/d1b/d46/da4/d113/d74/d133 0 2024-05-27T09:29:39.158 INFO:tasks.workunit.client.0.smithi119.stdout:8/930: chown d3/d8/d1b/d46/d92/d111 282 1 2024-05-27T09:29:39.159 INFO:tasks.workunit.client.0.smithi119.stdout:6/902: creat d0/d2/d1c/d21/f143 x:0 0 0 2024-05-27T09:29:39.164 INFO:tasks.workunit.client.0.smithi119.stdout:9/835: rename d18/db4/dbe/cca to d18/d19/d1b/d40/d44/c105 0 2024-05-27T09:29:39.170 INFO:tasks.workunit.client.0.smithi119.stdout:9/836: mknod d18/d19/dfa/c106 0 2024-05-27T09:29:39.170 INFO:tasks.workunit.client.0.smithi119.stdout:9/837: chown d18/d19/d1b/d29/f57 399181 1 2024-05-27T09:29:39.175 INFO:tasks.workunit.client.0.smithi119.stdout:0/919: rename d2/d6/d75 to d2/d6/d9/d24/d30/dcc/de6/d137 0 2024-05-27T09:29:39.177 INFO:tasks.workunit.client.0.smithi119.stdout:0/920: unlink d2/d6/d9/d24/d30/dcc/d124/f12c 0 2024-05-27T09:29:39.189 INFO:tasks.workunit.client.1.smithi138.stdout:8/414: dwrite d3/d8/d32/d34/d38/f46 [0,4194304] 0 2024-05-27T09:29:39.191 INFO:tasks.workunit.client.1.smithi138.stdout:8/415: mkdir d3/d8/d24/d4d/d7d 0 2024-05-27T09:29:39.193 INFO:tasks.workunit.client.1.smithi138.stdout:8/416: mkdir d3/d8/d24/d55/d7e 0 2024-05-27T09:29:39.195 INFO:tasks.workunit.client.0.smithi119.stdout:9/838: write d18/d19/d1b/d29/f2c [5703143,74819] 0 2024-05-27T09:29:39.197 INFO:tasks.workunit.client.1.smithi138.stdout:8/417: rename d3/d8/d64/c4f to d3/d8/d24/c7f 0 2024-05-27T09:29:39.199 INFO:tasks.workunit.client.1.smithi138.stdout:8/418: link d3/d60/c74 d3/d8/c80 0 2024-05-27T09:29:39.199 INFO:tasks.workunit.client.1.smithi138.stdout:8/419: write d3/d8/d32/f3b [2866036,114966] 0 2024-05-27T09:29:39.200 INFO:tasks.workunit.client.0.smithi119.stdout:2/934: dwrite d2/da/d8a/dd4/d89/d94/d87/f9a [4194304,4194304] 0 2024-05-27T09:29:39.201 INFO:tasks.workunit.client.1.smithi138.stdout:2/364: dwrite d1/d15/d2d/d17/d25/f28 [0,4194304] 0 2024-05-27T09:29:39.213 INFO:tasks.workunit.client.1.smithi138.stdout:4/508: dwrite d7/df/d35/d4c/d59/d5d/d64/d6a/f94 [0,4194304] 0 2024-05-27T09:29:39.222 INFO:tasks.workunit.client.0.smithi119.stdout:2/935: link d2/da/d2c/fe1 d2/da/d8a/dd4/d69/f13f 0 2024-05-27T09:29:39.222 INFO:tasks.workunit.client.0.smithi119.stdout:2/936: chown d2/c123 90101479 1 2024-05-27T09:29:39.225 INFO:tasks.workunit.client.1.smithi138.stdout:4/509: creat d7/d6e/faf x:0 0 0 2024-05-27T09:29:39.229 INFO:tasks.workunit.client.0.smithi119.stdout:2/937: link d2/da/d8a/d8f/d11e/f125 d2/da/d8a/dd4/d69/d134/f140 0 2024-05-27T09:29:39.238 INFO:tasks.workunit.client.0.smithi119.stdout:2/938: rename d2/da/d8a/d8f/lf4 to d2/da/d6f/l141 0 2024-05-27T09:29:39.241 INFO:tasks.workunit.client.0.smithi119.stdout:8/931: dwrite d3/d6e/f75 [0,4194304] 0 2024-05-27T09:29:39.241 INFO:tasks.workunit.client.0.smithi119.stdout:8/932: chown d3/d39/ce4 50389 1 2024-05-27T09:29:39.242 INFO:tasks.workunit.client.0.smithi119.stdout:1/979: sync 2024-05-27T09:29:39.243 INFO:tasks.workunit.client.0.smithi119.stdout:2/939: write d2/da/d8a/dd4/d89/d94/d87/ff3 [227463,83494] 0 2024-05-27T09:29:39.243 INFO:tasks.workunit.client.0.smithi119.stdout:0/921: dwrite d2/d6/d9/d2f/fe9 [0,4194304] 0 2024-05-27T09:29:39.244 INFO:tasks.workunit.client.0.smithi119.stdout:1/980: mknod d7/d40/c12c 0 2024-05-27T09:29:39.246 INFO:tasks.workunit.client.0.smithi119.stdout:2/940: symlink d2/da/d15/d53/l142 0 2024-05-27T09:29:39.248 INFO:tasks.workunit.client.1.smithi138.stdout:7/425: dwrite d0/d7/d43/f7a [0,4194304] 0 2024-05-27T09:29:39.248 INFO:tasks.workunit.client.0.smithi119.stdout:0/922: link d2/d6/d9/c78 d2/d6/d9/d24/d30/dcc/de3/d10a/c138 0 2024-05-27T09:29:39.249 INFO:tasks.workunit.client.0.smithi119.stdout:2/941: mknod d2/da/d8a/dd4/d89/d94/d81/c143 0 2024-05-27T09:29:39.250 INFO:tasks.workunit.client.1.smithi138.stdout:5/355: dwrite d3/d12/d26/d53/f58 [0,4194304] 0 2024-05-27T09:29:39.250 INFO:tasks.workunit.client.1.smithi138.stdout:6/556: dwrite d3/d11/dc0/d43/d1c/f59 [4194304,4194304] 0 2024-05-27T09:29:39.250 INFO:tasks.workunit.client.1.smithi138.stdout:7/426: mkdir d0/d7/d57/d91 0 2024-05-27T09:29:39.251 INFO:tasks.workunit.client.0.smithi119.stdout:0/923: rename d2/d6/d23/d7a/f62 to d2/d110/d96/f139 0 2024-05-27T09:29:39.251 INFO:tasks.workunit.client.0.smithi119.stdout:0/924: fsync d2/d6/d9/f2c 0 2024-05-27T09:29:39.253 INFO:tasks.workunit.client.1.smithi138.stdout:3/452: dwrite d0/d3f/d60/f95 [0,4194304] 0 2024-05-27T09:29:39.253 INFO:tasks.workunit.client.1.smithi138.stdout:7/427: getdents d0/d7/d60 0 2024-05-27T09:29:39.253 INFO:tasks.workunit.client.1.smithi138.stdout:7/428: truncate d0/d7/d60/f63 639575 0 2024-05-27T09:29:39.254 INFO:tasks.workunit.client.0.smithi119.stdout:0/925: rmdir d2/d6/d9/d24/d30/dcc 39 2024-05-27T09:29:39.254 INFO:tasks.workunit.client.0.smithi119.stdout:0/926: dread - d2/d6/d23/d7a/fd4 zero size 2024-05-27T09:29:39.254 INFO:tasks.workunit.client.0.smithi119.stdout:0/927: write d2/d6/d23/db8/d118/ff6 [68753,94615] 0 2024-05-27T09:29:39.255 INFO:tasks.workunit.client.0.smithi119.stdout:0/928: write d2/d6/d23/d7a/f114 [870162,84915] 0 2024-05-27T09:29:39.255 INFO:tasks.workunit.client.0.smithi119.stdout:0/929: creat d2/d6/d9/d24/d30/dcc/d124/f13a x:0 0 0 2024-05-27T09:29:39.255 INFO:tasks.workunit.client.0.smithi119.stdout:0/930: dread - d2/d110/fa7 zero size 2024-05-27T09:29:39.264 INFO:tasks.workunit.client.1.smithi138.stdout:7/429: symlink d0/d23/d49/d78/l92 0 2024-05-27T09:29:39.264 INFO:tasks.workunit.client.1.smithi138.stdout:7/430: readlink d0/l75 0 2024-05-27T09:29:39.264 INFO:tasks.workunit.client.1.smithi138.stdout:7/431: chown d0/f24 404 1 2024-05-27T09:29:39.264 INFO:tasks.workunit.client.1.smithi138.stdout:3/453: creat d0/d3f/d60/f96 x:0 0 0 2024-05-27T09:29:39.264 INFO:tasks.workunit.client.1.smithi138.stdout:3/454: creat d0/d3f/d50/f97 x:0 0 0 2024-05-27T09:29:39.264 INFO:tasks.workunit.client.1.smithi138.stdout:3/455: fdatasync d0/d3f/d60/f95 0 2024-05-27T09:29:39.264 INFO:tasks.workunit.client.1.smithi138.stdout:6/557: getdents d3/d11/dc0/d43 0 2024-05-27T09:29:39.264 INFO:tasks.workunit.client.1.smithi138.stdout:7/432: creat d0/d23/d3c/f93 x:0 0 0 2024-05-27T09:29:39.268 INFO:tasks.workunit.client.0.smithi119.stdout:0/931: link d2/d6/d9/c78 d2/d6/d9/d24/d30/dcc/de6/c13b 0 2024-05-27T09:29:39.270 INFO:tasks.workunit.client.0.smithi119.stdout:0/932: mknod d2/d6/d9/d24/c13c 0 2024-05-27T09:29:39.271 INFO:tasks.workunit.client.1.smithi138.stdout:7/433: unlink d0/d7/d60/f63 0 2024-05-27T09:29:39.273 INFO:tasks.workunit.client.0.smithi119.stdout:0/933: mknod d2/d6/d23/db8/c13d 0 2024-05-27T09:29:39.273 INFO:tasks.workunit.client.0.smithi119.stdout:0/934: dread - d2/d6/d9/f85 zero size 2024-05-27T09:29:39.273 INFO:tasks.workunit.client.0.smithi119.stdout:2/942: dread d2/da/f1c [0,4194304] 0 2024-05-27T09:29:39.273 INFO:tasks.workunit.client.0.smithi119.stdout:0/935: write d2/d6/d23/d7a/d53/dd1/f111 [58110,88446] 0 2024-05-27T09:29:39.273 INFO:tasks.workunit.client.0.smithi119.stdout:2/943: dread - d2/fbe zero size 2024-05-27T09:29:39.273 INFO:tasks.workunit.client.0.smithi119.stdout:2/944: truncate d2/fbe 230148 0 2024-05-27T09:29:39.274 INFO:tasks.workunit.client.1.smithi138.stdout:3/456: rmdir d0/d2/da/df/d10/d32 39 2024-05-27T09:29:39.277 INFO:tasks.workunit.client.1.smithi138.stdout:8/420: dwrite d3/d8/d32/d34/d38/f46 [0,4194304] 0 2024-05-27T09:29:39.278 INFO:tasks.workunit.client.0.smithi119.stdout:0/936: getdents d2/d1a/d20 0 2024-05-27T09:29:39.278 INFO:tasks.workunit.client.0.smithi119.stdout:0/937: dread - d2/d6/ffc zero size 2024-05-27T09:29:39.282 INFO:tasks.workunit.client.1.smithi138.stdout:8/421: stat d3/d8/d11/l1b 0 2024-05-27T09:29:39.282 INFO:tasks.workunit.client.1.smithi138.stdout:8/422: creat d3/d8/d32/f81 x:0 0 0 2024-05-27T09:29:39.284 INFO:tasks.workunit.client.1.smithi138.stdout:8/423: stat d3/d8/c65 0 2024-05-27T09:29:39.284 INFO:tasks.workunit.client.1.smithi138.stdout:8/424: write d3/d8/d64/f5e [861078,22193] 0 2024-05-27T09:29:39.284 INFO:tasks.workunit.client.1.smithi138.stdout:8/425: read - d3/d8/d32/f81 zero size 2024-05-27T09:29:39.288 INFO:tasks.workunit.client.1.smithi138.stdout:3/457: rmdir d0/d2/da/df 39 2024-05-27T09:29:39.288 INFO:tasks.workunit.client.1.smithi138.stdout:3/458: dread - d0/d2/da/f94 zero size 2024-05-27T09:29:39.288 INFO:tasks.workunit.client.1.smithi138.stdout:3/459: chown d0/d2/da/df/d10/l1e 49283 1 2024-05-27T09:29:39.288 INFO:tasks.workunit.client.1.smithi138.stdout:3/460: stat d0/d2/da/df/d10/d32/f1d 0 2024-05-27T09:29:39.288 INFO:tasks.workunit.client.1.smithi138.stdout:3/461: stat d0/d2/da/df/d10/d2a/d48/f5f 0 2024-05-27T09:29:39.289 INFO:tasks.workunit.client.0.smithi119.stdout:0/938: unlink d2/d6/d23/d7a/fd4 0 2024-05-27T09:29:39.289 INFO:tasks.workunit.client.0.smithi119.stdout:0/939: chown d2/d6/d9/d24/d69/l7b 24712 1 2024-05-27T09:29:39.291 INFO:tasks.workunit.client.1.smithi138.stdout:3/462: creat d0/d3f/f98 x:0 0 0 2024-05-27T09:29:39.291 INFO:tasks.workunit.client.0.smithi119.stdout:2/945: link d2/l52 d2/da/d8a/dd4/d89/de8/d12f/l144 0 2024-05-27T09:29:39.296 INFO:tasks.workunit.client.1.smithi138.stdout:2/365: truncate d1/d15/d2d/d17/d25/d4d/f5c 3471037 0 2024-05-27T09:29:39.298 INFO:tasks.workunit.client.1.smithi138.stdout:8/426: write d3/d8/d24/d4d/f61 [1211849,84633] 0 2024-05-27T09:29:39.299 INFO:tasks.workunit.client.1.smithi138.stdout:8/427: symlink d3/d8/d24/d55/d66/l82 0 2024-05-27T09:29:39.301 INFO:tasks.workunit.client.1.smithi138.stdout:8/428: mkdir d3/d8/d24/d55/d7e/d83 0 2024-05-27T09:29:39.302 INFO:tasks.workunit.client.0.smithi119.stdout:2/946: rename d2/deb to d2/da/d8a/d12d/d145 0 2024-05-27T09:29:39.314 INFO:tasks.workunit.client.1.smithi138.stdout:8/429: write d3/d8/f42 [115935,108442] 0 2024-05-27T09:29:39.316 INFO:tasks.workunit.client.1.smithi138.stdout:6/558: dwrite d3/d11/dc0/d43/d1c/f4d [0,4194304] 0 2024-05-27T09:29:39.316 INFO:tasks.workunit.client.1.smithi138.stdout:6/559: write d3/d11/dc0/d43/d1c/f9c [654652,13006] 0 2024-05-27T09:29:39.316 INFO:tasks.workunit.client.1.smithi138.stdout:6/560: truncate d3/d11/d2d/d37/d7e/f9f 125639 0 2024-05-27T09:29:39.317 INFO:tasks.workunit.client.1.smithi138.stdout:6/561: creat d3/d11/dc0/d43/d71/fca x:0 0 0 2024-05-27T09:29:39.323 INFO:tasks.workunit.client.1.smithi138.stdout:6/562: mknod d3/d11/d33/ccb 0 2024-05-27T09:29:39.323 INFO:tasks.workunit.client.1.smithi138.stdout:6/563: write d3/d11/d2d/d37/d7e/f9f [562237,73823] 0 2024-05-27T09:29:39.323 INFO:tasks.workunit.client.1.smithi138.stdout:6/564: fsync d3/d19/f41 0 2024-05-27T09:29:39.323 INFO:tasks.workunit.client.1.smithi138.stdout:6/565: creat d3/d11/dc0/d43/dbe/dc3/fcc x:0 0 0 2024-05-27T09:29:39.323 INFO:tasks.workunit.client.1.smithi138.stdout:6/566: symlink d3/d11/dc0/d5a/lcd 0 2024-05-27T09:29:39.324 INFO:tasks.workunit.client.1.smithi138.stdout:5/356: dwrite d3/d12/d32/f4f [0,4194304] 0 2024-05-27T09:29:39.324 INFO:tasks.workunit.client.1.smithi138.stdout:5/357: write d3/d12/d16/d1d/f55 [42122,98290] 0 2024-05-27T09:29:39.325 INFO:tasks.workunit.client.1.smithi138.stdout:5/358: stat d3/d12/d26/f45 0 2024-05-27T09:29:39.325 INFO:tasks.workunit.client.0.smithi119.stdout:8/933: dwrite d3/d8/d1b/d46/da4/dbb/ff8 [0,4194304] 0 2024-05-27T09:29:39.327 INFO:tasks.workunit.client.0.smithi119.stdout:8/934: creat d3/d8/d1b/d46/f134 x:0 0 0 2024-05-27T09:29:39.333 INFO:tasks.workunit.client.1.smithi138.stdout:6/567: dread d3/d11/dc0/d43/d1c/d32/d47/f51 [0,4194304] 0 2024-05-27T09:29:39.333 INFO:tasks.workunit.client.1.smithi138.stdout:6/568: readlink d3/d11/l62 0 2024-05-27T09:29:39.341 INFO:tasks.workunit.client.0.smithi119.stdout:8/935: rmdir d3/d8/d1b/d46/da4/d113/db7/de0 39 2024-05-27T09:29:39.341 INFO:tasks.workunit.client.0.smithi119.stdout:8/936: fsync d3/de/f109 0 2024-05-27T09:29:39.344 INFO:tasks.workunit.client.0.smithi119.stdout:0/940: dwrite d2/d110/d96/d132/f7d [0,4194304] 0 2024-05-27T09:29:39.345 INFO:tasks.workunit.client.0.smithi119.stdout:0/941: fdatasync d2/d6/d9/d12/d4f/f5a 0 2024-05-27T09:29:39.345 INFO:tasks.workunit.client.0.smithi119.stdout:0/942: chown d2/d110/d96/d132/dd7/d105/l131 780777 1 2024-05-27T09:29:39.345 INFO:tasks.workunit.client.0.smithi119.stdout:0/943: write d2/f11e [548954,31976] 0 2024-05-27T09:29:39.345 INFO:tasks.workunit.client.0.smithi119.stdout:8/937: rename d3/d8/d1b/d46/ld1 to d3/l135 0 2024-05-27T09:29:39.345 INFO:tasks.workunit.client.0.smithi119.stdout:8/938: write d3/d8/d1b/d46/da4/d113/d66/f119 [84127,70688] 0 2024-05-27T09:29:39.350 INFO:tasks.workunit.client.0.smithi119.stdout:8/939: mkdir d3/d8/d1b/db1/d136 0 2024-05-27T09:29:39.351 INFO:tasks.workunit.client.0.smithi119.stdout:8/940: symlink d3/d8/d1b/d46/da4/d113/l137 0 2024-05-27T09:29:39.351 INFO:tasks.workunit.client.0.smithi119.stdout:8/941: chown d3/d8/d1b/d46/da4/d113/db7/fee 16984 1 2024-05-27T09:29:39.352 INFO:tasks.workunit.client.0.smithi119.stdout:8/942: chown d3/d8/d4d/d58/f63 1810786 1 2024-05-27T09:29:39.352 INFO:tasks.workunit.client.0.smithi119.stdout:8/943: chown d3/d39 1955059 1 2024-05-27T09:29:39.353 INFO:tasks.workunit.client.0.smithi119.stdout:8/944: rename d3/d8/d4d/ccd to d3/d6e/d8e/c138 0 2024-05-27T09:29:39.363 INFO:tasks.workunit.client.1.smithi138.stdout:3/463: dwrite d0/d80/f92 [0,4194304] 0 2024-05-27T09:29:39.364 INFO:tasks.workunit.client.1.smithi138.stdout:5/359: dread d3/d12/d16/d1d/f24 [0,4194304] 0 2024-05-27T09:29:39.365 INFO:tasks.workunit.client.0.smithi119.stdout:2/947: dwrite d2/da/d15/fd7 [0,4194304] 0 2024-05-27T09:29:39.365 INFO:tasks.workunit.client.0.smithi119.stdout:2/948: chown d2/d1a/c43 536590860 1 2024-05-27T09:29:39.365 INFO:tasks.workunit.client.0.smithi119.stdout:2/949: truncate d2/da/d15/f2b 384818 0 2024-05-27T09:29:39.366 INFO:tasks.workunit.client.1.smithi138.stdout:8/430: dwrite d3/d8/d32/f81 [0,4194304] 0 2024-05-27T09:29:39.369 INFO:tasks.workunit.client.1.smithi138.stdout:8/431: symlink d3/d60/l84 0 2024-05-27T09:29:39.373 INFO:tasks.workunit.client.1.smithi138.stdout:8/432: truncate d3/d8/d32/d45/f4e 2298202 0 2024-05-27T09:29:39.387 INFO:tasks.workunit.client.1.smithi138.stdout:6/569: dwrite d3/d11/dc0/d43/f48 [4194304,4194304] 0 2024-05-27T09:29:39.387 INFO:tasks.workunit.client.1.smithi138.stdout:6/570: dread - d3/d11/dc0/d43/d8c/f96 zero size 2024-05-27T09:29:39.389 INFO:tasks.workunit.client.1.smithi138.stdout:6/571: link d3/d11/dc0/d43/d1c/d32/ca3 d3/d11/dc0/d43/d1c/d32/d9b/cce 0 2024-05-27T09:29:39.389 INFO:tasks.workunit.client.1.smithi138.stdout:6/572: chown d3/d11/dc0/d43/d1c/f59 49569979 1 2024-05-27T09:29:39.391 INFO:tasks.workunit.client.1.smithi138.stdout:6/573: mkdir d3/d11/dc0/d5a/dcf 0 2024-05-27T09:29:39.392 INFO:tasks.workunit.client.1.smithi138.stdout:6/574: write d3/d19/f22 [5120636,76931] 0 2024-05-27T09:29:39.394 INFO:tasks.workunit.client.0.smithi119.stdout:6/903: sync 2024-05-27T09:29:39.394 INFO:tasks.workunit.client.0.smithi119.stdout:1/981: sync 2024-05-27T09:29:39.394 INFO:tasks.workunit.client.0.smithi119.stdout:9/839: sync 2024-05-27T09:29:39.394 INFO:tasks.workunit.client.0.smithi119.stdout:2/950: read d2/da/d8a/f9f [3228054,37055] 0 2024-05-27T09:29:39.395 INFO:tasks.workunit.client.1.smithi138.stdout:6/575: creat d3/d11/fd0 x:0 0 0 2024-05-27T09:29:39.395 INFO:tasks.workunit.client.0.smithi119.stdout:1/982: mkdir d7/db/d12/ddb/d12d 0 2024-05-27T09:29:39.396 INFO:tasks.workunit.client.0.smithi119.stdout:6/904: creat d0/d2/d4e/f144 x:0 0 0 2024-05-27T09:29:39.397 INFO:tasks.workunit.client.0.smithi119.stdout:6/905: creat d0/d2/d57/d37/d74/df1/f145 x:0 0 0 2024-05-27T09:29:39.397 INFO:tasks.workunit.client.0.smithi119.stdout:6/906: truncate d0/d2/d1c/d21/ffd 729655 0 2024-05-27T09:29:39.398 INFO:tasks.workunit.client.1.smithi138.stdout:6/576: unlink d3/d11/dc0/d43/d1c/c23 0 2024-05-27T09:29:39.401 INFO:tasks.workunit.client.0.smithi119.stdout:2/951: write d2/da/fd [346491,17121] 0 2024-05-27T09:29:39.426 INFO:tasks.workunit.client.1.smithi138.stdout:6/577: dread d3/d19/f1e [0,4194304] 0 2024-05-27T09:29:39.428 INFO:tasks.workunit.client.1.smithi138.stdout:6/578: truncate d3/d11/dc0/f97 3947682 0 2024-05-27T09:29:39.428 INFO:tasks.workunit.client.1.smithi138.stdout:6/579: chown d3/d19 1 1 2024-05-27T09:29:39.444 INFO:tasks.workunit.client.1.smithi138.stdout:5/360: dwrite d3/f21 [0,4194304] 0 2024-05-27T09:29:39.458 INFO:tasks.workunit.client.1.smithi138.stdout:5/361: dread d3/d12/d16/d1d/d22/d2b/d36/f41 [0,4194304] 0 2024-05-27T09:29:39.459 INFO:tasks.workunit.client.1.smithi138.stdout:5/362: write d3/f1b [166369,101575] 0 2024-05-27T09:29:39.459 INFO:tasks.workunit.client.1.smithi138.stdout:5/363: chown d3/f10 3107142 1 2024-05-27T09:29:39.461 INFO:tasks.workunit.client.1.smithi138.stdout:5/364: mknod d3/d12/d16/d1d/d22/d2b/d36/c70 0 2024-05-27T09:29:39.462 INFO:tasks.workunit.client.1.smithi138.stdout:5/365: symlink d3/d12/d16/d1d/d23/l71 0 2024-05-27T09:29:39.463 INFO:tasks.workunit.client.1.smithi138.stdout:5/366: dread - d3/d12/d16/d1d/d22/d2b/f38 zero size 2024-05-27T09:29:39.463 INFO:tasks.workunit.client.1.smithi138.stdout:5/367: rename d3/d12/d16/d1d/d23 to d3/d12/d16/d1d/d23/d3b/d6f/d72 22 2024-05-27T09:29:39.464 INFO:tasks.workunit.client.0.smithi119.stdout:1/983: dwrite d7/dd/d44/f4e [0,4194304] 0 2024-05-27T09:29:39.464 INFO:tasks.workunit.client.0.smithi119.stdout:1/984: chown d7/db/d12/ddb/d38/d3c/l7f 57443592 1 2024-05-27T09:29:39.464 INFO:tasks.workunit.client.0.smithi119.stdout:1/985: chown d7/db/d12/ddb/d38/f107 29 1 2024-05-27T09:29:39.464 INFO:tasks.workunit.client.0.smithi119.stdout:1/986: chown d7/db 17632 1 2024-05-27T09:29:39.467 INFO:tasks.workunit.client.0.smithi119.stdout:1/987: symlink d7/db6/l12e 0 2024-05-27T09:29:39.468 INFO:tasks.workunit.client.1.smithi138.stdout:8/433: dwrite d3/d8/d64/f2f [0,4194304] 0 2024-05-27T09:29:39.470 INFO:tasks.workunit.client.0.smithi119.stdout:1/988: rename d7/dd/d25/d60/f123 to d7/dd/d36/d11f/f12f 0 2024-05-27T09:29:39.472 INFO:tasks.workunit.client.0.smithi119.stdout:1/989: link d7/dd/d44/faf d7/f130 0 2024-05-27T09:29:39.472 INFO:tasks.workunit.client.1.smithi138.stdout:8/434: unlink d3/d8/d32/d34/d38/f5d 0 2024-05-27T09:29:39.473 INFO:tasks.workunit.client.0.smithi119.stdout:2/952: dwrite d2/da/d8a/dd4/d89/d94/f133 [0,4194304] 0 2024-05-27T09:29:39.473 INFO:tasks.workunit.client.0.smithi119.stdout:8/945: dwrite d3/d8/d1b/d46/f2b [0,4194304] 0 2024-05-27T09:29:39.475 INFO:tasks.workunit.client.0.smithi119.stdout:2/953: mkdir d2/da/d8a/d8f/d11e/d146 0 2024-05-27T09:29:39.475 INFO:tasks.workunit.client.0.smithi119.stdout:2/954: write d2/f19 [518537,4867] 0 2024-05-27T09:29:39.475 INFO:tasks.workunit.client.0.smithi119.stdout:2/955: fdatasync d2/da/d8a/dd4/d89/f8d 0 2024-05-27T09:29:39.475 INFO:tasks.workunit.client.0.smithi119.stdout:9/840: dwrite d18/d19/d1b/d40/d44/f54 [4194304,4194304] 0 2024-05-27T09:29:39.476 INFO:tasks.workunit.client.1.smithi138.stdout:3/464: dwrite d0/d80/f92 [0,4194304] 0 2024-05-27T09:29:39.479 INFO:tasks.workunit.client.1.smithi138.stdout:3/465: creat d0/d2/da/df/d10/d2a/d61/f99 x:0 0 0 2024-05-27T09:29:39.479 INFO:tasks.workunit.client.1.smithi138.stdout:3/466: chown d0/d2/da/df/d10/d32/f9 59519 1 2024-05-27T09:29:39.479 INFO:tasks.workunit.client.0.smithi119.stdout:8/946: rmdir d3/d8/d1b/d46/da4/d113/d66/d85 39 2024-05-27T09:29:39.488 INFO:tasks.workunit.client.1.smithi138.stdout:3/467: dread d0/d3f/d50/f7c [0,4194304] 0 2024-05-27T09:29:39.494 INFO:tasks.workunit.client.1.smithi138.stdout:8/435: read f1 [2533908,119843] 0 2024-05-27T09:29:39.496 INFO:tasks.workunit.client.1.smithi138.stdout:8/436: rename d3/d8/d24/d55/l57 to d3/d8/d24/d55/l85 0 2024-05-27T09:29:39.496 INFO:tasks.workunit.client.0.smithi119.stdout:1/990: rename d7/d40 to d7/dd/d131 0 2024-05-27T09:29:39.497 INFO:tasks.workunit.client.1.smithi138.stdout:8/437: mkdir d3/d60/d86 0 2024-05-27T09:29:39.503 INFO:tasks.workunit.client.0.smithi119.stdout:1/991: creat d7/db/d52/f132 x:0 0 0 2024-05-27T09:29:39.503 INFO:tasks.workunit.client.0.smithi119.stdout:1/992: stat d7/d6e/dd0 0 2024-05-27T09:29:39.504 INFO:tasks.workunit.client.1.smithi138.stdout:6/580: dwrite d3/d11/dc0/d5a/d6a/f6b [0,4194304] 0 2024-05-27T09:29:39.507 INFO:tasks.workunit.client.0.smithi119.stdout:1/993: unlink d7/dd/d44/fc8 0 2024-05-27T09:29:39.507 INFO:tasks.workunit.client.0.smithi119.stdout:1/994: write d7/db/d12/d15/d1f/fed [890699,194] 0 2024-05-27T09:29:39.507 INFO:tasks.workunit.client.0.smithi119.stdout:1/995: dread - d7/f130 zero size 2024-05-27T09:29:39.513 INFO:tasks.workunit.client.0.smithi119.stdout:1/996: mknod d7/dd/d44/d47/c133 0 2024-05-27T09:29:39.513 INFO:tasks.workunit.client.0.smithi119.stdout:1/997: chown d7/db/d8f/d9b/dc9/ddc/c111 87416 1 2024-05-27T09:29:39.516 INFO:tasks.workunit.client.1.smithi138.stdout:6/581: rename d3/d11/dc0/d43/d1c/d32 to d3/d11/d2d/d37/dd1 0 2024-05-27T09:29:39.525 INFO:tasks.workunit.client.1.smithi138.stdout:6/582: symlink d3/d11/d2d/d37/d7e/ld2 0 2024-05-27T09:29:39.526 INFO:tasks.workunit.client.1.smithi138.stdout:3/468: dwrite d0/d2/da/df/d10/d2a/f56 [0,4194304] 0 2024-05-27T09:29:39.526 INFO:tasks.workunit.client.1.smithi138.stdout:6/583: creat d3/d11/dc0/fd3 x:0 0 0 2024-05-27T09:29:39.526 INFO:tasks.workunit.client.0.smithi119.stdout:1/998: creat d7/db/d8f/d9b/f134 x:0 0 0 2024-05-27T09:29:39.526 INFO:tasks.workunit.client.0.smithi119.stdout:1/999: chown d7/db/d52/l73 2078526436 1 2024-05-27T09:29:39.528 INFO:tasks.workunit.client.1.smithi138.stdout:3/469: rename d0/d3f/c43 to d0/d2/da/df/d10/d2a/d48/c9a 0 2024-05-27T09:29:39.529 INFO:tasks.workunit.client.0.smithi119.stdout:6/907: dwrite d0/d2/f7f [0,4194304] 0 2024-05-27T09:29:39.529 INFO:tasks.workunit.client.0.smithi119.stdout:6/908: fdatasync d0/d2/f133 0 2024-05-27T09:29:39.530 INFO:tasks.workunit.client.1.smithi138.stdout:3/470: mkdir d0/d3f/d50/d9b 0 2024-05-27T09:29:39.535 INFO:tasks.workunit.client.1.smithi138.stdout:3/471: rename d0/d2/l58 to d0/d3f/d50/l9c 0 2024-05-27T09:29:39.535 INFO:tasks.workunit.client.1.smithi138.stdout:3/472: chown d0/d2/da/df/d10/d2a/d48/f8c 0 1 2024-05-27T09:29:39.537 INFO:tasks.workunit.client.1.smithi138.stdout:3/473: link d0/d3f/d60/f96 d0/d2/da/df/d10/f9d 0 2024-05-27T09:29:39.542 INFO:tasks.workunit.client.0.smithi119.stdout:6/909: rmdir d0/d2/d57/d37/d74/d65 39 2024-05-27T09:29:39.543 INFO:tasks.workunit.client.0.smithi119.stdout:6/910: mkdir d0/d2/d4e/dc5/d146 0 2024-05-27T09:29:39.545 INFO:tasks.workunit.client.0.smithi119.stdout:6/911: mkdir d0/d2/d57/d37/de5/d147 0 2024-05-27T09:29:39.545 INFO:tasks.workunit.client.0.smithi119.stdout:8/947: dwrite d3/d8/d1b/d46/da4/d113/db7/de0/f12e [0,4194304] 0 2024-05-27T09:29:39.551 INFO:tasks.workunit.client.0.smithi119.stdout:9/841: dwrite d18/d19/d1b/d29/fa7 [0,4194304] 0 2024-05-27T09:29:39.552 INFO:tasks.workunit.client.0.smithi119.stdout:6/912: mkdir d0/d2/d57/d37/dad/db6/d148 0 2024-05-27T09:29:39.552 INFO:tasks.workunit.client.1.smithi138.stdout:6/584: dread d3/d11/d2d/d37/f3c [0,4194304] 0 2024-05-27T09:29:39.552 INFO:tasks.workunit.client.0.smithi119.stdout:2/956: dwrite d2/f112 [0,4194304] 0 2024-05-27T09:29:39.552 INFO:tasks.workunit.client.0.smithi119.stdout:2/957: fsync d2/da/d6f/fa5 0 2024-05-27T09:29:39.555 INFO:tasks.workunit.client.1.smithi138.stdout:8/438: dwrite f1 [0,4194304] 0 2024-05-27T09:29:39.556 INFO:tasks.workunit.client.0.smithi119.stdout:0/944: sync 2024-05-27T09:29:39.556 INFO:tasks.workunit.client.0.smithi119.stdout:0/945: fdatasync d2/d110/d96/fad 0 2024-05-27T09:29:39.558 INFO:tasks.workunit.client.0.smithi119.stdout:9/842: rename d18/d19/d1b/d2a/d9f/fd8 to d18/d19/d1b/d40/d44/f107 0 2024-05-27T09:29:39.558 INFO:tasks.workunit.client.0.smithi119.stdout:9/843: chown d18/d19/d1b/d2a/c3b 14502 1 2024-05-27T09:29:39.558 INFO:tasks.workunit.client.0.smithi119.stdout:9/844: write d18/d19/d1b/d40/db7/fb8 [1847183,104566] 0 2024-05-27T09:29:39.561 INFO:tasks.workunit.client.0.smithi119.stdout:2/958: rename d2/da/d6f/f7f to d2/da/d15/d53/dac/d124/f147 0 2024-05-27T09:29:39.561 INFO:tasks.workunit.client.0.smithi119.stdout:2/959: write d2/da/d6f/f111 [4623394,117320] 0 2024-05-27T09:29:39.561 INFO:tasks.workunit.client.0.smithi119.stdout:2/960: chown d2/da/lfe 3 1 2024-05-27T09:29:39.561 INFO:tasks.workunit.client.0.smithi119.stdout:2/961: chown d2/da/d15/d53/dac/lb8 36 1 2024-05-27T09:29:39.565 INFO:tasks.workunit.client.0.smithi119.stdout:9/845: rmdir d18/db4/dbe 39 2024-05-27T09:29:39.572 INFO:tasks.workunit.client.0.smithi119.stdout:9/846: dread d18/f76 [0,4194304] 0 2024-05-27T09:29:39.580 INFO:tasks.workunit.client.0.smithi119.stdout:0/946: write d2/d6/d9/d24/d69/fe1 [6072696,21464] 0 2024-05-27T09:29:39.583 INFO:tasks.workunit.client.0.smithi119.stdout:0/947: mknod d2/d6/d23/d7a/d116/c13e 0 2024-05-27T09:29:39.584 INFO:tasks.workunit.client.0.smithi119.stdout:0/948: rmdir d2/d110/d96 39 2024-05-27T09:29:39.584 INFO:tasks.workunit.client.0.smithi119.stdout:0/949: readlink d2/d6/d9/lc4 0 2024-05-27T09:29:39.585 INFO:tasks.workunit.client.0.smithi119.stdout:0/950: rmdir d2/d6/d9/d12 39 2024-05-27T09:29:39.586 INFO:tasks.workunit.client.1.smithi138.stdout:9/434: sync 2024-05-27T09:29:39.586 INFO:tasks.workunit.client.1.smithi138.stdout:1/440: sync 2024-05-27T09:29:39.586 INFO:tasks.workunit.client.1.smithi138.stdout:0/415: sync 2024-05-27T09:29:39.586 INFO:tasks.workunit.client.1.smithi138.stdout:0/416: chown da/d17/f31 10023 1 2024-05-27T09:29:39.587 INFO:tasks.workunit.client.0.smithi119.stdout:0/951: mknod d2/d6/d9/d2f/d95/c13f 0 2024-05-27T09:29:39.587 INFO:tasks.workunit.client.0.smithi119.stdout:0/952: truncate d2/d110/d96/d132/dd7/ffd 394749 0 2024-05-27T09:29:39.587 INFO:tasks.workunit.client.1.smithi138.stdout:1/441: mkdir d4/d3c/d8a 0 2024-05-27T09:29:39.590 INFO:tasks.workunit.client.1.smithi138.stdout:0/417: link da/d17/d1d/d1f/d42/d4a/l82 da/d17/d1d/d1f/d42/d4a/l8b 0 2024-05-27T09:29:39.591 INFO:tasks.workunit.client.0.smithi119.stdout:0/953: creat d2/d6/d9/d24/d30/d4a/f140 x:0 0 0 2024-05-27T09:29:39.591 INFO:tasks.workunit.client.0.smithi119.stdout:0/954: write d2/d6/d23/f33 [1715842,8400] 0 2024-05-27T09:29:39.593 INFO:tasks.workunit.client.1.smithi138.stdout:1/442: creat d4/d48/d39/d5f/d4d/f8b x:0 0 0 2024-05-27T09:29:39.593 INFO:tasks.workunit.client.1.smithi138.stdout:1/443: stat d4/f37 0 2024-05-27T09:29:39.593 INFO:tasks.workunit.client.1.smithi138.stdout:1/444: stat d4/d38/f44 0 2024-05-27T09:29:39.593 INFO:tasks.workunit.client.0.smithi119.stdout:8/948: dwrite d3/d39/f88 [0,4194304] 0 2024-05-27T09:29:39.601 INFO:tasks.workunit.client.1.smithi138.stdout:3/474: dwrite d0/d2/da/df/d10/d2a/f87 [0,4194304] 0 2024-05-27T09:29:39.617 INFO:tasks.workunit.client.0.smithi119.stdout:0/955: dread d2/d6/d9/fbd [0,4194304] 0 2024-05-27T09:29:39.617 INFO:tasks.workunit.client.0.smithi119.stdout:0/956: stat d2/d110/fa7 0 2024-05-27T09:29:39.619 INFO:tasks.workunit.client.0.smithi119.stdout:0/957: rename d2/d6/d9/d12/cfb to d2/d6/d9/d24/d30/dcc/d109/c141 0 2024-05-27T09:29:39.623 INFO:tasks.workunit.client.0.smithi119.stdout:8/949: read d3/d8/d4d/d58/ffc [2333478,119816] 0 2024-05-27T09:29:39.624 INFO:tasks.workunit.client.1.smithi138.stdout:8/439: dwrite d3/d8/d32/f49 [4194304,4194304] 0 2024-05-27T09:29:39.625 INFO:tasks.workunit.client.0.smithi119.stdout:6/913: dwrite d0/d2/d57/d37/d74/df1/d109/db2/fea [0,4194304] 0 2024-05-27T09:29:39.625 INFO:tasks.workunit.client.0.smithi119.stdout:8/950: truncate d3/d8/d1b/d46/f7c 904194 0 2024-05-27T09:29:39.625 INFO:tasks.workunit.client.0.smithi119.stdout:8/951: truncate d3/d8/d4d/d58/f63 1398628 0 2024-05-27T09:29:39.627 INFO:tasks.workunit.client.0.smithi119.stdout:0/958: dread d2/d6/d9/d24/fc8 [0,4194304] 0 2024-05-27T09:29:39.627 INFO:tasks.workunit.client.1.smithi138.stdout:8/440: mknod d3/d8/d64/c87 0 2024-05-27T09:29:39.627 INFO:tasks.workunit.client.1.smithi138.stdout:8/441: truncate d3/d8/d24/d4d/f5a 478529 0 2024-05-27T09:29:39.628 INFO:tasks.workunit.client.1.smithi138.stdout:8/442: mkdir d3/d8/d32/d88 0 2024-05-27T09:29:39.630 INFO:tasks.workunit.client.1.smithi138.stdout:8/443: link d3/d8/d24/f44 d3/d8/d24/d55/d7e/f89 0 2024-05-27T09:29:39.630 INFO:tasks.workunit.client.0.smithi119.stdout:0/959: rename d2/d110/d96/lba to d2/d110/d96/d132/dd7/l142 0 2024-05-27T09:29:39.632 INFO:tasks.workunit.client.1.smithi138.stdout:8/444: symlink d3/d8/l8a 0 2024-05-27T09:29:39.632 INFO:tasks.workunit.client.1.smithi138.stdout:8/445: chown d3/d8/d64/c5b 336215802 1 2024-05-27T09:29:39.632 INFO:tasks.workunit.client.1.smithi138.stdout:8/446: creat d3/d60/d86/f8b x:0 0 0 2024-05-27T09:29:39.634 INFO:tasks.workunit.client.0.smithi119.stdout:6/914: getdents d0/d2/d1c/d21/d58/dd3 0 2024-05-27T09:29:39.634 INFO:tasks.workunit.client.0.smithi119.stdout:8/952: getdents d3/d8/d1b/db1 0 2024-05-27T09:29:39.634 INFO:tasks.workunit.client.0.smithi119.stdout:8/953: readlink d3/d39/d5a/l93 0 2024-05-27T09:29:39.635 INFO:tasks.workunit.client.1.smithi138.stdout:8/447: unlink d3/l1d 0 2024-05-27T09:29:39.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:39 smithi138 ceph-mon[34885]: Adjusting osd_memory_target on smithi119 to 1216M 2024-05-27T09:29:39.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:39 smithi138 ceph-mon[34885]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:29:39.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:39 smithi138 ceph-mon[34885]: pgmap v17: 65 pgs: 65 active+clean; 2.8 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 230 MiB/s rd, 285 MiB/s wr, 416 op/s 2024-05-27T09:29:39.636 INFO:tasks.workunit.client.0.smithi119.stdout:8/954: symlink d3/d8/d1b/d46/d92/d111/l139 0 2024-05-27T09:29:39.637 INFO:tasks.workunit.client.0.smithi119.stdout:8/955: symlink d3/d6e/dcf/l13a 0 2024-05-27T09:29:39.638 INFO:tasks.workunit.client.0.smithi119.stdout:8/956: read d3/d8/d1b/f6d [179790,42588] 0 2024-05-27T09:29:39.648 INFO:tasks.workunit.client.1.smithi138.stdout:6/585: dwrite d3/d19/f83 [0,4194304] 0 2024-05-27T09:29:39.648 INFO:tasks.workunit.client.1.smithi138.stdout:6/586: read d3/d11/fe [2538969,42429] 0 2024-05-27T09:29:39.648 INFO:tasks.workunit.client.1.smithi138.stdout:6/587: dread - d3/d11/d2d/d37/d7e/fb0 zero size 2024-05-27T09:29:39.648 INFO:tasks.workunit.client.1.smithi138.stdout:6/588: write d3/d19/f89 [417352,35052] 0 2024-05-27T09:29:39.656 INFO:tasks.workunit.client.1.smithi138.stdout:6/589: mknod d3/d11/dc0/d74/cd4 0 2024-05-27T09:29:39.664 INFO:tasks.workunit.client.1.smithi138.stdout:6/590: dread d3/d19/fab [0,4194304] 0 2024-05-27T09:29:39.666 INFO:tasks.workunit.client.1.smithi138.stdout:6/591: symlink d3/d11/dc0/d5a/dcf/ld5 0 2024-05-27T09:29:39.666 INFO:tasks.workunit.client.1.smithi138.stdout:6/592: stat d3/d11/dc0/fd3 0 2024-05-27T09:29:39.668 INFO:tasks.workunit.client.1.smithi138.stdout:6/593: creat d3/d11/d2d/d37/d7e/fd6 x:0 0 0 2024-05-27T09:29:39.668 INFO:tasks.workunit.client.1.smithi138.stdout:6/594: chown d3/d19/c2f 254301 1 2024-05-27T09:29:39.672 INFO:tasks.workunit.client.1.smithi138.stdout:0/418: dread da/f10 [0,4194304] 0 2024-05-27T09:29:39.674 INFO:tasks.workunit.client.1.smithi138.stdout:0/419: truncate da/d17/d1d/d1f/d42/d4a/f4e 2120937 0 2024-05-27T09:29:39.674 INFO:tasks.workunit.client.1.smithi138.stdout:0/420: read da/f22 [3269430,38530] 0 2024-05-27T09:29:39.675 INFO:tasks.workunit.client.0.smithi119.stdout:0/960: dread d2/d6/d9/d12/f16 [0,4194304] 0 2024-05-27T09:29:39.676 INFO:tasks.workunit.client.1.smithi138.stdout:0/421: creat da/d17/f8c x:0 0 0 2024-05-27T09:29:39.678 INFO:tasks.workunit.client.1.smithi138.stdout:0/422: getdents da/d2a/d55 0 2024-05-27T09:29:39.678 INFO:tasks.workunit.client.1.smithi138.stdout:0/423: chown da/c14 4 1 2024-05-27T09:29:39.680 INFO:tasks.workunit.client.1.smithi138.stdout:9/435: dwrite d6/da/d29/d59/f70 [0,4194304] 0 2024-05-27T09:29:39.680 INFO:tasks.workunit.client.1.smithi138.stdout:3/475: dread d0/d3f/d50/f63 [0,4194304] 0 2024-05-27T09:29:39.681 INFO:tasks.workunit.client.1.smithi138.stdout:0/424: symlink da/l8d 0 2024-05-27T09:29:39.681 INFO:tasks.workunit.client.1.smithi138.stdout:9/436: creat d6/da/d29/f89 x:0 0 0 2024-05-27T09:29:39.681 INFO:tasks.workunit.client.1.smithi138.stdout:9/437: creat d6/da/d2f/d35/d55/f8a x:0 0 0 2024-05-27T09:29:39.684 INFO:tasks.workunit.client.1.smithi138.stdout:9/438: link d6/db/dc/f1e d6/da/d2f/f8b 0 2024-05-27T09:29:39.685 INFO:tasks.workunit.client.1.smithi138.stdout:9/439: mkdir d6/da/d2f/d35/d55/d75/d8c 0 2024-05-27T09:29:39.692 INFO:tasks.workunit.client.1.smithi138.stdout:8/448: dwrite d3/d8/d32/d34/d38/f63 [0,4194304] 0 2024-05-27T09:29:39.699 INFO:tasks.workunit.client.1.smithi138.stdout:3/476: dread d0/f12 [0,4194304] 0 2024-05-27T09:29:39.699 INFO:tasks.workunit.client.1.smithi138.stdout:8/449: mknod d3/d8/c8c 0 2024-05-27T09:29:39.700 INFO:tasks.workunit.client.1.smithi138.stdout:3/477: mknod d0/d3f/d50/c9e 0 2024-05-27T09:29:39.704 INFO:tasks.workunit.client.0.smithi119.stdout:8/957: dwrite d3/d8/d4d/f67 [0,4194304] 0 2024-05-27T09:29:39.704 INFO:tasks.workunit.client.0.smithi119.stdout:8/958: read d3/d8/d1b/f6d [69116,101480] 0 2024-05-27T09:29:39.704 INFO:tasks.workunit.client.0.smithi119.stdout:8/959: dread - d3/d8/d1b/d77/d95/fae zero size 2024-05-27T09:29:39.705 INFO:tasks.workunit.client.1.smithi138.stdout:6/595: write d3/d19/f63 [1194309,73839] 0 2024-05-27T09:29:39.705 INFO:tasks.workunit.client.1.smithi138.stdout:8/450: unlink d3/d8/d32/d45/d5c/f7a 0 2024-05-27T09:29:39.708 INFO:tasks.workunit.client.1.smithi138.stdout:0/425: rename da/d17/d1d to da/d2a/d8e 0 2024-05-27T09:29:39.708 INFO:tasks.workunit.client.1.smithi138.stdout:0/426: creat da/d2a/f8f x:0 0 0 2024-05-27T09:29:39.725 INFO:tasks.workunit.client.1.smithi138.stdout:1/445: dwrite d4/d3c/d51/f55 [0,4194304] 0 2024-05-27T09:29:39.738 INFO:tasks.workunit.client.0.smithi119.stdout:0/961: dwrite d2/d6/d9/f2c [0,4194304] 0 2024-05-27T09:29:39.739 INFO:tasks.workunit.client.0.smithi119.stdout:6/915: dwrite d0/d85/fae [0,4194304] 0 2024-05-27T09:29:39.739 INFO:tasks.workunit.client.0.smithi119.stdout:8/960: dread d3/d8/f7e [0,4194304] 0 2024-05-27T09:29:39.739 INFO:tasks.workunit.client.0.smithi119.stdout:8/961: fdatasync d3/d8/d1b/d46/da4/d113/d74/f98 0 2024-05-27T09:29:39.739 INFO:tasks.workunit.client.0.smithi119.stdout:8/962: write d3/d8/fd [244436,69189] 0 2024-05-27T09:29:39.747 INFO:tasks.workunit.client.0.smithi119.stdout:6/916: link d0/d2/d57/d37/d74/f70 d0/f149 0 2024-05-27T09:29:39.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:39 smithi119 ceph-mon[29082]: Adjusting osd_memory_target on smithi119 to 1216M 2024-05-27T09:29:39.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:39 smithi119 ceph-mon[29082]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:29:39.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:39 smithi119 ceph-mon[29082]: pgmap v17: 65 pgs: 65 active+clean; 2.8 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 230 MiB/s rd, 285 MiB/s wr, 416 op/s 2024-05-27T09:29:39.747 INFO:tasks.workunit.client.0.smithi119.stdout:6/917: write d0/d2/d4e/f144 [971677,102066] 0 2024-05-27T09:29:39.749 INFO:tasks.workunit.client.0.smithi119.stdout:8/963: rmdir d3/d6e/dcf/dfd 39 2024-05-27T09:29:39.757 INFO:tasks.workunit.client.1.smithi138.stdout:6/596: write d3/d11/dc0/d43/d1c/f4d [3288103,18051] 0 2024-05-27T09:29:39.757 INFO:tasks.workunit.client.1.smithi138.stdout:6/597: truncate d3/d11/dc0/d43/d8c/fa6 711874 0 2024-05-27T09:29:39.757 INFO:tasks.workunit.client.1.smithi138.stdout:6/598: chown d3/d11/dc0/d43/d8c/f96 11283 1 2024-05-27T09:29:39.759 INFO:tasks.workunit.client.1.smithi138.stdout:6/599: symlink d3/d11/dc0/d5a/db4/ld7 0 2024-05-27T09:29:39.763 INFO:tasks.workunit.client.1.smithi138.stdout:9/440: dwrite d6/da/d2f/f8b [0,4194304] 0 2024-05-27T09:29:39.764 INFO:tasks.workunit.client.1.smithi138.stdout:9/441: truncate d6/db/de/d18/f7b 1016993 0 2024-05-27T09:29:39.764 INFO:tasks.workunit.client.1.smithi138.stdout:8/451: dwrite d3/d8/d24/d4d/f61 [0,4194304] 0 2024-05-27T09:29:39.764 INFO:tasks.workunit.client.1.smithi138.stdout:8/452: stat d3/d8/d32 0 2024-05-27T09:29:39.765 INFO:tasks.workunit.client.0.smithi119.stdout:0/962: dread d2/d6/d9/d12/da3/fd3 [0,4194304] 0 2024-05-27T09:29:39.769 INFO:tasks.workunit.client.0.smithi119.stdout:2/962: sync 2024-05-27T09:29:39.769 INFO:tasks.workunit.client.0.smithi119.stdout:9/847: sync 2024-05-27T09:29:39.770 INFO:tasks.workunit.client.0.smithi119.stdout:2/963: creat d2/da/d2c/f148 x:0 0 0 2024-05-27T09:29:39.772 INFO:tasks.workunit.client.1.smithi138.stdout:8/453: dread d3/d8/f42 [0,4194304] 0 2024-05-27T09:29:39.772 INFO:tasks.workunit.client.0.smithi119.stdout:2/964: symlink d2/da/d8a/dd4/d69/df5/d108/l149 0 2024-05-27T09:29:39.774 INFO:tasks.workunit.client.0.smithi119.stdout:2/965: creat d2/da/d15/d53/d114/d116/d12a/f14a x:0 0 0 2024-05-27T09:29:39.775 INFO:tasks.workunit.client.1.smithi138.stdout:9/442: rmdir d6/da/d2f 39 2024-05-27T09:29:39.775 INFO:tasks.workunit.client.0.smithi119.stdout:8/964: write d3/d39/f88 [1170051,83635] 0 2024-05-27T09:29:39.777 INFO:tasks.workunit.client.1.smithi138.stdout:9/443: mkdir d6/db/d8d 0 2024-05-27T09:29:39.777 INFO:tasks.workunit.client.1.smithi138.stdout:9/444: write d6/db/f10 [1478506,93458] 0 2024-05-27T09:29:39.778 INFO:tasks.workunit.client.0.smithi119.stdout:8/965: unlink d3/d8/d1b/d46/da4/d113/fa3 0 2024-05-27T09:29:39.779 INFO:tasks.workunit.client.1.smithi138.stdout:9/445: creat d6/da/d2f/f8e x:0 0 0 2024-05-27T09:29:39.780 INFO:tasks.workunit.client.0.smithi119.stdout:8/966: getdents d3/d8/d1b 0 2024-05-27T09:29:39.782 INFO:tasks.workunit.client.1.smithi138.stdout:9/446: mkdir d6/da/d29/d8f 0 2024-05-27T09:29:39.782 INFO:tasks.workunit.client.1.smithi138.stdout:9/447: creat d6/da/d29/d59/f90 x:0 0 0 2024-05-27T09:29:39.783 INFO:tasks.workunit.client.1.smithi138.stdout:0/427: dwrite da/d2a/d8e/d1f/d42/d4a/d53/f57 [0,4194304] 0 2024-05-27T09:29:39.783 INFO:tasks.workunit.client.0.smithi119.stdout:8/967: creat d3/d6e/d80/f13b x:0 0 0 2024-05-27T09:29:39.784 INFO:tasks.workunit.client.1.smithi138.stdout:6/600: dread d3/d11/f2e [0,4194304] 0 2024-05-27T09:29:39.784 INFO:tasks.workunit.client.1.smithi138.stdout:4/510: sync 2024-05-27T09:29:39.784 INFO:tasks.workunit.client.1.smithi138.stdout:6/601: write d3/d19/f63 [128346,5590] 0 2024-05-27T09:29:39.784 INFO:tasks.workunit.client.1.smithi138.stdout:6/602: stat d3/d11/d2d/d37/daa 0 2024-05-27T09:29:39.784 INFO:tasks.workunit.client.1.smithi138.stdout:2/366: sync 2024-05-27T09:29:39.784 INFO:tasks.workunit.client.1.smithi138.stdout:2/367: readlink d1/d15/d2d/l20 0 2024-05-27T09:29:39.784 INFO:tasks.workunit.client.1.smithi138.stdout:4/511: creat d7/d27/d3a/fb0 x:0 0 0 2024-05-27T09:29:39.784 INFO:tasks.workunit.client.1.smithi138.stdout:9/448: symlink d6/db/de/d85/l91 0 2024-05-27T09:29:39.785 INFO:tasks.workunit.client.1.smithi138.stdout:5/368: sync 2024-05-27T09:29:39.785 INFO:tasks.workunit.client.1.smithi138.stdout:5/369: write d3/d12/d16/d1d/d22/d2b/d36/f43 [609006,2250] 0 2024-05-27T09:29:39.786 INFO:tasks.workunit.client.0.smithi119.stdout:8/968: stat d3/d8/d1b/d46/f7c 0 2024-05-27T09:29:39.787 INFO:tasks.workunit.client.1.smithi138.stdout:7/434: sync 2024-05-27T09:29:39.787 INFO:tasks.workunit.client.1.smithi138.stdout:0/428: readlink l0 0 2024-05-27T09:29:39.788 INFO:tasks.workunit.client.1.smithi138.stdout:7/435: getdents d0/d23/d49 0 2024-05-27T09:29:39.788 INFO:tasks.workunit.client.1.smithi138.stdout:7/436: stat d0/l2d 0 2024-05-27T09:29:39.791 INFO:tasks.workunit.client.0.smithi119.stdout:6/918: dwrite d0/d2/d1c/fd9 [0,4194304] 0 2024-05-27T09:29:39.791 INFO:tasks.workunit.client.0.smithi119.stdout:6/919: chown d0/d2/d57/d37/d74/df1/d109/ca4 13930730 1 2024-05-27T09:29:39.791 INFO:tasks.workunit.client.0.smithi119.stdout:6/920: fsync d0/d2/d4e/f1f 0 2024-05-27T09:29:39.792 INFO:tasks.workunit.client.0.smithi119.stdout:2/966: dread d2/da/d4e/fe3 [0,4194304] 0 2024-05-27T09:29:39.792 INFO:tasks.workunit.client.1.smithi138.stdout:4/512: mkdir d7/df/d5b/da1/db1 0 2024-05-27T09:29:39.792 INFO:tasks.workunit.client.1.smithi138.stdout:0/429: creat da/d2a/d8e/d1f/d42/d4a/d53/f90 x:0 0 0 2024-05-27T09:29:39.793 INFO:tasks.workunit.client.1.smithi138.stdout:4/513: mknod d7/df/d35/d4c/d59/d5d/d64/d6a/cb2 0 2024-05-27T09:29:39.793 INFO:tasks.workunit.client.1.smithi138.stdout:4/514: chown d7/df/d35/d4c/d59/d5d 32148 1 2024-05-27T09:29:39.793 INFO:tasks.workunit.client.0.smithi119.stdout:9/848: dread d18/d19/d1b/d29/f7c [0,4194304] 0 2024-05-27T09:29:39.795 INFO:tasks.workunit.client.0.smithi119.stdout:6/921: unlink d0/d2/d57/d37/d74/df1/d109/ca4 0 2024-05-27T09:29:39.795 INFO:tasks.workunit.client.1.smithi138.stdout:6/603: mkdir d3/d11/dc0/d5a/dcf/dd8 0 2024-05-27T09:29:39.795 INFO:tasks.workunit.client.1.smithi138.stdout:6/604: chown d3/d11/c93 14 1 2024-05-27T09:29:39.797 INFO:tasks.workunit.client.0.smithi119.stdout:8/969: dread d3/d8/d1b/d46/da4/d113/f107 [0,4194304] 0 2024-05-27T09:29:39.797 INFO:tasks.workunit.client.0.smithi119.stdout:2/967: rename d2/da/d2c/lda to d2/da/d8a/dd4/d69/df5/d108/l14b 0 2024-05-27T09:29:39.797 INFO:tasks.workunit.client.0.smithi119.stdout:2/968: stat d2/fa9 0 2024-05-27T09:29:39.801 INFO:tasks.workunit.client.0.smithi119.stdout:9/849: truncate d18/d19/d9b/fa0 3639345 0 2024-05-27T09:29:39.801 INFO:tasks.workunit.client.0.smithi119.stdout:9/850: fdatasync d18/d19/d61/f9d 0 2024-05-27T09:29:39.803 INFO:tasks.workunit.client.1.smithi138.stdout:4/515: unlink d7/df/d5b/f95 0 2024-05-27T09:29:39.804 INFO:tasks.workunit.client.1.smithi138.stdout:6/605: creat d3/d11/d2d/d37/daa/fd9 x:0 0 0 2024-05-27T09:29:39.807 INFO:tasks.workunit.client.1.smithi138.stdout:0/430: dread da/d2a/d8e/d1f/f38 [0,4194304] 0 2024-05-27T09:29:39.808 INFO:tasks.workunit.client.0.smithi119.stdout:2/969: creat d2/da/d8a/dd4/d89/d94/f14c x:0 0 0 2024-05-27T09:29:39.808 INFO:tasks.workunit.client.0.smithi119.stdout:2/970: fsync d2/da/fd 0 2024-05-27T09:29:39.808 INFO:tasks.workunit.client.0.smithi119.stdout:2/971: dread - d2/da/d15/d53/d114/d116/d12a/f14a zero size 2024-05-27T09:29:39.808 INFO:tasks.workunit.client.0.smithi119.stdout:9/851: mkdir d18/db4/d108 0 2024-05-27T09:29:39.809 INFO:tasks.workunit.client.0.smithi119.stdout:9/852: write d18/f8b [2017226,74032] 0 2024-05-27T09:29:39.809 INFO:tasks.workunit.client.0.smithi119.stdout:9/853: dread - d18/d19/d1b/d40/fc7 zero size 2024-05-27T09:29:39.810 INFO:tasks.workunit.client.1.smithi138.stdout:2/368: dread d1/d15/f48 [4194304,4194304] 0 2024-05-27T09:29:39.812 INFO:tasks.workunit.client.1.smithi138.stdout:2/369: mkdir d1/d15/d2d/d7b 0 2024-05-27T09:29:39.812 INFO:tasks.workunit.client.0.smithi119.stdout:2/972: truncate d2/da/d8a/dd4/f60 5993832 0 2024-05-27T09:29:39.814 INFO:tasks.workunit.client.1.smithi138.stdout:9/449: dread d6/da/d2f/f8b [0,4194304] 0 2024-05-27T09:29:39.815 INFO:tasks.workunit.client.1.smithi138.stdout:9/450: write d6/db/dc/f1c [76707,107508] 0 2024-05-27T09:29:39.815 INFO:tasks.workunit.client.1.smithi138.stdout:6/606: rename d3/cb5 to d3/d11/dc0/d43/d8c/cda 0 2024-05-27T09:29:39.820 INFO:tasks.workunit.client.1.smithi138.stdout:9/451: link d6/da/d29/f6d d6/db/de/d18/f92 0 2024-05-27T09:29:39.821 INFO:tasks.workunit.client.1.smithi138.stdout:5/370: write d3/d12/d26/f33 [4170494,65888] 0 2024-05-27T09:29:39.823 INFO:tasks.workunit.client.1.smithi138.stdout:9/452: unlink d6/db/dc/f1e 0 2024-05-27T09:29:39.823 INFO:tasks.workunit.client.1.smithi138.stdout:9/453: fsync f3 0 2024-05-27T09:29:39.824 INFO:tasks.workunit.client.0.smithi119.stdout:2/973: truncate d2/da/d8a/dd4/d89/d94/f133 1449712 0 2024-05-27T09:29:39.824 INFO:tasks.workunit.client.0.smithi119.stdout:8/970: fdatasync d3/d39/f88 0 2024-05-27T09:29:39.824 INFO:tasks.workunit.client.1.smithi138.stdout:6/607: symlink d3/d11/d2d/d37/dd1/ldb 0 2024-05-27T09:29:39.826 INFO:tasks.workunit.client.1.smithi138.stdout:5/371: rmdir d3/d12/d16/d1d/d23/d3b/d6f 0 2024-05-27T09:29:39.826 INFO:tasks.workunit.client.1.smithi138.stdout:5/372: write d3/fd [3155774,54502] 0 2024-05-27T09:29:39.826 INFO:tasks.workunit.client.1.smithi138.stdout:5/373: read d3/d12/d16/f39 [5148104,7171] 0 2024-05-27T09:29:39.826 INFO:tasks.workunit.client.1.smithi138.stdout:9/454: link d6/db/de/c19 d6/da/d26/d28/c93 0 2024-05-27T09:29:39.827 INFO:tasks.workunit.client.0.smithi119.stdout:2/974: mknod d2/da/d8a/d12d/d145/c14d 0 2024-05-27T09:29:39.831 INFO:tasks.workunit.client.0.smithi119.stdout:8/971: truncate d3/d8/d1b/db1/fbc 596656 0 2024-05-27T09:29:39.832 INFO:tasks.workunit.client.0.smithi119.stdout:9/854: write d18/d19/d1b/d40/d44/f4a [522146,89534] 0 2024-05-27T09:29:39.832 INFO:tasks.workunit.client.0.smithi119.stdout:9/855: creat d18/d19/d1b/d29/dc4/de1/f109 x:0 0 0 2024-05-27T09:29:39.832 INFO:tasks.workunit.client.0.smithi119.stdout:9/856: dread - d18/d19/d1b/d40/d44/fd3 zero size 2024-05-27T09:29:39.833 INFO:tasks.workunit.client.1.smithi138.stdout:4/516: dread d7/df/f4d [0,4194304] 0 2024-05-27T09:29:39.833 INFO:tasks.workunit.client.1.smithi138.stdout:4/517: fdatasync d7/d27/f98 0 2024-05-27T09:29:39.833 INFO:tasks.workunit.client.0.smithi119.stdout:9/857: creat d18/d19/d1b/d40/db7/f10a x:0 0 0 2024-05-27T09:29:39.836 INFO:tasks.workunit.client.1.smithi138.stdout:6/608: getdents d3/d11/d2d/d37/d8b 0 2024-05-27T09:29:39.837 INFO:tasks.workunit.client.1.smithi138.stdout:6/609: truncate d3/d11/dc0/f80 1031932 0 2024-05-27T09:29:39.838 INFO:tasks.workunit.client.0.smithi119.stdout:8/972: dread d3/d8/d1b/f6d [0,4194304] 0 2024-05-27T09:29:39.838 INFO:tasks.workunit.client.1.smithi138.stdout:0/431: dread da/d2a/d8e/d43/f48 [0,4194304] 0 2024-05-27T09:29:39.840 INFO:tasks.workunit.client.1.smithi138.stdout:5/374: getdents d3/d12/d16/d1d/d23/d68 0 2024-05-27T09:29:39.845 INFO:tasks.workunit.client.0.smithi119.stdout:9/858: mkdir d18/d19/d9b/d10b 0 2024-05-27T09:29:39.846 INFO:tasks.workunit.client.0.smithi119.stdout:9/859: read d18/db4/f102 [79670,2276] 0 2024-05-27T09:29:39.848 INFO:tasks.workunit.client.1.smithi138.stdout:0/432: creat da/d2a/d8e/d1f/d42/d63/f91 x:0 0 0 2024-05-27T09:29:39.848 INFO:tasks.workunit.client.1.smithi138.stdout:0/433: chown da/c3a 82424 1 2024-05-27T09:29:39.851 INFO:tasks.workunit.client.0.smithi119.stdout:8/973: mkdir d3/d8/d1b/d46/da4/dbb/d13c 0 2024-05-27T09:29:39.851 INFO:tasks.workunit.client.0.smithi119.stdout:8/974: creat d3/d8/d4d/d79/f13d x:0 0 0 2024-05-27T09:29:39.854 INFO:tasks.workunit.client.0.smithi119.stdout:0/963: dwrite d2/d6/d9/d12/da3/fd3 [0,4194304] 0 2024-05-27T09:29:39.854 INFO:tasks.workunit.client.0.smithi119.stdout:0/964: chown d2/d6/d9/d24/c38 7694633 1 2024-05-27T09:29:39.854 INFO:tasks.workunit.client.0.smithi119.stdout:0/965: chown d2/d6/d9/d24 322 1 2024-05-27T09:29:39.858 INFO:tasks.workunit.client.1.smithi138.stdout:0/434: mknod da/d2a/d8e/d1f/d42/c92 0 2024-05-27T09:29:39.863 INFO:tasks.workunit.client.0.smithi119.stdout:6/922: dwrite d0/d2/d57/d37/f12a [0,4194304] 0 2024-05-27T09:29:39.863 INFO:tasks.workunit.client.0.smithi119.stdout:6/923: read - d0/d2/d57/d37/d74/dfe/d10a/f10f zero size 2024-05-27T09:29:39.870 INFO:tasks.workunit.client.0.smithi119.stdout:6/924: truncate d0/d2/d4e/f104 651548 0 2024-05-27T09:29:39.870 INFO:tasks.workunit.client.0.smithi119.stdout:6/925: creat d0/d2/d57/d37/d74/dfe/d10a/f14a x:0 0 0 2024-05-27T09:29:39.870 INFO:tasks.workunit.client.0.smithi119.stdout:6/926: write d0/d2/fb1 [2102148,114648] 0 2024-05-27T09:29:39.873 INFO:tasks.workunit.client.0.smithi119.stdout:8/975: dread d3/d8/d4d/d58/f82 [0,4194304] 0 2024-05-27T09:29:39.874 INFO:tasks.workunit.client.0.smithi119.stdout:8/976: fsync d3/d8/d1b/d46/da4/dbb/f11b 0 2024-05-27T09:29:39.879 INFO:tasks.workunit.client.0.smithi119.stdout:8/977: symlink d3/d8/d1b/d46/da4/l13e 0 2024-05-27T09:29:39.885 INFO:tasks.workunit.client.0.smithi119.stdout:8/978: unlink d3/d8/c27 0 2024-05-27T09:29:39.888 INFO:tasks.workunit.client.0.smithi119.stdout:6/927: dread d0/d2/d57/d37/d74/df1/d109/f69 [0,4194304] 0 2024-05-27T09:29:39.888 INFO:tasks.workunit.client.0.smithi119.stdout:6/928: dread - d0/d11/f7d zero size 2024-05-27T09:29:39.889 INFO:tasks.workunit.client.1.smithi138.stdout:8/454: dwrite d3/d8/f33 [0,4194304] 0 2024-05-27T09:29:39.890 INFO:tasks.workunit.client.1.smithi138.stdout:8/455: fdatasync d3/d8/d64/f7c 0 2024-05-27T09:29:39.893 INFO:tasks.workunit.client.1.smithi138.stdout:2/370: dwrite d1/d15/d2d/d31/f3b [0,4194304] 0 2024-05-27T09:29:39.895 INFO:tasks.workunit.client.1.smithi138.stdout:5/375: dread d3/d12/d26/d53/f58 [0,4194304] 0 2024-05-27T09:29:39.895 INFO:tasks.workunit.client.1.smithi138.stdout:5/376: creat d3/d12/d16/d1d/f73 x:0 0 0 2024-05-27T09:29:39.899 INFO:tasks.workunit.client.1.smithi138.stdout:6/610: dwrite d3/d11/f2e [4194304,4194304] 0 2024-05-27T09:29:39.903 INFO:tasks.workunit.client.1.smithi138.stdout:2/371: rmdir d1/d15/d2d/d1d 39 2024-05-27T09:29:39.903 INFO:tasks.workunit.client.1.smithi138.stdout:2/372: write d1/d15/d2d/d17/f6b [437462,13848] 0 2024-05-27T09:29:39.904 INFO:tasks.workunit.client.1.smithi138.stdout:8/456: rmdir d3/d8/d24/d4d 39 2024-05-27T09:29:39.904 INFO:tasks.workunit.client.1.smithi138.stdout:8/457: chown d3/d8/d32/d34/d38/l43 159959 1 2024-05-27T09:29:39.904 INFO:tasks.workunit.client.1.smithi138.stdout:8/458: write d3/d8/d24/f54 [935879,30049] 0 2024-05-27T09:29:39.904 INFO:tasks.workunit.client.1.smithi138.stdout:8/459: readlink d3/d8/l2d 0 2024-05-27T09:29:39.905 INFO:tasks.workunit.client.1.smithi138.stdout:8/460: mknod d3/d8/d32/d34/d78/c8d 0 2024-05-27T09:29:39.908 INFO:tasks.workunit.client.1.smithi138.stdout:6/611: symlink d3/d11/dc0/d43/dbe/ldc 0 2024-05-27T09:29:39.908 INFO:tasks.workunit.client.1.smithi138.stdout:6/612: readlink d3/d19/l50 0 2024-05-27T09:29:39.912 INFO:tasks.workunit.client.0.smithi119.stdout:8/979: truncate d3/d39/f61 3039 0 2024-05-27T09:29:39.915 INFO:tasks.workunit.client.0.smithi119.stdout:6/929: symlink d0/d2/d4e/l14b 0 2024-05-27T09:29:39.916 INFO:tasks.workunit.client.1.smithi138.stdout:5/377: dread d3/d12/d16/d1d/f24 [0,4194304] 0 2024-05-27T09:29:39.926 INFO:tasks.workunit.client.0.smithi119.stdout:0/966: dwrite d2/d6/f8 [4194304,4194304] 0 2024-05-27T09:29:39.928 INFO:tasks.workunit.client.1.smithi138.stdout:7/437: dwrite d0/f45 [0,4194304] 0 2024-05-27T09:29:39.933 INFO:tasks.workunit.client.0.smithi119.stdout:6/930: write d0/d2/d1c/fb3 [1641580,21066] 0 2024-05-27T09:29:39.934 INFO:tasks.workunit.client.0.smithi119.stdout:6/931: read d0/d2/d57/d37/d74/f70 [861711,35335] 0 2024-05-27T09:29:39.948 INFO:tasks.workunit.client.1.smithi138.stdout:8/461: dread d3/d8/d32/d45/f4e [0,4194304] 0 2024-05-27T09:29:39.962 INFO:tasks.workunit.client.1.smithi138.stdout:9/455: dwrite d6/da/d2f/f5f [4194304,4194304] 0 2024-05-27T09:29:39.966 INFO:tasks.workunit.client.1.smithi138.stdout:9/456: creat d6/db/de/d85/f94 x:0 0 0 2024-05-27T09:29:39.971 INFO:tasks.workunit.client.1.smithi138.stdout:7/438: write d0/d7/d64/f7d [11899,103066] 0 2024-05-27T09:29:39.974 INFO:tasks.workunit.client.1.smithi138.stdout:7/439: rename d0/d7/c69 to d0/d23/d49/c94 0 2024-05-27T09:29:39.977 INFO:tasks.workunit.client.1.smithi138.stdout:7/440: rmdir d0/d23/d3c/d84 39 2024-05-27T09:29:39.977 INFO:tasks.workunit.client.1.smithi138.stdout:7/441: write d0/d7/f39 [3095419,5394] 0 2024-05-27T09:29:39.983 INFO:tasks.workunit.client.1.smithi138.stdout:8/462: dread d3/d8/d24/d4d/f61 [0,4194304] 0 2024-05-27T09:29:39.983 INFO:tasks.workunit.client.1.smithi138.stdout:4/518: dwrite d7/df/d5b/fa0 [0,4194304] 0 2024-05-27T09:29:39.983 INFO:tasks.workunit.client.1.smithi138.stdout:0/435: dwrite da/d2a/d2b/f50 [0,4194304] 0 2024-05-27T09:29:39.983 INFO:tasks.workunit.client.1.smithi138.stdout:0/436: truncate da/d2a/d2b/f4d 593038 0 2024-05-27T09:29:39.983 INFO:tasks.workunit.client.1.smithi138.stdout:8/463: read d3/d8/d11/f22 [2700377,54068] 0 2024-05-27T09:29:39.986 INFO:tasks.workunit.client.1.smithi138.stdout:8/464: creat d3/d8/d32/d88/f8e x:0 0 0 2024-05-27T09:29:39.990 INFO:tasks.workunit.client.1.smithi138.stdout:4/519: symlink d7/df/d5b/lb3 0 2024-05-27T09:29:39.991 INFO:tasks.workunit.client.1.smithi138.stdout:4/520: truncate d7/df/d35/d4c/d59/d5d/d64/d6a/f76 579959 0 2024-05-27T09:29:39.993 INFO:tasks.workunit.client.1.smithi138.stdout:4/521: mknod d7/d27/d3a/cb4 0 2024-05-27T09:29:39.993 INFO:tasks.workunit.client.1.smithi138.stdout:4/522: fdatasync d7/f44 0 2024-05-27T09:29:39.995 INFO:tasks.workunit.client.1.smithi138.stdout:2/373: dwrite d1/f7a [4194304,4194304] 0 2024-05-27T09:29:39.995 INFO:tasks.workunit.client.1.smithi138.stdout:2/374: readlink d1/d15/d2d/d31/l5a 0 2024-05-27T09:29:39.997 INFO:tasks.workunit.client.1.smithi138.stdout:2/375: mknod d1/d10/d52/c7c 0 2024-05-27T09:29:39.997 INFO:tasks.workunit.client.1.smithi138.stdout:2/376: creat d1/d2f/f7d x:0 0 0 2024-05-27T09:29:39.997 INFO:tasks.workunit.client.1.smithi138.stdout:2/377: stat d1/d15/d2d/d17/f58 0 2024-05-27T09:29:40.001 INFO:tasks.workunit.client.1.smithi138.stdout:2/378: mkdir d1/d10/d52/d5d/d7e 0 2024-05-27T09:29:40.001 INFO:tasks.workunit.client.1.smithi138.stdout:2/379: truncate d1/d15/d2d/d31/d4b/f4c 715885 0 2024-05-27T09:29:40.003 INFO:tasks.workunit.client.0.smithi119.stdout:8/980: dwrite d3/d8/d1b/db1/df9/f123 [0,4194304] 0 2024-05-27T09:29:40.004 INFO:tasks.workunit.client.1.smithi138.stdout:2/380: link d1/d15/l1e d1/d15/d2d/d17/d25/l7f 0 2024-05-27T09:29:40.004 INFO:tasks.workunit.client.1.smithi138.stdout:2/381: chown d1/d10/f76 228 1 2024-05-27T09:29:40.007 INFO:tasks.workunit.client.1.smithi138.stdout:2/382: symlink d1/l80 0 2024-05-27T09:29:40.008 INFO:tasks.workunit.client.1.smithi138.stdout:2/383: truncate d1/d15/d2d/f39 1291534 0 2024-05-27T09:29:40.008 INFO:tasks.workunit.client.0.smithi119.stdout:8/981: creat d3/d8/d1b/d46/da4/d113/d66/d85/f13f x:0 0 0 2024-05-27T09:29:40.010 INFO:tasks.workunit.client.0.smithi119.stdout:6/932: dwrite d0/d2/d1c/d21/f108 [0,4194304] 0 2024-05-27T09:29:40.012 INFO:tasks.workunit.client.0.smithi119.stdout:8/982: creat d3/d8/d4d/f140 x:0 0 0 2024-05-27T09:29:40.012 INFO:tasks.workunit.client.0.smithi119.stdout:8/983: readlink d3/d8/d1b/d46/da4/la6 0 2024-05-27T09:29:40.012 INFO:tasks.workunit.client.0.smithi119.stdout:8/984: fsync d3/fe5 0 2024-05-27T09:29:40.016 INFO:tasks.workunit.client.0.smithi119.stdout:6/933: rename d0/d85/lf7 to d0/d2/dec/l14c 0 2024-05-27T09:29:40.017 INFO:tasks.workunit.client.0.smithi119.stdout:8/985: fdatasync d3/d8/d1b/db1/ff0 0 2024-05-27T09:29:40.019 INFO:tasks.workunit.client.0.smithi119.stdout:8/986: creat d3/d6e/d8e/f141 x:0 0 0 2024-05-27T09:29:40.021 INFO:tasks.workunit.client.0.smithi119.stdout:6/934: mkdir d0/d2/d4e/d5a/d14d 0 2024-05-27T09:29:40.026 INFO:tasks.workunit.client.0.smithi119.stdout:8/987: mkdir d3/d8/d1b/d46/da4/d142 0 2024-05-27T09:29:40.034 INFO:tasks.workunit.client.1.smithi138.stdout:7/442: fsync d0/d7/d64/f7d 0 2024-05-27T09:29:40.037 INFO:tasks.workunit.client.1.smithi138.stdout:8/465: dread d3/d8/d32/d34/d38/f63 [0,4194304] 0 2024-05-27T09:29:40.038 INFO:tasks.workunit.client.1.smithi138.stdout:6/613: dwrite d3/fae [0,4194304] 0 2024-05-27T09:29:40.038 INFO:tasks.workunit.client.1.smithi138.stdout:6/614: write d3/d11/dc0/d43/d8c/f96 [53093,6202] 0 2024-05-27T09:29:40.039 INFO:tasks.workunit.client.1.smithi138.stdout:6/615: chown d3/d11/d2d/d37/d7e/fb0 140789 1 2024-05-27T09:29:40.042 INFO:tasks.workunit.client.1.smithi138.stdout:9/457: dwrite d6/db/de/d18/d24/f5c [0,4194304] 0 2024-05-27T09:29:40.042 INFO:tasks.workunit.client.1.smithi138.stdout:8/466: symlink d3/d8/d32/d34/d78/l8f 0 2024-05-27T09:29:40.043 INFO:tasks.workunit.client.1.smithi138.stdout:6/616: link d3/d11/dc0/d43/l98 d3/d11/d2d/d37/dd1/d47/ldd 0 2024-05-27T09:29:40.045 INFO:tasks.workunit.client.1.smithi138.stdout:9/458: mknod d6/db/de/d18/d24/d72/c95 0 2024-05-27T09:29:40.045 INFO:tasks.workunit.client.1.smithi138.stdout:8/467: creat d3/d8/d32/d45/f90 x:0 0 0 2024-05-27T09:29:40.047 INFO:tasks.workunit.client.1.smithi138.stdout:7/443: dread d0/d23/f47 [0,4194304] 0 2024-05-27T09:29:40.047 INFO:tasks.workunit.client.1.smithi138.stdout:8/468: rmdir d3/d8/d24/d55/d7e 39 2024-05-27T09:29:40.047 INFO:tasks.workunit.client.1.smithi138.stdout:8/469: truncate d3/d8/d32/d34/d38/f63 4275877 0 2024-05-27T09:29:40.048 INFO:tasks.workunit.client.1.smithi138.stdout:9/459: getdents d6/db/dc 0 2024-05-27T09:29:40.048 INFO:tasks.workunit.client.1.smithi138.stdout:9/460: chown d6/db/de/d18/d24/f5c 55551798 1 2024-05-27T09:29:40.052 INFO:tasks.workunit.client.1.smithi138.stdout:8/470: mknod d3/d8/c91 0 2024-05-27T09:29:40.052 INFO:tasks.workunit.client.1.smithi138.stdout:8/471: write d3/d8/d64/f5e [1826407,16051] 0 2024-05-27T09:29:40.052 INFO:tasks.workunit.client.1.smithi138.stdout:8/472: creat d3/d8/d32/d34/d38/f92 x:0 0 0 2024-05-27T09:29:40.052 INFO:tasks.workunit.client.1.smithi138.stdout:8/473: fsync d3/d8/d32/f3b 0 2024-05-27T09:29:40.052 INFO:tasks.workunit.client.1.smithi138.stdout:7/444: getdents d0/d23/d49 0 2024-05-27T09:29:40.053 INFO:tasks.workunit.client.1.smithi138.stdout:8/474: creat d3/d60/d86/f93 x:0 0 0 2024-05-27T09:29:40.053 INFO:tasks.workunit.client.1.smithi138.stdout:7/445: write d0/d23/d3a/f55 [839647,117219] 0 2024-05-27T09:29:40.053 INFO:tasks.workunit.client.1.smithi138.stdout:7/446: truncate d0/d23/d3a/f46 4264731 0 2024-05-27T09:29:40.055 INFO:tasks.workunit.client.1.smithi138.stdout:7/447: creat d0/d7/d57/f95 x:0 0 0 2024-05-27T09:29:40.057 INFO:tasks.workunit.client.1.smithi138.stdout:7/448: rmdir d0/d7/d64 39 2024-05-27T09:29:40.062 INFO:tasks.workunit.client.1.smithi138.stdout:4/523: dwrite d7/df/d35/fa2 [0,4194304] 0 2024-05-27T09:29:40.062 INFO:tasks.workunit.client.0.smithi119.stdout:0/967: dwrite d2/d6/d9/d12/f47 [0,4194304] 0 2024-05-27T09:29:40.066 INFO:tasks.workunit.client.1.smithi138.stdout:0/437: dwrite da/d2a/d8e/d1f/d71/f79 [0,4194304] 0 2024-05-27T09:29:40.072 INFO:tasks.workunit.client.1.smithi138.stdout:8/475: getdents d3/d8/d32/d45 0 2024-05-27T09:29:40.073 INFO:tasks.workunit.client.0.smithi119.stdout:0/968: mknod d2/d6/d23/d7a/d53/dd1/c143 0 2024-05-27T09:29:40.078 INFO:tasks.workunit.client.1.smithi138.stdout:8/476: mkdir d3/d8/d24/d55/d94 0 2024-05-27T09:29:40.078 INFO:tasks.workunit.client.1.smithi138.stdout:8/477: stat d3/d8/d64/f2e 0 2024-05-27T09:29:40.079 INFO:tasks.workunit.client.1.smithi138.stdout:8/478: write d3/d8/d32/f81 [4889344,32537] 0 2024-05-27T09:29:40.079 INFO:tasks.workunit.client.1.smithi138.stdout:7/449: rmdir d0 39 2024-05-27T09:29:40.079 INFO:tasks.workunit.client.1.smithi138.stdout:4/524: mknod d7/df/d35/d4c/d59/d5d/d64/d6a/cb5 0 2024-05-27T09:29:40.079 INFO:tasks.workunit.client.1.smithi138.stdout:4/525: write d7/df/d35/d4c/d59/d5d/d64/d6a/f76 [1271722,427] 0 2024-05-27T09:29:40.084 INFO:tasks.workunit.client.0.smithi119.stdout:2/975: sync 2024-05-27T09:29:40.084 INFO:tasks.workunit.client.0.smithi119.stdout:9/860: sync 2024-05-27T09:29:40.094 INFO:tasks.workunit.client.1.smithi138.stdout:8/479: write d3/d8/d64/f2f [3373062,40093] 0 2024-05-27T09:29:40.095 INFO:tasks.workunit.client.0.smithi119.stdout:0/969: truncate d2/f11e 343318 0 2024-05-27T09:29:40.095 INFO:tasks.workunit.client.1.smithi138.stdout:2/384: dwrite d1/d15/d2d/d17/d25/f28 [0,4194304] 0 2024-05-27T09:29:40.098 INFO:tasks.workunit.client.0.smithi119.stdout:6/935: dwrite d0/d85/de7/ffa [0,4194304] 0 2024-05-27T09:29:40.098 INFO:tasks.workunit.client.0.smithi119.stdout:6/936: stat d0/d2/d57/d37/de5/la5 0 2024-05-27T09:29:40.099 INFO:tasks.workunit.client.0.smithi119.stdout:0/970: rmdir d2/d6/d9/d24/d30/dcc/d109 39 2024-05-27T09:29:40.100 INFO:tasks.workunit.client.1.smithi138.stdout:4/526: symlink d7/df/d35/d4c/d59/lb6 0 2024-05-27T09:29:40.100 INFO:tasks.workunit.client.1.smithi138.stdout:2/385: creat d1/d15/d2d/d17/d25/d4d/d4e/f81 x:0 0 0 2024-05-27T09:29:40.101 INFO:tasks.workunit.client.1.smithi138.stdout:8/480: symlink d3/d8/d24/l95 0 2024-05-27T09:29:40.102 INFO:tasks.workunit.client.0.smithi119.stdout:9/861: getdents d18/d19/d1b/d40 0 2024-05-27T09:29:40.102 INFO:tasks.workunit.client.0.smithi119.stdout:9/862: stat d18/d19/d1b/d40/d44/dcc 0 2024-05-27T09:29:40.103 INFO:tasks.workunit.client.1.smithi138.stdout:2/386: symlink d1/d10/d3c/l82 0 2024-05-27T09:29:40.103 INFO:tasks.workunit.client.1.smithi138.stdout:8/481: rmdir d3/d8/d24/d4d/d7d 0 2024-05-27T09:29:40.104 INFO:tasks.workunit.client.1.smithi138.stdout:8/482: chown d3 607612419 1 2024-05-27T09:29:40.104 INFO:tasks.workunit.client.0.smithi119.stdout:2/976: dread d2/da/d15/f16 [0,4194304] 0 2024-05-27T09:29:40.104 INFO:tasks.workunit.client.0.smithi119.stdout:2/977: dread - d2/da/d8a/dd4/d89/db0/fca zero size 2024-05-27T09:29:40.105 INFO:tasks.workunit.client.0.smithi119.stdout:6/937: rename d0/d2/f67 to d0/d2/d57/d37/d74/dfe/d10a/f14e 0 2024-05-27T09:29:40.105 INFO:tasks.workunit.client.0.smithi119.stdout:6/938: creat d0/d2/d57/d37/dad/f14f x:0 0 0 2024-05-27T09:29:40.107 INFO:tasks.workunit.client.1.smithi138.stdout:7/450: dread d0/d7/f7c [0,4194304] 0 2024-05-27T09:29:40.108 INFO:tasks.workunit.client.0.smithi119.stdout:0/971: link d2/d6/d23/c17 d2/d1a/c144 0 2024-05-27T09:29:40.108 INFO:tasks.workunit.client.0.smithi119.stdout:9/863: symlink d18/d19/d1b/d40/db7/l10c 0 2024-05-27T09:29:40.110 INFO:tasks.workunit.client.0.smithi119.stdout:2/978: mknod d2/da/d8a/dd4/d89/d94/d87/dcf/de5/c14e 0 2024-05-27T09:29:40.112 INFO:tasks.workunit.client.1.smithi138.stdout:2/387: read d1/d15/d2d/d17/d25/f2a [387817,91056] 0 2024-05-27T09:29:40.114 INFO:tasks.workunit.client.0.smithi119.stdout:9/864: truncate d18/d19/f3f 949806 0 2024-05-27T09:29:40.115 INFO:tasks.workunit.client.0.smithi119.stdout:6/939: dread d0/d2/d57/d37/d74/fa1 [0,4194304] 0 2024-05-27T09:29:40.116 INFO:tasks.workunit.client.1.smithi138.stdout:8/483: rmdir d3/d8/d24/d55/d66 39 2024-05-27T09:29:40.117 INFO:tasks.workunit.client.0.smithi119.stdout:2/979: mknod d2/da/d8a/dd4/d89/d94/de2/c14f 0 2024-05-27T09:29:40.117 INFO:tasks.workunit.client.0.smithi119.stdout:2/980: fdatasync d2/fa9 0 2024-05-27T09:29:40.119 INFO:tasks.workunit.client.0.smithi119.stdout:8/988: dwrite d3/d8/d1b/f41 [4194304,4194304] 0 2024-05-27T09:29:40.120 INFO:tasks.workunit.client.0.smithi119.stdout:2/981: creat d2/d1a/d132/f150 x:0 0 0 2024-05-27T09:29:40.122 INFO:tasks.workunit.client.1.smithi138.stdout:7/451: mknod d0/d7/d57/d91/c96 0 2024-05-27T09:29:40.126 INFO:tasks.workunit.client.1.smithi138.stdout:8/484: link d3/d8/d32/d3f/f6b d3/d8/d32/d34/d78/f96 0 2024-05-27T09:29:40.129 INFO:tasks.workunit.client.0.smithi119.stdout:0/972: dread d2/d6/d9/d24/d30/dcc/de6/d137/fa2 [0,4194304] 0 2024-05-27T09:29:40.129 INFO:tasks.workunit.client.0.smithi119.stdout:0/973: chown d2/d6/d9/c12e 2814890 1 2024-05-27T09:29:40.129 INFO:tasks.workunit.client.0.smithi119.stdout:0/974: fsync d2/d6/ffc 0 2024-05-27T09:29:40.135 INFO:tasks.workunit.client.1.smithi138.stdout:7/452: unlink d0/d23/c33 0 2024-05-27T09:29:40.149 INFO:tasks.workunit.client.0.smithi119.stdout:2/982: dread d2/da/d8a/dd4/d89/d94/ffd [0,4194304] 0 2024-05-27T09:29:40.151 INFO:tasks.workunit.client.0.smithi119.stdout:6/940: read d0/ff5 [713764,98531] 0 2024-05-27T09:29:40.151 INFO:tasks.workunit.client.0.smithi119.stdout:2/983: mknod d2/da/d8a/dd4/d69/c151 0 2024-05-27T09:29:40.154 INFO:tasks.workunit.client.1.smithi138.stdout:9/461: dwrite d6/db/f10 [0,4194304] 0 2024-05-27T09:29:40.155 INFO:tasks.workunit.client.1.smithi138.stdout:9/462: link d6/f2e d6/db/de/d18/d27/f96 0 2024-05-27T09:29:40.157 INFO:tasks.workunit.client.1.smithi138.stdout:2/388: dwrite d1/d15/f4f [4194304,4194304] 0 2024-05-27T09:29:40.157 INFO:tasks.workunit.client.1.smithi138.stdout:2/389: truncate d1/d15/d2d/d17/d25/f28 4818998 0 2024-05-27T09:29:40.157 INFO:tasks.workunit.client.1.smithi138.stdout:2/390: stat d1/d15/d2d/d17/f6b 0 2024-05-27T09:29:40.160 INFO:tasks.workunit.client.1.smithi138.stdout:2/391: creat d1/d2f/f83 x:0 0 0 2024-05-27T09:29:40.161 INFO:tasks.workunit.client.1.smithi138.stdout:2/392: truncate d1/d15/d2d/d2e/f37 1830296 0 2024-05-27T09:29:40.161 INFO:tasks.workunit.client.1.smithi138.stdout:0/438: dwrite da/d2a/d8e/d43/f62 [0,4194304] 0 2024-05-27T09:29:40.162 INFO:tasks.workunit.client.0.smithi119.stdout:6/941: rename d0/d2/d57/d37/d74/df1/d109/db2 to d0/d2/d57/d37/d74/d65/d150 0 2024-05-27T09:29:40.162 INFO:tasks.workunit.client.0.smithi119.stdout:6/942: creat d0/d2/dec/f151 x:0 0 0 2024-05-27T09:29:40.162 INFO:tasks.workunit.client.0.smithi119.stdout:9/865: dread d18/d19/d1b/d40/d44/f54 [4194304,4194304] 0 2024-05-27T09:29:40.162 INFO:tasks.workunit.client.0.smithi119.stdout:9/866: write d18/d19/f74 [80677,54960] 0 2024-05-27T09:29:40.162 INFO:tasks.workunit.client.0.smithi119.stdout:9/867: creat d18/d19/d1b/d40/d44/f10d x:0 0 0 2024-05-27T09:29:40.163 INFO:tasks.workunit.client.1.smithi138.stdout:9/463: rename d6/da/d29/c5a to d6/da/d2f/d35/d55/d75/d8c/c97 0 2024-05-27T09:29:40.163 INFO:tasks.workunit.client.1.smithi138.stdout:9/464: chown d6/db/de 1411659 1 2024-05-27T09:29:40.163 INFO:tasks.workunit.client.1.smithi138.stdout:9/465: truncate d6/f2e 169968 0 2024-05-27T09:29:40.163 INFO:tasks.workunit.client.1.smithi138.stdout:2/393: creat d1/d10/d52/d5d/d7e/f84 x:0 0 0 2024-05-27T09:29:40.164 INFO:tasks.workunit.client.1.smithi138.stdout:9/466: mknod d6/db/de/c98 0 2024-05-27T09:29:40.164 INFO:tasks.workunit.client.1.smithi138.stdout:9/467: stat d6/da/d29/d59 0 2024-05-27T09:29:40.165 INFO:tasks.workunit.client.1.smithi138.stdout:2/394: mknod d1/d15/d2d/d2e/c85 0 2024-05-27T09:29:40.166 INFO:tasks.workunit.client.1.smithi138.stdout:9/468: truncate d6/db/de/f2b 7576767 0 2024-05-27T09:29:40.166 INFO:tasks.workunit.client.1.smithi138.stdout:9/469: truncate d6/da/d29/f74 148898 0 2024-05-27T09:29:40.167 INFO:tasks.workunit.client.1.smithi138.stdout:2/395: symlink d1/d15/d2d/d17/d25/d4d/d4e/d60/l86 0 2024-05-27T09:29:40.170 INFO:tasks.workunit.client.1.smithi138.stdout:9/470: rmdir d6/db/de/d18/d24 39 2024-05-27T09:29:40.170 INFO:tasks.workunit.client.1.smithi138.stdout:2/396: mkdir d1/d15/d2d/d17/d25/d4d/d6d/d87 0 2024-05-27T09:29:40.172 INFO:tasks.workunit.client.1.smithi138.stdout:2/397: rename d1/d15/d2d/d31/f3d to d1/d10/d52/d5d/f88 0 2024-05-27T09:29:40.179 INFO:tasks.workunit.client.1.smithi138.stdout:0/439: creat da/d2a/d8e/d1f/d71/f93 x:0 0 0 2024-05-27T09:29:40.179 INFO:tasks.workunit.client.1.smithi138.stdout:0/440: chown da/d2a/d8e/d1f/d42/d4a/c65 1490 1 2024-05-27T09:29:40.181 INFO:tasks.workunit.client.0.smithi119.stdout:6/943: rename d0/d2/d4e/d5a/d76/c116 to d0/d2/d1c/c152 0 2024-05-27T09:29:40.181 INFO:tasks.workunit.client.0.smithi119.stdout:6/944: readlink d0/d2/lb5 0 2024-05-27T09:29:40.187 INFO:tasks.workunit.client.1.smithi138.stdout:0/441: symlink da/d2a/d55/l94 0 2024-05-27T09:29:40.189 INFO:tasks.workunit.client.0.smithi119.stdout:6/945: mknod d0/d2/d57/dd0/d12b/c153 0 2024-05-27T09:29:40.196 INFO:tasks.workunit.client.1.smithi138.stdout:0/442: creat da/d2a/d55/f95 x:0 0 0 2024-05-27T09:29:40.196 INFO:tasks.workunit.client.0.smithi119.stdout:6/946: dread d0/d2/d57/d37/d74/df1/f11f [0,4194304] 0 2024-05-27T09:29:40.202 INFO:tasks.workunit.client.0.smithi119.stdout:6/947: rename d0/c54 to d0/d2/d57/d37/de5/d147/c154 0 2024-05-27T09:29:40.202 INFO:tasks.workunit.client.0.smithi119.stdout:6/948: truncate d0/d2/d4e/d5a/d76/fa3 1041822 0 2024-05-27T09:29:40.202 INFO:tasks.workunit.client.0.smithi119.stdout:6/949: chown d0/d2/d1c/d21/d58/c7c 351420348 1 2024-05-27T09:29:40.207 INFO:tasks.workunit.client.0.smithi119.stdout:6/950: mkdir d0/d155 0 2024-05-27T09:29:40.210 INFO:tasks.workunit.client.1.smithi138.stdout:6/617: dwrite d3/d11/dc0/d43/d1c/f4c [0,4194304] 0 2024-05-27T09:29:40.211 INFO:tasks.workunit.client.0.smithi119.stdout:6/951: stat d0/d2/d4e/d5a/le0 0 2024-05-27T09:29:40.211 INFO:tasks.workunit.client.0.smithi119.stdout:6/952: chown d0/d2/d1c/l71 0 1 2024-05-27T09:29:40.211 INFO:tasks.workunit.client.0.smithi119.stdout:6/953: stat d0/f18 0 2024-05-27T09:29:40.218 INFO:tasks.workunit.client.0.smithi119.stdout:6/954: rename d0/d2/d57/d37/d74/df1/d109/d118/f141 to d0/d85/f156 0 2024-05-27T09:29:40.218 INFO:tasks.workunit.client.1.smithi138.stdout:0/443: write da/d2a/d55/f61 [6556158,113578] 0 2024-05-27T09:29:40.220 INFO:tasks.workunit.client.1.smithi138.stdout:6/618: getdents d3/d11/dc0/d7d/da5 0 2024-05-27T09:29:40.223 INFO:tasks.workunit.client.1.smithi138.stdout:0/444: symlink da/d2a/d8e/d43/l96 0 2024-05-27T09:29:40.223 INFO:tasks.workunit.client.1.smithi138.stdout:0/445: write da/d2a/f8f [144752,23834] 0 2024-05-27T09:29:40.223 INFO:tasks.workunit.client.1.smithi138.stdout:6/619: getdents d3/d11/d2d/d37/dd1/d9b 0 2024-05-27T09:29:40.223 INFO:tasks.workunit.client.1.smithi138.stdout:6/620: fsync d3/fae 0 2024-05-27T09:29:40.223 INFO:tasks.workunit.client.1.smithi138.stdout:6/621: fsync d3/f17 0 2024-05-27T09:29:40.223 INFO:tasks.workunit.client.1.smithi138.stdout:0/446: mkdir da/d2a/d8e/d43/d74/d97 0 2024-05-27T09:29:40.226 INFO:tasks.workunit.client.1.smithi138.stdout:0/447: link da/d2a/d8e/d1f/d42/d4a/d53/f57 da/d2a/d55/f98 0 2024-05-27T09:29:40.226 INFO:tasks.workunit.client.1.smithi138.stdout:6/622: rmdir d3/d11/d2d/d37/d7e 39 2024-05-27T09:29:40.226 INFO:tasks.workunit.client.1.smithi138.stdout:6/623: creat d3/d11/fde x:0 0 0 2024-05-27T09:29:40.227 INFO:tasks.workunit.client.1.smithi138.stdout:6/624: read - d3/d11/d2d/d37/dd1/d9b/fb7 zero size 2024-05-27T09:29:40.230 INFO:tasks.workunit.client.1.smithi138.stdout:6/625: rmdir d3/d11/dc0/d5a/dcf/dd8 0 2024-05-27T09:29:40.230 INFO:tasks.workunit.client.1.smithi138.stdout:6/626: write d3/d11/dc0/fbd [750379,127497] 0 2024-05-27T09:29:40.230 INFO:tasks.workunit.client.1.smithi138.stdout:6/627: dread - d3/d11/d2d/d37/d7e/fd6 zero size 2024-05-27T09:29:40.230 INFO:tasks.workunit.client.1.smithi138.stdout:6/628: chown d3/d11/dc0/d5a/dcf 0 1 2024-05-27T09:29:40.230 INFO:tasks.workunit.client.1.smithi138.stdout:6/629: creat d3/d11/dc0/d43/d71/fdf x:0 0 0 2024-05-27T09:29:40.232 INFO:tasks.workunit.client.1.smithi138.stdout:6/630: mkdir d3/d11/dc0/d74/de0 0 2024-05-27T09:29:40.232 INFO:tasks.workunit.client.1.smithi138.stdout:6/631: truncate d3/d11/dc0/d7d/fc5 1001932 0 2024-05-27T09:29:40.233 INFO:tasks.workunit.client.1.smithi138.stdout:6/632: readlink d3/d11/d2d/l6e 0 2024-05-27T09:29:40.236 INFO:tasks.workunit.client.1.smithi138.stdout:7/453: dwrite d0/d2b/f5f [0,4194304] 0 2024-05-27T09:29:40.236 INFO:tasks.workunit.client.1.smithi138.stdout:7/454: write d0/d23/d3c/f42 [225815,104766] 0 2024-05-27T09:29:40.237 INFO:tasks.workunit.client.0.smithi119.stdout:2/984: dwrite d2/da/d8a/f9f [0,4194304] 0 2024-05-27T09:29:40.238 INFO:tasks.workunit.client.0.smithi119.stdout:2/985: rmdir d2/da/d8a/dd4/d69/df5 39 2024-05-27T09:29:40.239 INFO:tasks.workunit.client.1.smithi138.stdout:6/633: write d3/d11/d2d/d37/dd1/d47/f56 [301071,73896] 0 2024-05-27T09:29:40.240 INFO:tasks.workunit.client.1.smithi138.stdout:7/455: rename d0/d23/l26 to d0/d7/l97 0 2024-05-27T09:29:40.241 INFO:tasks.workunit.client.0.smithi119.stdout:2/986: rename d2/da/d8a/dd4/d89/d94/lf0 to d2/d1a/d132/l152 0 2024-05-27T09:29:40.242 INFO:tasks.workunit.client.1.smithi138.stdout:6/634: rmdir d3/d11/dc0/d43/dbe/dc3 39 2024-05-27T09:29:40.243 INFO:tasks.workunit.client.1.smithi138.stdout:7/456: mkdir d0/d7/d98 0 2024-05-27T09:29:40.245 INFO:tasks.workunit.client.0.smithi119.stdout:6/955: dread d0/d2/d4e/d5a/faa [0,4194304] 0 2024-05-27T09:29:40.248 INFO:tasks.workunit.client.1.smithi138.stdout:7/457: mknod d0/d23/d3c/d3f/c99 0 2024-05-27T09:29:40.248 INFO:tasks.workunit.client.1.smithi138.stdout:7/458: chown d0/d2b/c44 33994 1 2024-05-27T09:29:40.248 INFO:tasks.workunit.client.1.smithi138.stdout:7/459: write d0/f83 [524044,5334] 0 2024-05-27T09:29:40.248 INFO:tasks.workunit.client.1.smithi138.stdout:6/635: rename d3/d11/d2d/d37/d7e/d8d to d3/d11/dc2/de1 0 2024-05-27T09:29:40.261 INFO:tasks.workunit.client.1.smithi138.stdout:9/471: dwrite d6/db/de/d18/f92 [0,4194304] 0 2024-05-27T09:29:40.261 INFO:tasks.workunit.client.1.smithi138.stdout:9/472: chown d6/db/de/d18/l54 72629 1 2024-05-27T09:29:40.274 INFO:tasks.workunit.client.1.smithi138.stdout:9/473: creat d6/da/d2f/d35/d55/d75/f99 x:0 0 0 2024-05-27T09:29:40.274 INFO:tasks.workunit.client.1.smithi138.stdout:9/474: truncate d6/da/d29/f89 271543 0 2024-05-27T09:29:40.275 INFO:tasks.workunit.client.1.smithi138.stdout:9/475: mkdir d6/db/de/d18/d27/d9a 0 2024-05-27T09:29:40.286 INFO:tasks.workunit.client.0.smithi119.stdout:6/956: write d0/d2/d1c/d21/f108 [311221,83458] 0 2024-05-27T09:29:40.286 INFO:tasks.workunit.client.0.smithi119.stdout:6/957: truncate d0/d11/fdb 741023 0 2024-05-27T09:29:40.286 INFO:tasks.workunit.client.1.smithi138.stdout:3/478: sync 2024-05-27T09:29:40.286 INFO:tasks.workunit.client.1.smithi138.stdout:1/446: sync 2024-05-27T09:29:40.287 INFO:tasks.workunit.client.1.smithi138.stdout:1/447: chown d4/d1a/d2e/f79 13719 1 2024-05-27T09:29:40.292 INFO:tasks.workunit.client.1.smithi138.stdout:3/479: rmdir d0/d2/da/df/d10 39 2024-05-27T09:29:40.293 INFO:tasks.workunit.client.1.smithi138.stdout:3/480: mkdir d0/d3f/d50/d9b/d9f 0 2024-05-27T09:29:40.293 INFO:tasks.workunit.client.1.smithi138.stdout:3/481: truncate d0/d2/da/df/d10/d32/f3c 733566 0 2024-05-27T09:29:40.294 INFO:tasks.workunit.client.0.smithi119.stdout:6/958: link d0/d2/d57/d37/dad/db6/fcd d0/d2/d1c/d21/d58/f157 0 2024-05-27T09:29:40.294 INFO:tasks.workunit.client.1.smithi138.stdout:3/482: link d0/d2/da/df/d10/d32/f5b d0/d80/fa0 0 2024-05-27T09:29:40.299 INFO:tasks.workunit.client.1.smithi138.stdout:3/483: getdents d0/d2/da/df/d10/d2a/d61 0 2024-05-27T09:29:40.301 INFO:tasks.workunit.client.1.smithi138.stdout:0/448: dwrite da/f20 [0,4194304] 0 2024-05-27T09:29:40.301 INFO:tasks.workunit.client.1.smithi138.stdout:0/449: fdatasync da/d2a/d8e/d1f/d71/f93 0 2024-05-27T09:29:40.306 INFO:tasks.workunit.client.1.smithi138.stdout:1/448: unlink d4/d1a/d2e/d4a/c72 0 2024-05-27T09:29:40.309 INFO:tasks.workunit.client.1.smithi138.stdout:0/450: mkdir da/d2a/d8e/d6a/d99 0 2024-05-27T09:29:40.309 INFO:tasks.workunit.client.1.smithi138.stdout:0/451: stat da/c39 0 2024-05-27T09:29:40.314 INFO:tasks.workunit.client.0.smithi119.stdout:6/959: unlink d0/d2/f7f 0 2024-05-27T09:29:40.314 INFO:tasks.workunit.client.0.smithi119.stdout:6/960: write d0/d2/d57/d37/d74/fa1 [659967,55947] 0 2024-05-27T09:29:40.317 INFO:tasks.workunit.client.0.smithi119.stdout:6/961: creat d0/d2/d57/dd0/d12b/f158 x:0 0 0 2024-05-27T09:29:40.319 INFO:tasks.workunit.client.0.smithi119.stdout:6/962: getdents d0/d2/de1 0 2024-05-27T09:29:40.321 INFO:tasks.workunit.client.0.smithi119.stdout:6/963: truncate d0/d2/d1c/d21/d58/f89 2785260 0 2024-05-27T09:29:40.323 INFO:tasks.workunit.client.0.smithi119.stdout:6/964: write d0/d2/d4e/f1f [1943499,128651] 0 2024-05-27T09:29:40.323 INFO:tasks.workunit.client.0.smithi119.stdout:6/965: chown d0/d2/d57/d37/de5/ca2 264549952 1 2024-05-27T09:29:40.324 INFO:tasks.workunit.client.1.smithi138.stdout:6/636: dwrite d3/f53 [0,4194304] 0 2024-05-27T09:29:40.328 INFO:tasks.workunit.client.1.smithi138.stdout:6/637: rmdir d3/d11/dc0/d5a/dcf 39 2024-05-27T09:29:40.331 INFO:tasks.workunit.client.1.smithi138.stdout:7/460: dwrite d0/d7/d43/f67 [0,4194304] 0 2024-05-27T09:29:40.354 INFO:tasks.workunit.client.0.smithi119.stdout:2/987: dwrite d2/f66 [4194304,4194304] 0 2024-05-27T09:29:40.354 INFO:tasks.workunit.client.0.smithi119.stdout:2/988: chown d2/da/d8a/d12d/d145/ff8 1039 1 2024-05-27T09:29:40.357 INFO:tasks.workunit.client.1.smithi138.stdout:3/484: dwrite d0/d2/da/df/d10/d32/f28 [0,4194304] 0 2024-05-27T09:29:40.358 INFO:tasks.workunit.client.1.smithi138.stdout:3/485: write d0/d2/da/df/d10/d2a/f53 [877288,51608] 0 2024-05-27T09:29:40.360 INFO:tasks.workunit.client.1.smithi138.stdout:9/476: dwrite d6/da/d29/d61/f68 [0,4194304] 0 2024-05-27T09:29:40.360 INFO:tasks.workunit.client.1.smithi138.stdout:9/477: write d6/da/f5d [450938,114831] 0 2024-05-27T09:29:40.360 INFO:tasks.workunit.client.1.smithi138.stdout:9/478: creat d6/da/d2f/d35/d55/d75/f9b x:0 0 0 2024-05-27T09:29:40.360 INFO:tasks.workunit.client.1.smithi138.stdout:9/479: getdents d6/da/d29/d8f 0 2024-05-27T09:29:40.364 INFO:tasks.workunit.client.1.smithi138.stdout:7/461: write d0/f45 [1228421,12333] 0 2024-05-27T09:29:40.369 INFO:tasks.workunit.client.1.smithi138.stdout:9/480: link d6/da/l36 d6/da/d2f/d35/d55/l9c 0 2024-05-27T09:29:40.373 INFO:tasks.workunit.client.1.smithi138.stdout:7/462: rename d0/d23/f5d to d0/d23/d59/f9a 0 2024-05-27T09:29:40.373 INFO:tasks.workunit.client.1.smithi138.stdout:7/463: chown d0/d7/d60 758 1 2024-05-27T09:29:40.373 INFO:tasks.workunit.client.1.smithi138.stdout:7/464: fdatasync d0/d23/f48 0 2024-05-27T09:29:40.376 INFO:tasks.workunit.client.0.smithi119.stdout:2/989: creat d2/da/d2c/f153 x:0 0 0 2024-05-27T09:29:40.376 INFO:tasks.workunit.client.0.smithi119.stdout:2/990: fsync d2/fa9 0 2024-05-27T09:29:40.376 INFO:tasks.workunit.client.0.smithi119.stdout:2/991: write d2/da/d15/d53/d114/d116/d12a/f14a [736280,62151] 0 2024-05-27T09:29:40.378 INFO:tasks.workunit.client.1.smithi138.stdout:7/465: creat d0/d23/d49/d78/f9b x:0 0 0 2024-05-27T09:29:40.383 INFO:tasks.workunit.client.1.smithi138.stdout:7/466: mkdir d0/d23/d3c/d84/d9c 0 2024-05-27T09:29:40.392 INFO:tasks.workunit.client.1.smithi138.stdout:6/638: dwrite d3/d91/fa0 [0,4194304] 0 2024-05-27T09:29:40.392 INFO:tasks.workunit.client.1.smithi138.stdout:6/639: truncate d3/d11/d2d/d37/dd1/d47/f51 978907 0 2024-05-27T09:29:40.392 INFO:tasks.workunit.client.1.smithi138.stdout:6/640: truncate d3/d11/d2d/d37/d8b/fba 1216172 0 2024-05-27T09:29:40.397 INFO:tasks.workunit.client.0.smithi119.stdout:8/989: sync 2024-05-27T09:29:40.397 INFO:tasks.workunit.client.0.smithi119.stdout:0/975: sync 2024-05-27T09:29:40.397 INFO:tasks.workunit.client.0.smithi119.stdout:9/868: sync 2024-05-27T09:29:40.397 INFO:tasks.workunit.client.0.smithi119.stdout:9/869: chown d18 257141698 1 2024-05-27T09:29:40.397 INFO:tasks.workunit.client.0.smithi119.stdout:0/976: stat d2/d110/d96/d132/dd7/ffd 0 2024-05-27T09:29:40.397 INFO:tasks.workunit.client.0.smithi119.stdout:8/990: read - d3/de/f109 zero size 2024-05-27T09:29:40.397 INFO:tasks.workunit.client.0.smithi119.stdout:9/870: dread - d18/d19/d1b/d40/d44/fd3 zero size 2024-05-27T09:29:40.397 INFO:tasks.workunit.client.0.smithi119.stdout:0/977: write d2/d6/d9/f3a [3015902,59397] 0 2024-05-27T09:29:40.397 INFO:tasks.workunit.client.0.smithi119.stdout:0/978: chown d2/d6/ffc 0 1 2024-05-27T09:29:40.398 INFO:tasks.workunit.client.0.smithi119.stdout:0/979: dread - d2/d6/d9/d2f/f136 zero size 2024-05-27T09:29:40.398 INFO:tasks.workunit.client.0.smithi119.stdout:2/992: dread d2/d1a/f1f [0,4194304] 0 2024-05-27T09:29:40.398 INFO:tasks.workunit.client.1.smithi138.stdout:5/378: sync 2024-05-27T09:29:40.399 INFO:tasks.workunit.client.1.smithi138.stdout:5/379: write d3/d12/d16/f39 [6927072,111913] 0 2024-05-27T09:29:40.399 INFO:tasks.workunit.client.0.smithi119.stdout:9/871: mknod d18/d19/d1b/d2a/d63/c10e 0 2024-05-27T09:29:40.403 INFO:tasks.workunit.client.1.smithi138.stdout:6/641: link d3/d11/f4f d3/fe2 0 2024-05-27T09:29:40.403 INFO:tasks.workunit.client.1.smithi138.stdout:9/481: write d6/db/de/f31 [3030834,81762] 0 2024-05-27T09:29:40.403 INFO:tasks.workunit.client.1.smithi138.stdout:0/452: dwrite da/d17/f8c [0,4194304] 0 2024-05-27T09:29:40.413 INFO:tasks.workunit.client.0.smithi119.stdout:8/991: write d3/d8/d1b/d46/da4/dbb/ff8 [2139295,128390] 0 2024-05-27T09:29:40.413 INFO:tasks.workunit.client.0.smithi119.stdout:2/993: unlink d2/d1a/c3e 0 2024-05-27T09:29:40.413 INFO:tasks.workunit.client.0.smithi119.stdout:9/872: mknod d18/d19/d1b/d29/dc4/de1/c10f 0 2024-05-27T09:29:40.414 INFO:tasks.workunit.client.0.smithi119.stdout:9/873: getdents d18/d19/d1b/d40/d44/dcc 0 2024-05-27T09:29:40.414 INFO:tasks.workunit.client.0.smithi119.stdout:8/992: creat d3/d8/d1b/d46/da4/d113/d60/f143 x:0 0 0 2024-05-27T09:29:40.414 INFO:tasks.workunit.client.0.smithi119.stdout:8/993: readlink d3/d8/l5b 0 2024-05-27T09:29:40.414 INFO:tasks.workunit.client.0.smithi119.stdout:8/994: dread - d3/faf zero size 2024-05-27T09:29:40.417 INFO:tasks.workunit.client.1.smithi138.stdout:5/380: mknod d3/c74 0 2024-05-27T09:29:40.417 INFO:tasks.workunit.client.1.smithi138.stdout:5/381: write d3/d12/f31 [412929,67814] 0 2024-05-27T09:29:40.422 INFO:tasks.workunit.client.0.smithi119.stdout:2/994: rename d2/da/d8a/dd4/d89/de8/d12f/f131 to d2/da/d8a/dd4/d89/d94/d87/dcf/f154 0 2024-05-27T09:29:40.423 INFO:tasks.workunit.client.1.smithi138.stdout:5/382: rmdir d3/d12/d32/d6b/d6e 0 2024-05-27T09:29:40.423 INFO:tasks.workunit.client.1.smithi138.stdout:5/383: truncate d3/d12/d16/d1d/d22/f3a 599433 0 2024-05-27T09:29:40.425 INFO:tasks.workunit.client.0.smithi119.stdout:9/874: link d18/d19/d1b/d29/fa8 d18/d19/d1b/d2a/da2/f110 0 2024-05-27T09:29:40.426 INFO:tasks.workunit.client.0.smithi119.stdout:9/875: write d18/fa6 [743473,78795] 0 2024-05-27T09:29:40.426 INFO:tasks.workunit.client.0.smithi119.stdout:9/876: chown d18/d19/d1b/c3c 1052053 1 2024-05-27T09:29:40.427 INFO:tasks.workunit.client.1.smithi138.stdout:0/453: dread da/f29 [0,4194304] 0 2024-05-27T09:29:40.428 INFO:tasks.workunit.client.0.smithi119.stdout:8/995: creat d3/d39/f144 x:0 0 0 2024-05-27T09:29:40.433 INFO:tasks.workunit.client.1.smithi138.stdout:9/482: dread d6/da/d29/d61/f68 [0,4194304] 0 2024-05-27T09:29:40.433 INFO:tasks.workunit.client.1.smithi138.stdout:0/454: creat da/d2a/d8e/d1f/d42/f9a x:0 0 0 2024-05-27T09:29:40.433 INFO:tasks.workunit.client.1.smithi138.stdout:0/455: stat da/d2a/d2b/c69 0 2024-05-27T09:29:40.433 INFO:tasks.workunit.client.1.smithi138.stdout:0/456: chown da/d2a/d8e/d43/l96 20003 1 2024-05-27T09:29:40.434 INFO:tasks.workunit.client.1.smithi138.stdout:9/483: mknod d6/da/d2f/d35/d55/d75/d8c/c9d 0 2024-05-27T09:29:40.435 INFO:tasks.workunit.client.0.smithi119.stdout:9/877: rmdir d18/d19/d9b 39 2024-05-27T09:29:40.435 INFO:tasks.workunit.client.0.smithi119.stdout:9/878: write d18/d19/d1b/d29/dc4/de1/f109 [576820,12020] 0 2024-05-27T09:29:40.438 INFO:tasks.workunit.client.0.smithi119.stdout:6/966: dwrite d0/d2/d57/d37/f43 [0,4194304] 0 2024-05-27T09:29:40.438 INFO:tasks.workunit.client.1.smithi138.stdout:5/384: mkdir d3/d75 0 2024-05-27T09:29:40.438 INFO:tasks.workunit.client.1.smithi138.stdout:5/385: dread - d3/d12/f64 zero size 2024-05-27T09:29:40.438 INFO:tasks.workunit.client.1.smithi138.stdout:5/386: truncate d3/d12/d16/d1d/d23/f62 901743 0 2024-05-27T09:29:40.439 INFO:tasks.workunit.client.1.smithi138.stdout:9/484: creat d6/da/f9e x:0 0 0 2024-05-27T09:29:40.439 INFO:tasks.workunit.client.1.smithi138.stdout:9/485: write d6/da/d2f/d35/d55/d75/f9b [109762,42456] 0 2024-05-27T09:29:40.442 INFO:tasks.workunit.client.0.smithi119.stdout:8/996: symlink d3/de/dad/l145 0 2024-05-27T09:29:40.457 INFO:tasks.workunit.client.0.smithi119.stdout:6/967: creat d0/d2/f159 x:0 0 0 2024-05-27T09:29:40.462 INFO:tasks.workunit.client.1.smithi138.stdout:9/486: dread d6/da/d29/d61/f68 [0,4194304] 0 2024-05-27T09:29:40.462 INFO:tasks.workunit.client.1.smithi138.stdout:9/487: read d6/db/de/d18/d24/f4b [653926,122659] 0 2024-05-27T09:29:40.462 INFO:tasks.workunit.client.1.smithi138.stdout:9/488: dread - d6/da/d2f/d35/d55/d75/f99 zero size 2024-05-27T09:29:40.462 INFO:tasks.workunit.client.1.smithi138.stdout:9/489: truncate d6/f88 116889 0 2024-05-27T09:29:40.466 INFO:tasks.workunit.client.0.smithi119.stdout:8/997: truncate d3/d8/d1b/d46/f43 1714059 0 2024-05-27T09:29:40.467 INFO:tasks.workunit.client.1.smithi138.stdout:6/642: dwrite d3/d11/dc0/d43/d71/fdf [0,4194304] 0 2024-05-27T09:29:40.468 INFO:tasks.workunit.client.1.smithi138.stdout:6/643: chown d3/d11/dc0/f60 20809875 1 2024-05-27T09:29:40.468 INFO:tasks.workunit.client.1.smithi138.stdout:6/644: fsync d3/d11/dc0/fbd 0 2024-05-27T09:29:40.468 INFO:tasks.workunit.client.1.smithi138.stdout:6/645: dread - d3/d11/dc0/d7d/f92 zero size 2024-05-27T09:29:40.468 INFO:tasks.workunit.client.1.smithi138.stdout:6/646: chown d3/d11/dc0/d5a/d6a/l94 933 1 2024-05-27T09:29:40.468 INFO:tasks.workunit.client.1.smithi138.stdout:6/647: truncate d3/d11/dc0/d43/dbe/dc3/fcc 916351 0 2024-05-27T09:29:40.468 INFO:tasks.workunit.client.1.smithi138.stdout:9/490: symlink d6/da/d2f/d35/l9f 0 2024-05-27T09:29:40.471 INFO:tasks.workunit.client.0.smithi119.stdout:0/980: dwrite d2/f8c [8388608,4194304] 0 2024-05-27T09:29:40.474 INFO:tasks.workunit.client.1.smithi138.stdout:5/387: mknod d3/d12/d16/d1d/d22/d2b/c76 0 2024-05-27T09:29:40.487 INFO:tasks.workunit.client.1.smithi138.stdout:7/467: dwrite d0/d7/d43/f67 [0,4194304] 0 2024-05-27T09:29:40.493 INFO:tasks.workunit.client.0.smithi119.stdout:0/981: dread d2/d6/d23/db8/fea [0,4194304] 0 2024-05-27T09:29:40.493 INFO:tasks.workunit.client.0.smithi119.stdout:0/982: write d2/d6/d9/d12/f16 [257523,75705] 0 2024-05-27T09:29:40.493 INFO:tasks.workunit.client.0.smithi119.stdout:0/983: chown d2/d6/d9/d12/f16 1021542 1 2024-05-27T09:29:40.504 INFO:tasks.workunit.client.0.smithi119.stdout:6/968: truncate d0/d2/d57/d37/f86 6987773 0 2024-05-27T09:29:40.504 INFO:tasks.workunit.client.0.smithi119.stdout:6/969: chown d0/d2/d57/d37/f43 505891297 1 2024-05-27T09:29:40.515 INFO:tasks.workunit.client.0.smithi119.stdout:2/995: dwrite d2/da/d15/f18 [0,4194304] 0 2024-05-27T09:29:40.529 INFO:tasks.workunit.client.0.smithi119.stdout:6/970: read - d0/d2/d1c/d21/d58/f114 zero size 2024-05-27T09:29:40.531 INFO:tasks.workunit.client.0.smithi119.stdout:2/996: rename d2/d1a/l7c to d2/da/d8a/dd4/d89/d94/de2/l155 0 2024-05-27T09:29:40.536 INFO:tasks.workunit.client.0.smithi119.stdout:2/997: rename d2/da/d8a/d8f/d11e/d146 to d2/d1a/d156 0 2024-05-27T09:29:40.536 INFO:tasks.workunit.client.0.smithi119.stdout:2/998: dread - d2/da/d8a/dd4/d89/d94/d81/fdc zero size 2024-05-27T09:29:40.536 INFO:tasks.workunit.client.0.smithi119.stdout:2/999: chown d2/da/d8a/dd4/d89/d94/d87/fba 19127477 1 2024-05-27T09:29:40.536 INFO:tasks.workunit.client.0.smithi119.stdout:8/998: dwrite d3/d6e/dcf/dce/fd8 [0,4194304] 0 2024-05-27T09:29:40.541 INFO:tasks.workunit.client.0.smithi119.stdout:0/984: truncate d2/d6/d9/f3a 20406 0 2024-05-27T09:29:40.548 INFO:tasks.workunit.client.1.smithi138.stdout:6/648: dwrite d3/f53 [0,4194304] 0 2024-05-27T09:29:40.548 INFO:tasks.workunit.client.1.smithi138.stdout:6/649: write d3/d11/fd0 [524533,98349] 0 2024-05-27T09:29:40.552 INFO:tasks.workunit.client.0.smithi119.stdout:6/971: dread d0/d2/d4e/fc0 [0,4194304] 0 2024-05-27T09:29:40.558 INFO:tasks.workunit.client.0.smithi119.stdout:0/985: dread d2/d6/d9/d12/f77 [0,4194304] 0 2024-05-27T09:29:40.559 INFO:tasks.workunit.client.0.smithi119.stdout:6/972: link d0/d2/c33 d0/d2/d57/dd0/c15a 0 2024-05-27T09:29:40.560 INFO:tasks.workunit.client.0.smithi119.stdout:0/986: stat d2/d6/d23/d7a/d53/ca5 0 2024-05-27T09:29:40.561 INFO:tasks.workunit.client.0.smithi119.stdout:6/973: rmdir d0/d2/d57/d37/d115 39 2024-05-27T09:29:40.562 INFO:tasks.workunit.client.0.smithi119.stdout:6/974: write d0/d2/d1c/d21/f143 [152348,54780] 0 2024-05-27T09:29:40.565 INFO:tasks.workunit.client.0.smithi119.stdout:6/975: symlink d0/d2/d1c/l15b 0 2024-05-27T09:29:40.571 INFO:tasks.workunit.client.0.smithi119.stdout:6/976: link d0/d2/f159 d0/d2/d57/d37/d74/df1/d109/f15c 0 2024-05-27T09:29:40.571 INFO:tasks.workunit.client.0.smithi119.stdout:6/977: chown d0/d2/d1c/d21/ffd 50293 1 2024-05-27T09:29:40.573 INFO:tasks.workunit.client.0.smithi119.stdout:6/978: mkdir d0/d2/d1c/d21/d58/dd3/d15d 0 2024-05-27T09:29:40.574 INFO:tasks.workunit.client.1.smithi138.stdout:5/388: dread d3/d12/d16/f2e [0,4194304] 0 2024-05-27T09:29:40.575 INFO:tasks.workunit.client.1.smithi138.stdout:5/389: chown d3/fd 3 1 2024-05-27T09:29:40.575 INFO:tasks.workunit.client.1.smithi138.stdout:5/390: creat d3/d12/f77 x:0 0 0 2024-05-27T09:29:40.575 INFO:tasks.workunit.client.1.smithi138.stdout:5/391: readlink d3/d12/d16/d1d/d22/l65 0 2024-05-27T09:29:40.575 INFO:tasks.workunit.client.1.smithi138.stdout:5/392: write d3/d12/d16/d1d/f24 [1436474,101091] 0 2024-05-27T09:29:40.576 INFO:tasks.workunit.client.0.smithi119.stdout:6/979: creat d0/d2/d4e/d5a/d14d/f15e x:0 0 0 2024-05-27T09:29:40.586 INFO:tasks.workunit.client.1.smithi138.stdout:5/393: link d3/d12/d16/d1d/f24 d3/d12/d32/d6b/f78 0 2024-05-27T09:29:40.589 INFO:tasks.workunit.client.1.smithi138.stdout:4/527: sync 2024-05-27T09:29:40.589 INFO:tasks.workunit.client.1.smithi138.stdout:5/394: rmdir d3/d12/d16/d1d/d23/d68 39 2024-05-27T09:29:40.593 INFO:tasks.workunit.client.1.smithi138.stdout:4/528: rename d7/df/d35/d4c/d59/fa5 to d7/df/d35/d4c/d59/d5d/d64/fb7 0 2024-05-27T09:29:40.593 INFO:tasks.workunit.client.1.smithi138.stdout:4/529: getdents d7/df/d5b/da1/db1 0 2024-05-27T09:29:40.593 INFO:tasks.workunit.client.1.smithi138.stdout:7/468: dwrite d0/d7/f6b [0,4194304] 0 2024-05-27T09:29:40.601 INFO:tasks.workunit.client.1.smithi138.stdout:7/469: mkdir d0/d7/d60/d9d 0 2024-05-27T09:29:40.601 INFO:tasks.workunit.client.1.smithi138.stdout:4/530: symlink d7/lb8 0 2024-05-27T09:29:40.604 INFO:tasks.workunit.client.1.smithi138.stdout:4/531: link d7/d27/c49 d7/df/d5b/da1/db1/cb9 0 2024-05-27T09:29:40.611 INFO:tasks.workunit.client.0.smithi119.stdout:9/879: sync 2024-05-27T09:29:40.613 INFO:tasks.workunit.client.0.smithi119.stdout:9/880: creat d18/d19/dfa/f111 x:0 0 0 2024-05-27T09:29:40.617 INFO:tasks.workunit.client.0.smithi119.stdout:9/881: truncate fc 77793 0 2024-05-27T09:29:40.621 INFO:tasks.workunit.client.1.smithi138.stdout:6/650: dwrite d3/d11/d2d/d37/d7e/f9f [0,4194304] 0 2024-05-27T09:29:40.622 INFO:tasks.workunit.client.1.smithi138.stdout:6/651: truncate d3/d11/dc0/d43/f3f 645032 0 2024-05-27T09:29:40.622 INFO:tasks.workunit.client.0.smithi119.stdout:9/882: mknod d18/d19/d9b/d10b/c112 0 2024-05-27T09:29:40.622 INFO:tasks.workunit.client.0.smithi119.stdout:9/883: write d18/d19/d9b/fd5 [773095,46560] 0 2024-05-27T09:29:40.622 INFO:tasks.workunit.client.0.smithi119.stdout:9/884: write d18/db4/dbe/ff3 [324724,99558] 0 2024-05-27T09:29:40.625 INFO:tasks.workunit.client.1.smithi138.stdout:6/652: mkdir d3/d11/dc0/d7d/de3 0 2024-05-27T09:29:40.628 INFO:tasks.workunit.client.1.smithi138.stdout:6/653: creat d3/d11/dc0/d43/dbe/fe4 x:0 0 0 2024-05-27T09:29:40.629 INFO:tasks.workunit.client.1.smithi138.stdout:5/395: write d3/d12/d16/d1d/d22/f2a [1357320,75882] 0 2024-05-27T09:29:40.629 INFO:tasks.workunit.client.1.smithi138.stdout:5/396: creat d3/d12/f79 x:0 0 0 2024-05-27T09:29:40.631 INFO:tasks.workunit.client.0.smithi119.stdout:0/987: dwrite d2/d6/d23/d7a/f46 [0,4194304] 0 2024-05-27T09:29:40.631 INFO:tasks.workunit.client.0.smithi119.stdout:0/988: truncate d2/d6/d9/d24/d30/dcc/de6/d137/fa2 4823743 0 2024-05-27T09:29:40.632 INFO:tasks.workunit.client.1.smithi138.stdout:6/654: mknod d3/d19/ce5 0 2024-05-27T09:29:40.632 INFO:tasks.workunit.client.1.smithi138.stdout:6/655: dread - d3/d11/d2d/d37/dd1/d9b/fb7 zero size 2024-05-27T09:29:40.632 INFO:tasks.workunit.client.1.smithi138.stdout:6/656: truncate d3/d11/d2d/d37/daa/fb2 369514 0 2024-05-27T09:29:40.635 INFO:tasks.workunit.client.0.smithi119.stdout:0/989: creat d2/d6/d23/d7a/d53/f145 x:0 0 0 2024-05-27T09:29:40.636 INFO:tasks.workunit.client.1.smithi138.stdout:2/398: sync 2024-05-27T09:29:40.636 INFO:tasks.workunit.client.1.smithi138.stdout:8/485: sync 2024-05-27T09:29:40.638 INFO:tasks.workunit.client.1.smithi138.stdout:5/397: symlink d3/d12/d16/d1d/d22/l7a 0 2024-05-27T09:29:40.638 INFO:tasks.workunit.client.1.smithi138.stdout:2/399: getdents d1 0 2024-05-27T09:29:40.638 INFO:tasks.workunit.client.1.smithi138.stdout:2/400: write d1/d15/d2d/d17/d25/f6c [615107,52684] 0 2024-05-27T09:29:40.640 INFO:tasks.workunit.client.1.smithi138.stdout:6/657: unlink d3/d11/f88 0 2024-05-27T09:29:40.641 INFO:tasks.workunit.client.1.smithi138.stdout:8/486: creat d3/d8/d24/d4d/f97 x:0 0 0 2024-05-27T09:29:40.641 INFO:tasks.workunit.client.1.smithi138.stdout:8/487: truncate d3/d8/d32/d88/f8e 866340 0 2024-05-27T09:29:40.641 INFO:tasks.workunit.client.1.smithi138.stdout:8/488: chown d3/d8/d24/c7f 0 1 2024-05-27T09:29:40.641 INFO:tasks.workunit.client.1.smithi138.stdout:8/489: truncate d3/d8/d64/f5e 2877843 0 2024-05-27T09:29:40.644 INFO:tasks.workunit.client.0.smithi119.stdout:0/990: rename d2/d6/d23/db8/fda to d2/d6/d9/d12/da3/f146 0 2024-05-27T09:29:40.645 INFO:tasks.workunit.client.1.smithi138.stdout:2/401: rmdir d1/d2f 39 2024-05-27T09:29:40.645 INFO:tasks.workunit.client.1.smithi138.stdout:8/490: creat d3/d8/d6c/f98 x:0 0 0 2024-05-27T09:29:40.646 INFO:tasks.workunit.client.1.smithi138.stdout:2/402: dread - d1/d15/d2d/d31/f5b zero size 2024-05-27T09:29:40.647 INFO:tasks.workunit.client.1.smithi138.stdout:8/491: creat d3/d8/d11/f99 x:0 0 0 2024-05-27T09:29:40.647 INFO:tasks.workunit.client.1.smithi138.stdout:8/492: dread - d3/d8/d32/d45/f90 zero size 2024-05-27T09:29:40.648 INFO:tasks.workunit.client.0.smithi119.stdout:0/991: link d2/d6/d23/d7a/f11c d2/d6/d9/f147 0 2024-05-27T09:29:40.649 INFO:tasks.workunit.client.1.smithi138.stdout:8/493: creat d3/d8/d32/d34/f9a x:0 0 0 2024-05-27T09:29:40.658 INFO:tasks.workunit.client.1.smithi138.stdout:5/398: dread d3/d12/d26/f4b [0,4194304] 0 2024-05-27T09:29:40.658 INFO:tasks.workunit.client.1.smithi138.stdout:5/399: chown d3/d12/d16/d1d/d30/f4a 3311904 1 2024-05-27T09:29:40.668 INFO:tasks.workunit.client.0.smithi119.stdout:6/980: dwrite d0/d2/d1c/d21/d58/fd8 [0,4194304] 0 2024-05-27T09:29:40.671 INFO:tasks.workunit.client.1.smithi138.stdout:8/494: dread d3/d8/d32/d34/d78/f96 [0,4194304] 0 2024-05-27T09:29:40.678 INFO:tasks.workunit.client.1.smithi138.stdout:2/403: dread d1/d15/d2d/d2e/f37 [0,4194304] 0 2024-05-27T09:29:40.679 INFO:tasks.workunit.client.0.smithi119.stdout:6/981: truncate d0/d2/d57/f6d 2672015 0 2024-05-27T09:29:40.680 INFO:tasks.workunit.client.1.smithi138.stdout:2/404: creat d1/d10/d3c/f89 x:0 0 0 2024-05-27T09:29:40.680 INFO:tasks.workunit.client.1.smithi138.stdout:2/405: chown d1/d15/d2d/d1d/f43 205767250 1 2024-05-27T09:29:40.683 INFO:tasks.workunit.client.1.smithi138.stdout:2/406: link d1/d15/d2d/d17/d25/l26 d1/d10/l8a 0 2024-05-27T09:29:40.684 INFO:tasks.workunit.client.1.smithi138.stdout:2/407: truncate d1/d15/d2d/d17/d25/f28 2097046 0 2024-05-27T09:29:40.685 INFO:tasks.workunit.client.1.smithi138.stdout:2/408: chown d1/d15/d2d/d17/d25/d4d/c71 1784 1 2024-05-27T09:29:40.685 INFO:tasks.workunit.client.1.smithi138.stdout:5/400: dread d3/f18 [4194304,4194304] 0 2024-05-27T09:29:40.685 INFO:tasks.workunit.client.1.smithi138.stdout:5/401: write d3/d12/d26/f57 [22694,123081] 0 2024-05-27T09:29:40.686 INFO:tasks.workunit.client.1.smithi138.stdout:2/409: symlink d1/d15/d2d/d17/l8b 0 2024-05-27T09:29:40.687 INFO:tasks.workunit.client.1.smithi138.stdout:2/410: dread - d1/d10/d52/f73 zero size 2024-05-27T09:29:40.687 INFO:tasks.workunit.client.1.smithi138.stdout:2/411: readlink d1/d15/d2d/d17/d25/d4d/d4e/d60/l61 0 2024-05-27T09:29:40.687 INFO:tasks.workunit.client.1.smithi138.stdout:1/449: sync 2024-05-27T09:29:40.688 INFO:tasks.workunit.client.1.smithi138.stdout:2/412: symlink d1/d10/d52/d5d/d7e/l8c 0 2024-05-27T09:29:40.689 INFO:tasks.workunit.client.1.smithi138.stdout:2/413: chown d1/d15/d2d/d17/d25/l7f 99095243 1 2024-05-27T09:29:40.690 INFO:tasks.workunit.client.1.smithi138.stdout:2/414: symlink d1/d15/d2d/d2e/l8d 0 2024-05-27T09:29:40.692 INFO:tasks.workunit.client.1.smithi138.stdout:5/402: creat d3/d12/d32/d60/f7b x:0 0 0 2024-05-27T09:29:40.693 INFO:tasks.workunit.client.1.smithi138.stdout:2/415: creat d1/d15/d2d/f8e x:0 0 0 2024-05-27T09:29:40.697 INFO:tasks.workunit.client.0.smithi119.stdout:6/982: write d0/d2/d4e/d5a/faa [2109260,56070] 0 2024-05-27T09:29:40.700 INFO:tasks.workunit.client.1.smithi138.stdout:5/403: link d3/d12/d16/d1d/d22/d2b/d36/f43 d3/d12/d16/d1d/d23/d3b/f7c 0 2024-05-27T09:29:40.713 INFO:tasks.workunit.client.1.smithi138.stdout:5/404: mknod d3/c7d 0 2024-05-27T09:29:40.714 INFO:tasks.workunit.client.1.smithi138.stdout:7/470: dwrite d0/f45 [0,4194304] 0 2024-05-27T09:29:40.719 INFO:tasks.workunit.client.0.smithi119.stdout:9/885: dwrite d18/d19/d1b/d29/f57 [0,4194304] 0 2024-05-27T09:29:40.720 INFO:tasks.workunit.client.1.smithi138.stdout:5/405: write d3/d12/d16/f66 [5140032,122162] 0 2024-05-27T09:29:40.721 INFO:tasks.workunit.client.1.smithi138.stdout:8/495: write d3/d8/d24/d55/f67 [281107,79228] 0 2024-05-27T09:29:40.726 INFO:tasks.workunit.client.1.smithi138.stdout:6/658: dwrite d3/d11/d2d/d37/dd1/d47/f5e [0,4194304] 0 2024-05-27T09:29:40.726 INFO:tasks.workunit.client.1.smithi138.stdout:8/496: link d3/d8/c80 d3/d8/d32/d88/c9b 0 2024-05-27T09:29:40.726 INFO:tasks.workunit.client.1.smithi138.stdout:6/659: readlink d3/d11/lb8 0 2024-05-27T09:29:40.728 INFO:tasks.workunit.client.1.smithi138.stdout:8/497: rmdir d3/d8/d32/d34/d78 39 2024-05-27T09:29:40.728 INFO:tasks.workunit.client.0.smithi119.stdout:0/992: dwrite d2/d6/d23/db8/f103 [0,4194304] 0 2024-05-27T09:29:40.728 INFO:tasks.workunit.client.0.smithi119.stdout:0/993: read d2/d110/d96/d132/f58 [11813,9569] 0 2024-05-27T09:29:40.728 INFO:tasks.workunit.client.0.smithi119.stdout:0/994: chown d2/d110/d96/d132/dd7/f87 40785529 1 2024-05-27T09:29:40.728 INFO:tasks.workunit.client.0.smithi119.stdout:0/995: stat d2/d6/d9/d24/d30/c90 0 2024-05-27T09:29:40.730 INFO:tasks.workunit.client.1.smithi138.stdout:8/498: rename d3/d8/d32/d34/d78 to d3/d8/d32/d3f/d9c 0 2024-05-27T09:29:40.730 INFO:tasks.workunit.client.0.smithi119.stdout:9/886: stat d18/d19/d1b/l31 0 2024-05-27T09:29:40.731 INFO:tasks.workunit.client.1.smithi138.stdout:6/660: creat d3/d11/dc0/d5a/dcf/fe6 x:0 0 0 2024-05-27T09:29:40.731 INFO:tasks.workunit.client.1.smithi138.stdout:8/499: symlink d3/d8/d24/d55/l9d 0 2024-05-27T09:29:40.732 INFO:tasks.workunit.client.1.smithi138.stdout:3/486: sync 2024-05-27T09:29:40.732 INFO:tasks.workunit.client.0.smithi119.stdout:8/999: sync 2024-05-27T09:29:40.733 INFO:tasks.workunit.client.1.smithi138.stdout:8/500: mkdir d3/d8/d24/d55/d66/d9e 0 2024-05-27T09:29:40.734 INFO:tasks.workunit.client.1.smithi138.stdout:3/487: getdents d0/d2/da/df/d10/d32 0 2024-05-27T09:29:40.736 INFO:tasks.workunit.client.1.smithi138.stdout:3/488: getdents d0/d2/da/df/d10/d32/d37 0 2024-05-27T09:29:40.736 INFO:tasks.workunit.client.1.smithi138.stdout:3/489: dread - d0/d3f/f98 zero size 2024-05-27T09:29:40.736 INFO:tasks.workunit.client.1.smithi138.stdout:3/490: chown d0/d2/da/f2d 305474150 1 2024-05-27T09:29:40.736 INFO:tasks.workunit.client.1.smithi138.stdout:3/491: read d0/d2/fc [1128580,106742] 0 2024-05-27T09:29:40.736 INFO:tasks.workunit.client.1.smithi138.stdout:3/492: stat d0/d2/da/df/d10/d32/c46 0 2024-05-27T09:29:40.736 INFO:tasks.workunit.client.1.smithi138.stdout:3/493: fdatasync d0/fd 0 2024-05-27T09:29:40.739 INFO:tasks.workunit.client.1.smithi138.stdout:8/501: rename d3/d60 to d3/d8/d32/d3f/d9c/d9f 0 2024-05-27T09:29:40.743 INFO:tasks.workunit.client.1.smithi138.stdout:3/494: creat d0/d2/da/df/fa1 x:0 0 0 2024-05-27T09:29:40.744 INFO:tasks.workunit.client.1.smithi138.stdout:8/502: symlink d3/d8/d32/d45/d5c/la0 0 2024-05-27T09:29:40.746 INFO:tasks.workunit.client.1.smithi138.stdout:8/503: unlink d3/d8/d32/d34/d38/f63 0 2024-05-27T09:29:40.748 INFO:tasks.workunit.client.1.smithi138.stdout:8/504: symlink d3/d8/d24/d55/d94/la1 0 2024-05-27T09:29:40.752 INFO:tasks.workunit.client.1.smithi138.stdout:8/505: symlink d3/d8/d32/d3f/d9c/d9f/la2 0 2024-05-27T09:29:40.754 INFO:tasks.workunit.client.1.smithi138.stdout:5/406: write d3/f15 [1703534,71041] 0 2024-05-27T09:29:40.755 INFO:tasks.workunit.client.1.smithi138.stdout:8/506: rename d3/d8/d11/f22 to d3/d8/d24/d55/d66/fa3 0 2024-05-27T09:29:40.756 INFO:tasks.workunit.client.1.smithi138.stdout:8/507: symlink d3/d8/d24/d55/d94/la4 0 2024-05-27T09:29:40.758 INFO:tasks.workunit.client.1.smithi138.stdout:8/508: rename d3/d8/d24/d55/d66/d9e to d3/d8/d24/d55/d94/da5 0 2024-05-27T09:29:40.758 INFO:tasks.workunit.client.1.smithi138.stdout:8/509: chown d3/d8/d24/d55/d94/la4 471 1 2024-05-27T09:29:40.759 INFO:tasks.workunit.client.1.smithi138.stdout:5/407: mkdir d3/d12/d32/d7e 0 2024-05-27T09:29:40.759 INFO:tasks.workunit.client.1.smithi138.stdout:5/408: creat d3/d12/d16/d1d/d23/f7f x:0 0 0 2024-05-27T09:29:40.759 INFO:tasks.workunit.client.1.smithi138.stdout:8/510: rename d3/d8/d64/f40 to d3/d8/d11/fa6 0 2024-05-27T09:29:40.761 INFO:tasks.workunit.client.1.smithi138.stdout:8/511: creat d3/d8/d32/d3f/d9c/d9f/fa7 x:0 0 0 2024-05-27T09:29:40.761 INFO:tasks.workunit.client.1.smithi138.stdout:8/512: chown d3/d8/d32/f81 1103 1 2024-05-27T09:29:40.762 INFO:tasks.workunit.client.1.smithi138.stdout:8/513: write d3/d8/d64/f2e [2103968,29762] 0 2024-05-27T09:29:40.766 INFO:tasks.workunit.client.1.smithi138.stdout:5/409: unlink d3/d12/d16/d1d/l3f 0 2024-05-27T09:29:40.774 INFO:tasks.workunit.client.0.smithi119.stdout:0/996: write d2/d6/d23/d7a/f114 [711256,24318] 0 2024-05-27T09:29:40.777 INFO:tasks.workunit.client.1.smithi138.stdout:5/410: getdents d3/d12/d16 0 2024-05-27T09:29:40.777 INFO:tasks.workunit.client.1.smithi138.stdout:5/411: dread - d3/f4c zero size 2024-05-27T09:29:40.777 INFO:tasks.workunit.client.1.smithi138.stdout:5/412: dread - d3/d12/d16/d1d/d22/d2b/f38 zero size 2024-05-27T09:29:40.777 INFO:tasks.workunit.client.1.smithi138.stdout:5/413: read d3/f10 [796977,63640] 0 2024-05-27T09:29:40.777 INFO:tasks.workunit.client.1.smithi138.stdout:5/414: stat d3/d12/f31 0 2024-05-27T09:29:40.777 INFO:tasks.workunit.client.1.smithi138.stdout:5/415: dread - d3/d12/f77 zero size 2024-05-27T09:29:40.779 INFO:tasks.workunit.client.1.smithi138.stdout:3/495: dread d0/d2/da/df/d10/d32/f9 [4194304,4194304] 0 2024-05-27T09:29:40.781 INFO:tasks.workunit.client.1.smithi138.stdout:3/496: link d0/d3f/d60/l7a d0/d2/la2 0 2024-05-27T09:29:40.782 INFO:tasks.workunit.client.1.smithi138.stdout:3/497: write d0/d3f/d50/f76 [1856863,11040] 0 2024-05-27T09:29:40.782 INFO:tasks.workunit.client.1.smithi138.stdout:3/498: stat d0/d2/da/f2d 0 2024-05-27T09:29:40.783 INFO:tasks.workunit.client.1.smithi138.stdout:3/499: mkdir d0/d3f/d50/d9b/d9f/da3 0 2024-05-27T09:29:40.783 INFO:tasks.workunit.client.1.smithi138.stdout:3/500: dread - d0/d2/da/f94 zero size 2024-05-27T09:29:40.784 INFO:tasks.workunit.client.1.smithi138.stdout:3/501: fsync d0/d2/da/df/d10/d32/f71 0 2024-05-27T09:29:40.784 INFO:tasks.workunit.client.0.smithi119.stdout:0/997: rename d2/l3 to d2/d6/d9/d24/d30/dcc/de6/l148 0 2024-05-27T09:29:40.784 INFO:tasks.workunit.client.1.smithi138.stdout:4/532: dwrite d7/df/d35/d4c/d59/d5d/d64/d6a/f76 [0,4194304] 0 2024-05-27T09:29:40.785 INFO:tasks.workunit.client.1.smithi138.stdout:1/450: dwrite d4/d38/f44 [0,4194304] 0 2024-05-27T09:29:40.793 INFO:tasks.workunit.client.0.smithi119.stdout:0/998: creat d2/d6/d9/d24/d30/f149 x:0 0 0 2024-05-27T09:29:40.798 INFO:tasks.workunit.client.0.smithi119.stdout:6/983: dwrite d0/d2/d57/d37/f12a [4194304,4194304] 0 2024-05-27T09:29:40.804 INFO:tasks.workunit.client.1.smithi138.stdout:8/514: getdents d3/d8/d64 0 2024-05-27T09:29:40.806 INFO:tasks.workunit.client.1.smithi138.stdout:8/515: rename d3/d8/d32/l70 to d3/d8/d64/la8 0 2024-05-27T09:29:40.806 INFO:tasks.workunit.client.1.smithi138.stdout:5/416: symlink d3/d12/d16/d1d/d23/d3b/l80 0 2024-05-27T09:29:40.807 INFO:tasks.workunit.client.1.smithi138.stdout:8/516: write d3/d8/d6c/f98 [184446,126082] 0 2024-05-27T09:29:40.807 INFO:tasks.workunit.client.1.smithi138.stdout:8/517: fdatasync d3/d8/d24/f54 0 2024-05-27T09:29:40.808 INFO:tasks.workunit.client.0.smithi119.stdout:0/999: mkdir d2/d110/d96/d132/dd7/d14a 0 2024-05-27T09:29:40.814 INFO:tasks.workunit.client.0.smithi119.stdout:6/984: symlink d0/d2/d57/dd0/d12b/l15f 0 2024-05-27T09:29:40.814 INFO:tasks.workunit.client.0.smithi119.stdout:6/985: chown d0/d2/d57/dd0 128615 1 2024-05-27T09:29:40.818 INFO:tasks.workunit.client.0.smithi119.stdout:6/986: mkdir d0/d2/d57/dd0/d12b/d160 0 2024-05-27T09:29:40.819 INFO:tasks.workunit.client.1.smithi138.stdout:6/661: dwrite d3/d19/f89 [0,4194304] 0 2024-05-27T09:29:40.823 INFO:tasks.workunit.client.1.smithi138.stdout:1/451: mkdir d4/d1a/d2e/d32/d8c 0 2024-05-27T09:29:40.823 INFO:tasks.workunit.client.1.smithi138.stdout:8/518: dread d3/d8/d64/f2e [0,4194304] 0 2024-05-27T09:29:40.824 INFO:tasks.workunit.client.1.smithi138.stdout:2/416: dwrite d1/d15/d2d/d17/d25/f2a [0,4194304] 0 2024-05-27T09:29:40.826 INFO:tasks.workunit.client.0.smithi119.stdout:6/987: rename d0/d2/d57/d37/dad/db6/le6 to d0/d85/l161 0 2024-05-27T09:29:40.826 INFO:tasks.workunit.client.0.smithi119.stdout:6/988: chown d0/d11/l107 3157 1 2024-05-27T09:29:40.829 INFO:tasks.workunit.client.1.smithi138.stdout:5/417: symlink d3/d12/d16/d1d/d30/l81 0 2024-05-27T09:29:40.832 INFO:tasks.workunit.client.1.smithi138.stdout:8/519: mknod d3/d8/d24/ca9 0 2024-05-27T09:29:40.832 INFO:tasks.workunit.client.1.smithi138.stdout:8/520: chown d3/d8/l21 684313047 1 2024-05-27T09:29:40.834 INFO:tasks.workunit.client.1.smithi138.stdout:8/521: symlink d3/d8/d32/d34/laa 0 2024-05-27T09:29:40.835 INFO:tasks.workunit.client.1.smithi138.stdout:8/522: fsync d3/d8/d32/d3f/d9c/d9f/fa7 0 2024-05-27T09:29:40.836 INFO:tasks.workunit.client.1.smithi138.stdout:8/523: rmdir d3/d8/d6c 39 2024-05-27T09:29:40.837 INFO:tasks.workunit.client.1.smithi138.stdout:8/524: mkdir d3/d8/d32/d88/dab 0 2024-05-27T09:29:40.839 INFO:tasks.workunit.client.1.smithi138.stdout:9/491: sync 2024-05-27T09:29:40.839 INFO:tasks.workunit.client.1.smithi138.stdout:0/457: sync 2024-05-27T09:29:40.839 INFO:tasks.workunit.client.1.smithi138.stdout:0/458: truncate f1 9341527 0 2024-05-27T09:29:40.839 INFO:tasks.workunit.client.1.smithi138.stdout:8/525: creat d3/d8/fac x:0 0 0 2024-05-27T09:29:40.839 INFO:tasks.workunit.client.1.smithi138.stdout:9/492: write d6/da/f5d [1353761,116046] 0 2024-05-27T09:29:40.841 INFO:tasks.workunit.client.0.smithi119.stdout:6/989: creat d0/d2/d11b/f162 x:0 0 0 2024-05-27T09:29:40.844 INFO:tasks.workunit.client.1.smithi138.stdout:6/662: symlink d3/d11/dc2/le7 0 2024-05-27T09:29:40.844 INFO:tasks.workunit.client.1.smithi138.stdout:6/663: dread - d3/d11/dc0/d5a/fb3 zero size 2024-05-27T09:29:40.844 INFO:tasks.workunit.client.1.smithi138.stdout:6/664: truncate d3/d19/fab 904547 0 2024-05-27T09:29:40.846 INFO:tasks.workunit.client.1.smithi138.stdout:4/533: mkdir d7/dba 0 2024-05-27T09:29:40.846 INFO:tasks.workunit.client.1.smithi138.stdout:4/534: read d7/f39 [597065,111636] 0 2024-05-27T09:29:40.847 INFO:tasks.workunit.client.1.smithi138.stdout:1/452: mknod d4/d1a/c8d 0 2024-05-27T09:29:40.847 INFO:tasks.workunit.client.1.smithi138.stdout:1/453: chown d4/d3c 12869778 1 2024-05-27T09:29:40.847 INFO:tasks.workunit.client.1.smithi138.stdout:5/418: rename d3/d12/d16/d1d/d23/f7f to d3/d12/d32/d60/f82 0 2024-05-27T09:29:40.848 INFO:tasks.workunit.client.1.smithi138.stdout:6/665: unlink d3/d11/dc0/d5a/dcf/fe6 0 2024-05-27T09:29:40.848 INFO:tasks.workunit.client.1.smithi138.stdout:6/666: fdatasync d3/d11/dc0/d5a/fb3 0 2024-05-27T09:29:40.848 INFO:tasks.workunit.client.1.smithi138.stdout:6/667: fdatasync d3/fe2 0 2024-05-27T09:29:40.850 INFO:tasks.workunit.client.1.smithi138.stdout:9/493: rename d6/da/d2f/d35/d55/l7e to d6/da/la0 0 2024-05-27T09:29:40.850 INFO:tasks.workunit.client.1.smithi138.stdout:1/454: creat d4/d1a/d2e/d4a/f8e x:0 0 0 2024-05-27T09:29:40.850 INFO:tasks.workunit.client.1.smithi138.stdout:1/455: write d4/d1a/d2e/d4a/f80 [1021055,39900] 0 2024-05-27T09:29:40.852 INFO:tasks.workunit.client.1.smithi138.stdout:9/494: creat d6/db/de/d18/d27/d9a/fa1 x:0 0 0 2024-05-27T09:29:40.852 INFO:tasks.workunit.client.1.smithi138.stdout:5/419: symlink d3/d12/d16/d1d/d23/l83 0 2024-05-27T09:29:40.862 INFO:tasks.workunit.client.1.smithi138.stdout:4/535: read d7/df/d35/d4c/d59/d5d/d64/fb7 [1303627,25816] 0 2024-05-27T09:29:40.862 INFO:tasks.workunit.client.1.smithi138.stdout:4/536: fdatasync d7/df/d35/d4c/d59/d5d/d64/d6a/f76 0 2024-05-27T09:29:40.862 INFO:tasks.workunit.client.1.smithi138.stdout:4/537: write d7/df/d35/f92 [827129,54960] 0 2024-05-27T09:29:40.862 INFO:tasks.workunit.client.1.smithi138.stdout:4/538: write d7/df/f7b [367891,40984] 0 2024-05-27T09:29:40.870 INFO:tasks.workunit.client.1.smithi138.stdout:1/456: mkdir d4/d1a/d2e/d4a/d7e/d8f 0 2024-05-27T09:29:40.885 INFO:tasks.workunit.client.1.smithi138.stdout:5/420: unlink d3/d12/d16/d1d/d22/d2b/d36/c70 0 2024-05-27T09:29:40.885 INFO:tasks.workunit.client.1.smithi138.stdout:5/421: truncate d3/d12/d26/f45 1020359 0 2024-05-27T09:29:40.885 INFO:tasks.workunit.client.1.smithi138.stdout:5/422: truncate d3/f5b 1455021 0 2024-05-27T09:29:40.892 INFO:tasks.workunit.client.1.smithi138.stdout:4/539: read d7/f6f [184948,129864] 0 2024-05-27T09:29:40.896 INFO:tasks.workunit.client.1.smithi138.stdout:7/471: dwrite d0/f45 [0,4194304] 0 2024-05-27T09:29:40.898 INFO:tasks.workunit.client.0.smithi119.stdout:6/990: dwrite d0/d2/d57/d37/d74/f70 [0,4194304] 0 2024-05-27T09:29:40.898 INFO:tasks.workunit.client.0.smithi119.stdout:6/991: fsync d0/d11/fdb 0 2024-05-27T09:29:40.912 INFO:tasks.workunit.client.1.smithi138.stdout:8/526: dread d3/d8/d24/f44 [0,4194304] 0 2024-05-27T09:29:40.912 INFO:tasks.workunit.client.1.smithi138.stdout:8/527: write d3/d8/d6c/f98 [479927,61455] 0 2024-05-27T09:29:40.916 INFO:tasks.workunit.client.0.smithi119.stdout:6/992: mknod d0/d2/d57/c163 0 2024-05-27T09:29:40.917 INFO:tasks.workunit.client.1.smithi138.stdout:1/457: dread d4/d48/d39/f45 [0,4194304] 0 2024-05-27T09:29:40.921 INFO:tasks.workunit.client.1.smithi138.stdout:9/495: dwrite f1 [4194304,4194304] 0 2024-05-27T09:29:40.929 INFO:tasks.workunit.client.1.smithi138.stdout:5/423: dwrite d3/d12/d32/f4e [0,4194304] 0 2024-05-27T09:29:40.935 INFO:tasks.workunit.client.0.smithi119.stdout:6/993: dread d0/f149 [0,4194304] 0 2024-05-27T09:29:40.940 INFO:tasks.workunit.client.1.smithi138.stdout:2/417: dwrite d1/d15/d2d/d17/d25/d4d/d4e/d60/f66 [0,4194304] 0 2024-05-27T09:29:40.940 INFO:tasks.workunit.client.1.smithi138.stdout:2/418: dread - d1/d10/d52/d5d/d7e/f84 zero size 2024-05-27T09:29:40.940 INFO:tasks.workunit.client.1.smithi138.stdout:2/419: symlink d1/d2f/l8f 0 2024-05-27T09:29:40.940 INFO:tasks.workunit.client.1.smithi138.stdout:2/420: write d1/d10/f76 [773167,24579] 0 2024-05-27T09:29:40.940 INFO:tasks.workunit.client.1.smithi138.stdout:2/421: dread - d1/d15/d2d/d17/d25/d4d/d4e/f81 zero size 2024-05-27T09:29:40.940 INFO:tasks.workunit.client.1.smithi138.stdout:2/422: fsync d1/d15/d2d/d1d/f67 0 2024-05-27T09:29:40.941 INFO:tasks.workunit.client.1.smithi138.stdout:8/528: write d3/d8/d32/f81 [3343384,11211] 0 2024-05-27T09:29:40.941 INFO:tasks.workunit.client.1.smithi138.stdout:8/529: fdatasync d3/d8/d64/f7c 0 2024-05-27T09:29:40.941 INFO:tasks.workunit.client.1.smithi138.stdout:7/472: dread d0/d7/fe [0,4194304] 0 2024-05-27T09:29:40.943 INFO:tasks.workunit.client.1.smithi138.stdout:2/423: creat d1/d15/d2d/d2e/f90 x:0 0 0 2024-05-27T09:29:40.943 INFO:tasks.workunit.client.1.smithi138.stdout:1/458: dread d4/d3c/f6b [0,4194304] 0 2024-05-27T09:29:40.943 INFO:tasks.workunit.client.1.smithi138.stdout:2/424: write d1/d15/d2d/d2e/f50 [4204868,108350] 0 2024-05-27T09:29:40.945 INFO:tasks.workunit.client.1.smithi138.stdout:7/473: creat d0/d7/d57/f9e x:0 0 0 2024-05-27T09:29:40.949 INFO:tasks.workunit.client.1.smithi138.stdout:0/459: dwrite da/d2a/d8e/d43/f48 [0,4194304] 0 2024-05-27T09:29:40.951 INFO:tasks.workunit.client.1.smithi138.stdout:2/425: creat d1/d10/d52/d5d/f91 x:0 0 0 2024-05-27T09:29:40.951 INFO:tasks.workunit.client.1.smithi138.stdout:2/426: creat d1/d15/d2d/d17/d25/d4d/d4e/d60/f92 x:0 0 0 2024-05-27T09:29:40.951 INFO:tasks.workunit.client.1.smithi138.stdout:2/427: readlink d1/d15/d2d/d31/l5f 0 2024-05-27T09:29:40.952 INFO:tasks.workunit.client.1.smithi138.stdout:7/474: creat d0/d7/d60/f9f x:0 0 0 2024-05-27T09:29:40.953 INFO:tasks.workunit.client.0.smithi119.stdout:6/994: dread d0/f84 [0,4194304] 0 2024-05-27T09:29:40.953 INFO:tasks.workunit.client.0.smithi119.stdout:6/995: write d0/d2/d11b/f162 [878151,1943] 0 2024-05-27T09:29:40.958 INFO:tasks.workunit.client.0.smithi119.stdout:6/996: rmdir d0/d85/de7 39 2024-05-27T09:29:40.958 INFO:tasks.workunit.client.1.smithi138.stdout:2/428: rename d1/d15/d2d/d31/l5f to d1/d10/d52/l93 0 2024-05-27T09:29:40.958 INFO:tasks.workunit.client.1.smithi138.stdout:7/475: creat d0/d23/fa0 x:0 0 0 2024-05-27T09:29:40.958 INFO:tasks.workunit.client.1.smithi138.stdout:7/476: truncate d0/d23/d49/d78/f9b 1012358 0 2024-05-27T09:29:40.959 INFO:tasks.workunit.client.0.smithi119.stdout:6/997: unlink d0/d2/f96 0 2024-05-27T09:29:40.961 INFO:tasks.workunit.client.1.smithi138.stdout:0/460: dread da/d2a/f85 [0,4194304] 0 2024-05-27T09:29:40.961 INFO:tasks.workunit.client.1.smithi138.stdout:0/461: write da/d2a/d55/f61 [3289311,16440] 0 2024-05-27T09:29:40.961 INFO:tasks.workunit.client.1.smithi138.stdout:0/462: write da/f22 [1912190,92780] 0 2024-05-27T09:29:40.962 INFO:tasks.workunit.client.1.smithi138.stdout:2/429: rename d1/d15/d2d/d17/d25/d4d/f5c to d1/d15/d2d/d1d/f94 0 2024-05-27T09:29:40.962 INFO:tasks.workunit.client.1.smithi138.stdout:1/459: dread d4/d3c/f4e [0,4194304] 0 2024-05-27T09:29:40.963 INFO:tasks.workunit.client.1.smithi138.stdout:2/430: getdents d1/d15/d2d/d17/d25/d4d/d6d 0 2024-05-27T09:29:40.969 INFO:tasks.workunit.client.0.smithi119.stdout:6/998: dread d0/d2/d4e/dc5/f113 [0,4194304] 0 2024-05-27T09:29:40.969 INFO:tasks.workunit.client.1.smithi138.stdout:0/463: rename da/d2a/d2b/f50 to da/d2a/d8e/d1f/d42/d4a/f9b 0 2024-05-27T09:29:40.969 INFO:tasks.workunit.client.1.smithi138.stdout:0/464: chown f1 2501539 1 2024-05-27T09:29:40.971 INFO:tasks.workunit.client.1.smithi138.stdout:8/530: dread f1 [0,4194304] 0 2024-05-27T09:29:40.971 INFO:tasks.workunit.client.1.smithi138.stdout:1/460: mknod d4/d48/d39/d5f/d86/c90 0 2024-05-27T09:29:40.971 INFO:tasks.workunit.client.1.smithi138.stdout:1/461: fsync d4/d3c/f4e 0 2024-05-27T09:29:40.971 INFO:tasks.workunit.client.1.smithi138.stdout:1/462: chown d4/f78 427 1 2024-05-27T09:29:40.972 INFO:tasks.workunit.client.0.smithi119.stdout:9/887: sync 2024-05-27T09:29:40.972 INFO:tasks.workunit.client.0.smithi119.stdout:9/888: write d18/d19/d61/fed [1071229,110133] 0 2024-05-27T09:29:40.972 INFO:tasks.workunit.client.0.smithi119.stdout:9/889: read d18/f23 [678973,18374] 0 2024-05-27T09:29:40.973 INFO:tasks.workunit.client.1.smithi138.stdout:0/465: unlink da/d2a/d8e/d43/f62 0 2024-05-27T09:29:40.973 INFO:tasks.workunit.client.1.smithi138.stdout:0/466: chown da/d2a/d2b/l36 373640 1 2024-05-27T09:29:40.973 INFO:tasks.workunit.client.1.smithi138.stdout:2/431: rename d1/d15/d2d/d17/d25/f6c to d1/d15/d2d/d7b/f95 0 2024-05-27T09:29:40.977 INFO:tasks.workunit.client.0.smithi119.stdout:9/890: mknod d18/d19/d1b/d2a/d63/c113 0 2024-05-27T09:29:40.977 INFO:tasks.workunit.client.0.smithi119.stdout:9/891: truncate d18/d19/d1b/d40/d44/fd3 725321 0 2024-05-27T09:29:40.977 INFO:tasks.workunit.client.0.smithi119.stdout:9/892: creat d18/d19/d61/f114 x:0 0 0 2024-05-27T09:29:40.980 INFO:tasks.workunit.client.1.smithi138.stdout:9/496: dwrite d6/db/de/f2b [0,4194304] 0 2024-05-27T09:29:40.980 INFO:tasks.workunit.client.0.smithi119.stdout:9/893: rename d18/fa6 to d18/d19/d1b/f115 0 2024-05-27T09:29:40.982 INFO:tasks.workunit.client.0.smithi119.stdout:9/894: mkdir d18/d19/d1b/d40/d116 0 2024-05-27T09:29:40.982 INFO:tasks.workunit.client.0.smithi119.stdout:9/895: fsync d18/d19/d1b/f96 0 2024-05-27T09:29:40.984 INFO:tasks.workunit.client.1.smithi138.stdout:7/477: dread d0/f45 [0,4194304] 0 2024-05-27T09:29:40.984 INFO:tasks.workunit.client.1.smithi138.stdout:2/432: mknod d1/d15/d2d/d7b/c96 0 2024-05-27T09:29:40.984 INFO:tasks.workunit.client.1.smithi138.stdout:6/668: dwrite d3/d11/dc0/f60 [0,4194304] 0 2024-05-27T09:29:40.985 INFO:tasks.workunit.client.1.smithi138.stdout:9/497: fsync d6/da/d2f/f8e 0 2024-05-27T09:29:40.986 INFO:tasks.workunit.client.1.smithi138.stdout:0/467: link da/d2a/d8e/d1f/d4f/c78 da/d2a/d8e/d1f/c9c 0 2024-05-27T09:29:40.986 INFO:tasks.workunit.client.1.smithi138.stdout:0/468: fdatasync da/f3c 0 2024-05-27T09:29:40.986 INFO:tasks.workunit.client.1.smithi138.stdout:9/498: fsync d6/da/d29/f89 0 2024-05-27T09:29:40.986 INFO:tasks.workunit.client.1.smithi138.stdout:9/499: fdatasync d6/da/d29/d61/f68 0 2024-05-27T09:29:40.987 INFO:tasks.workunit.client.1.smithi138.stdout:9/500: write d6/da/d2f/d35/f80 [71968,9499] 0 2024-05-27T09:29:40.989 INFO:tasks.workunit.client.1.smithi138.stdout:2/433: creat d1/d15/d2d/d7b/f97 x:0 0 0 2024-05-27T09:29:40.990 INFO:tasks.workunit.client.1.smithi138.stdout:6/669: mknod d3/d11/d33/ce8 0 2024-05-27T09:29:40.990 INFO:tasks.workunit.client.1.smithi138.stdout:6/670: readlink d3/d11/dc0/d43/dbe/ldc 0 2024-05-27T09:29:40.990 INFO:tasks.workunit.client.1.smithi138.stdout:6/671: chown d3/d11/dc0/d43/f29 6495014 1 2024-05-27T09:29:40.990 INFO:tasks.workunit.client.1.smithi138.stdout:6/672: fdatasync d3/d11/d2d/f39 0 2024-05-27T09:29:40.995 INFO:tasks.workunit.client.1.smithi138.stdout:8/531: dread d3/d8/d6c/f98 [0,4194304] 0 2024-05-27T09:29:40.997 INFO:tasks.workunit.client.1.smithi138.stdout:6/673: creat d3/d11/dc0/d5a/fe9 x:0 0 0 2024-05-27T09:29:40.998 INFO:tasks.workunit.client.1.smithi138.stdout:6/674: creat d3/d11/dc0/d43/d71/fea x:0 0 0 2024-05-27T09:29:41.001 INFO:tasks.workunit.client.0.smithi119.stdout:9/896: dread d18/d19/d1b/d2a/d63/ff9 [0,4194304] 0 2024-05-27T09:29:41.002 INFO:tasks.workunit.client.1.smithi138.stdout:6/675: truncate d3/d11/dc0/d43/d1c/f59 4873407 0 2024-05-27T09:29:41.004 INFO:tasks.workunit.client.1.smithi138.stdout:1/463: dread d4/d48/d39/d5f/d4d/f3f [0,4194304] 0 2024-05-27T09:29:41.004 INFO:tasks.workunit.client.1.smithi138.stdout:2/434: dread d1/d10/d52/d5d/f88 [0,4194304] 0 2024-05-27T09:29:41.009 INFO:tasks.workunit.client.0.smithi119.stdout:9/897: mknod d18/d19/d1b/d40/c117 0 2024-05-27T09:29:41.009 INFO:tasks.workunit.client.0.smithi119.stdout:9/898: stat d18/d19/d1b/d2a/da2/f110 0 2024-05-27T09:29:41.010 INFO:tasks.workunit.client.1.smithi138.stdout:6/676: symlink d3/d11/d2d/d37/leb 0 2024-05-27T09:29:41.010 INFO:tasks.workunit.client.1.smithi138.stdout:1/464: mknod d4/c91 0 2024-05-27T09:29:41.013 INFO:tasks.workunit.client.0.smithi119.stdout:9/899: mkdir d18/d19/d1b/d40/d7b/dc6/d118 0 2024-05-27T09:29:41.013 INFO:tasks.workunit.client.0.smithi119.stdout:9/900: truncate d18/d19/d1b/d40/d44/f10d 576556 0 2024-05-27T09:29:41.016 INFO:tasks.workunit.client.1.smithi138.stdout:6/677: link d3/d19/c44 d3/d11/dc0/d74/de0/cec 0 2024-05-27T09:29:41.016 INFO:tasks.workunit.client.1.smithi138.stdout:6/678: truncate d3/d11/dc0/fd3 743177 0 2024-05-27T09:29:41.016 INFO:tasks.workunit.client.1.smithi138.stdout:8/532: dread d3/d8/d64/f5e [0,4194304] 0 2024-05-27T09:29:41.016 INFO:tasks.workunit.client.1.smithi138.stdout:8/533: fsync d3/d8/d11/fa6 0 2024-05-27T09:29:41.016 INFO:tasks.workunit.client.1.smithi138.stdout:8/534: creat d3/d8/d32/d34/d38/fad x:0 0 0 2024-05-27T09:29:41.018 INFO:tasks.workunit.client.1.smithi138.stdout:1/465: mknod d4/d48/d39/d5f/d4d/c92 0 2024-05-27T09:29:41.021 INFO:tasks.workunit.client.0.smithi119.stdout:9/901: symlink d18/d19/d1b/d29/dc4/l119 0 2024-05-27T09:29:41.021 INFO:tasks.workunit.client.0.smithi119.stdout:9/902: readlink d18/l67 0 2024-05-27T09:29:41.022 INFO:tasks.workunit.client.1.smithi138.stdout:7/478: dread d0/f10 [0,4194304] 0 2024-05-27T09:29:41.022 INFO:tasks.workunit.client.1.smithi138.stdout:7/479: dread - d0/d7/d60/f9f zero size 2024-05-27T09:29:41.024 INFO:tasks.workunit.client.1.smithi138.stdout:7/480: mknod d0/d23/d3c/ca1 0 2024-05-27T09:29:41.025 INFO:tasks.workunit.client.1.smithi138.stdout:5/424: dwrite d3/d12/d26/f5d [0,4194304] 0 2024-05-27T09:29:41.030 INFO:tasks.workunit.client.1.smithi138.stdout:1/466: mkdir d4/d48/d39/d5f/d93 0 2024-05-27T09:29:41.034 INFO:tasks.workunit.client.1.smithi138.stdout:0/469: dread f1 [4194304,4194304] 0 2024-05-27T09:29:41.037 INFO:tasks.workunit.client.0.smithi119.stdout:9/903: rename f2 to d18/d19/d1b/d40/d44/dcc/f11a 0 2024-05-27T09:29:41.037 INFO:tasks.workunit.client.0.smithi119.stdout:9/904: read d18/d19/d1b/d29/f71 [3007942,61196] 0 2024-05-27T09:29:41.040 INFO:tasks.workunit.client.1.smithi138.stdout:6/679: dread d3/d11/d2d/d37/f4a [0,4194304] 0 2024-05-27T09:29:41.042 INFO:tasks.workunit.client.0.smithi119.stdout:9/905: mkdir d18/d19/d1b/d29/d11b 0 2024-05-27T09:29:41.045 INFO:tasks.workunit.client.1.smithi138.stdout:2/435: write d1/d15/d2d/d31/d4b/f51 [2336560,119761] 0 2024-05-27T09:29:41.046 INFO:tasks.workunit.client.0.smithi119.stdout:6/999: write d0/f18 [2049182,101974] 0 2024-05-27T09:29:41.047 INFO:tasks.workunit.client.1.smithi138.stdout:5/425: creat d3/d12/d32/f84 x:0 0 0 2024-05-27T09:29:41.048 INFO:tasks.workunit.client.1.smithi138.stdout:1/467: mknod d4/d1a/d2e/d4a/c94 0 2024-05-27T09:29:41.048 INFO:tasks.workunit.client.1.smithi138.stdout:1/468: chown d4/d48/d39/d5f/d4d/c84 1 1 2024-05-27T09:29:41.048 INFO:tasks.workunit.client.1.smithi138.stdout:1/469: chown d4/d1a/d2e 0 1 2024-05-27T09:29:41.052 INFO:tasks.workunit.client.1.smithi138.stdout:6/680: mkdir d3/d11/d2d/d37/dd1/ded 0 2024-05-27T09:29:41.057 INFO:tasks.workunit.client.1.smithi138.stdout:5/426: link d3/d12/d16/d1d/f55 d3/d12/d26/d53/f85 0 2024-05-27T09:29:41.057 INFO:tasks.workunit.client.1.smithi138.stdout:5/427: truncate d3/f21 4684637 0 2024-05-27T09:29:41.058 INFO:tasks.workunit.client.1.smithi138.stdout:1/470: mknod d4/d3c/c95 0 2024-05-27T09:29:41.058 INFO:tasks.workunit.client.1.smithi138.stdout:1/471: write d4/d1a/d2e/d32/f7a [461988,130637] 0 2024-05-27T09:29:41.059 INFO:tasks.workunit.client.1.smithi138.stdout:1/472: chown d4/d48/d39/d5f/f6a 22 1 2024-05-27T09:29:41.059 INFO:tasks.workunit.client.1.smithi138.stdout:1/473: creat d4/d3c/d51/f96 x:0 0 0 2024-05-27T09:29:41.059 INFO:tasks.workunit.client.1.smithi138.stdout:1/474: readlink d4/l1d 0 2024-05-27T09:29:41.061 INFO:tasks.workunit.client.1.smithi138.stdout:5/428: mknod d3/d12/d16/d1d/d22/d2b/c86 0 2024-05-27T09:29:41.069 INFO:tasks.workunit.client.1.smithi138.stdout:9/501: dwrite d6/db/de/d18/d24/f7f [0,4194304] 0 2024-05-27T09:29:41.073 INFO:tasks.workunit.client.1.smithi138.stdout:2/436: getdents d1/d15/d2d/d17/d25/d4d 0 2024-05-27T09:29:41.076 INFO:tasks.workunit.client.1.smithi138.stdout:0/470: write da/d2a/d8e/d1f/d42/d4a/f4e [2940639,109653] 0 2024-05-27T09:29:41.078 INFO:tasks.workunit.client.1.smithi138.stdout:2/437: truncate d1/d15/d2d/d17/f58 1646464 0 2024-05-27T09:29:41.079 INFO:tasks.workunit.client.1.smithi138.stdout:0/471: mkdir da/d2a/d8e/d1f/d42/d63/d9d 0 2024-05-27T09:29:41.081 INFO:tasks.workunit.client.1.smithi138.stdout:2/438: rename d1/d15/d2d/c68 to d1/d2f/c98 0 2024-05-27T09:29:41.085 INFO:tasks.workunit.client.1.smithi138.stdout:0/472: creat da/d2a/d8e/d43/d74/d97/f9e x:0 0 0 2024-05-27T09:29:41.086 INFO:tasks.workunit.client.1.smithi138.stdout:7/481: dwrite d0/d23/f47 [0,4194304] 0 2024-05-27T09:29:41.086 INFO:tasks.workunit.client.1.smithi138.stdout:5/429: dread d3/d12/d26/f33 [0,4194304] 0 2024-05-27T09:29:41.086 INFO:tasks.workunit.client.1.smithi138.stdout:6/681: dread d3/d11/fe [0,4194304] 0 2024-05-27T09:29:41.086 INFO:tasks.workunit.client.1.smithi138.stdout:1/475: write d4/d48/d39/f54 [2242245,71764] 0 2024-05-27T09:29:41.088 INFO:tasks.workunit.client.1.smithi138.stdout:7/482: creat d0/d23/d59/fa2 x:0 0 0 2024-05-27T09:29:41.090 INFO:tasks.workunit.client.1.smithi138.stdout:6/682: symlink d3/d11/dc0/d43/d1c/lee 0 2024-05-27T09:29:41.090 INFO:tasks.workunit.client.1.smithi138.stdout:6/683: readlink d3/d11/dc0/d43/l98 0 2024-05-27T09:29:41.098 INFO:tasks.workunit.client.1.smithi138.stdout:5/430: mkdir d3/d12/d32/d87 0 2024-05-27T09:29:41.098 INFO:tasks.workunit.client.1.smithi138.stdout:7/483: symlink d0/d7/d60/la3 0 2024-05-27T09:29:41.100 INFO:tasks.workunit.client.1.smithi138.stdout:5/431: getdents d3/d12/d16/d1d/d23 0 2024-05-27T09:29:41.101 INFO:tasks.workunit.client.1.smithi138.stdout:5/432: dread - d3/d12/f64 zero size 2024-05-27T09:29:41.104 INFO:tasks.workunit.client.1.smithi138.stdout:5/433: link d3/f19 d3/f88 0 2024-05-27T09:29:41.104 INFO:tasks.workunit.client.1.smithi138.stdout:0/473: dread da/d2a/d8e/d1f/f38 [0,4194304] 0 2024-05-27T09:29:41.106 INFO:tasks.workunit.client.1.smithi138.stdout:0/474: getdents da/d2a/d8e/d6a 0 2024-05-27T09:29:41.110 INFO:tasks.workunit.client.1.smithi138.stdout:0/475: unlink da/le 0 2024-05-27T09:29:41.110 INFO:tasks.workunit.client.1.smithi138.stdout:0/476: fsync da/d2a/d8e/d1f/d42/d4a/f9b 0 2024-05-27T09:29:41.113 INFO:tasks.workunit.client.1.smithi138.stdout:7/484: dread d0/f1d [0,4194304] 0 2024-05-27T09:29:41.114 INFO:tasks.workunit.client.1.smithi138.stdout:0/477: symlink da/d2a/d8e/d1f/d42/d63/l9f 0 2024-05-27T09:29:41.117 INFO:tasks.workunit.client.0.smithi119.stdout:9/906: dwrite d18/d19/d1b/d40/d44/fdd [0,4194304] 0 2024-05-27T09:29:41.118 INFO:tasks.workunit.client.1.smithi138.stdout:1/476: write d4/d38/f44 [3897758,45568] 0 2024-05-27T09:29:41.118 INFO:tasks.workunit.client.1.smithi138.stdout:8/535: dwrite d3/d8/d64/f7c [0,4194304] 0 2024-05-27T09:29:41.118 INFO:tasks.workunit.client.1.smithi138.stdout:8/536: chown d3/d8/d32/d45/d5c 26 1 2024-05-27T09:29:41.118 INFO:tasks.workunit.client.1.smithi138.stdout:0/478: mknod da/d2a/d8e/d43/d74/ca0 0 2024-05-27T09:29:41.118 INFO:tasks.workunit.client.1.smithi138.stdout:1/477: mkdir d4/d38/d97 0 2024-05-27T09:29:41.121 INFO:tasks.workunit.client.1.smithi138.stdout:0/479: unlink da/d2a/d8e/d1f/d42/d4a/f9b 0 2024-05-27T09:29:41.121 INFO:tasks.workunit.client.1.smithi138.stdout:0/480: fdatasync da/d2a/d2b/f4b 0 2024-05-27T09:29:41.121 INFO:tasks.workunit.client.1.smithi138.stdout:0/481: stat da/d2a/d8e/d1f/d4f/l88 0 2024-05-27T09:29:41.124 INFO:tasks.workunit.client.1.smithi138.stdout:8/537: mkdir d3/d8/d32/d88/dab/dae 0 2024-05-27T09:29:41.126 INFO:tasks.workunit.client.1.smithi138.stdout:8/538: mknod d3/d8/d32/d34/d38/caf 0 2024-05-27T09:29:41.128 INFO:tasks.workunit.client.1.smithi138.stdout:8/539: creat d3/d8/d24/d55/d7e/d83/fb0 x:0 0 0 2024-05-27T09:29:41.139 INFO:tasks.workunit.client.1.smithi138.stdout:1/478: creat d4/d48/d39/d5f/d63/f98 x:0 0 0 2024-05-27T09:29:41.155 INFO:tasks.workunit.client.0.smithi119.stdout:9/907: creat d18/db4/f11c x:0 0 0 2024-05-27T09:29:41.159 INFO:tasks.workunit.client.1.smithi138.stdout:0/482: unlink da/c14 0 2024-05-27T09:29:41.161 INFO:tasks.workunit.client.1.smithi138.stdout:2/439: dwrite d1/d15/d2d/d17/d25/d4d/d4e/d60/f66 [0,4194304] 0 2024-05-27T09:29:41.162 INFO:tasks.workunit.client.1.smithi138.stdout:2/440: creat d1/d15/d2d/d1d/f99 x:0 0 0 2024-05-27T09:29:41.163 INFO:tasks.workunit.client.1.smithi138.stdout:2/441: chown d1/d15/d2d/d17/d25/d4d/c6f 134 1 2024-05-27T09:29:41.163 INFO:tasks.workunit.client.1.smithi138.stdout:2/442: truncate d1/d15/d2d/d7b/f97 763234 0 2024-05-27T09:29:41.163 INFO:tasks.workunit.client.1.smithi138.stdout:0/483: symlink da/d2a/d55/la1 0 2024-05-27T09:29:41.163 INFO:tasks.workunit.client.1.smithi138.stdout:0/484: chown da/d17/f44 2793 1 2024-05-27T09:29:41.163 INFO:tasks.workunit.client.1.smithi138.stdout:2/443: rename d1/c12 to d1/d15/d2d/d17/d25/d4d/c9a 0 2024-05-27T09:29:41.178 INFO:tasks.workunit.client.1.smithi138.stdout:0/485: creat da/d2a/d8e/d1f/fa2 x:0 0 0 2024-05-27T09:29:41.178 INFO:tasks.workunit.client.1.smithi138.stdout:0/486: mknod da/d2a/d8e/d1f/d42/d63/d9d/ca3 0 2024-05-27T09:29:41.181 INFO:tasks.workunit.client.1.smithi138.stdout:0/487: link da/d2a/d8e/d1f/d42/d63/d9d/ca3 da/d2a/d55/ca4 0 2024-05-27T09:29:41.183 INFO:tasks.workunit.client.1.smithi138.stdout:2/444: read d1/d15/d2d/d7b/f97 [109387,69758] 0 2024-05-27T09:29:41.184 INFO:tasks.workunit.client.1.smithi138.stdout:2/445: creat d1/d15/d2d/d17/f9b x:0 0 0 2024-05-27T09:29:41.187 INFO:tasks.workunit.client.1.smithi138.stdout:2/446: unlink d1/d15/d2d/d17/d25/d4d/c71 0 2024-05-27T09:29:41.190 INFO:tasks.workunit.client.1.smithi138.stdout:2/447: mknod d1/d15/d2d/d17/d25/d4d/d4e/d60/c9c 0 2024-05-27T09:29:41.199 INFO:tasks.workunit.client.1.smithi138.stdout:2/448: rename d1/d15/d2d/d31 to d1/d15/d2d/d17/d25/d9d 0 2024-05-27T09:29:41.202 INFO:tasks.workunit.client.0.smithi119.stdout:9/908: dread d18/d19/d1b/d2a/f8a [0,4194304] 0 2024-05-27T09:29:41.202 INFO:tasks.workunit.client.0.smithi119.stdout:9/909: creat d18/d19/d1b/d40/db7/f11d x:0 0 0 2024-05-27T09:29:41.202 INFO:tasks.workunit.client.0.smithi119.stdout:9/910: fdatasync d18/d19/d1b/d40/db7/f10a 0 2024-05-27T09:29:41.204 INFO:tasks.workunit.client.1.smithi138.stdout:2/449: link d1/d15/d2d/d17/d25/d9d/f5b d1/d15/d2d/d17/d25/d4d/d4e/f9e 0 2024-05-27T09:29:41.204 INFO:tasks.workunit.client.1.smithi138.stdout:2/450: stat d1/d15/d2d/d17/c70 0 2024-05-27T09:29:41.204 INFO:tasks.workunit.client.1.smithi138.stdout:2/451: fsync d1/d15/d2d/d17/d25/d9d/d4b/f51 0 2024-05-27T09:29:41.207 INFO:tasks.workunit.client.0.smithi119.stdout:9/911: creat d18/d19/d1b/d29/f11e x:0 0 0 2024-05-27T09:29:41.215 INFO:tasks.workunit.client.0.smithi119.stdout:9/912: link d18/ddf/fba d18/d19/d1b/d40/f11f 0 2024-05-27T09:29:41.215 INFO:tasks.workunit.client.1.smithi138.stdout:5/434: dwrite d3/d12/d16/f69 [0,4194304] 0 2024-05-27T09:29:41.217 INFO:tasks.workunit.client.0.smithi119.stdout:9/913: creat d18/d19/d1b/d29/d11b/f120 x:0 0 0 2024-05-27T09:29:41.218 INFO:tasks.workunit.client.1.smithi138.stdout:8/540: dwrite d3/d8/d32/d3f/f6b [0,4194304] 0 2024-05-27T09:29:41.218 INFO:tasks.workunit.client.1.smithi138.stdout:1/479: dwrite d4/d48/f14 [0,4194304] 0 2024-05-27T09:29:41.218 INFO:tasks.workunit.client.1.smithi138.stdout:1/480: getdents d4/d1a/d2e/d4a/d7e/d8f 0 2024-05-27T09:29:41.223 INFO:tasks.workunit.client.1.smithi138.stdout:9/502: dwrite d6/da/d2f/f8b [0,4194304] 0 2024-05-27T09:29:41.223 INFO:tasks.workunit.client.1.smithi138.stdout:9/503: chown d6/da/d29/d59 57035701 1 2024-05-27T09:29:41.228 INFO:tasks.workunit.client.1.smithi138.stdout:9/504: mkdir d6/db/de/d18/d27/da2 0 2024-05-27T09:29:41.231 INFO:tasks.workunit.client.1.smithi138.stdout:9/505: mknod d6/db/de/d18/d27/d9a/ca3 0 2024-05-27T09:29:41.236 INFO:tasks.workunit.client.1.smithi138.stdout:5/435: mkdir d3/d12/d32/d87/d89 0 2024-05-27T09:29:41.236 INFO:tasks.workunit.client.1.smithi138.stdout:9/506: truncate d6/db/de/f31 680353 0 2024-05-27T09:29:41.237 INFO:tasks.workunit.client.1.smithi138.stdout:2/452: dread d1/d15/d2d/d17/d25/d9d/d4b/f51 [0,4194304] 0 2024-05-27T09:29:41.253 INFO:tasks.workunit.client.1.smithi138.stdout:1/481: dread d4/d48/d22/d65/f6f [0,4194304] 0 2024-05-27T09:29:41.262 INFO:tasks.workunit.client.1.smithi138.stdout:1/482: rename d4/d48/c25 to d4/d1a/d2e/d32/c99 0 2024-05-27T09:29:41.262 INFO:tasks.workunit.client.1.smithi138.stdout:1/483: dread - d4/d48/d39/d5f/d63/f98 zero size 2024-05-27T09:29:41.274 INFO:tasks.workunit.client.1.smithi138.stdout:8/541: dread d3/d8/d24/d4d/f61 [0,4194304] 0 2024-05-27T09:29:41.274 INFO:tasks.workunit.client.1.smithi138.stdout:8/542: read d3/d8/f42 [171235,40178] 0 2024-05-27T09:29:41.275 INFO:tasks.workunit.client.1.smithi138.stdout:1/484: mknod d4/d38/d97/c9a 0 2024-05-27T09:29:41.275 INFO:tasks.workunit.client.1.smithi138.stdout:8/543: symlink d3/d8/d32/d34/d38/lb1 0 2024-05-27T09:29:41.276 INFO:tasks.workunit.client.1.smithi138.stdout:8/544: mknod d3/d8/d32/d34/d38/cb2 0 2024-05-27T09:29:41.278 INFO:tasks.workunit.client.1.smithi138.stdout:1/485: creat d4/f9b x:0 0 0 2024-05-27T09:29:41.278 INFO:tasks.workunit.client.1.smithi138.stdout:1/486: read d4/d48/d22/f35 [4846415,118963] 0 2024-05-27T09:29:41.279 INFO:tasks.workunit.client.1.smithi138.stdout:2/453: read d1/d15/d2d/d17/d25/d4d/d4e/d60/f66 [3812678,22602] 0 2024-05-27T09:29:41.279 INFO:tasks.workunit.client.1.smithi138.stdout:8/545: creat d3/d8/d24/d55/d94/da5/fb3 x:0 0 0 2024-05-27T09:29:41.283 INFO:tasks.workunit.client.1.smithi138.stdout:2/454: creat d1/d15/d2d/f9f x:0 0 0 2024-05-27T09:29:41.283 INFO:tasks.workunit.client.1.smithi138.stdout:0/488: dwrite da/f10 [0,4194304] 0 2024-05-27T09:29:41.283 INFO:tasks.workunit.client.1.smithi138.stdout:0/489: dread - da/d2a/d55/f95 zero size 2024-05-27T09:29:41.283 INFO:tasks.workunit.client.1.smithi138.stdout:1/487: rename d4/d1a/d2e/d4a/d7e/d8f to d4/d88/d9c 0 2024-05-27T09:29:41.284 INFO:tasks.workunit.client.1.smithi138.stdout:8/546: mkdir d3/db4 0 2024-05-27T09:29:41.289 INFO:tasks.workunit.client.1.smithi138.stdout:0/490: link da/d2a/d55/ca4 da/d2a/d8e/d1f/d4f/ca5 0 2024-05-27T09:29:41.290 INFO:tasks.workunit.client.1.smithi138.stdout:1/488: symlink d4/d1a/d2e/d4a/l9d 0 2024-05-27T09:29:41.293 INFO:tasks.workunit.client.1.smithi138.stdout:6/684: dwrite d3/d11/dc0/d43/f3f [0,4194304] 0 2024-05-27T09:29:41.305 INFO:tasks.workunit.client.1.smithi138.stdout:2/455: dread d1/d15/d2d/f39 [0,4194304] 0 2024-05-27T09:29:41.307 INFO:tasks.workunit.client.1.smithi138.stdout:0/491: write da/d2a/f30 [2723941,80665] 0 2024-05-27T09:29:41.319 INFO:tasks.workunit.client.0.smithi119.stdout:9/914: dwrite d18/d19/d1b/d29/f47 [4194304,4194304] 0 2024-05-27T09:29:41.321 INFO:tasks.workunit.client.1.smithi138.stdout:9/507: dwrite d6/da/d2f/d35/d55/d75/f99 [0,4194304] 0 2024-05-27T09:29:41.327 INFO:tasks.workunit.client.0.smithi119.stdout:9/915: creat d18/db4/d108/f121 x:0 0 0 2024-05-27T09:29:41.334 INFO:tasks.workunit.client.0.smithi119.stdout:9/916: link d18/d19/f74 d18/d19/d1b/d29/d11b/f122 0 2024-05-27T09:29:41.338 INFO:tasks.workunit.client.1.smithi138.stdout:2/456: dread d1/d15/f4f [0,4194304] 0 2024-05-27T09:29:41.338 INFO:tasks.workunit.client.1.smithi138.stdout:2/457: chown d1/d2f/f33 293263280 1 2024-05-27T09:29:41.342 INFO:tasks.workunit.client.1.smithi138.stdout:5/436: dwrite d3/d12/d16/d1d/f24 [0,4194304] 0 2024-05-27T09:29:41.354 INFO:tasks.workunit.client.1.smithi138.stdout:9/508: write d6/f2e [644891,128673] 0 2024-05-27T09:29:41.356 INFO:tasks.workunit.client.0.smithi119.stdout:9/917: getdents d18/d19/d1b/d2a/da2 0 2024-05-27T09:29:41.364 INFO:tasks.workunit.client.1.smithi138.stdout:0/492: dread da/f29 [0,4194304] 0 2024-05-27T09:29:41.364 INFO:tasks.workunit.client.1.smithi138.stdout:0/493: chown da/d2a/d8e/d1f/f3e 216635214 1 2024-05-27T09:29:41.364 INFO:tasks.workunit.client.1.smithi138.stdout:5/437: rename d3/d12/c2c to d3/d12/d32/d87/c8a 0 2024-05-27T09:29:41.364 INFO:tasks.workunit.client.1.smithi138.stdout:9/509: rename d6/da/d2f/d35 to d6/da/d2f/d35/d55/d75/da4 22 2024-05-27T09:29:41.364 INFO:tasks.workunit.client.1.smithi138.stdout:9/510: write d6/f3f [2534015,35094] 0 2024-05-27T09:29:41.364 INFO:tasks.workunit.client.1.smithi138.stdout:9/511: stat d6/db/de/d18/d24 0 2024-05-27T09:29:41.367 INFO:tasks.workunit.client.0.smithi119.stdout:9/918: mknod d18/d19/dfa/c123 0 2024-05-27T09:29:41.368 INFO:tasks.workunit.client.1.smithi138.stdout:5/438: write d3/d12/d16/d1d/d22/f3a [442732,89911] 0 2024-05-27T09:29:41.369 INFO:tasks.workunit.client.1.smithi138.stdout:9/512: link d6/db/l63 d6/la5 0 2024-05-27T09:29:41.369 INFO:tasks.workunit.client.1.smithi138.stdout:9/513: write d6/db/dc/f12 [1873290,2979] 0 2024-05-27T09:29:41.371 INFO:tasks.workunit.client.1.smithi138.stdout:6/685: dwrite d3/d11/d2d/d37/dd1/fa1 [4194304,4194304] 0 2024-05-27T09:29:41.375 INFO:tasks.workunit.client.0.smithi119.stdout:9/919: rename d18/d19/d1b/d40/d44/dcc/f11a to d18/d19/d9b/d10b/f124 0 2024-05-27T09:29:41.376 INFO:tasks.workunit.client.0.smithi119.stdout:9/920: write d18/fd1 [953012,56166] 0 2024-05-27T09:29:41.376 INFO:tasks.workunit.client.0.smithi119.stdout:9/921: readlink d18/d19/d1b/d40/d44/l78 0 2024-05-27T09:29:41.380 INFO:tasks.workunit.client.1.smithi138.stdout:4/540: sync 2024-05-27T09:29:41.381 INFO:tasks.workunit.client.1.smithi138.stdout:3/502: sync 2024-05-27T09:29:41.381 INFO:tasks.workunit.client.1.smithi138.stdout:4/541: readlink d7/lad 0 2024-05-27T09:29:41.381 INFO:tasks.workunit.client.1.smithi138.stdout:4/542: chown d7/f39 322530107 1 2024-05-27T09:29:41.381 INFO:tasks.workunit.client.1.smithi138.stdout:5/439: rename d3/d12/d32/f84 to d3/d12/d16/d1d/d30/f8b 0 2024-05-27T09:29:41.388 INFO:tasks.workunit.client.1.smithi138.stdout:5/440: symlink d3/d12/l8c 0 2024-05-27T09:29:41.388 INFO:tasks.workunit.client.1.smithi138.stdout:5/441: chown d3/d12/d16/d1d/d22/l65 203 1 2024-05-27T09:29:41.389 INFO:tasks.workunit.client.1.smithi138.stdout:5/442: readlink d3/l51 0 2024-05-27T09:29:41.389 INFO:tasks.workunit.client.1.smithi138.stdout:8/547: dwrite d3/f4a [0,4194304] 0 2024-05-27T09:29:41.390 INFO:tasks.workunit.client.1.smithi138.stdout:1/489: dwrite d4/d48/f58 [0,4194304] 0 2024-05-27T09:29:41.391 INFO:tasks.workunit.client.1.smithi138.stdout:1/490: fdatasync d4/d1a/d2e/d4a/f8e 0 2024-05-27T09:29:41.393 INFO:tasks.workunit.client.1.smithi138.stdout:8/548: truncate d3/f77 217879 0 2024-05-27T09:29:41.398 INFO:tasks.workunit.client.1.smithi138.stdout:5/443: rename d3/d12/d16/d1d/d30/f4a to d3/d12/d32/d87/f8d 0 2024-05-27T09:29:41.403 INFO:tasks.workunit.client.1.smithi138.stdout:5/444: creat d3/d12/d32/d60/f8e x:0 0 0 2024-05-27T09:29:41.405 INFO:tasks.workunit.client.1.smithi138.stdout:1/491: symlink d4/d1a/d2e/l9e 0 2024-05-27T09:29:41.410 INFO:tasks.workunit.client.1.smithi138.stdout:4/543: dread d7/df/d35/d4c/d59/f90 [0,4194304] 0 2024-05-27T09:29:41.419 INFO:tasks.workunit.client.1.smithi138.stdout:4/544: rename d7/d27/l85 to d7/d27/d3a/lbb 0 2024-05-27T09:29:41.422 INFO:tasks.workunit.client.1.smithi138.stdout:2/458: dwrite d1/d10/d52/d5d/f88 [0,4194304] 0 2024-05-27T09:29:41.422 INFO:tasks.workunit.client.1.smithi138.stdout:2/459: stat d1/d15/d2d/d17/f58 0 2024-05-27T09:29:41.423 INFO:tasks.workunit.client.1.smithi138.stdout:2/460: creat d1/d15/d2d/fa0 x:0 0 0 2024-05-27T09:29:41.424 INFO:tasks.workunit.client.1.smithi138.stdout:1/492: rmdir d4/d88/d9c 0 2024-05-27T09:29:41.425 INFO:tasks.workunit.client.1.smithi138.stdout:1/493: read d4/d1a/d2e/f41 [443917,73169] 0 2024-05-27T09:29:41.425 INFO:tasks.workunit.client.1.smithi138.stdout:1/494: chown d4/d3c/l5d 987376 1 2024-05-27T09:29:41.431 INFO:tasks.workunit.client.1.smithi138.stdout:2/461: link d1/d15/d2d/f39 d1/d15/d2d/d17/d25/d9d/d4b/fa1 0 2024-05-27T09:29:41.431 INFO:tasks.workunit.client.1.smithi138.stdout:2/462: chown d1/d15/d2d/d17/f58 11 1 2024-05-27T09:29:41.434 INFO:tasks.workunit.client.1.smithi138.stdout:2/463: write d1/d15/d2d/d17/d25/d9d/d4b/f51 [1400812,93889] 0 2024-05-27T09:29:41.437 INFO:tasks.workunit.client.1.smithi138.stdout:2/464: mknod d1/d15/d2d/d17/d25/d9d/ca2 0 2024-05-27T09:29:41.438 INFO:tasks.workunit.client.1.smithi138.stdout:1/495: mkdir d4/d88/d9f 0 2024-05-27T09:29:41.438 INFO:tasks.workunit.client.1.smithi138.stdout:1/496: creat d4/d3c/fa0 x:0 0 0 2024-05-27T09:29:41.444 INFO:tasks.workunit.client.1.smithi138.stdout:5/445: dread d3/d12/d16/f2e [0,4194304] 0 2024-05-27T09:29:41.445 INFO:tasks.workunit.client.1.smithi138.stdout:8/549: dread d3/d8/d32/d3f/f6b [0,4194304] 0 2024-05-27T09:29:41.448 INFO:tasks.workunit.client.1.smithi138.stdout:8/550: mkdir d3/db5 0 2024-05-27T09:29:41.451 INFO:tasks.workunit.client.1.smithi138.stdout:8/551: creat d3/d8/fb6 x:0 0 0 2024-05-27T09:29:41.459 INFO:tasks.workunit.client.1.smithi138.stdout:2/465: write d1/d15/d2d/d2e/f37 [31281,69135] 0 2024-05-27T09:29:41.476 INFO:tasks.workunit.client.1.smithi138.stdout:7/485: sync 2024-05-27T09:29:41.477 INFO:tasks.workunit.client.0.smithi119.stdout:9/922: dwrite d18/d19/d1b/d29/dc4/de1/ffc [0,4194304] 0 2024-05-27T09:29:41.478 INFO:tasks.workunit.client.1.smithi138.stdout:0/494: dwrite da/d2a/d8e/d1f/d42/d4a/d53/f57 [0,4194304] 0 2024-05-27T09:29:41.479 INFO:tasks.workunit.client.1.smithi138.stdout:7/486: truncate d0/d7/f54 2377243 0 2024-05-27T09:29:41.480 INFO:tasks.workunit.client.1.smithi138.stdout:7/487: truncate d0/f21 4202028 0 2024-05-27T09:29:41.486 INFO:tasks.workunit.client.1.smithi138.stdout:7/488: write d0/f83 [223167,32415] 0 2024-05-27T09:29:41.491 INFO:tasks.workunit.client.1.smithi138.stdout:7/489: creat d0/d7/d98/fa4 x:0 0 0 2024-05-27T09:29:41.491 INFO:tasks.workunit.client.1.smithi138.stdout:8/552: dread d3/d8/d32/f49 [4194304,4194304] 0 2024-05-27T09:29:41.492 INFO:tasks.workunit.client.1.smithi138.stdout:8/553: write d3/d8/d64/f2f [1007827,6456] 0 2024-05-27T09:29:41.492 INFO:tasks.workunit.client.1.smithi138.stdout:1/497: dwrite d4/f78 [0,4194304] 0 2024-05-27T09:29:41.494 INFO:tasks.workunit.client.1.smithi138.stdout:7/490: link d0/f77 d0/fa5 0 2024-05-27T09:29:41.495 INFO:tasks.workunit.client.1.smithi138.stdout:8/554: readlink d3/d8/d32/d45/d5c/l6e 0 2024-05-27T09:29:41.500 INFO:tasks.workunit.client.0.smithi119.stdout:9/923: getdents d18/d19/d61 0 2024-05-27T09:29:41.500 INFO:tasks.workunit.client.0.smithi119.stdout:9/924: chown d18/d19/d1b/d40/d44/l8d 544787724 1 2024-05-27T09:29:41.502 INFO:tasks.workunit.client.1.smithi138.stdout:1/498: truncate d4/d1a/d2e/d32/f49 173144 0 2024-05-27T09:29:41.503 INFO:tasks.workunit.client.1.smithi138.stdout:7/491: creat d0/d23/d3a/fa6 x:0 0 0 2024-05-27T09:29:41.505 INFO:tasks.workunit.client.1.smithi138.stdout:8/555: mknod d3/d8/cb7 0 2024-05-27T09:29:41.507 INFO:tasks.workunit.client.1.smithi138.stdout:7/492: rename d0/d7/d60/la3 to d0/d23/d3c/d84/la7 0 2024-05-27T09:29:41.507 INFO:tasks.workunit.client.1.smithi138.stdout:8/556: mkdir d3/d8/d32/d3f/d9c/db8 0 2024-05-27T09:29:41.509 INFO:tasks.workunit.client.0.smithi119.stdout:9/925: link d18/d19/d1b/d40/d44/f93 d18/ddf/f125 0 2024-05-27T09:29:41.511 INFO:tasks.workunit.client.1.smithi138.stdout:0/495: dread da/d2a/d8e/d1f/d71/f79 [0,4194304] 0 2024-05-27T09:29:41.511 INFO:tasks.workunit.client.1.smithi138.stdout:0/496: write da/d17/f31 [1535882,2539] 0 2024-05-27T09:29:41.512 INFO:tasks.workunit.client.1.smithi138.stdout:0/497: creat da/d2a/d3f/fa6 x:0 0 0 2024-05-27T09:29:41.513 INFO:tasks.workunit.client.1.smithi138.stdout:4/545: dwrite d7/df/d35/d4c/d59/f58 [0,4194304] 0 2024-05-27T09:29:41.513 INFO:tasks.workunit.client.1.smithi138.stdout:5/446: dwrite d3/fd [4194304,4194304] 0 2024-05-27T09:29:41.513 INFO:tasks.workunit.client.1.smithi138.stdout:1/499: symlink d4/d1a/d2e/d4a/d7e/la1 0 2024-05-27T09:29:41.516 INFO:tasks.workunit.client.0.smithi119.stdout:9/926: rename d18/d19/d1b/d40/d44/ffb to d18/d19/d1b/d40/d44/dcb/f126 0 2024-05-27T09:29:41.518 INFO:tasks.workunit.client.1.smithi138.stdout:3/503: dwrite d0/d2/da/f2d [0,4194304] 0 2024-05-27T09:29:41.518 INFO:tasks.workunit.client.1.smithi138.stdout:3/504: creat d0/fa4 x:0 0 0 2024-05-27T09:29:41.520 INFO:tasks.workunit.client.1.smithi138.stdout:3/505: getdents d0/d3f/d50/d9b 0 2024-05-27T09:29:41.526 INFO:tasks.workunit.client.1.smithi138.stdout:3/506: creat d0/d2/da/df/d10/d32/fa5 x:0 0 0 2024-05-27T09:29:41.527 INFO:tasks.workunit.client.1.smithi138.stdout:0/498: dread da/d2a/d8e/d1f/f38 [0,4194304] 0 2024-05-27T09:29:41.527 INFO:tasks.workunit.client.1.smithi138.stdout:3/507: creat d0/d3f/d50/fa6 x:0 0 0 2024-05-27T09:29:41.527 INFO:tasks.workunit.client.0.smithi119.stdout:9/927: creat d18/d19/d1b/d40/f127 x:0 0 0 2024-05-27T09:29:41.527 INFO:tasks.workunit.client.0.smithi119.stdout:9/928: chown d18/d19/d1b/d29/d11b/f120 83017 1 2024-05-27T09:29:41.528 INFO:tasks.workunit.client.0.smithi119.stdout:9/929: fdatasync d18/d19/d61/f9d 0 2024-05-27T09:29:41.528 INFO:tasks.workunit.client.0.smithi119.stdout:9/930: write d18/d19/d1b/d29/f11e [755474,99158] 0 2024-05-27T09:29:41.530 INFO:tasks.workunit.client.1.smithi138.stdout:5/447: creat d3/d12/d32/d87/f8f x:0 0 0 2024-05-27T09:29:41.530 INFO:tasks.workunit.client.1.smithi138.stdout:5/448: truncate d3/d12/f4d 1065421 0 2024-05-27T09:29:41.530 INFO:tasks.workunit.client.1.smithi138.stdout:5/449: readlink d3/d12/d16/d1d/d23/l71 0 2024-05-27T09:29:41.530 INFO:tasks.workunit.client.1.smithi138.stdout:3/508: symlink d0/d3f/d60/la7 0 2024-05-27T09:29:41.532 INFO:tasks.workunit.client.0.smithi119.stdout:9/931: link d18/ddf/fea d18/d19/d1b/d29/dc4/de1/f128 0 2024-05-27T09:29:41.533 INFO:tasks.workunit.client.1.smithi138.stdout:3/509: symlink d0/d2/da/df/la8 0 2024-05-27T09:29:41.533 INFO:tasks.workunit.client.1.smithi138.stdout:3/510: write d0/d3f/d60/f96 [853247,52145] 0 2024-05-27T09:29:41.533 INFO:tasks.workunit.client.1.smithi138.stdout:3/511: chown d0/d2/da/df/d10/d2a/f56 7 1 2024-05-27T09:29:41.533 INFO:tasks.workunit.client.1.smithi138.stdout:0/499: truncate da/d2a/d8e/d1f/f26 1196870 0 2024-05-27T09:29:41.533 INFO:tasks.workunit.client.1.smithi138.stdout:0/500: read - da/d2a/d8e/d1f/f7b zero size 2024-05-27T09:29:41.534 INFO:tasks.workunit.client.1.smithi138.stdout:3/512: symlink d0/d3f/d50/la9 0 2024-05-27T09:29:41.534 INFO:tasks.workunit.client.1.smithi138.stdout:3/513: chown d0/d2/da/df/d10/c75 6611 1 2024-05-27T09:29:41.536 INFO:tasks.workunit.client.1.smithi138.stdout:0/501: mknod da/d2a/d8e/ca7 0 2024-05-27T09:29:41.538 INFO:tasks.workunit.client.1.smithi138.stdout:0/502: unlink da/d2a/d3f/fa6 0 2024-05-27T09:29:41.546 INFO:tasks.workunit.client.1.smithi138.stdout:7/493: dread d0/d23/f58 [0,4194304] 0 2024-05-27T09:29:41.548 INFO:tasks.workunit.client.1.smithi138.stdout:2/466: dwrite d1/d15/d2d/d1d/f67 [0,4194304] 0 2024-05-27T09:29:41.550 INFO:tasks.workunit.client.1.smithi138.stdout:2/467: stat d1/d15/d2d/d17/d25/d9d/c75 0 2024-05-27T09:29:41.551 INFO:tasks.workunit.client.1.smithi138.stdout:2/468: stat d1/d15/d2d/d17/d25/d4d 0 2024-05-27T09:29:41.553 INFO:tasks.workunit.client.1.smithi138.stdout:2/469: getdents d1/d10/d52 0 2024-05-27T09:29:41.553 INFO:tasks.workunit.client.1.smithi138.stdout:2/470: readlink d1/d10/d3c/l40 0 2024-05-27T09:29:41.555 INFO:tasks.workunit.client.1.smithi138.stdout:5/450: write d3/d12/d32/d6b/f78 [1730038,684] 0 2024-05-27T09:29:41.555 INFO:tasks.workunit.client.1.smithi138.stdout:5/451: chown d3/l6 0 1 2024-05-27T09:29:41.555 INFO:tasks.workunit.client.1.smithi138.stdout:5/452: truncate d3/d12/f31 1172819 0 2024-05-27T09:29:41.558 INFO:tasks.workunit.client.1.smithi138.stdout:2/471: rename d1/d15/d2d/d1d/f67 to d1/d15/d2d/d17/d25/d4d/d6d/fa3 0 2024-05-27T09:29:41.561 INFO:tasks.workunit.client.1.smithi138.stdout:7/494: rename d0/d23/d59 to d0/d23/da8 0 2024-05-27T09:29:41.561 INFO:tasks.workunit.client.1.smithi138.stdout:2/472: mknod d1/d10/d52/ca4 0 2024-05-27T09:29:41.561 INFO:tasks.workunit.client.1.smithi138.stdout:2/473: dread - d1/d15/d2d/fa0 zero size 2024-05-27T09:29:41.562 INFO:tasks.workunit.client.1.smithi138.stdout:5/453: symlink d3/d12/d32/d6b/l90 0 2024-05-27T09:29:41.565 INFO:tasks.workunit.client.1.smithi138.stdout:2/474: link d1/d15/d2d/d17/d25/d9d/ca2 d1/d10/d52/d5d/d7e/ca5 0 2024-05-27T09:29:41.565 INFO:tasks.workunit.client.1.smithi138.stdout:2/475: fsync d1/d15/d2d/d17/d25/d4d/d4e/d60/f92 0 2024-05-27T09:29:41.577 INFO:tasks.workunit.client.1.smithi138.stdout:3/514: dread d0/d2/da/f1b [0,4194304] 0 2024-05-27T09:29:41.579 INFO:tasks.workunit.client.1.smithi138.stdout:3/515: mkdir d0/d3f/d60/daa 0 2024-05-27T09:29:41.584 INFO:tasks.workunit.client.1.smithi138.stdout:7/495: dread d0/d23/f4c [0,4194304] 0 2024-05-27T09:29:41.586 INFO:tasks.workunit.client.1.smithi138.stdout:2/476: dread d1/d15/d2d/d17/d25/d9d/f49 [0,4194304] 0 2024-05-27T09:29:41.586 INFO:tasks.workunit.client.1.smithi138.stdout:5/454: write d3/d12/d16/f39 [6917196,59521] 0 2024-05-27T09:29:41.587 INFO:tasks.workunit.client.1.smithi138.stdout:7/496: creat d0/d7/d43/fa9 x:0 0 0 2024-05-27T09:29:41.588 INFO:tasks.workunit.client.1.smithi138.stdout:7/497: write d0/f83 [497703,63423] 0 2024-05-27T09:29:41.588 INFO:tasks.workunit.client.1.smithi138.stdout:5/455: rename d3/f21 to d3/d12/d32/d60/f91 0 2024-05-27T09:29:41.588 INFO:tasks.workunit.client.1.smithi138.stdout:0/503: dread da/d2a/d8e/d1f/f3e [0,4194304] 0 2024-05-27T09:29:41.589 INFO:tasks.workunit.client.1.smithi138.stdout:2/477: truncate d1/d15/d2d/d17/d25/d4d/d6d/fa3 1088449 0 2024-05-27T09:29:41.589 INFO:tasks.workunit.client.1.smithi138.stdout:5/456: symlink d3/d12/d16/d1d/d23/d3b/l92 0 2024-05-27T09:29:41.590 INFO:tasks.workunit.client.1.smithi138.stdout:0/504: truncate f5 2099148 0 2024-05-27T09:29:41.591 INFO:tasks.workunit.client.1.smithi138.stdout:0/505: write da/d2a/d8e/d43/f6c [974157,106494] 0 2024-05-27T09:29:41.591 INFO:tasks.workunit.client.1.smithi138.stdout:2/478: rmdir d1/d15/d2d/d2e 39 2024-05-27T09:29:41.592 INFO:tasks.workunit.client.1.smithi138.stdout:0/506: creat da/d2a/d8e/d1f/d4f/fa8 x:0 0 0 2024-05-27T09:29:41.593 INFO:tasks.workunit.client.1.smithi138.stdout:5/457: mkdir d3/d12/d32/d60/d93 0 2024-05-27T09:29:41.593 INFO:tasks.workunit.client.1.smithi138.stdout:5/458: write d3/d12/d26/d53/f5a [997468,87460] 0 2024-05-27T09:29:41.594 INFO:tasks.workunit.client.1.smithi138.stdout:2/479: link d1/d15/f48 d1/d10/d52/fa6 0 2024-05-27T09:29:41.594 INFO:tasks.workunit.client.1.smithi138.stdout:0/507: symlink da/d2a/d8e/d1f/d42/d63/la9 0 2024-05-27T09:29:41.596 INFO:tasks.workunit.client.1.smithi138.stdout:1/500: dwrite d4/d3c/fa0 [0,4194304] 0 2024-05-27T09:29:41.596 INFO:tasks.workunit.client.1.smithi138.stdout:1/501: chown d4/d38 2 1 2024-05-27T09:29:41.601 INFO:tasks.workunit.client.1.smithi138.stdout:0/508: rename da/f29 to da/d2a/d8e/d43/d74/faa 0 2024-05-27T09:29:41.605 INFO:tasks.workunit.client.1.smithi138.stdout:0/509: mknod da/d2a/d8e/d6a/d99/cab 0 2024-05-27T09:29:41.611 INFO:tasks.workunit.client.1.smithi138.stdout:1/502: read d4/d3c/f6b [870329,103343] 0 2024-05-27T09:29:41.611 INFO:tasks.workunit.client.1.smithi138.stdout:1/503: read - d4/d48/d39/d5f/f6a zero size 2024-05-27T09:29:41.612 INFO:tasks.workunit.client.1.smithi138.stdout:1/504: write d4/f9b [885403,73128] 0 2024-05-27T09:29:41.612 INFO:tasks.workunit.client.1.smithi138.stdout:1/505: chown d4/d48/d22/d65/f6f 109344351 1 2024-05-27T09:29:41.617 INFO:tasks.workunit.client.1.smithi138.stdout:7/498: write d0/d7/d43/f67 [2446874,54554] 0 2024-05-27T09:29:41.618 INFO:tasks.workunit.client.1.smithi138.stdout:7/499: symlink d0/d7/d43/laa 0 2024-05-27T09:29:41.620 INFO:tasks.workunit.client.1.smithi138.stdout:7/500: creat d0/d2b/fab x:0 0 0 2024-05-27T09:29:41.621 INFO:tasks.workunit.client.1.smithi138.stdout:1/506: dread d4/d48/d39/d5f/d4d/d2b/f6c [0,4194304] 0 2024-05-27T09:29:41.621 INFO:tasks.workunit.client.1.smithi138.stdout:1/507: creat d4/fa2 x:0 0 0 2024-05-27T09:29:41.664 INFO:tasks.workunit.client.1.smithi138.stdout:0/510: truncate da/d2a/d8e/d43/f64 1823444 0 2024-05-27T09:29:41.665 INFO:tasks.workunit.client.1.smithi138.stdout:4/546: dwrite d7/df/d35/d4c/f69 [0,4194304] 0 2024-05-27T09:29:41.665 INFO:tasks.workunit.client.1.smithi138.stdout:4/547: readlink d7/df/d35/d4c/d59/d5d/d64/l7a 0 2024-05-27T09:29:41.665 INFO:tasks.workunit.client.1.smithi138.stdout:4/548: dread - d7/df/d5b/f7f zero size 2024-05-27T09:29:41.665 INFO:tasks.workunit.client.1.smithi138.stdout:4/549: chown d7/df/d35/d4c/d59/d5d/d64/fb7 30 1 2024-05-27T09:29:41.669 INFO:tasks.workunit.client.1.smithi138.stdout:1/508: unlink d4/d48/d22/f89 0 2024-05-27T09:29:41.671 INFO:tasks.workunit.client.1.smithi138.stdout:0/511: truncate da/d2a/d8e/d1f/f21 928592 0 2024-05-27T09:29:41.671 INFO:tasks.workunit.client.1.smithi138.stdout:0/512: creat da/d2a/d8e/d1f/d42/fac x:0 0 0 2024-05-27T09:29:41.671 INFO:tasks.workunit.client.1.smithi138.stdout:0/513: chown da/d2a/d8e/d1f/d42/c92 782 1 2024-05-27T09:29:41.672 INFO:tasks.workunit.client.1.smithi138.stdout:7/501: dread d0/f83 [0,4194304] 0 2024-05-27T09:29:41.675 INFO:tasks.workunit.client.1.smithi138.stdout:2/480: dwrite d1/d15/d2d/d7b/f95 [0,4194304] 0 2024-05-27T09:29:41.675 INFO:tasks.workunit.client.1.smithi138.stdout:2/481: fsync d1/d10/fe 0 2024-05-27T09:29:41.675 INFO:tasks.workunit.client.1.smithi138.stdout:2/482: fdatasync d1/d15/d2d/d17/d25/f2a 0 2024-05-27T09:29:41.675 INFO:tasks.workunit.client.1.smithi138.stdout:2/483: fsync d1/d10/d52/f73 0 2024-05-27T09:29:41.676 INFO:tasks.workunit.client.1.smithi138.stdout:4/550: mkdir d7/df/d5b/da1/dbc 0 2024-05-27T09:29:41.681 INFO:tasks.workunit.client.1.smithi138.stdout:8/557: dwrite d3/d8/d24/d55/d66/fa3 [4194304,4194304] 0 2024-05-27T09:29:41.681 INFO:tasks.workunit.client.1.smithi138.stdout:8/558: stat d3/d8/d24/d4d/f61 0 2024-05-27T09:29:41.681 INFO:tasks.workunit.client.1.smithi138.stdout:8/559: dread - d3/d8/d24/d55/d7e/d83/fb0 zero size 2024-05-27T09:29:41.681 INFO:tasks.workunit.client.1.smithi138.stdout:8/560: write d3/f77 [928891,111174] 0 2024-05-27T09:29:41.682 INFO:tasks.workunit.client.1.smithi138.stdout:4/551: symlink d7/d27/lbd 0 2024-05-27T09:29:41.688 INFO:tasks.workunit.client.1.smithi138.stdout:2/484: dread d1/d10/fe [0,4194304] 0 2024-05-27T09:29:41.692 INFO:tasks.workunit.client.1.smithi138.stdout:8/561: symlink d3/d8/d32/d34/lb9 0 2024-05-27T09:29:41.692 INFO:tasks.workunit.client.1.smithi138.stdout:4/552: rename d7/df/d35/d4c/d59/fa4 to d7/df/d35/d4c/d59/d5d/d64/d6a/fbe 0 2024-05-27T09:29:41.701 INFO:tasks.workunit.client.1.smithi138.stdout:0/514: dread da/d2a/d8e/d43/f6c [0,4194304] 0 2024-05-27T09:29:41.701 INFO:tasks.workunit.client.1.smithi138.stdout:0/515: stat da/d2a/d8e/d1f/d4f/f6f 0 2024-05-27T09:29:41.701 INFO:tasks.workunit.client.1.smithi138.stdout:0/516: getdents da/d2a/d8e/d6a 0 2024-05-27T09:29:41.701 INFO:tasks.workunit.client.1.smithi138.stdout:0/517: readlink da/d2a/d55/la1 0 2024-05-27T09:29:41.701 INFO:tasks.workunit.client.1.smithi138.stdout:0/518: creat da/d2a/d8e/d1f/fad x:0 0 0 2024-05-27T09:29:41.711 INFO:tasks.workunit.client.0.smithi119.stdout:9/932: sync 2024-05-27T09:29:41.720 INFO:tasks.workunit.client.0.smithi119.stdout:9/933: creat d18/d19/d1b/d40/d44/dcc/f129 x:0 0 0 2024-05-27T09:29:41.720 INFO:tasks.workunit.client.1.smithi138.stdout:8/562: dread d3/d8/d64/f7c [0,4194304] 0 2024-05-27T09:29:41.722 INFO:tasks.workunit.client.1.smithi138.stdout:8/563: mknod d3/d8/d24/d55/d94/cba 0 2024-05-27T09:29:41.723 INFO:tasks.workunit.client.1.smithi138.stdout:8/564: creat d3/d8/d11/fbb x:0 0 0 2024-05-27T09:29:41.726 INFO:tasks.workunit.client.1.smithi138.stdout:8/565: symlink d3/d8/d32/d3f/d9c/d9f/d86/lbc 0 2024-05-27T09:29:41.727 INFO:tasks.workunit.client.0.smithi119.stdout:9/934: write d18/d19/d1b/d40/d7b/f7e [1083390,10815] 0 2024-05-27T09:29:41.730 INFO:tasks.workunit.client.1.smithi138.stdout:0/519: dread da/d2a/d55/f98 [0,4194304] 0 2024-05-27T09:29:41.731 INFO:tasks.workunit.client.1.smithi138.stdout:0/520: creat da/d2a/d8e/d1f/d42/d63/d9d/fae x:0 0 0 2024-05-27T09:29:41.733 INFO:tasks.workunit.client.0.smithi119.stdout:9/935: symlink d18/d19/d1b/d2a/l12a 0 2024-05-27T09:29:41.733 INFO:tasks.workunit.client.0.smithi119.stdout:9/936: creat d18/d19/dfa/f12b x:0 0 0 2024-05-27T09:29:41.733 INFO:tasks.workunit.client.0.smithi119.stdout:9/937: dread - d18/d19/d1b/d2a/d9f/fec zero size 2024-05-27T09:29:41.736 INFO:tasks.workunit.client.0.smithi119.stdout:9/938: getdents d18/db4/dbe 0 2024-05-27T09:29:41.739 INFO:tasks.workunit.client.0.smithi119.stdout:9/939: rmdir d18/d19/d1b/d29/dc4 39 2024-05-27T09:29:41.742 INFO:tasks.workunit.client.0.smithi119.stdout:9/940: rmdir d18/d19/d61/d95 0 2024-05-27T09:29:41.743 INFO:tasks.workunit.client.1.smithi138.stdout:7/502: dwrite d0/d7/d57/f95 [0,4194304] 0 2024-05-27T09:29:41.744 INFO:tasks.workunit.client.1.smithi138.stdout:7/503: chown d0/f36 220 1 2024-05-27T09:29:41.744 INFO:tasks.workunit.client.1.smithi138.stdout:7/504: chown d0/d23/d3a/c6a 215884 1 2024-05-27T09:29:41.745 INFO:tasks.workunit.client.1.smithi138.stdout:1/509: dwrite d4/d1a/d2e/d4a/f8e [0,4194304] 0 2024-05-27T09:29:41.747 INFO:tasks.workunit.client.1.smithi138.stdout:8/566: dread d3/d8/d32/d3f/f6b [0,4194304] 0 2024-05-27T09:29:41.748 INFO:tasks.workunit.client.1.smithi138.stdout:6/686: sync 2024-05-27T09:29:41.748 INFO:tasks.workunit.client.1.smithi138.stdout:6/687: read - d3/d11/dc0/d43/d71/fca zero size 2024-05-27T09:29:41.751 INFO:tasks.workunit.client.0.smithi119.stdout:9/941: rename d18/d19/d1b/d2a/lda to d18/d19/d1b/d40/d7b/dc6/d118/l12c 0 2024-05-27T09:29:41.751 INFO:tasks.workunit.client.0.smithi119.stdout:9/942: write d18/d19/d1b/d40/fc7 [1041978,93927] 0 2024-05-27T09:29:41.751 INFO:tasks.workunit.client.0.smithi119.stdout:9/943: creat d18/db4/d108/f12d x:0 0 0 2024-05-27T09:29:41.753 INFO:tasks.workunit.client.1.smithi138.stdout:4/553: dwrite d7/f44 [0,4194304] 0 2024-05-27T09:29:41.754 INFO:tasks.workunit.client.1.smithi138.stdout:0/521: read da/d17/f44 [312651,20361] 0 2024-05-27T09:29:41.754 INFO:tasks.workunit.client.1.smithi138.stdout:0/522: fdatasync da/f20 0 2024-05-27T09:29:41.754 INFO:tasks.workunit.client.1.smithi138.stdout:0/523: stat da/d2a/f30 0 2024-05-27T09:29:41.761 INFO:tasks.workunit.client.1.smithi138.stdout:6/688: creat d3/d11/dc0/d7d/de3/fef x:0 0 0 2024-05-27T09:29:41.762 INFO:tasks.workunit.client.1.smithi138.stdout:4/554: mknod d7/d27/d3a/cbf 0 2024-05-27T09:29:41.763 INFO:tasks.workunit.client.0.smithi119.stdout:9/944: mknod d18/d19/d1b/d29/dc4/c12e 0 2024-05-27T09:29:41.764 INFO:tasks.workunit.client.1.smithi138.stdout:0/524: mkdir da/d2a/d8e/d6a/daf 0 2024-05-27T09:29:41.764 INFO:tasks.workunit.client.1.smithi138.stdout:0/525: fdatasync da/d2a/d8e/d43/d74/d97/f9e 0 2024-05-27T09:29:41.764 INFO:tasks.workunit.client.1.smithi138.stdout:6/689: truncate d3/d11/dc0/f80 1615630 0 2024-05-27T09:29:41.766 INFO:tasks.workunit.client.1.smithi138.stdout:4/555: mkdir d7/df/d35/d4c/dc0 0 2024-05-27T09:29:41.768 INFO:tasks.workunit.client.1.smithi138.stdout:0/526: symlink da/d2a/d8e/d1f/d42/d4a/d53/lb0 0 2024-05-27T09:29:41.770 INFO:tasks.workunit.client.1.smithi138.stdout:0/527: link da/d2a/d8e/d1f/d42/fac da/d2a/d8e/d1f/d60/fb1 0 2024-05-27T09:29:41.797 INFO:tasks.workunit.client.1.smithi138.stdout:4/556: write d7/df/d35/d4c/d59/d5d/d64/d6a/f76 [3382108,64803] 0 2024-05-27T09:29:41.823 INFO:tasks.workunit.client.1.smithi138.stdout:4/557: write d7/df/d5b/f7c [262957,68783] 0 2024-05-27T09:29:41.824 INFO:tasks.workunit.client.1.smithi138.stdout:4/558: write d7/df/d5b/f89 [28731,4581] 0 2024-05-27T09:29:41.825 INFO:tasks.workunit.client.1.smithi138.stdout:4/559: symlink d7/df/d35/lc1 0 2024-05-27T09:29:41.825 INFO:tasks.workunit.client.1.smithi138.stdout:4/560: creat d7/d6e/fc2 x:0 0 0 2024-05-27T09:29:41.825 INFO:tasks.workunit.client.1.smithi138.stdout:4/561: dread - d7/df/d35/f77 zero size 2024-05-27T09:29:41.827 INFO:tasks.workunit.client.1.smithi138.stdout:4/562: symlink d7/df/d35/d4c/lc3 0 2024-05-27T09:29:41.827 INFO:tasks.workunit.client.1.smithi138.stdout:8/567: dread d3/d8/f33 [0,4194304] 0 2024-05-27T09:29:41.829 INFO:tasks.workunit.client.1.smithi138.stdout:1/510: dwrite d4/f24 [4194304,4194304] 0 2024-05-27T09:29:41.831 INFO:tasks.workunit.client.1.smithi138.stdout:1/511: getdents d4/d48/d22 0 2024-05-27T09:29:41.831 INFO:tasks.workunit.client.1.smithi138.stdout:8/568: mkdir d3/d8/d32/d34/dbd 0 2024-05-27T09:29:41.833 INFO:tasks.workunit.client.1.smithi138.stdout:8/569: symlink d3/d8/d24/d55/lbe 0 2024-05-27T09:29:41.834 INFO:tasks.workunit.client.1.smithi138.stdout:8/570: mkdir d3/d8/d32/d3f/dbf 0 2024-05-27T09:29:41.835 INFO:tasks.workunit.client.1.smithi138.stdout:8/571: chown d3/d8/d32/d34/dbd 10289713 1 2024-05-27T09:29:41.836 INFO:tasks.workunit.client.1.smithi138.stdout:8/572: mknod d3/d8/d32/d3f/d9c/cc0 0 2024-05-27T09:29:41.839 INFO:tasks.workunit.client.1.smithi138.stdout:0/528: dwrite da/d2a/d8e/d1f/d42/d4a/f4e [0,4194304] 0 2024-05-27T09:29:41.839 INFO:tasks.workunit.client.1.smithi138.stdout:6/690: dwrite d3/d11/dc0/d43/dbe/fe4 [0,4194304] 0 2024-05-27T09:29:41.840 INFO:tasks.workunit.client.1.smithi138.stdout:8/573: rmdir d3/d8/d32/d3f/dbf 0 2024-05-27T09:29:41.840 INFO:tasks.workunit.client.1.smithi138.stdout:8/574: stat d3/d8/d32/f49 0 2024-05-27T09:29:41.841 INFO:tasks.workunit.client.1.smithi138.stdout:0/529: truncate da/d2a/d8e/d1f/d71/f79 814354 0 2024-05-27T09:29:41.842 INFO:tasks.workunit.client.1.smithi138.stdout:8/575: getdents d3/d8/d32/d34/d38 0 2024-05-27T09:29:41.844 INFO:tasks.workunit.client.1.smithi138.stdout:8/576: creat d3/d8/d24/fc1 x:0 0 0 2024-05-27T09:29:41.846 INFO:tasks.workunit.client.1.smithi138.stdout:8/577: mkdir d3/d8/d32/d45/dc2 0 2024-05-27T09:29:41.846 INFO:tasks.workunit.client.1.smithi138.stdout:8/578: truncate d3/d8/d11/fbb 810794 0 2024-05-27T09:29:41.848 INFO:tasks.workunit.client.1.smithi138.stdout:0/530: creat da/d2a/d8e/d6a/d99/fb2 x:0 0 0 2024-05-27T09:29:41.848 INFO:tasks.workunit.client.0.smithi119.stdout:9/945: dwrite d18/d19/d1b/d40/fe9 [0,4194304] 0 2024-05-27T09:29:41.848 INFO:tasks.workunit.client.0.smithi119.stdout:9/946: truncate d18/d19/d61/ff7 965761 0 2024-05-27T09:29:41.851 INFO:tasks.workunit.client.1.smithi138.stdout:1/512: dread d4/d1a/d2e/f62 [0,4194304] 0 2024-05-27T09:29:41.854 INFO:tasks.workunit.client.0.smithi119.stdout:9/947: mkdir d18/db4/d108/d12f 0 2024-05-27T09:29:41.856 INFO:tasks.workunit.client.0.smithi119.stdout:9/948: link d18/d19/d1b/l53 d18/d19/d1b/d2a/da2/l130 0 2024-05-27T09:29:41.856 INFO:tasks.workunit.client.0.smithi119.stdout:9/949: fdatasync d18/d19/d1b/f5d 0 2024-05-27T09:29:41.859 INFO:tasks.workunit.client.0.smithi119.stdout:9/950: symlink d18/ddf/l131 0 2024-05-27T09:29:41.859 INFO:tasks.workunit.client.0.smithi119.stdout:9/951: fdatasync d18/d19/d1b/d29/f11e 0 2024-05-27T09:29:41.859 INFO:tasks.workunit.client.1.smithi138.stdout:0/531: dread da/d2a/d8e/d1f/f21 [0,4194304] 0 2024-05-27T09:29:41.862 INFO:tasks.workunit.client.0.smithi119.stdout:9/952: truncate d18/d19/d1b/f4c 1338191 0 2024-05-27T09:29:41.866 INFO:tasks.workunit.client.1.smithi138.stdout:9/514: sync 2024-05-27T09:29:41.867 INFO:tasks.workunit.client.0.smithi119.stdout:9/953: creat d18/d19/d1b/d2a/dfe/f132 x:0 0 0 2024-05-27T09:29:41.869 INFO:tasks.workunit.client.0.smithi119.stdout:9/954: truncate d18/f9a 4078083 0 2024-05-27T09:29:41.870 INFO:tasks.workunit.client.0.smithi119.stdout:9/955: creat d18/d19/d1b/d40/d7b/f133 x:0 0 0 2024-05-27T09:29:41.870 INFO:tasks.workunit.client.1.smithi138.stdout:9/515: mknod d6/da/d2f/d35/d55/d75/ca6 0 2024-05-27T09:29:41.872 INFO:tasks.workunit.client.0.smithi119.stdout:9/956: rmdir d18/d19/d1b 39 2024-05-27T09:29:41.873 INFO:tasks.workunit.client.1.smithi138.stdout:9/516: creat d6/db/de/fa7 x:0 0 0 2024-05-27T09:29:41.874 INFO:tasks.workunit.client.0.smithi119.stdout:9/957: symlink d18/d19/d1b/d29/dc4/l134 0 2024-05-27T09:29:41.875 INFO:tasks.workunit.client.1.smithi138.stdout:1/513: write d4/d48/d39/d5f/f1f [2654112,18058] 0 2024-05-27T09:29:41.875 INFO:tasks.workunit.client.1.smithi138.stdout:1/514: readlink d4/l77 0 2024-05-27T09:29:41.876 INFO:tasks.workunit.client.1.smithi138.stdout:1/515: creat d4/d48/d39/d5f/d63/fa3 x:0 0 0 2024-05-27T09:29:41.878 INFO:tasks.workunit.client.1.smithi138.stdout:1/516: creat d4/d1a/d2e/d4a/d7e/fa4 x:0 0 0 2024-05-27T09:29:41.878 INFO:tasks.workunit.client.1.smithi138.stdout:9/517: unlink d6/db/de/d85/l91 0 2024-05-27T09:29:41.879 INFO:tasks.workunit.client.0.smithi119.stdout:9/958: truncate d18/d19/d1b/d40/d44/fdd 3388855 0 2024-05-27T09:29:41.879 INFO:tasks.workunit.client.0.smithi119.stdout:9/959: chown d18/d19/d1b/d40/d44/f4a 1005 1 2024-05-27T09:29:41.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:41 smithi138 ceph-mon[34885]: pgmap v18: 65 pgs: 65 active+clean; 2.8 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 170 MiB/s rd, 217 MiB/s wr, 298 op/s 2024-05-27T09:29:41.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:41 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:29:41.891 INFO:tasks.workunit.client.1.smithi138.stdout:1/517: dread d4/d1a/d2e/f62 [0,4194304] 0 2024-05-27T09:29:41.891 INFO:tasks.workunit.client.1.smithi138.stdout:6/691: dwrite d3/d11/d2d/d37/dd1/d47/f5e [0,4194304] 0 2024-05-27T09:29:41.891 INFO:tasks.workunit.client.1.smithi138.stdout:1/518: write d4/fa2 [721515,106998] 0 2024-05-27T09:29:41.892 INFO:tasks.workunit.client.1.smithi138.stdout:6/692: mkdir d3/d11/dc0/d5a/df0 0 2024-05-27T09:29:41.893 INFO:tasks.workunit.client.1.smithi138.stdout:1/519: unlink d4/d48/d39/d5f/f1f 0 2024-05-27T09:29:41.895 INFO:tasks.workunit.client.1.smithi138.stdout:1/520: truncate d4/d1a/d2e/f62 3738428 0 2024-05-27T09:29:41.896 INFO:tasks.workunit.client.1.smithi138.stdout:1/521: creat d4/d48/d22/fa5 x:0 0 0 2024-05-27T09:29:41.912 INFO:tasks.workunit.client.0.smithi119.stdout:9/960: dread d18/d19/d1b/d40/d44/fb0 [4194304,4194304] 0 2024-05-27T09:29:41.916 INFO:tasks.workunit.client.1.smithi138.stdout:1/522: dread d4/f78 [0,4194304] 0 2024-05-27T09:29:41.916 INFO:tasks.workunit.client.1.smithi138.stdout:1/523: dread - d4/d48/d39/d5f/d63/fa3 zero size 2024-05-27T09:29:41.920 INFO:tasks.workunit.client.1.smithi138.stdout:6/693: dread d3/d11/d2d/d37/f4a [0,4194304] 0 2024-05-27T09:29:41.921 INFO:tasks.workunit.client.1.smithi138.stdout:6/694: chown d3/d11/dc0/d5a/d6a/f6b 5 1 2024-05-27T09:29:41.921 INFO:tasks.workunit.client.1.smithi138.stdout:6/695: fsync d3/d11/dc0/d43/dbe/dc3/fcc 0 2024-05-27T09:29:41.934 INFO:tasks.workunit.client.0.smithi119.stdout:9/961: write d18/d19/d1b/d2a/d63/ff9 [3994359,12462] 0 2024-05-27T09:29:41.935 INFO:tasks.workunit.client.0.smithi119.stdout:9/962: creat d18/d19/d1b/d40/d44/dcc/f135 x:0 0 0 2024-05-27T09:29:41.935 INFO:tasks.workunit.client.0.smithi119.stdout:9/963: stat d18/f41 0 2024-05-27T09:29:41.942 INFO:tasks.workunit.client.1.smithi138.stdout:9/518: dwrite d6/db/dc/f4d [0,4194304] 0 2024-05-27T09:29:41.950 INFO:tasks.workunit.client.1.smithi138.stdout:9/519: mkdir d6/da/d29/d8f/da8 0 2024-05-27T09:29:41.956 INFO:tasks.workunit.client.1.smithi138.stdout:1/524: dwrite d4/d48/d22/fa5 [0,4194304] 0 2024-05-27T09:29:41.964 INFO:tasks.workunit.client.1.smithi138.stdout:6/696: dwrite d3/d11/f4f [4194304,4194304] 0 2024-05-27T09:29:41.964 INFO:tasks.workunit.client.1.smithi138.stdout:6/697: readlink d3/l5c 0 2024-05-27T09:29:41.964 INFO:tasks.workunit.client.1.smithi138.stdout:6/698: chown d3/d11/d2d/d37/daa/fd9 372 1 2024-05-27T09:29:41.964 INFO:tasks.workunit.client.1.smithi138.stdout:6/699: chown d3/d19/ce5 0 1 2024-05-27T09:29:41.967 INFO:tasks.workunit.client.1.smithi138.stdout:1/525: rmdir d4 39 2024-05-27T09:29:41.968 INFO:tasks.workunit.client.1.smithi138.stdout:6/700: creat d3/d11/d2d/d37/dd1/d9b/ff1 x:0 0 0 2024-05-27T09:29:41.969 INFO:tasks.workunit.client.1.smithi138.stdout:1/526: dread - d4/d48/d39/d5f/d63/fa3 zero size 2024-05-27T09:29:41.969 INFO:tasks.workunit.client.1.smithi138.stdout:6/701: getdents d3/d11/d33 0 2024-05-27T09:29:41.973 INFO:tasks.workunit.client.1.smithi138.stdout:1/527: creat d4/d1a/d2e/d4a/fa6 x:0 0 0 2024-05-27T09:29:41.975 INFO:tasks.workunit.client.1.smithi138.stdout:9/520: dread f1 [4194304,4194304] 0 2024-05-27T09:29:41.995 INFO:tasks.workunit.client.0.smithi119.stdout:9/964: dwrite d18/ddf/fba [0,4194304] 0 2024-05-27T09:29:41.996 INFO:tasks.workunit.client.0.smithi119.stdout:9/965: write d18/db4/d108/f121 [834306,108597] 0 2024-05-27T09:29:41.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:41 smithi119 ceph-mon[29082]: pgmap v18: 65 pgs: 65 active+clean; 2.8 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 170 MiB/s rd, 217 MiB/s wr, 298 op/s 2024-05-27T09:29:41.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:41 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:29:42.019 INFO:tasks.workunit.client.1.smithi138.stdout:6/702: link d3/d11/dc0/d43/d1c/l2b d3/d11/dc0/d43/dbe/dc3/lf2 0 2024-05-27T09:29:42.020 INFO:tasks.workunit.client.1.smithi138.stdout:6/703: chown d3/d11/dc0/d5a/c65 258372508 1 2024-05-27T09:29:42.020 INFO:tasks.workunit.client.1.smithi138.stdout:6/704: readlink d3/d11/dc0/d43/d71/lb1 0 2024-05-27T09:29:42.024 INFO:tasks.workunit.client.1.smithi138.stdout:1/528: dwrite d4/d48/d39/d5f/f50 [0,4194304] 0 2024-05-27T09:29:42.039 INFO:tasks.workunit.client.1.smithi138.stdout:9/521: dwrite f3 [0,4194304] 0 2024-05-27T09:29:42.039 INFO:tasks.workunit.client.1.smithi138.stdout:9/522: stat d6/db/dc/l7d 0 2024-05-27T09:29:42.039 INFO:tasks.workunit.client.1.smithi138.stdout:9/523: creat d6/da/fa9 x:0 0 0 2024-05-27T09:29:42.039 INFO:tasks.workunit.client.1.smithi138.stdout:9/524: truncate d6/da/d26/d28/f49 5046513 0 2024-05-27T09:29:42.041 INFO:tasks.workunit.client.1.smithi138.stdout:9/525: creat d6/da/d29/faa x:0 0 0 2024-05-27T09:29:42.042 INFO:tasks.workunit.client.1.smithi138.stdout:6/705: dread d3/d19/f89 [0,4194304] 0 2024-05-27T09:29:42.047 INFO:tasks.workunit.client.1.smithi138.stdout:9/526: symlink d6/da/d29/d8f/da8/lab 0 2024-05-27T09:29:42.071 INFO:tasks.workunit.client.1.smithi138.stdout:9/527: dread d6/f48 [0,4194304] 0 2024-05-27T09:29:42.093 INFO:tasks.workunit.client.1.smithi138.stdout:6/706: dwrite d3/d11/dc0/d43/d8c/fa6 [0,4194304] 0 2024-05-27T09:29:42.099 INFO:tasks.workunit.client.1.smithi138.stdout:1/529: truncate d4/d48/d39/d5f/d4d/f3f 1334564 0 2024-05-27T09:29:42.099 INFO:tasks.workunit.client.1.smithi138.stdout:1/530: dread - d4/d48/d39/d5f/f6a zero size 2024-05-27T09:29:42.155 INFO:tasks.workunit.client.1.smithi138.stdout:6/707: dwrite d3/d19/fab [0,4194304] 0 2024-05-27T09:29:42.155 INFO:tasks.workunit.client.1.smithi138.stdout:9/528: dwrite d6/db/de/d18/d24/f73 [0,4194304] 0 2024-05-27T09:29:42.155 INFO:tasks.workunit.client.1.smithi138.stdout:9/529: fdatasync d6/da/d2f/d35/d55/f3a 0 2024-05-27T09:29:42.162 INFO:tasks.workunit.client.1.smithi138.stdout:9/530: rename d6/db/de/c51 to d6/cac 0 2024-05-27T09:29:42.162 INFO:tasks.workunit.client.1.smithi138.stdout:9/531: chown d6/db/de/c62 1 1 2024-05-27T09:29:42.162 INFO:tasks.workunit.client.1.smithi138.stdout:9/532: truncate d6/da/d29/d61/f68 5100502 0 2024-05-27T09:29:42.164 INFO:tasks.workunit.client.1.smithi138.stdout:9/533: mknod d6/db/de/d18/d27/d4c/cad 0 2024-05-27T09:29:42.167 INFO:tasks.workunit.client.1.smithi138.stdout:9/534: creat d6/db/de/d18/d27/d4c/fae x:0 0 0 2024-05-27T09:29:42.167 INFO:tasks.workunit.client.1.smithi138.stdout:9/535: stat d6/da/d2f/d35/d55/d75/f99 0 2024-05-27T09:29:42.175 INFO:tasks.workunit.client.1.smithi138.stdout:9/536: dread d6/db/de/d18/d24/f4b [0,4194304] 0 2024-05-27T09:29:42.175 INFO:tasks.workunit.client.1.smithi138.stdout:9/537: stat d6/da/d29/d59/f90 0 2024-05-27T09:29:42.179 INFO:tasks.workunit.client.1.smithi138.stdout:6/708: truncate d3/d11/dc0/d5a/d6a/f6f 2874632 0 2024-05-27T09:29:42.186 INFO:tasks.workunit.client.1.smithi138.stdout:6/709: mkdir d3/d11/d2d/d37/daa/df3 0 2024-05-27T09:29:42.201 INFO:tasks.workunit.client.1.smithi138.stdout:1/531: symlink d4/d1a/la7 0 2024-05-27T09:29:42.208 INFO:tasks.workunit.client.1.smithi138.stdout:1/532: symlink d4/d48/d39/d5f/d86/la8 0 2024-05-27T09:29:42.214 INFO:tasks.workunit.client.1.smithi138.stdout:1/533: link d4/d48/d39/d5f/d4d/f6d d4/d1a/fa9 0 2024-05-27T09:29:42.229 INFO:tasks.workunit.client.1.smithi138.stdout:1/534: mkdir d4/daa 0 2024-05-27T09:29:42.231 INFO:tasks.workunit.client.1.smithi138.stdout:9/538: dwrite f3 [4194304,4194304] 0 2024-05-27T09:29:42.235 INFO:tasks.workunit.client.1.smithi138.stdout:1/535: symlink d4/d1a/d2e/d4a/lab 0 2024-05-27T09:29:42.235 INFO:tasks.workunit.client.1.smithi138.stdout:1/536: dread - d4/d1a/d2e/f79 zero size 2024-05-27T09:29:42.237 INFO:tasks.workunit.client.1.smithi138.stdout:9/539: rename d6/da/d2f/d35/d55/d75/f9b to d6/da/d29/d59/faf 0 2024-05-27T09:29:42.239 INFO:tasks.workunit.client.1.smithi138.stdout:1/537: creat d4/d48/d39/d5f/d4d/fac x:0 0 0 2024-05-27T09:29:42.240 INFO:tasks.workunit.client.1.smithi138.stdout:9/540: creat d6/da/d26/fb0 x:0 0 0 2024-05-27T09:29:42.240 INFO:tasks.workunit.client.1.smithi138.stdout:9/541: dread - d6/db/de/d18/d27/d4c/fae zero size 2024-05-27T09:29:42.241 INFO:tasks.workunit.client.1.smithi138.stdout:1/538: mknod d4/d48/d39/d5f/d63/cad 0 2024-05-27T09:29:42.241 INFO:tasks.workunit.client.1.smithi138.stdout:1/539: readlink d4/d48/d39/d5f/l87 0 2024-05-27T09:29:42.241 INFO:tasks.workunit.client.1.smithi138.stdout:1/540: readlink d4/d48/l57 0 2024-05-27T09:29:42.242 INFO:tasks.workunit.client.1.smithi138.stdout:6/710: dwrite d3/d11/d2d/d37/d7e/f9f [0,4194304] 0 2024-05-27T09:29:42.243 INFO:tasks.workunit.client.1.smithi138.stdout:9/542: creat d6/da/d2f/d35/fb1 x:0 0 0 2024-05-27T09:29:42.243 INFO:tasks.workunit.client.1.smithi138.stdout:9/543: chown d6/la5 31 1 2024-05-27T09:29:42.243 INFO:tasks.workunit.client.1.smithi138.stdout:9/544: write d6/da/d2f/d35/d55/f3a [198239,24246] 0 2024-05-27T09:29:42.243 INFO:tasks.workunit.client.1.smithi138.stdout:9/545: chown d6/db/de/d18/d27/da2 1 1 2024-05-27T09:29:42.247 INFO:tasks.workunit.client.1.smithi138.stdout:1/541: mknod d4/daa/cae 0 2024-05-27T09:29:42.250 INFO:tasks.workunit.client.1.smithi138.stdout:9/546: rename d6/db/de/d18/f92 to d6/da/d2f/d35/d55/d75/d8c/fb2 0 2024-05-27T09:29:42.250 INFO:tasks.workunit.client.1.smithi138.stdout:9/547: chown d6/db/de/d18 2469018 1 2024-05-27T09:29:42.253 INFO:tasks.workunit.client.1.smithi138.stdout:6/711: truncate d3/d11/d2d/d37/dd1/d47/f51 512292 0 2024-05-27T09:29:42.253 INFO:tasks.workunit.client.1.smithi138.stdout:3/516: sync 2024-05-27T09:29:42.256 INFO:tasks.workunit.client.1.smithi138.stdout:9/548: link d6/db/dc/l7d d6/da/d26/lb3 0 2024-05-27T09:29:42.256 INFO:tasks.workunit.client.1.smithi138.stdout:9/549: truncate d6/da/d29/f74 834050 0 2024-05-27T09:29:42.256 INFO:tasks.workunit.client.1.smithi138.stdout:9/550: chown d6/db/de/d18/d27/d46/c79 81958 1 2024-05-27T09:29:42.256 INFO:tasks.workunit.client.1.smithi138.stdout:9/551: stat d6/da/d29/d61 0 2024-05-27T09:29:42.256 INFO:tasks.workunit.client.1.smithi138.stdout:3/517: mkdir d0/d80/dab 0 2024-05-27T09:29:42.256 INFO:tasks.workunit.client.1.smithi138.stdout:3/518: stat d0/f54 0 2024-05-27T09:29:42.265 INFO:tasks.workunit.client.0.smithi119.stdout:9/966: sync 2024-05-27T09:29:42.274 INFO:tasks.workunit.client.0.smithi119.stdout:9/967: rename d18/d19/d1b/d40/d44/fdd to d18/d19/d1b/d2a/d63/f136 0 2024-05-27T09:29:42.277 INFO:tasks.workunit.client.1.smithi138.stdout:3/519: dread d0/d2/f4b [0,4194304] 0 2024-05-27T09:29:42.278 INFO:tasks.workunit.client.0.smithi119.stdout:9/968: rename d18/d19/d1b/d40/d44/l78 to d18/d19/d1b/d40/d7b/dc6/d118/l137 0 2024-05-27T09:29:42.278 INFO:tasks.workunit.client.0.smithi119.stdout:9/969: creat d18/d19/dfa/f138 x:0 0 0 2024-05-27T09:29:42.290 INFO:tasks.workunit.client.1.smithi138.stdout:2/485: sync 2024-05-27T09:29:42.291 INFO:tasks.workunit.client.1.smithi138.stdout:8/579: sync 2024-05-27T09:29:42.291 INFO:tasks.workunit.client.1.smithi138.stdout:5/459: sync 2024-05-27T09:29:42.294 INFO:tasks.workunit.client.1.smithi138.stdout:0/532: sync 2024-05-27T09:29:42.294 INFO:tasks.workunit.client.1.smithi138.stdout:4/563: sync 2024-05-27T09:29:42.294 INFO:tasks.workunit.client.1.smithi138.stdout:4/564: creat d7/d6e/fc4 x:0 0 0 2024-05-27T09:29:42.294 INFO:tasks.workunit.client.1.smithi138.stdout:7/505: sync 2024-05-27T09:29:42.294 INFO:tasks.workunit.client.1.smithi138.stdout:7/506: stat d0/d23/da8/fa2 0 2024-05-27T09:29:42.294 INFO:tasks.workunit.client.1.smithi138.stdout:7/507: chown d0/d23/d3a/l90 0 1 2024-05-27T09:29:42.295 INFO:tasks.workunit.client.1.smithi138.stdout:7/508: chown d0/d23/d3a/f55 1 1 2024-05-27T09:29:42.295 INFO:tasks.workunit.client.1.smithi138.stdout:7/509: fsync d0/d23/f4f 0 2024-05-27T09:29:42.295 INFO:tasks.workunit.client.1.smithi138.stdout:5/460: write d3/d12/d16/d1d/f24 [3552107,14786] 0 2024-05-27T09:29:42.295 INFO:tasks.workunit.client.1.smithi138.stdout:5/461: truncate d3/d12/d26/f45 1312178 0 2024-05-27T09:29:42.296 INFO:tasks.workunit.client.1.smithi138.stdout:5/462: truncate d3/d12/d32/d60/f7b 967208 0 2024-05-27T09:29:42.300 INFO:tasks.workunit.client.1.smithi138.stdout:2/486: symlink d1/d15/d2d/d17/d25/d4d/d4e/d60/la7 0 2024-05-27T09:29:42.300 INFO:tasks.workunit.client.1.smithi138.stdout:2/487: truncate d1/d15/d2d/d1d/f99 84818 0 2024-05-27T09:29:42.310 INFO:tasks.workunit.client.1.smithi138.stdout:4/565: mkdir d7/df/d35/d4c/d59/d5d/d64/dc5 0 2024-05-27T09:29:42.310 INFO:tasks.workunit.client.1.smithi138.stdout:4/566: read - d7/df/d35/d4c/d59/f9d zero size 2024-05-27T09:29:42.310 INFO:tasks.workunit.client.1.smithi138.stdout:4/567: stat d7/df/d35/c61 0 2024-05-27T09:29:42.313 INFO:tasks.workunit.client.1.smithi138.stdout:4/568: symlink d7/df/d5b/da1/lc6 0 2024-05-27T09:29:42.317 INFO:tasks.workunit.client.1.smithi138.stdout:9/552: dread d6/db/de/d18/d24/f73 [0,4194304] 0 2024-05-27T09:29:42.317 INFO:tasks.workunit.client.1.smithi138.stdout:9/553: write d6/da/f9e [209182,94720] 0 2024-05-27T09:29:42.317 INFO:tasks.workunit.client.1.smithi138.stdout:4/569: symlink d7/d6e/lc7 0 2024-05-27T09:29:42.320 INFO:tasks.workunit.client.1.smithi138.stdout:3/520: write d0/d2/da/df/d10/d32/f5b [3756401,16575] 0 2024-05-27T09:29:42.322 INFO:tasks.workunit.client.1.smithi138.stdout:6/712: dwrite d3/d11/d2d/d37/dd1/d47/f7a [0,4194304] 0 2024-05-27T09:29:42.322 INFO:tasks.workunit.client.1.smithi138.stdout:9/554: rename d6/da/d2f/d35/d55/d75/ca6 to d6/db/de/d18/d27/d4c/cb4 0 2024-05-27T09:29:42.326 INFO:tasks.workunit.client.1.smithi138.stdout:3/521: symlink d0/d2/da/df/d10/d2a/d48/lac 0 2024-05-27T09:29:42.328 INFO:tasks.workunit.client.1.smithi138.stdout:1/542: dwrite d4/d48/d39/d5f/d4d/d2b/f6c [0,4194304] 0 2024-05-27T09:29:42.331 INFO:tasks.workunit.client.1.smithi138.stdout:9/555: unlink f3 0 2024-05-27T09:29:42.332 INFO:tasks.workunit.client.0.smithi119.stdout:9/970: write d18/d19/d1b/d40/d44/f45 [1868324,71929] 0 2024-05-27T09:29:42.332 INFO:tasks.workunit.client.0.smithi119.stdout:9/971: readlink d18/d19/d1b/d40/l84 0 2024-05-27T09:29:42.334 INFO:tasks.workunit.client.1.smithi138.stdout:3/522: mkdir d0/d2/da/dad 0 2024-05-27T09:29:42.334 INFO:tasks.workunit.client.1.smithi138.stdout:1/543: read d4/d1a/d2e/f41 [3880110,29737] 0 2024-05-27T09:29:42.335 INFO:tasks.workunit.client.0.smithi119.stdout:9/972: creat d18/d19/d1b/d40/f139 x:0 0 0 2024-05-27T09:29:42.337 INFO:tasks.workunit.client.0.smithi119.stdout:9/973: symlink d18/d19/d1b/d2a/l13a 0 2024-05-27T09:29:42.337 INFO:tasks.workunit.client.0.smithi119.stdout:9/974: chown d18/d19/d1b/d40/d44/f45 111 1 2024-05-27T09:29:42.343 INFO:tasks.workunit.client.0.smithi119.stdout:9/975: truncate d18/f9a 4129082 0 2024-05-27T09:29:42.349 INFO:tasks.workunit.client.0.smithi119.stdout:9/976: mknod d18/d19/d61/c13b 0 2024-05-27T09:29:42.349 INFO:tasks.workunit.client.0.smithi119.stdout:9/977: write d18/d19/d1b/d40/d44/dcc/f135 [854631,59632] 0 2024-05-27T09:29:42.353 INFO:tasks.workunit.client.1.smithi138.stdout:4/570: rmdir d7/d6e 39 2024-05-27T09:29:42.359 INFO:tasks.workunit.client.1.smithi138.stdout:8/580: dread f1 [0,4194304] 0 2024-05-27T09:29:42.359 INFO:tasks.workunit.client.1.smithi138.stdout:8/581: chown d3/d8/d32/d34/d38/f92 648229 1 2024-05-27T09:29:42.359 INFO:tasks.workunit.client.1.smithi138.stdout:8/582: fdatasync d3/d8/d32/d3f/d9c/f96 0 2024-05-27T09:29:42.360 INFO:tasks.workunit.client.1.smithi138.stdout:8/583: mknod d3/d8/d24/d4d/cc3 0 2024-05-27T09:29:42.361 INFO:tasks.workunit.client.1.smithi138.stdout:8/584: readlink d3/d8/d64/la8 0 2024-05-27T09:29:42.368 INFO:tasks.workunit.client.0.smithi119.stdout:9/978: mkdir d18/db4/dbe/d13c 0 2024-05-27T09:29:42.368 INFO:tasks.workunit.client.0.smithi119.stdout:9/979: fsync d18/d19/d1b/d40/db7/f11d 0 2024-05-27T09:29:42.370 INFO:tasks.workunit.client.1.smithi138.stdout:4/571: rename d7/d27/c40 to d7/df/d35/d4c/d59/d5d/d64/dc5/cc8 0 2024-05-27T09:29:42.372 INFO:tasks.workunit.client.1.smithi138.stdout:4/572: creat d7/d27/daa/fc9 x:0 0 0 2024-05-27T09:29:42.375 INFO:tasks.workunit.client.1.smithi138.stdout:7/510: dread d0/d23/f47 [0,4194304] 0 2024-05-27T09:29:42.375 INFO:tasks.workunit.client.1.smithi138.stdout:7/511: write d0/d7/fe [1649162,122517] 0 2024-05-27T09:29:42.375 INFO:tasks.workunit.client.1.smithi138.stdout:2/488: dread d1/d15/d2d/d17/d25/d9d/f49 [0,4194304] 0 2024-05-27T09:29:42.375 INFO:tasks.workunit.client.1.smithi138.stdout:4/573: unlink d7/df/d35/d4c/d59/d5d/f87 0 2024-05-27T09:29:42.377 INFO:tasks.workunit.client.0.smithi119.stdout:9/980: symlink d18/d19/d1b/l13d 0 2024-05-27T09:29:42.378 INFO:tasks.workunit.client.1.smithi138.stdout:4/574: rename d7/df/d35/d4c/lc3 to d7/df/lca 0 2024-05-27T09:29:42.378 INFO:tasks.workunit.client.1.smithi138.stdout:4/575: fsync d7/df/d35/d4c/d59/f58 0 2024-05-27T09:29:42.379 INFO:tasks.workunit.client.1.smithi138.stdout:3/523: dread d0/d2/da/f6b [0,4194304] 0 2024-05-27T09:29:42.383 INFO:tasks.workunit.client.1.smithi138.stdout:4/576: mkdir d7/dba/dcb 0 2024-05-27T09:29:42.383 INFO:tasks.workunit.client.1.smithi138.stdout:4/577: write d7/d27/d3a/fb0 [274821,121058] 0 2024-05-27T09:29:42.384 INFO:tasks.workunit.client.1.smithi138.stdout:2/489: mknod d1/d15/d2d/d2e/ca8 0 2024-05-27T09:29:42.386 INFO:tasks.workunit.client.1.smithi138.stdout:2/490: symlink d1/d15/d2d/d17/d25/d4d/d6d/la9 0 2024-05-27T09:29:42.392 INFO:tasks.workunit.client.1.smithi138.stdout:7/512: link d0/d7/f39 d0/fac 0 2024-05-27T09:29:42.392 INFO:tasks.workunit.client.1.smithi138.stdout:7/513: dread - d0/d7/d98/fa4 zero size 2024-05-27T09:29:42.393 INFO:tasks.workunit.client.1.smithi138.stdout:7/514: fdatasync d0/d7/f12 0 2024-05-27T09:29:42.393 INFO:tasks.workunit.client.1.smithi138.stdout:7/515: chown d0/d23/d49/d78 0 1 2024-05-27T09:29:42.393 INFO:tasks.workunit.client.1.smithi138.stdout:7/516: dread - d0/f77 zero size 2024-05-27T09:29:42.394 INFO:tasks.workunit.client.1.smithi138.stdout:1/544: dread d4/d48/d39/f45 [0,4194304] 0 2024-05-27T09:29:42.394 INFO:tasks.workunit.client.1.smithi138.stdout:7/517: creat d0/d7/d60/d9d/fad x:0 0 0 2024-05-27T09:29:42.394 INFO:tasks.workunit.client.1.smithi138.stdout:7/518: write d0/d7/d64/f6e [188690,104812] 0 2024-05-27T09:29:42.395 INFO:tasks.workunit.client.1.smithi138.stdout:8/585: write d3/d8/d64/f2e [589532,47417] 0 2024-05-27T09:29:42.400 INFO:tasks.workunit.client.1.smithi138.stdout:7/519: truncate d0/f24 1163026 0 2024-05-27T09:29:42.400 INFO:tasks.workunit.client.1.smithi138.stdout:7/520: write d0/f1a [5482800,5396] 0 2024-05-27T09:29:42.402 INFO:tasks.workunit.client.0.smithi119.stdout:9/981: write d18/d19/d1b/d29/fa7 [2756216,123411] 0 2024-05-27T09:29:42.402 INFO:tasks.workunit.client.0.smithi119.stdout:9/982: chown d18/d19/d1b/d40/d7b/dc6/d118 134 1 2024-05-27T09:29:42.405 INFO:tasks.workunit.client.0.smithi119.stdout:9/983: mknod d18/d19/d1b/d2a/d9f/c13e 0 2024-05-27T09:29:42.406 INFO:tasks.workunit.client.0.smithi119.stdout:9/984: write d18/ddf/fea [48390,31194] 0 2024-05-27T09:29:42.408 INFO:tasks.workunit.client.1.smithi138.stdout:0/533: dwrite da/d17/f8c [0,4194304] 0 2024-05-27T09:29:42.411 INFO:tasks.workunit.client.1.smithi138.stdout:0/534: chown da/d2a/d8e/d43/f48 2576320 1 2024-05-27T09:29:42.411 INFO:tasks.workunit.client.1.smithi138.stdout:1/545: unlink d4/d3c/d51/l83 0 2024-05-27T09:29:42.412 INFO:tasks.workunit.client.1.smithi138.stdout:4/578: dread d7/fb [0,4194304] 0 2024-05-27T09:29:42.413 INFO:tasks.workunit.client.1.smithi138.stdout:7/521: dread d0/d23/d49/d78/f9b [0,4194304] 0 2024-05-27T09:29:42.414 INFO:tasks.workunit.client.0.smithi119.stdout:9/985: mkdir d18/d19/d1b/d40/d13f 0 2024-05-27T09:29:42.419 INFO:tasks.workunit.client.0.smithi119.stdout:9/986: creat d18/d19/d9b/f140 x:0 0 0 2024-05-27T09:29:42.419 INFO:tasks.workunit.client.0.smithi119.stdout:9/987: chown f10 51485939 1 2024-05-27T09:29:42.419 INFO:tasks.workunit.client.1.smithi138.stdout:1/546: rename d4/d1a/d2e/f7c to d4/d48/d39/d5f/d93/faf 0 2024-05-27T09:29:42.419 INFO:tasks.workunit.client.1.smithi138.stdout:1/547: chown d4/d1a/c8d 114 1 2024-05-27T09:29:42.419 INFO:tasks.workunit.client.1.smithi138.stdout:4/579: symlink d7/df/d35/d4c/d59/d5d/d64/dc5/lcc 0 2024-05-27T09:29:42.422 INFO:tasks.workunit.client.1.smithi138.stdout:5/463: dwrite d3/fd [0,4194304] 0 2024-05-27T09:29:42.425 INFO:tasks.workunit.client.1.smithi138.stdout:8/586: dread d3/d8/d32/d3f/d9c/f96 [0,4194304] 0 2024-05-27T09:29:42.425 INFO:tasks.workunit.client.1.smithi138.stdout:8/587: write d3/d8/d32/d34/d38/fad [793655,19807] 0 2024-05-27T09:29:42.425 INFO:tasks.workunit.client.1.smithi138.stdout:8/588: rename d3/d8/d24/d55/d66 to d3/d8/d24/d55/d66/dc4 22 2024-05-27T09:29:42.426 INFO:tasks.workunit.client.0.smithi119.stdout:9/988: rename d18/d19/d1b/d29/f2f to d18/db4/dbe/d13c/f141 0 2024-05-27T09:29:42.426 INFO:tasks.workunit.client.1.smithi138.stdout:1/548: getdents d4/d48/d39/d5f 0 2024-05-27T09:29:42.426 INFO:tasks.workunit.client.1.smithi138.stdout:3/524: dread d0/d80/f92 [0,4194304] 0 2024-05-27T09:29:42.432 INFO:tasks.workunit.client.1.smithi138.stdout:1/549: creat d4/d1a/d2e/fb0 x:0 0 0 2024-05-27T09:29:42.432 INFO:tasks.workunit.client.1.smithi138.stdout:9/556: dwrite d6/da/d29/f89 [0,4194304] 0 2024-05-27T09:29:42.434 INFO:tasks.workunit.client.1.smithi138.stdout:1/550: symlink d4/d1a/d2e/d4a/d7e/lb1 0 2024-05-27T09:29:42.435 INFO:tasks.workunit.client.1.smithi138.stdout:1/551: creat d4/d48/d39/d5f/d93/fb2 x:0 0 0 2024-05-27T09:29:42.435 INFO:tasks.workunit.client.1.smithi138.stdout:1/552: chown d4/d1a/d2e/d32/c43 25 1 2024-05-27T09:29:42.437 INFO:tasks.workunit.client.1.smithi138.stdout:1/553: symlink d4/d1a/d2e/d4a/d7e/lb3 0 2024-05-27T09:29:42.440 INFO:tasks.workunit.client.1.smithi138.stdout:8/589: mkdir d3/d8/d24/d55/d94/dc5 0 2024-05-27T09:29:42.456 INFO:tasks.workunit.client.1.smithi138.stdout:9/557: symlink d6/da/d2f/d35/d55/d75/d8c/lb5 0 2024-05-27T09:29:42.490 INFO:tasks.workunit.client.0.smithi119.stdout:9/989: dwrite d18/db4/d108/f121 [0,4194304] 0 2024-05-27T09:29:42.493 INFO:tasks.workunit.client.0.smithi119.stdout:9/990: symlink d18/d19/d1b/d29/d11b/l142 0 2024-05-27T09:29:42.494 INFO:tasks.workunit.client.0.smithi119.stdout:9/991: chown d18/d19/d1b/d2a/l12a 595237 1 2024-05-27T09:29:42.494 INFO:tasks.workunit.client.0.smithi119.stdout:9/992: write d18/d19/d9b/fb3 [1005697,1759] 0 2024-05-27T09:29:42.494 INFO:tasks.workunit.client.1.smithi138.stdout:8/590: read d3/d8/d11/f26 [987417,57432] 0 2024-05-27T09:29:42.498 INFO:tasks.workunit.client.1.smithi138.stdout:8/591: creat d3/d8/d32/d3f/d9c/d9f/d86/fc6 x:0 0 0 2024-05-27T09:29:42.503 INFO:tasks.workunit.client.1.smithi138.stdout:8/592: link d3/d8/d24/d55/d7e/f89 d3/d8/d32/d34/d38/fc7 0 2024-05-27T09:29:42.504 INFO:tasks.workunit.client.1.smithi138.stdout:0/535: dwrite da/d17/f44 [4194304,4194304] 0 2024-05-27T09:29:42.504 INFO:tasks.workunit.client.1.smithi138.stdout:0/536: write da/d2a/f30 [229619,77021] 0 2024-05-27T09:29:42.505 INFO:tasks.workunit.client.1.smithi138.stdout:0/537: chown da/d2a/d2b/c7f 238 1 2024-05-27T09:29:42.505 INFO:tasks.workunit.client.1.smithi138.stdout:0/538: read da/d2a/d8e/d1f/f21 [94153,47500] 0 2024-05-27T09:29:42.506 INFO:tasks.workunit.client.1.smithi138.stdout:0/539: creat da/d2a/d8e/d1f/d4f/fb3 x:0 0 0 2024-05-27T09:29:42.506 INFO:tasks.workunit.client.1.smithi138.stdout:0/540: dread - da/d2a/d8e/d1f/d42/d63/f91 zero size 2024-05-27T09:29:42.506 INFO:tasks.workunit.client.1.smithi138.stdout:0/541: creat da/d2a/d55/fb4 x:0 0 0 2024-05-27T09:29:42.517 INFO:tasks.workunit.client.1.smithi138.stdout:8/593: rmdir d3/d8/d24/d55 39 2024-05-27T09:29:42.525 INFO:tasks.workunit.client.1.smithi138.stdout:8/594: truncate d3/d8/d32/d3f/f6b 2681751 0 2024-05-27T09:29:42.526 INFO:tasks.workunit.client.0.smithi119.stdout:9/993: write d18/d19/d1b/d40/fe9 [1824446,74183] 0 2024-05-27T09:29:42.526 INFO:tasks.workunit.client.0.smithi119.stdout:9/994: fsync f10 0 2024-05-27T09:29:42.526 INFO:tasks.workunit.client.0.smithi119.stdout:9/995: write d18/db4/dbe/ff8 [160590,31947] 0 2024-05-27T09:29:42.527 INFO:tasks.workunit.client.1.smithi138.stdout:8/595: mknod d3/d8/d24/d55/d94/da5/cc8 0 2024-05-27T09:29:42.527 INFO:tasks.workunit.client.0.smithi119.stdout:9/996: creat d18/d19/d9b/f143 x:0 0 0 2024-05-27T09:29:42.536 INFO:tasks.workunit.client.1.smithi138.stdout:4/580: dwrite d7/d27/daa/fc9 [0,4194304] 0 2024-05-27T09:29:42.537 INFO:tasks.workunit.client.1.smithi138.stdout:9/558: dread d6/db/de/f2b [4194304,4194304] 0 2024-05-27T09:29:42.557 INFO:tasks.workunit.client.1.smithi138.stdout:7/522: dwrite d0/d23/fa0 [0,4194304] 0 2024-05-27T09:29:42.557 INFO:tasks.workunit.client.1.smithi138.stdout:7/523: fdatasync d0/d23/f50 0 2024-05-27T09:29:42.560 INFO:tasks.workunit.client.1.smithi138.stdout:4/581: dread d7/d27/f34 [0,4194304] 0 2024-05-27T09:29:42.560 INFO:tasks.workunit.client.1.smithi138.stdout:7/524: creat d0/d2b/d80/fae x:0 0 0 2024-05-27T09:29:42.560 INFO:tasks.workunit.client.1.smithi138.stdout:7/525: fsync d0/f24 0 2024-05-27T09:29:42.567 INFO:tasks.workunit.client.1.smithi138.stdout:3/525: dwrite d0/d3f/d50/f63 [0,4194304] 0 2024-05-27T09:29:42.567 INFO:tasks.workunit.client.1.smithi138.stdout:3/526: dread - d0/d3f/f7f zero size 2024-05-27T09:29:42.569 INFO:tasks.workunit.client.1.smithi138.stdout:3/527: creat d0/d3f/d60/daa/fae x:0 0 0 2024-05-27T09:29:42.570 INFO:tasks.workunit.client.1.smithi138.stdout:3/528: write d0/d2/da/df/d10/f77 [372540,111127] 0 2024-05-27T09:29:42.570 INFO:tasks.workunit.client.1.smithi138.stdout:3/529: write d0/d2/da/df/d10/d32/f1d [175077,82904] 0 2024-05-27T09:29:42.571 INFO:tasks.workunit.client.1.smithi138.stdout:5/464: dwrite d3/d12/d26/f4b [0,4194304] 0 2024-05-27T09:29:42.573 INFO:tasks.workunit.client.1.smithi138.stdout:3/530: mkdir d0/d3f/d60/daf 0 2024-05-27T09:29:42.583 INFO:tasks.workunit.client.0.smithi119.stdout:9/997: dwrite d18/d19/d61/f6d [0,4194304] 0 2024-05-27T09:29:42.584 INFO:tasks.workunit.client.1.smithi138.stdout:7/526: dread d0/fac [0,4194304] 0 2024-05-27T09:29:42.584 INFO:tasks.workunit.client.1.smithi138.stdout:7/527: chown d0/d23/c5b 1 1 2024-05-27T09:29:42.584 INFO:tasks.workunit.client.1.smithi138.stdout:7/528: write d0/d7/d43/fa9 [791323,119033] 0 2024-05-27T09:29:42.584 INFO:tasks.workunit.client.1.smithi138.stdout:1/554: dwrite d4/d3c/f4e [0,4194304] 0 2024-05-27T09:29:42.587 INFO:tasks.workunit.client.1.smithi138.stdout:4/582: creat d7/df/d35/d4c/d59/d5d/d64/d6a/fcd x:0 0 0 2024-05-27T09:29:42.591 INFO:tasks.workunit.client.1.smithi138.stdout:5/465: creat d3/d75/f94 x:0 0 0 2024-05-27T09:29:42.595 INFO:tasks.workunit.client.1.smithi138.stdout:7/529: rename d0/d7 to d0/d2b/daf 0 2024-05-27T09:29:42.597 INFO:tasks.workunit.client.1.smithi138.stdout:7/530: mknod d0/d2b/daf/d60/d9d/cb0 0 2024-05-27T09:29:42.597 INFO:tasks.workunit.client.1.smithi138.stdout:3/531: write d0/d2/da/df/d10/f38 [3733202,16032] 0 2024-05-27T09:29:42.606 INFO:tasks.workunit.client.1.smithi138.stdout:7/531: mknod d0/d23/da8/d8e/cb1 0 2024-05-27T09:29:42.606 INFO:tasks.workunit.client.1.smithi138.stdout:7/532: fdatasync d0/d2b/f40 0 2024-05-27T09:29:42.606 INFO:tasks.workunit.client.1.smithi138.stdout:0/542: getdents da/d2a/d55 0 2024-05-27T09:29:42.608 INFO:tasks.workunit.client.1.smithi138.stdout:3/532: dread d0/d2/f45 [0,4194304] 0 2024-05-27T09:29:42.609 INFO:tasks.workunit.client.1.smithi138.stdout:4/583: symlink d7/df/d5b/da1/db1/lce 0 2024-05-27T09:29:42.610 INFO:tasks.workunit.client.1.smithi138.stdout:3/533: mknod d0/d2/cb0 0 2024-05-27T09:29:42.611 INFO:tasks.workunit.client.1.smithi138.stdout:3/534: rmdir d0/d2/da/df/d10/d2a/d48/d90 39 2024-05-27T09:29:42.611 INFO:tasks.workunit.client.1.smithi138.stdout:7/533: write d0/d23/d49/f4a [4360442,8512] 0 2024-05-27T09:29:42.611 INFO:tasks.workunit.client.1.smithi138.stdout:9/559: dwrite d6/db/de/d18/d27/d4c/fae [0,4194304] 0 2024-05-27T09:29:42.612 INFO:tasks.workunit.client.1.smithi138.stdout:9/560: chown d6/da/d2f/d35/d55/l9c 71 1 2024-05-27T09:29:42.612 INFO:tasks.workunit.client.0.smithi119.stdout:9/998: dread d18/d19/d1b/d29/f47 [0,4194304] 0 2024-05-27T09:29:42.612 INFO:tasks.workunit.client.0.smithi119.stdout:9/999: chown d18/d19/d1b/d40/d7b/f7e 511618 1 2024-05-27T09:29:42.615 INFO:tasks.workunit.client.1.smithi138.stdout:8/596: dwrite d3/d8/d64/f7c [0,4194304] 0 2024-05-27T09:29:42.616 INFO:tasks.workunit.client.1.smithi138.stdout:9/561: creat d6/db/dc/fb6 x:0 0 0 2024-05-27T09:29:42.616 INFO:tasks.workunit.client.1.smithi138.stdout:7/534: getdents d0/d2b/daf/d60/d9d 0 2024-05-27T09:29:42.616 INFO:tasks.workunit.client.1.smithi138.stdout:9/562: write d6/db/de/d18/d24/f65 [362177,105714] 0 2024-05-27T09:29:42.617 INFO:tasks.workunit.client.1.smithi138.stdout:4/584: rename d7/d27 to d7/dba/dcf 0 2024-05-27T09:29:42.620 INFO:tasks.workunit.client.1.smithi138.stdout:7/535: creat d0/d23/d3c/d84/fb2 x:0 0 0 2024-05-27T09:29:42.621 INFO:tasks.workunit.client.1.smithi138.stdout:7/536: mknod d0/d2b/d80/cb3 0 2024-05-27T09:29:42.621 INFO:tasks.workunit.client.1.smithi138.stdout:7/537: chown d0/f83 446515 1 2024-05-27T09:29:42.623 INFO:tasks.workunit.client.1.smithi138.stdout:7/538: link d0/d2b/f5f d0/d2b/daf/d57/d91/fb4 0 2024-05-27T09:29:42.623 INFO:tasks.workunit.client.1.smithi138.stdout:7/539: fdatasync d0/f45 0 2024-05-27T09:29:42.623 INFO:tasks.workunit.client.1.smithi138.stdout:3/535: rmdir d0/d2/da/df/d10/d2a/d48 39 2024-05-27T09:29:42.625 INFO:tasks.workunit.client.0.smithi119.stderr:+ rm -rf -- ./tmp.uwTadhjwLg 2024-05-27T09:29:42.626 INFO:tasks.workunit.client.1.smithi138.stdout:7/540: truncate d0/d2b/daf/d57/d91/fb4 1782576 0 2024-05-27T09:29:42.627 INFO:tasks.workunit.client.1.smithi138.stdout:3/536: creat d0/d3f/d60/daf/fb1 x:0 0 0 2024-05-27T09:29:42.628 INFO:tasks.workunit.client.1.smithi138.stdout:7/541: symlink d0/d2b/daf/d57/d91/lb5 0 2024-05-27T09:29:42.631 INFO:tasks.workunit.client.1.smithi138.stdout:5/466: dwrite d3/d12/d32/d60/f8e [0,4194304] 0 2024-05-27T09:29:42.631 INFO:tasks.workunit.client.1.smithi138.stdout:5/467: write d3/d12/f64 [1039384,25971] 0 2024-05-27T09:29:42.633 INFO:tasks.workunit.client.1.smithi138.stdout:8/597: dread d3/d8/f33 [0,4194304] 0 2024-05-27T09:29:42.633 INFO:tasks.workunit.client.1.smithi138.stdout:8/598: dread - d3/d8/d24/d55/d94/da5/fb3 zero size 2024-05-27T09:29:42.634 INFO:tasks.workunit.client.1.smithi138.stdout:8/599: mknod d3/d8/d72/cc9 0 2024-05-27T09:29:42.634 INFO:tasks.workunit.client.1.smithi138.stdout:5/468: creat d3/d12/d16/d1d/d23/d3b/f95 x:0 0 0 2024-05-27T09:29:42.634 INFO:tasks.workunit.client.1.smithi138.stdout:3/537: mkdir d0/d2/da/df/d10/d2a/db2 0 2024-05-27T09:29:42.640 INFO:tasks.workunit.client.1.smithi138.stdout:8/600: creat d3/d8/d64/fca x:0 0 0 2024-05-27T09:29:42.640 INFO:tasks.workunit.client.1.smithi138.stdout:8/601: dread - d3/d8/d11/f99 zero size 2024-05-27T09:29:42.640 INFO:tasks.workunit.client.1.smithi138.stdout:8/602: dread - d3/d8/d24/d4d/f97 zero size 2024-05-27T09:29:42.641 INFO:tasks.workunit.client.1.smithi138.stdout:3/538: symlink d0/d2/da/df/d10/d2a/d48/d90/lb3 0 2024-05-27T09:29:42.641 INFO:tasks.workunit.client.1.smithi138.stdout:3/539: dread - d0/d3f/d60/daa/fae zero size 2024-05-27T09:29:42.645 INFO:tasks.workunit.client.1.smithi138.stdout:9/563: dread d6/db/de/d18/d27/d46/f47 [0,4194304] 0 2024-05-27T09:29:42.645 INFO:tasks.workunit.client.1.smithi138.stdout:9/564: fdatasync d6/da/f5d 0 2024-05-27T09:29:42.705 INFO:tasks.workunit.client.1.smithi138.stdout:4/585: dwrite d7/dba/dcf/f9a [0,4194304] 0 2024-05-27T09:29:42.706 INFO:tasks.workunit.client.1.smithi138.stdout:4/586: chown d7/dba/dcf/d3a/c82 15323 1 2024-05-27T09:29:42.706 INFO:tasks.workunit.client.1.smithi138.stdout:4/587: write d7/dba/dcf/f48 [1706251,14751] 0 2024-05-27T09:29:42.707 INFO:tasks.workunit.client.1.smithi138.stdout:4/588: creat d7/df/fd0 x:0 0 0 2024-05-27T09:29:42.710 INFO:tasks.workunit.client.1.smithi138.stdout:4/589: creat d7/d6e/fd1 x:0 0 0 2024-05-27T09:29:42.710 INFO:tasks.workunit.client.1.smithi138.stdout:4/590: fdatasync d7/df/d35/d4c/f69 0 2024-05-27T09:29:42.729 INFO:tasks.workunit.client.1.smithi138.stdout:7/542: dwrite d0/d2b/daf/d60/f9f [0,4194304] 0 2024-05-27T09:29:42.729 INFO:tasks.workunit.client.1.smithi138.stdout:8/603: dread d3/d8/d24/d4d/f61 [0,4194304] 0 2024-05-27T09:29:42.732 INFO:tasks.workunit.client.1.smithi138.stdout:7/543: mkdir d0/d2b/daf/d98/db6 0 2024-05-27T09:29:42.734 INFO:tasks.workunit.client.1.smithi138.stdout:5/469: dwrite d3/d12/f4d [0,4194304] 0 2024-05-27T09:29:42.734 INFO:tasks.workunit.client.1.smithi138.stdout:5/470: stat d3/d12/d16/d1d/l5f 0 2024-05-27T09:29:42.740 INFO:tasks.workunit.client.1.smithi138.stdout:5/471: unlink d3/f10 0 2024-05-27T09:29:42.741 INFO:tasks.workunit.client.1.smithi138.stdout:8/604: dread d3/d8/d64/f5e [0,4194304] 0 2024-05-27T09:29:42.748 INFO:tasks.workunit.client.1.smithi138.stdout:5/472: rename d3/d12/d32/l50 to d3/d12/d16/l96 0 2024-05-27T09:29:42.748 INFO:tasks.workunit.client.1.smithi138.stdout:5/473: write d3/d12/d16/d1d/f55 [85629,71794] 0 2024-05-27T09:29:42.748 INFO:tasks.workunit.client.1.smithi138.stdout:5/474: write d3/d12/d16/d1d/d22/d2b/d36/f43 [912387,44849] 0 2024-05-27T09:29:42.748 INFO:tasks.workunit.client.1.smithi138.stdout:5/475: write d3/f1b [4657295,108532] 0 2024-05-27T09:29:42.760 INFO:tasks.workunit.client.1.smithi138.stdout:5/476: dread d3/f19 [0,4194304] 0 2024-05-27T09:29:42.762 INFO:tasks.workunit.client.1.smithi138.stdout:5/477: rename d3/d12/d26/f33 to d3/d12/d16/d1d/f97 0 2024-05-27T09:29:42.762 INFO:tasks.workunit.client.1.smithi138.stdout:5/478: write d3/d12/d32/d87/f8f [780723,90424] 0 2024-05-27T09:29:42.764 INFO:tasks.workunit.client.1.smithi138.stdout:5/479: mknod d3/d12/d32/d60/c98 0 2024-05-27T09:29:42.764 INFO:tasks.workunit.client.1.smithi138.stdout:7/544: dwrite d0/d23/f50 [0,4194304] 0 2024-05-27T09:29:42.766 INFO:tasks.workunit.client.1.smithi138.stdout:5/480: mkdir d3/d12/d16/d1d/d22/d2b/d99 0 2024-05-27T09:29:42.770 INFO:tasks.workunit.client.1.smithi138.stdout:5/481: truncate d3/d12/d16/d1d/f24 2001911 0 2024-05-27T09:29:42.779 INFO:tasks.workunit.client.1.smithi138.stdout:5/482: dread d3/d12/d16/d1d/d22/f3a [0,4194304] 0 2024-05-27T09:29:42.779 INFO:tasks.workunit.client.1.smithi138.stdout:5/483: dread - d3/d75/f94 zero size 2024-05-27T09:29:42.779 INFO:tasks.workunit.client.1.smithi138.stdout:5/484: stat d3/f18 0 2024-05-27T09:29:42.783 INFO:tasks.workunit.client.1.smithi138.stdout:5/485: rename d3/d12/d16/f66 to d3/d12/d16/d1d/d22/d2b/d36/f9a 0 2024-05-27T09:29:42.783 INFO:tasks.workunit.client.1.smithi138.stdout:5/486: read d3/d12/d26/f57 [133513,58826] 0 2024-05-27T09:29:42.783 INFO:tasks.workunit.client.1.smithi138.stdout:5/487: creat d3/d12/f9b x:0 0 0 2024-05-27T09:29:42.783 INFO:tasks.workunit.client.1.smithi138.stdout:5/488: readlink d3/d12/d16/d1d/d23/l56 0 2024-05-27T09:29:42.785 INFO:tasks.workunit.client.1.smithi138.stdout:9/565: dwrite d6/db/de/d18/f57 [4194304,4194304] 0 2024-05-27T09:29:42.787 INFO:tasks.workunit.client.1.smithi138.stdout:7/545: dread d0/d23/d3a/f46 [0,4194304] 0 2024-05-27T09:29:42.787 INFO:tasks.workunit.client.1.smithi138.stdout:3/540: dwrite d0/d2/f13 [0,4194304] 0 2024-05-27T09:29:42.790 INFO:tasks.workunit.client.1.smithi138.stdout:5/489: creat d3/d12/d16/d1d/d30/f9c x:0 0 0 2024-05-27T09:29:42.790 INFO:tasks.workunit.client.1.smithi138.stdout:5/490: chown d3/d12/d16 2663 1 2024-05-27T09:29:42.790 INFO:tasks.workunit.client.1.smithi138.stdout:5/491: truncate d3/d75/f94 405896 0 2024-05-27T09:29:42.793 INFO:tasks.workunit.client.1.smithi138.stdout:7/546: mkdir d0/d23/d3c/d3f/db7 0 2024-05-27T09:29:42.805 INFO:tasks.workunit.client.1.smithi138.stdout:7/547: mkdir d0/d2b/daf/d57/d91/db8 0 2024-05-27T09:29:42.808 INFO:tasks.workunit.client.1.smithi138.stdout:9/566: rename d6/db/de/d18/d27/d9a to d6/da/d29/d8f/db7 0 2024-05-27T09:29:42.808 INFO:tasks.workunit.client.1.smithi138.stdout:9/567: stat d6/da/d29/d61/l66 0 2024-05-27T09:29:42.813 INFO:tasks.workunit.client.1.smithi138.stdout:7/548: rename d0/d23/d49/f4a to d0/d23/d3c/fb9 0 2024-05-27T09:29:42.817 INFO:tasks.workunit.client.1.smithi138.stdout:8/605: dwrite d3/d8/fac [0,4194304] 0 2024-05-27T09:29:42.820 INFO:tasks.workunit.client.1.smithi138.stdout:9/568: creat d6/da/d29/d8f/db7/fb8 x:0 0 0 2024-05-27T09:29:42.821 INFO:tasks.workunit.client.1.smithi138.stdout:9/569: write d6/db/de/f31 [1089497,59993] 0 2024-05-27T09:29:42.830 INFO:tasks.workunit.client.1.smithi138.stdout:9/570: mknod d6/db/de/d18/cb9 0 2024-05-27T09:29:42.835 INFO:tasks.workunit.client.1.smithi138.stdout:7/549: mkdir d0/d2b/daf/dba 0 2024-05-27T09:29:42.838 INFO:tasks.workunit.client.1.smithi138.stdout:9/571: truncate d6/db/dc/f12 1396747 0 2024-05-27T09:29:42.842 INFO:tasks.workunit.client.1.smithi138.stdout:3/541: dwrite d0/d2/da/df/d10/f77 [0,4194304] 0 2024-05-27T09:29:42.853 INFO:tasks.workunit.client.1.smithi138.stdout:9/572: rename d6/db/de/c98 to d6/db/de/d18/d27/d46/cba 0 2024-05-27T09:29:42.854 INFO:tasks.workunit.client.1.smithi138.stdout:5/492: dwrite d3/d12/f77 [0,4194304] 0 2024-05-27T09:29:42.865 INFO:tasks.workunit.client.1.smithi138.stdout:8/606: dwrite d3/d8/d32/d3f/d9c/d9f/d86/f93 [0,4194304] 0 2024-05-27T09:29:42.865 INFO:tasks.workunit.client.1.smithi138.stdout:9/573: mknod d6/db/de/d18/d27/cbb 0 2024-05-27T09:29:42.865 INFO:tasks.workunit.client.1.smithi138.stdout:9/574: chown d6/da/d26/fb0 0 1 2024-05-27T09:29:42.867 INFO:tasks.workunit.client.1.smithi138.stdout:9/575: creat d6/fbc x:0 0 0 2024-05-27T09:29:42.867 INFO:tasks.workunit.client.1.smithi138.stdout:5/493: symlink d3/d12/d16/d1d/d22/d2b/d99/l9d 0 2024-05-27T09:29:42.867 INFO:tasks.workunit.client.1.smithi138.stdout:5/494: readlink d3/l51 0 2024-05-27T09:29:42.868 INFO:tasks.workunit.client.1.smithi138.stdout:5/495: chown d3/d12/d26/f2f 53661 1 2024-05-27T09:29:42.875 INFO:tasks.workunit.client.1.smithi138.stdout:3/542: dread d0/d2/da/f8a [0,4194304] 0 2024-05-27T09:29:42.875 INFO:tasks.workunit.client.1.smithi138.stdout:3/543: truncate d0/d2/da/df/d10/d2a/d48/f79 591074 0 2024-05-27T09:29:42.876 INFO:tasks.workunit.client.1.smithi138.stdout:3/544: rmdir d0/d2/da/df/d10/d2a/d48 39 2024-05-27T09:29:42.876 INFO:tasks.workunit.client.1.smithi138.stdout:3/545: dread - d0/d2/da/df/fa1 zero size 2024-05-27T09:29:42.880 INFO:tasks.workunit.client.1.smithi138.stdout:7/550: dwrite d0/d2b/d80/fae [0,4194304] 0 2024-05-27T09:29:42.889 INFO:tasks.workunit.client.1.smithi138.stdout:7/551: creat d0/d23/d49/fbb x:0 0 0 2024-05-27T09:29:42.889 INFO:tasks.workunit.client.1.smithi138.stdout:8/607: write d3/f4a [3405640,64157] 0 2024-05-27T09:29:42.889 INFO:tasks.workunit.client.1.smithi138.stdout:9/576: rename d6/db/de/d18/d27 to d6/da/d29/d8f/db7/dbd 0 2024-05-27T09:29:42.889 INFO:tasks.workunit.client.1.smithi138.stdout:9/577: fsync d6/da/d29/d59/f6e 0 2024-05-27T09:29:42.889 INFO:tasks.workunit.client.1.smithi138.stdout:9/578: fdatasync d6/da/d2f/d35/d55/f3a 0 2024-05-27T09:29:42.889 INFO:tasks.workunit.client.1.smithi138.stdout:9/579: chown d6/da/d26/d28/f49 80763 1 2024-05-27T09:29:42.890 INFO:tasks.workunit.client.1.smithi138.stdout:8/608: mknod d3/d8/d32/d3f/d9c/d9f/d86/ccb 0 2024-05-27T09:29:42.892 INFO:tasks.workunit.client.1.smithi138.stdout:9/580: creat d6/db/de/d18/fbe x:0 0 0 2024-05-27T09:29:42.897 INFO:tasks.workunit.client.1.smithi138.stdout:9/581: symlink d6/db/d8d/lbf 0 2024-05-27T09:29:42.923 INFO:tasks.workunit.client.1.smithi138.stdout:6/713: sync 2024-05-27T09:29:42.928 INFO:tasks.workunit.client.1.smithi138.stdout:7/552: write d0/d2b/daf/d43/f7a [534114,96771] 0 2024-05-27T09:29:42.928 INFO:tasks.workunit.client.1.smithi138.stdout:3/546: dwrite d0/fa4 [0,4194304] 0 2024-05-27T09:29:42.928 INFO:tasks.workunit.client.1.smithi138.stdout:3/547: chown d0/d2/da/df/d10/f4a 1075 1 2024-05-27T09:29:42.947 INFO:tasks.workunit.client.1.smithi138.stdout:6/714: dread d3/d11/d2d/f75 [0,4194304] 0 2024-05-27T09:29:42.947 INFO:tasks.workunit.client.1.smithi138.stdout:6/715: fsync d3/d11/dc0/d43/d71/fea 0 2024-05-27T09:29:42.948 INFO:tasks.workunit.client.1.smithi138.stdout:2/491: sync 2024-05-27T09:29:42.949 INFO:tasks.workunit.client.1.smithi138.stdout:8/609: dwrite d3/d8/d32/d3f/d9c/f96 [0,4194304] 0 2024-05-27T09:29:42.949 INFO:tasks.workunit.client.1.smithi138.stdout:8/610: chown d3/d8/d32/d45/dc2 3737112 1 2024-05-27T09:29:42.954 INFO:tasks.workunit.client.1.smithi138.stdout:7/553: dread d0/f24 [0,4194304] 0 2024-05-27T09:29:42.954 INFO:tasks.workunit.client.1.smithi138.stdout:7/554: write d0/d23/f58 [4563152,29483] 0 2024-05-27T09:29:42.954 INFO:tasks.workunit.client.1.smithi138.stdout:6/716: unlink d3/d11/d2d/d37/l40 0 2024-05-27T09:29:42.955 INFO:tasks.workunit.client.1.smithi138.stdout:6/717: write d3/d11/d2d/f39 [455650,19654] 0 2024-05-27T09:29:42.955 INFO:tasks.workunit.client.1.smithi138.stdout:8/611: mknod d3/d8/d32/d45/dc2/ccc 0 2024-05-27T09:29:42.955 INFO:tasks.workunit.client.1.smithi138.stdout:9/582: dwrite d6/da/d29/d8f/db7/dbd/d4c/fae [4194304,4194304] 0 2024-05-27T09:29:42.969 INFO:tasks.workunit.client.1.smithi138.stdout:8/612: dread d3/d8/d32/d34/d38/fc7 [0,4194304] 0 2024-05-27T09:29:42.979 INFO:tasks.workunit.client.1.smithi138.stdout:2/492: dread d1/d2f/f33 [0,4194304] 0 2024-05-27T09:29:42.979 INFO:tasks.workunit.client.1.smithi138.stdout:8/613: rename d3/d8/c23 to d3/d8/d24/d55/d94/da5/ccd 0 2024-05-27T09:29:42.981 INFO:tasks.workunit.client.1.smithi138.stdout:8/614: rmdir d3/d8/d24/d55/d94 39 2024-05-27T09:29:42.982 INFO:tasks.workunit.client.1.smithi138.stdout:6/718: write d3/d19/fab [2579089,42224] 0 2024-05-27T09:29:42.982 INFO:tasks.workunit.client.1.smithi138.stdout:6/719: stat d3/d11/dc0/d5a/f68 0 2024-05-27T09:29:42.982 INFO:tasks.workunit.client.1.smithi138.stdout:6/720: readlink d3/d11/dc0/d43/d8c/lc6 0 2024-05-27T09:29:42.985 INFO:tasks.workunit.client.1.smithi138.stdout:8/615: link d3/d8/d11/fa6 d3/d8/d24/d4d/fce 0 2024-05-27T09:29:42.985 INFO:tasks.workunit.client.1.smithi138.stdout:6/721: stat d3/d11/cc 0 2024-05-27T09:29:42.990 INFO:tasks.workunit.client.1.smithi138.stdout:8/616: creat d3/d8/d32/d3f/d9c/db8/fcf x:0 0 0 2024-05-27T09:29:42.991 INFO:tasks.workunit.client.1.smithi138.stdout:8/617: read - d3/d8/d11/f99 zero size 2024-05-27T09:29:42.991 INFO:tasks.workunit.client.1.smithi138.stdout:8/618: write d3/d8/d24/d4d/f97 [779646,66532] 0 2024-05-27T09:29:42.991 INFO:tasks.workunit.client.1.smithi138.stdout:8/619: stat d3/d8/d32/d3f/f6b 0 2024-05-27T09:29:42.991 INFO:tasks.workunit.client.1.smithi138.stdout:8/620: fdatasync d3/d8/d32/d3f/d9c/f96 0 2024-05-27T09:29:43.000 INFO:tasks.workunit.client.1.smithi138.stdout:8/621: truncate d3/d8/d32/d88/f8e 54480 0 2024-05-27T09:29:43.006 INFO:tasks.workunit.client.1.smithi138.stdout:8/622: rename d3/d8/d24/d55/d66 to d3/d8/d32/d34/d38/dd0 0 2024-05-27T09:29:43.007 INFO:tasks.workunit.client.1.smithi138.stdout:6/722: dread d3/d11/d2d/d37/dd1/f6c [0,4194304] 0 2024-05-27T09:29:43.007 INFO:tasks.workunit.client.1.smithi138.stdout:9/583: dwrite d6/db/de/d18/f7b [0,4194304] 0 2024-05-27T09:29:43.009 INFO:tasks.workunit.client.1.smithi138.stdout:6/723: creat d3/d11/d2d/ff4 x:0 0 0 2024-05-27T09:29:43.014 INFO:tasks.workunit.client.1.smithi138.stdout:3/548: dwrite d0/d2/f4b [0,4194304] 0 2024-05-27T09:29:43.016 INFO:tasks.workunit.client.1.smithi138.stdout:1/555: sync 2024-05-27T09:29:43.016 INFO:tasks.workunit.client.1.smithi138.stdout:1/556: creat d4/d1a/d2e/fb4 x:0 0 0 2024-05-27T09:29:43.016 INFO:tasks.workunit.client.1.smithi138.stdout:1/557: dread - d4/d1a/d2e/d4a/fa6 zero size 2024-05-27T09:29:43.017 INFO:tasks.workunit.client.1.smithi138.stdout:0/543: sync 2024-05-27T09:29:43.022 INFO:tasks.workunit.client.1.smithi138.stdout:0/544: rename da/d17/f31 to da/d2a/d8e/d1f/d42/d63/fb5 0 2024-05-27T09:29:43.025 INFO:tasks.workunit.client.1.smithi138.stdout:9/584: symlink d6/da/d2f/d35/d55/lc0 0 2024-05-27T09:29:43.028 INFO:tasks.workunit.client.1.smithi138.stdout:3/549: mknod d0/d2/da/df/d10/d2a/d61/cb4 0 2024-05-27T09:29:43.029 INFO:tasks.workunit.client.1.smithi138.stdout:9/585: mknod d6/da/d29/d59/cc1 0 2024-05-27T09:29:43.029 INFO:tasks.workunit.client.1.smithi138.stdout:3/550: write d0/d3f/f84 [210739,103448] 0 2024-05-27T09:29:43.029 INFO:tasks.workunit.client.1.smithi138.stdout:3/551: stat d0/d2/da/l16 0 2024-05-27T09:29:43.030 INFO:tasks.workunit.client.1.smithi138.stdout:3/552: chown d0/d2/da/l67 4 1 2024-05-27T09:29:43.033 INFO:tasks.workunit.client.1.smithi138.stdout:2/493: dwrite d1/d10/f76 [0,4194304] 0 2024-05-27T09:29:43.034 INFO:tasks.workunit.client.1.smithi138.stdout:3/553: creat d0/d2/da/df/d10/d2a/d61/fb5 x:0 0 0 2024-05-27T09:29:43.038 INFO:tasks.workunit.client.1.smithi138.stdout:2/494: unlink d1/d15/d2d/d2e/c85 0 2024-05-27T09:29:43.040 INFO:tasks.workunit.client.1.smithi138.stdout:7/555: dwrite d0/d2b/daf/d43/f67 [4194304,4194304] 0 2024-05-27T09:29:43.041 INFO:tasks.workunit.client.1.smithi138.stdout:2/495: link d1/d15/d2d/d17/d25/d9d/d4b/c79 d1/d10/d52/d5d/caa 0 2024-05-27T09:29:43.042 INFO:tasks.workunit.client.1.smithi138.stdout:6/724: write d3/d11/dc0/f97 [653222,72227] 0 2024-05-27T09:29:43.044 INFO:tasks.workunit.client.1.smithi138.stdout:1/558: dread d4/d3c/fa0 [0,4194304] 0 2024-05-27T09:29:43.044 INFO:tasks.workunit.client.1.smithi138.stdout:1/559: write d4/f9b [1991372,71078] 0 2024-05-27T09:29:43.045 INFO:tasks.workunit.client.1.smithi138.stdout:7/556: symlink d0/d2b/daf/d57/d91/lbc 0 2024-05-27T09:29:43.049 INFO:tasks.workunit.client.1.smithi138.stdout:6/725: symlink d3/d11/dc0/d43/d71/lf5 0 2024-05-27T09:29:43.059 INFO:tasks.workunit.client.1.smithi138.stdout:7/557: creat d0/d2b/daf/d57/d91/fbd x:0 0 0 2024-05-27T09:29:43.060 INFO:tasks.workunit.client.1.smithi138.stdout:7/558: chown d0/d23/d3a/l8b 35 1 2024-05-27T09:29:43.074 INFO:tasks.workunit.client.1.smithi138.stdout:1/560: dread d4/d48/d22/f5a [0,4194304] 0 2024-05-27T09:29:43.086 INFO:tasks.workunit.client.1.smithi138.stdout:3/554: dread d0/d2/da/df/d10/d32/f9 [4194304,4194304] 0 2024-05-27T09:29:43.088 INFO:tasks.workunit.client.1.smithi138.stdout:1/561: creat d4/d88/d9f/fb5 x:0 0 0 2024-05-27T09:29:43.095 INFO:tasks.workunit.client.1.smithi138.stdout:7/559: dread d0/d2b/daf/f6b [0,4194304] 0 2024-05-27T09:29:43.098 INFO:tasks.workunit.client.1.smithi138.stdout:7/560: getdents d0/d23/d49/d78 0 2024-05-27T09:29:43.101 INFO:tasks.workunit.client.1.smithi138.stdout:7/561: read - d0/d23/d3a/fa6 zero size 2024-05-27T09:29:43.101 INFO:tasks.workunit.client.1.smithi138.stdout:3/555: dread d0/d3f/d60/f96 [0,4194304] 0 2024-05-27T09:29:43.102 INFO:tasks.workunit.client.1.smithi138.stdout:3/556: mknod d0/cb6 0 2024-05-27T09:29:43.105 INFO:tasks.workunit.client.1.smithi138.stdout:8/623: dwrite d3/d8/d32/d45/f90 [0,4194304] 0 2024-05-27T09:29:43.105 INFO:tasks.workunit.client.1.smithi138.stdout:8/624: write d3/d8/d32/f59 [974645,101288] 0 2024-05-27T09:29:43.112 INFO:tasks.workunit.client.1.smithi138.stdout:0/545: dwrite da/d2a/d8e/d1f/f26 [0,4194304] 0 2024-05-27T09:29:43.114 INFO:tasks.workunit.client.1.smithi138.stdout:0/546: stat da/d2a/d8e/d1f/c9c 0 2024-05-27T09:29:43.114 INFO:tasks.workunit.client.1.smithi138.stdout:0/547: chown da/d2a/d8e/d1f/fa2 577 1 2024-05-27T09:29:43.117 INFO:tasks.workunit.client.1.smithi138.stdout:2/496: dwrite d1/d10/d52/d5d/d7e/f84 [0,4194304] 0 2024-05-27T09:29:43.125 INFO:tasks.workunit.client.1.smithi138.stdout:2/497: write d1/d15/d2d/d17/f6b [12913,57685] 0 2024-05-27T09:29:43.127 INFO:tasks.workunit.client.1.smithi138.stdout:1/562: dread d4/d1a/d2e/d32/f49 [0,4194304] 0 2024-05-27T09:29:43.130 INFO:tasks.workunit.client.1.smithi138.stdout:1/563: mknod d4/d48/d22/cb6 0 2024-05-27T09:29:43.131 INFO:tasks.workunit.client.1.smithi138.stdout:1/564: write d4/d88/d9f/fb5 [877005,125889] 0 2024-05-27T09:29:43.131 INFO:tasks.workunit.client.1.smithi138.stdout:1/565: truncate d4/d48/d39/d5f/d93/fb2 663606 0 2024-05-27T09:29:43.134 INFO:tasks.workunit.client.1.smithi138.stdout:1/566: mkdir d4/d1a/d2e/d4a/d7e/db7 0 2024-05-27T09:29:43.135 INFO:tasks.workunit.client.1.smithi138.stdout:3/557: dread d0/fa4 [0,4194304] 0 2024-05-27T09:29:43.138 INFO:tasks.workunit.client.1.smithi138.stdout:0/548: dread da/d2a/d8e/d1f/d42/d4a/d53/f57 [0,4194304] 0 2024-05-27T09:29:43.139 INFO:tasks.workunit.client.1.smithi138.stdout:0/549: write da/f22 [3948399,106143] 0 2024-05-27T09:29:43.139 INFO:tasks.workunit.client.1.smithi138.stdout:6/726: dwrite d3/d11/dc0/d43/d1c/fa9 [0,4194304] 0 2024-05-27T09:29:43.141 INFO:tasks.workunit.client.1.smithi138.stdout:1/567: symlink d4/d1a/d2e/lb8 0 2024-05-27T09:29:43.143 INFO:tasks.workunit.client.1.smithi138.stdout:6/727: mknod d3/d11/dc0/d43/cf6 0 2024-05-27T09:29:43.150 INFO:tasks.workunit.client.1.smithi138.stdout:1/568: symlink d4/d48/d39/d5f/lb9 0 2024-05-27T09:29:43.153 INFO:tasks.workunit.client.1.smithi138.stdout:1/569: rmdir d4/d1a/d2e/d32 39 2024-05-27T09:29:43.154 INFO:tasks.workunit.client.1.smithi138.stdout:2/498: dread d1/d10/f76 [0,4194304] 0 2024-05-27T09:29:43.158 INFO:tasks.workunit.client.1.smithi138.stdout:7/562: dwrite d0/d2b/daf/d64/f7d [0,4194304] 0 2024-05-27T09:29:43.158 INFO:tasks.workunit.client.1.smithi138.stdout:7/563: stat d0/l8a 0 2024-05-27T09:29:43.159 INFO:tasks.workunit.client.1.smithi138.stdout:2/499: rename d1/d15/d2d/d17/d25/d4d/d6d/fa3 to d1/d15/d2d/d17/d25/d4d/d6d/fab 0 2024-05-27T09:29:43.160 INFO:tasks.workunit.client.1.smithi138.stdout:9/586: dwrite d6/da/d26/d28/f49 [0,4194304] 0 2024-05-27T09:29:43.163 INFO:tasks.workunit.client.1.smithi138.stdout:2/500: mkdir d1/d15/d2d/d17/dac 0 2024-05-27T09:29:43.163 INFO:tasks.workunit.client.1.smithi138.stdout:0/550: dread da/d2a/d8e/d1f/d4f/f6f [0,4194304] 0 2024-05-27T09:29:43.163 INFO:tasks.workunit.client.1.smithi138.stdout:0/551: stat da/d17/f8c 0 2024-05-27T09:29:43.169 INFO:tasks.workunit.client.1.smithi138.stdout:1/570: symlink d4/d48/d22/d65/lba 0 2024-05-27T09:29:43.171 INFO:tasks.workunit.client.1.smithi138.stdout:9/587: rename d6/da/d2f/l42 to d6/da/d26/d28/lc2 0 2024-05-27T09:29:43.174 INFO:tasks.workunit.client.1.smithi138.stdout:9/588: symlink d6/da/d2f/d35/d55/d75/d8c/lc3 0 2024-05-27T09:29:43.180 INFO:tasks.workunit.client.1.smithi138.stdout:9/589: unlink d6/da/d2f/l67 0 2024-05-27T09:29:43.191 INFO:tasks.workunit.client.1.smithi138.stdout:4/591: sync 2024-05-27T09:29:43.191 INFO:tasks.workunit.client.1.smithi138.stdout:1/571: truncate d4/d1a/d2e/d4a/f8e 903992 0 2024-05-27T09:29:43.195 INFO:tasks.workunit.client.1.smithi138.stdout:2/501: dread d1/d10/d52/fa6 [4194304,4194304] 0 2024-05-27T09:29:43.196 INFO:tasks.workunit.client.1.smithi138.stdout:4/592: unlink d7/d6e/f81 0 2024-05-27T09:29:43.196 INFO:tasks.workunit.client.1.smithi138.stdout:4/593: write d7/dba/dcf/d3a/f6b [1365060,121478] 0 2024-05-27T09:29:43.198 INFO:tasks.workunit.client.1.smithi138.stdout:2/502: fdatasync d1/d15/d2d/d17/d25/d4d/d4e/f81 0 2024-05-27T09:29:43.204 INFO:tasks.workunit.client.1.smithi138.stdout:1/572: rename d4/f37 to d4/d3c/d51/fbb 0 2024-05-27T09:29:43.206 INFO:tasks.workunit.client.1.smithi138.stdout:1/573: write d4/d3c/d51/f96 [972878,50214] 0 2024-05-27T09:29:43.212 INFO:tasks.workunit.client.1.smithi138.stdout:8/625: dwrite d3/d8/d24/f54 [0,4194304] 0 2024-05-27T09:29:43.215 INFO:tasks.workunit.client.1.smithi138.stdout:4/594: unlink d7/df/d35/f5a 0 2024-05-27T09:29:43.219 INFO:tasks.workunit.client.1.smithi138.stdout:7/564: dwrite d0/d2b/fab [0,4194304] 0 2024-05-27T09:29:43.225 INFO:tasks.workunit.client.1.smithi138.stdout:1/574: mknod d4/d48/d39/cbc 0 2024-05-27T09:29:43.226 INFO:tasks.workunit.client.1.smithi138.stdout:9/590: read d6/db/f10 [646817,77117] 0 2024-05-27T09:29:43.227 INFO:tasks.workunit.client.1.smithi138.stdout:8/626: dread d3/d8/f58 [0,4194304] 0 2024-05-27T09:29:43.231 INFO:tasks.workunit.client.1.smithi138.stdout:7/565: rename d0/d23/da8/c70 to d0/d23/d3c/d3f/cbe 0 2024-05-27T09:29:43.232 INFO:tasks.workunit.client.1.smithi138.stdout:7/566: dread - d0/d23/d3a/fa6 zero size 2024-05-27T09:29:43.237 INFO:tasks.workunit.client.1.smithi138.stdout:4/595: symlink d7/df/d5b/da1/dbc/ld2 0 2024-05-27T09:29:43.237 INFO:tasks.workunit.client.1.smithi138.stdout:8/627: mknod d3/d8/d32/d34/dbd/cd1 0 2024-05-27T09:29:43.244 INFO:tasks.workunit.client.1.smithi138.stdout:1/575: unlink d4/d48/d22/cb6 0 2024-05-27T09:29:43.244 INFO:tasks.workunit.client.1.smithi138.stdout:1/576: fdatasync d4/d48/d39/d5f/f6a 0 2024-05-27T09:29:43.246 INFO:tasks.workunit.client.1.smithi138.stdout:3/558: dwrite d0/d2/da/f2d [0,4194304] 0 2024-05-27T09:29:43.251 INFO:tasks.workunit.client.1.smithi138.stdout:3/559: write d0/d2/da/df/d10/f4a [2389844,92498] 0 2024-05-27T09:29:43.251 INFO:tasks.workunit.client.1.smithi138.stdout:7/567: getdents d0/d2b/daf/d43 0 2024-05-27T09:29:43.251 INFO:tasks.workunit.client.1.smithi138.stdout:5/496: sync 2024-05-27T09:29:43.251 INFO:tasks.workunit.client.1.smithi138.stdout:1/577: symlink d4/d88/lbd 0 2024-05-27T09:29:43.251 INFO:tasks.workunit.client.1.smithi138.stdout:1/578: write d4/d48/d39/f54 [5133383,2083] 0 2024-05-27T09:29:43.251 INFO:tasks.workunit.client.1.smithi138.stdout:1/579: truncate d4/d1a/d2e/d4a/fa6 165271 0 2024-05-27T09:29:43.261 INFO:tasks.workunit.client.1.smithi138.stdout:4/596: write d7/dba/dcf/d3a/f6b [7240314,27351] 0 2024-05-27T09:29:43.261 INFO:tasks.workunit.client.1.smithi138.stdout:4/597: truncate d7/df/f41 1127925 0 2024-05-27T09:29:43.262 INFO:tasks.workunit.client.1.smithi138.stdout:2/503: dwrite d1/f1b [0,4194304] 0 2024-05-27T09:29:43.262 INFO:tasks.workunit.client.1.smithi138.stdout:2/504: write d1/d15/d2d/d1d/f99 [715108,123568] 0 2024-05-27T09:29:43.265 INFO:tasks.workunit.client.1.smithi138.stdout:2/505: symlink d1/d10/d52/lad 0 2024-05-27T09:29:43.265 INFO:tasks.workunit.client.1.smithi138.stdout:2/506: write d1/d10/d52/f73 [127079,35012] 0 2024-05-27T09:29:43.265 INFO:tasks.workunit.client.1.smithi138.stdout:2/507: fsync d1/d15/d2d/d17/d25/d4d/d4e/f9e 0 2024-05-27T09:29:43.270 INFO:tasks.workunit.client.1.smithi138.stdout:2/508: creat d1/d15/d2d/d17/d25/d4d/d6d/d87/fae x:0 0 0 2024-05-27T09:29:43.272 INFO:tasks.workunit.client.1.smithi138.stdout:2/509: creat d1/d15/d2d/d2e/faf x:0 0 0 2024-05-27T09:29:43.276 INFO:tasks.workunit.client.1.smithi138.stdout:2/510: rename d1/d10/d52/d5d/caa to d1/d15/d2d/d17/d25/d4d/cb0 0 2024-05-27T09:29:43.276 INFO:tasks.workunit.client.1.smithi138.stdout:7/568: write d0/d2b/fab [9522,4160] 0 2024-05-27T09:29:43.276 INFO:tasks.workunit.client.1.smithi138.stdout:2/511: chown d1/d2f/c4a 494 1 2024-05-27T09:29:43.276 INFO:tasks.workunit.client.1.smithi138.stdout:7/569: fsync d0/d23/d3c/f93 0 2024-05-27T09:29:43.276 INFO:tasks.workunit.client.1.smithi138.stdout:7/570: fdatasync d0/d2b/f5f 0 2024-05-27T09:29:43.276 INFO:tasks.workunit.client.1.smithi138.stdout:7/571: chown d0/d23/da8 24374991 1 2024-05-27T09:29:43.277 INFO:tasks.workunit.client.1.smithi138.stdout:7/572: mkdir d0/d2b/daf/dbf 0 2024-05-27T09:29:43.279 INFO:tasks.workunit.client.1.smithi138.stdout:5/497: rmdir d3/d12 39 2024-05-27T09:29:43.279 INFO:tasks.workunit.client.1.smithi138.stdout:8/628: dwrite d3/d8/d32/d34/d38/f46 [0,4194304] 0 2024-05-27T09:29:43.280 INFO:tasks.workunit.client.1.smithi138.stdout:5/498: write d3/d12/d32/d60/f7b [926972,59052] 0 2024-05-27T09:29:43.284 INFO:tasks.workunit.client.1.smithi138.stdout:4/598: rename d7/df/c8c to d7/df/cd3 0 2024-05-27T09:29:43.286 INFO:tasks.workunit.client.1.smithi138.stdout:5/499: mkdir d3/d12/d16/d1d/d22/d2b/d9e 0 2024-05-27T09:29:43.286 INFO:tasks.workunit.client.1.smithi138.stdout:5/500: chown d3/d12/d16/f39 43704 1 2024-05-27T09:29:43.289 INFO:tasks.workunit.client.1.smithi138.stdout:0/552: dwrite da/d2a/d8e/d1f/d42/d4a/d53/f6e [0,4194304] 0 2024-05-27T09:29:43.289 INFO:tasks.workunit.client.1.smithi138.stdout:0/553: creat da/d2a/d8e/d1f/d42/d63/d9d/fb6 x:0 0 0 2024-05-27T09:29:43.294 INFO:tasks.workunit.client.1.smithi138.stdout:4/599: creat d7/df/d35/d4c/d59/d5d/fd4 x:0 0 0 2024-05-27T09:29:43.294 INFO:tasks.workunit.client.1.smithi138.stdout:4/600: dread - d7/df/d5b/f7f zero size 2024-05-27T09:29:43.294 INFO:tasks.workunit.client.1.smithi138.stdout:4/601: chown d7/dba/dcf/f2c 256593 1 2024-05-27T09:29:43.294 INFO:tasks.workunit.client.1.smithi138.stdout:5/501: creat d3/d12/d32/d7e/f9f x:0 0 0 2024-05-27T09:29:43.324 INFO:tasks.workunit.client.1.smithi138.stdout:8/629: write d3/d8/d32/d3f/d9c/f96 [2043310,86334] 0 2024-05-27T09:29:43.325 INFO:tasks.workunit.client.1.smithi138.stdout:9/591: dwrite d6/db/dc/f50 [0,4194304] 0 2024-05-27T09:29:43.327 INFO:tasks.workunit.client.1.smithi138.stdout:8/630: rename d3/d8/d32/d34/d38/dd0/l82 to d3/d8/d24/d55/d7e/ld2 0 2024-05-27T09:29:43.328 INFO:tasks.workunit.client.1.smithi138.stdout:9/592: creat d6/da/d29/d8f/db7/fc4 x:0 0 0 2024-05-27T09:29:43.328 INFO:tasks.workunit.client.1.smithi138.stdout:9/593: readlink d6/da/d29/d8f/da8/lab 0 2024-05-27T09:29:43.331 INFO:tasks.workunit.client.1.smithi138.stdout:9/594: link d6/f48 d6/da/d29/d8f/db7/dbd/fc5 0 2024-05-27T09:29:43.334 INFO:tasks.workunit.client.1.smithi138.stdout:9/595: creat d6/da/d2f/d35/d55/fc6 x:0 0 0 2024-05-27T09:29:43.334 INFO:tasks.workunit.client.1.smithi138.stdout:9/596: read d6/f2e [3410330,114367] 0 2024-05-27T09:29:43.334 INFO:tasks.workunit.client.1.smithi138.stdout:1/580: dwrite d4/d48/d39/d5f/f5b [0,4194304] 0 2024-05-27T09:29:43.336 INFO:tasks.workunit.client.1.smithi138.stdout:9/597: mkdir d6/da/dc7 0 2024-05-27T09:29:43.339 INFO:tasks.workunit.client.1.smithi138.stdout:9/598: mkdir d6/dc8 0 2024-05-27T09:29:43.339 INFO:tasks.workunit.client.1.smithi138.stdout:9/599: chown d6/da/d2f/d35/l84 229598 1 2024-05-27T09:29:43.339 INFO:tasks.workunit.client.1.smithi138.stdout:9/600: creat d6/db/de/d85/fc9 x:0 0 0 2024-05-27T09:29:43.343 INFO:tasks.workunit.client.1.smithi138.stdout:1/581: symlink d4/d38/lbe 0 2024-05-27T09:29:43.356 INFO:tasks.workunit.client.1.smithi138.stdout:5/502: dread d3/d12/d26/d53/f5a [0,4194304] 0 2024-05-27T09:29:43.356 INFO:tasks.workunit.client.1.smithi138.stdout:7/573: dwrite d0/f21 [0,4194304] 0 2024-05-27T09:29:43.357 INFO:tasks.workunit.client.1.smithi138.stdout:8/631: dread d3/d8/d32/f81 [0,4194304] 0 2024-05-27T09:29:43.357 INFO:tasks.workunit.client.1.smithi138.stdout:8/632: dread - d3/d8/d32/d34/d38/f79 zero size 2024-05-27T09:29:43.357 INFO:tasks.workunit.client.1.smithi138.stdout:8/633: fdatasync d3/d8/d24/d4d/fce 0 2024-05-27T09:29:43.358 INFO:tasks.workunit.client.1.smithi138.stdout:1/582: rmdir d4/d48/d39/d5f/d93 39 2024-05-27T09:29:43.358 INFO:tasks.workunit.client.1.smithi138.stdout:1/583: stat d4/d1a/d2e/d4a/d7e/lb1 0 2024-05-27T09:29:43.359 INFO:tasks.workunit.client.1.smithi138.stdout:1/584: write d4/d48/d22/f5a [4742891,60995] 0 2024-05-27T09:29:43.360 INFO:tasks.workunit.client.1.smithi138.stdout:7/574: rename d0/d23/d49/d78/c89 to d0/d23/d3a/cc0 0 2024-05-27T09:29:43.360 INFO:tasks.workunit.client.1.smithi138.stdout:7/575: chown d0/d23/f58 23 1 2024-05-27T09:29:43.360 INFO:tasks.workunit.client.1.smithi138.stdout:7/576: creat d0/d23/d49/fc1 x:0 0 0 2024-05-27T09:29:43.360 INFO:tasks.workunit.client.1.smithi138.stdout:7/577: fdatasync d0/d2b/daf/d60/f9f 0 2024-05-27T09:29:43.360 INFO:tasks.workunit.client.1.smithi138.stdout:7/578: truncate d0/d2b/daf/d57/d91/fbd 2979 0 2024-05-27T09:29:43.360 INFO:tasks.workunit.client.1.smithi138.stdout:0/554: dwrite da/d2a/d55/fb4 [0,4194304] 0 2024-05-27T09:29:43.365 INFO:tasks.workunit.client.1.smithi138.stdout:1/585: getdents d4/d48 0 2024-05-27T09:29:43.366 INFO:tasks.workunit.client.1.smithi138.stdout:1/586: write d4/d1a/d2e/fb0 [917472,54315] 0 2024-05-27T09:29:43.366 INFO:tasks.workunit.client.1.smithi138.stdout:1/587: dread - d4/d48/d39/d5f/d63/f98 zero size 2024-05-27T09:29:43.368 INFO:tasks.workunit.client.1.smithi138.stdout:5/503: link d3/d12/d32/c6c d3/d12/d32/ca0 0 2024-05-27T09:29:43.369 INFO:tasks.workunit.client.1.smithi138.stdout:1/588: symlink d4/d48/d39/d5f/d4d/d2b/lbf 0 2024-05-27T09:29:43.369 INFO:tasks.workunit.client.1.smithi138.stdout:1/589: readlink d4/d88/lbd 0 2024-05-27T09:29:43.369 INFO:tasks.workunit.client.1.smithi138.stdout:1/590: readlink d4/d48/d39/d5f/l66 0 2024-05-27T09:29:43.369 INFO:tasks.workunit.client.1.smithi138.stdout:1/591: dread - d4/d48/d39/d5f/f6a zero size 2024-05-27T09:29:43.372 INFO:tasks.workunit.client.1.smithi138.stdout:8/634: rename d3/db4 to d3/d8/d6c/dd3 0 2024-05-27T09:29:43.373 INFO:tasks.workunit.client.1.smithi138.stdout:8/635: fsync f1 0 2024-05-27T09:29:43.373 INFO:tasks.workunit.client.1.smithi138.stdout:5/504: creat d3/d12/d16/d1d/d23/d68/fa1 x:0 0 0 2024-05-27T09:29:43.373 INFO:tasks.workunit.client.1.smithi138.stdout:5/505: stat d3/d12/d16/d1d/d30/c52 0 2024-05-27T09:29:43.373 INFO:tasks.workunit.client.1.smithi138.stdout:7/579: readlink d0/d23/l2a 0 2024-05-27T09:29:43.373 INFO:tasks.workunit.client.1.smithi138.stdout:7/580: write d0/d23/f48 [1816394,17180] 0 2024-05-27T09:29:43.373 INFO:tasks.workunit.client.1.smithi138.stdout:7/581: write d0/f32 [198851,89427] 0 2024-05-27T09:29:43.373 INFO:tasks.workunit.client.1.smithi138.stdout:7/582: write d0/f8c [700167,40774] 0 2024-05-27T09:29:43.373 INFO:tasks.workunit.client.1.smithi138.stdout:7/583: read - d0/d23/d3c/f93 zero size 2024-05-27T09:29:43.377 INFO:tasks.workunit.client.1.smithi138.stdout:8/636: unlink d3/d8/d32/d34/d38/fad 0 2024-05-27T09:29:43.377 INFO:tasks.workunit.client.1.smithi138.stdout:8/637: stat d3/d8/d32/d3f/d9c/d9f 0 2024-05-27T09:29:43.389 INFO:tasks.workunit.client.1.smithi138.stdout:7/584: fsync d0/d2b/daf/d43/f67 0 2024-05-27T09:29:43.390 INFO:tasks.workunit.client.1.smithi138.stdout:9/601: dwrite d6/db/f6a [0,4194304] 0 2024-05-27T09:29:43.391 INFO:tasks.workunit.client.1.smithi138.stdout:7/585: link d0/d2b/c44 d0/d2b/daf/dbf/cc2 0 2024-05-27T09:29:43.393 INFO:tasks.workunit.client.1.smithi138.stdout:9/602: truncate d6/f9 474097 0 2024-05-27T09:29:43.396 INFO:tasks.workunit.client.1.smithi138.stdout:7/586: rename d0/d2b/d80 to d0/d23/d3c/d84/d9c/dc3 0 2024-05-27T09:29:43.398 INFO:tasks.workunit.client.1.smithi138.stdout:7/587: mkdir d0/d23/dc4 0 2024-05-27T09:29:43.402 INFO:tasks.workunit.client.1.smithi138.stdout:4/602: dwrite d7/dba/dcf/d3a/f6b [8388608,4194304] 0 2024-05-27T09:29:43.404 INFO:tasks.workunit.client.1.smithi138.stdout:5/506: dread d3/d12/d32/d60/f8e [0,4194304] 0 2024-05-27T09:29:43.404 INFO:tasks.workunit.client.1.smithi138.stdout:5/507: chown d3 322 1 2024-05-27T09:29:43.406 INFO:tasks.workunit.client.1.smithi138.stdout:1/592: write d4/f9b [553637,116393] 0 2024-05-27T09:29:43.410 INFO:tasks.workunit.client.1.smithi138.stdout:5/508: getdents d3/d12/d16/d1d/d23/d3b 0 2024-05-27T09:29:43.412 INFO:tasks.workunit.client.1.smithi138.stdout:1/593: mkdir d4/d1a/d2e/d4a/d7e/dc0 0 2024-05-27T09:29:43.414 INFO:tasks.workunit.client.1.smithi138.stdout:5/509: link d3/d12/d32/d6b/f78 d3/d12/d16/d1d/d23/d3b/fa2 0 2024-05-27T09:29:43.415 INFO:tasks.workunit.client.1.smithi138.stdout:5/510: write d3/d12/f64 [1694044,78301] 0 2024-05-27T09:29:43.415 INFO:tasks.workunit.client.1.smithi138.stdout:5/511: fsync d3/d12/d16/d1d/d23/f62 0 2024-05-27T09:29:43.415 INFO:tasks.workunit.client.1.smithi138.stdout:5/512: chown d3/d12/d32/d87/c8a 36 1 2024-05-27T09:29:43.417 INFO:tasks.workunit.client.1.smithi138.stdout:1/594: truncate d4/d48/d22/f5a 4528858 0 2024-05-27T09:29:43.422 INFO:tasks.workunit.client.1.smithi138.stdout:5/513: getdents d3/d12/d26 0 2024-05-27T09:29:43.426 INFO:tasks.workunit.client.1.smithi138.stdout:1/595: rmdir d4/d1a/d2e/d4a/d7e 39 2024-05-27T09:29:43.426 INFO:tasks.workunit.client.1.smithi138.stdout:9/603: dread d6/db/de/d18/d24/f7f [0,4194304] 0 2024-05-27T09:29:43.426 INFO:tasks.workunit.client.1.smithi138.stdout:9/604: readlink d6/db/l63 0 2024-05-27T09:29:43.426 INFO:tasks.workunit.client.1.smithi138.stdout:9/605: readlink d6/da/d2f/d35/d55/l34 0 2024-05-27T09:29:43.442 INFO:tasks.workunit.client.1.smithi138.stdout:0/555: dwrite da/d2a/d8e/d1f/f38 [0,4194304] 0 2024-05-27T09:29:43.452 INFO:tasks.workunit.client.1.smithi138.stdout:0/556: getdents da/d2a/d8e/d43/d74 0 2024-05-27T09:29:43.452 INFO:tasks.workunit.client.1.smithi138.stdout:0/557: write da/d2a/d8e/d1f/f3b [1064070,46750] 0 2024-05-27T09:29:43.463 INFO:tasks.workunit.client.1.smithi138.stdout:0/558: creat da/d2a/d2b/fb7 x:0 0 0 2024-05-27T09:29:43.488 INFO:tasks.workunit.client.1.smithi138.stdout:7/588: dwrite d0/d2b/daf/f54 [0,4194304] 0 2024-05-27T09:29:43.488 INFO:tasks.workunit.client.1.smithi138.stdout:5/514: dwrite d3/d12/f9b [0,4194304] 0 2024-05-27T09:29:43.488 INFO:tasks.workunit.client.1.smithi138.stdout:7/589: fsync d0/f1a 0 2024-05-27T09:29:43.489 INFO:tasks.workunit.client.1.smithi138.stdout:8/638: dwrite d3/d8/d32/d3f/d9c/d9f/d86/fc6 [0,4194304] 0 2024-05-27T09:29:43.494 INFO:tasks.workunit.client.1.smithi138.stdout:7/590: rmdir d0/d23/da8 39 2024-05-27T09:29:43.494 INFO:tasks.workunit.client.1.smithi138.stdout:9/606: dwrite d6/da/d26/fb0 [0,4194304] 0 2024-05-27T09:29:43.496 INFO:tasks.workunit.client.1.smithi138.stdout:8/639: creat d3/d8/d6c/fd4 x:0 0 0 2024-05-27T09:29:43.499 INFO:tasks.workunit.client.1.smithi138.stdout:5/515: rename d3/d12/f9b to d3/d12/d32/d60/fa3 0 2024-05-27T09:29:43.499 INFO:tasks.workunit.client.1.smithi138.stdout:5/516: chown d3/d12/d32/d7e/f9f 1 1 2024-05-27T09:29:43.499 INFO:tasks.workunit.client.1.smithi138.stdout:5/517: write d3/d12/d16/d1d/d22/d2b/f38 [991478,16247] 0 2024-05-27T09:29:43.500 INFO:tasks.workunit.client.1.smithi138.stdout:7/591: dread d0/d2b/daf/d43/fa9 [0,4194304] 0 2024-05-27T09:29:43.501 INFO:tasks.workunit.client.1.smithi138.stdout:7/592: write d0/d2b/daf/d57/f9e [500644,19501] 0 2024-05-27T09:29:43.505 INFO:tasks.workunit.client.1.smithi138.stdout:7/593: rmdir d0/d2b/daf/d57/d91 39 2024-05-27T09:29:43.505 INFO:tasks.workunit.client.1.smithi138.stdout:7/594: write d0/f1d [914642,118023] 0 2024-05-27T09:29:43.505 INFO:tasks.workunit.client.1.smithi138.stdout:7/595: write d0/d2b/daf/f39 [2032015,55375] 0 2024-05-27T09:29:43.505 INFO:tasks.workunit.client.1.smithi138.stdout:7/596: chown d0/d2b/daf 660 1 2024-05-27T09:29:43.505 INFO:tasks.workunit.client.1.smithi138.stdout:9/607: dread d6/da/d2f/d35/f80 [0,4194304] 0 2024-05-27T09:29:43.505 INFO:tasks.workunit.client.1.smithi138.stdout:9/608: chown d6/db/de/d6b 15 1 2024-05-27T09:29:43.507 INFO:tasks.workunit.client.1.smithi138.stdout:7/597: unlink d0/d23/d3c/d84/la7 0 2024-05-27T09:29:43.509 INFO:tasks.workunit.client.1.smithi138.stdout:9/609: mkdir d6/db/dc/dca 0 2024-05-27T09:29:43.509 INFO:tasks.workunit.client.1.smithi138.stdout:9/610: write d6/da/d2f/d35/d55/d75/d8c/fb2 [3286973,5622] 0 2024-05-27T09:29:43.510 INFO:tasks.workunit.client.1.smithi138.stdout:1/596: dwrite d4/d48/d39/f45 [0,4194304] 0 2024-05-27T09:29:43.511 INFO:tasks.workunit.client.1.smithi138.stdout:1/597: chown d4/d48/d39/d5f 342635 1 2024-05-27T09:29:43.511 INFO:tasks.workunit.client.1.smithi138.stdout:5/518: link d3/f1b d3/d12/d32/d87/fa4 0 2024-05-27T09:29:43.512 INFO:tasks.workunit.client.1.smithi138.stdout:9/611: mknod d6/db/de/d18/d24/d72/ccb 0 2024-05-27T09:29:43.513 INFO:tasks.workunit.client.1.smithi138.stdout:7/598: symlink d0/d2b/daf/lc5 0 2024-05-27T09:29:43.513 INFO:tasks.workunit.client.1.smithi138.stdout:7/599: write d0/d23/d3c/f42 [1099792,109684] 0 2024-05-27T09:29:43.515 INFO:tasks.workunit.client.1.smithi138.stdout:9/612: creat d6/da/d29/d8f/db7/dbd/d4c/fcc x:0 0 0 2024-05-27T09:29:43.515 INFO:tasks.workunit.client.1.smithi138.stdout:9/613: creat d6/da/d29/d61/fcd x:0 0 0 2024-05-27T09:29:43.516 INFO:tasks.workunit.client.1.smithi138.stdout:1/598: symlink d4/d88/d9f/lc1 0 2024-05-27T09:29:43.517 INFO:tasks.workunit.client.1.smithi138.stdout:7/600: link d0/d2b/daf/l97 d0/d2b/daf/d60/d9d/lc6 0 2024-05-27T09:29:43.520 INFO:tasks.workunit.client.1.smithi138.stdout:9/614: symlink d6/db/lce 0 2024-05-27T09:29:43.520 INFO:tasks.workunit.client.1.smithi138.stdout:1/599: creat d4/d3c/d51/fc2 x:0 0 0 2024-05-27T09:29:43.520 INFO:tasks.workunit.client.1.smithi138.stdout:1/600: write d4/f18 [276697,49657] 0 2024-05-27T09:29:43.523 INFO:tasks.workunit.client.1.smithi138.stdout:0/559: dwrite da/d2a/d2b/f4d [0,4194304] 0 2024-05-27T09:29:43.529 INFO:tasks.workunit.client.1.smithi138.stdout:9/615: link d6/db/de/d18/d24/c5b d6/ccf 0 2024-05-27T09:29:43.529 INFO:tasks.workunit.client.1.smithi138.stdout:9/616: write d6/da/d29/d8f/db7/dbd/f96 [3212265,17401] 0 2024-05-27T09:29:43.529 INFO:tasks.workunit.client.1.smithi138.stdout:9/617: readlink d6/da/d29/d8f/da8/lab 0 2024-05-27T09:29:43.533 INFO:tasks.workunit.client.1.smithi138.stdout:6/728: sync 2024-05-27T09:29:43.534 INFO:tasks.workunit.client.1.smithi138.stdout:0/560: creat da/fb8 x:0 0 0 2024-05-27T09:29:43.535 INFO:tasks.workunit.client.1.smithi138.stdout:0/561: chown da/d2a/d3f/f59 4811398 1 2024-05-27T09:29:43.539 INFO:tasks.workunit.client.1.smithi138.stdout:7/601: read d0/d2b/daf/d60/f9f [1797706,104452] 0 2024-05-27T09:29:43.539 INFO:tasks.workunit.client.1.smithi138.stdout:7/602: write d0/d2b/daf/fe [3975810,65668] 0 2024-05-27T09:29:43.539 INFO:tasks.workunit.client.1.smithi138.stdout:7/603: write d0/d2b/f35 [1639614,84807] 0 2024-05-27T09:29:43.542 INFO:tasks.workunit.client.1.smithi138.stdout:6/729: unlink d3/d19/c26 0 2024-05-27T09:29:43.555 INFO:tasks.workunit.client.1.smithi138.stdout:6/730: mknod d3/d11/dc0/d5a/db4/cf7 0 2024-05-27T09:29:43.569 INFO:tasks.workunit.client.1.smithi138.stdout:7/604: dread d0/d2b/f35 [0,4194304] 0 2024-05-27T09:29:43.571 INFO:tasks.workunit.client.1.smithi138.stdout:6/731: dread d3/d11/dc0/d5a/d6a/f7f [0,4194304] 0 2024-05-27T09:29:43.590 INFO:tasks.workunit.client.1.smithi138.stdout:5/519: dwrite d3/d12/d16/d1d/d23/d68/fa1 [0,4194304] 0 2024-05-27T09:29:43.594 INFO:tasks.workunit.client.1.smithi138.stdout:5/520: unlink d3/d12/d16/f39 0 2024-05-27T09:29:43.594 INFO:tasks.workunit.client.1.smithi138.stdout:5/521: getdents d3/d12/d32/d60/d93 0 2024-05-27T09:29:43.596 INFO:tasks.workunit.client.1.smithi138.stdout:5/522: mknod d3/ca5 0 2024-05-27T09:29:43.599 INFO:tasks.workunit.client.1.smithi138.stdout:5/523: creat d3/d12/d26/fa6 x:0 0 0 2024-05-27T09:29:43.600 INFO:tasks.workunit.client.1.smithi138.stdout:5/524: truncate d3/d12/d16/d1d/d23/d3b/f7c 1057986 0 2024-05-27T09:29:43.634 INFO:tasks.workunit.client.1.smithi138.stdout:8/640: dwrite d3/f4a [4194304,4194304] 0 2024-05-27T09:29:43.641 INFO:tasks.workunit.client.1.smithi138.stdout:8/641: creat d3/d8/d6c/dd3/fd5 x:0 0 0 2024-05-27T09:29:43.646 INFO:tasks.workunit.client.1.smithi138.stdout:1/601: dwrite d4/d3c/d51/f61 [0,4194304] 0 2024-05-27T09:29:43.648 INFO:tasks.workunit.client.1.smithi138.stdout:8/642: rmdir d3/d8/d32/d3f 39 2024-05-27T09:29:43.649 INFO:tasks.workunit.client.1.smithi138.stdout:1/602: unlink d4/d1a/d2e/d4a/d7e/fa4 0 2024-05-27T09:29:43.653 INFO:tasks.workunit.client.1.smithi138.stdout:7/605: dread d0/d23/f48 [0,4194304] 0 2024-05-27T09:29:43.653 INFO:tasks.workunit.client.1.smithi138.stdout:1/603: mknod d4/d1a/d2e/d4a/cc3 0 2024-05-27T09:29:43.653 INFO:tasks.workunit.client.1.smithi138.stdout:1/604: truncate d4/d48/d22/fa5 2021441 0 2024-05-27T09:29:43.654 INFO:tasks.workunit.client.1.smithi138.stdout:1/605: fdatasync d4/f18 0 2024-05-27T09:29:43.654 INFO:tasks.workunit.client.1.smithi138.stdout:9/618: dwrite d6/da/d2f/d35/d55/f8a [0,4194304] 0 2024-05-27T09:29:43.654 INFO:tasks.workunit.client.1.smithi138.stdout:6/732: dwrite d3/d11/d2d/d37/d8b/fba [0,4194304] 0 2024-05-27T09:29:43.654 INFO:tasks.workunit.client.1.smithi138.stdout:6/733: write d3/d11/dc0/d43/d71/fea [739670,82883] 0 2024-05-27T09:29:43.654 INFO:tasks.workunit.client.1.smithi138.stdout:6/734: creat d3/d11/dc0/d5a/dcf/ff8 x:0 0 0 2024-05-27T09:29:43.656 INFO:tasks.workunit.client.1.smithi138.stdout:1/606: symlink d4/d48/d39/d5f/d63/lc4 0 2024-05-27T09:29:43.657 INFO:tasks.workunit.client.1.smithi138.stdout:5/525: rmdir d3/d12/d26 39 2024-05-27T09:29:43.663 INFO:tasks.workunit.client.1.smithi138.stdout:6/735: link d3/d19/f34 d3/d11/d2d/d37/daa/df3/ff9 0 2024-05-27T09:29:43.665 INFO:tasks.workunit.client.1.smithi138.stdout:9/619: unlink d6/da/d26/c3d 0 2024-05-27T09:29:43.665 INFO:tasks.workunit.client.1.smithi138.stdout:9/620: creat d6/da/d29/d8f/db7/dbd/d4c/fd0 x:0 0 0 2024-05-27T09:29:43.665 INFO:tasks.workunit.client.1.smithi138.stdout:9/621: truncate d6/db/f6a 4980983 0 2024-05-27T09:29:43.665 INFO:tasks.workunit.client.1.smithi138.stdout:1/607: symlink d4/d1a/d2e/d4a/d7e/dc0/lc5 0 2024-05-27T09:29:43.666 INFO:tasks.workunit.client.1.smithi138.stdout:8/643: dread d3/d8/d32/f59 [0,4194304] 0 2024-05-27T09:29:43.671 INFO:tasks.workunit.client.1.smithi138.stdout:6/736: creat d3/d11/dc0/d43/ffa x:0 0 0 2024-05-27T09:29:43.676 INFO:tasks.workunit.client.1.smithi138.stdout:6/737: stat d3/d11/dc0/d5a/d6a/l94 0 2024-05-27T09:29:43.677 INFO:tasks.workunit.client.1.smithi138.stdout:7/606: write d0/d23/d3a/f46 [2573499,95186] 0 2024-05-27T09:29:43.678 INFO:tasks.workunit.client.1.smithi138.stdout:7/607: write d0/d23/da8/fa2 [167790,128557] 0 2024-05-27T09:29:43.682 INFO:tasks.workunit.client.1.smithi138.stdout:6/738: link d3/d11/dc0/d5a/db4/fc9 d3/d11/d2d/d37/dd1/ffb 0 2024-05-27T09:29:43.682 INFO:tasks.workunit.client.1.smithi138.stdout:6/739: chown d3/d11/dc0/d74/cd4 42 1 2024-05-27T09:29:43.682 INFO:tasks.workunit.client.1.smithi138.stdout:9/622: dread d6/db/f10 [0,4194304] 0 2024-05-27T09:29:43.683 INFO:tasks.workunit.client.1.smithi138.stdout:6/740: symlink d3/d11/d2d/lfc 0 2024-05-27T09:29:43.684 INFO:tasks.workunit.client.1.smithi138.stdout:6/741: write d3/d11/d2d/d37/dd1/d47/f51 [1491963,79040] 0 2024-05-27T09:29:43.684 INFO:tasks.workunit.client.1.smithi138.stdout:6/742: truncate d3/d11/dc0/d43/d1c/fbc 782642 0 2024-05-27T09:29:43.685 INFO:tasks.workunit.client.1.smithi138.stdout:8/644: mkdir d3/d8/d32/dd6 0 2024-05-27T09:29:43.686 INFO:tasks.workunit.client.1.smithi138.stdout:1/608: dread d4/d48/f58 [0,4194304] 0 2024-05-27T09:29:43.686 INFO:tasks.workunit.client.1.smithi138.stdout:6/743: creat d3/d11/d33/ffd x:0 0 0 2024-05-27T09:29:43.687 INFO:tasks.workunit.client.1.smithi138.stdout:5/526: dread d3/f1b [0,4194304] 0 2024-05-27T09:29:43.688 INFO:tasks.workunit.client.1.smithi138.stdout:5/527: stat d3/d12/d32/l49 0 2024-05-27T09:29:43.688 INFO:tasks.workunit.client.1.smithi138.stdout:5/528: fsync d3/d12/d26/f5d 0 2024-05-27T09:29:43.689 INFO:tasks.workunit.client.1.smithi138.stdout:7/608: rename d0/d23/d49/f72 to d0/d23/da8/fc7 0 2024-05-27T09:29:43.691 INFO:tasks.workunit.client.1.smithi138.stdout:5/529: unlink d3/d12/d26/d53/f5a 0 2024-05-27T09:29:43.691 INFO:tasks.workunit.client.1.smithi138.stdout:5/530: chown d3/d12/l29 6 1 2024-05-27T09:29:43.699 INFO:tasks.workunit.client.1.smithi138.stdout:1/609: write d4/d3c/fa0 [353380,48827] 0 2024-05-27T09:29:43.699 INFO:tasks.workunit.client.1.smithi138.stdout:1/610: fdatasync d4/d48/d22/fa5 0 2024-05-27T09:29:43.702 INFO:tasks.workunit.client.1.smithi138.stdout:9/623: symlink d6/db/d8d/ld1 0 2024-05-27T09:29:43.702 INFO:tasks.workunit.client.1.smithi138.stdout:4/603: sync 2024-05-27T09:29:43.702 INFO:tasks.workunit.client.1.smithi138.stdout:3/560: sync 2024-05-27T09:29:43.703 INFO:tasks.workunit.client.1.smithi138.stdout:2/512: sync 2024-05-27T09:29:43.703 INFO:tasks.workunit.client.1.smithi138.stdout:7/609: rmdir d0/d23/d3c/d3f/db7 0 2024-05-27T09:29:43.703 INFO:tasks.workunit.client.1.smithi138.stdout:2/513: write d1/d15/d2d/d17/d25/d4d/d4e/f9e [396186,47642] 0 2024-05-27T09:29:43.704 INFO:tasks.workunit.client.1.smithi138.stdout:4/604: mknod d7/df/d35/d4c/d59/cd5 0 2024-05-27T09:29:43.704 INFO:tasks.workunit.client.1.smithi138.stdout:4/605: chown d7/df/d35/c61 341 1 2024-05-27T09:29:43.704 INFO:tasks.workunit.client.1.smithi138.stdout:7/610: mkdir d0/d2b/daf/dbf/dc8 0 2024-05-27T09:29:43.705 INFO:tasks.workunit.client.1.smithi138.stdout:2/514: truncate d1/d15/d2d/d17/d25/f2a 1998742 0 2024-05-27T09:29:43.708 INFO:tasks.workunit.client.1.smithi138.stdout:2/515: symlink d1/d10/d52/d5d/lb1 0 2024-05-27T09:29:43.715 INFO:tasks.workunit.client.1.smithi138.stdout:5/531: dread d3/d12/d16/d1d/d22/d2b/f38 [0,4194304] 0 2024-05-27T09:29:43.715 INFO:tasks.workunit.client.1.smithi138.stdout:4/606: link d7/df/d35/d4c/d59/d5d/d64/d6a/la8 d7/df/d5b/ld6 0 2024-05-27T09:29:43.715 INFO:tasks.workunit.client.1.smithi138.stdout:4/607: chown d7/c2f 98581 1 2024-05-27T09:29:43.715 INFO:tasks.workunit.client.1.smithi138.stdout:9/624: dread d6/da/d2f/d35/d55/f3a [0,4194304] 0 2024-05-27T09:29:43.717 INFO:tasks.workunit.client.1.smithi138.stdout:4/608: symlink d7/df/d5b/da1/db1/ld7 0 2024-05-27T09:29:43.720 INFO:tasks.workunit.client.1.smithi138.stdout:4/609: mknod d7/dba/dcb/cd8 0 2024-05-27T09:29:43.721 INFO:tasks.workunit.client.1.smithi138.stdout:1/611: rename d4/d3c/d51 to d4/d1a/d2e/d32/d8c/dc6 0 2024-05-27T09:29:43.722 INFO:tasks.workunit.client.1.smithi138.stdout:1/612: dread - d4/d48/d39/d5f/d63/fa3 zero size 2024-05-27T09:29:43.728 INFO:tasks.workunit.client.1.smithi138.stdout:4/610: mknod d7/df/d5b/da1/cd9 0 2024-05-27T09:29:43.728 INFO:tasks.workunit.client.1.smithi138.stdout:4/611: chown d7/df/d5b/f7f 1 1 2024-05-27T09:29:43.728 INFO:tasks.workunit.client.1.smithi138.stdout:3/561: write d0/d2/f13 [2139886,81800] 0 2024-05-27T09:29:43.729 INFO:tasks.workunit.client.1.smithi138.stdout:1/613: unlink d4/d48/d39/d5f/f50 0 2024-05-27T09:29:43.742 INFO:tasks.workunit.client.1.smithi138.stdout:7/611: write d0/d2b/daf/d57/f95 [2193430,16693] 0 2024-05-27T09:29:43.743 INFO:tasks.workunit.client.1.smithi138.stdout:7/612: stat d0/d23/d49/l4b 0 2024-05-27T09:29:43.746 INFO:tasks.workunit.client.1.smithi138.stdout:2/516: dread d1/d10/d52/fa6 [0,4194304] 0 2024-05-27T09:29:43.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:43 smithi119 ceph-mon[29082]: pgmap v19: 65 pgs: 65 active+clean; 3.3 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 256 MiB/s rd, 312 MiB/s wr, 445 op/s 2024-05-27T09:29:43.751 INFO:tasks.workunit.client.1.smithi138.stdout:8/645: dwrite d3/d8/d32/d3f/d9c/d9f/d86/f93 [0,4194304] 0 2024-05-27T09:29:43.753 INFO:tasks.workunit.client.1.smithi138.stdout:3/562: symlink d0/d3f/d50/d9b/d9f/da3/lb7 0 2024-05-27T09:29:43.763 INFO:tasks.workunit.client.1.smithi138.stdout:7/613: creat d0/d2b/daf/d98/fc9 x:0 0 0 2024-05-27T09:29:43.765 INFO:tasks.workunit.client.1.smithi138.stdout:2/517: mknod d1/d15/d2d/d17/d25/d4d/cb2 0 2024-05-27T09:29:43.769 INFO:tasks.workunit.client.1.smithi138.stdout:8/646: symlink d3/d8/d24/d55/d94/ld7 0 2024-05-27T09:29:43.769 INFO:tasks.workunit.client.1.smithi138.stdout:8/647: truncate d3/d8/d64/fca 471159 0 2024-05-27T09:29:43.770 INFO:tasks.workunit.client.1.smithi138.stdout:3/563: rmdir d0/d3f/d50/d9b 39 2024-05-27T09:29:43.776 INFO:tasks.workunit.client.1.smithi138.stdout:7/614: rename d0/f21 to d0/d23/d3c/fca 0 2024-05-27T09:29:43.779 INFO:tasks.workunit.client.1.smithi138.stdout:9/625: dwrite d6/da/d29/d8f/db7/fb8 [0,4194304] 0 2024-05-27T09:29:43.779 INFO:tasks.workunit.client.1.smithi138.stdout:2/518: truncate d1/d15/d2d/d17/d25/d4d/d4e/d60/f66 4018302 0 2024-05-27T09:29:43.780 INFO:tasks.workunit.client.1.smithi138.stdout:6/744: dwrite d3/d19/f3d [0,4194304] 0 2024-05-27T09:29:43.782 INFO:tasks.workunit.client.1.smithi138.stdout:8/648: rmdir d3/d8/d32 39 2024-05-27T09:29:43.782 INFO:tasks.workunit.client.1.smithi138.stdout:8/649: truncate d3/d8/d64/fca 1210472 0 2024-05-27T09:29:43.783 INFO:tasks.workunit.client.1.smithi138.stdout:3/564: mknod d0/d2/da/df/d10/d2a/cb8 0 2024-05-27T09:29:43.783 INFO:tasks.workunit.client.1.smithi138.stdout:3/565: chown d0/d2/da/df/d10/d2a/l72 722827723 1 2024-05-27T09:29:43.788 INFO:tasks.workunit.client.1.smithi138.stdout:6/745: rename d3/d11/d2d/d37/dd1/d47/l76 to d3/d11/d33/lfe 0 2024-05-27T09:29:43.788 INFO:tasks.workunit.client.1.smithi138.stdout:9/626: symlink d6/da/d29/d8f/ld2 0 2024-05-27T09:29:43.788 INFO:tasks.workunit.client.1.smithi138.stdout:6/746: creat d3/d11/dc0/d5a/fff x:0 0 0 2024-05-27T09:29:43.788 INFO:tasks.workunit.client.1.smithi138.stdout:2/519: stat d1/d15/d2d/d17/d25/l7f 0 2024-05-27T09:29:43.793 INFO:tasks.workunit.client.1.smithi138.stdout:6/747: mkdir d3/d11/d33/d100 0 2024-05-27T09:29:43.793 INFO:tasks.workunit.client.1.smithi138.stdout:8/650: mknod d3/d8/d24/d4d/cd8 0 2024-05-27T09:29:43.793 INFO:tasks.workunit.client.1.smithi138.stdout:9/627: chown d6/da/d2f/c83 2162 1 2024-05-27T09:29:43.793 INFO:tasks.workunit.client.1.smithi138.stdout:3/566: mkdir d0/d3f/d60/d93/db9 0 2024-05-27T09:29:43.803 INFO:tasks.workunit.client.1.smithi138.stdout:8/651: rename d3/d8/d24/fc1 to d3/db5/fd9 0 2024-05-27T09:29:43.805 INFO:tasks.workunit.client.1.smithi138.stdout:8/652: getdents d3/d8/d32/d45/d5c 0 2024-05-27T09:29:43.807 INFO:tasks.workunit.client.1.smithi138.stdout:8/653: creat d3/d8/d32/d45/d5c/fda x:0 0 0 2024-05-27T09:29:43.807 INFO:tasks.workunit.client.1.smithi138.stdout:8/654: creat d3/d8/d32/d45/d5c/fdb x:0 0 0 2024-05-27T09:29:43.807 INFO:tasks.workunit.client.1.smithi138.stdout:8/655: rename d3/d8/d32/d3f/d9c/db8 to d3/d8/d32/d3f/d9c/db8/ddc 22 2024-05-27T09:29:43.808 INFO:tasks.workunit.client.1.smithi138.stdout:2/520: dread d1/d15/d2d/d17/f6b [0,4194304] 0 2024-05-27T09:29:43.808 INFO:tasks.workunit.client.1.smithi138.stdout:2/521: chown d1/d15/d2d/d17/d25/l26 107557 1 2024-05-27T09:29:43.811 INFO:tasks.workunit.client.1.smithi138.stdout:8/656: mknod d3/d8/d6c/cdd 0 2024-05-27T09:29:43.811 INFO:tasks.workunit.client.1.smithi138.stdout:8/657: fsync f1 0 2024-05-27T09:29:43.812 INFO:tasks.workunit.client.1.smithi138.stdout:4/612: dwrite d7/f18 [0,4194304] 0 2024-05-27T09:29:43.813 INFO:tasks.workunit.client.1.smithi138.stdout:8/658: creat d3/d8/d72/fde x:0 0 0 2024-05-27T09:29:43.814 INFO:tasks.workunit.client.1.smithi138.stdout:8/659: getdents d3/d8/d32/d34/d38 0 2024-05-27T09:29:43.820 INFO:tasks.workunit.client.1.smithi138.stdout:8/660: symlink d3/d8/d24/d55/d94/dc5/ldf 0 2024-05-27T09:29:43.821 INFO:tasks.workunit.client.1.smithi138.stdout:5/532: dwrite d3/d75/f94 [0,4194304] 0 2024-05-27T09:29:43.821 INFO:tasks.workunit.client.1.smithi138.stdout:8/661: mkdir d3/d8/d24/d55/d7e/de0 0 2024-05-27T09:29:43.824 INFO:tasks.workunit.client.1.smithi138.stdout:8/662: rename d3/d8/d32/d45/d5c/l6f to d3/d8/d32/d45/dc2/le1 0 2024-05-27T09:29:43.827 INFO:tasks.workunit.client.1.smithi138.stdout:8/663: unlink d3/d8/d11/f99 0 2024-05-27T09:29:43.827 INFO:tasks.workunit.client.1.smithi138.stdout:8/664: read - d3/d8/d32/d34/d38/f79 zero size 2024-05-27T09:29:43.828 INFO:tasks.workunit.client.1.smithi138.stdout:8/665: mknod d3/d8/d32/dd6/ce2 0 2024-05-27T09:29:43.853 INFO:tasks.workunit.client.1.smithi138.stdout:8/666: getdents d3/d8/d32/d45/dc2 0 2024-05-27T09:29:43.854 INFO:tasks.workunit.client.1.smithi138.stdout:4/613: dread d7/df/d35/d4c/d59/d5d/d64/d6a/f76 [0,4194304] 0 2024-05-27T09:29:43.861 INFO:tasks.workunit.client.1.smithi138.stdout:1/614: dwrite d4/d48/d39/d5f/d4d/d2b/f67 [0,4194304] 0 2024-05-27T09:29:43.861 INFO:tasks.workunit.client.1.smithi138.stdout:1/615: dread - d4/d48/d39/d5f/d4d/f8b zero size 2024-05-27T09:29:43.872 INFO:tasks.workunit.client.1.smithi138.stdout:1/616: truncate d4/d48/d39/d5f/d93/faf 2325262 0 2024-05-27T09:29:43.872 INFO:tasks.workunit.client.1.smithi138.stdout:5/533: dwrite d3/d12/d16/d1d/d22/f3a [0,4194304] 0 2024-05-27T09:29:43.878 INFO:tasks.workunit.client.1.smithi138.stdout:5/534: symlink d3/d12/d16/d1d/d23/d68/la7 0 2024-05-27T09:29:43.880 INFO:tasks.workunit.client.1.smithi138.stdout:5/535: truncate d3/d12/d16/d1d/d23/d3b/fa2 1041784 0 2024-05-27T09:29:43.880 INFO:tasks.workunit.client.1.smithi138.stdout:5/536: chown d3/d12/d16/d1d/d23/d68/f6a 460093472 1 2024-05-27T09:29:43.884 INFO:tasks.workunit.client.1.smithi138.stdout:9/628: dwrite d6/da/d29/d8f/db7/dbd/d46/f47 [4194304,4194304] 0 2024-05-27T09:29:43.884 INFO:tasks.workunit.client.1.smithi138.stdout:9/629: creat d6/da/d29/d8f/db7/fd3 x:0 0 0 2024-05-27T09:29:43.884 INFO:tasks.workunit.client.1.smithi138.stdout:9/630: fdatasync d6/f3f 0 2024-05-27T09:29:43.884 INFO:tasks.workunit.client.1.smithi138.stdout:4/614: dread d7/df/f4d [0,4194304] 0 2024-05-27T09:29:43.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:43 smithi138 ceph-mon[34885]: pgmap v19: 65 pgs: 65 active+clean; 3.3 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 256 MiB/s rd, 312 MiB/s wr, 445 op/s 2024-05-27T09:29:43.886 INFO:tasks.workunit.client.1.smithi138.stdout:9/631: creat d6/db/fd4 x:0 0 0 2024-05-27T09:29:43.886 INFO:tasks.workunit.client.1.smithi138.stdout:9/632: creat d6/db/de/d85/fd5 x:0 0 0 2024-05-27T09:29:43.886 INFO:tasks.workunit.client.1.smithi138.stdout:9/633: chown d6/db/de/d85/fc9 325549837 1 2024-05-27T09:29:43.886 INFO:tasks.workunit.client.1.smithi138.stdout:9/634: dread - d6/fbc zero size 2024-05-27T09:29:43.886 INFO:tasks.workunit.client.1.smithi138.stdout:7/615: dwrite d0/d2b/daf/f12 [0,4194304] 0 2024-05-27T09:29:43.886 INFO:tasks.workunit.client.1.smithi138.stdout:1/617: dread d4/d1a/d2e/d32/f7a [0,4194304] 0 2024-05-27T09:29:43.889 INFO:tasks.workunit.client.1.smithi138.stdout:4/615: mkdir d7/dba/dcf/daa/dda 0 2024-05-27T09:29:43.889 INFO:tasks.workunit.client.1.smithi138.stdout:4/616: chown d7/df/f68 0 1 2024-05-27T09:29:43.889 INFO:tasks.workunit.client.1.smithi138.stdout:4/617: fsync d7/d6e/fc4 0 2024-05-27T09:29:43.892 INFO:tasks.workunit.client.1.smithi138.stdout:9/635: dread d6/da/d2f/d35/d55/f3a [0,4194304] 0 2024-05-27T09:29:43.892 INFO:tasks.workunit.client.1.smithi138.stdout:9/636: creat d6/da/d26/fd6 x:0 0 0 2024-05-27T09:29:43.893 INFO:tasks.workunit.client.1.smithi138.stdout:7/616: mknod d0/d2b/daf/d98/db6/ccb 0 2024-05-27T09:29:43.895 INFO:tasks.workunit.client.1.smithi138.stdout:7/617: symlink d0/d2b/daf/d98/lcc 0 2024-05-27T09:29:43.895 INFO:tasks.workunit.client.1.smithi138.stdout:7/618: fsync d0/d23/f48 0 2024-05-27T09:29:43.897 INFO:tasks.workunit.client.1.smithi138.stdout:7/619: truncate d0/d23/f47 1205585 0 2024-05-27T09:29:43.897 INFO:tasks.workunit.client.1.smithi138.stdout:7/620: chown d0/d2b/daf/d60/d9d 440317 1 2024-05-27T09:29:43.899 INFO:tasks.workunit.client.1.smithi138.stdout:7/621: rename d0/d23/d3c/d3f/c73 to d0/d23/d49/d78/ccd 0 2024-05-27T09:29:43.900 INFO:tasks.workunit.client.1.smithi138.stdout:0/562: sync 2024-05-27T09:29:43.900 INFO:tasks.workunit.client.1.smithi138.stdout:0/563: fsync f5 0 2024-05-27T09:29:43.901 INFO:tasks.workunit.client.1.smithi138.stdout:7/622: symlink d0/d2b/daf/d64/lce 0 2024-05-27T09:29:43.901 INFO:tasks.workunit.client.1.smithi138.stdout:5/537: truncate d3/f18 833581 0 2024-05-27T09:29:43.902 INFO:tasks.workunit.client.1.smithi138.stdout:7/623: unlink d0/d23/da8/fc7 0 2024-05-27T09:29:43.903 INFO:tasks.workunit.client.1.smithi138.stdout:7/624: symlink d0/d2b/daf/d57/lcf 0 2024-05-27T09:29:43.904 INFO:tasks.workunit.client.1.smithi138.stdout:0/564: rename da/d2a/d8e/d1f/d42/d4a/d53/c5a to da/d2a/d8e/d1f/d71/cb9 0 2024-05-27T09:29:43.905 INFO:tasks.workunit.client.1.smithi138.stdout:7/625: symlink d0/d2b/daf/d60/d9d/ld0 0 2024-05-27T09:29:43.906 INFO:tasks.workunit.client.1.smithi138.stdout:7/626: getdents d0/d2b/daf/dbf 0 2024-05-27T09:29:43.906 INFO:tasks.workunit.client.1.smithi138.stdout:7/627: stat d0/d2b/daf/d57/d91 0 2024-05-27T09:29:43.906 INFO:tasks.workunit.client.1.smithi138.stdout:7/628: creat d0/d23/d3c/d84/fd1 x:0 0 0 2024-05-27T09:29:43.906 INFO:tasks.workunit.client.1.smithi138.stdout:7/629: read d0/d2b/daf/fe [2204864,104145] 0 2024-05-27T09:29:43.909 INFO:tasks.workunit.client.1.smithi138.stdout:2/522: dwrite d1/f1b [4194304,4194304] 0 2024-05-27T09:29:43.909 INFO:tasks.workunit.client.1.smithi138.stdout:8/667: dwrite d3/d8/d32/d88/f8e [0,4194304] 0 2024-05-27T09:29:43.913 INFO:tasks.workunit.client.1.smithi138.stdout:5/538: rename d3/d12/d26/f4b to d3/d12/d16/d1d/d23/d3b/fa8 0 2024-05-27T09:29:43.915 INFO:tasks.workunit.client.1.smithi138.stdout:2/523: getdents d1/d15/d2d/d17/d25/d4d 0 2024-05-27T09:29:43.915 INFO:tasks.workunit.client.1.smithi138.stdout:8/668: symlink d3/d8/d24/d55/d94/dc5/le3 0 2024-05-27T09:29:43.917 INFO:tasks.workunit.client.1.smithi138.stdout:8/669: creat d3/d8/d32/d34/dbd/fe4 x:0 0 0 2024-05-27T09:29:43.917 INFO:tasks.workunit.client.1.smithi138.stdout:8/670: readlink d3/d8/d64/l2c 0 2024-05-27T09:29:43.919 INFO:tasks.workunit.client.1.smithi138.stdout:8/671: mknod d3/d8/d24/ce5 0 2024-05-27T09:29:43.921 INFO:tasks.workunit.client.1.smithi138.stdout:1/618: dread d4/d3c/f4e [0,4194304] 0 2024-05-27T09:29:43.926 INFO:tasks.workunit.client.1.smithi138.stdout:0/565: rename da/d2a/d8e/d1f/c84 to da/d2a/d8e/d43/cba 0 2024-05-27T09:29:43.927 INFO:tasks.workunit.client.1.smithi138.stdout:5/539: creat d3/d12/d16/fa9 x:0 0 0 2024-05-27T09:29:43.932 INFO:tasks.workunit.client.1.smithi138.stdout:3/567: dwrite d0/d2/da/f1b [0,4194304] 0 2024-05-27T09:29:43.934 INFO:tasks.workunit.client.1.smithi138.stdout:0/566: rmdir da/d2a/d8e/d43/d74 39 2024-05-27T09:29:43.935 INFO:tasks.workunit.client.1.smithi138.stdout:5/540: truncate d3/d12/d16/d1d/d22/d2b/d36/f43 509593 0 2024-05-27T09:29:43.937 INFO:tasks.workunit.client.1.smithi138.stdout:7/630: dread d0/d23/f58 [0,4194304] 0 2024-05-27T09:29:43.942 INFO:tasks.workunit.client.1.smithi138.stdout:2/524: rename d1/d15/d2d/d17/d25/d9d/d4b/f4c to d1/d15/d2d/d17/d25/d9d/fb3 0 2024-05-27T09:29:43.942 INFO:tasks.workunit.client.1.smithi138.stdout:7/631: rmdir d0/d2b/daf/d57 39 2024-05-27T09:29:43.942 INFO:tasks.workunit.client.1.smithi138.stdout:1/619: mkdir d4/dc7 0 2024-05-27T09:29:43.942 INFO:tasks.workunit.client.1.smithi138.stdout:7/632: creat d0/d2b/daf/dbf/fd2 x:0 0 0 2024-05-27T09:29:43.948 INFO:tasks.workunit.client.1.smithi138.stdout:0/567: creat da/d2a/d8e/d1f/d42/d4a/fbb x:0 0 0 2024-05-27T09:29:43.948 INFO:tasks.workunit.client.1.smithi138.stdout:1/620: creat d4/d3c/fc8 x:0 0 0 2024-05-27T09:29:43.948 INFO:tasks.workunit.client.1.smithi138.stdout:0/568: write da/f66 [6365,3789] 0 2024-05-27T09:29:43.955 INFO:tasks.workunit.client.1.smithi138.stdout:8/672: write d3/d8/d64/f7c [3578319,1207] 0 2024-05-27T09:29:43.962 INFO:tasks.workunit.client.1.smithi138.stdout:8/673: creat d3/d8/d32/fe6 x:0 0 0 2024-05-27T09:29:43.964 INFO:tasks.workunit.client.1.smithi138.stdout:8/674: mknod d3/d8/d32/d34/dbd/ce7 0 2024-05-27T09:29:43.964 INFO:tasks.workunit.client.1.smithi138.stdout:8/675: write d3/d8/d32/d3f/f6b [2385787,60200] 0 2024-05-27T09:29:43.971 INFO:tasks.workunit.client.1.smithi138.stdout:0/569: getdents da/d2a/d8e/d1f/d71 0 2024-05-27T09:29:43.979 INFO:tasks.workunit.client.1.smithi138.stdout:0/570: mknod da/d2a/d8e/d1f/cbc 0 2024-05-27T09:29:43.980 INFO:tasks.workunit.client.1.smithi138.stdout:4/618: dwrite d7/dba/dcf/d3a/fb0 [0,4194304] 0 2024-05-27T09:29:43.991 INFO:tasks.workunit.client.1.smithi138.stdout:5/541: dwrite d3/fd [0,4194304] 0 2024-05-27T09:29:43.991 INFO:tasks.workunit.client.1.smithi138.stdout:5/542: chown d3/d12/d32/d60/f61 4115 1 2024-05-27T09:29:43.997 INFO:tasks.workunit.client.1.smithi138.stdout:4/619: creat d7/df/d35/d4c/dc0/fdb x:0 0 0 2024-05-27T09:29:44.002 INFO:tasks.workunit.client.1.smithi138.stdout:4/620: creat d7/fdc x:0 0 0 2024-05-27T09:29:44.002 INFO:tasks.workunit.client.1.smithi138.stdout:4/621: chown d7/df/d35/f5c 2806795 1 2024-05-27T09:29:44.003 INFO:tasks.workunit.client.1.smithi138.stdout:5/543: dread d3/d12/d26/d53/f85 [0,4194304] 0 2024-05-27T09:29:44.004 INFO:tasks.workunit.client.1.smithi138.stdout:5/544: mknod d3/d12/d16/d1d/d23/caa 0 2024-05-27T09:29:44.004 INFO:tasks.workunit.client.1.smithi138.stdout:5/545: write d3/d12/d16/d1d/f97 [91525,10961] 0 2024-05-27T09:29:44.005 INFO:tasks.workunit.client.1.smithi138.stdout:7/633: dwrite d0/d23/da8/fa2 [0,4194304] 0 2024-05-27T09:29:44.010 INFO:tasks.workunit.client.1.smithi138.stdout:2/525: dwrite d1/d15/d2d/fa0 [0,4194304] 0 2024-05-27T09:29:44.010 INFO:tasks.workunit.client.1.smithi138.stdout:2/526: chown d1/d15/d2d/d17/d25/d9d/ca2 0 1 2024-05-27T09:29:44.026 INFO:tasks.workunit.client.1.smithi138.stdout:7/634: write d0/d23/fa0 [3783411,5623] 0 2024-05-27T09:29:44.031 INFO:tasks.workunit.client.1.smithi138.stdout:7/635: stat d0/d2b/daf/c17 0 2024-05-27T09:29:44.031 INFO:tasks.workunit.client.1.smithi138.stdout:7/636: readlink d0/d23/d49/d78/l7f 0 2024-05-27T09:29:44.031 INFO:tasks.workunit.client.1.smithi138.stdout:2/527: dread d1/d2f/f33 [0,4194304] 0 2024-05-27T09:29:44.031 INFO:tasks.workunit.client.1.smithi138.stdout:2/528: chown d1/d15/d2d/d17/d25/d4d/d6d/fab 86924305 1 2024-05-27T09:29:44.031 INFO:tasks.workunit.client.1.smithi138.stdout:2/529: fsync d1/d10/fe 0 2024-05-27T09:29:44.034 INFO:tasks.workunit.client.1.smithi138.stdout:1/621: dread d4/d48/d22/f5a [0,4194304] 0 2024-05-27T09:29:44.042 INFO:tasks.workunit.client.1.smithi138.stdout:3/568: dwrite d0/d2/da/df/d10/d2a/f7e [0,4194304] 0 2024-05-27T09:29:44.043 INFO:tasks.workunit.client.1.smithi138.stdout:3/569: truncate d0/d3f/d50/f97 213843 0 2024-05-27T09:29:44.043 INFO:tasks.workunit.client.1.smithi138.stdout:3/570: dread - d0/d2/da/df/d10/d2a/d61/fb5 zero size 2024-05-27T09:29:44.073 INFO:tasks.workunit.client.1.smithi138.stdout:3/571: dread d0/d80/f92 [0,4194304] 0 2024-05-27T09:29:44.074 INFO:tasks.workunit.client.1.smithi138.stdout:3/572: truncate d0/d2/da/df/d10/f4a 3269001 0 2024-05-27T09:29:44.076 INFO:tasks.workunit.client.1.smithi138.stdout:3/573: unlink d0/d3f/l69 0 2024-05-27T09:29:44.082 INFO:tasks.workunit.client.1.smithi138.stdout:3/574: rename d0/c66 to d0/d3f/d60/daa/cba 0 2024-05-27T09:29:44.086 INFO:tasks.workunit.client.1.smithi138.stdout:3/575: creat d0/d2/da/df/d10/d2a/db2/fbb x:0 0 0 2024-05-27T09:29:44.088 INFO:tasks.workunit.client.1.smithi138.stdout:3/576: symlink d0/d2/da/df/d10/d2a/db2/lbc 0 2024-05-27T09:29:44.091 INFO:tasks.workunit.client.1.smithi138.stdout:3/577: mknod d0/d3f/d60/d93/db9/cbd 0 2024-05-27T09:29:44.091 INFO:tasks.workunit.client.1.smithi138.stdout:3/578: dread - d0/d3f/d50/f5c zero size 2024-05-27T09:29:44.095 INFO:tasks.workunit.client.1.smithi138.stdout:3/579: creat d0/d2/da/df/d10/d2a/fbe x:0 0 0 2024-05-27T09:29:44.095 INFO:tasks.workunit.client.1.smithi138.stdout:2/530: dwrite d1/d15/d2d/d17/d25/d9d/f3b [0,4194304] 0 2024-05-27T09:29:44.095 INFO:tasks.workunit.client.1.smithi138.stdout:2/531: fsync d1/d15/d2d/d1d/f43 0 2024-05-27T09:29:44.095 INFO:tasks.workunit.client.1.smithi138.stdout:7/637: dwrite d0/d2b/daf/f56 [0,4194304] 0 2024-05-27T09:29:44.098 INFO:tasks.workunit.client.1.smithi138.stdout:2/532: rename d1/d15/d2d/d2e/f90 to d1/d2f/fb4 0 2024-05-27T09:29:44.101 INFO:tasks.workunit.client.1.smithi138.stdout:2/533: rename d1/d10/d52/c7c to d1/d15/d2d/d17/d25/d4d/cb5 0 2024-05-27T09:29:44.106 INFO:tasks.workunit.client.1.smithi138.stdout:1/622: dwrite d4/d3c/f42 [0,4194304] 0 2024-05-27T09:29:44.106 INFO:tasks.workunit.client.1.smithi138.stdout:1/623: truncate d4/d48/d39/d5f/f6a 945521 0 2024-05-27T09:29:44.108 INFO:tasks.workunit.client.1.smithi138.stdout:1/624: getdents d4/d1a 0 2024-05-27T09:29:44.108 INFO:tasks.workunit.client.1.smithi138.stdout:7/638: write d0/f10 [1041313,125114] 0 2024-05-27T09:29:44.108 INFO:tasks.workunit.client.1.smithi138.stdout:7/639: fdatasync d0/d23/d3a/f55 0 2024-05-27T09:29:44.108 INFO:tasks.workunit.client.1.smithi138.stdout:7/640: chown d0/d2b/daf/dba 0 1 2024-05-27T09:29:44.117 INFO:tasks.workunit.client.1.smithi138.stdout:1/625: link d4/d48/d39/d5f/d4d/f8b d4/d48/d39/d5f/d93/fc9 0 2024-05-27T09:29:44.121 INFO:tasks.workunit.client.1.smithi138.stdout:3/580: dread d0/d2/da/df/d10/f77 [0,4194304] 0 2024-05-27T09:29:44.125 INFO:tasks.workunit.client.1.smithi138.stdout:7/641: dread d0/d2b/daf/d57/d91/fb4 [0,4194304] 0 2024-05-27T09:29:44.125 INFO:tasks.workunit.client.1.smithi138.stdout:7/642: dread - d0/d2b/daf/d98/fc9 zero size 2024-05-27T09:29:44.129 INFO:tasks.workunit.client.1.smithi138.stdout:7/643: unlink d0/d23/d49/c94 0 2024-05-27T09:29:44.171 INFO:tasks.workunit.client.1.smithi138.stdout:2/534: dwrite d1/d2f/f7d [0,4194304] 0 2024-05-27T09:29:44.179 INFO:tasks.workunit.client.1.smithi138.stdout:2/535: dread d1/d15/d2d/d17/d25/d9d/f5b [0,4194304] 0 2024-05-27T09:29:44.197 INFO:tasks.workunit.client.1.smithi138.stdout:2/536: dread d1/d15/d2d/d17/d25/d9d/f3b [0,4194304] 0 2024-05-27T09:29:44.197 INFO:tasks.workunit.client.1.smithi138.stdout:2/537: chown d1/d10/d3c/c63 1928546 1 2024-05-27T09:29:44.198 INFO:tasks.workunit.client.1.smithi138.stdout:2/538: mkdir d1/d15/d2d/d17/d25/d4d/db6 0 2024-05-27T09:29:44.201 INFO:tasks.workunit.client.1.smithi138.stdout:2/539: link d1/d15/d2d/d17/d25/d9d/d4b/fa1 d1/d15/d2d/d17/d25/d4d/d6d/fb7 0 2024-05-27T09:29:44.202 INFO:tasks.workunit.client.1.smithi138.stdout:2/540: stat d1/d10/d52/d5d/f91 0 2024-05-27T09:29:44.202 INFO:tasks.workunit.client.1.smithi138.stdout:2/541: rename d1/d15/d2d/d17/d25 to d1/d15/d2d/d17/d25/d4d/db6/db8 22 2024-05-27T09:29:44.203 INFO:tasks.workunit.client.1.smithi138.stdout:2/542: mkdir d1/d10/d52/db9 0 2024-05-27T09:29:44.203 INFO:tasks.workunit.client.1.smithi138.stdout:2/543: dread - d1/d15/d2d/d17/f9b zero size 2024-05-27T09:29:44.204 INFO:tasks.workunit.client.1.smithi138.stdout:2/544: creat d1/d15/d2d/d17/d25/d4d/d6d/fba x:0 0 0 2024-05-27T09:29:44.205 INFO:tasks.workunit.client.1.smithi138.stdout:2/545: rename d1/d10 to d1/d10/d52/d5d/dbb 22 2024-05-27T09:29:44.205 INFO:tasks.workunit.client.1.smithi138.stdout:2/546: stat d1/d15/d2d/d17/d25/d4d/d4e/d60/f92 0 2024-05-27T09:29:44.205 INFO:tasks.workunit.client.1.smithi138.stdout:2/547: write d1/d15/d2d/d17/d25/d4d/d6d/d87/fae [175293,110824] 0 2024-05-27T09:29:44.205 INFO:tasks.workunit.client.1.smithi138.stdout:2/548: mknod d1/d15/d2d/d7b/cbc 0 2024-05-27T09:29:44.206 INFO:tasks.workunit.client.1.smithi138.stdout:2/549: readlink d1/d15/d2d/d17/l30 0 2024-05-27T09:29:44.206 INFO:tasks.workunit.client.1.smithi138.stdout:2/550: creat d1/d10/d52/d5d/fbd x:0 0 0 2024-05-27T09:29:44.208 INFO:tasks.workunit.client.1.smithi138.stdout:2/551: creat d1/d15/d2d/d17/d25/d9d/d4b/fbe x:0 0 0 2024-05-27T09:29:44.208 INFO:tasks.workunit.client.1.smithi138.stdout:2/552: readlink d1/d10/d3c/l82 0 2024-05-27T09:29:44.209 INFO:tasks.workunit.client.1.smithi138.stdout:2/553: chown d1/d15/d2d/f39 22577108 1 2024-05-27T09:29:44.210 INFO:tasks.workunit.client.1.smithi138.stdout:7/644: dwrite d0/d2b/daf/f6b [0,4194304] 0 2024-05-27T09:29:44.218 INFO:tasks.workunit.client.1.smithi138.stdout:2/554: dread d1/d15/d2d/d17/d25/d4d/d6d/fab [0,4194304] 0 2024-05-27T09:29:44.218 INFO:tasks.workunit.client.1.smithi138.stdout:2/555: truncate d1/d15/d2d/d17/d25/d4d/d4e/f9e 482250 0 2024-05-27T09:29:44.218 INFO:tasks.workunit.client.1.smithi138.stdout:2/556: truncate d1/d15/d2d/d17/d25/d4d/d4e/f9e 548426 0 2024-05-27T09:29:44.219 INFO:tasks.workunit.client.1.smithi138.stdout:2/557: unlink d1/d15/d2d/d7b/c96 0 2024-05-27T09:29:44.242 INFO:tasks.workunit.client.1.smithi138.stdout:7/645: dwrite d0/f32 [4194304,4194304] 0 2024-05-27T09:29:44.243 INFO:tasks.workunit.client.1.smithi138.stdout:7/646: creat d0/d2b/daf/d60/fd3 x:0 0 0 2024-05-27T09:29:44.248 INFO:tasks.workunit.client.1.smithi138.stdout:7/647: read d0/f8c [477313,68895] 0 2024-05-27T09:29:44.255 INFO:tasks.workunit.client.1.smithi138.stdout:7/648: dread d0/d23/d49/d78/f9b [0,4194304] 0 2024-05-27T09:29:44.273 INFO:tasks.workunit.client.1.smithi138.stdout:7/649: dread d0/d2b/daf/d64/f7d [0,4194304] 0 2024-05-27T09:29:44.319 INFO:tasks.workunit.client.1.smithi138.stdout:7/650: dwrite d0/f83 [0,4194304] 0 2024-05-27T09:29:44.319 INFO:tasks.workunit.client.1.smithi138.stdout:7/651: readlink d0/d2b/l2c 0 2024-05-27T09:29:44.319 INFO:tasks.workunit.client.1.smithi138.stdout:7/652: read - d0/d2b/daf/d60/d9d/fad zero size 2024-05-27T09:29:44.328 INFO:tasks.workunit.client.1.smithi138.stdout:6/748: sync 2024-05-27T09:29:44.328 INFO:tasks.workunit.client.1.smithi138.stdout:9/637: sync 2024-05-27T09:29:44.328 INFO:tasks.workunit.client.1.smithi138.stdout:9/638: dread - d6/da/d29/d59/f90 zero size 2024-05-27T09:29:44.330 INFO:tasks.workunit.client.1.smithi138.stdout:6/749: creat d3/d11/d2d/d37/dd1/ded/f101 x:0 0 0 2024-05-27T09:29:44.333 INFO:tasks.workunit.client.1.smithi138.stdout:6/750: mkdir d3/d11/d2d/d37/d102 0 2024-05-27T09:29:44.341 INFO:tasks.workunit.client.1.smithi138.stdout:9/639: link d6/db/dc/cf d6/db/d8d/cd7 0 2024-05-27T09:29:44.350 INFO:tasks.workunit.client.1.smithi138.stdout:9/640: getdents d6/db/d8d 0 2024-05-27T09:29:44.351 INFO:tasks.workunit.client.1.smithi138.stdout:9/641: write d6/da/d29/f89 [4603206,62534] 0 2024-05-27T09:29:44.353 INFO:tasks.workunit.client.1.smithi138.stdout:9/642: mkdir d6/da/d2f/d35/d55/d75/dd8 0 2024-05-27T09:29:44.356 INFO:tasks.workunit.client.1.smithi138.stdout:9/643: truncate d6/da/d26/fb0 621685 0 2024-05-27T09:29:44.356 INFO:tasks.workunit.client.1.smithi138.stdout:9/644: creat d6/da/d29/d8f/db7/dbd/d4c/fd9 x:0 0 0 2024-05-27T09:29:44.356 INFO:tasks.workunit.client.1.smithi138.stdout:9/645: truncate d6/db/de/d18/d24/f5c 4467754 0 2024-05-27T09:29:44.392 INFO:tasks.workunit.client.1.smithi138.stdout:7/653: dwrite d0/d2b/daf/f56 [0,4194304] 0 2024-05-27T09:29:44.408 INFO:tasks.workunit.client.1.smithi138.stdout:6/751: dwrite d3/d11/d2d/d37/f3c [4194304,4194304] 0 2024-05-27T09:29:44.409 INFO:tasks.workunit.client.1.smithi138.stdout:6/752: fdatasync d3/d11/dc0/fd3 0 2024-05-27T09:29:44.425 INFO:tasks.workunit.client.1.smithi138.stdout:9/646: dwrite d6/da/fa9 [0,4194304] 0 2024-05-27T09:29:44.430 INFO:tasks.workunit.client.1.smithi138.stdout:9/647: creat d6/da/d29/d8f/fda x:0 0 0 2024-05-27T09:29:44.431 INFO:tasks.workunit.client.1.smithi138.stdout:9/648: mkdir d6/db/de/d18/ddb 0 2024-05-27T09:29:44.432 INFO:tasks.workunit.client.1.smithi138.stdout:9/649: fsync d6/db/de/fa7 0 2024-05-27T09:29:44.436 INFO:tasks.workunit.client.1.smithi138.stdout:9/650: unlink d6/db/dc/f1c 0 2024-05-27T09:29:44.436 INFO:tasks.workunit.client.1.smithi138.stdout:9/651: dread - d6/da/d2f/d35/d55/fc6 zero size 2024-05-27T09:29:44.438 INFO:tasks.workunit.client.1.smithi138.stdout:9/652: creat d6/db/d8d/fdc x:0 0 0 2024-05-27T09:29:44.438 INFO:tasks.workunit.client.1.smithi138.stdout:9/653: write d6/da/d2f/d35/d55/f3a [1112702,17752] 0 2024-05-27T09:29:44.438 INFO:tasks.workunit.client.1.smithi138.stdout:9/654: write d6/da/d2f/d35/d55/d75/d8c/fb2 [4623849,24107] 0 2024-05-27T09:29:44.457 INFO:tasks.workunit.client.1.smithi138.stdout:7/654: dwrite d0/d23/d3a/f55 [0,4194304] 0 2024-05-27T09:29:44.460 INFO:tasks.workunit.client.1.smithi138.stdout:7/655: truncate d0/d23/d3c/fca 2723844 0 2024-05-27T09:29:44.464 INFO:tasks.workunit.client.1.smithi138.stdout:7/656: creat d0/d23/d3c/d84/d9c/dc3/fd4 x:0 0 0 2024-05-27T09:29:44.464 INFO:tasks.workunit.client.1.smithi138.stdout:7/657: fdatasync d0/d2b/f5e 0 2024-05-27T09:29:44.464 INFO:tasks.workunit.client.1.smithi138.stdout:7/658: creat d0/d23/d3a/fd5 x:0 0 0 2024-05-27T09:29:44.480 INFO:tasks.workunit.client.1.smithi138.stdout:6/753: dwrite d3/d11/dc0/d5a/fe9 [0,4194304] 0 2024-05-27T09:29:44.480 INFO:tasks.workunit.client.1.smithi138.stdout:6/754: creat d3/d11/f103 x:0 0 0 2024-05-27T09:29:44.481 INFO:tasks.workunit.client.1.smithi138.stdout:6/755: write d3/d11/d2d/f39 [1456884,7875] 0 2024-05-27T09:29:44.486 INFO:tasks.workunit.client.1.smithi138.stdout:7/659: write d0/d23/d3a/f55 [3735042,111168] 0 2024-05-27T09:29:44.491 INFO:tasks.workunit.client.1.smithi138.stdout:7/660: creat d0/d2b/daf/d57/d91/db8/fd6 x:0 0 0 2024-05-27T09:29:44.492 INFO:tasks.workunit.client.1.smithi138.stdout:7/661: creat d0/d23/da8/d8e/fd7 x:0 0 0 2024-05-27T09:29:44.494 INFO:tasks.workunit.client.1.smithi138.stdout:7/662: symlink d0/d2b/daf/dbf/dc8/ld8 0 2024-05-27T09:29:44.494 INFO:tasks.workunit.client.1.smithi138.stdout:7/663: stat d0/d23/d3c 0 2024-05-27T09:29:44.495 INFO:tasks.workunit.client.1.smithi138.stdout:6/756: mkdir d3/d11/dc0/d5a/df0/d104 0 2024-05-27T09:29:44.499 INFO:tasks.workunit.client.1.smithi138.stdout:6/757: creat d3/d11/dc0/d5a/df0/d104/f105 x:0 0 0 2024-05-27T09:29:44.499 INFO:tasks.workunit.client.1.smithi138.stdout:6/758: fdatasync d3/d11/dc0/d43/d1c/f4c 0 2024-05-27T09:29:44.499 INFO:tasks.workunit.client.1.smithi138.stdout:6/759: creat d3/d11/dc0/d43/dbe/f106 x:0 0 0 2024-05-27T09:29:44.500 INFO:tasks.workunit.client.1.smithi138.stdout:6/760: readlink d3/d11/l1d 0 2024-05-27T09:29:44.506 INFO:tasks.workunit.client.1.smithi138.stdout:6/761: unlink d3/f5 0 2024-05-27T09:29:44.529 INFO:tasks.workunit.client.1.smithi138.stdout:6/762: write d3/fe2 [6753851,102108] 0 2024-05-27T09:29:44.554 INFO:tasks.workunit.client.1.smithi138.stdout:6/763: dread d3/d11/dc0/d5a/d6a/f6b [0,4194304] 0 2024-05-27T09:29:44.554 INFO:tasks.workunit.client.1.smithi138.stdout:6/764: fsync d3/d11/dc0/d5a/dcf/ff8 0 2024-05-27T09:29:44.554 INFO:tasks.workunit.client.1.smithi138.stdout:7/664: dwrite d0/d2b/daf/f56 [4194304,4194304] 0 2024-05-27T09:29:44.555 INFO:tasks.workunit.client.1.smithi138.stdout:7/665: chown d0/d2b/daf/d43/f67 1487 1 2024-05-27T09:29:44.556 INFO:tasks.workunit.client.1.smithi138.stdout:7/666: unlink d0/d2b/fab 0 2024-05-27T09:29:44.557 INFO:tasks.workunit.client.1.smithi138.stdout:7/667: write d0/d2b/daf/d43/f7a [898983,114175] 0 2024-05-27T09:29:44.574 INFO:tasks.workunit.client.1.smithi138.stdout:7/668: read d0/f36 [1760550,9679] 0 2024-05-27T09:29:44.575 INFO:tasks.workunit.client.1.smithi138.stdout:7/669: truncate d0/d2b/daf/d57/f9e 1298781 0 2024-05-27T09:29:44.575 INFO:tasks.workunit.client.1.smithi138.stdout:7/670: stat d0/d23/d3a/c6f 0 2024-05-27T09:29:44.575 INFO:tasks.workunit.client.1.smithi138.stdout:7/671: fdatasync d0/f1a 0 2024-05-27T09:29:44.575 INFO:tasks.workunit.client.1.smithi138.stdout:7/672: read d0/d23/f4c [1889028,83850] 0 2024-05-27T09:29:44.576 INFO:tasks.workunit.client.1.smithi138.stdout:7/673: symlink d0/d23/d49/ld9 0 2024-05-27T09:29:44.577 INFO:tasks.workunit.client.1.smithi138.stdout:7/674: fsync d0/d23/f4f 0 2024-05-27T09:29:44.579 INFO:tasks.workunit.client.1.smithi138.stdout:7/675: symlink d0/d23/d3c/lda 0 2024-05-27T09:29:44.582 INFO:tasks.workunit.client.1.smithi138.stdout:5/546: sync 2024-05-27T09:29:44.582 INFO:tasks.workunit.client.1.smithi138.stdout:6/765: dread d3/d19/f3d [0,4194304] 0 2024-05-27T09:29:44.582 INFO:tasks.workunit.client.1.smithi138.stdout:6/766: getdents d3/d11/dc0/d7d/db9 0 2024-05-27T09:29:44.582 INFO:tasks.workunit.client.1.smithi138.stdout:0/571: sync 2024-05-27T09:29:44.582 INFO:tasks.workunit.client.1.smithi138.stdout:4/622: sync 2024-05-27T09:29:44.582 INFO:tasks.workunit.client.1.smithi138.stdout:0/572: chown da/d2a/d3f 211506743 1 2024-05-27T09:29:44.582 INFO:tasks.workunit.client.1.smithi138.stdout:4/623: readlink d7/df/d5b/lb3 0 2024-05-27T09:29:44.584 INFO:tasks.workunit.client.1.smithi138.stdout:8/676: sync 2024-05-27T09:29:44.584 INFO:tasks.workunit.client.1.smithi138.stdout:8/677: truncate d3/d8/d32/d45/d5c/fdb 853692 0 2024-05-27T09:29:44.584 INFO:tasks.workunit.client.1.smithi138.stdout:8/678: getdents d3/d8/d32/d45/dc2 0 2024-05-27T09:29:44.585 INFO:tasks.workunit.client.1.smithi138.stdout:7/676: rename d0/d23/fa0 to d0/d2b/daf/fdb 0 2024-05-27T09:29:44.585 INFO:tasks.workunit.client.1.smithi138.stdout:7/677: read d0/f19 [3349586,37249] 0 2024-05-27T09:29:44.585 INFO:tasks.workunit.client.1.smithi138.stdout:7/678: chown d0/d2b/daf/d60/d9d/lc6 1 1 2024-05-27T09:29:44.587 INFO:tasks.workunit.client.1.smithi138.stdout:7/679: getdents d0 0 2024-05-27T09:29:44.587 INFO:tasks.workunit.client.1.smithi138.stdout:7/680: stat d0/d2b/daf/dbf/dc8/ld8 0 2024-05-27T09:29:44.587 INFO:tasks.workunit.client.1.smithi138.stdout:7/681: fsync d0/d2b/daf/f39 0 2024-05-27T09:29:44.587 INFO:tasks.workunit.client.1.smithi138.stdout:7/682: readlink d0/d2b/daf/d57/d91/lb5 0 2024-05-27T09:29:44.588 INFO:tasks.workunit.client.1.smithi138.stdout:5/547: getdents d3/d12/d32/d7e 0 2024-05-27T09:29:44.588 INFO:tasks.workunit.client.1.smithi138.stdout:5/548: readlink d3/d12/d16/d1d/d23/l56 0 2024-05-27T09:29:44.588 INFO:tasks.workunit.client.1.smithi138.stdout:5/549: read d3/d12/d32/d60/f7b [495985,79801] 0 2024-05-27T09:29:44.591 INFO:tasks.workunit.client.1.smithi138.stdout:7/683: rename d0/d23/d3a/l90 to d0/d2b/daf/d98/db6/ldc 0 2024-05-27T09:29:44.591 INFO:tasks.workunit.client.1.smithi138.stdout:7/684: fsync d0/d23/d3c/d84/fd1 0 2024-05-27T09:29:44.591 INFO:tasks.workunit.client.1.smithi138.stdout:7/685: write d0/f77 [983708,65087] 0 2024-05-27T09:29:44.596 INFO:tasks.workunit.client.1.smithi138.stdout:6/767: dread d3/d19/f38 [0,4194304] 0 2024-05-27T09:29:44.612 INFO:tasks.workunit.client.1.smithi138.stdout:8/679: write d3/d8/d32/d3f/d9c/d9f/d86/f93 [1892206,55425] 0 2024-05-27T09:29:44.612 INFO:tasks.workunit.client.1.smithi138.stdout:8/680: write d3/d8/f42 [1174136,105634] 0 2024-05-27T09:29:44.612 INFO:tasks.workunit.client.1.smithi138.stdout:8/681: chown d3/d8/d64/f7c 48 1 2024-05-27T09:29:44.615 INFO:tasks.workunit.client.1.smithi138.stdout:8/682: symlink d3/d8/d32/d88/dab/dae/le8 0 2024-05-27T09:29:44.616 INFO:tasks.workunit.client.1.smithi138.stdout:5/550: write d3/d12/d16/d1d/d23/d68/fa1 [2718155,64219] 0 2024-05-27T09:29:44.618 INFO:tasks.workunit.client.1.smithi138.stdout:8/683: symlink d3/d8/d32/d34/d38/dd0/le9 0 2024-05-27T09:29:44.618 INFO:tasks.workunit.client.1.smithi138.stdout:8/684: fsync d3/d8/d11/fbb 0 2024-05-27T09:29:44.621 INFO:tasks.workunit.client.1.smithi138.stdout:8/685: getdents d3/d8/d24/d55/d7e 0 2024-05-27T09:29:44.621 INFO:tasks.workunit.client.1.smithi138.stdout:8/686: chown d3/d8/d32/d3f/d9c/d9f/c74 25456 1 2024-05-27T09:29:44.621 INFO:tasks.workunit.client.1.smithi138.stdout:8/687: stat d3/d8/d64/l2c 0 2024-05-27T09:29:44.628 INFO:tasks.workunit.client.1.smithi138.stdout:5/551: mkdir d3/dab 0 2024-05-27T09:29:44.634 INFO:tasks.workunit.client.1.smithi138.stdout:5/552: creat d3/dab/fac x:0 0 0 2024-05-27T09:29:44.645 INFO:tasks.workunit.client.1.smithi138.stdout:5/553: dread d3/d12/d16/d1d/d22/d2b/d36/f41 [0,4194304] 0 2024-05-27T09:29:44.645 INFO:tasks.workunit.client.1.smithi138.stdout:5/554: truncate d3/d12/d26/fa6 326668 0 2024-05-27T09:29:44.645 INFO:tasks.workunit.client.1.smithi138.stdout:5/555: write d3/d12/f79 [19624,969] 0 2024-05-27T09:29:44.646 INFO:tasks.workunit.client.1.smithi138.stdout:5/556: mknod d3/d12/cad 0 2024-05-27T09:29:44.653 INFO:tasks.workunit.client.1.smithi138.stdout:5/557: rename d3/d12/d16/d1d/d23/d3b/l3d to d3/d12/d16/d1d/d22/d2b/d9e/lae 0 2024-05-27T09:29:44.657 INFO:tasks.workunit.client.1.smithi138.stdout:5/558: mkdir d3/d12/d16/d1d/d22/d2b/daf 0 2024-05-27T09:29:44.658 INFO:tasks.workunit.client.1.smithi138.stdout:5/559: write d3/d12/d32/d87/fa4 [1594204,74048] 0 2024-05-27T09:29:44.660 INFO:tasks.workunit.client.1.smithi138.stdout:5/560: rename d3/d12/d16/d1d/d23/d68/la7 to d3/d12/d26/d53/lb0 0 2024-05-27T09:29:44.665 INFO:tasks.workunit.client.1.smithi138.stdout:5/561: symlink d3/d12/d32/d87/d89/lb1 0 2024-05-27T09:29:44.668 INFO:tasks.workunit.client.1.smithi138.stdout:4/624: dwrite d7/df/d35/d4c/d59/f73 [0,4194304] 0 2024-05-27T09:29:44.683 INFO:tasks.workunit.client.1.smithi138.stdout:7/686: dwrite d0/f36 [0,4194304] 0 2024-05-27T09:29:44.683 INFO:tasks.workunit.client.1.smithi138.stdout:7/687: chown d0/d2b/daf/d57/d91/lb5 64770131 1 2024-05-27T09:29:44.716 INFO:tasks.workunit.client.1.smithi138.stdout:6/768: dwrite d3/d19/fa7 [0,4194304] 0 2024-05-27T09:29:44.718 INFO:tasks.workunit.client.1.smithi138.stdout:8/688: dwrite d3/d8/d32/d88/f8e [4194304,4194304] 0 2024-05-27T09:29:44.722 INFO:tasks.workunit.client.1.smithi138.stdout:6/769: rename d3/d11/dc0/d43/dbe/dc3 to d3/d11/dc0/d43/d107 0 2024-05-27T09:29:44.725 INFO:tasks.workunit.client.1.smithi138.stdout:6/770: fdatasync d3/d19/f3d 0 2024-05-27T09:29:44.729 INFO:tasks.workunit.client.1.smithi138.stdout:6/771: symlink d3/d11/dc0/d43/d107/l108 0 2024-05-27T09:29:44.765 INFO:tasks.workunit.client.1.smithi138.stdout:7/688: dwrite d0/f8c [0,4194304] 0 2024-05-27T09:29:44.765 INFO:tasks.workunit.client.1.smithi138.stdout:7/689: stat d0/c15 0 2024-05-27T09:29:44.766 INFO:tasks.workunit.client.1.smithi138.stdout:5/562: dwrite d3/d12/d16/f2e [4194304,4194304] 0 2024-05-27T09:29:44.772 INFO:tasks.workunit.client.1.smithi138.stdout:7/690: creat d0/d2b/daf/d98/db6/fdd x:0 0 0 2024-05-27T09:29:44.772 INFO:tasks.workunit.client.1.smithi138.stdout:5/563: mkdir d3/d12/d16/d1d/d22/d2b/daf/db2 0 2024-05-27T09:29:44.774 INFO:tasks.workunit.client.1.smithi138.stdout:7/691: rmdir d0/d2b/daf 39 2024-05-27T09:29:44.777 INFO:tasks.workunit.client.1.smithi138.stdout:5/564: mkdir d3/d75/db3 0 2024-05-27T09:29:44.779 INFO:tasks.workunit.client.1.smithi138.stdout:5/565: symlink d3/d12/d32/d60/lb4 0 2024-05-27T09:29:44.783 INFO:tasks.workunit.client.1.smithi138.stdout:5/566: getdents d3/d12/d16/d1d/d23/d3b 0 2024-05-27T09:29:44.788 INFO:tasks.workunit.client.1.smithi138.stdout:7/692: dread d0/d23/f4f [0,4194304] 0 2024-05-27T09:29:44.789 INFO:tasks.workunit.client.1.smithi138.stdout:7/693: symlink d0/d2b/daf/d60/d9d/lde 0 2024-05-27T09:29:44.789 INFO:tasks.workunit.client.1.smithi138.stdout:7/694: write d0/d23/f4f [399453,76588] 0 2024-05-27T09:29:44.791 INFO:tasks.workunit.client.1.smithi138.stdout:5/567: dread d3/d12/d16/d1d/d23/f62 [0,4194304] 0 2024-05-27T09:29:44.793 INFO:tasks.workunit.client.1.smithi138.stdout:7/695: rename d0/d2b/daf/d57 to d0/d23/dc4/ddf 0 2024-05-27T09:29:44.793 INFO:tasks.workunit.client.1.smithi138.stdout:7/696: fsync d0/f45 0 2024-05-27T09:29:44.794 INFO:tasks.workunit.client.1.smithi138.stdout:8/689: dwrite d3/d8/d24/d55/d7e/d83/fb0 [0,4194304] 0 2024-05-27T09:29:44.795 INFO:tasks.workunit.client.1.smithi138.stdout:8/690: creat d3/d8/d64/fea x:0 0 0 2024-05-27T09:29:44.795 INFO:tasks.workunit.client.1.smithi138.stdout:7/697: rmdir d0/d23/dc4/ddf/d91/db8 39 2024-05-27T09:29:44.797 INFO:tasks.workunit.client.1.smithi138.stdout:6/772: dwrite d3/d11/dc0/d43/d8c/f96 [0,4194304] 0 2024-05-27T09:29:44.797 INFO:tasks.workunit.client.1.smithi138.stdout:6/773: chown d3/d11/dc0/fbd 110333086 1 2024-05-27T09:29:44.798 INFO:tasks.workunit.client.1.smithi138.stdout:3/581: sync 2024-05-27T09:29:44.798 INFO:tasks.workunit.client.1.smithi138.stdout:1/626: sync 2024-05-27T09:29:44.798 INFO:tasks.workunit.client.1.smithi138.stdout:3/582: write d0/d2/da/df/d10/f38 [884229,40990] 0 2024-05-27T09:29:44.799 INFO:tasks.workunit.client.1.smithi138.stdout:8/691: unlink d3/d8/d32/d34/d38/lb1 0 2024-05-27T09:29:44.799 INFO:tasks.workunit.client.1.smithi138.stdout:1/627: chown d4/c91 247 1 2024-05-27T09:29:44.799 INFO:tasks.workunit.client.1.smithi138.stdout:1/628: chown d4/d1a/d2e/d4a/l9d 10698 1 2024-05-27T09:29:44.799 INFO:tasks.workunit.client.1.smithi138.stdout:8/692: chown d3/d8/d32/d3f/d9c/d9f/d86/f93 17 1 2024-05-27T09:29:44.799 INFO:tasks.workunit.client.1.smithi138.stdout:5/568: dread d3/d12/d16/d1d/d22/d2b/d36/f41 [0,4194304] 0 2024-05-27T09:29:44.800 INFO:tasks.workunit.client.1.smithi138.stdout:7/698: mkdir d0/d23/d3a/de0 0 2024-05-27T09:29:44.800 INFO:tasks.workunit.client.1.smithi138.stdout:7/699: readlink d0/d23/dc4/ddf/d91/lb5 0 2024-05-27T09:29:44.800 INFO:tasks.workunit.client.1.smithi138.stdout:7/700: creat d0/d2b/fe1 x:0 0 0 2024-05-27T09:29:44.801 INFO:tasks.workunit.client.1.smithi138.stdout:3/583: mknod d0/d2/da/df/d10/d2a/d61/cbf 0 2024-05-27T09:29:44.801 INFO:tasks.workunit.client.1.smithi138.stdout:8/693: truncate d3/d8/d24/d4d/f61 2975771 0 2024-05-27T09:29:44.802 INFO:tasks.workunit.client.1.smithi138.stdout:7/701: symlink d0/d23/d3c/le2 0 2024-05-27T09:29:44.806 INFO:tasks.workunit.client.1.smithi138.stdout:3/584: read d0/d2/da/df/d10/d2a/d48/f79 [431635,32652] 0 2024-05-27T09:29:44.806 INFO:tasks.workunit.client.1.smithi138.stdout:7/702: getdents d0/d23/d49 0 2024-05-27T09:29:44.809 INFO:tasks.workunit.client.1.smithi138.stdout:7/703: unlink d0/d23/dc4/ddf/d91/lb5 0 2024-05-27T09:29:44.809 INFO:tasks.workunit.client.1.smithi138.stdout:3/585: mkdir d0/d3f/d60/daa/dc0 0 2024-05-27T09:29:44.811 INFO:tasks.workunit.client.1.smithi138.stdout:8/694: rename d3/d8/d6c to d3/d8/d11/deb 0 2024-05-27T09:29:44.811 INFO:tasks.workunit.client.1.smithi138.stdout:8/695: fsync d3/d8/d64/f2e 0 2024-05-27T09:29:44.811 INFO:tasks.workunit.client.1.smithi138.stdout:8/696: stat d3/d8/d11/deb/dd3 0 2024-05-27T09:29:44.812 INFO:tasks.workunit.client.1.smithi138.stdout:3/586: creat d0/d3f/d50/d9b/d9f/da3/fc1 x:0 0 0 2024-05-27T09:29:44.812 INFO:tasks.workunit.client.1.smithi138.stdout:8/697: mknod d3/d8/d24/d55/d7e/d83/cec 0 2024-05-27T09:29:44.814 INFO:tasks.workunit.client.1.smithi138.stdout:3/587: link d0/d2/da/df/d10/l1e d0/d2/da/df/d10/d32/d37/lc2 0 2024-05-27T09:29:44.814 INFO:tasks.workunit.client.1.smithi138.stdout:3/588: getdents d0/d2/da/df/d10/d2a/d61/d73 0 2024-05-27T09:29:44.815 INFO:tasks.workunit.client.1.smithi138.stdout:8/698: mknod d3/d8/d11/ced 0 2024-05-27T09:29:44.815 INFO:tasks.workunit.client.1.smithi138.stdout:8/699: creat d3/d8/d24/d4d/fee x:0 0 0 2024-05-27T09:29:44.824 INFO:tasks.workunit.client.1.smithi138.stdout:3/589: creat d0/d3f/d60/d93/db9/fc3 x:0 0 0 2024-05-27T09:29:44.851 INFO:tasks.workunit.client.1.smithi138.stdout:6/774: dwrite d3/d11/d2d/d37/dd1/d47/f7a [4194304,4194304] 0 2024-05-27T09:29:44.853 INFO:tasks.workunit.client.1.smithi138.stdout:6/775: creat d3/d11/dc0/d5a/d6a/f109 x:0 0 0 2024-05-27T09:29:44.854 INFO:tasks.workunit.client.1.smithi138.stdout:3/590: write d0/d2/da/f8a [1360140,124054] 0 2024-05-27T09:29:44.866 INFO:tasks.workunit.client.1.smithi138.stdout:3/591: rmdir d0/d3f/d50 39 2024-05-27T09:29:44.866 INFO:tasks.workunit.client.1.smithi138.stdout:7/704: dread d0/d2b/f40 [0,4194304] 0 2024-05-27T09:29:44.868 INFO:tasks.workunit.client.1.smithi138.stdout:9/655: sync 2024-05-27T09:29:44.868 INFO:tasks.workunit.client.1.smithi138.stdout:2/558: sync 2024-05-27T09:29:44.868 INFO:tasks.workunit.client.1.smithi138.stdout:2/559: chown d1/f7a 222899943 1 2024-05-27T09:29:44.868 INFO:tasks.workunit.client.1.smithi138.stdout:9/656: truncate d6/da/d2f/d35/d55/f76 305856 0 2024-05-27T09:29:44.869 INFO:tasks.workunit.client.1.smithi138.stdout:9/657: read - d6/fbc zero size 2024-05-27T09:29:44.869 INFO:tasks.workunit.client.1.smithi138.stdout:9/658: chown d6/da/d29/d8f/db7/fa1 7748838 1 2024-05-27T09:29:44.869 INFO:tasks.workunit.client.1.smithi138.stdout:9/659: dread - d6/da/d2f/f8e zero size 2024-05-27T09:29:44.869 INFO:tasks.workunit.client.1.smithi138.stdout:7/705: link d0/d2b/daf/fdb d0/d2b/daf/d64/fe3 0 2024-05-27T09:29:44.869 INFO:tasks.workunit.client.1.smithi138.stdout:3/592: write d0/d2/da/df/d10/d2a/f53 [297164,101507] 0 2024-05-27T09:29:44.869 INFO:tasks.workunit.client.1.smithi138.stdout:7/706: chown d0/fac 1984 1 2024-05-27T09:29:44.870 INFO:tasks.workunit.client.1.smithi138.stdout:9/660: creat d6/da/d29/d8f/da8/fdd x:0 0 0 2024-05-27T09:29:44.870 INFO:tasks.workunit.client.1.smithi138.stdout:2/560: write d1/d15/d2d/d17/d25/f3a [42149,89646] 0 2024-05-27T09:29:44.871 INFO:tasks.workunit.client.1.smithi138.stdout:1/629: dwrite d4/d48/d39/d5f/d93/fc9 [0,4194304] 0 2024-05-27T09:29:44.874 INFO:tasks.workunit.client.1.smithi138.stdout:3/593: link d0/d2/da/df/d10/d2a/f42 d0/fc4 0 2024-05-27T09:29:44.874 INFO:tasks.workunit.client.1.smithi138.stdout:7/707: mknod d0/d23/da8/d8e/ce4 0 2024-05-27T09:29:44.878 INFO:tasks.workunit.client.1.smithi138.stdout:3/594: symlink d0/d80/lc5 0 2024-05-27T09:29:44.881 INFO:tasks.workunit.client.1.smithi138.stdout:5/569: dwrite d3/f88 [0,4194304] 0 2024-05-27T09:29:44.881 INFO:tasks.workunit.client.1.smithi138.stdout:5/570: chown d3/fd 27310 1 2024-05-27T09:29:44.881 INFO:tasks.workunit.client.1.smithi138.stdout:7/708: truncate d0/d23/f4f 536152 0 2024-05-27T09:29:44.884 INFO:tasks.workunit.client.1.smithi138.stdout:9/661: rename d6/da/d29/d59/f6e to d6/db/de/d18/d24/d72/fde 0 2024-05-27T09:29:44.885 INFO:tasks.workunit.client.1.smithi138.stdout:3/595: mkdir d0/d3f/d50/d9b/d9f/da3/dc6 0 2024-05-27T09:29:44.886 INFO:tasks.workunit.client.1.smithi138.stdout:2/561: getdents d1/d15/d2d/d17/d25/d4d/d4e/d60 0 2024-05-27T09:29:44.886 INFO:tasks.workunit.client.1.smithi138.stdout:2/562: creat d1/d15/d2d/d17/d25/d4d/d6d/d87/fbf x:0 0 0 2024-05-27T09:29:44.886 INFO:tasks.workunit.client.1.smithi138.stdout:2/563: creat d1/d15/d2d/d17/d25/d4d/d6d/d87/fc0 x:0 0 0 2024-05-27T09:29:44.886 INFO:tasks.workunit.client.1.smithi138.stdout:2/564: chown d1/d15/d2d/f53 165 1 2024-05-27T09:29:44.886 INFO:tasks.workunit.client.1.smithi138.stdout:6/776: dread d3/d19/f83 [0,4194304] 0 2024-05-27T09:29:44.886 INFO:tasks.workunit.client.1.smithi138.stdout:6/777: readlink d3/d11/d2d/lfc 0 2024-05-27T09:29:44.886 INFO:tasks.workunit.client.1.smithi138.stdout:6/778: fsync d3/d11/d2d/f39 0 2024-05-27T09:29:44.886 INFO:tasks.workunit.client.1.smithi138.stdout:6/779: fdatasync d3/d91/fa0 0 2024-05-27T09:29:44.889 INFO:tasks.workunit.client.1.smithi138.stdout:6/780: unlink d3/d11/d2d/d37/d7e/fd6 0 2024-05-27T09:29:44.891 INFO:tasks.workunit.client.1.smithi138.stdout:8/700: dwrite d3/d8/f33 [4194304,4194304] 0 2024-05-27T09:29:44.892 INFO:tasks.workunit.client.1.smithi138.stdout:8/701: chown d3/d8/d24/d55/d94/da5/cc8 2 1 2024-05-27T09:29:44.899 INFO:tasks.workunit.client.1.smithi138.stdout:9/662: symlink d6/db/de/d6b/ldf 0 2024-05-27T09:29:44.917 INFO:tasks.workunit.client.1.smithi138.stdout:8/702: symlink d3/d8/d32/lef 0 2024-05-27T09:29:44.917 INFO:tasks.workunit.client.1.smithi138.stdout:7/709: mkdir d0/d23/de5 0 2024-05-27T09:29:44.917 INFO:tasks.workunit.client.1.smithi138.stdout:7/710: fdatasync d0/d2b/daf/d64/f7d 0 2024-05-27T09:29:44.918 INFO:tasks.workunit.client.1.smithi138.stdout:7/711: dread - d0/d23/da8/d8e/fd7 zero size 2024-05-27T09:29:44.918 INFO:tasks.workunit.client.1.smithi138.stdout:9/663: mknod d6/db/de/d18/ce0 0 2024-05-27T09:29:44.918 INFO:tasks.workunit.client.1.smithi138.stdout:2/565: rmdir d1/d15/d2d/d7b 39 2024-05-27T09:29:44.918 INFO:tasks.workunit.client.1.smithi138.stdout:2/566: chown d1/d15/d2d/d17/d25/d4d/d4e/d60/la7 87 1 2024-05-27T09:29:44.921 INFO:tasks.workunit.client.1.smithi138.stdout:8/703: rename d3/d8/d24/d55/d7e/d83/cec to d3/d8/d24/d55/cf0 0 2024-05-27T09:29:44.922 INFO:tasks.workunit.client.1.smithi138.stdout:7/712: mkdir d0/d23/d3c/d84/d9c/dc3/de6 0 2024-05-27T09:29:44.925 INFO:tasks.workunit.client.1.smithi138.stdout:9/664: link d6/db/de/d85/f94 d6/da/d29/d8f/db7/dbd/da2/fe1 0 2024-05-27T09:29:44.925 INFO:tasks.workunit.client.1.smithi138.stdout:9/665: write d6/db/de/f31 [312043,129994] 0 2024-05-27T09:29:44.929 INFO:tasks.workunit.client.1.smithi138.stdout:2/567: symlink d1/d15/d2d/d17/d25/d4d/d6d/d87/lc1 0 2024-05-27T09:29:44.931 INFO:tasks.workunit.client.1.smithi138.stdout:9/666: symlink d6/da/d26/d28/le2 0 2024-05-27T09:29:44.937 INFO:tasks.workunit.client.1.smithi138.stdout:8/704: rename d3/d8/d32/d88/dab to d3/d8/d11/df1 0 2024-05-27T09:29:44.937 INFO:tasks.workunit.client.1.smithi138.stdout:8/705: chown d3/d8/d24/ce5 1606 1 2024-05-27T09:29:44.937 INFO:tasks.workunit.client.1.smithi138.stdout:8/706: write d3/d8/fb6 [986256,116615] 0 2024-05-27T09:29:44.939 INFO:tasks.workunit.client.1.smithi138.stdout:5/571: dwrite d3/fd [8388608,4194304] 0 2024-05-27T09:29:44.939 INFO:tasks.workunit.client.1.smithi138.stdout:5/572: fsync d3/d12/d32/d60/f7b 0 2024-05-27T09:29:44.943 INFO:tasks.workunit.client.1.smithi138.stdout:4/625: sync 2024-05-27T09:29:44.943 INFO:tasks.workunit.client.1.smithi138.stdout:4/626: fsync d7/df/d35/f8b 0 2024-05-27T09:29:44.944 INFO:tasks.workunit.client.1.smithi138.stdout:0/573: sync 2024-05-27T09:29:44.944 INFO:tasks.workunit.client.1.smithi138.stdout:0/574: write da/d2a/d8e/d1f/d42/d4a/d53/f72 [1456327,9067] 0 2024-05-27T09:29:44.944 INFO:tasks.workunit.client.1.smithi138.stdout:0/575: write da/d2a/d8e/d1f/fa2 [490069,33136] 0 2024-05-27T09:29:44.944 INFO:tasks.workunit.client.1.smithi138.stdout:0/576: fsync da/d2a/d8e/d1f/f7b 0 2024-05-27T09:29:44.944 INFO:tasks.workunit.client.1.smithi138.stdout:9/667: rename d6/da/d29/d59/f70 to d6/da/d29/fe3 0 2024-05-27T09:29:44.944 INFO:tasks.workunit.client.1.smithi138.stdout:9/668: fdatasync d6/da/fa9 0 2024-05-27T09:29:44.944 INFO:tasks.workunit.client.1.smithi138.stdout:9/669: readlink d6/da/d29/d8f/da8/lab 0 2024-05-27T09:29:44.945 INFO:tasks.workunit.client.1.smithi138.stdout:9/670: read - d6/db/d8d/fdc zero size 2024-05-27T09:29:44.946 INFO:tasks.workunit.client.1.smithi138.stdout:9/671: creat d6/db/de/d18/ddb/fe4 x:0 0 0 2024-05-27T09:29:44.954 INFO:tasks.workunit.client.1.smithi138.stdout:1/630: dwrite d4/f18 [0,4194304] 0 2024-05-27T09:29:44.966 INFO:tasks.workunit.client.1.smithi138.stdout:6/781: dwrite d3/d11/d2d/d37/dd1/d47/f56 [0,4194304] 0 2024-05-27T09:29:44.967 INFO:tasks.workunit.client.1.smithi138.stdout:1/631: creat d4/d1a/d2e/d4a/fca x:0 0 0 2024-05-27T09:29:44.967 INFO:tasks.workunit.client.1.smithi138.stdout:1/632: readlink d4/d48/d39/l30 0 2024-05-27T09:29:44.967 INFO:tasks.workunit.client.1.smithi138.stdout:1/633: creat d4/d48/d39/d5f/d4d/fcb x:0 0 0 2024-05-27T09:29:44.970 INFO:tasks.workunit.client.1.smithi138.stdout:4/627: dread d7/df/d35/d4c/d59/d5d/d64/f96 [0,4194304] 0 2024-05-27T09:29:44.970 INFO:tasks.workunit.client.1.smithi138.stdout:4/628: chown f3 31252 1 2024-05-27T09:29:44.976 INFO:tasks.workunit.client.1.smithi138.stdout:8/707: dread d3/d8/d24/f44 [0,4194304] 0 2024-05-27T09:29:44.981 INFO:tasks.workunit.client.1.smithi138.stdout:3/596: dwrite d0/d2/da/df/d10/d32/f9 [8388608,4194304] 0 2024-05-27T09:29:44.988 INFO:tasks.workunit.client.1.smithi138.stdout:3/597: link d0/fa4 d0/d3f/d60/daa/dc0/fc7 0 2024-05-27T09:29:44.988 INFO:tasks.workunit.client.1.smithi138.stdout:8/708: dread d3/d8/d32/d45/d5c/fdb [0,4194304] 0 2024-05-27T09:29:44.990 INFO:tasks.workunit.client.1.smithi138.stdout:4/629: getdents d7 0 2024-05-27T09:29:44.999 INFO:tasks.workunit.client.1.smithi138.stdout:5/573: write d3/d12/d32/d60/f8e [3200126,121799] 0 2024-05-27T09:29:45.003 INFO:tasks.workunit.client.1.smithi138.stdout:2/568: dwrite d1/d15/d2d/f9f [0,4194304] 0 2024-05-27T09:29:45.005 INFO:tasks.workunit.client.1.smithi138.stdout:2/569: symlink d1/d10/d52/db9/lc2 0 2024-05-27T09:29:45.008 INFO:tasks.workunit.client.1.smithi138.stdout:2/570: unlink d1/d15/d2d/d17/d25/d9d/d4b/f51 0 2024-05-27T09:29:45.009 INFO:tasks.workunit.client.1.smithi138.stdout:2/571: symlink d1/d15/d2d/d17/d25/d4d/d4e/d60/lc3 0 2024-05-27T09:29:45.011 INFO:tasks.workunit.client.1.smithi138.stdout:2/572: symlink d1/d15/d2d/d17/d25/d9d/d4b/lc4 0 2024-05-27T09:29:45.011 INFO:tasks.workunit.client.1.smithi138.stdout:2/573: truncate d1/d2f/f83 253567 0 2024-05-27T09:29:45.014 INFO:tasks.workunit.client.1.smithi138.stdout:9/672: dwrite d6/db/de/d18/f64 [0,4194304] 0 2024-05-27T09:29:45.015 INFO:tasks.workunit.client.1.smithi138.stdout:4/630: rename d7/df/d35/f6c to d7/fdd 0 2024-05-27T09:29:45.015 INFO:tasks.workunit.client.1.smithi138.stdout:0/577: dwrite da/d2a/d2b/f33 [0,4194304] 0 2024-05-27T09:29:45.015 INFO:tasks.workunit.client.1.smithi138.stdout:0/578: write f5 [1182825,119118] 0 2024-05-27T09:29:45.017 INFO:tasks.workunit.client.1.smithi138.stdout:5/574: link d3/f4c d3/d12/d16/d1d/d22/d2b/d99/fb5 0 2024-05-27T09:29:45.024 INFO:tasks.workunit.client.1.smithi138.stdout:0/579: mkdir da/d2a/d8e/d1f/d60/dbd 0 2024-05-27T09:29:45.026 INFO:tasks.workunit.client.1.smithi138.stdout:4/631: rmdir d7/df/d35/d4c/d59 39 2024-05-27T09:29:45.026 INFO:tasks.workunit.client.1.smithi138.stdout:4/632: creat d7/df/d35/d4c/dc0/fde x:0 0 0 2024-05-27T09:29:45.028 INFO:tasks.workunit.client.1.smithi138.stdout:8/709: dwrite d3/d8/d32/d3f/d9c/d9f/fa7 [0,4194304] 0 2024-05-27T09:29:45.028 INFO:tasks.workunit.client.1.smithi138.stdout:4/633: readlink d7/dba/dcf/d3a/lbb 0 2024-05-27T09:29:45.030 INFO:tasks.workunit.client.1.smithi138.stdout:3/598: dwrite d0/d2/da/df/d10/f38 [0,4194304] 0 2024-05-27T09:29:45.033 INFO:tasks.workunit.client.1.smithi138.stdout:8/710: creat d3/d8/d32/d45/ff2 x:0 0 0 2024-05-27T09:29:45.033 INFO:tasks.workunit.client.1.smithi138.stdout:4/634: truncate d7/d6e/fd1 949088 0 2024-05-27T09:29:45.034 INFO:tasks.workunit.client.1.smithi138.stdout:8/711: mknod d3/d8/d32/d88/cf3 0 2024-05-27T09:29:45.035 INFO:tasks.workunit.client.1.smithi138.stdout:8/712: truncate d3/d8/d32/d45/ff2 1015726 0 2024-05-27T09:29:45.036 INFO:tasks.workunit.client.1.smithi138.stdout:8/713: creat d3/d8/d11/deb/dd3/ff4 x:0 0 0 2024-05-27T09:29:45.039 INFO:tasks.workunit.client.1.smithi138.stdout:8/714: symlink d3/d8/d32/lf5 0 2024-05-27T09:29:45.039 INFO:tasks.workunit.client.1.smithi138.stdout:4/635: truncate d7/fdc 480167 0 2024-05-27T09:29:45.041 INFO:tasks.workunit.client.1.smithi138.stdout:8/715: creat d3/d8/d32/d3f/d9c/ff6 x:0 0 0 2024-05-27T09:29:45.042 INFO:tasks.workunit.client.1.smithi138.stdout:8/716: mknod d3/d8/d64/cf7 0 2024-05-27T09:29:45.043 INFO:tasks.workunit.client.1.smithi138.stdout:8/717: dread - d3/d8/d11/deb/dd3/fd5 zero size 2024-05-27T09:29:45.043 INFO:tasks.workunit.client.1.smithi138.stdout:4/636: symlink d7/df/ldf 0 2024-05-27T09:29:45.044 INFO:tasks.workunit.client.1.smithi138.stdout:8/718: mkdir d3/d8/d32/df8 0 2024-05-27T09:29:45.045 INFO:tasks.workunit.client.1.smithi138.stdout:5/575: rename d3/d12/d16/d1d/d22/d2b/d36 to d3/d12/d16/d1d/d22/d2b/d9e/db6 0 2024-05-27T09:29:45.045 INFO:tasks.workunit.client.1.smithi138.stdout:8/719: symlink d3/d8/d32/d45/dc2/lf9 0 2024-05-27T09:29:45.046 INFO:tasks.workunit.client.1.smithi138.stdout:4/637: rmdir d7/dba/dcf/daa 39 2024-05-27T09:29:45.048 INFO:tasks.workunit.client.1.smithi138.stdout:0/580: rename da/d2a/d8e/d1f/d4f/l5b to da/d2a/d8e/d1f/d71/lbe 0 2024-05-27T09:29:45.049 INFO:tasks.workunit.client.1.smithi138.stdout:5/576: mkdir d3/d75/db3/db7 0 2024-05-27T09:29:45.050 INFO:tasks.workunit.client.1.smithi138.stdout:3/599: rename d0/d2/da/df/d10/d32/f71 to d0/d2/fc8 0 2024-05-27T09:29:45.051 INFO:tasks.workunit.client.1.smithi138.stdout:3/600: creat d0/d3f/d50/d9b/d9f/fc9 x:0 0 0 2024-05-27T09:29:45.052 INFO:tasks.workunit.client.1.smithi138.stdout:5/577: symlink d3/d12/d26/lb8 0 2024-05-27T09:29:45.052 INFO:tasks.workunit.client.1.smithi138.stdout:3/601: mkdir d0/d3f/d60/d93/db9/dca 0 2024-05-27T09:29:45.054 INFO:tasks.workunit.client.1.smithi138.stdout:5/578: truncate d3/f19 2411686 0 2024-05-27T09:29:45.057 INFO:tasks.workunit.client.1.smithi138.stdout:5/579: rename d3/d12/d16/d1d/d22/d2b/d99 to d3/d12/d16/d1d/d22/db9 0 2024-05-27T09:29:45.062 INFO:tasks.workunit.client.1.smithi138.stdout:5/580: creat d3/d12/d16/d1d/d23/fba x:0 0 0 2024-05-27T09:29:45.062 INFO:tasks.workunit.client.1.smithi138.stdout:5/581: dread - d3/f4c zero size 2024-05-27T09:29:45.063 INFO:tasks.workunit.client.1.smithi138.stdout:4/638: read d7/dba/dcf/d3a/f6b [12528421,101212] 0 2024-05-27T09:29:45.065 INFO:tasks.workunit.client.1.smithi138.stdout:4/639: creat d7/dba/dcf/daa/fe0 x:0 0 0 2024-05-27T09:29:45.069 INFO:tasks.workunit.client.1.smithi138.stdout:8/720: write d3/d8/d32/d88/f8e [5899955,113499] 0 2024-05-27T09:29:45.070 INFO:tasks.workunit.client.1.smithi138.stdout:4/640: write d7/df/d35/f92 [355559,111320] 0 2024-05-27T09:29:45.070 INFO:tasks.workunit.client.1.smithi138.stdout:3/602: read d0/d2/f4b [288725,103958] 0 2024-05-27T09:29:45.070 INFO:tasks.workunit.client.1.smithi138.stdout:5/582: dread d3/d12/f79 [0,4194304] 0 2024-05-27T09:29:45.071 INFO:tasks.workunit.client.1.smithi138.stdout:3/603: mkdir d0/d3f/d60/dcb 0 2024-05-27T09:29:45.071 INFO:tasks.workunit.client.1.smithi138.stdout:5/583: truncate d3/d12/d16/d1d/d22/d2b/f38 995158 0 2024-05-27T09:29:45.073 INFO:tasks.workunit.client.1.smithi138.stdout:3/604: getdents d0 0 2024-05-27T09:29:45.075 INFO:tasks.workunit.client.1.smithi138.stdout:5/584: mknod d3/d75/cbb 0 2024-05-27T09:29:45.075 INFO:tasks.workunit.client.1.smithi138.stdout:3/605: symlink d0/d3f/d50/d9b/d9f/da3/lcc 0 2024-05-27T09:29:45.077 INFO:tasks.workunit.client.1.smithi138.stdout:5/585: symlink d3/d12/d16/d1d/d22/d2b/daf/lbc 0 2024-05-27T09:29:45.077 INFO:tasks.workunit.client.1.smithi138.stdout:3/606: creat d0/d3f/d50/d9b/d9f/da3/dc6/fcd x:0 0 0 2024-05-27T09:29:45.080 INFO:tasks.workunit.client.1.smithi138.stdout:3/607: creat d0/d3f/d60/d93/db9/dca/fce x:0 0 0 2024-05-27T09:29:45.080 INFO:tasks.workunit.client.1.smithi138.stdout:3/608: fsync d0/d3f/d50/d9b/d9f/da3/fc1 0 2024-05-27T09:29:45.080 INFO:tasks.workunit.client.1.smithi138.stdout:3/609: read d0/d2/da/df/d10/d2a/f53 [88204,123899] 0 2024-05-27T09:29:45.080 INFO:tasks.workunit.client.1.smithi138.stdout:3/610: readlink d0/d3f/d60/la7 0 2024-05-27T09:29:45.080 INFO:tasks.workunit.client.1.smithi138.stdout:3/611: chown d0/d2/l57 280851 1 2024-05-27T09:29:45.080 INFO:tasks.workunit.client.1.smithi138.stdout:0/581: dread da/d2a/d8e/d43/f48 [0,4194304] 0 2024-05-27T09:29:45.082 INFO:tasks.workunit.client.1.smithi138.stdout:5/586: unlink d3/d12/d32/d60/f61 0 2024-05-27T09:29:45.089 INFO:tasks.workunit.client.1.smithi138.stdout:8/721: dread d3/d8/d24/f54 [0,4194304] 0 2024-05-27T09:29:45.094 INFO:tasks.workunit.client.1.smithi138.stdout:3/612: rmdir d0/d3f/d50/d9b/d9f 39 2024-05-27T09:29:45.095 INFO:tasks.workunit.client.1.smithi138.stdout:5/587: dread d3/d12/f64 [0,4194304] 0 2024-05-27T09:29:45.100 INFO:tasks.workunit.client.1.smithi138.stdout:5/588: symlink d3/d12/d16/d1d/d30/lbd 0 2024-05-27T09:29:45.100 INFO:tasks.workunit.client.1.smithi138.stdout:5/589: fsync d3/d12/f79 0 2024-05-27T09:29:45.105 INFO:tasks.workunit.client.1.smithi138.stdout:3/613: dread d0/d3f/f84 [0,4194304] 0 2024-05-27T09:29:45.105 INFO:tasks.workunit.client.1.smithi138.stdout:5/590: creat d3/d12/d16/d1d/d22/d2b/d9e/fbe x:0 0 0 2024-05-27T09:29:45.105 INFO:tasks.workunit.client.1.smithi138.stdout:3/614: chown d0/d2/c20 11 1 2024-05-27T09:29:45.105 INFO:tasks.workunit.client.1.smithi138.stdout:3/615: chown d0/l26 257565072 1 2024-05-27T09:29:45.105 INFO:tasks.workunit.client.1.smithi138.stdout:3/616: creat d0/d2/da/fcf x:0 0 0 2024-05-27T09:29:45.105 INFO:tasks.workunit.client.1.smithi138.stdout:0/582: truncate da/d2a/d8e/d1f/f3b 4079878 0 2024-05-27T09:29:45.106 INFO:tasks.workunit.client.1.smithi138.stdout:0/583: creat da/d2a/d8e/d1f/d42/d4a/d53/fbf x:0 0 0 2024-05-27T09:29:45.108 INFO:tasks.workunit.client.1.smithi138.stdout:3/617: mknod d0/d2/da/df/d10/d32/d37/cd0 0 2024-05-27T09:29:45.108 INFO:tasks.workunit.client.1.smithi138.stdout:3/618: write d0/d2/da/df/d10/d2a/f42 [1353297,89221] 0 2024-05-27T09:29:45.110 INFO:tasks.workunit.client.1.smithi138.stdout:3/619: rmdir d0/d2 39 2024-05-27T09:29:45.110 INFO:tasks.workunit.client.1.smithi138.stdout:3/620: fdatasync d0/d2/da/f6b 0 2024-05-27T09:29:45.117 INFO:tasks.workunit.client.1.smithi138.stdout:3/621: rename d0/d3f/d60/daa to d0/d2/da/df/d10/d2a/db2/dd1 0 2024-05-27T09:29:45.118 INFO:tasks.workunit.client.1.smithi138.stdout:3/622: creat d0/d80/fd2 x:0 0 0 2024-05-27T09:29:45.118 INFO:tasks.workunit.client.1.smithi138.stdout:3/623: creat d0/d3f/d60/d93/db9/fd3 x:0 0 0 2024-05-27T09:29:45.118 INFO:tasks.workunit.client.1.smithi138.stdout:3/624: truncate d0/d2/da/df/d10/f9d 1177293 0 2024-05-27T09:29:45.125 INFO:tasks.workunit.client.1.smithi138.stdout:3/625: unlink d0/d3f/d50/la9 0 2024-05-27T09:29:45.131 INFO:tasks.workunit.client.1.smithi138.stdout:3/626: symlink d0/d3f/d60/d93/db9/ld4 0 2024-05-27T09:29:45.163 INFO:tasks.workunit.client.1.smithi138.stdout:4/641: dwrite d7/df/d35/d4c/d59/d5d/d64/d6a/f6d [0,4194304] 0 2024-05-27T09:29:45.171 INFO:tasks.workunit.client.1.smithi138.stdout:8/722: dwrite d3/d8/d24/d55/f67 [0,4194304] 0 2024-05-27T09:29:45.174 INFO:tasks.workunit.client.1.smithi138.stdout:8/723: creat d3/d8/d32/d3f/d9c/db8/ffa x:0 0 0 2024-05-27T09:29:45.174 INFO:tasks.workunit.client.1.smithi138.stdout:8/724: creat d3/d8/d32/d45/d5c/ffb x:0 0 0 2024-05-27T09:29:45.174 INFO:tasks.workunit.client.1.smithi138.stdout:8/725: write d3/d8/d32/d34/d38/f92 [591498,47457] 0 2024-05-27T09:29:45.181 INFO:tasks.workunit.client.1.smithi138.stdout:3/627: dwrite d0/fa4 [0,4194304] 0 2024-05-27T09:29:45.182 INFO:tasks.workunit.client.1.smithi138.stdout:8/726: symlink d3/d8/d32/d3f/lfc 0 2024-05-27T09:29:45.190 INFO:tasks.workunit.client.1.smithi138.stdout:0/584: dwrite da/d2a/d8e/d1f/d42/d63/d9d/fb6 [0,4194304] 0 2024-05-27T09:29:45.190 INFO:tasks.workunit.client.1.smithi138.stdout:0/585: chown da/f3c 219990 1 2024-05-27T09:29:45.190 INFO:tasks.workunit.client.1.smithi138.stdout:0/586: fdatasync da/d2a/d3f/f4c 0 2024-05-27T09:29:45.198 INFO:tasks.workunit.client.1.smithi138.stdout:3/628: dread d0/d2/da/df/d10/d32/f1d [0,4194304] 0 2024-05-27T09:29:45.198 INFO:tasks.workunit.client.1.smithi138.stdout:8/727: dread d3/d8/d64/f2e [0,4194304] 0 2024-05-27T09:29:45.199 INFO:tasks.workunit.client.1.smithi138.stdout:5/591: dwrite d3/d12/d16/d1d/d23/d3b/fa8 [0,4194304] 0 2024-05-27T09:29:45.200 INFO:tasks.workunit.client.1.smithi138.stdout:3/629: mknod d0/d2/da/df/d10/d2a/d61/d73/cd5 0 2024-05-27T09:29:45.204 INFO:tasks.workunit.client.1.smithi138.stdout:3/630: symlink d0/d2/ld6 0 2024-05-27T09:29:45.218 INFO:tasks.workunit.client.1.smithi138.stdout:4/642: dwrite d7/df/d35/d4c/d59/d5d/d64/d6a/f94 [4194304,4194304] 0 2024-05-27T09:29:45.218 INFO:tasks.workunit.client.1.smithi138.stdout:4/643: dread - d7/df/d5b/f7f zero size 2024-05-27T09:29:45.228 INFO:tasks.workunit.client.1.smithi138.stdout:8/728: rmdir d3/d8 39 2024-05-27T09:29:45.230 INFO:tasks.workunit.client.1.smithi138.stdout:3/631: getdents d0/d3f/d50/d9b/d9f/da3/dc6 0 2024-05-27T09:29:45.231 INFO:tasks.workunit.client.1.smithi138.stdout:8/729: mknod d3/d8/d32/d34/dbd/cfd 0 2024-05-27T09:29:45.233 INFO:tasks.workunit.client.1.smithi138.stdout:3/632: link d0/d2/da/df/d10/d32/d37/lc2 d0/d3f/d60/ld7 0 2024-05-27T09:29:45.234 INFO:tasks.workunit.client.1.smithi138.stdout:8/730: symlink d3/d8/d32/d3f/d9c/lfe 0 2024-05-27T09:29:45.245 INFO:tasks.workunit.client.1.smithi138.stdout:8/731: write d3/d8/d24/d4d/f61 [1532382,47515] 0 2024-05-27T09:29:45.246 INFO:tasks.workunit.client.1.smithi138.stdout:8/732: truncate d3/db5/fd9 873012 0 2024-05-27T09:29:45.246 INFO:tasks.workunit.client.1.smithi138.stdout:8/733: chown d3/d8/d24/d4d/cd8 61051720 1 2024-05-27T09:29:45.246 INFO:tasks.workunit.client.1.smithi138.stdout:8/734: readlink d3/d8/d32/d34/laa 0 2024-05-27T09:29:45.251 INFO:tasks.workunit.client.1.smithi138.stdout:5/592: dwrite d3/d12/d16/d1d/f24 [0,4194304] 0 2024-05-27T09:29:45.251 INFO:tasks.workunit.client.1.smithi138.stdout:5/593: stat d3/d12/d32/f4f 0 2024-05-27T09:29:45.267 INFO:tasks.workunit.client.1.smithi138.stdout:5/594: dread d3/d12/d16/d1d/d23/d3b/fa2 [0,4194304] 0 2024-05-27T09:29:45.267 INFO:tasks.workunit.client.1.smithi138.stdout:5/595: dread - d3/d12/d16/d1d/d23/f67 zero size 2024-05-27T09:29:45.267 INFO:tasks.workunit.client.1.smithi138.stdout:8/735: dread d3/d8/d32/d3f/d9c/d9f/d86/fc6 [0,4194304] 0 2024-05-27T09:29:45.267 INFO:tasks.workunit.client.1.smithi138.stdout:8/736: truncate d3/f50 968855 0 2024-05-27T09:29:45.270 INFO:tasks.workunit.client.1.smithi138.stdout:8/737: symlink d3/d8/d32/d45/lff 0 2024-05-27T09:29:45.273 INFO:tasks.workunit.client.1.smithi138.stdout:3/633: dwrite d0/d2/da/df/d10/d32/f5b [0,4194304] 0 2024-05-27T09:29:45.273 INFO:tasks.workunit.client.1.smithi138.stdout:8/738: creat d3/d8/d32/d3f/d9c/d9f/f100 x:0 0 0 2024-05-27T09:29:45.275 INFO:tasks.workunit.client.1.smithi138.stdout:3/634: mkdir d0/d2/da/df/d10/d2a/db2/dd1/dd8 0 2024-05-27T09:29:45.277 INFO:tasks.workunit.client.1.smithi138.stdout:8/739: symlink d3/d8/d24/d55/l101 0 2024-05-27T09:29:45.277 INFO:tasks.workunit.client.1.smithi138.stdout:8/740: chown d3/d8/f42 3506 1 2024-05-27T09:29:45.279 INFO:tasks.workunit.client.1.smithi138.stdout:3/635: mknod d0/d3f/d50/d9b/d9f/cd9 0 2024-05-27T09:29:45.279 INFO:tasks.workunit.client.1.smithi138.stdout:3/636: stat d0/d80/fd2 0 2024-05-27T09:29:45.284 INFO:tasks.workunit.client.1.smithi138.stdout:8/741: unlink d3/d8/c91 0 2024-05-27T09:29:45.286 INFO:tasks.workunit.client.1.smithi138.stdout:0/587: unlink da/d2a/d8e/d1f/l5d 0 2024-05-27T09:29:45.286 INFO:tasks.workunit.client.1.smithi138.stdout:0/588: chown da/d2a/d2b/f33 1079214830 1 2024-05-27T09:29:45.288 INFO:tasks.workunit.client.1.smithi138.stdout:4/644: readlink d7/lb8 0 2024-05-27T09:29:45.291 INFO:tasks.workunit.client.1.smithi138.stdout:5/596: symlink d3/d12/d16/d1d/d22/d2b/daf/db2/lbf 0 2024-05-27T09:29:45.291 INFO:tasks.workunit.client.1.smithi138.stdout:5/597: truncate d3/d12/d26/f57 842461 0 2024-05-27T09:29:45.292 INFO:tasks.workunit.client.1.smithi138.stdout:0/589: unlink da/d2a/d8e/d1f/d42/c5e 0 2024-05-27T09:29:45.292 INFO:tasks.workunit.client.1.smithi138.stdout:4/645: chown d7/f39 3 1 2024-05-27T09:29:45.294 INFO:tasks.workunit.client.1.smithi138.stdout:5/598: mkdir d3/d12/d32/d87/dc0 0 2024-05-27T09:29:45.294 INFO:tasks.workunit.client.1.smithi138.stdout:5/599: fdatasync d3/d12/d26/d53/f85 0 2024-05-27T09:29:45.294 INFO:tasks.workunit.client.1.smithi138.stdout:0/590: mkdir da/d2a/d8e/dc0 0 2024-05-27T09:29:45.295 INFO:tasks.workunit.client.1.smithi138.stdout:4/646: chown d7/c88 557900922 1 2024-05-27T09:29:45.297 INFO:tasks.workunit.client.1.smithi138.stdout:5/600: creat d3/d12/d16/d1d/d22/d2b/fc1 x:0 0 0 2024-05-27T09:29:45.298 INFO:tasks.workunit.client.1.smithi138.stdout:5/601: read d3/d12/d26/d53/f85 [91463,81293] 0 2024-05-27T09:29:45.298 INFO:tasks.workunit.client.1.smithi138.stdout:5/602: readlink d3/d12/l29 0 2024-05-27T09:29:45.298 INFO:tasks.workunit.client.1.smithi138.stdout:5/603: chown d3/d12/d32/d87/f8f 46 1 2024-05-27T09:29:45.298 INFO:tasks.workunit.client.1.smithi138.stdout:5/604: dread - d3/d12/d16/d1d/d22/d2b/fc1 zero size 2024-05-27T09:29:45.298 INFO:tasks.workunit.client.1.smithi138.stdout:5/605: dread - d3/d12/d16/d1d/f73 zero size 2024-05-27T09:29:45.298 INFO:tasks.workunit.client.1.smithi138.stdout:0/591: link da/d2a/d8e/d1f/d42/d4a/l8b da/d2a/d8e/d1f/d4f/lc1 0 2024-05-27T09:29:45.298 INFO:tasks.workunit.client.1.smithi138.stdout:0/592: dread - da/d2a/d8e/d43/d74/d97/f9e zero size 2024-05-27T09:29:45.299 INFO:tasks.workunit.client.1.smithi138.stdout:0/593: creat da/d2a/d8e/d43/d74/d97/fc2 x:0 0 0 2024-05-27T09:29:45.299 INFO:tasks.workunit.client.1.smithi138.stdout:0/594: creat da/d2a/d8e/d6a/d99/fc3 x:0 0 0 2024-05-27T09:29:45.299 INFO:tasks.workunit.client.1.smithi138.stdout:0/595: readlink da/d2a/d2b/l46 0 2024-05-27T09:29:45.299 INFO:tasks.workunit.client.1.smithi138.stdout:5/606: mkdir d3/dab/dc2 0 2024-05-27T09:29:45.300 INFO:tasks.workunit.client.1.smithi138.stdout:5/607: write d3/f1b [639942,11307] 0 2024-05-27T09:29:45.301 INFO:tasks.workunit.client.1.smithi138.stdout:8/742: read d3/d8/d32/d45/f90 [3862455,44518] 0 2024-05-27T09:29:45.301 INFO:tasks.workunit.client.1.smithi138.stdout:8/743: stat d3/d8/d64/f76 0 2024-05-27T09:29:45.301 INFO:tasks.workunit.client.1.smithi138.stdout:8/744: chown d3/d8/d32/d34/f9a 14 1 2024-05-27T09:29:45.304 INFO:tasks.workunit.client.1.smithi138.stdout:8/745: creat d3/d8/d32/d3f/f102 x:0 0 0 2024-05-27T09:29:45.304 INFO:tasks.workunit.client.1.smithi138.stdout:8/746: chown d3/d8/d24/d55/d7e/ld2 125714 1 2024-05-27T09:29:45.304 INFO:tasks.workunit.client.1.smithi138.stdout:4/647: link d7/df/d35/d4c/d59/d5d/d64/d6a/c8f d7/dba/dcf/ce1 0 2024-05-27T09:29:45.304 INFO:tasks.workunit.client.1.smithi138.stdout:8/747: write d3/d8/d11/fa6 [980447,66323] 0 2024-05-27T09:29:45.304 INFO:tasks.workunit.client.1.smithi138.stdout:4/648: stat d7/df/d35/d4c/d59/d5d/fd4 0 2024-05-27T09:29:45.304 INFO:tasks.workunit.client.1.smithi138.stdout:8/748: read d3/d8/d64/f7c [2727631,33794] 0 2024-05-27T09:29:45.305 INFO:tasks.workunit.client.1.smithi138.stdout:0/596: mkdir da/d2a/d8e/d6a/daf/dc4 0 2024-05-27T09:29:45.305 INFO:tasks.workunit.client.1.smithi138.stdout:0/597: read da/d2a/f8f [166508,94193] 0 2024-05-27T09:29:45.306 INFO:tasks.workunit.client.1.smithi138.stdout:0/598: dread - da/d2a/d8e/d1f/d42/d63/f91 zero size 2024-05-27T09:29:45.308 INFO:tasks.workunit.client.1.smithi138.stdout:5/608: rmdir d3/d12/d32/d6b 39 2024-05-27T09:29:45.308 INFO:tasks.workunit.client.1.smithi138.stdout:5/609: chown d3/d12/d32/c6c 53 1 2024-05-27T09:29:45.311 INFO:tasks.workunit.client.1.smithi138.stdout:4/649: symlink d7/dba/dcf/daa/dda/le2 0 2024-05-27T09:29:45.311 INFO:tasks.workunit.client.1.smithi138.stdout:4/650: write d7/d6e/fd1 [1869442,42993] 0 2024-05-27T09:29:45.316 INFO:tasks.workunit.client.1.smithi138.stdout:0/599: unlink da/f15 0 2024-05-27T09:29:45.322 INFO:tasks.workunit.client.1.smithi138.stdout:5/610: mkdir d3/d12/d32/d87/d89/dc3 0 2024-05-27T09:29:45.322 INFO:tasks.workunit.client.1.smithi138.stdout:5/611: write d3/d12/d16/d1d/d22/d2b/d9e/db6/f43 [1117712,15152] 0 2024-05-27T09:29:45.324 INFO:tasks.workunit.client.1.smithi138.stdout:4/651: creat d7/df/d5b/da1/dbc/fe3 x:0 0 0 2024-05-27T09:29:45.324 INFO:tasks.workunit.client.1.smithi138.stdout:4/652: dread - d7/df/d35/d4c/d59/d5d/d64/fa7 zero size 2024-05-27T09:29:45.324 INFO:tasks.workunit.client.1.smithi138.stdout:4/653: write d7/df/d35/d4c/d59/d5d/d64/fa7 [115147,14614] 0 2024-05-27T09:29:45.326 INFO:tasks.workunit.client.1.smithi138.stdout:5/612: creat d3/d12/d16/d1d/d22/d2b/daf/db2/fc4 x:0 0 0 2024-05-27T09:29:45.333 INFO:tasks.workunit.client.1.smithi138.stdout:5/613: mknod d3/d12/d32/cc5 0 2024-05-27T09:29:45.333 INFO:tasks.workunit.client.1.smithi138.stdout:3/637: dwrite d0/fd [8388608,4194304] 0 2024-05-27T09:29:45.337 INFO:tasks.workunit.client.1.smithi138.stdout:0/600: write da/d2a/d8e/d1f/d42/d63/d9d/fb6 [3979230,117089] 0 2024-05-27T09:29:45.338 INFO:tasks.workunit.client.1.smithi138.stdout:3/638: mkdir d0/dda 0 2024-05-27T09:29:45.342 INFO:tasks.workunit.client.1.smithi138.stdout:0/601: unlink da/d2a/d8e/d1f/d42/d4a/l82 0 2024-05-27T09:29:45.342 INFO:tasks.workunit.client.1.smithi138.stdout:0/602: dread - da/d2a/d8e/d6a/d99/fb2 zero size 2024-05-27T09:29:45.342 INFO:tasks.workunit.client.1.smithi138.stdout:0/603: chown da/d2a/d8e/d43 40 1 2024-05-27T09:29:45.343 INFO:tasks.workunit.client.1.smithi138.stdout:0/604: write f5 [2982823,106750] 0 2024-05-27T09:29:45.343 INFO:tasks.workunit.client.1.smithi138.stdout:0/605: write f1 [1399188,122950] 0 2024-05-27T09:29:45.344 INFO:tasks.workunit.client.1.smithi138.stdout:4/654: write d7/df/d35/fa2 [280387,95452] 0 2024-05-27T09:29:45.344 INFO:tasks.workunit.client.1.smithi138.stdout:4/655: chown d7/df/d5b/f86 10508629 1 2024-05-27T09:29:45.347 INFO:tasks.workunit.client.1.smithi138.stdout:0/606: mkdir da/d2a/d8e/d6a/dc5 0 2024-05-27T09:29:45.349 INFO:tasks.workunit.client.1.smithi138.stdout:4/656: symlink d7/df/d35/d4c/le4 0 2024-05-27T09:29:45.351 INFO:tasks.workunit.client.1.smithi138.stdout:0/607: creat da/d2a/d8e/d1f/d60/fc6 x:0 0 0 2024-05-27T09:29:45.354 INFO:tasks.workunit.client.1.smithi138.stdout:4/657: mkdir d7/dba/dcb/de5 0 2024-05-27T09:29:45.358 INFO:tasks.workunit.client.1.smithi138.stdout:4/658: getdents d7/df/d35/d4c/dc0 0 2024-05-27T09:29:45.358 INFO:tasks.workunit.client.1.smithi138.stdout:4/659: fsync d7/df/d35/f77 0 2024-05-27T09:29:45.359 INFO:tasks.workunit.client.1.smithi138.stdout:3/639: dread d0/fd [0,4194304] 0 2024-05-27T09:29:45.360 INFO:tasks.workunit.client.1.smithi138.stdout:3/640: dread - d0/d3f/d50/d9b/d9f/fc9 zero size 2024-05-27T09:29:45.360 INFO:tasks.workunit.client.1.smithi138.stdout:8/749: dwrite d3/d8/d32/d3f/d9c/d9f/fa7 [0,4194304] 0 2024-05-27T09:29:45.360 INFO:tasks.workunit.client.1.smithi138.stdout:8/750: chown d3/d8/d32/d3f/d9c/d9f/d86/f93 609 1 2024-05-27T09:29:45.362 INFO:tasks.workunit.client.1.smithi138.stdout:4/660: rename d7/dba/dcf/d3a/f6b to d7/df/d5b/da1/db1/fe6 0 2024-05-27T09:29:45.368 INFO:tasks.workunit.client.1.smithi138.stdout:4/661: rmdir d7/dba/dcb/de5 0 2024-05-27T09:29:45.368 INFO:tasks.workunit.client.1.smithi138.stdout:4/662: dread - d7/df/d5b/da1/dbc/fe3 zero size 2024-05-27T09:29:45.368 INFO:tasks.workunit.client.1.smithi138.stdout:4/663: write d7/d6e/fd1 [2294463,81849] 0 2024-05-27T09:29:45.372 INFO:tasks.workunit.client.1.smithi138.stdout:4/664: symlink d7/dba/dcf/d3a/le7 0 2024-05-27T09:29:45.372 INFO:tasks.workunit.client.1.smithi138.stdout:4/665: creat d7/df/d35/d4c/d59/d5d/fe8 x:0 0 0 2024-05-27T09:29:45.372 INFO:tasks.workunit.client.1.smithi138.stdout:4/666: fsync d7/df/d5b/f89 0 2024-05-27T09:29:45.377 INFO:tasks.workunit.client.1.smithi138.stdout:4/667: truncate d7/df/d35/f5c 3571178 0 2024-05-27T09:29:45.389 INFO:tasks.workunit.client.1.smithi138.stdout:4/668: creat d7/dba/dcb/fe9 x:0 0 0 2024-05-27T09:29:45.391 INFO:tasks.workunit.client.1.smithi138.stdout:4/669: truncate d7/dba/dcf/d3a/fb0 1551530 0 2024-05-27T09:29:45.396 INFO:tasks.workunit.client.1.smithi138.stdout:4/670: mkdir d7/dea 0 2024-05-27T09:29:45.398 INFO:tasks.workunit.client.1.smithi138.stdout:4/671: creat d7/df/d35/d4c/d59/d5d/d64/d6a/feb x:0 0 0 2024-05-27T09:29:45.405 INFO:tasks.workunit.client.1.smithi138.stdout:4/672: write d7/df/d35/d4c/d59/d5d/d64/d6a/fbe [776790,41565] 0 2024-05-27T09:29:45.405 INFO:tasks.workunit.client.1.smithi138.stdout:4/673: read - d7/d6e/faf zero size 2024-05-27T09:29:45.406 INFO:tasks.workunit.client.1.smithi138.stdout:4/674: creat d7/df/d35/d4c/fec x:0 0 0 2024-05-27T09:29:45.406 INFO:tasks.workunit.client.1.smithi138.stdout:4/675: readlink d7/dba/dcf/daa/dda/le2 0 2024-05-27T09:29:45.407 INFO:tasks.workunit.client.1.smithi138.stdout:4/676: chown d7/dba/dcf/ce1 651327 1 2024-05-27T09:29:45.407 INFO:tasks.workunit.client.1.smithi138.stdout:5/614: dwrite d3/d12/d32/f4e [0,4194304] 0 2024-05-27T09:29:45.413 INFO:tasks.workunit.client.1.smithi138.stdout:8/751: dwrite d3/d8/d32/d34/d38/fc7 [0,4194304] 0 2024-05-27T09:29:45.423 INFO:tasks.workunit.client.1.smithi138.stdout:4/677: fsync d7/df/d35/d4c/fec 0 2024-05-27T09:29:45.424 INFO:tasks.workunit.client.1.smithi138.stdout:4/678: creat d7/dba/dcb/fed x:0 0 0 2024-05-27T09:29:45.461 INFO:tasks.workunit.client.1.smithi138.stdout:0/608: dwrite da/d2a/d8e/d1f/f21 [0,4194304] 0 2024-05-27T09:29:45.464 INFO:tasks.workunit.client.1.smithi138.stdout:0/609: mkdir da/d2a/d55/dc7 0 2024-05-27T09:29:45.470 INFO:tasks.workunit.client.1.smithi138.stdout:0/610: creat da/d2a/d8e/d6a/daf/dc4/fc8 x:0 0 0 2024-05-27T09:29:45.470 INFO:tasks.workunit.client.1.smithi138.stdout:0/611: write da/d2a/d8e/d1f/d71/f93 [284919,12533] 0 2024-05-27T09:29:45.476 INFO:tasks.workunit.client.1.smithi138.stdout:0/612: write da/d2a/d8e/d1f/fa2 [323410,33888] 0 2024-05-27T09:29:45.477 INFO:tasks.workunit.client.1.smithi138.stdout:4/679: dwrite d7/df/d35/d4c/f69 [4194304,4194304] 0 2024-05-27T09:29:45.477 INFO:tasks.workunit.client.1.smithi138.stdout:4/680: chown d7/df/d5b/da1/dbc 226796 1 2024-05-27T09:29:45.478 INFO:tasks.workunit.client.1.smithi138.stdout:4/681: chown d7/d6e/faf 802 1 2024-05-27T09:29:45.478 INFO:tasks.workunit.client.1.smithi138.stdout:4/682: stat d7/df/d35/d4c/l97 0 2024-05-27T09:29:45.484 INFO:tasks.workunit.client.1.smithi138.stdout:4/683: rename d7/f60 to d7/df/d35/d4c/d59/d5d/fee 0 2024-05-27T09:29:45.488 INFO:tasks.workunit.client.1.smithi138.stdout:4/684: mkdir d7/dba/dcf/daa/def 0 2024-05-27T09:29:45.488 INFO:tasks.workunit.client.1.smithi138.stdout:4/685: chown f3 244441 1 2024-05-27T09:29:45.493 INFO:tasks.workunit.client.1.smithi138.stdout:4/686: mknod d7/dba/cf0 0 2024-05-27T09:29:45.493 INFO:tasks.workunit.client.1.smithi138.stdout:8/752: dwrite d3/d8/d32/d34/d38/f92 [0,4194304] 0 2024-05-27T09:29:45.509 INFO:tasks.workunit.client.1.smithi138.stdout:3/641: dwrite d0/d2/da/df/d10/f77 [4194304,4194304] 0 2024-05-27T09:29:45.509 INFO:tasks.workunit.client.1.smithi138.stdout:3/642: chown d0/d2/da/cb 12 1 2024-05-27T09:29:45.509 INFO:tasks.workunit.client.1.smithi138.stdout:3/643: write d0/d2/da/df/d10/d2a/f8f [80932,96185] 0 2024-05-27T09:29:45.514 INFO:tasks.workunit.client.1.smithi138.stdout:8/753: truncate d3/d8/d24/d4d/f61 1371666 0 2024-05-27T09:29:45.529 INFO:tasks.workunit.client.1.smithi138.stdout:0/613: write da/d2a/d8e/d1f/d42/d4a/d53/f6e [441336,109964] 0 2024-05-27T09:29:45.529 INFO:tasks.workunit.client.1.smithi138.stdout:8/754: mkdir d3/d8/d32/d3f/d9c/d9f/d103 0 2024-05-27T09:29:45.529 INFO:tasks.workunit.client.1.smithi138.stdout:0/614: read da/d17/f44 [3426660,54370] 0 2024-05-27T09:29:45.529 INFO:tasks.workunit.client.1.smithi138.stdout:8/755: chown d3/d8/d32/d34/d38/dd0/le9 5 1 2024-05-27T09:29:45.531 INFO:tasks.workunit.client.1.smithi138.stdout:8/756: link d3/d8/d24/d4d/f5a d3/db5/f104 0 2024-05-27T09:29:45.531 INFO:tasks.workunit.client.1.smithi138.stdout:0/615: mkdir da/d2a/dc9 0 2024-05-27T09:29:45.531 INFO:tasks.workunit.client.1.smithi138.stdout:0/616: chown da/d17/c1b 1 1 2024-05-27T09:29:45.531 INFO:tasks.workunit.client.1.smithi138.stdout:8/757: write d3/d8/d32/fe6 [383272,1782] 0 2024-05-27T09:29:45.533 INFO:tasks.workunit.client.1.smithi138.stdout:8/758: creat d3/d8/d32/f105 x:0 0 0 2024-05-27T09:29:45.536 INFO:tasks.workunit.client.1.smithi138.stdout:5/615: dwrite d3/d12/d32/d7e/f9f [0,4194304] 0 2024-05-27T09:29:45.545 INFO:tasks.workunit.client.1.smithi138.stdout:4/687: dwrite d7/d6e/fc2 [0,4194304] 0 2024-05-27T09:29:45.549 INFO:tasks.workunit.client.1.smithi138.stdout:8/759: rmdir d3/d8/d32 39 2024-05-27T09:29:45.550 INFO:tasks.workunit.client.1.smithi138.stdout:8/760: mknod d3/db5/c106 0 2024-05-27T09:29:45.552 INFO:tasks.workunit.client.1.smithi138.stdout:4/688: symlink d7/dba/dcf/daa/def/lf1 0 2024-05-27T09:29:45.564 INFO:tasks.workunit.client.1.smithi138.stdout:5/616: rmdir d3/d12/d26 39 2024-05-27T09:29:45.564 INFO:tasks.workunit.client.1.smithi138.stdout:5/617: chown d3/d12/d32/d87/f8f 12604956 1 2024-05-27T09:29:45.564 INFO:tasks.workunit.client.1.smithi138.stdout:5/618: chown d3/d12/f79 0 1 2024-05-27T09:29:45.568 INFO:tasks.workunit.client.1.smithi138.stdout:4/689: mknod d7/df/d35/cf2 0 2024-05-27T09:29:45.573 INFO:tasks.workunit.client.1.smithi138.stdout:4/690: creat d7/df/d5b/da1/db1/ff3 x:0 0 0 2024-05-27T09:29:45.574 INFO:tasks.workunit.client.1.smithi138.stdout:5/619: getdents d3/d12/d16/d1d/d23/d68 0 2024-05-27T09:29:45.574 INFO:tasks.workunit.client.1.smithi138.stdout:5/620: truncate d3/d12/d16/d1d/d22/d2b/fc1 905005 0 2024-05-27T09:29:45.584 INFO:tasks.workunit.client.1.smithi138.stdout:4/691: creat d7/df/d35/d4c/d59/d5d/d64/d6a/ff4 x:0 0 0 2024-05-27T09:29:45.585 INFO:tasks.workunit.client.1.smithi138.stdout:5/621: mkdir d3/dab/dc2/dc6 0 2024-05-27T09:29:45.585 INFO:tasks.workunit.client.1.smithi138.stdout:6/782: sync 2024-05-27T09:29:45.585 INFO:tasks.workunit.client.1.smithi138.stdout:1/634: sync 2024-05-27T09:29:45.585 INFO:tasks.workunit.client.1.smithi138.stdout:2/574: sync 2024-05-27T09:29:45.585 INFO:tasks.workunit.client.1.smithi138.stdout:7/713: sync 2024-05-27T09:29:45.585 INFO:tasks.workunit.client.1.smithi138.stdout:9/673: sync 2024-05-27T09:29:45.585 INFO:tasks.workunit.client.1.smithi138.stdout:6/783: fdatasync d3/d11/d2d/d37/f3c 0 2024-05-27T09:29:45.591 INFO:tasks.workunit.client.1.smithi138.stdout:5/622: unlink d3/d12/d16/f69 0 2024-05-27T09:29:45.591 INFO:tasks.workunit.client.1.smithi138.stdout:5/623: dread - d3/d12/d16/d1d/d22/d2b/d9e/fbe zero size 2024-05-27T09:29:45.591 INFO:tasks.workunit.client.1.smithi138.stdout:5/624: creat d3/d12/d16/d1d/d22/d2b/d9e/fc7 x:0 0 0 2024-05-27T09:29:45.592 INFO:tasks.workunit.client.1.smithi138.stdout:6/784: rename d3/d91 to d3/d11/d2d/d37/dd1/d9b/d10a 0 2024-05-27T09:29:45.611 INFO:tasks.workunit.client.1.smithi138.stdout:7/714: dread d0/d2b/daf/d64/fe3 [0,4194304] 0 2024-05-27T09:29:45.614 INFO:tasks.workunit.client.1.smithi138.stdout:7/715: rename d0/d2b/daf/d43/laa to d0/d2b/le7 0 2024-05-27T09:29:45.618 INFO:tasks.workunit.client.1.smithi138.stdout:7/716: creat d0/d23/d3c/d84/d9c/dc3/de6/fe8 x:0 0 0 2024-05-27T09:29:45.622 INFO:tasks.workunit.client.1.smithi138.stdout:7/717: symlink d0/d23/d3a/de0/le9 0 2024-05-27T09:29:45.624 INFO:tasks.workunit.client.1.smithi138.stdout:3/644: dwrite d0/d3f/d50/f7c [0,4194304] 0 2024-05-27T09:29:45.629 INFO:tasks.workunit.client.1.smithi138.stdout:3/645: rename d0/d2/da/df/d10/d32/d37/f65 to d0/d3f/d60/dcb/fdb 0 2024-05-27T09:29:45.629 INFO:tasks.workunit.client.1.smithi138.stdout:3/646: creat d0/d80/fdc x:0 0 0 2024-05-27T09:29:45.631 INFO:tasks.workunit.client.1.smithi138.stdout:3/647: mknod d0/d2/da/df/d10/d2a/db2/dd1/dd8/cdd 0 2024-05-27T09:29:45.634 INFO:tasks.workunit.client.1.smithi138.stdout:3/648: creat d0/d2/da/df/d10/d2a/d48/d90/fde x:0 0 0 2024-05-27T09:29:45.650 INFO:tasks.workunit.client.1.smithi138.stdout:4/692: dwrite d7/df/d35/d4c/dc0/fdb [0,4194304] 0 2024-05-27T09:29:45.650 INFO:tasks.workunit.client.1.smithi138.stdout:4/693: chown d7/d6e 56575218 1 2024-05-27T09:29:45.652 INFO:tasks.workunit.client.1.smithi138.stdout:1/635: mkdir d4/dcc 0 2024-05-27T09:29:45.655 INFO:tasks.workunit.client.1.smithi138.stdout:5/625: symlink d3/d12/d32/d87/dc0/lc8 0 2024-05-27T09:29:45.655 INFO:tasks.workunit.client.1.smithi138.stdout:5/626: fdatasync d3/d12/d32/d6b/f78 0 2024-05-27T09:29:45.657 INFO:tasks.workunit.client.1.smithi138.stdout:4/694: symlink d7/d6e/lf5 0 2024-05-27T09:29:45.658 INFO:tasks.workunit.client.1.smithi138.stdout:4/695: read d7/df/d35/f92 [854153,8356] 0 2024-05-27T09:29:45.658 INFO:tasks.workunit.client.1.smithi138.stdout:4/696: chown d7/dba/dcb/cd8 154 1 2024-05-27T09:29:45.659 INFO:tasks.workunit.client.1.smithi138.stdout:7/718: dread d0/fac [0,4194304] 0 2024-05-27T09:29:45.659 INFO:tasks.workunit.client.1.smithi138.stdout:7/719: fdatasync d0/d2b/daf/f6b 0 2024-05-27T09:29:45.660 INFO:tasks.workunit.client.1.smithi138.stdout:1/636: unlink d4/d48/d39/d5f/d63/c73 0 2024-05-27T09:29:45.662 INFO:tasks.workunit.client.1.smithi138.stdout:7/720: unlink d0/d23/d3c/d3f/l4e 0 2024-05-27T09:29:45.662 INFO:tasks.workunit.client.1.smithi138.stdout:7/721: readlink d0/d2b/daf/d60/d9d/ld0 0 2024-05-27T09:29:45.662 INFO:tasks.workunit.client.1.smithi138.stdout:6/785: write d3/f53 [2969195,109629] 0 2024-05-27T09:29:45.662 INFO:tasks.workunit.client.1.smithi138.stdout:0/617: dwrite da/d2a/d3f/f59 [0,4194304] 0 2024-05-27T09:29:45.662 INFO:tasks.workunit.client.1.smithi138.stdout:5/627: symlink d3/d12/d32/d87/d89/dc3/lc9 0 2024-05-27T09:29:45.663 INFO:tasks.workunit.client.1.smithi138.stdout:2/575: dwrite d1/d10/d52/fa6 [4194304,4194304] 0 2024-05-27T09:29:45.664 INFO:tasks.workunit.client.1.smithi138.stdout:7/722: creat d0/d23/d49/d78/fea x:0 0 0 2024-05-27T09:29:45.664 INFO:tasks.workunit.client.1.smithi138.stdout:7/723: creat d0/d23/d3c/d84/d9c/dc3/de6/feb x:0 0 0 2024-05-27T09:29:45.671 INFO:tasks.workunit.client.1.smithi138.stdout:6/786: mknod d3/d11/d2d/d37/c10b 0 2024-05-27T09:29:45.673 INFO:tasks.workunit.client.1.smithi138.stdout:5/628: creat d3/d12/d32/d6b/fca x:0 0 0 2024-05-27T09:29:45.673 INFO:tasks.workunit.client.1.smithi138.stdout:5/629: fsync d3/d12/d16/fa9 0 2024-05-27T09:29:45.673 INFO:tasks.workunit.client.1.smithi138.stdout:5/630: chown d3/d12/d32/d7e/f9f 256339 1 2024-05-27T09:29:45.675 INFO:tasks.workunit.client.1.smithi138.stdout:0/618: unlink da/f3c 0 2024-05-27T09:29:45.675 INFO:tasks.workunit.client.1.smithi138.stdout:7/724: rename d0/d2b/daf/d98/db6/ccb to d0/d23/d3c/d84/d9c/dc3/de6/cec 0 2024-05-27T09:29:45.675 INFO:tasks.workunit.client.1.smithi138.stdout:7/725: write d0/d2b/daf/d60/f9f [1409204,58617] 0 2024-05-27T09:29:45.680 INFO:tasks.workunit.client.1.smithi138.stdout:1/637: write d4/d3c/f42 [640212,80660] 0 2024-05-27T09:29:45.681 INFO:tasks.workunit.client.1.smithi138.stdout:6/787: symlink d3/d11/dc0/d7d/l10c 0 2024-05-27T09:29:45.685 INFO:tasks.workunit.client.1.smithi138.stdout:0/619: mkdir da/d2a/dc9/dca 0 2024-05-27T09:29:45.687 INFO:tasks.workunit.client.1.smithi138.stdout:0/620: truncate da/d2a/d8e/d1f/d42/d4a/d53/f87 305616 0 2024-05-27T09:29:45.689 INFO:tasks.workunit.client.1.smithi138.stdout:6/788: mkdir d3/d11/dc0/d7d/da5/d10d 0 2024-05-27T09:29:45.690 INFO:tasks.workunit.client.1.smithi138.stdout:1/638: getdents d4/d48/d39/d5f 0 2024-05-27T09:29:45.693 INFO:tasks.workunit.client.1.smithi138.stdout:1/639: symlink d4/d48/d39/d5f/d93/lcd 0 2024-05-27T09:29:45.693 INFO:tasks.workunit.client.1.smithi138.stdout:0/621: write da/d2a/d8e/d1f/d42/d63/fb5 [72708,120628] 0 2024-05-27T09:29:45.702 INFO:tasks.workunit.client.1.smithi138.stdout:7/726: dread d0/f8c [0,4194304] 0 2024-05-27T09:29:45.702 INFO:tasks.workunit.client.1.smithi138.stdout:1/640: link d4/d48/d39/d5f/d4d/fac d4/d48/d22/d65/fce 0 2024-05-27T09:29:45.705 INFO:tasks.workunit.client.1.smithi138.stdout:0/622: rename da/d2a/d8e/d43/d74/d97/f9e to da/fcb 0 2024-05-27T09:29:45.706 INFO:tasks.workunit.client.1.smithi138.stdout:7/727: truncate d0/d23/d3a/f55 264974 0 2024-05-27T09:29:45.710 INFO:tasks.workunit.client.1.smithi138.stdout:7/728: mknod d0/d2b/daf/d60/d9d/ced 0 2024-05-27T09:29:45.713 INFO:tasks.workunit.client.1.smithi138.stdout:1/641: rename d4/d1a/d2e/d32/f7a to d4/d1a/d2e/d4a/d7e/fcf 0 2024-05-27T09:29:45.716 INFO:tasks.workunit.client.1.smithi138.stdout:4/697: dwrite d7/df/d35/d4c/f66 [4194304,4194304] 0 2024-05-27T09:29:45.716 INFO:tasks.workunit.client.1.smithi138.stdout:4/698: write d7/dba/dcb/fed [430173,29145] 0 2024-05-27T09:29:45.720 INFO:tasks.workunit.client.1.smithi138.stdout:7/729: rename d0/d23/dc4/ddf/l66 to d0/lee 0 2024-05-27T09:29:45.720 INFO:tasks.workunit.client.1.smithi138.stdout:4/699: mkdir d7/df/d35/d4c/d59/d5d/d64/df6 0 2024-05-27T09:29:45.721 INFO:tasks.workunit.client.1.smithi138.stdout:4/700: write d7/f78 [1171694,72707] 0 2024-05-27T09:29:45.721 INFO:tasks.workunit.client.1.smithi138.stdout:4/701: creat d7/df/d35/d4c/d59/d5d/ff7 x:0 0 0 2024-05-27T09:29:45.736 INFO:tasks.workunit.client.1.smithi138.stdout:7/730: write d0/d2b/daf/f6b [530224,74064] 0 2024-05-27T09:29:45.737 INFO:tasks.workunit.client.1.smithi138.stdout:0/623: dread da/d2a/d55/fb4 [0,4194304] 0 2024-05-27T09:29:45.738 INFO:tasks.workunit.client.1.smithi138.stdout:6/789: dwrite d3/fe2 [4194304,4194304] 0 2024-05-27T09:29:45.742 INFO:tasks.workunit.client.1.smithi138.stdout:1/642: dread d4/d48/d39/d5f/d4d/d2b/f67 [0,4194304] 0 2024-05-27T09:29:45.743 INFO:tasks.workunit.client.1.smithi138.stdout:0/624: getdents da/d2a/d8e/d1f/d42/d63/d9d 0 2024-05-27T09:29:45.743 INFO:tasks.workunit.client.1.smithi138.stdout:0/625: chown da/d2a/d8e/d6a/daf/dc4/fc8 1997 1 2024-05-27T09:29:45.745 INFO:tasks.workunit.client.1.smithi138.stdout:1/643: creat d4/d1a/d2e/d4a/d7e/dc0/fd0 x:0 0 0 2024-05-27T09:29:45.745 INFO:tasks.workunit.client.1.smithi138.stdout:1/644: fsync d4/d48/d22/d65/fce 0 2024-05-27T09:29:45.745 INFO:tasks.workunit.client.1.smithi138.stdout:1/645: chown d4/d48/d22/fa5 68817 1 2024-05-27T09:29:45.747 INFO:tasks.workunit.client.1.smithi138.stdout:5/631: dwrite d3/d12/d26/d53/f85 [0,4194304] 0 2024-05-27T09:29:45.748 INFO:tasks.workunit.client.1.smithi138.stdout:1/646: mknod d4/dc7/cd1 0 2024-05-27T09:29:45.753 INFO:tasks.workunit.client.1.smithi138.stdout:7/731: rmdir d0/d23/d3a 39 2024-05-27T09:29:45.757 INFO:tasks.workunit.client.1.smithi138.stdout:7/732: rename d0/d23/dc4/ddf/f9e to d0/d23/dc4/ddf/d91/db8/fef 0 2024-05-27T09:29:45.759 INFO:tasks.workunit.client.1.smithi138.stdout:7/733: mknod d0/d2b/daf/d64/cf0 0 2024-05-27T09:29:45.761 INFO:tasks.workunit.client.1.smithi138.stdout:0/626: dread da/d2a/d55/f98 [0,4194304] 0 2024-05-27T09:29:45.762 INFO:tasks.workunit.client.1.smithi138.stdout:7/734: rename d0/d23/d3c/ca1 to d0/d2b/daf/dbf/cf1 0 2024-05-27T09:29:45.763 INFO:tasks.workunit.client.1.smithi138.stdout:7/735: chown d0/d23/dc4/ddf/f95 761248 1 2024-05-27T09:29:45.763 INFO:tasks.workunit.client.1.smithi138.stdout:5/632: unlink d3/d12/d16/d1d/l5f 0 2024-05-27T09:29:45.763 INFO:tasks.workunit.client.1.smithi138.stdout:5/633: dread - d3/d12/d16/d1d/d30/f8b zero size 2024-05-27T09:29:45.763 INFO:tasks.workunit.client.1.smithi138.stdout:5/634: read d3/d12/d32/d60/f7b [331789,85479] 0 2024-05-27T09:29:45.766 INFO:tasks.workunit.client.1.smithi138.stdout:1/647: creat d4/d48/d39/d5f/d63/fd2 x:0 0 0 2024-05-27T09:29:45.768 INFO:tasks.workunit.client.1.smithi138.stdout:6/790: read d3/d11/dc0/d5a/f68 [4108060,84032] 0 2024-05-27T09:29:45.768 INFO:tasks.workunit.client.1.smithi138.stdout:6/791: truncate d3/d11/dc0/d5a/db4/fb6 653829 0 2024-05-27T09:29:45.768 INFO:tasks.workunit.client.1.smithi138.stdout:0/627: symlink da/d2a/d2b/lcc 0 2024-05-27T09:29:45.772 INFO:tasks.workunit.client.1.smithi138.stdout:5/635: mkdir d3/d12/d16/d1d/d22/d2b/d9e/dcb 0 2024-05-27T09:29:45.772 INFO:tasks.workunit.client.1.smithi138.stdout:5/636: write d3/dab/fac [729207,72934] 0 2024-05-27T09:29:45.773 INFO:tasks.workunit.client.1.smithi138.stdout:1/648: mkdir d4/d48/d39/d5f/dd3 0 2024-05-27T09:29:45.774 INFO:tasks.workunit.client.1.smithi138.stdout:5/637: creat d3/d12/d16/d1d/d22/db9/fcc x:0 0 0 2024-05-27T09:29:45.776 INFO:tasks.workunit.client.1.smithi138.stdout:1/649: creat d4/d38/fd4 x:0 0 0 2024-05-27T09:29:45.777 INFO:tasks.workunit.client.1.smithi138.stdout:1/650: symlink d4/d48/d39/d5f/ld5 0 2024-05-27T09:29:45.779 INFO:tasks.workunit.client.1.smithi138.stdout:1/651: creat d4/d1a/d2e/d32/d8c/dc6/fd6 x:0 0 0 2024-05-27T09:29:45.783 INFO:tasks.workunit.client.1.smithi138.stdout:1/652: getdents d4/d48/d22 0 2024-05-27T09:29:45.786 INFO:tasks.workunit.client.1.smithi138.stdout:4/702: dwrite d7/dba/dcf/d3a/f8e [0,4194304] 0 2024-05-27T09:29:45.835 INFO:tasks.workunit.client.1.smithi138.stdout:7/736: dwrite d0/d23/dc4/ddf/d91/db8/fef [0,4194304] 0 2024-05-27T09:29:45.839 INFO:tasks.workunit.client.1.smithi138.stdout:7/737: getdents d0/d2b/daf/dbf/dc8 0 2024-05-27T09:29:45.839 INFO:tasks.workunit.client.1.smithi138.stdout:7/738: fsync d0/f10 0 2024-05-27T09:29:45.841 INFO:tasks.workunit.client.1.smithi138.stdout:5/638: dwrite d3/d12/d26/fa6 [0,4194304] 0 2024-05-27T09:29:45.841 INFO:tasks.workunit.client.1.smithi138.stdout:5/639: fsync d3/d12/d32/f4f 0 2024-05-27T09:29:45.841 INFO:tasks.workunit.client.1.smithi138.stdout:5/640: read d3/d12/f79 [19566,76079] 0 2024-05-27T09:29:45.841 INFO:tasks.workunit.client.1.smithi138.stdout:7/739: rmdir d0/d23/d3c/d3f 39 2024-05-27T09:29:45.842 INFO:tasks.workunit.client.1.smithi138.stdout:7/740: chown d0/d2b/daf/dbf/cc2 13 1 2024-05-27T09:29:45.842 INFO:tasks.workunit.client.1.smithi138.stdout:7/741: write d0/fac [239265,111937] 0 2024-05-27T09:29:45.843 INFO:tasks.workunit.client.1.smithi138.stdout:7/742: truncate d0/d2b/daf/f56 8142362 0 2024-05-27T09:29:45.845 INFO:tasks.workunit.client.1.smithi138.stdout:5/641: symlink d3/d12/d32/d87/dc0/lcd 0 2024-05-27T09:29:45.846 INFO:tasks.workunit.client.1.smithi138.stdout:5/642: symlink d3/d12/d16/d1d/lce 0 2024-05-27T09:29:45.847 INFO:tasks.workunit.client.1.smithi138.stdout:5/643: mkdir d3/d12/dcf 0 2024-05-27T09:29:45.849 INFO:tasks.workunit.client.1.smithi138.stdout:5/644: mkdir d3/d12/d32/d60/d93/dd0 0 2024-05-27T09:29:45.849 INFO:tasks.workunit.client.1.smithi138.stdout:5/645: truncate d3/d12/d16/d1d/d23/d3b/f95 567105 0 2024-05-27T09:29:45.850 INFO:tasks.workunit.client.1.smithi138.stdout:5/646: truncate d3/d12/d32/d6b/f78 281134 0 2024-05-27T09:29:45.866 INFO:tasks.workunit.client.1.smithi138.stdout:5/647: read d3/d12/d16/d1d/d22/d2b/d9e/db6/f9a [1795232,5221] 0 2024-05-27T09:29:45.868 INFO:tasks.workunit.client.1.smithi138.stdout:5/648: symlink d3/d12/d32/d6b/ld1 0 2024-05-27T09:29:45.868 INFO:tasks.workunit.client.1.smithi138.stdout:5/649: creat d3/d12/d32/d6b/fd2 x:0 0 0 2024-05-27T09:29:45.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:45 smithi138 ceph-mon[34885]: pgmap v20: 65 pgs: 65 active+clean; 3.3 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 162 MiB/s rd, 197 MiB/s wr, 291 op/s 2024-05-27T09:29:45.918 INFO:tasks.workunit.client.1.smithi138.stdout:0/628: dwrite da/d2a/d55/f95 [0,4194304] 0 2024-05-27T09:29:45.919 INFO:tasks.workunit.client.1.smithi138.stdout:0/629: chown da/d2a/d8e/d1f/d4f/lc1 3204838 1 2024-05-27T09:29:45.920 INFO:tasks.workunit.client.1.smithi138.stdout:4/703: dwrite d7/df/d35/d4c/d59/f73 [0,4194304] 0 2024-05-27T09:29:45.924 INFO:tasks.workunit.client.1.smithi138.stdout:0/630: link da/d2a/d8e/d1f/d42/d63/f91 da/d2a/dc9/fcd 0 2024-05-27T09:29:45.924 INFO:tasks.workunit.client.1.smithi138.stdout:7/743: truncate d0/fac 3867644 0 2024-05-27T09:29:45.924 INFO:tasks.workunit.client.1.smithi138.stdout:7/744: write d0/d2b/daf/d64/f6e [1084747,17033] 0 2024-05-27T09:29:45.924 INFO:tasks.workunit.client.1.smithi138.stdout:7/745: fsync d0/d23/f4c 0 2024-05-27T09:29:45.925 INFO:tasks.workunit.client.1.smithi138.stdout:4/704: rename d7/dea to d7/d6e/df8 0 2024-05-27T09:29:45.927 INFO:tasks.workunit.client.1.smithi138.stdout:0/631: rmdir da/d2a/d8e/d1f/d42/d4a/d53/d86 0 2024-05-27T09:29:45.929 INFO:tasks.workunit.client.1.smithi138.stdout:7/746: unlink d0/d23/d3c/d84/d9c/dc3/fd4 0 2024-05-27T09:29:45.931 INFO:tasks.workunit.client.1.smithi138.stdout:4/705: getdents d7/df/d35/d4c 0 2024-05-27T09:29:45.934 INFO:tasks.workunit.client.1.smithi138.stdout:0/632: truncate da/d2a/d8e/d1f/d42/d4a/f4e 3604726 0 2024-05-27T09:29:45.934 INFO:tasks.workunit.client.1.smithi138.stdout:4/706: mkdir d7/dba/df9 0 2024-05-27T09:29:45.935 INFO:tasks.workunit.client.1.smithi138.stdout:6/792: dwrite d3/d11/dc0/d5a/d6a/f6b [0,4194304] 0 2024-05-27T09:29:45.935 INFO:tasks.workunit.client.1.smithi138.stdout:4/707: truncate d7/d6e/faf 810517 0 2024-05-27T09:29:45.947 INFO:tasks.workunit.client.1.smithi138.stdout:4/708: mkdir d7/dba/dfa 0 2024-05-27T09:29:45.947 INFO:tasks.workunit.client.1.smithi138.stdout:6/793: mknod d3/d11/dc0/d7d/de3/c10e 0 2024-05-27T09:29:45.947 INFO:tasks.workunit.client.1.smithi138.stdout:6/794: chown d3/d11/l16 19737884 1 2024-05-27T09:29:45.947 INFO:tasks.workunit.client.1.smithi138.stdout:6/795: stat d3/d11/dc0/d5a/fff 0 2024-05-27T09:29:45.952 INFO:tasks.workunit.client.1.smithi138.stdout:7/747: dread d0/d2b/daf/f12 [0,4194304] 0 2024-05-27T09:29:45.952 INFO:tasks.workunit.client.1.smithi138.stdout:7/748: fdatasync d0/d2b/daf/d60/f9f 0 2024-05-27T09:29:45.952 INFO:tasks.workunit.client.1.smithi138.stdout:7/749: creat d0/d2b/daf/d98/db6/ff2 x:0 0 0 2024-05-27T09:29:45.952 INFO:tasks.workunit.client.1.smithi138.stdout:6/796: mknod d3/d11/dc0/d7d/db9/c10f 0 2024-05-27T09:29:45.952 INFO:tasks.workunit.client.1.smithi138.stdout:6/797: write d3/d11/dc0/fd3 [1721243,116951] 0 2024-05-27T09:29:45.964 INFO:tasks.workunit.client.1.smithi138.stdout:0/633: dread da/f20 [0,4194304] 0 2024-05-27T09:29:45.965 INFO:tasks.workunit.client.1.smithi138.stdout:4/709: read d7/dba/dcf/f9a [1964440,84619] 0 2024-05-27T09:29:45.965 INFO:tasks.workunit.client.1.smithi138.stdout:4/710: fsync d7/df/f2e 0 2024-05-27T09:29:45.971 INFO:tasks.workunit.client.1.smithi138.stdout:0/634: link da/d2a/d2b/f4d da/d2a/d8e/dc0/fce 0 2024-05-27T09:29:45.972 INFO:tasks.workunit.client.1.smithi138.stdout:4/711: mkdir d7/df/d35/d4c/dc0/dfb 0 2024-05-27T09:29:45.976 INFO:tasks.workunit.client.1.smithi138.stdout:0/635: rmdir da/d2a/d8e/d1f/d42/d63/d9d 39 2024-05-27T09:29:45.977 INFO:tasks.workunit.client.1.smithi138.stdout:0/636: stat da/d2a/d8e/d6a/dc5 0 2024-05-27T09:29:45.988 INFO:tasks.workunit.client.1.smithi138.stdout:4/712: dread d7/df/f19 [0,4194304] 0 2024-05-27T09:29:45.989 INFO:tasks.workunit.client.1.smithi138.stdout:4/713: mkdir d7/dba/dcf/daa/dda/dfc 0 2024-05-27T09:29:45.990 INFO:tasks.workunit.client.1.smithi138.stdout:4/714: fdatasync d7/df/f4d 0 2024-05-27T09:29:45.994 INFO:tasks.workunit.client.1.smithi138.stdout:6/798: dread d3/d11/dc0/d5a/d6a/f7f [0,4194304] 0 2024-05-27T09:29:45.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:45 smithi119 ceph-mon[29082]: pgmap v20: 65 pgs: 65 active+clean; 3.3 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 162 MiB/s rd, 197 MiB/s wr, 291 op/s 2024-05-27T09:29:45.998 INFO:tasks.workunit.client.1.smithi138.stdout:4/715: dread d7/df/d5b/f89 [0,4194304] 0 2024-05-27T09:29:45.999 INFO:tasks.workunit.client.1.smithi138.stdout:4/716: truncate d7/df/d35/d4c/d59/f90 124856 0 2024-05-27T09:29:45.999 INFO:tasks.workunit.client.1.smithi138.stdout:4/717: write d7/df/d35/d4c/d59/d5d/d64/d6a/fcd [27583,13931] 0 2024-05-27T09:29:46.000 INFO:tasks.workunit.client.1.smithi138.stdout:4/718: symlink d7/dba/dfa/lfd 0 2024-05-27T09:29:46.002 INFO:tasks.workunit.client.1.smithi138.stdout:6/799: dread d3/d11/dc0/d5a/d6a/fac [0,4194304] 0 2024-05-27T09:29:46.002 INFO:tasks.workunit.client.1.smithi138.stdout:6/800: chown d3/d11/dc0/d43/d71/lb1 2667005 1 2024-05-27T09:29:46.002 INFO:tasks.workunit.client.1.smithi138.stdout:6/801: creat d3/d11/d2d/d37/daa/f110 x:0 0 0 2024-05-27T09:29:46.002 INFO:tasks.workunit.client.1.smithi138.stdout:6/802: chown d3/d11/d2d/d37/dd1/d47 7 1 2024-05-27T09:29:46.005 INFO:tasks.workunit.client.1.smithi138.stdout:4/719: rename d7/df/d35/d4c/d59/d5d/fd4 to d7/dba/dcf/d3a/ffe 0 2024-05-27T09:29:46.005 INFO:tasks.workunit.client.1.smithi138.stdout:4/720: creat d7/dba/dcb/fff x:0 0 0 2024-05-27T09:29:46.006 INFO:tasks.workunit.client.1.smithi138.stdout:4/721: write d7/df/d35/f8b [787775,118737] 0 2024-05-27T09:29:46.006 INFO:tasks.workunit.client.1.smithi138.stdout:4/722: write d7/d6e/faf [1230282,96806] 0 2024-05-27T09:29:46.006 INFO:tasks.workunit.client.1.smithi138.stdout:4/723: truncate d7/df/f2e 950512 0 2024-05-27T09:29:46.006 INFO:tasks.workunit.client.1.smithi138.stdout:6/803: dread d3/d11/d2d/d37/daa/fb2 [0,4194304] 0 2024-05-27T09:29:46.009 INFO:tasks.workunit.client.1.smithi138.stdout:7/750: dwrite d0/d23/f5a [0,4194304] 0 2024-05-27T09:29:46.012 INFO:tasks.workunit.client.1.smithi138.stdout:6/804: link d3/d11/dc0/d5a/db4/cf7 d3/d11/dc0/d7d/de3/c111 0 2024-05-27T09:29:46.024 INFO:tasks.workunit.client.1.smithi138.stdout:7/751: creat d0/d2b/daf/dba/ff3 x:0 0 0 2024-05-27T09:29:46.031 INFO:tasks.workunit.client.1.smithi138.stdout:6/805: dread d3/d11/d2d/d37/f4a [0,4194304] 0 2024-05-27T09:29:46.032 INFO:tasks.workunit.client.1.smithi138.stdout:0/637: dwrite da/d2a/d8e/d1f/d42/d4a/d53/fbf [0,4194304] 0 2024-05-27T09:29:46.037 INFO:tasks.workunit.client.1.smithi138.stdout:0/638: mknod da/d2a/d8e/d1f/d60/dbd/ccf 0 2024-05-27T09:29:46.037 INFO:tasks.workunit.client.1.smithi138.stdout:0/639: read da/d2a/f30 [1122453,78878] 0 2024-05-27T09:29:46.046 INFO:tasks.workunit.client.1.smithi138.stdout:7/752: dread d0/d23/d3a/f46 [0,4194304] 0 2024-05-27T09:29:46.056 INFO:tasks.workunit.client.1.smithi138.stdout:7/753: mkdir d0/d23/da8/d8e/df4 0 2024-05-27T09:29:46.056 INFO:tasks.workunit.client.1.smithi138.stdout:7/754: chown d0/d23/d3c/f93 11395995 1 2024-05-27T09:29:46.057 INFO:tasks.workunit.client.1.smithi138.stdout:7/755: fdatasync d0/d23/f50 0 2024-05-27T09:29:46.057 INFO:tasks.workunit.client.1.smithi138.stdout:0/640: dread da/f66 [0,4194304] 0 2024-05-27T09:29:46.059 INFO:tasks.workunit.client.1.smithi138.stdout:0/641: mknod da/d2a/cd0 0 2024-05-27T09:29:46.060 INFO:tasks.workunit.client.1.smithi138.stdout:0/642: mknod da/d2a/d8e/d6a/daf/cd1 0 2024-05-27T09:29:46.064 INFO:tasks.workunit.client.1.smithi138.stdout:7/756: rename d0/d23/l3d to d0/d2b/daf/d60/d9d/lf5 0 2024-05-27T09:29:46.073 INFO:tasks.workunit.client.1.smithi138.stdout:7/757: rename d0/d23/de5 to d0/d23/d49/df6 0 2024-05-27T09:29:46.078 INFO:tasks.workunit.client.1.smithi138.stdout:7/758: getdents d0/d23/d3c/d84/d9c/dc3/de6 0 2024-05-27T09:29:46.081 INFO:tasks.workunit.client.1.smithi138.stdout:7/759: rmdir d0/d2b 39 2024-05-27T09:29:46.085 INFO:tasks.workunit.client.1.smithi138.stdout:7/760: mkdir d0/df7 0 2024-05-27T09:29:46.085 INFO:tasks.workunit.client.1.smithi138.stdout:7/761: write d0/d23/d3a/f46 [3503792,119479] 0 2024-05-27T09:29:46.085 INFO:tasks.workunit.client.1.smithi138.stdout:7/762: getdents d0/d2b/daf/dbf/dc8 0 2024-05-27T09:29:46.087 INFO:tasks.workunit.client.1.smithi138.stdout:7/763: symlink d0/d23/d3c/d84/lf8 0 2024-05-27T09:29:46.087 INFO:tasks.workunit.client.1.smithi138.stdout:7/764: fdatasync d0/d2b/daf/d98/fa4 0 2024-05-27T09:29:46.091 INFO:tasks.workunit.client.1.smithi138.stdout:4/724: dwrite d7/df/d5b/f7c [0,4194304] 0 2024-05-27T09:29:46.091 INFO:tasks.workunit.client.1.smithi138.stdout:4/725: getdents d7/d6e/df8 0 2024-05-27T09:29:46.095 INFO:tasks.workunit.client.1.smithi138.stdout:7/765: mknod d0/d23/cf9 0 2024-05-27T09:29:46.095 INFO:tasks.workunit.client.1.smithi138.stdout:7/766: truncate d0/d2b/daf/dba/ff3 901065 0 2024-05-27T09:29:46.095 INFO:tasks.workunit.client.1.smithi138.stdout:7/767: write d0/f36 [4606019,125177] 0 2024-05-27T09:29:46.096 INFO:tasks.workunit.client.1.smithi138.stdout:7/768: write d0/d2b/daf/d43/f7a [5226823,82315] 0 2024-05-27T09:29:46.114 INFO:tasks.workunit.client.1.smithi138.stdout:0/643: dwrite da/d2a/d8e/d1f/d42/d4a/f4e [0,4194304] 0 2024-05-27T09:29:46.114 INFO:tasks.workunit.client.1.smithi138.stdout:0/644: chown da/d2a/d8e/d1f 7878792 1 2024-05-27T09:29:46.118 INFO:tasks.workunit.client.1.smithi138.stdout:0/645: truncate da/d2a/f8f 127281 0 2024-05-27T09:29:46.120 INFO:tasks.workunit.client.1.smithi138.stdout:0/646: symlink da/d2a/d8e/dc0/ld2 0 2024-05-27T09:29:46.126 INFO:tasks.workunit.client.1.smithi138.stdout:0/647: mkdir da/d2a/d8e/d6a/dc5/dd3 0 2024-05-27T09:29:46.134 INFO:tasks.workunit.client.1.smithi138.stdout:4/726: dwrite d7/df/d35/f92 [0,4194304] 0 2024-05-27T09:29:46.137 INFO:tasks.workunit.client.1.smithi138.stdout:6/806: dwrite d3/d11/dc0/d7d/fc5 [0,4194304] 0 2024-05-27T09:29:46.137 INFO:tasks.workunit.client.1.smithi138.stdout:6/807: chown d3/d11/f103 59257 1 2024-05-27T09:29:46.142 INFO:tasks.workunit.client.1.smithi138.stdout:4/727: read d7/f78 [48328,102511] 0 2024-05-27T09:29:46.150 INFO:tasks.workunit.client.1.smithi138.stdout:0/648: dread da/d2a/d8e/d1f/f21 [0,4194304] 0 2024-05-27T09:29:46.150 INFO:tasks.workunit.client.1.smithi138.stdout:4/728: link d7/df/d5b/da1/dbc/ld2 d7/df/d35/d4c/d59/d5d/l100 0 2024-05-27T09:29:46.151 INFO:tasks.workunit.client.1.smithi138.stdout:4/729: creat d7/df/f101 x:0 0 0 2024-05-27T09:29:46.157 INFO:tasks.workunit.client.1.smithi138.stdout:7/769: dwrite d0/d23/d3c/d84/d9c/dc3/de6/feb [0,4194304] 0 2024-05-27T09:29:46.157 INFO:tasks.workunit.client.1.smithi138.stdout:7/770: write d0/d23/d3c/fb9 [2433727,84796] 0 2024-05-27T09:29:46.170 INFO:tasks.workunit.client.1.smithi138.stdout:7/771: dread d0/d2b/f5f [0,4194304] 0 2024-05-27T09:29:46.181 INFO:tasks.workunit.client.1.smithi138.stdout:7/772: symlink d0/d23/lfa 0 2024-05-27T09:29:46.182 INFO:tasks.workunit.client.1.smithi138.stdout:7/773: write d0/d23/f48 [1639428,23047] 0 2024-05-27T09:29:46.185 INFO:tasks.workunit.client.1.smithi138.stdout:7/774: creat d0/d2b/daf/d43/ffb x:0 0 0 2024-05-27T09:29:46.193 INFO:tasks.workunit.client.1.smithi138.stdout:1/653: sync 2024-05-27T09:29:46.193 INFO:tasks.workunit.client.1.smithi138.stdout:5/650: sync 2024-05-27T09:29:46.193 INFO:tasks.workunit.client.1.smithi138.stdout:2/576: sync 2024-05-27T09:29:46.193 INFO:tasks.workunit.client.1.smithi138.stdout:8/761: sync 2024-05-27T09:29:46.193 INFO:tasks.workunit.client.1.smithi138.stdout:5/651: creat d3/d12/d16/d1d/d23/fd3 x:0 0 0 2024-05-27T09:29:46.193 INFO:tasks.workunit.client.1.smithi138.stdout:2/577: creat d1/d15/d2d/d17/d25/d9d/d4b/fc5 x:0 0 0 2024-05-27T09:29:46.193 INFO:tasks.workunit.client.1.smithi138.stdout:3/649: sync 2024-05-27T09:29:46.193 INFO:tasks.workunit.client.1.smithi138.stdout:5/652: dread - d3/d12/d16/d1d/d22/d2b/daf/db2/fc4 zero size 2024-05-27T09:29:46.193 INFO:tasks.workunit.client.1.smithi138.stdout:2/578: fsync d1/d15/d2d/f39 0 2024-05-27T09:29:46.194 INFO:tasks.workunit.client.1.smithi138.stdout:8/762: write d3/db5/fd9 [625041,59858] 0 2024-05-27T09:29:46.194 INFO:tasks.workunit.client.1.smithi138.stdout:8/763: write d3/d8/d32/d34/f9a [1034269,77724] 0 2024-05-27T09:29:46.194 INFO:tasks.workunit.client.1.smithi138.stdout:9/674: sync 2024-05-27T09:29:46.194 INFO:tasks.workunit.client.1.smithi138.stdout:9/675: dread - d6/db/de/fa7 zero size 2024-05-27T09:29:46.198 INFO:tasks.workunit.client.1.smithi138.stdout:5/653: symlink d3/d12/d16/d1d/d22/db9/ld4 0 2024-05-27T09:29:46.198 INFO:tasks.workunit.client.1.smithi138.stdout:5/654: write d3/d12/d16/d1d/f24 [715029,9995] 0 2024-05-27T09:29:46.200 INFO:tasks.workunit.client.1.smithi138.stdout:2/579: mknod d1/cc6 0 2024-05-27T09:29:46.200 INFO:tasks.workunit.client.1.smithi138.stdout:8/764: link d3/d8/d32/d3f/d9c/d9f/f100 d3/d8/d32/dd6/f107 0 2024-05-27T09:29:46.200 INFO:tasks.workunit.client.1.smithi138.stdout:0/649: dwrite da/fb8 [0,4194304] 0 2024-05-27T09:29:46.202 INFO:tasks.workunit.client.1.smithi138.stdout:5/655: truncate d3/f5b 104742 0 2024-05-27T09:29:46.202 INFO:tasks.workunit.client.1.smithi138.stdout:5/656: dread - d3/f4c zero size 2024-05-27T09:29:46.203 INFO:tasks.workunit.client.1.smithi138.stdout:2/580: creat d1/d15/d2d/d17/d25/d9d/fc7 x:0 0 0 2024-05-27T09:29:46.204 INFO:tasks.workunit.client.1.smithi138.stdout:8/765: symlink d3/d8/d11/df1/dae/l108 0 2024-05-27T09:29:46.204 INFO:tasks.workunit.client.1.smithi138.stdout:9/676: write d6/db/dc/f12 [910330,26151] 0 2024-05-27T09:29:46.209 INFO:tasks.workunit.client.1.smithi138.stdout:8/766: mkdir d3/d8/d72/d109 0 2024-05-27T09:29:46.209 INFO:tasks.workunit.client.1.smithi138.stdout:9/677: write d6/da/d29/f6d [4393574,60289] 0 2024-05-27T09:29:46.209 INFO:tasks.workunit.client.1.smithi138.stdout:0/650: unlink da/d2a/d8e/dc0/ld2 0 2024-05-27T09:29:46.210 INFO:tasks.workunit.client.1.smithi138.stdout:1/654: rmdir d4/d1a/d2e 39 2024-05-27T09:29:46.210 INFO:tasks.workunit.client.1.smithi138.stdout:1/655: write d4/d1a/d2e/fb0 [1328304,60813] 0 2024-05-27T09:29:46.214 INFO:tasks.workunit.client.1.smithi138.stdout:9/678: symlink d6/da/d2f/le5 0 2024-05-27T09:29:46.214 INFO:tasks.workunit.client.1.smithi138.stdout:9/679: chown d6/da/d29/d8f/db7/dbd/cbb 3 1 2024-05-27T09:29:46.216 INFO:tasks.workunit.client.1.smithi138.stdout:9/680: symlink d6/da/d29/d8f/db7/dbd/da2/le6 0 2024-05-27T09:29:46.227 INFO:tasks.workunit.client.1.smithi138.stdout:2/581: read d1/d2f/f7d [1886497,82178] 0 2024-05-27T09:29:46.229 INFO:tasks.workunit.client.1.smithi138.stdout:2/582: mkdir d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8 0 2024-05-27T09:29:46.229 INFO:tasks.workunit.client.1.smithi138.stdout:2/583: truncate d1/d15/d2d/d17/d25/d4d/d6d/d87/fc0 63283 0 2024-05-27T09:29:46.231 INFO:tasks.workunit.client.1.smithi138.stdout:6/808: dwrite d3/d11/dc0/d5a/d6a/f6b [4194304,4194304] 0 2024-05-27T09:29:46.232 INFO:tasks.workunit.client.1.smithi138.stdout:2/584: mknod d1/d10/d3c/cc9 0 2024-05-27T09:29:46.232 INFO:tasks.workunit.client.1.smithi138.stdout:2/585: creat d1/d15/d2d/d1d/fca x:0 0 0 2024-05-27T09:29:46.232 INFO:tasks.workunit.client.1.smithi138.stdout:2/586: write d1/d10/d3c/f89 [613536,24960] 0 2024-05-27T09:29:46.234 INFO:tasks.workunit.client.1.smithi138.stdout:8/767: read d3/d8/d24/d55/d7e/d83/fb0 [745588,51103] 0 2024-05-27T09:29:46.234 INFO:tasks.workunit.client.1.smithi138.stdout:8/768: read - d3/d8/d72/fde zero size 2024-05-27T09:29:46.241 INFO:tasks.workunit.client.1.smithi138.stdout:7/775: dread d0/d2b/daf/d43/f7a [0,4194304] 0 2024-05-27T09:29:46.243 INFO:tasks.workunit.client.1.smithi138.stdout:4/730: dwrite d7/dba/dcb/fff [0,4194304] 0 2024-05-27T09:29:46.243 INFO:tasks.workunit.client.1.smithi138.stdout:4/731: write d7/df/f101 [575471,62042] 0 2024-05-27T09:29:46.249 INFO:tasks.workunit.client.1.smithi138.stdout:3/650: dread d0/d2/da/df/d10/d2a/f56 [0,4194304] 0 2024-05-27T09:29:46.258 INFO:tasks.workunit.client.1.smithi138.stdout:6/809: mkdir d3/d11/dc0/d43/dbe/d112 0 2024-05-27T09:29:46.258 INFO:tasks.workunit.client.1.smithi138.stdout:6/810: write d3/d11/dc0/d43/dbe/fe4 [4944386,17895] 0 2024-05-27T09:29:46.265 INFO:tasks.workunit.client.1.smithi138.stdout:6/811: write d3/d19/f63 [889318,126552] 0 2024-05-27T09:29:46.265 INFO:tasks.workunit.client.1.smithi138.stdout:6/812: fsync d3/d11/dc0/fd3 0 2024-05-27T09:29:46.265 INFO:tasks.workunit.client.1.smithi138.stdout:6/813: fdatasync d3/d11/d2d/d37/dd1/d9b/ff1 0 2024-05-27T09:29:46.265 INFO:tasks.workunit.client.1.smithi138.stdout:0/651: dread da/d17/f44 [0,4194304] 0 2024-05-27T09:29:46.268 INFO:tasks.workunit.client.1.smithi138.stdout:6/814: rmdir d3/d11/dc2/de1/da8 0 2024-05-27T09:29:46.271 INFO:tasks.workunit.client.1.smithi138.stdout:8/769: read d3/d8/d32/d34/d38/f46 [1596418,127469] 0 2024-05-27T09:29:46.271 INFO:tasks.workunit.client.1.smithi138.stdout:0/652: mknod da/d2a/d55/dc7/cd4 0 2024-05-27T09:29:46.273 INFO:tasks.workunit.client.1.smithi138.stdout:6/815: mknod d3/d11/dc0/d43/dbe/c113 0 2024-05-27T09:29:46.276 INFO:tasks.workunit.client.1.smithi138.stdout:5/657: dwrite d3/d12/d16/d1d/d23/f67 [0,4194304] 0 2024-05-27T09:29:46.282 INFO:tasks.workunit.client.1.smithi138.stdout:0/653: mkdir da/d2a/d8e/d1f/dd5 0 2024-05-27T09:29:46.286 INFO:tasks.workunit.client.1.smithi138.stdout:6/816: symlink d3/d11/d2d/d37/l114 0 2024-05-27T09:29:46.290 INFO:tasks.workunit.client.1.smithi138.stdout:5/658: mkdir d3/dab/dc2/dd5 0 2024-05-27T09:29:46.292 INFO:tasks.workunit.client.1.smithi138.stdout:0/654: unlink da/d2a/d2b/l46 0 2024-05-27T09:29:46.292 INFO:tasks.workunit.client.1.smithi138.stdout:0/655: read - da/d2a/d8e/d1f/d42/d4a/d53/f90 zero size 2024-05-27T09:29:46.294 INFO:tasks.workunit.client.1.smithi138.stdout:6/817: write d3/d11/d2d/d37/dd1/d47/f51 [1193796,49781] 0 2024-05-27T09:29:46.299 INFO:tasks.workunit.client.1.smithi138.stdout:5/659: link d3/d12/d32/d87/c8a d3/d12/d32/d60/d93/dd0/cd6 0 2024-05-27T09:29:46.299 INFO:tasks.workunit.client.1.smithi138.stdout:5/660: write d3/d12/d16/d1d/d23/d3b/f95 [580475,78857] 0 2024-05-27T09:29:46.300 INFO:tasks.workunit.client.1.smithi138.stdout:0/656: creat da/d2a/d8e/d43/fd6 x:0 0 0 2024-05-27T09:29:46.305 INFO:tasks.workunit.client.1.smithi138.stdout:6/818: rename d3/d11/d2d/d37/d7e/ld2 to d3/d11/l115 0 2024-05-27T09:29:46.312 INFO:tasks.workunit.client.1.smithi138.stdout:0/657: rmdir da/d2a/d8e/d6a/dc5/dd3 0 2024-05-27T09:29:46.316 INFO:tasks.workunit.client.1.smithi138.stdout:7/776: dwrite d0/fac [0,4194304] 0 2024-05-27T09:29:46.316 INFO:tasks.workunit.client.1.smithi138.stdout:7/777: write d0/d2b/daf/d98/db6/fdd [119191,15361] 0 2024-05-27T09:29:46.317 INFO:tasks.workunit.client.1.smithi138.stdout:6/819: dread d3/d11/d33/f78 [0,4194304] 0 2024-05-27T09:29:46.318 INFO:tasks.workunit.client.1.smithi138.stdout:7/778: mknod d0/d2b/daf/d43/cfc 0 2024-05-27T09:29:46.327 INFO:tasks.workunit.client.1.smithi138.stdout:4/732: dwrite d7/df/d35/f92 [0,4194304] 0 2024-05-27T09:29:46.333 INFO:tasks.workunit.client.1.smithi138.stdout:1/656: dwrite d4/d48/d39/f85 [0,4194304] 0 2024-05-27T09:29:46.333 INFO:tasks.workunit.client.1.smithi138.stdout:2/587: dwrite d1/d10/f76 [0,4194304] 0 2024-05-27T09:29:46.337 INFO:tasks.workunit.client.1.smithi138.stdout:3/651: dwrite d0/d3f/d50/f97 [0,4194304] 0 2024-05-27T09:29:46.337 INFO:tasks.workunit.client.1.smithi138.stdout:3/652: fsync d0/d2/da/df/d10/d2a/db2/dd1/fae 0 2024-05-27T09:29:46.337 INFO:tasks.workunit.client.1.smithi138.stdout:3/653: write d0/d2/da/fcf [532278,21037] 0 2024-05-27T09:29:46.342 INFO:tasks.workunit.client.1.smithi138.stdout:5/661: getdents d3/d12/d16/d1d/d22/d2b/daf/db2 0 2024-05-27T09:29:46.343 INFO:tasks.workunit.client.1.smithi138.stdout:3/654: symlink d0/d2/da/df/ldf 0 2024-05-27T09:29:46.343 INFO:tasks.workunit.client.1.smithi138.stdout:3/655: fdatasync d0/d3f/f55 0 2024-05-27T09:29:46.344 INFO:tasks.workunit.client.1.smithi138.stdout:7/779: write d0/d2b/daf/d64/f7d [3554377,81640] 0 2024-05-27T09:29:46.346 INFO:tasks.workunit.client.1.smithi138.stdout:3/656: truncate d0/d3f/d60/f95 291124 0 2024-05-27T09:29:46.352 INFO:tasks.workunit.client.1.smithi138.stdout:7/780: mknod d0/d23/d3a/de0/cfd 0 2024-05-27T09:29:46.353 INFO:tasks.workunit.client.1.smithi138.stdout:4/733: truncate d7/df/d35/d4c/d59/d5d/d64/d6a/f76 782990 0 2024-05-27T09:29:46.363 INFO:tasks.workunit.client.1.smithi138.stdout:8/770: dwrite d3/d8/d32/f49 [0,4194304] 0 2024-05-27T09:29:46.363 INFO:tasks.workunit.client.1.smithi138.stdout:8/771: fsync d3/d8/d64/f76 0 2024-05-27T09:29:46.363 INFO:tasks.workunit.client.1.smithi138.stdout:8/772: chown d3/f50 6296992 1 2024-05-27T09:29:46.367 INFO:tasks.workunit.client.1.smithi138.stdout:8/773: rename d3/d8/fac to d3/d8/d24/f10a 0 2024-05-27T09:29:46.367 INFO:tasks.workunit.client.1.smithi138.stdout:8/774: fsync d3/d8/d32/d3f/d9c/d9f/d86/f8b 0 2024-05-27T09:29:46.367 INFO:tasks.workunit.client.1.smithi138.stdout:8/775: truncate d3/d8/d24/d55/d94/da5/fb3 252926 0 2024-05-27T09:29:46.367 INFO:tasks.workunit.client.1.smithi138.stdout:8/776: write d3/d8/d11/deb/dd3/ff4 [590035,2903] 0 2024-05-27T09:29:46.368 INFO:tasks.workunit.client.1.smithi138.stdout:8/777: write d3/d8/d72/fde [779615,16469] 0 2024-05-27T09:29:46.368 INFO:tasks.workunit.client.1.smithi138.stdout:8/778: readlink d3/d8/d32/d34/d38/dd0/le9 0 2024-05-27T09:29:46.371 INFO:tasks.workunit.client.1.smithi138.stdout:6/820: dread d3/d11/dc0/d43/d8c/fa6 [0,4194304] 0 2024-05-27T09:29:46.372 INFO:tasks.workunit.client.1.smithi138.stdout:8/779: creat d3/d8/d32/dd6/f10b x:0 0 0 2024-05-27T09:29:46.381 INFO:tasks.workunit.client.1.smithi138.stdout:6/821: truncate d3/d11/dc0/d43/d1c/f4c 2629474 0 2024-05-27T09:29:46.381 INFO:tasks.workunit.client.1.smithi138.stdout:6/822: readlink d3/d11/d2d/d37/d8b/laf 0 2024-05-27T09:29:46.395 INFO:tasks.workunit.client.1.smithi138.stdout:1/657: dwrite d4/d1a/d2e/d32/d8c/dc6/f61 [0,4194304] 0 2024-05-27T09:29:46.395 INFO:tasks.workunit.client.1.smithi138.stdout:4/734: dread d7/dba/dcf/f2c [0,4194304] 0 2024-05-27T09:29:46.420 INFO:tasks.workunit.client.1.smithi138.stdout:3/657: dwrite d0/d2/da/df/d10/d32/f3c [0,4194304] 0 2024-05-27T09:29:46.420 INFO:tasks.workunit.client.1.smithi138.stdout:4/735: mknod d7/df/d5b/da1/c102 0 2024-05-27T09:29:46.422 INFO:tasks.workunit.client.1.smithi138.stdout:3/658: creat d0/d2/fe0 x:0 0 0 2024-05-27T09:29:46.422 INFO:tasks.workunit.client.1.smithi138.stdout:3/659: chown d0/d2/da/df/c74 5 1 2024-05-27T09:29:46.422 INFO:tasks.workunit.client.1.smithi138.stdout:3/660: dread - d0/d2/da/df/d10/d2a/d61/fb5 zero size 2024-05-27T09:29:46.422 INFO:tasks.workunit.client.1.smithi138.stdout:3/661: fdatasync d0/d2/da/df/d10/f77 0 2024-05-27T09:29:46.428 INFO:tasks.workunit.client.1.smithi138.stdout:5/662: dwrite d3/d12/d16/d1d/f24 [0,4194304] 0 2024-05-27T09:29:46.431 INFO:tasks.workunit.client.1.smithi138.stdout:0/658: dwrite da/d17/f8c [4194304,4194304] 0 2024-05-27T09:29:46.431 INFO:tasks.workunit.client.1.smithi138.stdout:0/659: write da/fcb [589484,109936] 0 2024-05-27T09:29:46.448 INFO:tasks.workunit.client.1.smithi138.stdout:2/588: dwrite d1/d2f/fb4 [0,4194304] 0 2024-05-27T09:29:46.448 INFO:tasks.workunit.client.1.smithi138.stdout:2/589: chown d1/d10/f76 488368 1 2024-05-27T09:29:46.455 INFO:tasks.workunit.client.1.smithi138.stdout:7/781: dwrite d0/d2b/daf/f12 [0,4194304] 0 2024-05-27T09:29:46.460 INFO:tasks.workunit.client.1.smithi138.stdout:6/823: dwrite d3/d11/dc0/fbd [0,4194304] 0 2024-05-27T09:29:46.463 INFO:tasks.workunit.client.1.smithi138.stdout:2/590: rmdir d1/d15/d2d/d1d 39 2024-05-27T09:29:46.470 INFO:tasks.workunit.client.1.smithi138.stdout:3/662: write d0/d3f/d50/f63 [2742369,125600] 0 2024-05-27T09:29:46.484 INFO:tasks.workunit.client.1.smithi138.stdout:4/736: dwrite d7/df/d35/d4c/d59/d5d/fee [0,4194304] 0 2024-05-27T09:29:46.490 INFO:tasks.workunit.client.1.smithi138.stdout:7/782: write d0/d23/da8/fa2 [4107230,94456] 0 2024-05-27T09:29:46.490 INFO:tasks.workunit.client.1.smithi138.stdout:7/783: mkdir d0/d23/d49/dfe 0 2024-05-27T09:29:46.490 INFO:tasks.workunit.client.1.smithi138.stdout:7/784: creat d0/d2b/daf/d43/fff x:0 0 0 2024-05-27T09:29:46.492 INFO:tasks.workunit.client.1.smithi138.stdout:1/658: dwrite d4/d48/d39/f54 [4194304,4194304] 0 2024-05-27T09:29:46.492 INFO:tasks.workunit.client.1.smithi138.stdout:1/659: readlink d4/d1a/d2e/l59 0 2024-05-27T09:29:46.493 INFO:tasks.workunit.client.1.smithi138.stdout:2/591: write d1/f7a [1762550,83144] 0 2024-05-27T09:29:46.494 INFO:tasks.workunit.client.1.smithi138.stdout:7/785: link d0/d2b/daf/d43/f7a d0/d23/da8/d8e/df4/f100 0 2024-05-27T09:29:46.498 INFO:tasks.workunit.client.1.smithi138.stdout:7/786: creat d0/d23/d49/dfe/f101 x:0 0 0 2024-05-27T09:29:46.502 INFO:tasks.workunit.client.1.smithi138.stdout:4/737: dread d7/d6e/fd1 [0,4194304] 0 2024-05-27T09:29:46.507 INFO:tasks.workunit.client.1.smithi138.stdout:3/663: write d0/d80/f92 [3085162,107699] 0 2024-05-27T09:29:46.507 INFO:tasks.workunit.client.1.smithi138.stdout:0/660: dwrite da/d2a/d55/fb4 [0,4194304] 0 2024-05-27T09:29:46.508 INFO:tasks.workunit.client.1.smithi138.stdout:3/664: rename d0/d2/da/df/d10/d2a/db2/dd1/dc0 to d0/d2/da/df/d10/d2a/d61/de1 0 2024-05-27T09:29:46.509 INFO:tasks.workunit.client.1.smithi138.stdout:8/780: dwrite d3/d8/d32/d3f/d9c/db8/ffa [0,4194304] 0 2024-05-27T09:29:46.511 INFO:tasks.workunit.client.1.smithi138.stdout:7/787: dread d0/d23/dc4/ddf/d91/fb4 [0,4194304] 0 2024-05-27T09:29:46.514 INFO:tasks.workunit.client.1.smithi138.stdout:7/788: readlink d0/d2b/daf/l2e 0 2024-05-27T09:29:46.517 INFO:tasks.workunit.client.1.smithi138.stdout:7/789: creat d0/d23/dc4/ddf/d91/f102 x:0 0 0 2024-05-27T09:29:46.517 INFO:tasks.workunit.client.1.smithi138.stdout:7/790: stat d0/d23/d49/d78/f9b 0 2024-05-27T09:29:46.518 INFO:tasks.workunit.client.1.smithi138.stdout:3/665: write d0/d2/da/df/d10/d2a/d48/f5f [229235,7164] 0 2024-05-27T09:29:46.519 INFO:tasks.workunit.client.1.smithi138.stdout:7/791: truncate d0/d23/d3c/d84/d9c/dc3/de6/feb 4039971 0 2024-05-27T09:29:46.519 INFO:tasks.workunit.client.1.smithi138.stdout:7/792: chown d0/d2b/daf/fdb 31 1 2024-05-27T09:29:46.524 INFO:tasks.workunit.client.1.smithi138.stdout:0/661: dread da/f66 [0,4194304] 0 2024-05-27T09:29:46.527 INFO:tasks.workunit.client.1.smithi138.stdout:3/666: write d0/d3f/f84 [658202,13157] 0 2024-05-27T09:29:46.528 INFO:tasks.workunit.client.1.smithi138.stdout:1/660: mknod d4/d48/d39/d5f/dd3/cd7 0 2024-05-27T09:29:46.528 INFO:tasks.workunit.client.1.smithi138.stdout:1/661: dread - d4/d48/d22/d65/fce zero size 2024-05-27T09:29:46.528 INFO:tasks.workunit.client.1.smithi138.stdout:1/662: fsync d4/f24 0 2024-05-27T09:29:46.529 INFO:tasks.workunit.client.1.smithi138.stdout:4/738: dread d7/d6e/faf [0,4194304] 0 2024-05-27T09:29:46.530 INFO:tasks.workunit.client.1.smithi138.stdout:8/781: dread d3/d8/d32/d34/f9a [0,4194304] 0 2024-05-27T09:29:46.532 INFO:tasks.workunit.client.1.smithi138.stdout:1/663: mkdir d4/d1a/d2e/d32/d8c/dc6/dd8 0 2024-05-27T09:29:46.532 INFO:tasks.workunit.client.1.smithi138.stdout:1/664: truncate d4/d48/d22/d65/f75 425925 0 2024-05-27T09:29:46.533 INFO:tasks.workunit.client.1.smithi138.stdout:1/665: chown d4/d1a/d2e/lb8 516621 1 2024-05-27T09:29:46.533 INFO:tasks.workunit.client.1.smithi138.stdout:1/666: fsync d4/d48/f14 0 2024-05-27T09:29:46.534 INFO:tasks.workunit.client.1.smithi138.stdout:4/739: creat d7/df/d35/d4c/d59/f103 x:0 0 0 2024-05-27T09:29:46.534 INFO:tasks.workunit.client.1.smithi138.stdout:8/782: truncate d3/d8/d32/d34/f9a 51928 0 2024-05-27T09:29:46.536 INFO:tasks.workunit.client.1.smithi138.stdout:8/783: rename d3/d8/d24/d55/d7e/de0 to d3/d8/d11/df1/d10c 0 2024-05-27T09:29:46.536 INFO:tasks.workunit.client.1.smithi138.stdout:8/784: readlink d3/d8/d32/d3f/d9c/d9f/l84 0 2024-05-27T09:29:46.536 INFO:tasks.workunit.client.1.smithi138.stdout:1/667: link d4/d48/d39/d5f/d63/lc4 d4/d3c/ld9 0 2024-05-27T09:29:46.537 INFO:tasks.workunit.client.1.smithi138.stdout:1/668: mknod d4/d1a/d2e/d32/d8c/cda 0 2024-05-27T09:29:46.540 INFO:tasks.workunit.client.1.smithi138.stdout:8/785: rename d3/d8/d32/d45/d5c/fdb to d3/d8/d32/dd6/f10d 0 2024-05-27T09:29:46.557 INFO:tasks.workunit.client.1.smithi138.stdout:5/663: truncate d3/d12/d16/d1d/f24 3699647 0 2024-05-27T09:29:46.563 INFO:tasks.workunit.client.1.smithi138.stdout:2/592: dwrite d1/d15/d2d/d17/d25/d9d/fb3 [0,4194304] 0 2024-05-27T09:29:46.567 INFO:tasks.workunit.client.1.smithi138.stdout:5/664: rename d3/d12/d26/f5d to d3/fd7 0 2024-05-27T09:29:46.572 INFO:tasks.workunit.client.1.smithi138.stdout:5/665: rename d3/d12/cad to d3/dab/dc2/dc6/cd8 0 2024-05-27T09:29:46.577 INFO:tasks.workunit.client.1.smithi138.stdout:5/666: mkdir d3/d12/d16/d1d/d22/dd9 0 2024-05-27T09:29:46.580 INFO:tasks.workunit.client.1.smithi138.stdout:8/786: dread f1 [0,4194304] 0 2024-05-27T09:29:46.595 INFO:tasks.workunit.client.1.smithi138.stdout:5/667: rename d3/d12/d16/d1d/d22/d2b/d9e/db6/f41 to d3/d12/d32/d60/fda 0 2024-05-27T09:29:46.621 INFO:tasks.workunit.client.1.smithi138.stdout:5/668: rmdir d3/d12/d16/d1d/d22/d2b/d9e/dcb 0 2024-05-27T09:29:46.624 INFO:tasks.workunit.client.1.smithi138.stdout:5/669: symlink d3/d12/d32/d87/dc0/ldb 0 2024-05-27T09:29:46.625 INFO:tasks.workunit.client.1.smithi138.stdout:7/793: dwrite d0/d2b/f35 [0,4194304] 0 2024-05-27T09:29:46.625 INFO:tasks.workunit.client.1.smithi138.stdout:8/787: dread d3/d8/d32/d34/f9a [0,4194304] 0 2024-05-27T09:29:46.629 INFO:tasks.workunit.client.1.smithi138.stdout:0/662: dwrite da/d2a/d8e/d43/d74/d97/fc2 [0,4194304] 0 2024-05-27T09:29:46.631 INFO:tasks.workunit.client.1.smithi138.stdout:7/794: mkdir d0/d23/d3c/d84/d9c/d103 0 2024-05-27T09:29:46.631 INFO:tasks.workunit.client.1.smithi138.stdout:7/795: fsync d0/d2b/daf/f6b 0 2024-05-27T09:29:46.632 INFO:tasks.workunit.client.1.smithi138.stdout:8/788: mknod d3/d8/d11/df1/d10c/c10e 0 2024-05-27T09:29:46.632 INFO:tasks.workunit.client.1.smithi138.stdout:8/789: fsync d3/d8/d32/d45/f4e 0 2024-05-27T09:29:46.634 INFO:tasks.workunit.client.1.smithi138.stdout:7/796: creat d0/d23/d3c/d84/d9c/dc3/de6/f104 x:0 0 0 2024-05-27T09:29:46.635 INFO:tasks.workunit.client.1.smithi138.stdout:1/669: dwrite d4/d38/f44 [0,4194304] 0 2024-05-27T09:29:46.635 INFO:tasks.workunit.client.1.smithi138.stdout:1/670: stat d4/d48/d39/l30 0 2024-05-27T09:29:46.635 INFO:tasks.workunit.client.1.smithi138.stdout:1/671: fdatasync d4/f18 0 2024-05-27T09:29:46.645 INFO:tasks.workunit.client.1.smithi138.stdout:7/797: dread d0/d23/f4f [0,4194304] 0 2024-05-27T09:29:46.645 INFO:tasks.workunit.client.1.smithi138.stdout:7/798: stat d0/d23 0 2024-05-27T09:29:46.650 INFO:tasks.workunit.client.1.smithi138.stdout:7/799: truncate d0/d23/d3a/f46 360707 0 2024-05-27T09:29:46.650 INFO:tasks.workunit.client.1.smithi138.stdout:7/800: chown d0/d2b/daf/f56 0 1 2024-05-27T09:29:46.650 INFO:tasks.workunit.client.1.smithi138.stdout:7/801: read d0/f10 [1964457,16221] 0 2024-05-27T09:29:46.654 INFO:tasks.workunit.client.1.smithi138.stdout:0/663: write da/d2a/d55/fb4 [4156686,77705] 0 2024-05-27T09:29:46.654 INFO:tasks.workunit.client.1.smithi138.stdout:7/802: creat d0/d23/da8/d8e/df4/f105 x:0 0 0 2024-05-27T09:29:46.683 INFO:tasks.workunit.client.1.smithi138.stdout:3/667: dwrite d0/d2/da/df/d10/d2a/db2/fbb [0,4194304] 0 2024-05-27T09:29:46.689 INFO:tasks.workunit.client.1.smithi138.stdout:1/672: dwrite d4/d48/d39/f54 [4194304,4194304] 0 2024-05-27T09:29:46.691 INFO:tasks.workunit.client.1.smithi138.stdout:0/664: read f1 [8314308,78244] 0 2024-05-27T09:29:46.693 INFO:tasks.workunit.client.1.smithi138.stdout:0/665: mknod da/d2a/d8e/d43/d74/d97/cd7 0 2024-05-27T09:29:46.695 INFO:tasks.workunit.client.1.smithi138.stdout:0/666: creat da/d2a/d8e/d6a/fd8 x:0 0 0 2024-05-27T09:29:46.699 INFO:tasks.workunit.client.1.smithi138.stdout:3/668: dread d0/d2/da/df/d10/f9d [0,4194304] 0 2024-05-27T09:29:46.699 INFO:tasks.workunit.client.1.smithi138.stdout:0/667: mknod da/d2a/d8e/d1f/d42/d63/cd9 0 2024-05-27T09:29:46.699 INFO:tasks.workunit.client.1.smithi138.stdout:0/668: chown da/d2a/cd0 401 1 2024-05-27T09:29:46.700 INFO:tasks.workunit.client.1.smithi138.stdout:3/669: getdents d0/d2/da/df/d10 0 2024-05-27T09:29:46.700 INFO:tasks.workunit.client.1.smithi138.stdout:0/669: mkdir da/d2a/d8e/dda 0 2024-05-27T09:29:46.700 INFO:tasks.workunit.client.1.smithi138.stdout:3/670: fsync d0/d2/da/df/f6f 0 2024-05-27T09:29:46.701 INFO:tasks.workunit.client.1.smithi138.stdout:1/673: dread d4/fa2 [0,4194304] 0 2024-05-27T09:29:46.701 INFO:tasks.workunit.client.1.smithi138.stdout:1/674: fsync d4/d1a/d2e/d32/d8c/dc6/f7b 0 2024-05-27T09:29:46.702 INFO:tasks.workunit.client.1.smithi138.stdout:0/670: unlink da/d2a/f30 0 2024-05-27T09:29:46.703 INFO:tasks.workunit.client.1.smithi138.stdout:0/671: write da/d2a/d8e/d1f/f7b [960450,81055] 0 2024-05-27T09:29:46.703 INFO:tasks.workunit.client.1.smithi138.stdout:1/675: mknod d4/cdb 0 2024-05-27T09:29:46.705 INFO:tasks.workunit.client.1.smithi138.stdout:1/676: mknod d4/d3c/d8a/cdc 0 2024-05-27T09:29:46.705 INFO:tasks.workunit.client.1.smithi138.stdout:1/677: truncate d4/d48/d39/d5f/d63/f98 364160 0 2024-05-27T09:29:46.705 INFO:tasks.workunit.client.1.smithi138.stdout:1/678: chown d4/cdb 277 1 2024-05-27T09:29:46.707 INFO:tasks.workunit.client.1.smithi138.stdout:1/679: creat d4/d48/d39/d5f/d4d/d2b/fdd x:0 0 0 2024-05-27T09:29:46.707 INFO:tasks.workunit.client.1.smithi138.stdout:1/680: truncate d4/d48/d22/d65/fce 264158 0 2024-05-27T09:29:46.708 INFO:tasks.workunit.client.1.smithi138.stdout:7/803: dwrite d0/d2b/daf/d43/ffb [0,4194304] 0 2024-05-27T09:29:46.709 INFO:tasks.workunit.client.1.smithi138.stdout:7/804: write d0/d2b/daf/d98/db6/ff2 [345506,55991] 0 2024-05-27T09:29:46.709 INFO:tasks.workunit.client.1.smithi138.stdout:5/670: dwrite d3/d12/d16/d1d/d22/d2b/d9e/db6/f43 [0,4194304] 0 2024-05-27T09:29:46.709 INFO:tasks.workunit.client.1.smithi138.stdout:5/671: getdents d3/d12/d16/d1d/d22/d2b/daf/db2 0 2024-05-27T09:29:46.709 INFO:tasks.workunit.client.1.smithi138.stdout:5/672: stat d3/d12/d32/f4e 0 2024-05-27T09:29:46.713 INFO:tasks.workunit.client.1.smithi138.stdout:7/805: symlink d0/l106 0 2024-05-27T09:29:46.715 INFO:tasks.workunit.client.1.smithi138.stdout:5/673: mkdir d3/ddc 0 2024-05-27T09:29:46.717 INFO:tasks.workunit.client.1.smithi138.stdout:5/674: creat d3/d12/d16/d1d/d23/d68/fdd x:0 0 0 2024-05-27T09:29:46.725 INFO:tasks.workunit.client.1.smithi138.stdout:5/675: mknod d3/d12/cde 0 2024-05-27T09:29:46.725 INFO:tasks.workunit.client.1.smithi138.stdout:5/676: stat d3/d12/d26/lb8 0 2024-05-27T09:29:46.726 INFO:tasks.workunit.client.1.smithi138.stdout:0/672: getdents da/d2a/d8e/d1f/d42/d63 0 2024-05-27T09:29:46.728 INFO:tasks.workunit.client.1.smithi138.stdout:9/681: sync 2024-05-27T09:29:46.733 INFO:tasks.workunit.client.1.smithi138.stdout:0/673: creat da/d2a/d8e/d1f/d4f/fdb x:0 0 0 2024-05-27T09:29:46.733 INFO:tasks.workunit.client.1.smithi138.stdout:9/682: link d6/da/d29/d8f/db7/ca3 d6/ce7 0 2024-05-27T09:29:46.733 INFO:tasks.workunit.client.1.smithi138.stdout:9/683: chown d6/da/l36 23599700 1 2024-05-27T09:29:46.737 INFO:tasks.workunit.client.1.smithi138.stdout:9/684: mkdir d6/db/de/d18/ddb/de8 0 2024-05-27T09:29:46.737 INFO:tasks.workunit.client.1.smithi138.stdout:9/685: fdatasync d6/f7c 0 2024-05-27T09:29:46.737 INFO:tasks.workunit.client.1.smithi138.stdout:9/686: stat d6/db/de/d18/d24/d72/c95 0 2024-05-27T09:29:46.739 INFO:tasks.workunit.client.1.smithi138.stdout:9/687: creat d6/da/fe9 x:0 0 0 2024-05-27T09:29:46.743 INFO:tasks.workunit.client.1.smithi138.stdout:9/688: creat d6/da/d26/d28/fea x:0 0 0 2024-05-27T09:29:46.743 INFO:tasks.workunit.client.1.smithi138.stdout:9/689: readlink d6/db/l56 0 2024-05-27T09:29:46.743 INFO:tasks.workunit.client.1.smithi138.stdout:9/690: fdatasync d6/da/d29/d8f/db7/dbd/fc5 0 2024-05-27T09:29:46.743 INFO:tasks.workunit.client.1.smithi138.stdout:9/691: truncate d6/da/d29/d59/f90 1012607 0 2024-05-27T09:29:46.749 INFO:tasks.workunit.client.1.smithi138.stdout:9/692: truncate d6/db/de/d18/f64 1608858 0 2024-05-27T09:29:46.764 INFO:tasks.workunit.client.1.smithi138.stdout:7/806: dwrite d0/fac [0,4194304] 0 2024-05-27T09:29:46.768 INFO:tasks.workunit.client.1.smithi138.stdout:7/807: creat d0/d23/d49/f107 x:0 0 0 2024-05-27T09:29:46.781 INFO:tasks.workunit.client.1.smithi138.stdout:5/677: dwrite d3/d12/d32/d6b/fd2 [0,4194304] 0 2024-05-27T09:29:46.787 INFO:tasks.workunit.client.1.smithi138.stdout:5/678: mkdir d3/d12/d16/d1d/d22/db9/ddf 0 2024-05-27T09:29:46.792 INFO:tasks.workunit.client.1.smithi138.stdout:0/674: dwrite da/d2a/d8e/d1f/d4f/f6f [0,4194304] 0 2024-05-27T09:29:46.801 INFO:tasks.workunit.client.1.smithi138.stdout:0/675: creat da/d2a/d55/fdc x:0 0 0 2024-05-27T09:29:46.810 INFO:tasks.workunit.client.1.smithi138.stdout:0/676: rename da/d2a/d8e/d1f/d71/lbe to da/d2a/d8e/d1f/d42/d4a/d53/ldd 0 2024-05-27T09:29:46.818 INFO:tasks.workunit.client.1.smithi138.stdout:7/808: dwrite d0/d2b/f40 [0,4194304] 0 2024-05-27T09:29:46.820 INFO:tasks.workunit.client.1.smithi138.stdout:7/809: rmdir d0/d23/d49/df6 0 2024-05-27T09:29:46.820 INFO:tasks.workunit.client.1.smithi138.stdout:7/810: chown d0/d23/dc4/ddf/d91 666 1 2024-05-27T09:29:46.822 INFO:tasks.workunit.client.1.smithi138.stdout:7/811: mkdir d0/d23/dc4/ddf/d91/d108 0 2024-05-27T09:29:46.822 INFO:tasks.workunit.client.1.smithi138.stdout:7/812: stat d0 0 2024-05-27T09:29:46.824 INFO:tasks.workunit.client.1.smithi138.stdout:7/813: creat d0/d23/d3c/d84/d9c/f109 x:0 0 0 2024-05-27T09:29:46.850 INFO:tasks.workunit.client.1.smithi138.stdout:9/693: dwrite d6/f9 [0,4194304] 0 2024-05-27T09:29:46.863 INFO:tasks.workunit.client.1.smithi138.stdout:7/814: dread d0/d2b/daf/fdb [0,4194304] 0 2024-05-27T09:29:46.863 INFO:tasks.workunit.client.1.smithi138.stdout:7/815: read - d0/d23/d49/fbb zero size 2024-05-27T09:29:46.867 INFO:tasks.workunit.client.1.smithi138.stdout:5/679: dwrite d3/d12/f79 [0,4194304] 0 2024-05-27T09:29:46.868 INFO:tasks.workunit.client.1.smithi138.stdout:5/680: chown d3/d12/d32/d6b/fca 16555743 1 2024-05-27T09:29:46.871 INFO:tasks.workunit.client.1.smithi138.stdout:5/681: mknod d3/d12/d16/d1d/ce0 0 2024-05-27T09:29:46.875 INFO:tasks.workunit.client.1.smithi138.stdout:7/816: rename d0/d2b/daf/d43/f7a to d0/d2b/daf/d43/f10a 0 2024-05-27T09:29:46.875 INFO:tasks.workunit.client.1.smithi138.stdout:7/817: mknod d0/d2b/daf/d98/db6/c10b 0 2024-05-27T09:29:46.880 INFO:tasks.workunit.client.1.smithi138.stdout:7/818: write d0/d2b/f5f [1677875,11953] 0 2024-05-27T09:29:46.882 INFO:tasks.workunit.client.1.smithi138.stdout:7/819: rename d0/d2b/daf/l2e to d0/d23/d3c/d84/d9c/d103/l10c 0 2024-05-27T09:29:46.886 INFO:tasks.workunit.client.1.smithi138.stdout:7/820: rename d0/d2b/daf/c8f to d0/d23/dc4/ddf/d91/db8/c10d 0 2024-05-27T09:29:46.886 INFO:tasks.workunit.client.1.smithi138.stdout:7/821: getdents d0/d2b/daf/dbf/dc8 0 2024-05-27T09:29:46.898 INFO:tasks.workunit.client.1.smithi138.stdout:5/682: write d3/d12/d32/f4e [1230686,118929] 0 2024-05-27T09:29:46.901 INFO:tasks.workunit.client.1.smithi138.stdout:5/683: mknod d3/d12/dcf/ce1 0 2024-05-27T09:29:46.903 INFO:tasks.workunit.client.1.smithi138.stdout:5/684: mkdir d3/d12/d32/d87/d89/dc3/de2 0 2024-05-27T09:29:46.903 INFO:tasks.workunit.client.1.smithi138.stdout:5/685: write d3/d12/d32/d60/f82 [604742,37850] 0 2024-05-27T09:29:46.904 INFO:tasks.workunit.client.1.smithi138.stdout:5/686: rmdir d3/d12/d32/d87/d89/dc3 39 2024-05-27T09:29:46.907 INFO:tasks.workunit.client.1.smithi138.stdout:5/687: mknod d3/d12/d26/d53/ce3 0 2024-05-27T09:29:46.946 INFO:tasks.workunit.client.1.smithi138.stdout:9/694: dwrite d6/da/d2f/d35/d55/f3a [0,4194304] 0 2024-05-27T09:29:46.952 INFO:tasks.workunit.client.1.smithi138.stdout:5/688: write d3/d12/d26/f45 [825755,86651] 0 2024-05-27T09:29:46.955 INFO:tasks.workunit.client.1.smithi138.stdout:5/689: creat d3/dab/dc2/dd5/fe4 x:0 0 0 2024-05-27T09:29:46.956 INFO:tasks.workunit.client.1.smithi138.stdout:9/695: dread d6/da/d2f/d35/d55/f76 [0,4194304] 0 2024-05-27T09:29:46.958 INFO:tasks.workunit.client.1.smithi138.stdout:9/696: mknod d6/ceb 0 2024-05-27T09:29:46.960 INFO:tasks.workunit.client.1.smithi138.stdout:9/697: creat d6/db/dc/dca/fec x:0 0 0 2024-05-27T09:29:46.960 INFO:tasks.workunit.client.1.smithi138.stdout:9/698: fsync d6/db/dc/f12 0 2024-05-27T09:29:46.961 INFO:tasks.workunit.client.1.smithi138.stdout:9/699: chown d6/da/d2f/f40 0 1 2024-05-27T09:29:46.961 INFO:tasks.workunit.client.1.smithi138.stdout:9/700: fdatasync d6/db/dc/f4d 0 2024-05-27T09:29:46.961 INFO:tasks.workunit.client.1.smithi138.stdout:9/701: creat d6/da/d26/fed x:0 0 0 2024-05-27T09:29:46.966 INFO:tasks.workunit.client.1.smithi138.stdout:9/702: rename d6/da/d2f/d35 to d6/da/d29/d59/dee 0 2024-05-27T09:29:46.966 INFO:tasks.workunit.client.1.smithi138.stdout:9/703: chown d6/db/dc/l7d 763957784 1 2024-05-27T09:29:46.968 INFO:tasks.workunit.client.1.smithi138.stdout:9/704: creat d6/fef x:0 0 0 2024-05-27T09:29:46.978 INFO:tasks.workunit.client.1.smithi138.stdout:9/705: dread d6/da/f5d [0,4194304] 0 2024-05-27T09:29:46.981 INFO:tasks.workunit.client.1.smithi138.stdout:9/706: symlink d6/db/dc/lf0 0 2024-05-27T09:29:47.041 INFO:tasks.workunit.client.1.smithi138.stdout:7/822: dwrite d0/d2b/daf/d43/f67 [4194304,4194304] 0 2024-05-27T09:29:47.043 INFO:tasks.workunit.client.1.smithi138.stdout:7/823: rmdir d0/d23/d3c 39 2024-05-27T09:29:47.046 INFO:tasks.workunit.client.1.smithi138.stdout:5/690: dwrite d3/d12/d26/d53/f85 [4194304,4194304] 0 2024-05-27T09:29:47.046 INFO:tasks.workunit.client.1.smithi138.stdout:5/691: chown d3/d12/d16/d1d/d22 73284 1 2024-05-27T09:29:47.054 INFO:tasks.workunit.client.1.smithi138.stdout:9/707: dwrite d6/db/de/d18/d24/f73 [0,4194304] 0 2024-05-27T09:29:47.060 INFO:tasks.workunit.client.1.smithi138.stdout:9/708: creat d6/da/d29/d8f/db7/dbd/d46/ff1 x:0 0 0 2024-05-27T09:29:47.068 INFO:tasks.workunit.client.1.smithi138.stdout:9/709: link d6/da/fa9 d6/da/d29/ff2 0 2024-05-27T09:29:47.068 INFO:tasks.workunit.client.1.smithi138.stdout:9/710: truncate d6/da/d29/d59/dee/d55/d75/d8c/fb2 5485905 0 2024-05-27T09:29:47.068 INFO:tasks.workunit.client.1.smithi138.stdout:9/711: chown d6/da/d29/d59/dee/d55/d75/d8c/fb2 4 1 2024-05-27T09:29:47.068 INFO:tasks.workunit.client.1.smithi138.stdout:9/712: truncate d6/db/de/d18/fbe 231872 0 2024-05-27T09:29:47.069 INFO:tasks.workunit.client.1.smithi138.stdout:9/713: mknod d6/dc8/cf3 0 2024-05-27T09:29:47.074 INFO:tasks.workunit.client.1.smithi138.stdout:5/692: dread d3/d12/d26/fa6 [0,4194304] 0 2024-05-27T09:29:47.094 INFO:tasks.workunit.client.1.smithi138.stdout:7/824: dwrite d0/d23/dc4/ddf/d91/db8/fef [0,4194304] 0 2024-05-27T09:29:47.100 INFO:tasks.workunit.client.1.smithi138.stdout:7/825: symlink d0/d23/l10e 0 2024-05-27T09:29:47.103 INFO:tasks.workunit.client.1.smithi138.stdout:7/826: mknod d0/df7/c10f 0 2024-05-27T09:29:47.110 INFO:tasks.workunit.client.1.smithi138.stdout:7/827: creat d0/f110 x:0 0 0 2024-05-27T09:29:47.110 INFO:tasks.workunit.client.1.smithi138.stdout:7/828: chown d0/d23/d3a/cc0 1791 1 2024-05-27T09:29:47.110 INFO:tasks.workunit.client.1.smithi138.stdout:7/829: write d0/d2b/daf/f7c [1008510,112616] 0 2024-05-27T09:29:47.143 INFO:tasks.workunit.client.1.smithi138.stdout:9/714: dwrite d6/db/de/d18/d24/f4b [0,4194304] 0 2024-05-27T09:29:47.148 INFO:tasks.workunit.client.1.smithi138.stdout:5/693: dwrite d3/d12/d16/d1d/d23/fd3 [0,4194304] 0 2024-05-27T09:29:47.149 INFO:tasks.workunit.client.1.smithi138.stdout:5/694: write d3/d12/d16/d1d/d23/f62 [588202,14545] 0 2024-05-27T09:29:47.149 INFO:tasks.workunit.client.1.smithi138.stdout:5/695: write d3/f15 [1603347,114461] 0 2024-05-27T09:29:47.171 INFO:tasks.workunit.client.1.smithi138.stdout:5/696: dread d3/d12/d16/d1d/d23/d3b/f7c [0,4194304] 0 2024-05-27T09:29:47.236 INFO:tasks.workunit.client.1.smithi138.stdout:5/697: dwrite d3/d12/d16/d1d/d30/f9c [0,4194304] 0 2024-05-27T09:29:47.253 INFO:tasks.workunit.client.1.smithi138.stdout:5/698: dread d3/d12/d32/f4f [0,4194304] 0 2024-05-27T09:29:47.255 INFO:tasks.workunit.client.1.smithi138.stdout:5/699: mknod d3/d12/d16/d1d/d22/d2b/ce5 0 2024-05-27T09:29:47.292 INFO:tasks.workunit.client.1.smithi138.stdout:4/740: sync 2024-05-27T09:29:47.292 INFO:tasks.workunit.client.1.smithi138.stdout:6/824: sync 2024-05-27T09:29:47.292 INFO:tasks.workunit.client.1.smithi138.stdout:2/593: sync 2024-05-27T09:29:47.301 INFO:tasks.workunit.client.1.smithi138.stdout:4/741: dread d7/df/f2e [0,4194304] 0 2024-05-27T09:29:47.302 INFO:tasks.workunit.client.1.smithi138.stdout:4/742: truncate d7/f39 2595134 0 2024-05-27T09:29:47.305 INFO:tasks.workunit.client.1.smithi138.stdout:4/743: rmdir d7/df/d35/d4c/d59/d5d/d64/dc5 39 2024-05-27T09:29:47.306 INFO:tasks.workunit.client.1.smithi138.stdout:4/744: symlink d7/dba/l104 0 2024-05-27T09:29:47.308 INFO:tasks.workunit.client.1.smithi138.stdout:4/745: rmdir d7/dba/dfa 39 2024-05-27T09:29:47.310 INFO:tasks.workunit.client.1.smithi138.stdout:2/594: write d1/d15/d2d/d17/d25/f28 [531123,103474] 0 2024-05-27T09:29:47.310 INFO:tasks.workunit.client.1.smithi138.stdout:2/595: write d1/d15/d2d/d17/d25/d4d/d4e/f9e [1373927,47600] 0 2024-05-27T09:29:47.310 INFO:tasks.workunit.client.1.smithi138.stdout:2/596: read - d1/d15/d2d/d17/d25/d9d/fc7 zero size 2024-05-27T09:29:47.310 INFO:tasks.workunit.client.1.smithi138.stdout:4/746: mkdir d7/df/d35/d4c/d59/d5d/d64/df6/d105 0 2024-05-27T09:29:47.314 INFO:tasks.workunit.client.1.smithi138.stdout:2/597: mknod d1/d15/d2d/d17/d25/d4d/d6d/ccb 0 2024-05-27T09:29:47.323 INFO:tasks.workunit.client.1.smithi138.stdout:2/598: rename d1/d2f to d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc 0 2024-05-27T09:29:47.327 INFO:tasks.workunit.client.1.smithi138.stdout:5/700: dwrite d3/d12/d32/d6b/f78 [0,4194304] 0 2024-05-27T09:29:47.348 INFO:tasks.workunit.client.1.smithi138.stdout:4/747: write d7/df/d35/d4c/f69 [6673511,36265] 0 2024-05-27T09:29:47.348 INFO:tasks.workunit.client.1.smithi138.stdout:4/748: truncate d7/dba/dcf/d3a/fb0 2087242 0 2024-05-27T09:29:47.348 INFO:tasks.workunit.client.1.smithi138.stdout:5/701: read d3/fd7 [978529,59855] 0 2024-05-27T09:29:47.353 INFO:tasks.workunit.client.1.smithi138.stdout:4/749: rename d7/f39 to d7/d6e/df8/f106 0 2024-05-27T09:29:47.355 INFO:tasks.workunit.client.1.smithi138.stdout:4/750: rename d7/df/d35/f92 to d7/d6e/df8/f107 0 2024-05-27T09:29:47.365 INFO:tasks.workunit.client.1.smithi138.stdout:6/825: dwrite d3/d19/f1e [0,4194304] 0 2024-05-27T09:29:47.367 INFO:tasks.workunit.client.1.smithi138.stdout:6/826: truncate d3/d11/d2d/d37/d7e/f9f 3396493 0 2024-05-27T09:29:47.369 INFO:tasks.workunit.client.1.smithi138.stdout:2/599: dwrite d1/d15/d2d/d1d/fca [0,4194304] 0 2024-05-27T09:29:47.369 INFO:tasks.workunit.client.1.smithi138.stdout:2/600: stat d1/d15/d2d/d7b 0 2024-05-27T09:29:47.371 INFO:tasks.workunit.client.1.smithi138.stdout:6/827: rmdir d3/d11/d2d/d37/dd1/d9b/d10a 39 2024-05-27T09:29:47.371 INFO:tasks.workunit.client.1.smithi138.stdout:6/828: read d3/d11/dc0/d43/f29 [2525801,54164] 0 2024-05-27T09:29:47.371 INFO:tasks.workunit.client.1.smithi138.stdout:6/829: write d3/d11/dc0/d5a/d6a/fac [676357,63390] 0 2024-05-27T09:29:47.374 INFO:tasks.workunit.client.1.smithi138.stdout:6/830: creat d3/d11/dc0/d5a/f116 x:0 0 0 2024-05-27T09:29:47.374 INFO:tasks.workunit.client.1.smithi138.stdout:6/831: dread - d3/d11/dc0/d43/ffa zero size 2024-05-27T09:29:47.374 INFO:tasks.workunit.client.1.smithi138.stdout:6/832: readlink d3/d11/dc0/d43/d8c/la2 0 2024-05-27T09:29:47.374 INFO:tasks.workunit.client.1.smithi138.stdout:6/833: chown d3/d11/dc0/d5a/fff 2676 1 2024-05-27T09:29:47.382 INFO:tasks.workunit.client.1.smithi138.stdout:2/601: rename d1/d15/d2d/c2c to d1/d15/d2d/d17/d25/d4d/d6d/d87/ccd 0 2024-05-27T09:29:47.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:47 smithi138 ceph-mon[34885]: pgmap v21: 65 pgs: 65 active+clean; 3.3 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 162 MiB/s rd, 197 MiB/s wr, 291 op/s 2024-05-27T09:29:47.386 INFO:tasks.workunit.client.1.smithi138.stdout:2/602: symlink d1/d15/d2d/lce 0 2024-05-27T09:29:47.393 INFO:tasks.workunit.client.1.smithi138.stdout:4/751: dread d7/dba/dcf/f34 [0,4194304] 0 2024-05-27T09:29:47.393 INFO:tasks.workunit.client.1.smithi138.stdout:4/752: creat d7/df/d35/d4c/d59/f108 x:0 0 0 2024-05-27T09:29:47.393 INFO:tasks.workunit.client.1.smithi138.stdout:4/753: fsync d7/fdc 0 2024-05-27T09:29:47.397 INFO:tasks.workunit.client.1.smithi138.stdout:4/754: mkdir d7/df/d35/d4c/d59/d5d/d64/dc5/d109 0 2024-05-27T09:29:47.400 INFO:tasks.workunit.client.1.smithi138.stdout:4/755: rename d7/df/f101 to d7/df/d35/d4c/d59/d5d/d64/dc5/d109/f10a 0 2024-05-27T09:29:47.400 INFO:tasks.workunit.client.1.smithi138.stdout:4/756: dread - d7/df/d35/d4c/d59/d5d/d64/d6a/feb zero size 2024-05-27T09:29:47.400 INFO:tasks.workunit.client.1.smithi138.stdout:4/757: readlink d7/df/d5b/da1/lc6 0 2024-05-27T09:29:47.401 INFO:tasks.workunit.client.1.smithi138.stdout:4/758: dread - d7/df/d35/f77 zero size 2024-05-27T09:29:47.405 INFO:tasks.workunit.client.1.smithi138.stdout:4/759: mknod d7/dba/dcf/daa/dda/dfc/c10b 0 2024-05-27T09:29:47.406 INFO:tasks.workunit.client.1.smithi138.stdout:6/834: dread d3/d19/fab [0,4194304] 0 2024-05-27T09:29:47.410 INFO:tasks.workunit.client.1.smithi138.stdout:4/760: mknod d7/dba/dcf/daa/def/c10c 0 2024-05-27T09:29:47.411 INFO:tasks.workunit.client.1.smithi138.stdout:4/761: chown d7/dba/dcf/daa/fe0 246670 1 2024-05-27T09:29:47.413 INFO:tasks.workunit.client.1.smithi138.stdout:6/835: truncate d3/fae 3605389 0 2024-05-27T09:29:47.417 INFO:tasks.workunit.client.1.smithi138.stdout:6/836: unlink d3/d11/dc0/d5a/d6a/f6b 0 2024-05-27T09:29:47.417 INFO:tasks.workunit.client.1.smithi138.stdout:6/837: write d3/d11/fd0 [1144854,62980] 0 2024-05-27T09:29:47.418 INFO:tasks.workunit.client.1.smithi138.stdout:4/762: read d7/df/d5b/f89 [27469,62353] 0 2024-05-27T09:29:47.421 INFO:tasks.workunit.client.1.smithi138.stdout:5/702: dwrite d3/d12/d16/d1d/d22/d2b/d9e/fbe [0,4194304] 0 2024-05-27T09:29:47.423 INFO:tasks.workunit.client.1.smithi138.stdout:6/838: rename d3/d11/dc0/d74/de0/cec to d3/d11/d2d/d37/daa/df3/c117 0 2024-05-27T09:29:47.423 INFO:tasks.workunit.client.1.smithi138.stdout:6/839: truncate d3/d11/d2d/d37/d8b/fc7 858932 0 2024-05-27T09:29:47.423 INFO:tasks.workunit.client.1.smithi138.stdout:6/840: truncate d3/d11/f103 504273 0 2024-05-27T09:29:47.423 INFO:tasks.workunit.client.1.smithi138.stdout:4/763: mkdir d7/df/d35/d4c/d59/d5d/d64/df6/d10d 0 2024-05-27T09:29:47.429 INFO:tasks.workunit.client.1.smithi138.stdout:5/703: mkdir d3/d12/d32/d60/de6 0 2024-05-27T09:29:47.432 INFO:tasks.workunit.client.1.smithi138.stdout:5/704: mknod d3/d75/db3/ce7 0 2024-05-27T09:29:47.432 INFO:tasks.workunit.client.1.smithi138.stdout:5/705: fdatasync d3/d12/d16/d1d/d22/d2b/d9e/fbe 0 2024-05-27T09:29:47.434 INFO:tasks.workunit.client.1.smithi138.stdout:4/764: dread d7/df/d35/d4c/d59/d5d/d64/d6a/fbe [0,4194304] 0 2024-05-27T09:29:47.435 INFO:tasks.workunit.client.1.smithi138.stdout:5/706: rename d3/d12/d16/d1d/f55 to d3/d12/d26/fe8 0 2024-05-27T09:29:47.437 INFO:tasks.workunit.client.1.smithi138.stdout:4/765: rename d7/df/d35/d4c/le4 to d7/df/d35/d4c/l10e 0 2024-05-27T09:29:47.439 INFO:tasks.workunit.client.1.smithi138.stdout:5/707: unlink d3/d75/f94 0 2024-05-27T09:29:47.441 INFO:tasks.workunit.client.1.smithi138.stdout:5/708: rmdir d3/d12/d32/d60/d93/dd0 39 2024-05-27T09:29:47.441 INFO:tasks.workunit.client.1.smithi138.stdout:5/709: fsync d3/d12/d16/d1d/d30/f8b 0 2024-05-27T09:29:47.445 INFO:tasks.workunit.client.1.smithi138.stdout:6/841: dread d3/d11/d2d/d37/f4a [0,4194304] 0 2024-05-27T09:29:47.445 INFO:tasks.workunit.client.1.smithi138.stdout:6/842: chown d3/d11/dc0/d43/d107 19485 1 2024-05-27T09:29:47.445 INFO:tasks.workunit.client.1.smithi138.stdout:5/710: rmdir d3/d12/d32/d87/d89/dc3 39 2024-05-27T09:29:47.448 INFO:tasks.workunit.client.1.smithi138.stdout:6/843: mkdir d3/d11/dc0/d5a/df0/d118 0 2024-05-27T09:29:47.463 INFO:tasks.workunit.client.1.smithi138.stdout:2/603: dwrite d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc/f33 [4194304,4194304] 0 2024-05-27T09:29:47.463 INFO:tasks.workunit.client.1.smithi138.stdout:2/604: write d1/d15/d2d/d17/d25/d9d/d4b/fbe [768942,101176] 0 2024-05-27T09:29:47.463 INFO:tasks.workunit.client.1.smithi138.stdout:6/844: write d3/d19/f41 [1693637,7324] 0 2024-05-27T09:29:47.468 INFO:tasks.workunit.client.1.smithi138.stdout:6/845: rmdir d3/d11/dc0/d43/dbe/d112 0 2024-05-27T09:29:47.475 INFO:tasks.workunit.client.1.smithi138.stdout:6/846: rename d3/d11/dc0/d43/f48 to d3/d11/d2d/d37/d7e/f119 0 2024-05-27T09:29:47.475 INFO:tasks.workunit.client.1.smithi138.stdout:6/847: chown d3/d11/d2d/d37/dd1/d9b/c9d 65414 1 2024-05-27T09:29:47.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:47 smithi119 ceph-mon[29082]: pgmap v21: 65 pgs: 65 active+clean; 3.3 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 162 MiB/s rd, 197 MiB/s wr, 291 op/s 2024-05-27T09:29:47.498 INFO:tasks.workunit.client.1.smithi138.stdout:5/711: dwrite d3/d12/d32/d60/fda [0,4194304] 0 2024-05-27T09:29:47.498 INFO:tasks.workunit.client.1.smithi138.stdout:5/712: truncate d3/d12/d16/d1d/d23/fba 909297 0 2024-05-27T09:29:47.498 INFO:tasks.workunit.client.1.smithi138.stdout:5/713: chown d3/d12/d32/d87 519913682 1 2024-05-27T09:29:47.499 INFO:tasks.workunit.client.1.smithi138.stdout:5/714: creat d3/d12/d16/d1d/d22/d2b/d9e/db6/fe9 x:0 0 0 2024-05-27T09:29:47.499 INFO:tasks.workunit.client.1.smithi138.stdout:5/715: write d3/d12/d16/d1d/d30/f9c [4881256,64438] 0 2024-05-27T09:29:47.503 INFO:tasks.workunit.client.1.smithi138.stdout:1/681: sync 2024-05-27T09:29:47.503 INFO:tasks.workunit.client.1.smithi138.stdout:3/671: sync 2024-05-27T09:29:47.508 INFO:tasks.workunit.client.1.smithi138.stdout:0/677: sync 2024-05-27T09:29:47.508 INFO:tasks.workunit.client.1.smithi138.stdout:8/790: sync 2024-05-27T09:29:47.508 INFO:tasks.workunit.client.1.smithi138.stdout:3/672: creat d0/d2/da/df/fe2 x:0 0 0 2024-05-27T09:29:47.508 INFO:tasks.workunit.client.1.smithi138.stdout:3/673: chown d0/d2/da/df/c2c 203856 1 2024-05-27T09:29:47.508 INFO:tasks.workunit.client.1.smithi138.stdout:6/848: dread d3/d11/dc0/d43/dbe/fe4 [0,4194304] 0 2024-05-27T09:29:47.511 INFO:tasks.workunit.client.1.smithi138.stdout:8/791: creat d3/d8/d32/d34/f10f x:0 0 0 2024-05-27T09:29:47.513 INFO:tasks.workunit.client.1.smithi138.stdout:8/792: symlink d3/d8/d32/dd6/l110 0 2024-05-27T09:29:47.518 INFO:tasks.workunit.client.1.smithi138.stdout:8/793: mkdir d3/d8/d24/d55/d94/da5/d111 0 2024-05-27T09:29:47.518 INFO:tasks.workunit.client.1.smithi138.stdout:0/678: rename f5 to da/d2a/d8e/d1f/fde 0 2024-05-27T09:29:47.521 INFO:tasks.workunit.client.1.smithi138.stdout:0/679: write da/d2a/d55/fdc [553734,47174] 0 2024-05-27T09:29:47.521 INFO:tasks.workunit.client.1.smithi138.stdout:0/680: truncate da/d2a/d8e/d1f/d60/fb1 551897 0 2024-05-27T09:29:47.521 INFO:tasks.workunit.client.1.smithi138.stdout:6/849: write d3/d11/dc0/d43/d1c/f4c [2123381,73337] 0 2024-05-27T09:29:47.521 INFO:tasks.workunit.client.1.smithi138.stdout:6/850: fsync d3/d11/dc0/d7d/f99 0 2024-05-27T09:29:47.527 INFO:tasks.workunit.client.1.smithi138.stdout:3/674: rename d0/d3f/d50/d9b/d9f/da3/dc6/fcd to d0/d3f/d60/d93/db9/fe3 0 2024-05-27T09:29:47.527 INFO:tasks.workunit.client.1.smithi138.stdout:3/675: readlink d0/d2/da/df/d10/d32/d37/l3e 0 2024-05-27T09:29:47.529 INFO:tasks.workunit.client.1.smithi138.stdout:4/766: dwrite f3 [0,4194304] 0 2024-05-27T09:29:47.529 INFO:tasks.workunit.client.1.smithi138.stdout:4/767: creat d7/dba/dcf/daa/f10f x:0 0 0 2024-05-27T09:29:47.530 INFO:tasks.workunit.client.1.smithi138.stdout:3/676: mknod d0/d2/da/df/d10/d2a/d48/d90/ce4 0 2024-05-27T09:29:47.530 INFO:tasks.workunit.client.1.smithi138.stdout:3/677: dread - d0/d3f/d60/dcb/fdb zero size 2024-05-27T09:29:47.531 INFO:tasks.workunit.client.1.smithi138.stdout:8/794: mknod d3/d8/d24/d55/d7e/c112 0 2024-05-27T09:29:47.531 INFO:tasks.workunit.client.1.smithi138.stdout:8/795: chown d3/d8/d32/d3f/d9c/d9f/fa7 21801 1 2024-05-27T09:29:47.531 INFO:tasks.workunit.client.1.smithi138.stdout:8/796: chown d3/d8/d24/d55/d7e/c112 7 1 2024-05-27T09:29:47.532 INFO:tasks.workunit.client.1.smithi138.stdout:8/797: write d3/d8/d64/f5e [3378180,15481] 0 2024-05-27T09:29:47.532 INFO:tasks.workunit.client.1.smithi138.stdout:8/798: dread - d3/d8/d32/d3f/d9c/d9f/f100 zero size 2024-05-27T09:29:47.532 INFO:tasks.workunit.client.1.smithi138.stdout:8/799: chown d3/d8/d32/d34/d38/dd0 0 1 2024-05-27T09:29:47.532 INFO:tasks.workunit.client.1.smithi138.stdout:8/800: chown d3/d8/d24/d55/lbe 827730 1 2024-05-27T09:29:47.532 INFO:tasks.workunit.client.1.smithi138.stdout:8/801: chown d3/d8/d32/d3f/d9c/db8/ffa 0 1 2024-05-27T09:29:47.532 INFO:tasks.workunit.client.1.smithi138.stdout:8/802: fdatasync d3/d8/d24/d4d/fce 0 2024-05-27T09:29:47.534 INFO:tasks.workunit.client.1.smithi138.stdout:0/681: dread da/d2a/f85 [0,4194304] 0 2024-05-27T09:29:47.538 INFO:tasks.workunit.client.1.smithi138.stdout:2/605: dwrite d1/d15/d2d/d17/d25/f3a [0,4194304] 0 2024-05-27T09:29:47.539 INFO:tasks.workunit.client.1.smithi138.stdout:0/682: mkdir da/d2a/d8e/dda/ddf 0 2024-05-27T09:29:47.542 INFO:tasks.workunit.client.1.smithi138.stdout:4/768: unlink d7/df/d35/d4c/d59/d5d/d64/d6a/l7e 0 2024-05-27T09:29:47.545 INFO:tasks.workunit.client.1.smithi138.stdout:2/606: read d1/d15/d2d/d17/d25/d9d/d4b/fa1 [1016164,120635] 0 2024-05-27T09:29:47.547 INFO:tasks.workunit.client.1.smithi138.stdout:8/803: creat d3/d8/d24/d55/f113 x:0 0 0 2024-05-27T09:29:47.553 INFO:tasks.workunit.client.1.smithi138.stdout:5/716: dwrite d3/d12/d16/f3c [4194304,4194304] 0 2024-05-27T09:29:47.553 INFO:tasks.workunit.client.1.smithi138.stdout:1/682: dwrite d4/d48/d39/d5f/f6a [0,4194304] 0 2024-05-27T09:29:47.553 INFO:tasks.workunit.client.1.smithi138.stdout:1/683: chown d4/d48/d22/d65/f75 4 1 2024-05-27T09:29:47.555 INFO:tasks.workunit.client.1.smithi138.stdout:2/607: rename d1/d10/f1a to d1/d10/d52/db9/fcf 0 2024-05-27T09:29:47.556 INFO:tasks.workunit.client.1.smithi138.stdout:8/804: symlink d3/d8/d32/d45/l114 0 2024-05-27T09:29:47.557 INFO:tasks.workunit.client.1.smithi138.stdout:8/805: write d3/d8/d24/f44 [4854357,85966] 0 2024-05-27T09:29:47.559 INFO:tasks.workunit.client.1.smithi138.stdout:8/806: creat d3/d8/d72/f115 x:0 0 0 2024-05-27T09:29:47.564 INFO:tasks.workunit.client.1.smithi138.stdout:3/678: rename d0/d3f/d60/d93/db9 to d0/d3f/d60/d93/de5 0 2024-05-27T09:29:47.564 INFO:tasks.workunit.client.1.smithi138.stdout:3/679: creat d0/d2/da/fe6 x:0 0 0 2024-05-27T09:29:47.567 INFO:tasks.workunit.client.1.smithi138.stdout:4/769: dread d7/df/d35/f8b [0,4194304] 0 2024-05-27T09:29:47.576 INFO:tasks.workunit.client.1.smithi138.stdout:1/684: dread d4/d3c/fa0 [0,4194304] 0 2024-05-27T09:29:47.577 INFO:tasks.workunit.client.1.smithi138.stdout:7/830: sync 2024-05-27T09:29:47.577 INFO:tasks.workunit.client.1.smithi138.stdout:9/715: sync 2024-05-27T09:29:47.577 INFO:tasks.workunit.client.1.smithi138.stdout:9/716: chown d6/dc8 6296 1 2024-05-27T09:29:47.578 INFO:tasks.workunit.client.1.smithi138.stdout:7/831: chown d0/d2b/daf/l97 861814685 1 2024-05-27T09:29:47.580 INFO:tasks.workunit.client.1.smithi138.stdout:9/717: symlink d6/da/d29/d59/lf4 0 2024-05-27T09:29:47.581 INFO:tasks.workunit.client.1.smithi138.stdout:3/680: symlink d0/d3f/d60/daf/le7 0 2024-05-27T09:29:47.583 INFO:tasks.workunit.client.1.smithi138.stdout:3/681: rename d0/d2/da/df/d10/d2a/d48/d90/c91 to d0/d3f/d60/d93/de5/dca/ce8 0 2024-05-27T09:29:47.584 INFO:tasks.workunit.client.1.smithi138.stdout:3/682: readlink d0/d2/da/l5a 0 2024-05-27T09:29:47.586 INFO:tasks.workunit.client.1.smithi138.stdout:3/683: unlink d0/d2/f4b 0 2024-05-27T09:29:47.589 INFO:tasks.workunit.client.1.smithi138.stdout:5/717: dread d3/d12/d26/f2f [0,4194304] 0 2024-05-27T09:29:47.599 INFO:tasks.workunit.client.1.smithi138.stdout:8/807: write d3/d8/d24/f10a [2696331,112357] 0 2024-05-27T09:29:47.612 INFO:tasks.workunit.client.1.smithi138.stdout:4/770: dread d7/df/d35/d4c/d59/d5d/d64/d6a/f6d [0,4194304] 0 2024-05-27T09:29:47.616 INFO:tasks.workunit.client.1.smithi138.stdout:4/771: symlink d7/df/d5b/da1/dbc/l110 0 2024-05-27T09:29:47.633 INFO:tasks.workunit.client.1.smithi138.stdout:6/851: dwrite d3/d11/dc0/d43/d1c/f59 [0,4194304] 0 2024-05-27T09:29:47.633 INFO:tasks.workunit.client.1.smithi138.stdout:9/718: dread d6/db/de/d18/d24/f5c [0,4194304] 0 2024-05-27T09:29:47.633 INFO:tasks.workunit.client.1.smithi138.stdout:6/852: write d3/d11/d2d/d37/dd1/f6c [1004207,76559] 0 2024-05-27T09:29:47.636 INFO:tasks.workunit.client.1.smithi138.stdout:6/853: chown d3/d19 4120999 1 2024-05-27T09:29:47.636 INFO:tasks.workunit.client.1.smithi138.stdout:9/719: write d6/da/d29/d59/f90 [1531266,64057] 0 2024-05-27T09:29:47.636 INFO:tasks.workunit.client.1.smithi138.stdout:2/608: dwrite d1/d15/d2d/d17/d25/d4d/d6d/fab [0,4194304] 0 2024-05-27T09:29:47.637 INFO:tasks.workunit.client.1.smithi138.stdout:4/772: dread d7/df/d35/f8b [0,4194304] 0 2024-05-27T09:29:47.637 INFO:tasks.workunit.client.1.smithi138.stdout:2/609: read d1/d15/d2d/d17/d25/d4d/d6d/d87/fc0 [32024,72704] 0 2024-05-27T09:29:47.637 INFO:tasks.workunit.client.1.smithi138.stdout:2/610: stat d1/d10/d52/d5d/d7e/l8c 0 2024-05-27T09:29:47.637 INFO:tasks.workunit.client.1.smithi138.stdout:2/611: read d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc/f7d [1117071,69985] 0 2024-05-27T09:29:47.637 INFO:tasks.workunit.client.1.smithi138.stdout:4/773: creat d7/df/d5b/da1/db1/f111 x:0 0 0 2024-05-27T09:29:47.637 INFO:tasks.workunit.client.1.smithi138.stdout:4/774: chown d7/df/cd3 27552 1 2024-05-27T09:29:47.639 INFO:tasks.workunit.client.1.smithi138.stdout:2/612: link d1/d10/d52/d5d/d7e/ca5 d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/cd0 0 2024-05-27T09:29:47.641 INFO:tasks.workunit.client.1.smithi138.stdout:2/613: rmdir d1/d15/d2d 39 2024-05-27T09:29:47.643 INFO:tasks.workunit.client.1.smithi138.stdout:2/614: symlink d1/d15/d2d/d2e/ld1 0 2024-05-27T09:29:47.646 INFO:tasks.workunit.client.1.smithi138.stdout:2/615: mkdir d1/d15/d2d/dd2 0 2024-05-27T09:29:47.646 INFO:tasks.workunit.client.1.smithi138.stdout:2/616: write d1/d15/d2d/d17/d25/d9d/d4b/fa1 [2234465,93515] 0 2024-05-27T09:29:47.647 INFO:tasks.workunit.client.1.smithi138.stdout:4/775: rename d7/c2f to d7/dba/dcb/c112 0 2024-05-27T09:29:47.657 INFO:tasks.workunit.client.1.smithi138.stdout:7/832: dwrite d0/d2b/f35 [0,4194304] 0 2024-05-27T09:29:47.659 INFO:tasks.workunit.client.1.smithi138.stdout:1/685: dwrite d4/d1a/d2e/d4a/f80 [0,4194304] 0 2024-05-27T09:29:47.659 INFO:tasks.workunit.client.1.smithi138.stdout:7/833: mknod d0/d23/dc4/ddf/d91/c111 0 2024-05-27T09:29:47.659 INFO:tasks.workunit.client.1.smithi138.stdout:1/686: creat d4/d48/d39/d5f/d63/fde x:0 0 0 2024-05-27T09:29:47.663 INFO:tasks.workunit.client.1.smithi138.stdout:7/834: symlink d0/d23/d3c/d84/d9c/l112 0 2024-05-27T09:29:47.665 INFO:tasks.workunit.client.1.smithi138.stdout:7/835: mknod d0/d23/d3c/d84/d9c/d103/c113 0 2024-05-27T09:29:47.665 INFO:tasks.workunit.client.1.smithi138.stdout:7/836: readlink d0/d23/d49/ld9 0 2024-05-27T09:29:47.667 INFO:tasks.workunit.client.1.smithi138.stdout:7/837: symlink d0/d23/d3c/l114 0 2024-05-27T09:29:47.667 INFO:tasks.workunit.client.1.smithi138.stdout:7/838: chown d0/d2b/daf/d98/db6/ldc 90 1 2024-05-27T09:29:47.667 INFO:tasks.workunit.client.1.smithi138.stdout:7/839: fsync d0/f19 0 2024-05-27T09:29:47.667 INFO:tasks.workunit.client.1.smithi138.stdout:7/840: fdatasync d0/d23/f50 0 2024-05-27T09:29:47.672 INFO:tasks.workunit.client.1.smithi138.stdout:8/808: dwrite d3/d8/d32/f105 [0,4194304] 0 2024-05-27T09:29:47.672 INFO:tasks.workunit.client.1.smithi138.stdout:8/809: stat d3/d8/d32/d34/dbd/cd1 0 2024-05-27T09:29:47.694 INFO:tasks.workunit.client.1.smithi138.stdout:6/854: dwrite d3/d11/d2d/d37/dd1/f6c [0,4194304] 0 2024-05-27T09:29:47.698 INFO:tasks.workunit.client.1.smithi138.stdout:6/855: creat d3/d11/d2d/d37/dd1/d9b/d10a/f11a x:0 0 0 2024-05-27T09:29:47.699 INFO:tasks.workunit.client.1.smithi138.stdout:6/856: symlink d3/d11/d2d/d37/dd1/l11b 0 2024-05-27T09:29:47.699 INFO:tasks.workunit.client.1.smithi138.stdout:6/857: fsync d3/d11/d2d/d37/d7e/f119 0 2024-05-27T09:29:47.700 INFO:tasks.workunit.client.1.smithi138.stdout:6/858: write d3/d11/d2d/d37/dd1/d9b/d10a/f11a [934671,108844] 0 2024-05-27T09:29:47.707 INFO:tasks.workunit.client.1.smithi138.stdout:5/718: dwrite d3/d12/d16/d1d/d22/d2b/d9e/db6/f43 [0,4194304] 0 2024-05-27T09:29:47.707 INFO:tasks.workunit.client.1.smithi138.stdout:5/719: readlink d3/d12/d16/d1d/d23/d3b/l80 0 2024-05-27T09:29:47.714 INFO:tasks.workunit.client.1.smithi138.stdout:5/720: mkdir d3/d12/d32/d87/dea 0 2024-05-27T09:29:47.717 INFO:tasks.workunit.client.1.smithi138.stdout:5/721: creat d3/dab/dc2/feb x:0 0 0 2024-05-27T09:29:47.726 INFO:tasks.workunit.client.1.smithi138.stdout:5/722: rename d3/dab/dc2/dd5 to d3/d12/d16/d1d/d23/d3b/dec 0 2024-05-27T09:29:47.729 INFO:tasks.workunit.client.1.smithi138.stdout:6/859: write d3/d11/d2d/d37/d8b/fc7 [848089,16468] 0 2024-05-27T09:29:47.729 INFO:tasks.workunit.client.1.smithi138.stdout:2/617: dwrite d1/d10/d3c/f62 [0,4194304] 0 2024-05-27T09:29:47.729 INFO:tasks.workunit.client.1.smithi138.stdout:2/618: write d1/d10/f64 [1706453,128587] 0 2024-05-27T09:29:47.729 INFO:tasks.workunit.client.1.smithi138.stdout:5/723: mknod d3/dab/dc2/dc6/ced 0 2024-05-27T09:29:47.729 INFO:tasks.workunit.client.1.smithi138.stdout:5/724: chown d3/d12/d32/d87/dc0/lcd 441 1 2024-05-27T09:29:47.730 INFO:tasks.workunit.client.1.smithi138.stdout:2/619: mknod d1/d15/d2d/d17/cd3 0 2024-05-27T09:29:47.736 INFO:tasks.workunit.client.1.smithi138.stdout:6/860: dread d3/d11/dc0/d5a/d6a/f7f [0,4194304] 0 2024-05-27T09:29:47.738 INFO:tasks.workunit.client.1.smithi138.stdout:6/861: truncate d3/fa 1694928 0 2024-05-27T09:29:47.740 INFO:tasks.workunit.client.1.smithi138.stdout:6/862: creat d3/d11/dc0/d43/f11c x:0 0 0 2024-05-27T09:29:47.741 INFO:tasks.workunit.client.1.smithi138.stdout:8/810: dwrite d3/db5/f104 [0,4194304] 0 2024-05-27T09:29:47.752 INFO:tasks.workunit.client.1.smithi138.stdout:1/687: dwrite d4/f78 [4194304,4194304] 0 2024-05-27T09:29:47.754 INFO:tasks.workunit.client.1.smithi138.stdout:1/688: truncate d4/d1a/d2e/d4a/d7e/fcf 556564 0 2024-05-27T09:29:47.758 INFO:tasks.workunit.client.1.smithi138.stdout:1/689: rename d4/cdb to d4/d48/d39/d5f/d4d/d2b/cdf 0 2024-05-27T09:29:47.759 INFO:tasks.workunit.client.1.smithi138.stdout:8/811: dread d3/d8/d24/f54 [0,4194304] 0 2024-05-27T09:29:47.762 INFO:tasks.workunit.client.1.smithi138.stdout:8/812: creat d3/d8/d24/d55/d7e/f116 x:0 0 0 2024-05-27T09:29:47.784 INFO:tasks.workunit.client.1.smithi138.stdout:5/725: write d3/d12/d32/d6b/f78 [2761335,119619] 0 2024-05-27T09:29:47.818 INFO:tasks.workunit.client.1.smithi138.stdout:6/863: dwrite d3/d11/dc0/d43/d8c/fa6 [0,4194304] 0 2024-05-27T09:29:47.820 INFO:tasks.workunit.client.1.smithi138.stdout:6/864: getdents d3/d11/d2d/d37/d7e 0 2024-05-27T09:29:47.821 INFO:tasks.workunit.client.1.smithi138.stdout:6/865: getdents d3/d11/dc2 0 2024-05-27T09:29:47.822 INFO:tasks.workunit.client.1.smithi138.stdout:5/726: write d3/d12/d32/d6b/fd2 [3993751,2496] 0 2024-05-27T09:29:47.824 INFO:tasks.workunit.client.1.smithi138.stdout:5/727: mknod d3/d12/d16/d1d/d22/db9/ddf/cee 0 2024-05-27T09:29:47.827 INFO:tasks.workunit.client.1.smithi138.stdout:8/813: dwrite d3/d8/d72/fde [0,4194304] 0 2024-05-27T09:29:47.830 INFO:tasks.workunit.client.1.smithi138.stdout:8/814: truncate d3/d8/fb6 1030970 0 2024-05-27T09:29:47.831 INFO:tasks.workunit.client.1.smithi138.stdout:1/690: dwrite d4/d48/d39/d5f/d4d/d2b/f6c [0,4194304] 0 2024-05-27T09:29:47.832 INFO:tasks.workunit.client.1.smithi138.stdout:6/866: dread d3/d11/d2d/d37/d8b/fc7 [0,4194304] 0 2024-05-27T09:29:47.832 INFO:tasks.workunit.client.1.smithi138.stdout:8/815: rename d3/d8/d32/d45 to d3/d8/d24/d55/d94/da5/d111/d117 0 2024-05-27T09:29:47.833 INFO:tasks.workunit.client.1.smithi138.stdout:8/816: stat d3/d8/d32/d34/d38/l43 0 2024-05-27T09:29:47.835 INFO:tasks.workunit.client.1.smithi138.stdout:6/867: symlink d3/d11/d2d/d37/d7e/l11d 0 2024-05-27T09:29:47.835 INFO:tasks.workunit.client.1.smithi138.stdout:6/868: chown d3/d11/d2d/d37/dd1/d47/c72 238 1 2024-05-27T09:29:47.835 INFO:tasks.workunit.client.1.smithi138.stdout:6/869: getdents d3/d11/d2d/d37/d102 0 2024-05-27T09:29:47.851 INFO:tasks.workunit.client.1.smithi138.stdout:8/817: dread d3/f4a [0,4194304] 0 2024-05-27T09:29:47.854 INFO:tasks.workunit.client.1.smithi138.stdout:8/818: creat d3/d8/f118 x:0 0 0 2024-05-27T09:29:47.917 INFO:tasks.workunit.client.1.smithi138.stdout:5/728: dwrite d3/d12/d26/f57 [0,4194304] 0 2024-05-27T09:29:47.921 INFO:tasks.workunit.client.1.smithi138.stdout:6/870: dwrite d3/d11/fe [4194304,4194304] 0 2024-05-27T09:29:47.925 INFO:tasks.workunit.client.1.smithi138.stdout:5/729: read d3/f88 [1824627,114861] 0 2024-05-27T09:29:47.926 INFO:tasks.workunit.client.1.smithi138.stdout:5/730: readlink d3/l34 0 2024-05-27T09:29:47.927 INFO:tasks.workunit.client.1.smithi138.stdout:5/731: mknod d3/dab/dc2/cef 0 2024-05-27T09:29:47.930 INFO:tasks.workunit.client.1.smithi138.stdout:5/732: unlink d3/d12/d16/d1d/d22/d2b/f38 0 2024-05-27T09:29:47.930 INFO:tasks.workunit.client.1.smithi138.stdout:5/733: write d3/f88 [1523012,130552] 0 2024-05-27T09:29:47.935 INFO:tasks.workunit.client.1.smithi138.stdout:8/819: dwrite d3/f4a [4194304,4194304] 0 2024-05-27T09:29:47.937 INFO:tasks.workunit.client.1.smithi138.stdout:5/734: rmdir d3/d12/d32/d60/d93 39 2024-05-27T09:29:47.939 INFO:tasks.workunit.client.1.smithi138.stdout:8/820: symlink d3/d8/d11/df1/l119 0 2024-05-27T09:29:47.945 INFO:tasks.workunit.client.1.smithi138.stdout:8/821: rename d3/d8/d11/deb/dd3/ff4 to d3/d8/d11/df1/f11a 0 2024-05-27T09:29:47.965 INFO:tasks.workunit.client.1.smithi138.stdout:5/735: write d3/d12/d26/fe8 [8152924,84185] 0 2024-05-27T09:29:47.996 INFO:tasks.workunit.client.1.smithi138.stdout:6/871: dwrite d3/d11/d2d/d37/daa/df3/ff9 [0,4194304] 0 2024-05-27T09:29:48.023 INFO:tasks.workunit.client.1.smithi138.stdout:6/872: write d3/d19/fa7 [2769,116998] 0 2024-05-27T09:29:48.025 INFO:tasks.workunit.client.1.smithi138.stdout:6/873: mkdir d3/d11/d2d/d37/dd1/ded/d11e 0 2024-05-27T09:29:48.025 INFO:tasks.workunit.client.1.smithi138.stdout:6/874: dread - d3/d11/d2d/d37/d7e/fb0 zero size 2024-05-27T09:29:48.025 INFO:tasks.workunit.client.1.smithi138.stdout:8/822: dwrite d3/d8/d64/f2e [0,4194304] 0 2024-05-27T09:29:48.026 INFO:tasks.workunit.client.1.smithi138.stdout:6/875: mkdir d3/d11/dc0/d43/d11f 0 2024-05-27T09:29:48.028 INFO:tasks.workunit.client.1.smithi138.stdout:6/876: creat d3/d11/d2d/d37/daa/df3/f120 x:0 0 0 2024-05-27T09:29:48.028 INFO:tasks.workunit.client.1.smithi138.stdout:8/823: symlink d3/d8/d24/d55/d94/da5/d111/l11b 0 2024-05-27T09:29:48.028 INFO:tasks.workunit.client.1.smithi138.stdout:8/824: fsync d3/d8/d32/d3f/d9c/ff6 0 2024-05-27T09:29:48.030 INFO:tasks.workunit.client.1.smithi138.stdout:6/877: unlink d3/d19/f28 0 2024-05-27T09:29:48.030 INFO:tasks.workunit.client.1.smithi138.stdout:6/878: chown d3/d11/dc0/d43/d1c/f9c 288304791 1 2024-05-27T09:29:48.031 INFO:tasks.workunit.client.1.smithi138.stdout:8/825: mkdir d3/d8/d32/d34/d11c 0 2024-05-27T09:29:48.031 INFO:tasks.workunit.client.1.smithi138.stdout:8/826: write d3/d8/d32/d3f/d9c/d9f/f100 [931956,105229] 0 2024-05-27T09:29:48.093 INFO:tasks.workunit.client.1.smithi138.stdout:8/827: dwrite d3/db5/f104 [0,4194304] 0 2024-05-27T09:29:48.093 INFO:tasks.workunit.client.1.smithi138.stdout:8/828: chown d3/d8/d11/deb/cdd 3 1 2024-05-27T09:29:48.095 INFO:tasks.workunit.client.1.smithi138.stdout:6/879: dwrite d3/f53 [0,4194304] 0 2024-05-27T09:29:48.100 INFO:tasks.workunit.client.1.smithi138.stdout:6/880: fdatasync d3/d19/fa7 0 2024-05-27T09:29:48.102 INFO:tasks.workunit.client.1.smithi138.stdout:6/881: mknod d3/d11/d2d/c121 0 2024-05-27T09:29:48.105 INFO:tasks.workunit.client.1.smithi138.stdout:6/882: rename d3/d11/d2d/d37/dd1/d9b/cce to d3/d11/d2d/d37/d7e/c122 0 2024-05-27T09:29:48.105 INFO:tasks.workunit.client.1.smithi138.stdout:6/883: read d3/d11/dc0/d43/d1c/f4d [4005394,73268] 0 2024-05-27T09:29:48.106 INFO:tasks.workunit.client.1.smithi138.stdout:6/884: mkdir d3/d11/dc0/d43/d11f/d123 0 2024-05-27T09:29:48.108 INFO:tasks.workunit.client.1.smithi138.stdout:6/885: rename d3/fa to d3/d11/dc0/d43/d8c/f124 0 2024-05-27T09:29:48.113 INFO:tasks.workunit.client.1.smithi138.stdout:6/886: creat d3/d11/dc2/de1/f125 x:0 0 0 2024-05-27T09:29:48.116 INFO:tasks.workunit.client.1.smithi138.stdout:6/887: creat d3/d11/f126 x:0 0 0 2024-05-27T09:29:48.116 INFO:tasks.workunit.client.1.smithi138.stdout:6/888: dread - d3/d11/dc0/d43/d71/fca zero size 2024-05-27T09:29:48.119 INFO:tasks.workunit.client.1.smithi138.stdout:6/889: unlink d3/d11/dc0/d5a/db4/fb6 0 2024-05-27T09:29:48.119 INFO:tasks.workunit.client.1.smithi138.stdout:8/829: write d3/d8/d32/d3f/d9c/d9f/fa7 [3179568,33953] 0 2024-05-27T09:29:48.119 INFO:tasks.workunit.client.1.smithi138.stdout:8/830: dread - d3/d8/d72/f115 zero size 2024-05-27T09:29:48.120 INFO:tasks.workunit.client.1.smithi138.stdout:6/890: unlink d3/d11/dc0/d5a/f116 0 2024-05-27T09:29:48.121 INFO:tasks.workunit.client.1.smithi138.stdout:8/831: symlink d3/d8/d72/d109/l11d 0 2024-05-27T09:29:48.134 INFO:tasks.workunit.client.1.smithi138.stdout:8/832: mkdir d3/d8/d24/d55/d94/da5/d111/d117/d11e 0 2024-05-27T09:29:48.138 INFO:tasks.workunit.client.1.smithi138.stdout:8/833: symlink d3/l11f 0 2024-05-27T09:29:48.138 INFO:tasks.workunit.client.1.smithi138.stdout:8/834: chown d3/d8/d24 1 1 2024-05-27T09:29:48.202 INFO:tasks.workunit.client.1.smithi138.stdout:6/891: dwrite d3/d11/dc0/d43/ffa [0,4194304] 0 2024-05-27T09:29:48.203 INFO:tasks.workunit.client.1.smithi138.stdout:6/892: write d3/d19/fab [986744,97726] 0 2024-05-27T09:29:48.203 INFO:tasks.workunit.client.1.smithi138.stdout:6/893: truncate d3/d11/d2d/d37/f6d 78821 0 2024-05-27T09:29:48.211 INFO:tasks.workunit.client.1.smithi138.stdout:8/835: dwrite d3/d8/f58 [0,4194304] 0 2024-05-27T09:29:48.218 INFO:tasks.workunit.client.1.smithi138.stdout:8/836: mkdir d3/d8/d11/deb/d120 0 2024-05-27T09:29:48.218 INFO:tasks.workunit.client.1.smithi138.stdout:8/837: write d3/f29 [3629253,112820] 0 2024-05-27T09:29:48.241 INFO:tasks.workunit.client.1.smithi138.stdout:8/838: dread d3/d8/d32/d3f/d9c/d9f/fa7 [0,4194304] 0 2024-05-27T09:29:48.251 INFO:tasks.workunit.client.1.smithi138.stdout:0/683: sync 2024-05-27T09:29:48.251 INFO:tasks.workunit.client.1.smithi138.stdout:0/684: dread - da/d2a/d8e/d1f/d42/d4a/d53/f90 zero size 2024-05-27T09:29:48.251 INFO:tasks.workunit.client.1.smithi138.stdout:6/894: dwrite d3/d19/f41 [0,4194304] 0 2024-05-27T09:29:48.251 INFO:tasks.workunit.client.1.smithi138.stdout:6/895: creat d3/d11/d2d/d37/daa/f127 x:0 0 0 2024-05-27T09:29:48.259 INFO:tasks.workunit.client.1.smithi138.stdout:8/839: write d3/d8/f33 [7017239,35528] 0 2024-05-27T09:29:48.263 INFO:tasks.workunit.client.1.smithi138.stdout:8/840: mknod d3/d8/d11/df1/dae/c121 0 2024-05-27T09:29:48.264 INFO:tasks.workunit.client.1.smithi138.stdout:6/896: write d3/d11/dc0/d43/d8c/f124 [236044,84124] 0 2024-05-27T09:29:48.265 INFO:tasks.workunit.client.1.smithi138.stdout:6/897: rmdir d3/d11/d2d/d37/d7e 39 2024-05-27T09:29:48.265 INFO:tasks.workunit.client.1.smithi138.stdout:6/898: fdatasync d3/d11/dc0/d43/ffa 0 2024-05-27T09:29:48.286 INFO:tasks.workunit.client.1.smithi138.stdout:6/899: dread d3/f84 [0,4194304] 0 2024-05-27T09:29:48.286 INFO:tasks.workunit.client.1.smithi138.stdout:6/900: getdents d3/d11/dc0/d5a/df0/d118 0 2024-05-27T09:29:48.286 INFO:tasks.workunit.client.1.smithi138.stdout:6/901: creat d3/d11/f128 x:0 0 0 2024-05-27T09:29:48.286 INFO:tasks.workunit.client.1.smithi138.stdout:6/902: fdatasync d3/d11/dc0/d7d/f99 0 2024-05-27T09:29:48.286 INFO:tasks.workunit.client.1.smithi138.stdout:6/903: truncate d3/d11/dc0/d7d/de3/fef 738618 0 2024-05-27T09:29:48.286 INFO:tasks.workunit.client.1.smithi138.stdout:6/904: creat d3/d11/d2d/d37/daa/df3/f129 x:0 0 0 2024-05-27T09:29:48.294 INFO:tasks.workunit.client.1.smithi138.stdout:6/905: getdents d3/d11/dc0/d5a/d6a 0 2024-05-27T09:29:48.333 INFO:tasks.workunit.client.1.smithi138.stdout:8/841: dwrite d3/f50 [0,4194304] 0 2024-05-27T09:29:48.333 INFO:tasks.workunit.client.1.smithi138.stdout:8/842: fsync d3/d8/d32/d34/dbd/fe4 0 2024-05-27T09:29:48.338 INFO:tasks.workunit.client.1.smithi138.stdout:8/843: symlink d3/d8/d24/d55/d94/da5/d111/d117/d11e/l122 0 2024-05-27T09:29:48.338 INFO:tasks.workunit.client.1.smithi138.stdout:8/844: truncate d3/d8/d32/dd6/f10d 1662135 0 2024-05-27T09:29:48.342 INFO:tasks.workunit.client.1.smithi138.stdout:8/845: rename d3/d8/d11/deb/f98 to d3/f123 0 2024-05-27T09:29:48.347 INFO:tasks.workunit.client.1.smithi138.stdout:8/846: mknod d3/d8/d32/d34/c124 0 2024-05-27T09:29:48.352 INFO:tasks.workunit.client.1.smithi138.stdout:8/847: creat d3/d8/d24/d55/d94/da5/d111/d117/d5c/f125 x:0 0 0 2024-05-27T09:29:48.360 INFO:tasks.workunit.client.1.smithi138.stdout:6/906: dwrite d3/d11/dc0/d43/dbe/f106 [0,4194304] 0 2024-05-27T09:29:48.367 INFO:tasks.workunit.client.1.smithi138.stdout:6/907: link d3/d19/c2f d3/d11/dc0/d7d/de3/c12a 0 2024-05-27T09:29:48.370 INFO:tasks.workunit.client.1.smithi138.stdout:6/908: unlink d3/d11/dc0/d7d/db9/c10f 0 2024-05-27T09:29:48.370 INFO:tasks.workunit.client.1.smithi138.stdout:6/909: truncate d3/d11/dc0/d74/fc8 241763 0 2024-05-27T09:29:48.373 INFO:tasks.workunit.client.1.smithi138.stdout:6/910: rename d3/d19/f34 to d3/d11/d2d/d37/dd1/f12b 0 2024-05-27T09:29:48.374 INFO:tasks.workunit.client.1.smithi138.stdout:6/911: creat d3/d11/dc0/d43/d11f/f12c x:0 0 0 2024-05-27T09:29:48.426 INFO:tasks.workunit.client.1.smithi138.stdout:8/848: dwrite d3/d8/f42 [0,4194304] 0 2024-05-27T09:29:48.426 INFO:tasks.workunit.client.1.smithi138.stdout:8/849: chown d3/d8/c18 440131146 1 2024-05-27T09:29:48.429 INFO:tasks.workunit.client.1.smithi138.stdout:8/850: getdents d3/d8/d24/d55/d94/dc5 0 2024-05-27T09:29:48.431 INFO:tasks.workunit.client.1.smithi138.stdout:8/851: mkdir d3/d8/d24/d55/d94/da5/d111/d117/d126 0 2024-05-27T09:29:48.433 INFO:tasks.workunit.client.1.smithi138.stdout:6/912: dwrite d3/d11/dc0/d5a/fff [0,4194304] 0 2024-05-27T09:29:48.442 INFO:tasks.workunit.client.1.smithi138.stdout:6/913: link d3/d19/f89 d3/d11/dc2/de1/f12d 0 2024-05-27T09:29:48.457 INFO:tasks.workunit.client.1.smithi138.stdout:6/914: dread d3/d11/dc0/d43/d1c/f59 [4194304,4194304] 0 2024-05-27T09:29:48.461 INFO:tasks.workunit.client.1.smithi138.stdout:9/720: sync 2024-05-27T09:29:48.461 INFO:tasks.workunit.client.1.smithi138.stdout:3/684: sync 2024-05-27T09:29:48.461 INFO:tasks.workunit.client.1.smithi138.stdout:9/721: readlink d6/da/d29/d8f/da8/lab 0 2024-05-27T09:29:48.461 INFO:tasks.workunit.client.1.smithi138.stdout:9/722: dread - d6/db/dc/dca/fec zero size 2024-05-27T09:29:48.464 INFO:tasks.workunit.client.1.smithi138.stdout:9/723: creat d6/db/de/d6b/ff5 x:0 0 0 2024-05-27T09:29:48.470 INFO:tasks.workunit.client.1.smithi138.stdout:0/685: sync 2024-05-27T09:29:48.470 INFO:tasks.workunit.client.1.smithi138.stdout:4/776: sync 2024-05-27T09:29:48.470 INFO:tasks.workunit.client.1.smithi138.stdout:7/841: sync 2024-05-27T09:29:48.471 INFO:tasks.workunit.client.1.smithi138.stdout:7/842: chown d0/l38 10505 1 2024-05-27T09:29:48.471 INFO:tasks.workunit.client.1.smithi138.stdout:7/843: chown d0/d23/d3a/f46 1324 1 2024-05-27T09:29:48.471 INFO:tasks.workunit.client.1.smithi138.stdout:2/620: sync 2024-05-27T09:29:48.471 INFO:tasks.workunit.client.1.smithi138.stdout:7/844: fdatasync d0/d23/d3c/f42 0 2024-05-27T09:29:48.471 INFO:tasks.workunit.client.1.smithi138.stdout:1/691: sync 2024-05-27T09:29:48.471 INFO:tasks.workunit.client.1.smithi138.stdout:1/692: chown d4/d38/fd4 384 1 2024-05-27T09:29:48.471 INFO:tasks.workunit.client.1.smithi138.stdout:5/736: sync 2024-05-27T09:29:48.471 INFO:tasks.workunit.client.1.smithi138.stdout:5/737: chown d3/d12/f31 1 1 2024-05-27T09:29:48.471 INFO:tasks.workunit.client.1.smithi138.stdout:3/685: rmdir d0/d2 39 2024-05-27T09:29:48.471 INFO:tasks.workunit.client.1.smithi138.stdout:3/686: write d0/d3f/d60/daf/fb1 [739571,126161] 0 2024-05-27T09:29:48.473 INFO:tasks.workunit.client.1.smithi138.stdout:3/687: creat d0/d2/da/df/d10/fe9 x:0 0 0 2024-05-27T09:29:48.473 INFO:tasks.workunit.client.1.smithi138.stdout:3/688: truncate d0/d2/da/df/f85 942254 0 2024-05-27T09:29:48.505 INFO:tasks.workunit.client.1.smithi138.stdout:4/777: symlink d7/d6e/l113 0 2024-05-27T09:29:48.507 INFO:tasks.workunit.client.1.smithi138.stdout:1/693: rename d4/d48/c10 to d4/d1a/d2e/d32/d8c/ce0 0 2024-05-27T09:29:48.507 INFO:tasks.workunit.client.1.smithi138.stdout:1/694: truncate d4/d1a/d2e/d32/d8c/dc6/f7b 565677 0 2024-05-27T09:29:48.507 INFO:tasks.workunit.client.1.smithi138.stdout:1/695: truncate d4/d1a/d2e/fb4 604776 0 2024-05-27T09:29:48.512 INFO:tasks.workunit.client.1.smithi138.stdout:6/915: write d3/d11/dc0/f60 [1188068,100601] 0 2024-05-27T09:29:48.512 INFO:tasks.workunit.client.1.smithi138.stdout:5/738: mknod d3/d12/d16/d1d/d22/d2b/d9e/db6/cf0 0 2024-05-27T09:29:48.520 INFO:tasks.workunit.client.1.smithi138.stdout:4/778: unlink d7/df/d5b/f89 0 2024-05-27T09:29:48.520 INFO:tasks.workunit.client.1.smithi138.stdout:4/779: write d7/df/f2e [1736798,109627] 0 2024-05-27T09:29:48.522 INFO:tasks.workunit.client.1.smithi138.stdout:1/696: mknod d4/d38/ce1 0 2024-05-27T09:29:48.527 INFO:tasks.workunit.client.1.smithi138.stdout:5/739: rename d3/d12/d16/d1d/d30/lbd to d3/d12/d32/d60/lf1 0 2024-05-27T09:29:48.530 INFO:tasks.workunit.client.1.smithi138.stdout:8/852: dwrite d3/d8/d24/d55/d94/da5/d111/d117/d5c/f125 [0,4194304] 0 2024-05-27T09:29:48.535 INFO:tasks.workunit.client.1.smithi138.stdout:7/845: dwrite d0/d23/f47 [0,4194304] 0 2024-05-27T09:29:48.535 INFO:tasks.workunit.client.1.smithi138.stdout:7/846: fsync d0/d23/da8/d8e/df4/f100 0 2024-05-27T09:29:48.540 INFO:tasks.workunit.client.1.smithi138.stdout:7/847: write d0/d2b/daf/d43/fa9 [886097,7136] 0 2024-05-27T09:29:48.540 INFO:tasks.workunit.client.1.smithi138.stdout:1/697: symlink d4/d1a/d2e/d4a/le2 0 2024-05-27T09:29:48.540 INFO:tasks.workunit.client.1.smithi138.stdout:1/698: creat d4/d48/d39/d5f/d63/fe3 x:0 0 0 2024-05-27T09:29:48.540 INFO:tasks.workunit.client.1.smithi138.stdout:4/780: link d7/ca d7/df/c114 0 2024-05-27T09:29:48.540 INFO:tasks.workunit.client.1.smithi138.stdout:7/848: getdents d0/d23/d3c/d3f 0 2024-05-27T09:29:48.540 INFO:tasks.workunit.client.1.smithi138.stdout:7/849: chown d0/d23/d3c/f42 6960 1 2024-05-27T09:29:48.540 INFO:tasks.workunit.client.1.smithi138.stdout:1/699: rename d4/d38/c47 to d4/dcc/ce4 0 2024-05-27T09:29:48.560 INFO:tasks.workunit.client.1.smithi138.stdout:8/853: dread d3/d8/d64/f5e [0,4194304] 0 2024-05-27T09:29:48.564 INFO:tasks.workunit.client.1.smithi138.stdout:7/850: dread d0/d23/f47 [0,4194304] 0 2024-05-27T09:29:48.565 INFO:tasks.workunit.client.1.smithi138.stdout:8/854: rmdir d3/d8/d64 39 2024-05-27T09:29:48.567 INFO:tasks.workunit.client.1.smithi138.stdout:8/855: truncate d3/d8/d32/d34/d38/f46 3373582 0 2024-05-27T09:29:48.568 INFO:tasks.workunit.client.1.smithi138.stdout:7/851: rename d0/l75 to d0/d23/l115 0 2024-05-27T09:29:48.568 INFO:tasks.workunit.client.1.smithi138.stdout:7/852: write d0/d23/dc4/ddf/d91/fb4 [2063122,74116] 0 2024-05-27T09:29:48.569 INFO:tasks.workunit.client.1.smithi138.stdout:8/856: creat d3/d8/d32/d34/d38/dd0/f127 x:0 0 0 2024-05-27T09:29:48.570 INFO:tasks.workunit.client.1.smithi138.stdout:7/853: rmdir d0/d23/d3a 39 2024-05-27T09:29:48.570 INFO:tasks.workunit.client.1.smithi138.stdout:8/857: getdents d3/d8/d32/d34/dbd 0 2024-05-27T09:29:48.571 INFO:tasks.workunit.client.1.smithi138.stdout:2/621: dwrite d1/d15/d2d/d17/d25/d9d/d4b/fc5 [0,4194304] 0 2024-05-27T09:29:48.574 INFO:tasks.workunit.client.1.smithi138.stdout:2/622: mkdir d1/dd4 0 2024-05-27T09:29:48.574 INFO:tasks.workunit.client.1.smithi138.stdout:8/858: rmdir d3/d8/d32/dd6 39 2024-05-27T09:29:48.574 INFO:tasks.workunit.client.1.smithi138.stdout:8/859: fdatasync d3/db5/fd9 0 2024-05-27T09:29:48.575 INFO:tasks.workunit.client.1.smithi138.stdout:1/700: read d4/d1a/d2e/f62 [1171182,46047] 0 2024-05-27T09:29:48.575 INFO:tasks.workunit.client.1.smithi138.stdout:1/701: readlink d4/d1a/la7 0 2024-05-27T09:29:48.576 INFO:tasks.workunit.client.1.smithi138.stdout:8/860: mknod d3/d8/d24/d55/d94/da5/d111/d117/d11e/c128 0 2024-05-27T09:29:48.577 INFO:tasks.workunit.client.1.smithi138.stdout:8/861: symlink d3/d8/l129 0 2024-05-27T09:29:48.577 INFO:tasks.workunit.client.1.smithi138.stdout:8/862: dread - d3/d8/d24/d55/d94/da5/d111/d117/d5c/ffb zero size 2024-05-27T09:29:48.582 INFO:tasks.workunit.client.1.smithi138.stdout:1/702: link d4/d48/d22/f35 d4/d48/d39/d5f/d4d/d2b/fe5 0 2024-05-27T09:29:48.583 INFO:tasks.workunit.client.1.smithi138.stdout:3/689: read d0/d2/da/f2d [158938,62800] 0 2024-05-27T09:29:48.583 INFO:tasks.workunit.client.1.smithi138.stdout:3/690: fsync d0/d3f/d60/daf/fb1 0 2024-05-27T09:29:48.583 INFO:tasks.workunit.client.1.smithi138.stdout:0/686: dwrite da/d2a/d2b/f4b [0,4194304] 0 2024-05-27T09:29:48.583 INFO:tasks.workunit.client.1.smithi138.stdout:0/687: truncate da/d2a/d8e/d1f/d42/f9a 532579 0 2024-05-27T09:29:48.584 INFO:tasks.workunit.client.1.smithi138.stdout:3/691: creat d0/dda/fea x:0 0 0 2024-05-27T09:29:48.585 INFO:tasks.workunit.client.1.smithi138.stdout:3/692: chown d0/d2/da 3133846 1 2024-05-27T09:29:48.585 INFO:tasks.workunit.client.1.smithi138.stdout:6/916: write d3/d11/dc0/d43/d1c/fbc [784087,127954] 0 2024-05-27T09:29:48.587 INFO:tasks.workunit.client.1.smithi138.stdout:9/724: dwrite d6/db/de/d18/d24/d72/fde [0,4194304] 0 2024-05-27T09:29:48.587 INFO:tasks.workunit.client.1.smithi138.stdout:3/693: mkdir d0/d2/da/df/d10/d2a/d61/d73/deb 0 2024-05-27T09:29:48.591 INFO:tasks.workunit.client.1.smithi138.stdout:1/703: truncate d4/d48/d39/d5f/f5b 680201 0 2024-05-27T09:29:48.591 INFO:tasks.workunit.client.1.smithi138.stdout:3/694: mknod d0/d3f/d60/dcb/cec 0 2024-05-27T09:29:48.591 INFO:tasks.workunit.client.1.smithi138.stdout:9/725: symlink d6/da/d29/lf6 0 2024-05-27T09:29:48.593 INFO:tasks.workunit.client.1.smithi138.stdout:9/726: mknod d6/da/dc7/cf7 0 2024-05-27T09:29:48.595 INFO:tasks.workunit.client.1.smithi138.stdout:6/917: truncate d3/d11/dc0/d43/d8c/fa6 3690085 0 2024-05-27T09:29:48.597 INFO:tasks.workunit.client.1.smithi138.stdout:0/688: rmdir da/d2a/d55 39 2024-05-27T09:29:48.597 INFO:tasks.workunit.client.1.smithi138.stdout:0/689: write da/f66 [1236169,106775] 0 2024-05-27T09:29:48.599 INFO:tasks.workunit.client.1.smithi138.stdout:8/863: dread d3/d8/d11/f26 [0,4194304] 0 2024-05-27T09:29:48.599 INFO:tasks.workunit.client.1.smithi138.stdout:4/781: dwrite d7/f18 [0,4194304] 0 2024-05-27T09:29:48.601 INFO:tasks.workunit.client.1.smithi138.stdout:6/918: mknod d3/d11/dc0/d7d/c12e 0 2024-05-27T09:29:48.601 INFO:tasks.workunit.client.1.smithi138.stdout:8/864: rename d3/d8/d32/dd6/f10b to d3/d8/d32/d34/f12a 0 2024-05-27T09:29:48.605 INFO:tasks.workunit.client.1.smithi138.stdout:1/704: read d4/d48/f58 [838208,80673] 0 2024-05-27T09:29:48.614 INFO:tasks.workunit.client.1.smithi138.stdout:1/705: mkdir d4/d48/d39/d5f/d63/de6 0 2024-05-27T09:29:48.614 INFO:tasks.workunit.client.1.smithi138.stdout:1/706: write d4/d1a/d2e/d32/f60 [301484,61153] 0 2024-05-27T09:29:48.614 INFO:tasks.workunit.client.1.smithi138.stdout:1/707: write d4/d1a/d2e/d32/d8c/dc6/f55 [4451694,62297] 0 2024-05-27T09:29:48.615 INFO:tasks.workunit.client.1.smithi138.stdout:9/727: write d6/db/f6a [23007,76551] 0 2024-05-27T09:29:48.615 INFO:tasks.workunit.client.1.smithi138.stdout:4/782: write d7/dba/dcf/f48 [1202168,40673] 0 2024-05-27T09:29:48.621 INFO:tasks.workunit.client.1.smithi138.stdout:8/865: dread d3/d8/d24/d4d/f97 [0,4194304] 0 2024-05-27T09:29:48.622 INFO:tasks.workunit.client.1.smithi138.stdout:8/866: write d3/d8/d32/d3f/d9c/d9f/fa7 [5059630,15520] 0 2024-05-27T09:29:48.622 INFO:tasks.workunit.client.1.smithi138.stdout:8/867: readlink d3/d8/d11/l1b 0 2024-05-27T09:29:48.625 INFO:tasks.workunit.client.1.smithi138.stdout:1/708: mknod d4/d1a/d2e/d4a/ce7 0 2024-05-27T09:29:48.631 INFO:tasks.workunit.client.1.smithi138.stdout:0/690: write da/d17/f8c [1539378,49923] 0 2024-05-27T09:29:48.634 INFO:tasks.workunit.client.1.smithi138.stdout:4/783: creat d7/df/d35/f115 x:0 0 0 2024-05-27T09:29:48.634 INFO:tasks.workunit.client.1.smithi138.stdout:4/784: stat d7/df/f7b 0 2024-05-27T09:29:48.634 INFO:tasks.workunit.client.1.smithi138.stdout:4/785: chown d7/df/d35/d4c/dc0/fdb 1 1 2024-05-27T09:29:48.640 INFO:tasks.workunit.client.1.smithi138.stdout:1/709: rename d4/d38/d97/c9a to d4/d1a/ce8 0 2024-05-27T09:29:48.648 INFO:tasks.workunit.client.1.smithi138.stdout:0/691: chown da/d2a/d8e/d1f/d4f/fa8 1048195 1 2024-05-27T09:29:48.683 INFO:tasks.workunit.client.1.smithi138.stdout:3/695: dwrite d0/d2/f13 [0,4194304] 0 2024-05-27T09:29:48.687 INFO:tasks.workunit.client.1.smithi138.stdout:3/696: creat d0/d2/da/df/d10/d2a/db2/fed x:0 0 0 2024-05-27T09:29:48.687 INFO:tasks.workunit.client.1.smithi138.stdout:3/697: readlink d0/d3f/d60/l7a 0 2024-05-27T09:29:48.687 INFO:tasks.workunit.client.1.smithi138.stdout:3/698: readlink d0/d2/da/df/d10/d2a/d48/l70 0 2024-05-27T09:29:48.696 INFO:tasks.workunit.client.1.smithi138.stdout:8/868: dread d3/d8/d24/d4d/fce [0,4194304] 0 2024-05-27T09:29:48.698 INFO:tasks.workunit.client.1.smithi138.stdout:0/692: dwrite da/d2a/d8e/d43/f48 [4194304,4194304] 0 2024-05-27T09:29:48.703 INFO:tasks.workunit.client.1.smithi138.stdout:8/869: write d3/d8/d32/d34/d38/fc7 [4407928,656] 0 2024-05-27T09:29:48.703 INFO:tasks.workunit.client.1.smithi138.stdout:8/870: stat d3/d8/d11/deb/dd3/fd5 0 2024-05-27T09:29:48.703 INFO:tasks.workunit.client.1.smithi138.stdout:8/871: chown d3/d8/d24/d55/d94/da5/fb3 5 1 2024-05-27T09:29:48.705 INFO:tasks.workunit.client.1.smithi138.stdout:0/693: mknod da/d2a/d8e/d1f/dd5/ce0 0 2024-05-27T09:29:48.705 INFO:tasks.workunit.client.1.smithi138.stdout:8/872: mknod d3/d8/d24/d55/d94/da5/d111/d117/d11e/c12b 0 2024-05-27T09:29:48.705 INFO:tasks.workunit.client.1.smithi138.stdout:8/873: chown d3/d8/d32/lef 316649 1 2024-05-27T09:29:48.706 INFO:tasks.workunit.client.1.smithi138.stdout:8/874: symlink d3/d8/d32/l12c 0 2024-05-27T09:29:48.712 INFO:tasks.workunit.client.1.smithi138.stdout:6/919: dwrite d3/d11/dc0/d43/d1c/f59 [4194304,4194304] 0 2024-05-27T09:29:48.712 INFO:tasks.workunit.client.1.smithi138.stdout:1/710: dwrite d4/d1a/d2e/d32/d8c/dc6/f55 [0,4194304] 0 2024-05-27T09:29:48.712 INFO:tasks.workunit.client.1.smithi138.stdout:6/920: write d3/d11/dc0/d5a/d6a/f7f [906545,85016] 0 2024-05-27T09:29:48.713 INFO:tasks.workunit.client.1.smithi138.stdout:3/699: write d0/d3f/d50/f7c [2724733,80383] 0 2024-05-27T09:29:48.714 INFO:tasks.workunit.client.1.smithi138.stdout:1/711: symlink d4/d1a/d2e/d32/d8c/dc6/dd8/le9 0 2024-05-27T09:29:48.715 INFO:tasks.workunit.client.1.smithi138.stdout:3/700: mknod d0/d2/da/df/d10/d2a/d61/d73/deb/cee 0 2024-05-27T09:29:48.716 INFO:tasks.workunit.client.1.smithi138.stdout:6/921: symlink d3/d11/dc2/l12f 0 2024-05-27T09:29:48.716 INFO:tasks.workunit.client.1.smithi138.stdout:6/922: getdents d3/d11/dc0/d5a/d6a 0 2024-05-27T09:29:48.716 INFO:tasks.workunit.client.1.smithi138.stdout:3/701: dread - d0/d3f/f7f zero size 2024-05-27T09:29:48.716 INFO:tasks.workunit.client.1.smithi138.stdout:1/712: creat d4/d88/fea x:0 0 0 2024-05-27T09:29:48.719 INFO:tasks.workunit.client.1.smithi138.stdout:3/702: unlink d0/d2/da/df/f6f 0 2024-05-27T09:29:48.719 INFO:tasks.workunit.client.1.smithi138.stdout:3/703: stat d0/d2/da/df/d10/d2a/d48/f5f 0 2024-05-27T09:29:48.719 INFO:tasks.workunit.client.1.smithi138.stdout:3/704: getdents d0/d3f/d50/d9b 0 2024-05-27T09:29:48.719 INFO:tasks.workunit.client.1.smithi138.stdout:3/705: chown d0/d2/da/df/fe2 14348374 1 2024-05-27T09:29:48.720 INFO:tasks.workunit.client.1.smithi138.stdout:1/713: mkdir d4/d1a/deb 0 2024-05-27T09:29:48.720 INFO:tasks.workunit.client.1.smithi138.stdout:9/728: dwrite d6/da/fe9 [0,4194304] 0 2024-05-27T09:29:48.722 INFO:tasks.workunit.client.1.smithi138.stdout:1/714: mknod d4/daa/cec 0 2024-05-27T09:29:48.722 INFO:tasks.workunit.client.1.smithi138.stdout:1/715: stat d4/d48/d39/d5f/l87 0 2024-05-27T09:29:48.726 INFO:tasks.workunit.client.1.smithi138.stdout:8/875: write d3/f50 [3915573,38188] 0 2024-05-27T09:29:48.726 INFO:tasks.workunit.client.1.smithi138.stdout:8/876: readlink d3/d8/l21 0 2024-05-27T09:29:48.730 INFO:tasks.workunit.client.1.smithi138.stdout:1/716: rename d4/d1a/la7 to d4/d48/d22/led 0 2024-05-27T09:29:48.730 INFO:tasks.workunit.client.1.smithi138.stdout:3/706: symlink d0/d3f/d50/d9b/lef 0 2024-05-27T09:29:48.742 INFO:tasks.workunit.client.1.smithi138.stdout:9/729: dread d6/da/d29/ff2 [0,4194304] 0 2024-05-27T09:29:48.746 INFO:tasks.workunit.client.1.smithi138.stdout:4/786: dwrite d7/df/d35/d4c/dc0/fdb [0,4194304] 0 2024-05-27T09:29:48.747 INFO:tasks.workunit.client.1.smithi138.stdout:9/730: rmdir d6/db/de/d18/ddb/de8 0 2024-05-27T09:29:48.747 INFO:tasks.workunit.client.1.smithi138.stdout:9/731: creat d6/db/d8d/ff8 x:0 0 0 2024-05-27T09:29:48.751 INFO:tasks.workunit.client.1.smithi138.stdout:4/787: rename d7/df/d35/d4c/d59/d5d/d64/d6a/f94 to d7/df/d35/d4c/f116 0 2024-05-27T09:29:48.755 INFO:tasks.workunit.client.1.smithi138.stdout:4/788: creat d7/df/d35/d4c/d59/d5d/d64/df6/d10d/f117 x:0 0 0 2024-05-27T09:29:48.765 INFO:tasks.workunit.client.1.smithi138.stdout:1/717: dread d4/d1a/d2e/d4a/f8e [0,4194304] 0 2024-05-27T09:29:48.769 INFO:tasks.workunit.client.1.smithi138.stdout:1/718: truncate d4/d48/d39/d5f/f2f 297140 0 2024-05-27T09:29:48.770 INFO:tasks.workunit.client.1.smithi138.stdout:9/732: write d6/da/fe9 [238921,58278] 0 2024-05-27T09:29:48.770 INFO:tasks.workunit.client.1.smithi138.stdout:9/733: stat d6/da/d29/d61/l66 0 2024-05-27T09:29:48.770 INFO:tasks.workunit.client.1.smithi138.stdout:9/734: chown d6/db/l56 147 1 2024-05-27T09:29:48.770 INFO:tasks.workunit.client.1.smithi138.stdout:1/719: chown d4/d48/d39/d5f/l87 119 1 2024-05-27T09:29:48.770 INFO:tasks.workunit.client.1.smithi138.stdout:9/735: creat d6/da/d26/ff9 x:0 0 0 2024-05-27T09:29:48.773 INFO:tasks.workunit.client.1.smithi138.stdout:9/736: symlink d6/db/dc/dca/lfa 0 2024-05-27T09:29:48.776 INFO:tasks.workunit.client.1.smithi138.stdout:9/737: symlink d6/da/d29/d8f/db7/dbd/d46/lfb 0 2024-05-27T09:29:48.785 INFO:tasks.workunit.client.1.smithi138.stdout:6/923: dread d3/d11/dc0/d5a/d6a/f7f [0,4194304] 0 2024-05-27T09:29:48.801 INFO:tasks.workunit.client.1.smithi138.stdout:1/720: read d4/d1a/d2e/d32/d8c/dc6/f96 [366255,81099] 0 2024-05-27T09:29:48.801 INFO:tasks.workunit.client.1.smithi138.stdout:1/721: chown d4/d48/d39/d5f/d63/lc4 1047 1 2024-05-27T09:29:48.801 INFO:tasks.workunit.client.1.smithi138.stdout:1/722: write d4/d48/d22/d65/fce [1155370,69092] 0 2024-05-27T09:29:48.801 INFO:tasks.workunit.client.1.smithi138.stdout:1/723: chown d4/d48/d39/d5f 255087839 1 2024-05-27T09:29:48.803 INFO:tasks.workunit.client.1.smithi138.stdout:9/738: write d6/da/d29/fe3 [197368,75469] 0 2024-05-27T09:29:48.803 INFO:tasks.workunit.client.1.smithi138.stdout:0/694: dwrite da/d17/f8c [0,4194304] 0 2024-05-27T09:29:48.806 INFO:tasks.workunit.client.1.smithi138.stdout:1/724: mkdir d4/d3c/d8a/dee 0 2024-05-27T09:29:48.807 INFO:tasks.workunit.client.1.smithi138.stdout:9/739: link d6/da/d29/d59/lf4 d6/db/de/d18/lfc 0 2024-05-27T09:29:48.807 INFO:tasks.workunit.client.1.smithi138.stdout:6/924: write d3/d11/dc0/d43/dbe/f106 [839483,100908] 0 2024-05-27T09:29:48.809 INFO:tasks.workunit.client.1.smithi138.stdout:9/740: unlink d6/da/d26/d28/le2 0 2024-05-27T09:29:48.810 INFO:tasks.workunit.client.1.smithi138.stdout:9/741: truncate d6/da/d29/d59/f90 1013540 0 2024-05-27T09:29:48.813 INFO:tasks.workunit.client.1.smithi138.stdout:9/742: truncate d6/da/d29/d59/dee/d55/f8a 2535634 0 2024-05-27T09:29:48.818 INFO:tasks.workunit.client.1.smithi138.stdout:0/695: mknod da/d2a/d8e/d6a/daf/dc4/ce1 0 2024-05-27T09:29:48.819 INFO:tasks.workunit.client.1.smithi138.stdout:8/877: dwrite d3/d8/d32/d34/f10f [0,4194304] 0 2024-05-27T09:29:48.821 INFO:tasks.workunit.client.1.smithi138.stdout:8/878: symlink d3/l12d 0 2024-05-27T09:29:48.821 INFO:tasks.workunit.client.1.smithi138.stdout:8/879: truncate d3/d8/d32/d3f/f102 793755 0 2024-05-27T09:29:48.824 INFO:tasks.workunit.client.1.smithi138.stdout:1/725: mkdir d4/def 0 2024-05-27T09:29:48.826 INFO:tasks.workunit.client.1.smithi138.stdout:1/726: write d4/d3c/f6b [4699378,25313] 0 2024-05-27T09:29:48.826 INFO:tasks.workunit.client.1.smithi138.stdout:1/727: chown d4/d38/l4f 141656 1 2024-05-27T09:29:48.826 INFO:tasks.workunit.client.1.smithi138.stdout:8/880: symlink d3/d8/d11/df1/d10c/l12e 0 2024-05-27T09:29:48.827 INFO:tasks.workunit.client.1.smithi138.stdout:8/881: chown d3/d8/d24/f44 143133 1 2024-05-27T09:29:48.828 INFO:tasks.workunit.client.1.smithi138.stdout:8/882: getdents d3/d8/d24/d55/d94/da5/d111/d117/d5c 0 2024-05-27T09:29:48.828 INFO:tasks.workunit.client.1.smithi138.stdout:8/883: write d3/d8/d64/fca [2236715,60289] 0 2024-05-27T09:29:48.828 INFO:tasks.workunit.client.1.smithi138.stdout:8/884: write d3/d8/d32/d34/dbd/fe4 [647625,95490] 0 2024-05-27T09:29:48.829 INFO:tasks.workunit.client.1.smithi138.stdout:8/885: stat d3/d8/d24/d55/d94/la4 0 2024-05-27T09:29:48.832 INFO:tasks.workunit.client.1.smithi138.stdout:0/696: read da/d2a/d8e/d43/f64 [18396,42763] 0 2024-05-27T09:29:48.832 INFO:tasks.workunit.client.1.smithi138.stdout:8/886: write d3/d8/d64/fca [1765036,129038] 0 2024-05-27T09:29:48.833 INFO:tasks.workunit.client.1.smithi138.stdout:8/887: read d3/d8/d32/d34/f9a [41757,92925] 0 2024-05-27T09:29:48.838 INFO:tasks.workunit.client.1.smithi138.stdout:3/707: dwrite d0/f54 [0,4194304] 0 2024-05-27T09:29:48.841 INFO:tasks.workunit.client.1.smithi138.stdout:8/888: creat d3/d8/d72/d109/f12f x:0 0 0 2024-05-27T09:29:48.844 INFO:tasks.workunit.client.1.smithi138.stdout:0/697: rename da/d2a/d8e/d1f/d42/d63/d9d/fae to da/fe2 0 2024-05-27T09:29:48.846 INFO:tasks.workunit.client.1.smithi138.stdout:8/889: rmdir d3/d8/d24/d55/d7e 39 2024-05-27T09:29:48.846 INFO:tasks.workunit.client.1.smithi138.stdout:8/890: mknod d3/d8/d24/d55/c130 0 2024-05-27T09:29:48.863 INFO:tasks.workunit.client.1.smithi138.stdout:1/728: dread d4/d1a/d2e/fb0 [0,4194304] 0 2024-05-27T09:29:48.863 INFO:tasks.workunit.client.1.smithi138.stdout:1/729: fdatasync d4/d48/d39/d5f/d63/f98 0 2024-05-27T09:29:48.871 INFO:tasks.workunit.client.1.smithi138.stdout:2/623: sync 2024-05-27T09:29:48.877 INFO:tasks.workunit.client.1.smithi138.stdout:2/624: symlink d1/d15/d2d/d17/d25/ld5 0 2024-05-27T09:29:48.877 INFO:tasks.workunit.client.1.smithi138.stdout:1/730: dread d4/d1a/d2e/fb0 [0,4194304] 0 2024-05-27T09:29:48.878 INFO:tasks.workunit.client.1.smithi138.stdout:1/731: write d4/d3c/f42 [807016,70045] 0 2024-05-27T09:29:48.878 INFO:tasks.workunit.client.1.smithi138.stdout:5/740: sync 2024-05-27T09:29:48.878 INFO:tasks.workunit.client.1.smithi138.stdout:7/854: sync 2024-05-27T09:29:48.878 INFO:tasks.workunit.client.1.smithi138.stdout:7/855: readlink d0/d23/da8/l5c 0 2024-05-27T09:29:48.878 INFO:tasks.workunit.client.1.smithi138.stdout:7/856: dread - d0/d23/d49/fc1 zero size 2024-05-27T09:29:48.879 INFO:tasks.workunit.client.1.smithi138.stdout:5/741: read - d3/d12/d16/d1d/d22/d2b/d9e/db6/fe9 zero size 2024-05-27T09:29:48.879 INFO:tasks.workunit.client.1.smithi138.stdout:7/857: creat d0/d2b/daf/dbf/f116 x:0 0 0 2024-05-27T09:29:48.879 INFO:tasks.workunit.client.1.smithi138.stdout:2/625: rename d1/d15/d2d/d1d to d1/d15/dd6 0 2024-05-27T09:29:48.880 INFO:tasks.workunit.client.1.smithi138.stdout:9/743: dwrite d6/da/d29/d8f/db7/dbd/d4c/fd0 [0,4194304] 0 2024-05-27T09:29:48.882 INFO:tasks.workunit.client.1.smithi138.stdout:2/626: symlink d1/d10/d52/d5d/ld7 0 2024-05-27T09:29:48.884 INFO:tasks.workunit.client.1.smithi138.stdout:2/627: getdents d1/d15/d2d/d17/d25/d4d 0 2024-05-27T09:29:48.888 INFO:tasks.workunit.client.1.smithi138.stdout:2/628: write d1/d10/d52/db9/fcf [3035891,66998] 0 2024-05-27T09:29:48.888 INFO:tasks.workunit.client.1.smithi138.stdout:2/629: stat d1/d15/d2d/d17/d25/d9d/fb3 0 2024-05-27T09:29:48.888 INFO:tasks.workunit.client.1.smithi138.stdout:2/630: dread - d1/d15/d2d/d17/d25/d4d/d6d/fba zero size 2024-05-27T09:29:48.888 INFO:tasks.workunit.client.1.smithi138.stdout:2/631: dread - d1/d15/d2d/d17/d25/d9d/fc7 zero size 2024-05-27T09:29:48.888 INFO:tasks.workunit.client.1.smithi138.stdout:7/858: read d0/d2b/daf/d64/f6e [1013053,117030] 0 2024-05-27T09:29:48.888 INFO:tasks.workunit.client.1.smithi138.stdout:9/744: rmdir d6/db/d8d 39 2024-05-27T09:29:48.890 INFO:tasks.workunit.client.1.smithi138.stdout:9/745: link d6/da/d2f/f40 d6/da/d29/ffd 0 2024-05-27T09:29:48.896 INFO:tasks.workunit.client.1.smithi138.stdout:5/742: unlink d3/d12/d16/d1d/d30/f8b 0 2024-05-27T09:29:48.896 INFO:tasks.workunit.client.1.smithi138.stdout:1/732: read d4/d48/d39/d5f/f2f [216872,99835] 0 2024-05-27T09:29:48.897 INFO:tasks.workunit.client.1.smithi138.stdout:1/733: fdatasync d4/d1a/d2e/d32/d8c/dc6/f61 0 2024-05-27T09:29:48.897 INFO:tasks.workunit.client.1.smithi138.stdout:1/734: chown d4/daa 1 1 2024-05-27T09:29:48.901 INFO:tasks.workunit.client.1.smithi138.stdout:5/743: mkdir d3/d12/d32/d60/de6/df2 0 2024-05-27T09:29:48.901 INFO:tasks.workunit.client.1.smithi138.stdout:1/735: creat d4/d48/d39/d5f/d86/ff0 x:0 0 0 2024-05-27T09:29:48.906 INFO:tasks.workunit.client.1.smithi138.stdout:4/789: dwrite d7/df/f1a [0,4194304] 0 2024-05-27T09:29:48.908 INFO:tasks.workunit.client.1.smithi138.stdout:4/790: rmdir d7/dba/dcb 39 2024-05-27T09:29:48.914 INFO:tasks.workunit.client.1.smithi138.stdout:4/791: link d7/df/d35/d4c/d59/f9d d7/df/d35/d4c/d59/f118 0 2024-05-27T09:29:48.914 INFO:tasks.workunit.client.1.smithi138.stdout:4/792: chown d7/df/d35/d4c/f116 13 1 2024-05-27T09:29:48.924 INFO:tasks.workunit.client.1.smithi138.stdout:1/736: dread d4/d48/d22/d65/f6f [0,4194304] 0 2024-05-27T09:29:48.930 INFO:tasks.workunit.client.1.smithi138.stdout:8/891: dwrite d3/d8/d64/f2f [0,4194304] 0 2024-05-27T09:29:48.930 INFO:tasks.workunit.client.1.smithi138.stdout:8/892: stat d3/d8/d32/d34/f9a 0 2024-05-27T09:29:48.939 INFO:tasks.workunit.client.1.smithi138.stdout:1/737: creat d4/d1a/d2e/d4a/d7e/ff1 x:0 0 0 2024-05-27T09:29:48.945 INFO:tasks.workunit.client.1.smithi138.stdout:1/738: truncate d4/d1a/d2e/d32/d8c/dc6/f96 357297 0 2024-05-27T09:29:48.952 INFO:tasks.workunit.client.1.smithi138.stdout:8/893: getdents d3/d8/d24 0 2024-05-27T09:29:48.952 INFO:tasks.workunit.client.1.smithi138.stdout:8/894: truncate d3/d8/d11/df1/f11a 1393314 0 2024-05-27T09:29:48.953 INFO:tasks.workunit.client.1.smithi138.stdout:8/895: dread - d3/d8/f118 zero size 2024-05-27T09:29:48.953 INFO:tasks.workunit.client.1.smithi138.stdout:8/896: write d3/d8/d32/f59 [720832,12863] 0 2024-05-27T09:29:48.973 INFO:tasks.workunit.client.1.smithi138.stdout:8/897: dread d3/d8/d32/d3f/d9c/d9f/d86/fc6 [0,4194304] 0 2024-05-27T09:29:48.976 INFO:tasks.workunit.client.1.smithi138.stdout:9/746: dread d6/da/d29/d59/dee/d55/f8a [0,4194304] 0 2024-05-27T09:29:48.976 INFO:tasks.workunit.client.1.smithi138.stdout:6/925: dwrite d3/d11/dc0/d7d/de3/fef [0,4194304] 0 2024-05-27T09:29:48.979 INFO:tasks.workunit.client.1.smithi138.stdout:8/898: mkdir d3/d8/d32/df8/d131 0 2024-05-27T09:29:48.981 INFO:tasks.workunit.client.1.smithi138.stdout:9/747: mknod d6/cfe 0 2024-05-27T09:29:48.992 INFO:tasks.workunit.client.1.smithi138.stdout:0/698: dwrite da/d2a/dc9/fcd [0,4194304] 0 2024-05-27T09:29:48.992 INFO:tasks.workunit.client.1.smithi138.stdout:2/632: write d1/d15/d2d/d17/d25/d4d/d4e/f9e [1145572,61822] 0 2024-05-27T09:29:48.992 INFO:tasks.workunit.client.1.smithi138.stdout:2/633: readlink d1/d15/d2d/d17/d25/d9d/l5a 0 2024-05-27T09:29:49.000 INFO:tasks.workunit.client.1.smithi138.stdout:2/634: creat d1/d10/d52/d5d/fd8 x:0 0 0 2024-05-27T09:29:49.001 INFO:tasks.workunit.client.1.smithi138.stdout:6/926: dread d3/f53 [0,4194304] 0 2024-05-27T09:29:49.001 INFO:tasks.workunit.client.1.smithi138.stdout:5/744: dwrite d3/f15 [0,4194304] 0 2024-05-27T09:29:49.003 INFO:tasks.workunit.client.1.smithi138.stdout:2/635: unlink d1/d15/d2d/d17/l8b 0 2024-05-27T09:29:49.005 INFO:tasks.workunit.client.1.smithi138.stdout:5/745: mkdir d3/d12/d32/d7e/df3 0 2024-05-27T09:29:49.005 INFO:tasks.workunit.client.1.smithi138.stdout:2/636: symlink d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc/ld9 0 2024-05-27T09:29:49.006 INFO:tasks.workunit.client.1.smithi138.stdout:9/748: dread d6/da/d2f/f8b [0,4194304] 0 2024-05-27T09:29:49.008 INFO:tasks.workunit.client.1.smithi138.stdout:9/749: mknod d6/da/d29/d59/dee/d55/d75/dd8/cff 0 2024-05-27T09:29:49.008 INFO:tasks.workunit.client.1.smithi138.stdout:5/746: link d3/d12/f79 d3/d12/d26/ff4 0 2024-05-27T09:29:49.008 INFO:tasks.workunit.client.1.smithi138.stdout:5/747: read d3/d12/d26/fe8 [4609587,62765] 0 2024-05-27T09:29:49.008 INFO:tasks.workunit.client.1.smithi138.stdout:5/748: read d3/d12/d16/f2e [1164537,16575] 0 2024-05-27T09:29:49.008 INFO:tasks.workunit.client.1.smithi138.stdout:5/749: fdatasync d3/f4c 0 2024-05-27T09:29:49.009 INFO:tasks.workunit.client.1.smithi138.stdout:9/750: mknod d6/da/d29/d59/dee/d55/d75/c100 0 2024-05-27T09:29:49.009 INFO:tasks.workunit.client.1.smithi138.stdout:9/751: fsync d6/da/d26/ff9 0 2024-05-27T09:29:49.012 INFO:tasks.workunit.client.1.smithi138.stdout:7/859: dwrite d0/f24 [0,4194304] 0 2024-05-27T09:29:49.014 INFO:tasks.workunit.client.1.smithi138.stdout:7/860: mkdir d0/df7/d117 0 2024-05-27T09:29:49.015 INFO:tasks.workunit.client.1.smithi138.stdout:7/861: creat d0/d2b/daf/dbf/f118 x:0 0 0 2024-05-27T09:29:49.016 INFO:tasks.workunit.client.1.smithi138.stdout:5/750: symlink d3/d12/d16/d1d/d22/d2b/daf/db2/lf5 0 2024-05-27T09:29:49.017 INFO:tasks.workunit.client.1.smithi138.stdout:8/899: dread d3/d8/d24/d55/f67 [0,4194304] 0 2024-05-27T09:29:49.017 INFO:tasks.workunit.client.1.smithi138.stdout:7/862: creat d0/d23/d3c/f119 x:0 0 0 2024-05-27T09:29:49.018 INFO:tasks.workunit.client.1.smithi138.stdout:7/863: unlink d0/d23/l115 0 2024-05-27T09:29:49.019 INFO:tasks.workunit.client.1.smithi138.stdout:8/900: mknod d3/d8/d32/d3f/c132 0 2024-05-27T09:29:49.019 INFO:tasks.workunit.client.1.smithi138.stdout:2/637: dread d1/d15/d2d/d2e/f37 [0,4194304] 0 2024-05-27T09:29:49.019 INFO:tasks.workunit.client.1.smithi138.stdout:7/864: mknod d0/df7/c11a 0 2024-05-27T09:29:49.020 INFO:tasks.workunit.client.1.smithi138.stdout:8/901: rmdir d3/d8/d32/d3f 39 2024-05-27T09:29:49.024 INFO:tasks.workunit.client.1.smithi138.stdout:8/902: creat d3/d8/d11/df1/f133 x:0 0 0 2024-05-27T09:29:49.025 INFO:tasks.workunit.client.1.smithi138.stdout:8/903: creat d3/d8/d24/d55/d7e/d83/f134 x:0 0 0 2024-05-27T09:29:49.025 INFO:tasks.workunit.client.1.smithi138.stdout:8/904: dread - d3/d8/d24/d55/f113 zero size 2024-05-27T09:29:49.026 INFO:tasks.workunit.client.1.smithi138.stdout:8/905: creat d3/d8/d32/d3f/d9c/d9f/d86/f135 x:0 0 0 2024-05-27T09:29:49.032 INFO:tasks.workunit.client.1.smithi138.stdout:0/699: dread da/d2a/d8e/d1f/f3e [0,4194304] 0 2024-05-27T09:29:49.038 INFO:tasks.workunit.client.1.smithi138.stdout:8/906: write d3/d8/d11/df1/f11a [1309955,1292] 0 2024-05-27T09:29:49.039 INFO:tasks.workunit.client.1.smithi138.stdout:8/907: stat d3/l11f 0 2024-05-27T09:29:49.039 INFO:tasks.workunit.client.1.smithi138.stdout:8/908: dread - d3/d8/d11/deb/fd4 zero size 2024-05-27T09:29:49.042 INFO:tasks.workunit.client.1.smithi138.stdout:8/909: symlink d3/d8/d32/d34/d38/l136 0 2024-05-27T09:29:49.042 INFO:tasks.workunit.client.1.smithi138.stdout:8/910: dread - d3/d8/d32/d34/d38/f79 zero size 2024-05-27T09:29:49.070 INFO:tasks.workunit.client.1.smithi138.stdout:8/911: write d3/d8/d24/d55/d7e/f89 [1414823,114775] 0 2024-05-27T09:29:49.071 INFO:tasks.workunit.client.1.smithi138.stdout:8/912: creat d3/f137 x:0 0 0 2024-05-27T09:29:49.074 INFO:tasks.workunit.client.1.smithi138.stdout:8/913: rename d3/d8/d24/d55/d94/da5/d111/d117/d5c/f125 to d3/d8/d24/d55/d94/f138 0 2024-05-27T09:29:49.074 INFO:tasks.workunit.client.1.smithi138.stdout:8/914: chown d3/d8/d32/d88/f8e 2808957 1 2024-05-27T09:29:49.079 INFO:tasks.workunit.client.1.smithi138.stdout:6/927: dwrite d3/d11/dc0/d43/f3f [0,4194304] 0 2024-05-27T09:29:49.082 INFO:tasks.workunit.client.1.smithi138.stdout:6/928: creat d3/d11/d2d/d37/d102/f130 x:0 0 0 2024-05-27T09:29:49.082 INFO:tasks.workunit.client.1.smithi138.stdout:6/929: write d3/d11/dc0/f60 [1633983,40769] 0 2024-05-27T09:29:49.088 INFO:tasks.workunit.client.1.smithi138.stdout:6/930: rmdir d3/d11/dc0/d5a/df0/d104 39 2024-05-27T09:29:49.089 INFO:tasks.workunit.client.1.smithi138.stdout:6/931: creat d3/d11/dc0/d5a/f131 x:0 0 0 2024-05-27T09:29:49.096 INFO:tasks.workunit.client.1.smithi138.stdout:6/932: rename d3/d11/d2d/d37/dd1/c9a to d3/d11/d33/c132 0 2024-05-27T09:29:49.100 INFO:tasks.workunit.client.1.smithi138.stdout:8/915: write d3/d8/d32/d3f/d9c/d9f/d86/fc6 [388497,86333] 0 2024-05-27T09:29:49.100 INFO:tasks.workunit.client.1.smithi138.stdout:8/916: stat d3/d8/f42 0 2024-05-27T09:29:49.101 INFO:tasks.workunit.client.1.smithi138.stdout:8/917: chown d3/d8/d11/df1/f133 61 1 2024-05-27T09:29:49.101 INFO:tasks.workunit.client.1.smithi138.stdout:8/918: dread - d3/d8/d32/d3f/d9c/d9f/d86/f8b zero size 2024-05-27T09:29:49.101 INFO:tasks.workunit.client.1.smithi138.stdout:8/919: fdatasync d3/d8/d32/f3b 0 2024-05-27T09:29:49.104 INFO:tasks.workunit.client.1.smithi138.stdout:6/933: creat d3/d11/dc0/d43/d8c/f133 x:0 0 0 2024-05-27T09:29:49.104 INFO:tasks.workunit.client.1.smithi138.stdout:6/934: fdatasync d3/d11/d2d/d37/dd1/f12b 0 2024-05-27T09:29:49.106 INFO:tasks.workunit.client.1.smithi138.stdout:6/935: creat d3/d11/dc0/d43/dbe/f134 x:0 0 0 2024-05-27T09:29:49.107 INFO:tasks.workunit.client.1.smithi138.stdout:6/936: chown d3/d11/dc0/f80 1922 1 2024-05-27T09:29:49.109 INFO:tasks.workunit.client.1.smithi138.stdout:2/638: dwrite d1/d10/fe [0,4194304] 0 2024-05-27T09:29:49.110 INFO:tasks.workunit.client.1.smithi138.stdout:5/751: dwrite d3/d12/d32/d60/fa3 [0,4194304] 0 2024-05-27T09:29:49.111 INFO:tasks.workunit.client.1.smithi138.stdout:2/639: mknod d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc/cda 0 2024-05-27T09:29:49.111 INFO:tasks.workunit.client.1.smithi138.stdout:5/752: creat d3/d12/d16/d1d/d22/d2b/d9e/ff6 x:0 0 0 2024-05-27T09:29:49.116 INFO:tasks.workunit.client.1.smithi138.stdout:5/753: mknod d3/d12/d32/d60/cf7 0 2024-05-27T09:29:49.119 INFO:tasks.workunit.client.1.smithi138.stdout:8/920: dread d3/d8/d24/f10a [0,4194304] 0 2024-05-27T09:29:49.121 INFO:tasks.workunit.client.1.smithi138.stdout:9/752: dwrite d6/db/de/d18/fbe [0,4194304] 0 2024-05-27T09:29:49.125 INFO:tasks.workunit.client.1.smithi138.stdout:9/753: mkdir d6/da/d101 0 2024-05-27T09:29:49.126 INFO:tasks.workunit.client.1.smithi138.stdout:9/754: mkdir d6/da/d26/d28/d102 0 2024-05-27T09:29:49.126 INFO:tasks.workunit.client.1.smithi138.stdout:9/755: fsync d6/da/d26/fed 0 2024-05-27T09:29:49.141 INFO:tasks.workunit.client.1.smithi138.stdout:5/754: dread d3/d12/d26/d53/f58 [0,4194304] 0 2024-05-27T09:29:49.141 INFO:tasks.workunit.client.1.smithi138.stdout:5/755: readlink d3/d12/d16/d1d/d22/db9/l9d 0 2024-05-27T09:29:49.155 INFO:tasks.workunit.client.1.smithi138.stdout:5/756: creat d3/d12/d32/d87/dc0/ff8 x:0 0 0 2024-05-27T09:29:49.155 INFO:tasks.workunit.client.1.smithi138.stdout:9/756: read d6/da/d26/d28/f49 [1898220,42817] 0 2024-05-27T09:29:49.155 INFO:tasks.workunit.client.1.smithi138.stdout:9/757: truncate d6/da/d29/d59/dee/d55/f8a 3068027 0 2024-05-27T09:29:49.156 INFO:tasks.workunit.client.1.smithi138.stdout:8/921: write d3/d8/d32/d34/f10f [652939,125006] 0 2024-05-27T09:29:49.158 INFO:tasks.workunit.client.1.smithi138.stdout:9/758: symlink d6/db/de/d18/l103 0 2024-05-27T09:29:49.159 INFO:tasks.workunit.client.1.smithi138.stdout:8/922: mknod d3/d8/d32/d34/d11c/c139 0 2024-05-27T09:29:49.171 INFO:tasks.workunit.client.1.smithi138.stdout:5/757: dread d3/f15 [0,4194304] 0 2024-05-27T09:29:49.172 INFO:tasks.workunit.client.1.smithi138.stdout:5/758: truncate d3/d12/d32/d60/f8e 2813472 0 2024-05-27T09:29:49.172 INFO:tasks.workunit.client.1.smithi138.stdout:5/759: fsync d3/fd7 0 2024-05-27T09:29:49.172 INFO:tasks.workunit.client.1.smithi138.stdout:5/760: creat d3/dab/dc2/ff9 x:0 0 0 2024-05-27T09:29:49.180 INFO:tasks.workunit.client.1.smithi138.stdout:8/923: write d3/d8/d64/f2e [1989956,63786] 0 2024-05-27T09:29:49.180 INFO:tasks.workunit.client.1.smithi138.stdout:8/924: read d3/d8/d64/f2e [601134,2134] 0 2024-05-27T09:29:49.180 INFO:tasks.workunit.client.1.smithi138.stdout:8/925: write d3/d8/d32/d3f/d9c/f96 [1307392,72957] 0 2024-05-27T09:29:49.191 INFO:tasks.workunit.client.1.smithi138.stdout:8/926: write d3/d8/d24/f54 [3463180,93202] 0 2024-05-27T09:29:49.191 INFO:tasks.workunit.client.1.smithi138.stdout:8/927: dread - d3/d8/d24/d55/d7e/f116 zero size 2024-05-27T09:29:49.191 INFO:tasks.workunit.client.1.smithi138.stdout:8/928: truncate d3/d8/d32/d34/d38/fc7 5911458 0 2024-05-27T09:29:49.231 INFO:tasks.workunit.client.1.smithi138.stdout:6/937: dwrite d3/d11/d2d/d37/dd1/d47/f7a [4194304,4194304] 0 2024-05-27T09:29:49.231 INFO:tasks.workunit.client.1.smithi138.stdout:2/640: dwrite d1/d15/d2d/d17/f6b [0,4194304] 0 2024-05-27T09:29:49.233 INFO:tasks.workunit.client.1.smithi138.stdout:6/938: mknod d3/d11/d33/d100/c135 0 2024-05-27T09:29:49.234 INFO:tasks.workunit.client.1.smithi138.stdout:6/939: creat d3/d11/dc0/d7d/f136 x:0 0 0 2024-05-27T09:29:49.234 INFO:tasks.workunit.client.1.smithi138.stdout:6/940: dread - d3/d11/d33/ffd zero size 2024-05-27T09:29:49.238 INFO:tasks.workunit.client.1.smithi138.stdout:9/759: dwrite d6/da/fa9 [0,4194304] 0 2024-05-27T09:29:49.240 INFO:tasks.workunit.client.1.smithi138.stdout:6/941: link d3/d11/d33/f78 d3/d11/dc0/d5a/df0/d118/f137 0 2024-05-27T09:29:49.241 INFO:tasks.workunit.client.1.smithi138.stdout:8/929: dwrite d3/d8/d11/df1/f11a [0,4194304] 0 2024-05-27T09:29:49.241 INFO:tasks.workunit.client.1.smithi138.stdout:8/930: fsync d3/d8/d72/f115 0 2024-05-27T09:29:49.242 INFO:tasks.workunit.client.1.smithi138.stdout:9/760: mkdir d6/db/de/d6b/d104 0 2024-05-27T09:29:49.242 INFO:tasks.workunit.client.1.smithi138.stdout:6/942: unlink d3/d11/dc0/d43/l98 0 2024-05-27T09:29:49.244 INFO:tasks.workunit.client.1.smithi138.stdout:6/943: symlink d3/d11/d2d/d37/d8b/l138 0 2024-05-27T09:29:49.246 INFO:tasks.workunit.client.1.smithi138.stdout:8/931: rename d3/d8/d32/d34/d38/l136 to d3/d8/d11/df1/dae/l13a 0 2024-05-27T09:29:49.246 INFO:tasks.workunit.client.1.smithi138.stdout:8/932: write d3/d8/d32/d34/dbd/fe4 [1265508,48451] 0 2024-05-27T09:29:49.246 INFO:tasks.workunit.client.1.smithi138.stdout:8/933: write d3/d8/d24/d55/d94/da5/d111/d117/f4e [2513639,61964] 0 2024-05-27T09:29:49.247 INFO:tasks.workunit.client.1.smithi138.stdout:6/944: unlink d3/d11/dc0/d43/dbe/f106 0 2024-05-27T09:29:49.247 INFO:tasks.workunit.client.1.smithi138.stdout:9/761: symlink d6/da/d29/d8f/db7/dbd/d46/l105 0 2024-05-27T09:29:49.249 INFO:tasks.workunit.client.1.smithi138.stdout:5/761: dwrite d3/d12/d16/d1d/d22/d2b/fc1 [0,4194304] 0 2024-05-27T09:29:49.249 INFO:tasks.workunit.client.1.smithi138.stdout:5/762: readlink d3/l6 0 2024-05-27T09:29:49.249 INFO:tasks.workunit.client.1.smithi138.stdout:8/934: getdents d3/d8/d24/d55/d94/da5/d111/d117/dc2 0 2024-05-27T09:29:49.253 INFO:tasks.workunit.client.1.smithi138.stdout:2/641: dread d1/d15/f48 [4194304,4194304] 0 2024-05-27T09:29:49.254 INFO:tasks.workunit.client.1.smithi138.stdout:8/935: unlink d3/f29 0 2024-05-27T09:29:49.256 INFO:tasks.workunit.client.1.smithi138.stdout:9/762: link d6/da/d29/d8f/db7/dbd/d4c/fcc d6/db/de/d18/d24/f106 0 2024-05-27T09:29:49.257 INFO:tasks.workunit.client.1.smithi138.stdout:2/642: link d1/d15/d2d/d17/d25/d4d/d4e/f81 d1/d10/d52/fdb 0 2024-05-27T09:29:49.258 INFO:tasks.workunit.client.1.smithi138.stdout:2/643: chown d1/d15/l1e 12585 1 2024-05-27T09:29:49.269 INFO:tasks.workunit.client.1.smithi138.stdout:8/936: unlink d3/d8/d24/d55/f113 0 2024-05-27T09:29:49.269 INFO:tasks.workunit.client.1.smithi138.stdout:8/937: dread - d3/d8/d11/deb/dd3/fd5 zero size 2024-05-27T09:29:49.269 INFO:tasks.workunit.client.1.smithi138.stdout:8/938: fdatasync d3/d8/d32/d3f/d9c/d9f/d86/f93 0 2024-05-27T09:29:49.269 INFO:tasks.workunit.client.1.smithi138.stdout:9/763: rmdir d6/da/d101 0 2024-05-27T09:29:49.269 INFO:tasks.workunit.client.1.smithi138.stdout:5/763: dread d3/f15 [0,4194304] 0 2024-05-27T09:29:49.269 INFO:tasks.workunit.client.1.smithi138.stdout:5/764: dread - d3/d12/d16/d1d/d22/d2b/d9e/fc7 zero size 2024-05-27T09:29:49.271 INFO:tasks.workunit.client.1.smithi138.stdout:5/765: truncate d3/d12/d16/d1d/d23/f67 1715952 0 2024-05-27T09:29:49.271 INFO:tasks.workunit.client.1.smithi138.stdout:8/939: symlink d3/d8/d32/d34/l13b 0 2024-05-27T09:29:49.277 INFO:tasks.workunit.client.1.smithi138.stdout:5/766: rename d3/d12/d32/d60/fda to d3/d12/d32/d87/d89/dc3/de2/ffa 0 2024-05-27T09:29:49.277 INFO:tasks.workunit.client.1.smithi138.stdout:5/767: dread - d3/d12/d16/d1d/d22/db9/fb5 zero size 2024-05-27T09:29:49.277 INFO:tasks.workunit.client.1.smithi138.stdout:5/768: stat d3/d12/d16/d1d/d23/d68 0 2024-05-27T09:29:49.277 INFO:tasks.workunit.client.1.smithi138.stdout:9/764: symlink d6/da/d29/d8f/db7/dbd/l107 0 2024-05-27T09:29:49.277 INFO:tasks.workunit.client.1.smithi138.stdout:9/765: link d6/db/de/d18/d24/f7f d6/da/d29/d8f/db7/dbd/f108 0 2024-05-27T09:29:49.277 INFO:tasks.workunit.client.1.smithi138.stdout:9/766: fdatasync d6/da/d29/d8f/da8/fdd 0 2024-05-27T09:29:49.278 INFO:tasks.workunit.client.1.smithi138.stdout:9/767: symlink d6/db/dc/dca/l109 0 2024-05-27T09:29:49.278 INFO:tasks.workunit.client.1.smithi138.stdout:9/768: mkdir d6/db/de/d6b/d104/d10a 0 2024-05-27T09:29:49.280 INFO:tasks.workunit.client.1.smithi138.stdout:9/769: mkdir d6/db/dc/dca/d10b 0 2024-05-27T09:29:49.292 INFO:tasks.workunit.client.1.smithi138.stdout:5/769: dread d3/d12/d26/f57 [0,4194304] 0 2024-05-27T09:29:49.295 INFO:tasks.workunit.client.1.smithi138.stdout:5/770: mkdir d3/d12/d16/d1d/d22/db9/dfb 0 2024-05-27T09:29:49.301 INFO:tasks.workunit.client.1.smithi138.stdout:5/771: creat d3/d12/d16/d1d/d22/db9/dfb/ffc x:0 0 0 2024-05-27T09:29:49.301 INFO:tasks.workunit.client.1.smithi138.stdout:3/708: sync 2024-05-27T09:29:49.301 INFO:tasks.workunit.client.1.smithi138.stdout:3/709: chown d0/d3f/f88 19605 1 2024-05-27T09:29:49.301 INFO:tasks.workunit.client.1.smithi138.stdout:3/710: rmdir d0/d2/da/df/d10/d2a/db2/dd1 39 2024-05-27T09:29:49.303 INFO:tasks.workunit.client.1.smithi138.stdout:3/711: creat d0/d2/da/df/d10/d2a/d61/d73/deb/ff0 x:0 0 0 2024-05-27T09:29:49.305 INFO:tasks.workunit.client.1.smithi138.stdout:3/712: rename d0/d3f/d60/daf/le7 to d0/d2/da/lf1 0 2024-05-27T09:29:49.306 INFO:tasks.workunit.client.1.smithi138.stdout:3/713: dread - d0/d3f/d60/dcb/fdb zero size 2024-05-27T09:29:49.307 INFO:tasks.workunit.client.1.smithi138.stdout:3/714: creat d0/d3f/ff2 x:0 0 0 2024-05-27T09:29:49.309 INFO:tasks.workunit.client.1.smithi138.stdout:8/940: write d3/d8/d32/d3f/d9c/db8/ffa [2613688,40859] 0 2024-05-27T09:29:49.309 INFO:tasks.workunit.client.1.smithi138.stdout:3/715: mknod d0/d2/da/df/d10/d32/d37/cf3 0 2024-05-27T09:29:49.313 INFO:tasks.workunit.client.1.smithi138.stdout:3/716: rename d0/d2/da/cb to d0/d3f/d60/d93/de5/dca/cf4 0 2024-05-27T09:29:49.313 INFO:tasks.workunit.client.1.smithi138.stdout:3/717: write d0/d3f/f98 [1005632,19990] 0 2024-05-27T09:29:49.313 INFO:tasks.workunit.client.1.smithi138.stdout:3/718: fsync d0/d2/da/df/d10/d2a/f83 0 2024-05-27T09:29:49.313 INFO:tasks.workunit.client.1.smithi138.stdout:3/719: dread - d0/d2/da/df/d10/d2a/db2/dd1/fae zero size 2024-05-27T09:29:49.313 INFO:tasks.workunit.client.1.smithi138.stdout:3/720: read d0/d2/da/f8a [143021,81734] 0 2024-05-27T09:29:49.314 INFO:tasks.workunit.client.1.smithi138.stdout:8/941: mkdir d3/d8/d32/d3f/d13c 0 2024-05-27T09:29:49.314 INFO:tasks.workunit.client.1.smithi138.stdout:3/721: creat d0/d3f/d50/ff5 x:0 0 0 2024-05-27T09:29:49.314 INFO:tasks.workunit.client.1.smithi138.stdout:3/722: write d0/d3f/d50/f5c [786369,40646] 0 2024-05-27T09:29:49.316 INFO:tasks.workunit.client.1.smithi138.stdout:3/723: creat d0/d3f/d60/ff6 x:0 0 0 2024-05-27T09:29:49.316 INFO:tasks.workunit.client.1.smithi138.stdout:3/724: chown d0/d2/da/lf1 0 1 2024-05-27T09:29:49.316 INFO:tasks.workunit.client.1.smithi138.stdout:8/942: mknod d3/d8/d11/deb/dd3/c13d 0 2024-05-27T09:29:49.316 INFO:tasks.workunit.client.1.smithi138.stdout:6/945: dwrite d3/d11/dc0/d5a/d6a/fac [0,4194304] 0 2024-05-27T09:29:49.316 INFO:tasks.workunit.client.1.smithi138.stdout:9/770: write d6/db/dc/f50 [3963303,110035] 0 2024-05-27T09:29:49.317 INFO:tasks.workunit.client.1.smithi138.stdout:9/771: dread - d6/da/d29/d61/fcd zero size 2024-05-27T09:29:49.317 INFO:tasks.workunit.client.1.smithi138.stdout:9/772: write d6/db/de/fa7 [249842,18666] 0 2024-05-27T09:29:49.317 INFO:tasks.workunit.client.1.smithi138.stdout:9/773: write d6/db/de/d18/ddb/fe4 [874619,44744] 0 2024-05-27T09:29:49.317 INFO:tasks.workunit.client.1.smithi138.stdout:8/943: link d3/d8/d32/d34/lb9 d3/d8/d11/df1/d10c/l13e 0 2024-05-27T09:29:49.321 INFO:tasks.workunit.client.1.smithi138.stdout:9/774: mkdir d6/db/de/d18/d24/d10c 0 2024-05-27T09:29:49.329 INFO:tasks.workunit.client.1.smithi138.stdout:6/946: getdents d3/d11/d2d 0 2024-05-27T09:29:49.332 INFO:tasks.workunit.client.1.smithi138.stdout:6/947: rename d3/d11/d2d/d37/dd1/ded/f101 to d3/d11/d2d/d37/d8b/f139 0 2024-05-27T09:29:49.333 INFO:tasks.workunit.client.1.smithi138.stdout:8/944: dread d3/d8/d24/d4d/fce [0,4194304] 0 2024-05-27T09:29:49.334 INFO:tasks.workunit.client.1.smithi138.stdout:8/945: getdents d3/d8/d32/d34/dbd 0 2024-05-27T09:29:49.338 INFO:tasks.workunit.client.1.smithi138.stdout:6/948: rename d3/d11/d2d/f39 to d3/d11/dc0/d7d/da5/d10d/f13a 0 2024-05-27T09:29:49.341 INFO:tasks.workunit.client.1.smithi138.stdout:6/949: dread - d3/d11/dc0/d43/dbe/f134 zero size 2024-05-27T09:29:49.345 INFO:tasks.workunit.client.1.smithi138.stdout:9/775: dread d6/da/d2f/f5f [4194304,4194304] 0 2024-05-27T09:29:49.354 INFO:tasks.workunit.client.1.smithi138.stdout:8/946: write d3/d8/f4c [953895,6123] 0 2024-05-27T09:29:49.360 INFO:tasks.workunit.client.1.smithi138.stdout:5/772: dwrite d3/d12/d16/d1d/d22/d2b/fc1 [0,4194304] 0 2024-05-27T09:29:49.374 INFO:tasks.workunit.client.1.smithi138.stdout:9/776: dread d6/da/d29/d59/dee/d55/d75/f99 [0,4194304] 0 2024-05-27T09:29:49.385 INFO:tasks.workunit.client.1.smithi138.stdout:5/773: dread d3/d12/d16/d1d/d23/f62 [0,4194304] 0 2024-05-27T09:29:49.385 INFO:tasks.workunit.client.1.smithi138.stdout:3/725: dwrite d0/d2/da/df/d10/d2a/f83 [0,4194304] 0 2024-05-27T09:29:49.394 INFO:tasks.workunit.client.1.smithi138.stdout:6/950: dwrite d3/d11/dc0/d43/d71/fca [0,4194304] 0 2024-05-27T09:29:49.394 INFO:tasks.workunit.client.1.smithi138.stdout:6/951: chown d3/d11/dc0/d5a/db4/cf7 6411825 1 2024-05-27T09:29:49.397 INFO:tasks.workunit.client.1.smithi138.stdout:8/947: dwrite d3/d8/d64/f2e [0,4194304] 0 2024-05-27T09:29:49.397 INFO:tasks.workunit.client.1.smithi138.stdout:8/948: stat d3/d8/d32/f49 0 2024-05-27T09:29:49.400 INFO:tasks.workunit.client.1.smithi138.stdout:6/952: creat d3/d11/d2d/d37/d7e/f13b x:0 0 0 2024-05-27T09:29:49.400 INFO:tasks.workunit.client.1.smithi138.stdout:6/953: dread - d3/d11/d2d/d37/daa/df3/f129 zero size 2024-05-27T09:29:49.400 INFO:tasks.workunit.client.1.smithi138.stdout:6/954: dread - d3/d11/dc0/d5a/d6a/f109 zero size 2024-05-27T09:29:49.401 INFO:tasks.workunit.client.1.smithi138.stdout:6/955: creat d3/d11/d2d/d37/dd1/ded/f13c x:0 0 0 2024-05-27T09:29:49.401 INFO:tasks.workunit.client.1.smithi138.stdout:8/949: getdents d3/d8/d32/d34/d38/dd0 0 2024-05-27T09:29:49.403 INFO:tasks.workunit.client.1.smithi138.stdout:5/774: write d3/fd [5630931,42084] 0 2024-05-27T09:29:49.403 INFO:tasks.workunit.client.1.smithi138.stdout:3/726: rmdir d0/d3f/d60 39 2024-05-27T09:29:49.403 INFO:tasks.workunit.client.1.smithi138.stdout:3/727: creat d0/d2/da/df/d10/d2a/db2/ff7 x:0 0 0 2024-05-27T09:29:49.403 INFO:tasks.workunit.client.1.smithi138.stdout:3/728: fsync d0/d2/da/df/d10/d2a/f56 0 2024-05-27T09:29:49.403 INFO:tasks.workunit.client.1.smithi138.stdout:8/950: mknod d3/d8/d32/d34/d38/dd0/c13f 0 2024-05-27T09:29:49.411 INFO:tasks.workunit.client.1.smithi138.stdout:3/729: rename d0/d3f/d50/f5c to d0/d2/da/ff8 0 2024-05-27T09:29:49.412 INFO:tasks.workunit.client.1.smithi138.stdout:3/730: chown d0/d2/da/df/d10/d32/c14 2742 1 2024-05-27T09:29:49.476 INFO:tasks.workunit.client.1.smithi138.stdout:5/775: dwrite d3/d12/d16/d1d/f24 [0,4194304] 0 2024-05-27T09:29:49.477 INFO:tasks.workunit.client.1.smithi138.stdout:5/776: write d3/d12/d16/fa9 [1029422,118407] 0 2024-05-27T09:29:49.477 INFO:tasks.workunit.client.1.smithi138.stdout:8/951: dwrite d3/d8/f33 [8388608,4194304] 0 2024-05-27T09:29:49.480 INFO:tasks.workunit.client.1.smithi138.stdout:8/952: rename d3/d8/d24/d55/d94/da5/d111/d117/dc2/lf9 to d3/d8/d32/d3f/d9c/db8/l140 0 2024-05-27T09:29:49.480 INFO:tasks.workunit.client.1.smithi138.stdout:8/953: readlink d3/d8/d24/d55/d94/ld7 0 2024-05-27T09:29:49.481 INFO:tasks.workunit.client.1.smithi138.stdout:8/954: rmdir d3/d8/d32/d34/d38 39 2024-05-27T09:29:49.482 INFO:tasks.workunit.client.1.smithi138.stdout:8/955: creat d3/d8/d24/d55/d94/f141 x:0 0 0 2024-05-27T09:29:49.482 INFO:tasks.workunit.client.1.smithi138.stdout:5/777: symlink d3/d12/d16/d1d/d23/d3b/dec/lfd 0 2024-05-27T09:29:49.483 INFO:tasks.workunit.client.1.smithi138.stdout:8/956: mknod d3/d8/d24/d55/d94/da5/d111/c142 0 2024-05-27T09:29:49.484 INFO:tasks.workunit.client.1.smithi138.stdout:8/957: mkdir d3/d8/d72/d109/d143 0 2024-05-27T09:29:49.484 INFO:tasks.workunit.client.1.smithi138.stdout:8/958: stat d3/d8/d32/dd6/f10d 0 2024-05-27T09:29:49.486 INFO:tasks.workunit.client.1.smithi138.stdout:8/959: rename d3/d8/d24/d55/d94/la1 to d3/d8/d24/d55/d94/da5/l144 0 2024-05-27T09:29:49.487 INFO:tasks.workunit.client.1.smithi138.stdout:8/960: symlink d3/d8/d32/d3f/d9c/d9f/d86/l145 0 2024-05-27T09:29:49.489 INFO:tasks.workunit.client.1.smithi138.stdout:3/731: dwrite d0/d2/da/ff8 [0,4194304] 0 2024-05-27T09:29:49.497 INFO:tasks.workunit.client.1.smithi138.stdout:1/739: sync 2024-05-27T09:29:49.497 INFO:tasks.workunit.client.1.smithi138.stdout:1/740: chown d4/d1a/d2e/lb8 0 1 2024-05-27T09:29:49.497 INFO:tasks.workunit.client.1.smithi138.stdout:1/741: creat d4/d88/ff2 x:0 0 0 2024-05-27T09:29:49.499 INFO:tasks.workunit.client.1.smithi138.stdout:6/956: dwrite d3/d11/d2d/d37/dd1/d9b/d10a/f11a [0,4194304] 0 2024-05-27T09:29:49.499 INFO:tasks.workunit.client.1.smithi138.stdout:6/957: write d3/d11/dc0/d43/d1c/fbc [209867,32365] 0 2024-05-27T09:29:49.499 INFO:tasks.workunit.client.1.smithi138.stdout:4/793: sync 2024-05-27T09:29:49.499 INFO:tasks.workunit.client.1.smithi138.stdout:4/794: stat d7/df/fd0 0 2024-05-27T09:29:49.500 INFO:tasks.workunit.client.1.smithi138.stdout:6/958: creat d3/d11/d2d/d37/dd1/d9b/f13d x:0 0 0 2024-05-27T09:29:49.504 INFO:tasks.workunit.client.1.smithi138.stdout:1/742: mknod d4/d48/d39/d5f/d4d/cf3 0 2024-05-27T09:29:49.504 INFO:tasks.workunit.client.1.smithi138.stdout:5/778: write d3/d12/f77 [385285,15031] 0 2024-05-27T09:29:49.508 INFO:tasks.workunit.client.1.smithi138.stdout:8/961: dread f1 [0,4194304] 0 2024-05-27T09:29:49.516 INFO:tasks.workunit.client.1.smithi138.stdout:6/959: rename d3/d11/dc0/d43/d8c/f124 to d3/d11/dc0/d7d/f13e 0 2024-05-27T09:29:49.525 INFO:tasks.workunit.client.1.smithi138.stdout:5/779: creat d3/d12/ffe x:0 0 0 2024-05-27T09:29:49.528 INFO:tasks.workunit.client.1.smithi138.stdout:1/743: rename d4/d48/l3a to d4/d1a/d2e/d4a/d7e/lf4 0 2024-05-27T09:29:49.530 INFO:tasks.workunit.client.1.smithi138.stdout:4/795: write d7/df/d35/d4c/d59/d5d/fee [2593177,122246] 0 2024-05-27T09:29:49.531 INFO:tasks.workunit.client.1.smithi138.stdout:6/960: dread d3/d19/f38 [0,4194304] 0 2024-05-27T09:29:49.533 INFO:tasks.workunit.client.1.smithi138.stdout:8/962: rename d3/d8/d24/d4d/f97 to d3/d8/d32/d3f/d9c/d9f/d86/f146 0 2024-05-27T09:29:49.533 INFO:tasks.workunit.client.1.smithi138.stdout:8/963: chown d3/d8/d24/d55/d94/da5/d111/d117/d5c/l6e 11602614 1 2024-05-27T09:29:49.534 INFO:tasks.workunit.client.1.smithi138.stdout:1/744: mknod d4/d1a/d2e/cf5 0 2024-05-27T09:29:49.535 INFO:tasks.workunit.client.1.smithi138.stdout:8/964: link d3/d8/d32/d88/c9b d3/d8/d32/d3f/d9c/d9f/d103/c147 0 2024-05-27T09:29:49.538 INFO:tasks.workunit.client.1.smithi138.stdout:8/965: creat d3/f148 x:0 0 0 2024-05-27T09:29:49.542 INFO:tasks.workunit.client.1.smithi138.stdout:6/961: getdents d3/d11/d2d/d37/dd1 0 2024-05-27T09:29:49.547 INFO:tasks.workunit.client.1.smithi138.stdout:5/780: rename d3/d12/d32/d87/dc0/lcd to d3/d12/d32/d60/de6/lff 0 2024-05-27T09:29:49.547 INFO:tasks.workunit.client.1.smithi138.stdout:5/781: truncate d3/d12/d32/d6b/fca 27326 0 2024-05-27T09:29:49.547 INFO:tasks.workunit.client.1.smithi138.stdout:5/782: chown d3/d12/c1f 17 1 2024-05-27T09:29:49.571 INFO:tasks.workunit.client.1.smithi138.stdout:6/962: write d3/f31 [4282201,51745] 0 2024-05-27T09:29:49.572 INFO:tasks.workunit.client.1.smithi138.stdout:5/783: dread d3/d12/d32/d60/f8e [0,4194304] 0 2024-05-27T09:29:49.573 INFO:tasks.workunit.client.1.smithi138.stdout:6/963: rmdir d3/d11/d2d/d37/dd1/ded/d11e 0 2024-05-27T09:29:49.576 INFO:tasks.workunit.client.1.smithi138.stdout:6/964: creat d3/d11/dc0/d5a/d6a/f13f x:0 0 0 2024-05-27T09:29:49.580 INFO:tasks.workunit.client.1.smithi138.stdout:6/965: mknod d3/d11/d33/d100/c140 0 2024-05-27T09:29:49.586 INFO:tasks.workunit.client.1.smithi138.stdout:6/966: unlink d3/d11/d2d/cbb 0 2024-05-27T09:29:49.586 INFO:tasks.workunit.client.1.smithi138.stdout:6/967: truncate d3/d11/d2d/d37/d7e/fb0 581903 0 2024-05-27T09:29:49.589 INFO:tasks.workunit.client.1.smithi138.stdout:6/968: fdatasync d3/d19/f22 0 2024-05-27T09:29:49.589 INFO:tasks.workunit.client.1.smithi138.stdout:6/969: creat d3/d11/d2d/d37/d7e/f141 x:0 0 0 2024-05-27T09:29:49.594 INFO:tasks.workunit.client.1.smithi138.stdout:1/745: write d4/d48/d39/d5f/f2f [1001177,80344] 0 2024-05-27T09:29:49.602 INFO:tasks.workunit.client.1.smithi138.stdout:1/746: symlink d4/d3c/lf6 0 2024-05-27T09:29:49.604 INFO:tasks.workunit.client.1.smithi138.stdout:8/966: dread d3/f123 [0,4194304] 0 2024-05-27T09:29:49.606 INFO:tasks.workunit.client.1.smithi138.stdout:8/967: mknod d3/d8/d24/d55/d94/c149 0 2024-05-27T09:29:49.624 INFO:tasks.workunit.client.1.smithi138.stdout:8/968: read d3/d8/d24/d55/d94/da5/d111/d117/f4e [1362634,20410] 0 2024-05-27T09:29:49.626 INFO:tasks.workunit.client.1.smithi138.stdout:3/732: dwrite d0/fa4 [4194304,4194304] 0 2024-05-27T09:29:49.629 INFO:tasks.workunit.client.1.smithi138.stdout:3/733: getdents d0/d2/da/df/d10/d2a/d61 0 2024-05-27T09:29:49.656 INFO:tasks.workunit.client.1.smithi138.stdout:8/969: write d3/f4a [5297678,47250] 0 2024-05-27T09:29:49.658 INFO:tasks.workunit.client.1.smithi138.stdout:4/796: dwrite d7/df/d35/d4c/d59/d5d/d64/d6a/f76 [0,4194304] 0 2024-05-27T09:29:49.663 INFO:tasks.workunit.client.1.smithi138.stdout:4/797: creat d7/dba/f119 x:0 0 0 2024-05-27T09:29:49.663 INFO:tasks.workunit.client.1.smithi138.stdout:4/798: truncate d7/df/fd0 982682 0 2024-05-27T09:29:49.663 INFO:tasks.workunit.client.1.smithi138.stdout:5/784: dwrite d3/d12/d16/d1d/d22/d2b/d9e/db6/f9a [4194304,4194304] 0 2024-05-27T09:29:49.663 INFO:tasks.workunit.client.1.smithi138.stdout:0/700: sync 2024-05-27T09:29:49.663 INFO:tasks.workunit.client.1.smithi138.stdout:4/799: read - d7/dba/dcb/fe9 zero size 2024-05-27T09:29:49.663 INFO:tasks.workunit.client.1.smithi138.stdout:0/701: write da/d2a/d55/fb4 [2349393,13789] 0 2024-05-27T09:29:49.666 INFO:tasks.workunit.client.1.smithi138.stdout:7/865: sync 2024-05-27T09:29:49.667 INFO:tasks.workunit.client.1.smithi138.stdout:7/866: creat d0/d23/d3c/d84/d9c/f11b x:0 0 0 2024-05-27T09:29:49.667 INFO:tasks.workunit.client.1.smithi138.stdout:7/867: fsync d0/d23/d3a/fd5 0 2024-05-27T09:29:49.670 INFO:tasks.workunit.client.1.smithi138.stdout:7/868: mknod d0/d23/d3c/d84/d9c/d103/c11c 0 2024-05-27T09:29:49.672 INFO:tasks.workunit.client.1.smithi138.stdout:7/869: mkdir d0/d23/da8/d11d 0 2024-05-27T09:29:49.673 INFO:tasks.workunit.client.1.smithi138.stdout:7/870: write d0/d23/d3c/d84/d9c/f11b [535883,75971] 0 2024-05-27T09:29:49.673 INFO:tasks.workunit.client.1.smithi138.stdout:7/871: write d0/f19 [3088528,30494] 0 2024-05-27T09:29:49.676 INFO:tasks.workunit.client.1.smithi138.stdout:7/872: mknod d0/d2b/daf/dbf/c11e 0 2024-05-27T09:29:49.676 INFO:tasks.workunit.client.1.smithi138.stdout:7/873: truncate d0/d23/da8/fa2 5227700 0 2024-05-27T09:29:49.679 INFO:tasks.workunit.client.1.smithi138.stdout:5/785: mkdir d3/d12/d16/d1d/d23/d3b/dec/d100 0 2024-05-27T09:29:49.683 INFO:tasks.workunit.client.1.smithi138.stdout:4/800: getdents d7/dba/dcf/d3a 0 2024-05-27T09:29:49.686 INFO:tasks.workunit.client.1.smithi138.stdout:7/874: rmdir d0/d2b/daf/dba 39 2024-05-27T09:29:49.691 INFO:tasks.workunit.client.1.smithi138.stdout:7/875: chown d0/d2b/daf/l14 1 1 2024-05-27T09:29:49.694 INFO:tasks.workunit.client.1.smithi138.stdout:5/786: creat d3/d12/d32/d87/d89/f101 x:0 0 0 2024-05-27T09:29:49.694 INFO:tasks.workunit.client.1.smithi138.stdout:5/787: fsync d3/d12/d16/d1d/d23/d68/f6a 0 2024-05-27T09:29:49.694 INFO:tasks.workunit.client.1.smithi138.stdout:5/788: fdatasync d3/d12/d16/d1d/d23/d3b/fa2 0 2024-05-27T09:29:49.699 INFO:tasks.workunit.client.1.smithi138.stdout:5/789: rmdir d3/d12/d16/d1d/d22/db9/dfb 39 2024-05-27T09:29:49.699 INFO:tasks.workunit.client.1.smithi138.stdout:2/644: sync 2024-05-27T09:29:49.700 INFO:tasks.workunit.client.1.smithi138.stdout:9/777: sync 2024-05-27T09:29:49.700 INFO:tasks.workunit.client.1.smithi138.stdout:9/778: dread - d6/db/dc/dca/fec zero size 2024-05-27T09:29:49.700 INFO:tasks.workunit.client.1.smithi138.stdout:9/779: chown d6/da/dc7/cf7 73 1 2024-05-27T09:29:49.700 INFO:tasks.workunit.client.1.smithi138.stdout:2/645: chown d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc/f33 197551759 1 2024-05-27T09:29:49.701 INFO:tasks.workunit.client.1.smithi138.stdout:5/790: symlink d3/d12/d16/d1d/d22/db9/ddf/l102 0 2024-05-27T09:29:49.701 INFO:tasks.workunit.client.1.smithi138.stdout:5/791: read d3/d12/d26/fe8 [6049668,44091] 0 2024-05-27T09:29:49.702 INFO:tasks.workunit.client.1.smithi138.stdout:9/780: creat d6/db/de/d18/d24/f10d x:0 0 0 2024-05-27T09:29:49.703 INFO:tasks.workunit.client.1.smithi138.stdout:5/792: creat d3/d12/d16/d1d/d22/db9/ddf/f103 x:0 0 0 2024-05-27T09:29:49.709 INFO:tasks.workunit.client.1.smithi138.stdout:7/876: dread d0/d2b/daf/d43/f10a [4194304,4194304] 0 2024-05-27T09:29:49.709 INFO:tasks.workunit.client.1.smithi138.stdout:9/781: rename d6/da/d26 to d6/da/d29/d8f/db7/dbd/d10e 0 2024-05-27T09:29:49.718 INFO:tasks.workunit.client.1.smithi138.stdout:7/877: getdents d0/d23/d3c 0 2024-05-27T09:29:49.718 INFO:tasks.workunit.client.1.smithi138.stdout:1/747: dwrite d4/d1a/d2e/d4a/f8e [0,4194304] 0 2024-05-27T09:29:49.718 INFO:tasks.workunit.client.1.smithi138.stdout:9/782: mkdir d6/da/d29/d8f/db7/dbd/d46/d10f 0 2024-05-27T09:29:49.718 INFO:tasks.workunit.client.1.smithi138.stdout:6/970: dwrite d3/f53 [0,4194304] 0 2024-05-27T09:29:49.718 INFO:tasks.workunit.client.1.smithi138.stdout:9/783: truncate d6/da/d29/d8f/db7/dbd/fc5 1593355 0 2024-05-27T09:29:49.723 INFO:tasks.workunit.client.1.smithi138.stdout:1/748: rename d4/d48/d22/d65/lba to d4/d48/d39/lf7 0 2024-05-27T09:29:49.729 INFO:tasks.workunit.client.1.smithi138.stdout:6/971: symlink d3/d11/dc0/d43/dbe/l142 0 2024-05-27T09:29:49.737 INFO:tasks.workunit.client.1.smithi138.stdout:9/784: read d6/da/d29/d8f/db7/dbd/f108 [2681097,46892] 0 2024-05-27T09:29:49.738 INFO:tasks.workunit.client.1.smithi138.stdout:3/734: dwrite d0/fc4 [0,4194304] 0 2024-05-27T09:29:49.745 INFO:tasks.workunit.client.1.smithi138.stdout:1/749: creat d4/d38/d97/ff8 x:0 0 0 2024-05-27T09:29:49.746 INFO:tasks.workunit.client.1.smithi138.stdout:1/750: stat d4/d1a/d2e/d4a/d7e/dc0 0 2024-05-27T09:29:49.750 INFO:tasks.workunit.client.1.smithi138.stdout:4/801: dread f3 [4194304,4194304] 0 2024-05-27T09:29:49.750 INFO:tasks.workunit.client.1.smithi138.stdout:7/878: dread d0/d23/f4f [0,4194304] 0 2024-05-27T09:29:49.751 INFO:tasks.workunit.client.1.smithi138.stdout:7/879: write d0/d2b/daf/f12 [4374227,125694] 0 2024-05-27T09:29:49.754 INFO:tasks.workunit.client.1.smithi138.stdout:7/880: unlink d0/d2b/daf/d64/f7d 0 2024-05-27T09:29:49.757 INFO:tasks.workunit.client.1.smithi138.stdout:4/802: rename d7/df/f2e to d7/dba/dfa/f11a 0 2024-05-27T09:29:49.757 INFO:tasks.workunit.client.1.smithi138.stdout:4/803: stat d7/df/d35/d4c/f116 0 2024-05-27T09:29:49.760 INFO:tasks.workunit.client.1.smithi138.stdout:6/972: dread d3/d11/d2d/f75 [0,4194304] 0 2024-05-27T09:29:49.763 INFO:tasks.workunit.client.1.smithi138.stdout:4/804: mkdir d7/df/d35/d4c/dc0/dfb/d11b 0 2024-05-27T09:29:49.766 INFO:tasks.workunit.client.1.smithi138.stdout:6/973: getdents d3/d11/dc0/d5a/df0/d104 0 2024-05-27T09:29:49.766 INFO:tasks.workunit.client.1.smithi138.stdout:6/974: dread - d3/d11/d2d/d37/d8b/f139 zero size 2024-05-27T09:29:49.769 INFO:tasks.workunit.client.1.smithi138.stdout:8/970: dwrite d3/d8/d32/d34/d38/dd0/fa3 [0,4194304] 0 2024-05-27T09:29:49.769 INFO:tasks.workunit.client.1.smithi138.stdout:8/971: chown d3/d8/d11/deb/d120 301 1 2024-05-27T09:29:49.770 INFO:tasks.workunit.client.1.smithi138.stdout:8/972: stat d3/d8/d24/d55/d94/da5/d111/l11b 0 2024-05-27T09:29:49.777 INFO:tasks.workunit.client.1.smithi138.stdout:4/805: creat d7/df/d35/d4c/d59/d5d/d64/df6/f11c x:0 0 0 2024-05-27T09:29:49.797 INFO:tasks.workunit.client.1.smithi138.stdout:9/785: dread d6/db/de/d18/fbe [0,4194304] 0 2024-05-27T09:29:49.809 INFO:tasks.workunit.client.1.smithi138.stdout:0/702: dwrite da/d2a/d8e/d1f/d42/d4a/d53/f87 [0,4194304] 0 2024-05-27T09:29:49.810 INFO:tasks.workunit.client.1.smithi138.stdout:2/646: dwrite d1/d15/d2d/d17/f58 [0,4194304] 0 2024-05-27T09:29:49.810 INFO:tasks.workunit.client.1.smithi138.stdout:5/793: dwrite d3/d12/d16/d1d/f24 [0,4194304] 0 2024-05-27T09:29:49.821 INFO:tasks.workunit.client.1.smithi138.stdout:0/703: creat da/d2a/d8e/d1f/d42/fe3 x:0 0 0 2024-05-27T09:29:49.821 INFO:tasks.workunit.client.1.smithi138.stdout:7/881: write d0/d23/d3c/fca [1175716,120157] 0 2024-05-27T09:29:49.821 INFO:tasks.workunit.client.1.smithi138.stdout:7/882: creat d0/d23/d49/f11f x:0 0 0 2024-05-27T09:29:49.821 INFO:tasks.workunit.client.1.smithi138.stdout:7/883: chown d0/d23/d3a/f46 86027 1 2024-05-27T09:29:49.824 INFO:tasks.workunit.client.1.smithi138.stdout:5/794: unlink d3/d12/d32/d87/dc0/ldb 0 2024-05-27T09:29:49.825 INFO:tasks.workunit.client.1.smithi138.stdout:5/795: mknod d3/d12/d32/d60/de6/df2/c104 0 2024-05-27T09:29:49.826 INFO:tasks.workunit.client.1.smithi138.stdout:3/735: dwrite d0/d3f/f88 [0,4194304] 0 2024-05-27T09:29:49.831 INFO:tasks.workunit.client.1.smithi138.stdout:5/796: getdents d3/d12/d16/d1d/d22 0 2024-05-27T09:29:49.835 INFO:tasks.workunit.client.1.smithi138.stdout:5/797: creat d3/d12/d16/d1d/f105 x:0 0 0 2024-05-27T09:29:49.842 INFO:tasks.workunit.client.1.smithi138.stdout:5/798: rmdir d3/dab 39 2024-05-27T09:29:49.855 INFO:tasks.workunit.client.1.smithi138.stdout:3/736: mknod d0/d3f/d60/cf9 0 2024-05-27T09:29:49.862 INFO:tasks.workunit.client.1.smithi138.stdout:1/751: dwrite d4/d1a/d2e/d4a/d7e/dc0/fd0 [0,4194304] 0 2024-05-27T09:29:49.862 INFO:tasks.workunit.client.1.smithi138.stdout:3/737: unlink d0/d80/fdc 0 2024-05-27T09:29:49.862 INFO:tasks.workunit.client.1.smithi138.stdout:3/738: readlink d0/d2/da/df/ldf 0 2024-05-27T09:29:49.862 INFO:tasks.workunit.client.1.smithi138.stdout:3/739: getdents d0/d2/da/df/d10/d32/d37 0 2024-05-27T09:29:49.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:49 smithi138 ceph-mon[34885]: pgmap v22: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 215 MiB/s rd, 266 MiB/s wr, 412 op/s 2024-05-27T09:29:49.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:49 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:49.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:49 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:49.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:49 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:49.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:49 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:49.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:49 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:49.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:49 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:49.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:49 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:49.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:49 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:49.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:49 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:49.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:49 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:29:49.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:49 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:29:49.900 INFO:tasks.workunit.client.1.smithi138.stdout:6/975: dwrite d3/d11/dc0/d7d/fc5 [0,4194304] 0 2024-05-27T09:29:49.900 INFO:tasks.workunit.client.1.smithi138.stdout:3/740: write d0/d3f/d50/f97 [2518178,102955] 0 2024-05-27T09:29:49.904 INFO:tasks.workunit.client.1.smithi138.stdout:6/976: mknod d3/d11/dc0/d43/d71/c143 0 2024-05-27T09:29:49.909 INFO:tasks.workunit.client.1.smithi138.stdout:6/977: mknod d3/d11/dc0/d7d/da5/d10d/c144 0 2024-05-27T09:29:49.909 INFO:tasks.workunit.client.1.smithi138.stdout:6/978: stat d3/d11/dc0/d43/f11c 0 2024-05-27T09:29:49.909 INFO:tasks.workunit.client.1.smithi138.stdout:6/979: truncate d3/d11/dc0/d5a/db4/fc9 511721 0 2024-05-27T09:29:49.912 INFO:tasks.workunit.client.1.smithi138.stdout:0/704: dwrite da/fb8 [0,4194304] 0 2024-05-27T09:29:49.913 INFO:tasks.workunit.client.1.smithi138.stdout:6/980: symlink d3/d11/d2d/d37/dd1/d47/l145 0 2024-05-27T09:29:49.915 INFO:tasks.workunit.client.1.smithi138.stdout:4/806: dwrite d7/df/d35/d4c/d59/d5d/fe8 [0,4194304] 0 2024-05-27T09:29:49.915 INFO:tasks.workunit.client.1.smithi138.stdout:4/807: write d7/df/d35/d4c/d59/d5d/d64/df6/d10d/f117 [600745,111101] 0 2024-05-27T09:29:49.916 INFO:tasks.workunit.client.1.smithi138.stdout:6/981: mknod d3/d11/dc0/d43/d71/c146 0 2024-05-27T09:29:49.917 INFO:tasks.workunit.client.1.smithi138.stdout:6/982: write d3/d11/dc2/de1/f12d [5001815,126259] 0 2024-05-27T09:29:49.917 INFO:tasks.workunit.client.1.smithi138.stdout:4/808: creat d7/df/d35/d4c/d59/d5d/d64/df6/d10d/f11d x:0 0 0 2024-05-27T09:29:49.917 INFO:tasks.workunit.client.1.smithi138.stdout:4/809: fdatasync d7/df/d35/d4c/d59/f108 0 2024-05-27T09:29:49.917 INFO:tasks.workunit.client.1.smithi138.stdout:4/810: fsync d7/df/d35/d4c/d59/d5d/d64/fa7 0 2024-05-27T09:29:49.917 INFO:tasks.workunit.client.1.smithi138.stdout:4/811: truncate d7/df/d5b/f7f 516841 0 2024-05-27T09:29:49.923 INFO:tasks.workunit.client.1.smithi138.stdout:2/647: dwrite d1/d15/d2d/f39 [0,4194304] 0 2024-05-27T09:29:49.928 INFO:tasks.workunit.client.1.smithi138.stdout:2/648: link d1/d10/d52/l93 d1/d15/d2d/d17/d25/ldc 0 2024-05-27T09:29:49.928 INFO:tasks.workunit.client.1.smithi138.stdout:2/649: fdatasync d1/d15/d2d/d17/d25/d9d/f3b 0 2024-05-27T09:29:49.929 INFO:tasks.workunit.client.1.smithi138.stdout:2/650: creat d1/d15/d2d/dd2/fdd x:0 0 0 2024-05-27T09:29:49.931 INFO:tasks.workunit.client.1.smithi138.stdout:1/752: dwrite d4/d3c/fc8 [0,4194304] 0 2024-05-27T09:29:49.932 INFO:tasks.workunit.client.1.smithi138.stdout:1/753: write d4/d3c/f42 [464169,81153] 0 2024-05-27T09:29:49.934 INFO:tasks.workunit.client.1.smithi138.stdout:6/983: rename d3/d11/dc0/d5a/dcf to d3/d11/dc0/d147 0 2024-05-27T09:29:49.949 INFO:tasks.workunit.client.1.smithi138.stdout:1/754: symlink d4/d1a/d2e/d32/d8c/dc6/lf9 0 2024-05-27T09:29:49.951 INFO:tasks.workunit.client.1.smithi138.stdout:6/984: mknod d3/d11/d2d/d37/dd1/d9b/d10a/c148 0 2024-05-27T09:29:49.953 INFO:tasks.workunit.client.1.smithi138.stdout:6/985: creat d3/d11/dc0/d7d/db9/f149 x:0 0 0 2024-05-27T09:29:49.956 INFO:tasks.workunit.client.1.smithi138.stdout:6/986: rename d3/l8f to d3/d11/d2d/d37/d7e/l14a 0 2024-05-27T09:29:49.957 INFO:tasks.workunit.client.1.smithi138.stdout:6/987: creat d3/d11/dc0/d43/f14b x:0 0 0 2024-05-27T09:29:49.963 INFO:tasks.workunit.client.1.smithi138.stdout:5/799: dwrite d3/d12/d26/d53/f58 [0,4194304] 0 2024-05-27T09:29:49.963 INFO:tasks.workunit.client.1.smithi138.stdout:9/786: dwrite d6/da/d29/d8f/db7/fc4 [0,4194304] 0 2024-05-27T09:29:49.964 INFO:tasks.workunit.client.1.smithi138.stdout:9/787: read - d6/da/d29/d8f/db7/dbd/d10e/ff9 zero size 2024-05-27T09:29:49.968 INFO:tasks.workunit.client.1.smithi138.stdout:9/788: getdents d6/db 0 2024-05-27T09:29:49.969 INFO:tasks.workunit.client.1.smithi138.stdout:5/800: mknod d3/ddc/c106 0 2024-05-27T09:29:49.969 INFO:tasks.workunit.client.1.smithi138.stdout:5/801: fsync d3/d12/d32/d87/dc0/ff8 0 2024-05-27T09:29:49.979 INFO:tasks.workunit.client.1.smithi138.stdout:1/755: dread d4/d1a/fa9 [0,4194304] 0 2024-05-27T09:29:49.980 INFO:tasks.workunit.client.1.smithi138.stdout:1/756: dread - d4/d48/d39/d5f/d4d/d2b/fdd zero size 2024-05-27T09:29:49.980 INFO:tasks.workunit.client.1.smithi138.stdout:1/757: write d4/fa2 [1153109,35972] 0 2024-05-27T09:29:49.983 INFO:tasks.workunit.client.1.smithi138.stdout:1/758: mknod d4/d88/cfa 0 2024-05-27T09:29:49.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:49 smithi119 ceph-mon[29082]: pgmap v22: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 215 MiB/s rd, 266 MiB/s wr, 412 op/s 2024-05-27T09:29:49.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:49 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:49.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:49 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:49.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:49 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:49.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:49 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:49.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:49 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:49.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:49 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:49.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:49 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:49.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:49 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T09:29:49.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:49 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:49.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:49 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:29:49.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:49 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:29:49.999 INFO:tasks.workunit.client.1.smithi138.stdout:5/802: dread d3/d12/f79 [0,4194304] 0 2024-05-27T09:29:49.999 INFO:tasks.workunit.client.1.smithi138.stdout:5/803: chown d3/d12/cde 22456 1 2024-05-27T09:29:50.004 INFO:tasks.workunit.client.1.smithi138.stdout:5/804: rename d3/d12/d32/d60/de6/df2/c104 to d3/d12/d16/d1d/d22/db9/c107 0 2024-05-27T09:29:50.009 INFO:tasks.workunit.client.1.smithi138.stdout:1/759: dread d4/d1a/d2e/d4a/f80 [0,4194304] 0 2024-05-27T09:29:50.009 INFO:tasks.workunit.client.1.smithi138.stdout:0/705: dwrite da/d17/f8c [8388608,4194304] 0 2024-05-27T09:29:50.014 INFO:tasks.workunit.client.1.smithi138.stdout:1/760: getdents d4/d48/d39/d5f/d93 0 2024-05-27T09:29:50.019 INFO:tasks.workunit.client.1.smithi138.stdout:3/741: dwrite d0/d3f/d50/d9b/d9f/da3/fc1 [0,4194304] 0 2024-05-27T09:29:50.019 INFO:tasks.workunit.client.1.smithi138.stdout:3/742: write d0/d3f/d60/d93/de5/fd3 [690482,103589] 0 2024-05-27T09:29:50.021 INFO:tasks.workunit.client.1.smithi138.stdout:3/743: creat d0/d2/da/df/d10/d2a/db2/dd1/ffa x:0 0 0 2024-05-27T09:29:50.022 INFO:tasks.workunit.client.1.smithi138.stdout:3/744: readlink d0/l3a 0 2024-05-27T09:29:50.022 INFO:tasks.workunit.client.1.smithi138.stdout:3/745: fdatasync d0/d2/da/ff8 0 2024-05-27T09:29:50.022 INFO:tasks.workunit.client.1.smithi138.stdout:3/746: dread - d0/d3f/d50/ff5 zero size 2024-05-27T09:29:50.022 INFO:tasks.workunit.client.1.smithi138.stdout:3/747: chown d0/d2/da/df/d10/d2a/d48/c89 4 1 2024-05-27T09:29:50.040 INFO:tasks.workunit.client.1.smithi138.stdout:4/812: dwrite d7/df/d35/d4c/d59/d5d/d64/d6a/feb [0,4194304] 0 2024-05-27T09:29:50.040 INFO:tasks.workunit.client.1.smithi138.stdout:4/813: write d7/df/d5b/da1/db1/f111 [369730,77969] 0 2024-05-27T09:29:50.040 INFO:tasks.workunit.client.1.smithi138.stdout:4/814: truncate d7/f44 431103 0 2024-05-27T09:29:50.040 INFO:tasks.workunit.client.1.smithi138.stdout:5/805: dread d3/f88 [0,4194304] 0 2024-05-27T09:29:50.053 INFO:tasks.workunit.client.1.smithi138.stdout:5/806: getdents d3 0 2024-05-27T09:29:50.053 INFO:tasks.workunit.client.1.smithi138.stdout:5/807: creat d3/d12/d16/f108 x:0 0 0 2024-05-27T09:29:50.060 INFO:tasks.workunit.client.1.smithi138.stdout:9/789: write d6/da/d29/d59/dee/d55/f3a [3338008,126797] 0 2024-05-27T09:29:50.060 INFO:tasks.workunit.client.1.smithi138.stdout:9/790: creat d6/db/de/d18/ddb/f110 x:0 0 0 2024-05-27T09:29:50.065 INFO:tasks.workunit.client.1.smithi138.stdout:9/791: link d6/da/l36 d6/db/dc/dca/d10b/l111 0 2024-05-27T09:29:50.066 INFO:tasks.workunit.client.1.smithi138.stdout:5/808: unlink d3/dab/dc2/cef 0 2024-05-27T09:29:50.066 INFO:tasks.workunit.client.1.smithi138.stdout:5/809: creat d3/d12/d16/d1d/d22/d2b/d9e/db6/f109 x:0 0 0 2024-05-27T09:29:50.067 INFO:tasks.workunit.client.1.smithi138.stdout:9/792: link d6/db/de/c62 d6/da/d29/d8f/db7/c112 0 2024-05-27T09:29:50.068 INFO:tasks.workunit.client.1.smithi138.stdout:9/793: readlink d6/db/dc/dca/l109 0 2024-05-27T09:29:50.068 INFO:tasks.workunit.client.1.smithi138.stdout:9/794: chown d6/db/de/d6b/ldf 21121 1 2024-05-27T09:29:50.073 INFO:tasks.workunit.client.1.smithi138.stdout:6/988: dwrite d3/d19/fa7 [0,4194304] 0 2024-05-27T09:29:50.073 INFO:tasks.workunit.client.1.smithi138.stdout:9/795: creat d6/db/de/d18/d24/d72/f113 x:0 0 0 2024-05-27T09:29:50.074 INFO:tasks.workunit.client.1.smithi138.stdout:1/761: dread d4/d48/d39/d5f/d4d/f8b [0,4194304] 0 2024-05-27T09:29:50.081 INFO:tasks.workunit.client.1.smithi138.stdout:5/810: rename d3/d12/d16/d1d/d23/fd3 to d3/d12/d16/d1d/d22/d2b/d9e/db6/f10a 0 2024-05-27T09:29:50.088 INFO:tasks.workunit.client.1.smithi138.stdout:1/762: mkdir d4/d1a/d2e/d4a/d7e/db7/dfb 0 2024-05-27T09:29:50.088 INFO:tasks.workunit.client.1.smithi138.stdout:6/989: unlink d3/d19/fab 0 2024-05-27T09:29:50.092 INFO:tasks.workunit.client.1.smithi138.stdout:1/763: truncate d4/d48/d39/d5f/d4d/f8b 1057330 0 2024-05-27T09:29:50.092 INFO:tasks.workunit.client.1.smithi138.stdout:1/764: dread - d4/d48/d39/d5f/d63/fde zero size 2024-05-27T09:29:50.092 INFO:tasks.workunit.client.1.smithi138.stdout:1/765: chown d4/d3c/c95 988608171 1 2024-05-27T09:29:50.093 INFO:tasks.workunit.client.1.smithi138.stdout:6/990: mknod d3/d11/d2d/d37/d8b/dc1/c14c 0 2024-05-27T09:29:50.097 INFO:tasks.workunit.client.1.smithi138.stdout:6/991: truncate d3/d11/dc0/d43/d71/fdf 3333577 0 2024-05-27T09:29:50.098 INFO:tasks.workunit.client.1.smithi138.stdout:0/706: dwrite da/d2a/d8e/d6a/fd8 [0,4194304] 0 2024-05-27T09:29:50.098 INFO:tasks.workunit.client.1.smithi138.stdout:0/707: fsync da/d2a/d2b/f4d 0 2024-05-27T09:29:50.098 INFO:tasks.workunit.client.1.smithi138.stdout:0/708: chown da/d2a/d8e/d1f/d42/d63/l9f 3968 1 2024-05-27T09:29:50.099 INFO:tasks.workunit.client.1.smithi138.stdout:4/815: dwrite d7/df/d35/f8b [0,4194304] 0 2024-05-27T09:29:50.099 INFO:tasks.workunit.client.1.smithi138.stdout:1/766: write d4/d48/d39/d5f/f2f [860694,46556] 0 2024-05-27T09:29:50.100 INFO:tasks.workunit.client.1.smithi138.stdout:6/992: truncate d3/d11/f4f 2336655 0 2024-05-27T09:29:50.101 INFO:tasks.workunit.client.1.smithi138.stdout:9/796: getdents d6/da/d29/d8f/db7 0 2024-05-27T09:29:50.101 INFO:tasks.workunit.client.1.smithi138.stdout:0/709: truncate da/d2a/d8e/d1f/f26 2951928 0 2024-05-27T09:29:50.103 INFO:tasks.workunit.client.1.smithi138.stdout:9/797: mknod d6/da/d29/d8f/db7/dbd/d10e/c114 0 2024-05-27T09:29:50.104 INFO:tasks.workunit.client.1.smithi138.stdout:9/798: mkdir d6/db/d8d/d115 0 2024-05-27T09:29:50.106 INFO:tasks.workunit.client.1.smithi138.stdout:9/799: mknod d6/db/de/d6b/d104/c116 0 2024-05-27T09:29:50.106 INFO:tasks.workunit.client.1.smithi138.stdout:9/800: chown d6/db/de/d18/d24/d72/ccb 5 1 2024-05-27T09:29:50.108 INFO:tasks.workunit.client.1.smithi138.stdout:9/801: mknod d6/da/d29/d59/dee/d55/d75/d8c/c117 0 2024-05-27T09:29:50.110 INFO:tasks.workunit.client.1.smithi138.stdout:9/802: fdatasync d6/da/d2f/f32 0 2024-05-27T09:29:50.110 INFO:tasks.workunit.client.1.smithi138.stdout:9/803: readlink d6/da/d29/d8f/ld2 0 2024-05-27T09:29:50.110 INFO:tasks.workunit.client.1.smithi138.stdout:9/804: readlink d6/da/d29/d59/dee/d55/l34 0 2024-05-27T09:29:50.110 INFO:tasks.workunit.client.1.smithi138.stdout:9/805: chown d6/da/d29/d8f/db7/dbd/d10e/d28 9 1 2024-05-27T09:29:50.111 INFO:tasks.workunit.client.1.smithi138.stdout:1/767: rename d4/d48/d39/lf7 to d4/def/lfc 0 2024-05-27T09:29:50.112 INFO:tasks.workunit.client.1.smithi138.stdout:9/806: rmdir d6/da/d29/d8f/db7/dbd/d46 39 2024-05-27T09:29:50.112 INFO:tasks.workunit.client.1.smithi138.stdout:9/807: write d6/db/dc/f4d [4487462,76462] 0 2024-05-27T09:29:50.114 INFO:tasks.workunit.client.1.smithi138.stdout:9/808: creat d6/da/d29/d8f/db7/dbd/d10e/f118 x:0 0 0 2024-05-27T09:29:50.115 INFO:tasks.workunit.client.1.smithi138.stdout:0/710: creat da/d2a/d8e/dda/ddf/fe4 x:0 0 0 2024-05-27T09:29:50.115 INFO:tasks.workunit.client.1.smithi138.stdout:9/809: mkdir d6/db/de/d85/d119 0 2024-05-27T09:29:50.121 INFO:tasks.workunit.client.1.smithi138.stdout:3/748: dwrite d0/d2/da/df/d10/d32/f9 [4194304,4194304] 0 2024-05-27T09:29:50.121 INFO:tasks.workunit.client.1.smithi138.stdout:3/749: readlink d0/d2/da/df/d10/l1e 0 2024-05-27T09:29:50.128 INFO:tasks.workunit.client.1.smithi138.stdout:4/816: mknod d7/c11e 0 2024-05-27T09:29:50.134 INFO:tasks.workunit.client.1.smithi138.stdout:6/993: dread d3/d11/dc0/d43/d8c/f96 [0,4194304] 0 2024-05-27T09:29:50.136 INFO:tasks.workunit.client.1.smithi138.stdout:1/768: write d4/d3c/f4e [1432453,66067] 0 2024-05-27T09:29:50.141 INFO:tasks.workunit.client.1.smithi138.stdout:0/711: creat da/d2a/d8e/d43/fe5 x:0 0 0 2024-05-27T09:29:50.141 INFO:tasks.workunit.client.1.smithi138.stdout:0/712: stat da/d17 0 2024-05-27T09:29:50.141 INFO:tasks.workunit.client.1.smithi138.stdout:9/810: dread d6/da/d29/d8f/db7/dbd/d46/f47 [4194304,4194304] 0 2024-05-27T09:29:50.141 INFO:tasks.workunit.client.1.smithi138.stdout:0/713: dread - da/fe2 zero size 2024-05-27T09:29:50.141 INFO:tasks.workunit.client.1.smithi138.stdout:9/811: fsync d6/da/d29/fe3 0 2024-05-27T09:29:50.141 INFO:tasks.workunit.client.1.smithi138.stdout:4/817: creat d7/dba/dcf/d3a/f11f x:0 0 0 2024-05-27T09:29:50.144 INFO:tasks.workunit.client.1.smithi138.stdout:9/812: write d6/da/d29/f89 [4264723,60135] 0 2024-05-27T09:29:50.145 INFO:tasks.workunit.client.1.smithi138.stdout:3/750: dread d0/d3f/d50/f97 [0,4194304] 0 2024-05-27T09:29:50.147 INFO:tasks.workunit.client.1.smithi138.stdout:3/751: rmdir d0/d3f/d50/d9b 39 2024-05-27T09:29:50.148 INFO:tasks.workunit.client.1.smithi138.stdout:3/752: symlink d0/d2/lfb 0 2024-05-27T09:29:50.150 INFO:tasks.workunit.client.1.smithi138.stdout:3/753: symlink d0/d3f/d50/d9b/lfc 0 2024-05-27T09:29:50.150 INFO:tasks.workunit.client.1.smithi138.stdout:3/754: creat d0/dda/ffd x:0 0 0 2024-05-27T09:29:50.155 INFO:tasks.workunit.client.1.smithi138.stdout:0/714: write da/f66 [3594320,42946] 0 2024-05-27T09:29:50.156 INFO:tasks.workunit.client.1.smithi138.stdout:6/994: dread d3/d11/d2d/d37/f6d [0,4194304] 0 2024-05-27T09:29:50.156 INFO:tasks.workunit.client.1.smithi138.stdout:6/995: chown d3/d11/lb8 1733 1 2024-05-27T09:29:50.161 INFO:tasks.workunit.client.1.smithi138.stdout:1/769: rmdir d4/d1a/d2e/d32/d8c/dc6 39 2024-05-27T09:29:50.163 INFO:tasks.workunit.client.1.smithi138.stdout:4/818: rename d7/df/d35/d4c/d59/d5d/d64/d6a/f76 to d7/f120 0 2024-05-27T09:29:50.163 INFO:tasks.workunit.client.1.smithi138.stdout:4/819: dread - d7/df/d35/d4c/d59/d5d/d64/df6/d10d/f11d zero size 2024-05-27T09:29:50.164 INFO:tasks.workunit.client.1.smithi138.stdout:1/770: mknod d4/d88/cfd 0 2024-05-27T09:29:50.165 INFO:tasks.workunit.client.1.smithi138.stdout:1/771: rmdir d4/d48 39 2024-05-27T09:29:50.166 INFO:tasks.workunit.client.1.smithi138.stdout:9/813: rename d6/da/d29/d59/dee/d55/f76 to d6/da/d29/d61/f11a 0 2024-05-27T09:29:50.166 INFO:tasks.workunit.client.1.smithi138.stdout:1/772: mkdir d4/d38/d97/dfe 0 2024-05-27T09:29:50.166 INFO:tasks.workunit.client.1.smithi138.stdout:9/814: write d6/da/d29/d8f/db7/dbd/d10e/fed [462904,122884] 0 2024-05-27T09:29:50.175 INFO:tasks.workunit.client.1.smithi138.stdout:1/773: dread d4/d1a/d2e/d32/d8c/dc6/fbb [0,4194304] 0 2024-05-27T09:29:50.175 INFO:tasks.workunit.client.1.smithi138.stdout:1/774: chown d4/d48/d39/f85 406 1 2024-05-27T09:29:50.179 INFO:tasks.workunit.client.1.smithi138.stdout:5/811: dwrite d3/d12/d32/f4f [0,4194304] 0 2024-05-27T09:29:50.180 INFO:tasks.workunit.client.1.smithi138.stdout:0/715: dread da/d17/f44 [4194304,4194304] 0 2024-05-27T09:29:50.193 INFO:tasks.workunit.client.1.smithi138.stdout:3/755: rmdir d0/dda 39 2024-05-27T09:29:50.193 INFO:tasks.workunit.client.1.smithi138.stdout:3/756: rmdir d0/d3f/d60 39 2024-05-27T09:29:50.194 INFO:tasks.workunit.client.1.smithi138.stdout:3/757: creat d0/d2/da/df/d10/d2a/d61/d73/deb/ffe x:0 0 0 2024-05-27T09:29:50.194 INFO:tasks.workunit.client.1.smithi138.stdout:5/812: rename d3/f5b to d3/d12/d26/d53/f10b 0 2024-05-27T09:29:50.202 INFO:tasks.workunit.client.1.smithi138.stdout:0/716: getdents da/d17 0 2024-05-27T09:29:50.207 INFO:tasks.workunit.client.1.smithi138.stdout:5/813: link d3/d12/d16/d1d/d22/d2b/d9e/fc7 d3/d12/d16/d1d/d22/d2b/d9e/db6/f10c 0 2024-05-27T09:29:50.215 INFO:tasks.workunit.client.1.smithi138.stdout:7/884: sync 2024-05-27T09:29:50.215 INFO:tasks.workunit.client.1.smithi138.stdout:8/973: sync 2024-05-27T09:29:50.218 INFO:tasks.workunit.client.1.smithi138.stdout:7/885: symlink d0/d2b/daf/d98/db6/l120 0 2024-05-27T09:29:50.221 INFO:tasks.workunit.client.1.smithi138.stdout:0/717: truncate da/d2a/d8e/d43/d74/d97/fc2 2397363 0 2024-05-27T09:29:50.221 INFO:tasks.workunit.client.1.smithi138.stdout:0/718: chown da/d2a/d8e/d43/l96 1983 1 2024-05-27T09:29:50.222 INFO:tasks.workunit.client.1.smithi138.stdout:8/974: rmdir d3/d8 39 2024-05-27T09:29:50.223 INFO:tasks.workunit.client.1.smithi138.stdout:7/886: link d0/d23/d3c/d84/d9c/dc3/l85 d0/d2b/daf/d43/l121 0 2024-05-27T09:29:50.224 INFO:tasks.workunit.client.1.smithi138.stdout:8/975: rename d3/d8/d24/d55/d94/da5/d111/d117/f90 to d3/d8/d24/d55/d94/da5/d111/d117/d126/f14a 0 2024-05-27T09:29:50.229 INFO:tasks.workunit.client.1.smithi138.stdout:3/758: read d0/d2/f13 [2950419,29083] 0 2024-05-27T09:29:50.229 INFO:tasks.workunit.client.1.smithi138.stdout:8/976: unlink d3/d8/f4c 0 2024-05-27T09:29:50.229 INFO:tasks.workunit.client.1.smithi138.stdout:7/887: rmdir d0/d23/da8 39 2024-05-27T09:29:50.229 INFO:tasks.workunit.client.1.smithi138.stdout:7/888: fsync d0/f110 0 2024-05-27T09:29:50.230 INFO:tasks.workunit.client.1.smithi138.stdout:7/889: creat d0/d23/d3c/d84/d9c/dc3/f122 x:0 0 0 2024-05-27T09:29:50.241 INFO:tasks.workunit.client.1.smithi138.stdout:4/820: dwrite d7/df/d35/f8b [0,4194304] 0 2024-05-27T09:29:50.249 INFO:tasks.workunit.client.1.smithi138.stdout:1/775: dwrite d4/d48/d22/fa5 [0,4194304] 0 2024-05-27T09:29:50.256 INFO:tasks.workunit.client.1.smithi138.stdout:1/776: mknod d4/d38/d97/cff 0 2024-05-27T09:29:50.257 INFO:tasks.workunit.client.1.smithi138.stdout:3/759: write d0/d2/da/df/d10/f38 [3474510,16026] 0 2024-05-27T09:29:50.261 INFO:tasks.workunit.client.1.smithi138.stdout:9/815: dwrite d6/da/fa9 [0,4194304] 0 2024-05-27T09:29:50.261 INFO:tasks.workunit.client.1.smithi138.stdout:1/777: mkdir d4/d48/d22/d65/d100 0 2024-05-27T09:29:50.261 INFO:tasks.workunit.client.1.smithi138.stdout:1/778: write d4/d48/d39/d5f/d4d/d2b/fdd [73628,44059] 0 2024-05-27T09:29:50.261 INFO:tasks.workunit.client.1.smithi138.stdout:7/890: getdents d0/d2b/daf/d43 0 2024-05-27T09:29:50.261 INFO:tasks.workunit.client.1.smithi138.stdout:3/760: mknod d0/d2/da/df/d10/d2a/db2/cff 0 2024-05-27T09:29:50.262 INFO:tasks.workunit.client.1.smithi138.stdout:3/761: mkdir d0/d2/da/df/d100 0 2024-05-27T09:29:50.265 INFO:tasks.workunit.client.1.smithi138.stdout:1/779: creat d4/d1a/deb/f101 x:0 0 0 2024-05-27T09:29:50.267 INFO:tasks.workunit.client.1.smithi138.stdout:5/814: dwrite d3/f88 [0,4194304] 0 2024-05-27T09:29:50.267 INFO:tasks.workunit.client.1.smithi138.stdout:5/815: fsync d3/d12/d32/d87/f8d 0 2024-05-27T09:29:50.267 INFO:tasks.workunit.client.1.smithi138.stdout:1/780: creat d4/d1a/d2e/f102 x:0 0 0 2024-05-27T09:29:50.269 INFO:tasks.workunit.client.1.smithi138.stdout:5/816: symlink d3/d12/d16/d1d/d22/d2b/daf/db2/l10d 0 2024-05-27T09:29:50.269 INFO:tasks.workunit.client.1.smithi138.stdout:5/817: fsync d3/d12/f4d 0 2024-05-27T09:29:50.271 INFO:tasks.workunit.client.1.smithi138.stdout:5/818: creat d3/d12/d32/d60/de6/df2/f10e x:0 0 0 2024-05-27T09:29:50.272 INFO:tasks.workunit.client.1.smithi138.stdout:5/819: stat d3/ddc/c106 0 2024-05-27T09:29:50.275 INFO:tasks.workunit.client.1.smithi138.stdout:5/820: mkdir d3/d12/d32/d87/d89/d10f 0 2024-05-27T09:29:50.280 INFO:tasks.workunit.client.1.smithi138.stdout:0/719: dread da/d2a/d8e/d1f/d42/d4a/d53/f6e [0,4194304] 0 2024-05-27T09:29:50.280 INFO:tasks.workunit.client.1.smithi138.stdout:0/720: fdatasync da/d2a/d8e/dda/ddf/fe4 0 2024-05-27T09:29:50.283 INFO:tasks.workunit.client.1.smithi138.stdout:7/891: read d0/d2b/daf/f54 [4098260,75682] 0 2024-05-27T09:29:50.283 INFO:tasks.workunit.client.1.smithi138.stdout:7/892: fsync d0/f45 0 2024-05-27T09:29:50.283 INFO:tasks.workunit.client.1.smithi138.stdout:7/893: stat d0/d2b/daf/d98/fa4 0 2024-05-27T09:29:50.283 INFO:tasks.workunit.client.1.smithi138.stdout:5/821: symlink d3/dab/l110 0 2024-05-27T09:29:50.286 INFO:tasks.workunit.client.1.smithi138.stdout:0/721: rename da/d2a/d55/la1 to da/d2a/d8e/le6 0 2024-05-27T09:29:50.287 INFO:tasks.workunit.client.1.smithi138.stdout:5/822: getdents d3/d12/d16/d1d/d22/db9/dfb 0 2024-05-27T09:29:50.289 INFO:tasks.workunit.client.1.smithi138.stdout:0/722: symlink da/d2a/d8e/d1f/d42/d63/d9d/le7 0 2024-05-27T09:29:50.289 INFO:tasks.workunit.client.1.smithi138.stdout:0/723: readlink da/d2a/l8a 0 2024-05-27T09:29:50.291 INFO:tasks.workunit.client.1.smithi138.stdout:5/823: rename d3/d12/d32/d87/d89/dc3/de2 to d3/d12/d32/d60/de6/d111 0 2024-05-27T09:29:50.291 INFO:tasks.workunit.client.1.smithi138.stdout:5/824: stat d3/d12/d16/d1d/d22/d2b/c86 0 2024-05-27T09:29:50.291 INFO:tasks.workunit.client.1.smithi138.stdout:5/825: chown d3/d12/d16/d1d/f97 18402 1 2024-05-27T09:29:50.294 INFO:tasks.workunit.client.1.smithi138.stdout:0/724: mknod da/d2a/d8e/d43/d74/ce8 0 2024-05-27T09:29:50.295 INFO:tasks.workunit.client.1.smithi138.stdout:5/826: read d3/dab/fac [410615,7552] 0 2024-05-27T09:29:50.296 INFO:tasks.workunit.client.1.smithi138.stdout:5/827: symlink d3/d12/d32/d87/d89/dc3/l112 0 2024-05-27T09:29:50.299 INFO:tasks.workunit.client.1.smithi138.stdout:5/828: getdents d3/d12/d16/d1d/d23/d3b/dec 0 2024-05-27T09:29:50.301 INFO:tasks.workunit.client.1.smithi138.stdout:8/977: dwrite d3/d8/d24/d55/d7e/d83/f134 [0,4194304] 0 2024-05-27T09:29:50.305 INFO:tasks.workunit.client.1.smithi138.stdout:4/821: dread d7/df/d35/d4c/d59/d5d/d64/df6/d10d/f117 [0,4194304] 0 2024-05-27T09:29:50.305 INFO:tasks.workunit.client.1.smithi138.stdout:4/822: chown d7/df/d35/f115 31239161 1 2024-05-27T09:29:50.305 INFO:tasks.workunit.client.1.smithi138.stdout:5/829: creat d3/d12/d16/d1d/d22/d2b/daf/db2/f113 x:0 0 0 2024-05-27T09:29:50.306 INFO:tasks.workunit.client.1.smithi138.stdout:4/823: symlink d7/df/d35/d4c/dc0/dfb/d11b/l121 0 2024-05-27T09:29:50.306 INFO:tasks.workunit.client.1.smithi138.stdout:4/824: creat d7/dba/dcf/daa/f122 x:0 0 0 2024-05-27T09:29:50.309 INFO:tasks.workunit.client.1.smithi138.stdout:5/830: rename d3/d12/f4d to d3/d12/d26/d53/f114 0 2024-05-27T09:29:50.313 INFO:tasks.workunit.client.1.smithi138.stdout:4/825: rename d7/df/fd0 to d7/dba/dcf/daa/dda/dfc/f123 0 2024-05-27T09:29:50.314 INFO:tasks.workunit.client.1.smithi138.stdout:7/894: dread d0/f10 [0,4194304] 0 2024-05-27T09:29:50.314 INFO:tasks.workunit.client.1.smithi138.stdout:7/895: dread - d0/d23/d3c/f93 zero size 2024-05-27T09:29:50.317 INFO:tasks.workunit.client.1.smithi138.stdout:5/831: truncate d3/d12/d32/d6b/f78 2805757 0 2024-05-27T09:29:50.317 INFO:tasks.workunit.client.1.smithi138.stdout:5/832: creat d3/d12/f115 x:0 0 0 2024-05-27T09:29:50.317 INFO:tasks.workunit.client.1.smithi138.stdout:5/833: write d3/d12/d16/d1d/d22/db9/dfb/ffc [747175,91494] 0 2024-05-27T09:29:50.317 INFO:tasks.workunit.client.1.smithi138.stdout:5/834: chown d3/dab/dc2/dc6 4 1 2024-05-27T09:29:50.324 INFO:tasks.workunit.client.1.smithi138.stdout:8/978: dread d3/d8/d24/f10a [0,4194304] 0 2024-05-27T09:29:50.324 INFO:tasks.workunit.client.1.smithi138.stdout:1/781: dwrite d4/d48/f58 [0,4194304] 0 2024-05-27T09:29:50.337 INFO:tasks.workunit.client.1.smithi138.stdout:4/826: dread d7/df/d35/d4c/d59/f73 [0,4194304] 0 2024-05-27T09:29:50.337 INFO:tasks.workunit.client.1.smithi138.stdout:4/827: creat d7/df/d5b/da1/dbc/f124 x:0 0 0 2024-05-27T09:29:50.338 INFO:tasks.workunit.client.1.smithi138.stdout:4/828: creat d7/dba/f125 x:0 0 0 2024-05-27T09:29:50.340 INFO:tasks.workunit.client.1.smithi138.stdout:1/782: creat d4/d1a/d2e/d4a/d7e/db7/dfb/f103 x:0 0 0 2024-05-27T09:29:50.341 INFO:tasks.workunit.client.1.smithi138.stdout:1/783: fdatasync d4/d1a/d2e/f102 0 2024-05-27T09:29:50.369 INFO:tasks.workunit.client.1.smithi138.stdout:1/784: creat d4/d48/f104 x:0 0 0 2024-05-27T09:29:50.369 INFO:tasks.workunit.client.1.smithi138.stdout:1/785: readlink d4/l77 0 2024-05-27T09:29:50.371 INFO:tasks.workunit.client.1.smithi138.stdout:2/651: sync 2024-05-27T09:29:50.380 INFO:tasks.workunit.client.1.smithi138.stdout:1/786: getdents d4/d3c/d8a 0 2024-05-27T09:29:50.384 INFO:tasks.workunit.client.1.smithi138.stdout:8/979: dwrite d3/d8/d32/d34/dbd/fe4 [0,4194304] 0 2024-05-27T09:29:50.387 INFO:tasks.workunit.client.1.smithi138.stdout:1/787: mknod d4/d48/d22/d65/c105 0 2024-05-27T09:29:50.392 INFO:tasks.workunit.client.1.smithi138.stdout:4/829: dwrite d7/dba/dcf/f9a [4194304,4194304] 0 2024-05-27T09:29:50.393 INFO:tasks.workunit.client.1.smithi138.stdout:7/896: dwrite d0/d23/f5a [0,4194304] 0 2024-05-27T09:29:50.393 INFO:tasks.workunit.client.1.smithi138.stdout:7/897: write d0/d23/d3c/d84/fd1 [593054,87438] 0 2024-05-27T09:29:50.393 INFO:tasks.workunit.client.1.smithi138.stdout:7/898: write d0/d23/d3a/f55 [1176776,18948] 0 2024-05-27T09:29:50.395 INFO:tasks.workunit.client.1.smithi138.stdout:7/899: getdents d0/d2b/daf/d60 0 2024-05-27T09:29:50.405 INFO:tasks.workunit.client.1.smithi138.stdout:4/830: chown d7/df/d35/d4c/d59/d5d/d64/d6a/cb2 2220733 1 2024-05-27T09:29:50.412 INFO:tasks.workunit.client.1.smithi138.stdout:4/831: unlink d7/df/ldf 0 2024-05-27T09:29:50.412 INFO:tasks.workunit.client.1.smithi138.stdout:1/788: write d4/d3c/fc8 [739057,75455] 0 2024-05-27T09:29:50.416 INFO:tasks.workunit.client.1.smithi138.stdout:1/789: unlink d4/f24 0 2024-05-27T09:29:50.416 INFO:tasks.workunit.client.1.smithi138.stdout:1/790: chown d4/dcc/ce4 3 1 2024-05-27T09:29:50.423 INFO:tasks.workunit.client.1.smithi138.stdout:1/791: dread d4/d48/d39/d5f/d4d/d2b/fdd [0,4194304] 0 2024-05-27T09:29:50.430 INFO:tasks.workunit.client.1.smithi138.stdout:4/832: read d7/d6e/df8/f107 [1389562,39994] 0 2024-05-27T09:29:50.434 INFO:tasks.workunit.client.1.smithi138.stdout:1/792: creat d4/dc7/f106 x:0 0 0 2024-05-27T09:29:50.434 INFO:tasks.workunit.client.1.smithi138.stdout:1/793: fsync d4/f9b 0 2024-05-27T09:29:50.446 INFO:tasks.workunit.client.1.smithi138.stdout:1/794: truncate d4/d1a/d2e/d32/d8c/dc6/f96 52377 0 2024-05-27T09:29:50.463 INFO:tasks.workunit.client.1.smithi138.stdout:1/795: rename d4/d88/fea to d4/d48/d39/f107 0 2024-05-27T09:29:50.463 INFO:tasks.workunit.client.1.smithi138.stdout:1/796: creat d4/d3c/f108 x:0 0 0 2024-05-27T09:29:50.488 INFO:tasks.workunit.client.1.smithi138.stdout:2/652: dwrite d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc/fb4 [0,4194304] 0 2024-05-27T09:29:50.492 INFO:tasks.workunit.client.1.smithi138.stdout:4/833: dwrite d7/df/d35/d4c/fec [0,4194304] 0 2024-05-27T09:29:50.506 INFO:tasks.workunit.client.1.smithi138.stdout:2/653: creat d1/d15/d2d/d17/dac/fde x:0 0 0 2024-05-27T09:29:50.506 INFO:tasks.workunit.client.1.smithi138.stdout:2/654: chown d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc 349470692 1 2024-05-27T09:29:50.521 INFO:tasks.workunit.client.1.smithi138.stdout:4/834: write d7/dba/dcb/fff [34270,114494] 0 2024-05-27T09:29:50.521 INFO:tasks.workunit.client.1.smithi138.stdout:7/900: dwrite d0/d23/da8/fa2 [0,4194304] 0 2024-05-27T09:29:50.522 INFO:tasks.workunit.client.1.smithi138.stdout:7/901: chown d0/d23/d3a/cc0 1840 1 2024-05-27T09:29:50.527 INFO:tasks.workunit.client.1.smithi138.stdout:8/980: dwrite d3/d8/d24/f54 [0,4194304] 0 2024-05-27T09:29:50.544 INFO:tasks.workunit.client.1.smithi138.stdout:1/797: dwrite d4/d48/d39/f45 [0,4194304] 0 2024-05-27T09:29:50.549 INFO:tasks.workunit.client.1.smithi138.stdout:1/798: creat d4/d1a/d2e/d32/f109 x:0 0 0 2024-05-27T09:29:50.558 INFO:tasks.workunit.client.1.smithi138.stdout:1/799: rename d4/d3c/f4e to d4/d48/d22/d65/f10a 0 2024-05-27T09:29:50.563 INFO:tasks.workunit.client.1.smithi138.stdout:2/655: dwrite d1/d15/d2d/d17/d25/d9d/d4b/fa1 [0,4194304] 0 2024-05-27T09:29:50.565 INFO:tasks.workunit.client.1.smithi138.stdout:4/835: dread d7/df/d35/d4c/fec [0,4194304] 0 2024-05-27T09:29:50.567 INFO:tasks.workunit.client.1.smithi138.stdout:2/656: rename d1/d10/d3c to d1/d15/d2d/d17/d25/d4d/d4e/ddf 0 2024-05-27T09:29:50.570 INFO:tasks.workunit.client.1.smithi138.stdout:4/836: link d7/df/d35/d4c/d59/d5d/d64/d6a/cb5 d7/df/d35/d4c/d59/d5d/d64/df6/c126 0 2024-05-27T09:29:50.570 INFO:tasks.workunit.client.1.smithi138.stdout:4/837: write d7/df/d35/d4c/d59/d5d/fee [3358689,8993] 0 2024-05-27T09:29:50.573 INFO:tasks.workunit.client.1.smithi138.stdout:4/838: write d7/df/d35/d4c/d59/d5d/d64/df6/f11c [857183,31709] 0 2024-05-27T09:29:50.574 INFO:tasks.workunit.client.1.smithi138.stdout:4/839: mkdir d7/dba/dfa/d127 0 2024-05-27T09:29:50.579 INFO:tasks.workunit.client.1.smithi138.stdout:8/981: dread d3/d8/d11/fa6 [0,4194304] 0 2024-05-27T09:29:50.583 INFO:tasks.workunit.client.1.smithi138.stdout:4/840: truncate d7/dba/dcf/daa/dda/dfc/f123 879509 0 2024-05-27T09:29:50.583 INFO:tasks.workunit.client.1.smithi138.stdout:1/800: dread d4/d1a/fa9 [0,4194304] 0 2024-05-27T09:29:50.589 INFO:tasks.workunit.client.1.smithi138.stdout:7/902: dwrite d0/d23/f50 [4194304,4194304] 0 2024-05-27T09:29:50.589 INFO:tasks.workunit.client.1.smithi138.stdout:7/903: dread - d0/d2b/f5e zero size 2024-05-27T09:29:50.591 INFO:tasks.workunit.client.1.smithi138.stdout:7/904: mkdir d0/d23/d3c/d123 0 2024-05-27T09:29:50.591 INFO:tasks.workunit.client.1.smithi138.stdout:7/905: chown d0/d23/d49 449 1 2024-05-27T09:29:50.593 INFO:tasks.workunit.client.1.smithi138.stdout:2/657: write d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc/f83 [304961,55907] 0 2024-05-27T09:29:50.593 INFO:tasks.workunit.client.1.smithi138.stdout:2/658: dread - d1/d15/d2d/d17/d25/d4d/d6d/d87/fbf zero size 2024-05-27T09:29:50.593 INFO:tasks.workunit.client.1.smithi138.stdout:2/659: stat d1/d15/d2d/d17/c70 0 2024-05-27T09:29:50.593 INFO:tasks.workunit.client.1.smithi138.stdout:2/660: creat d1/d15/d2d/dd2/fe0 x:0 0 0 2024-05-27T09:29:50.593 INFO:tasks.workunit.client.1.smithi138.stdout:2/661: chown d1/d15/l55 2251306 1 2024-05-27T09:29:50.593 INFO:tasks.workunit.client.1.smithi138.stdout:7/906: symlink d0/d23/d49/dfe/l124 0 2024-05-27T09:29:50.593 INFO:tasks.workunit.client.1.smithi138.stdout:7/907: write d0/d2b/f5e [719203,98671] 0 2024-05-27T09:29:50.593 INFO:tasks.workunit.client.1.smithi138.stdout:7/908: write d0/d23/da8/d8e/fd7 [899238,110594] 0 2024-05-27T09:29:50.593 INFO:tasks.workunit.client.1.smithi138.stdout:7/909: chown d0/d23/d3c/d84/d9c/l112 62467099 1 2024-05-27T09:29:50.595 INFO:tasks.workunit.client.1.smithi138.stdout:2/662: symlink d1/d15/d2d/d17/d25/d4d/db6/le1 0 2024-05-27T09:29:50.595 INFO:tasks.workunit.client.1.smithi138.stdout:2/663: write d1/d15/d2d/d17/d25/d4d/d6d/d87/fc0 [987028,88181] 0 2024-05-27T09:29:50.596 INFO:tasks.workunit.client.1.smithi138.stdout:7/910: symlink d0/d2b/daf/dbf/dc8/l125 0 2024-05-27T09:29:50.596 INFO:tasks.workunit.client.1.smithi138.stdout:2/664: unlink d1/c21 0 2024-05-27T09:29:50.602 INFO:tasks.workunit.client.1.smithi138.stdout:1/801: mkdir d4/d48/d39/d5f/dd3/d10b 0 2024-05-27T09:29:50.602 INFO:tasks.workunit.client.1.smithi138.stdout:1/802: fsync d4/d1a/d2e/d32/f109 0 2024-05-27T09:29:50.602 INFO:tasks.workunit.client.1.smithi138.stdout:1/803: readlink d4/d3c/l5d 0 2024-05-27T09:29:50.602 INFO:tasks.workunit.client.1.smithi138.stdout:1/804: readlink d4/d38/l4f 0 2024-05-27T09:29:50.602 INFO:tasks.workunit.client.1.smithi138.stdout:1/805: fdatasync d4/d3c/f108 0 2024-05-27T09:29:50.602 INFO:tasks.workunit.client.1.smithi138.stdout:1/806: write d4/d48/d39/d5f/d86/ff0 [546851,106617] 0 2024-05-27T09:29:50.603 INFO:tasks.workunit.client.1.smithi138.stdout:1/807: write d4/d1a/d2e/f41 [4752028,45777] 0 2024-05-27T09:29:50.605 INFO:tasks.workunit.client.1.smithi138.stdout:6/996: sync 2024-05-27T09:29:50.624 INFO:tasks.workunit.client.1.smithi138.stdout:1/808: mknod d4/d88/d9f/c10c 0 2024-05-27T09:29:50.624 INFO:tasks.workunit.client.1.smithi138.stdout:6/997: rename d3/d11/d2d/d37/dd1/d47 to d3/d11/d2d/d37/d8b/d14d 0 2024-05-27T09:29:50.624 INFO:tasks.workunit.client.1.smithi138.stdout:9/816: sync 2024-05-27T09:29:50.624 INFO:tasks.workunit.client.1.smithi138.stdout:9/817: stat d6/db/de/d18/d24/f106 0 2024-05-27T09:29:50.624 INFO:tasks.workunit.client.1.smithi138.stdout:3/762: sync 2024-05-27T09:29:50.624 INFO:tasks.workunit.client.1.smithi138.stdout:3/763: fdatasync d0/d2/da/f33 0 2024-05-27T09:29:50.626 INFO:tasks.workunit.client.1.smithi138.stdout:9/818: truncate d6/db/de/d18/fbe 2920194 0 2024-05-27T09:29:50.627 INFO:tasks.workunit.client.1.smithi138.stdout:9/819: readlink d6/da/d29/l60 0 2024-05-27T09:29:50.627 INFO:tasks.workunit.client.1.smithi138.stdout:9/820: write d6/da/fe9 [12134,98597] 0 2024-05-27T09:29:50.627 INFO:tasks.workunit.client.1.smithi138.stdout:0/725: sync 2024-05-27T09:29:50.627 INFO:tasks.workunit.client.1.smithi138.stdout:5/835: sync 2024-05-27T09:29:50.627 INFO:tasks.workunit.client.1.smithi138.stdout:5/836: chown d3/d12/dcf/ce1 0 1 2024-05-27T09:29:50.639 INFO:tasks.workunit.client.1.smithi138.stdout:9/821: creat d6/da/d29/d8f/db7/dbd/d10e/f11b x:0 0 0 2024-05-27T09:29:50.639 INFO:tasks.workunit.client.1.smithi138.stdout:9/822: fsync d6/f9 0 2024-05-27T09:29:50.640 INFO:tasks.workunit.client.1.smithi138.stdout:9/823: write d6/fbc [636268,128903] 0 2024-05-27T09:29:50.643 INFO:tasks.workunit.client.1.smithi138.stdout:9/824: creat d6/da/d29/d8f/db7/dbd/d10e/d28/d102/f11c x:0 0 0 2024-05-27T09:29:50.648 INFO:tasks.workunit.client.1.smithi138.stdout:6/998: dread d3/d11/dc0/d43/d1c/f9c [0,4194304] 0 2024-05-27T09:29:50.648 INFO:tasks.workunit.client.1.smithi138.stdout:6/999: write d3/d11/d2d/d37/f3c [2798297,85965] 0 2024-05-27T09:29:50.649 INFO:tasks.workunit.client.1.smithi138.stdout:0/726: symlink da/d2a/d8e/d1f/le9 0 2024-05-27T09:29:50.657 INFO:tasks.workunit.client.1.smithi138.stdout:0/727: mkdir da/d2a/d8e/d1f/d42/d63/dea 0 2024-05-27T09:29:50.663 INFO:tasks.workunit.client.1.smithi138.stdout:5/837: dread d3/d12/d32/d7e/f9f [0,4194304] 0 2024-05-27T09:29:50.664 INFO:tasks.workunit.client.1.smithi138.stdout:5/838: readlink d3/d12/d16/d1d/lce 0 2024-05-27T09:29:50.674 INFO:tasks.workunit.client.1.smithi138.stdout:3/764: read d0/d2/da/ff8 [2428415,56114] 0 2024-05-27T09:29:50.676 INFO:tasks.workunit.client.1.smithi138.stdout:0/728: unlink da/d2a/d8e/d1f/d42/d4a/c81 0 2024-05-27T09:29:50.677 INFO:tasks.workunit.client.1.smithi138.stdout:3/765: creat d0/d2/da/df/d10/d2a/d61/de1/f101 x:0 0 0 2024-05-27T09:29:50.678 INFO:tasks.workunit.client.1.smithi138.stdout:3/766: symlink d0/d3f/d50/d9b/d9f/da3/l102 0 2024-05-27T09:29:50.680 INFO:tasks.workunit.client.1.smithi138.stdout:3/767: getdents d0/d3f/d60 0 2024-05-27T09:29:50.685 INFO:tasks.workunit.client.1.smithi138.stdout:4/841: dwrite d7/df/d35/d4c/d59/f9d [0,4194304] 0 2024-05-27T09:29:50.686 INFO:tasks.workunit.client.1.smithi138.stdout:7/911: dwrite d0/f8c [0,4194304] 0 2024-05-27T09:29:50.687 INFO:tasks.workunit.client.1.smithi138.stdout:5/839: creat d3/dab/f116 x:0 0 0 2024-05-27T09:29:50.732 INFO:tasks.workunit.client.1.smithi138.stdout:1/809: dwrite d4/d1a/d2e/f79 [0,4194304] 0 2024-05-27T09:29:50.732 INFO:tasks.workunit.client.1.smithi138.stdout:9/825: dwrite d6/da/d29/d8f/db7/dbd/d10e/d28/fea [0,4194304] 0 2024-05-27T09:29:50.733 INFO:tasks.workunit.client.1.smithi138.stdout:2/665: dwrite d1/d10/f64 [0,4194304] 0 2024-05-27T09:29:50.733 INFO:tasks.workunit.client.1.smithi138.stdout:1/810: write d4/d48/d22/f35 [8850152,117107] 0 2024-05-27T09:29:50.749 INFO:tasks.workunit.client.1.smithi138.stdout:9/826: symlink d6/da/d29/d8f/db7/l11d 0 2024-05-27T09:29:50.749 INFO:tasks.workunit.client.1.smithi138.stdout:8/982: dwrite d3/f148 [0,4194304] 0 2024-05-27T09:29:50.749 INFO:tasks.workunit.client.1.smithi138.stdout:8/983: chown d3/d8/d24/d55/d94/da5/d111/d117/d5c/fda 807 1 2024-05-27T09:29:50.749 INFO:tasks.workunit.client.1.smithi138.stdout:2/666: mknod d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/ce2 0 2024-05-27T09:29:50.749 INFO:tasks.workunit.client.1.smithi138.stdout:1/811: mknod d4/d1a/d2e/d4a/d7e/dc0/c10d 0 2024-05-27T09:29:50.749 INFO:tasks.workunit.client.1.smithi138.stdout:1/812: symlink d4/d38/l10e 0 2024-05-27T09:29:50.749 INFO:tasks.workunit.client.1.smithi138.stdout:2/667: symlink d1/d15/d2d/d17/dac/le3 0 2024-05-27T09:29:50.753 INFO:tasks.workunit.client.1.smithi138.stdout:1/813: creat d4/d48/d22/f10f x:0 0 0 2024-05-27T09:29:50.753 INFO:tasks.workunit.client.1.smithi138.stdout:1/814: stat d4/d48/d22 0 2024-05-27T09:29:50.753 INFO:tasks.workunit.client.1.smithi138.stdout:1/815: chown d4/dcc/ce4 164829 1 2024-05-27T09:29:50.759 INFO:tasks.workunit.client.1.smithi138.stdout:3/768: dwrite d0/d2/da/df/d10/d2a/d61/de1/fc7 [0,4194304] 0 2024-05-27T09:29:50.759 INFO:tasks.workunit.client.1.smithi138.stdout:4/842: dread d7/df/d35/d4c/d59/f118 [0,4194304] 0 2024-05-27T09:29:50.759 INFO:tasks.workunit.client.1.smithi138.stdout:4/843: fdatasync d7/d6e/fc2 0 2024-05-27T09:29:50.766 INFO:tasks.workunit.client.1.smithi138.stdout:4/844: creat d7/df/d35/d4c/dc0/dfb/f128 x:0 0 0 2024-05-27T09:29:50.771 INFO:tasks.workunit.client.1.smithi138.stdout:4/845: creat d7/dba/dcf/daa/dda/dfc/f129 x:0 0 0 2024-05-27T09:29:50.771 INFO:tasks.workunit.client.1.smithi138.stdout:4/846: readlink d7/dba/dcf/daa/def/lf1 0 2024-05-27T09:29:50.771 INFO:tasks.workunit.client.1.smithi138.stdout:4/847: chown d7/dba/dcf/daa/def/c10c 108 1 2024-05-27T09:29:50.773 INFO:tasks.workunit.client.1.smithi138.stdout:4/848: creat d7/f12a x:0 0 0 2024-05-27T09:29:50.778 INFO:tasks.workunit.client.1.smithi138.stdout:0/729: dwrite da/d2a/d8e/d1f/d42/fe3 [0,4194304] 0 2024-05-27T09:29:50.790 INFO:tasks.workunit.client.1.smithi138.stdout:4/849: dread d7/df/d35/d4c/d59/f118 [0,4194304] 0 2024-05-27T09:29:50.794 INFO:tasks.workunit.client.1.smithi138.stdout:4/850: symlink d7/df/l12b 0 2024-05-27T09:29:50.795 INFO:tasks.workunit.client.1.smithi138.stdout:4/851: creat d7/df/d35/d4c/dc0/dfb/f12c x:0 0 0 2024-05-27T09:29:50.795 INFO:tasks.workunit.client.1.smithi138.stdout:4/852: stat d7/df/d35/d4c/d59/d5d/d64/fb7 0 2024-05-27T09:29:50.795 INFO:tasks.workunit.client.1.smithi138.stdout:4/853: dread - d7/df/d5b/da1/dbc/fe3 zero size 2024-05-27T09:29:50.795 INFO:tasks.workunit.client.1.smithi138.stdout:4/854: chown d7/dba/dfa 1483823 1 2024-05-27T09:29:50.795 INFO:tasks.workunit.client.1.smithi138.stdout:4/855: dread - d7/dba/dcf/d3a/f11f zero size 2024-05-27T09:29:50.799 INFO:tasks.workunit.client.1.smithi138.stdout:2/668: dread d1/d15/d2d/d17/d25/f3a [0,4194304] 0 2024-05-27T09:29:50.799 INFO:tasks.workunit.client.1.smithi138.stdout:7/912: dwrite d0/d2b/f40 [4194304,4194304] 0 2024-05-27T09:29:50.800 INFO:tasks.workunit.client.1.smithi138.stdout:4/856: mknod d7/df/d35/d4c/dc0/dfb/c12d 0 2024-05-27T09:29:50.801 INFO:tasks.workunit.client.1.smithi138.stdout:7/913: getdents d0/d2b 0 2024-05-27T09:29:50.801 INFO:tasks.workunit.client.1.smithi138.stdout:7/914: chown d0/d2b/daf/d98/fa4 1423 1 2024-05-27T09:29:50.801 INFO:tasks.workunit.client.1.smithi138.stdout:7/915: chown d0/d2b/daf/d64/cf0 281 1 2024-05-27T09:29:50.804 INFO:tasks.workunit.client.1.smithi138.stdout:4/857: link d7/dba/dcf/daa/f10f d7/df/d35/d4c/d59/d5d/d64/df6/d10d/f12e 0 2024-05-27T09:29:50.810 INFO:tasks.workunit.client.1.smithi138.stdout:4/858: mkdir d7/df/d12f 0 2024-05-27T09:29:50.812 INFO:tasks.workunit.client.1.smithi138.stdout:4/859: creat d7/df/f130 x:0 0 0 2024-05-27T09:29:50.815 INFO:tasks.workunit.client.1.smithi138.stdout:4/860: rmdir d7/dba/dcf/daa/def 39 2024-05-27T09:29:50.818 INFO:tasks.workunit.client.1.smithi138.stdout:4/861: creat d7/df/d35/d4c/d59/d5d/d64/dc5/f131 x:0 0 0 2024-05-27T09:29:50.820 INFO:tasks.workunit.client.1.smithi138.stdout:4/862: mkdir d7/dba/dfa/d127/d132 0 2024-05-27T09:29:50.820 INFO:tasks.workunit.client.1.smithi138.stdout:4/863: chown d7/df/d5b/da1/c102 9099 1 2024-05-27T09:29:50.820 INFO:tasks.workunit.client.1.smithi138.stdout:4/864: creat d7/df/d35/f133 x:0 0 0 2024-05-27T09:29:50.821 INFO:tasks.workunit.client.1.smithi138.stdout:4/865: truncate d7/df/d35/d4c/d59/d5d/d64/d6a/feb 3343488 0 2024-05-27T09:29:50.823 INFO:tasks.workunit.client.1.smithi138.stdout:4/866: mkdir d7/df/d35/d4c/d59/d5d/d64/df6/d134 0 2024-05-27T09:29:50.826 INFO:tasks.workunit.client.1.smithi138.stdout:4/867: rename d7/df/d35/d4c/d59/f73 to d7/dba/f135 0 2024-05-27T09:29:50.826 INFO:tasks.workunit.client.1.smithi138.stdout:4/868: chown d7/df/d5b/da1/db1 11 1 2024-05-27T09:29:50.826 INFO:tasks.workunit.client.1.smithi138.stdout:4/869: chown d7/df/d35/d4c/ca3 164565 1 2024-05-27T09:29:50.827 INFO:tasks.workunit.client.1.smithi138.stdout:4/870: mkdir d7/dba/dfa/d136 0 2024-05-27T09:29:50.840 INFO:tasks.workunit.client.1.smithi138.stdout:9/827: dwrite d6/da/d29/ff2 [0,4194304] 0 2024-05-27T09:29:50.840 INFO:tasks.workunit.client.1.smithi138.stdout:9/828: truncate d6/db/fd4 758387 0 2024-05-27T09:29:50.842 INFO:tasks.workunit.client.1.smithi138.stdout:9/829: mknod d6/da/d29/d8f/db7/dbd/d4c/c11e 0 2024-05-27T09:29:50.855 INFO:tasks.workunit.client.1.smithi138.stdout:8/984: dwrite d3/d8/d24/d4d/fce [0,4194304] 0 2024-05-27T09:29:50.856 INFO:tasks.workunit.client.1.smithi138.stdout:8/985: write d3/d8/d32/d3f/d9c/d9f/d86/f146 [1784670,109632] 0 2024-05-27T09:29:50.861 INFO:tasks.workunit.client.1.smithi138.stdout:8/986: mknod d3/d8/d11/df1/c14b 0 2024-05-27T09:29:50.861 INFO:tasks.workunit.client.1.smithi138.stdout:8/987: fsync d3/d8/d24/d4d/fce 0 2024-05-27T09:29:50.861 INFO:tasks.workunit.client.1.smithi138.stdout:8/988: truncate d3/d8/d24/d55/d7e/f89 6257690 0 2024-05-27T09:29:50.863 INFO:tasks.workunit.client.1.smithi138.stdout:8/989: mkdir d3/d8/d72/d14c 0 2024-05-27T09:29:50.863 INFO:tasks.workunit.client.1.smithi138.stdout:8/990: truncate d3/d8/d24/d55/d94/f141 585497 0 2024-05-27T09:29:50.863 INFO:tasks.workunit.client.1.smithi138.stdout:8/991: chown d3/d8/d72/d109/f12f 24233487 1 2024-05-27T09:29:50.863 INFO:tasks.workunit.client.1.smithi138.stdout:8/992: readlink d3/d8/d24/d55/d94/da5/d111/d117/d5c/l6e 0 2024-05-27T09:29:50.863 INFO:tasks.workunit.client.1.smithi138.stdout:8/993: chown d3/d8/d72/fde 9937280 1 2024-05-27T09:29:50.863 INFO:tasks.workunit.client.1.smithi138.stdout:8/994: write d3/d8/d32/d3f/d9c/d9f/d86/f135 [967266,65561] 0 2024-05-27T09:29:50.870 INFO:tasks.workunit.client.1.smithi138.stdout:8/995: rename d3/d8/d32/dd6/f10d to d3/d8/d11/df1/f14d 0 2024-05-27T09:29:50.874 INFO:tasks.workunit.client.1.smithi138.stdout:8/996: symlink d3/d8/d24/d55/l14e 0 2024-05-27T09:29:50.874 INFO:tasks.workunit.client.1.smithi138.stdout:8/997: readlink d3/d8/d64/la8 0 2024-05-27T09:29:50.875 INFO:tasks.workunit.client.1.smithi138.stdout:1/816: dwrite d4/d48/d39/d5f/d93/fc9 [0,4194304] 0 2024-05-27T09:29:50.875 INFO:tasks.workunit.client.1.smithi138.stdout:3/769: dwrite d0/fa4 [4194304,4194304] 0 2024-05-27T09:29:50.875 INFO:tasks.workunit.client.1.smithi138.stdout:3/770: dread - d0/d2/da/df/d10/d2a/db2/fed zero size 2024-05-27T09:29:50.876 INFO:tasks.workunit.client.1.smithi138.stdout:7/916: dread d0/d23/f50 [4194304,4194304] 0 2024-05-27T09:29:50.876 INFO:tasks.workunit.client.1.smithi138.stdout:3/771: symlink d0/d3f/d50/d9b/d9f/da3/l103 0 2024-05-27T09:29:50.877 INFO:tasks.workunit.client.1.smithi138.stdout:1/817: getdents d4/d48/d22/d65 0 2024-05-27T09:29:50.877 INFO:tasks.workunit.client.1.smithi138.stdout:1/818: dread - d4/d1a/d2e/d4a/d7e/ff1 zero size 2024-05-27T09:29:50.877 INFO:tasks.workunit.client.1.smithi138.stdout:3/772: mknod d0/d2/da/df/d10/d2a/db2/c104 0 2024-05-27T09:29:50.877 INFO:tasks.workunit.client.1.smithi138.stdout:7/917: getdents d0/d23/d3c/d84 0 2024-05-27T09:29:50.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:50 smithi138 ceph-mon[34885]: Adjusting osd_memory_target on smithi138 to 2240M 2024-05-27T09:29:50.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:50 smithi138 ceph-mon[34885]: Updating smithi119:/etc/ceph/ceph.conf 2024-05-27T09:29:50.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:50 smithi138 ceph-mon[34885]: Updating smithi138:/etc/ceph/ceph.conf 2024-05-27T09:29:50.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:50 smithi138 ceph-mon[34885]: Updating smithi119:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:29:50.891 INFO:tasks.workunit.client.1.smithi138.stdout:0/730: dwrite da/d2a/d8e/d1f/d60/fc6 [0,4194304] 0 2024-05-27T09:29:50.896 INFO:tasks.workunit.client.1.smithi138.stdout:9/830: dwrite d6/db/de/d18/d24/f5c [4194304,4194304] 0 2024-05-27T09:29:50.899 INFO:tasks.workunit.client.1.smithi138.stdout:4/871: dwrite d7/d6e/df8/f107 [0,4194304] 0 2024-05-27T09:29:50.899 INFO:tasks.workunit.client.1.smithi138.stdout:4/872: readlink d7/df/d35/d4c/d59/d5d/d64/dc5/lcc 0 2024-05-27T09:29:50.907 INFO:tasks.workunit.client.1.smithi138.stdout:0/731: unlink da/d2a/d8e/d43/fd6 0 2024-05-27T09:29:50.915 INFO:tasks.workunit.client.1.smithi138.stdout:8/998: dwrite d3/f137 [0,4194304] 0 2024-05-27T09:29:50.918 INFO:tasks.workunit.client.1.smithi138.stdout:8/999: mknod d3/d8/d24/c14f 0 2024-05-27T09:29:50.922 INFO:tasks.workunit.client.1.smithi138.stdout:4/873: rename d7/df/d35/c99 to d7/dba/dfa/d127/c137 0 2024-05-27T09:29:50.926 INFO:tasks.workunit.client.1.smithi138.stdout:0/732: symlink da/d2a/d2b/leb 0 2024-05-27T09:29:50.928 INFO:tasks.workunit.client.1.smithi138.stdout:4/874: mkdir d7/dba/dcf/daa/dda/d138 0 2024-05-27T09:29:50.928 INFO:tasks.workunit.client.1.smithi138.stdout:4/875: truncate d7/d6e/faf 1422716 0 2024-05-27T09:29:50.930 INFO:tasks.workunit.client.1.smithi138.stdout:9/831: dread d6/db/de/d18/ddb/fe4 [0,4194304] 0 2024-05-27T09:29:50.933 INFO:tasks.workunit.client.1.smithi138.stdout:9/832: mkdir d6/d11f 0 2024-05-27T09:29:50.937 INFO:tasks.workunit.client.1.smithi138.stdout:1/819: dwrite d4/d1a/d2e/d4a/f80 [4194304,4194304] 0 2024-05-27T09:29:50.947 INFO:tasks.workunit.client.1.smithi138.stdout:4/876: dread d7/df/d35/d4c/d59/d5d/d64/d6a/f6d [0,4194304] 0 2024-05-27T09:29:50.954 INFO:tasks.workunit.client.1.smithi138.stdout:0/733: dread da/d2a/d2b/f4d [0,4194304] 0 2024-05-27T09:29:50.954 INFO:tasks.workunit.client.1.smithi138.stdout:7/918: dwrite d0/d23/d3c/fb9 [0,4194304] 0 2024-05-27T09:29:50.955 INFO:tasks.workunit.client.1.smithi138.stdout:1/820: dread d4/d3c/fc8 [0,4194304] 0 2024-05-27T09:29:50.955 INFO:tasks.workunit.client.1.smithi138.stdout:1/821: truncate d4/d38/d97/ff8 999768 0 2024-05-27T09:29:50.955 INFO:tasks.workunit.client.1.smithi138.stdout:1/822: write d4/d48/d22/d65/f10a [1094583,42116] 0 2024-05-27T09:29:50.955 INFO:tasks.workunit.client.1.smithi138.stdout:7/919: creat d0/d2b/daf/dba/f126 x:0 0 0 2024-05-27T09:29:50.958 INFO:tasks.workunit.client.1.smithi138.stdout:7/920: getdents d0 0 2024-05-27T09:29:50.958 INFO:tasks.workunit.client.1.smithi138.stdout:7/921: write d0/d23/d3c/fca [2349759,126159] 0 2024-05-27T09:29:50.959 INFO:tasks.workunit.client.1.smithi138.stdout:7/922: mkdir d0/d2b/daf/d60/d127 0 2024-05-27T09:29:50.960 INFO:tasks.workunit.client.1.smithi138.stdout:4/877: rmdir d7/df/d35/d4c/d59/d5d 39 2024-05-27T09:29:50.962 INFO:tasks.workunit.client.1.smithi138.stdout:7/923: symlink d0/d23/d3c/l128 0 2024-05-27T09:29:50.962 INFO:tasks.workunit.client.1.smithi138.stdout:9/833: read d6/da/fa9 [1729875,5107] 0 2024-05-27T09:29:50.963 INFO:tasks.workunit.client.1.smithi138.stdout:9/834: creat d6/db/dc/f120 x:0 0 0 2024-05-27T09:29:50.963 INFO:tasks.workunit.client.1.smithi138.stdout:9/835: write d6/fef [970979,36799] 0 2024-05-27T09:29:50.967 INFO:tasks.workunit.client.1.smithi138.stdout:9/836: write d6/db/de/d18/ddb/fe4 [263164,54147] 0 2024-05-27T09:29:50.972 INFO:tasks.workunit.client.1.smithi138.stdout:0/734: getdents da/d2a/d8e/d6a/daf 0 2024-05-27T09:29:50.972 INFO:tasks.workunit.client.1.smithi138.stdout:0/735: creat da/d2a/d8e/d43/fec x:0 0 0 2024-05-27T09:29:50.974 INFO:tasks.workunit.client.1.smithi138.stdout:1/823: rename d4/d88/cfd to d4/d3c/c110 0 2024-05-27T09:29:50.974 INFO:tasks.workunit.client.1.smithi138.stdout:1/824: stat d4/d1a/fa9 0 2024-05-27T09:29:50.981 INFO:tasks.workunit.client.1.smithi138.stdout:0/736: unlink da/d2a/d8e/d1f/c41 0 2024-05-27T09:29:50.984 INFO:tasks.workunit.client.1.smithi138.stdout:7/924: dread d0/d2b/daf/fe [0,4194304] 0 2024-05-27T09:29:50.984 INFO:tasks.workunit.client.1.smithi138.stdout:4/878: read d7/df/f29 [294918,105297] 0 2024-05-27T09:29:50.984 INFO:tasks.workunit.client.1.smithi138.stdout:4/879: creat d7/dba/dcf/d3a/f139 x:0 0 0 2024-05-27T09:29:50.987 INFO:tasks.workunit.client.1.smithi138.stdout:7/925: truncate d0/d2b/daf/d43/fa9 798508 0 2024-05-27T09:29:50.987 INFO:tasks.workunit.client.1.smithi138.stdout:0/737: mknod da/d2a/d8e/d1f/d42/d63/dea/ced 0 2024-05-27T09:29:50.987 INFO:tasks.workunit.client.1.smithi138.stdout:0/738: truncate da/d2a/d8e/d1f/d71/f79 1679191 0 2024-05-27T09:29:50.991 INFO:tasks.workunit.client.1.smithi138.stdout:7/926: creat d0/d23/f129 x:0 0 0 2024-05-27T09:29:50.993 INFO:tasks.workunit.client.1.smithi138.stdout:7/927: creat d0/d2b/daf/d60/d127/f12a x:0 0 0 2024-05-27T09:29:50.997 INFO:tasks.workunit.client.1.smithi138.stdout:7/928: creat d0/d2b/daf/d98/f12b x:0 0 0 2024-05-27T09:29:50.997 INFO:tasks.workunit.client.1.smithi138.stdout:7/929: chown d0/d2b/daf/d43/c51 99305 1 2024-05-27T09:29:50.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:50 smithi119 ceph-mon[29082]: Adjusting osd_memory_target on smithi138 to 2240M 2024-05-27T09:29:50.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:50 smithi119 ceph-mon[29082]: Updating smithi119:/etc/ceph/ceph.conf 2024-05-27T09:29:50.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:50 smithi119 ceph-mon[29082]: Updating smithi138:/etc/ceph/ceph.conf 2024-05-27T09:29:50.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:50 smithi119 ceph-mon[29082]: Updating smithi119:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:29:51.002 INFO:tasks.workunit.client.1.smithi138.stdout:7/930: symlink d0/d2b/daf/d60/d127/l12c 0 2024-05-27T09:29:51.024 INFO:tasks.workunit.client.1.smithi138.stdout:4/880: dwrite f3 [8388608,4194304] 0 2024-05-27T09:29:51.025 INFO:tasks.workunit.client.1.smithi138.stdout:4/881: chown d7/df/d35/d4c/f116 733685696 1 2024-05-27T09:29:51.025 INFO:tasks.workunit.client.1.smithi138.stdout:4/882: chown d7/df/d35/d4c/d59/lb6 38903 1 2024-05-27T09:29:51.025 INFO:tasks.workunit.client.1.smithi138.stdout:7/931: dread d0/f1a [4194304,4194304] 0 2024-05-27T09:29:51.025 INFO:tasks.workunit.client.1.smithi138.stdout:7/932: write d0/d23/dc4/ddf/d91/fb4 [90353,93451] 0 2024-05-27T09:29:51.027 INFO:tasks.workunit.client.1.smithi138.stdout:5/840: sync 2024-05-27T09:29:51.027 INFO:tasks.workunit.client.1.smithi138.stdout:7/933: creat d0/d23/dc4/ddf/d91/d108/f12d x:0 0 0 2024-05-27T09:29:51.027 INFO:tasks.workunit.client.1.smithi138.stdout:5/841: write d3/d12/d16/d1d/d23/d68/fa1 [5005171,19863] 0 2024-05-27T09:29:51.029 INFO:tasks.workunit.client.1.smithi138.stdout:7/934: rename d0/d23/d3c/d84/fd1 to d0/d23/d3c/d3f/f12e 0 2024-05-27T09:29:51.031 INFO:tasks.workunit.client.1.smithi138.stdout:7/935: mkdir d0/d2b/daf/d60/d9d/d12f 0 2024-05-27T09:29:51.031 INFO:tasks.workunit.client.1.smithi138.stdout:7/936: dread - d0/d23/d49/f107 zero size 2024-05-27T09:29:51.037 INFO:tasks.workunit.client.1.smithi138.stdout:4/883: dread d7/dba/dcf/f34 [0,4194304] 0 2024-05-27T09:29:51.037 INFO:tasks.workunit.client.1.smithi138.stdout:4/884: creat d7/df/d5b/da1/dbc/f13a x:0 0 0 2024-05-27T09:29:51.039 INFO:tasks.workunit.client.1.smithi138.stdout:4/885: creat d7/dba/dcf/daa/dda/d138/f13b x:0 0 0 2024-05-27T09:29:51.039 INFO:tasks.workunit.client.1.smithi138.stdout:4/886: chown d7/f44 1 1 2024-05-27T09:29:51.039 INFO:tasks.workunit.client.1.smithi138.stdout:4/887: chown d7/f78 34640274 1 2024-05-27T09:29:51.039 INFO:tasks.workunit.client.1.smithi138.stdout:5/842: dread d3/d12/d32/d60/f82 [0,4194304] 0 2024-05-27T09:29:51.039 INFO:tasks.workunit.client.1.smithi138.stdout:5/843: creat d3/d12/d16/d1d/d22/db9/f117 x:0 0 0 2024-05-27T09:29:51.044 INFO:tasks.workunit.client.1.smithi138.stdout:4/888: creat d7/df/d35/d4c/d59/d5d/d64/f13c x:0 0 0 2024-05-27T09:29:51.050 INFO:tasks.workunit.client.1.smithi138.stdout:1/825: dread d4/d1a/d2e/d32/d8c/dc6/f55 [0,4194304] 0 2024-05-27T09:29:51.050 INFO:tasks.workunit.client.1.smithi138.stdout:9/837: dwrite d6/da/d29/fe3 [4194304,4194304] 0 2024-05-27T09:29:51.050 INFO:tasks.workunit.client.1.smithi138.stdout:1/826: write d4/d48/d39/d5f/d63/fa3 [290274,58167] 0 2024-05-27T09:29:51.050 INFO:tasks.workunit.client.1.smithi138.stdout:1/827: write d4/d1a/d2e/d32/d8c/dc6/fbb [799744,85922] 0 2024-05-27T09:29:51.054 INFO:tasks.workunit.client.1.smithi138.stdout:9/838: rename d6/da/fe9 to d6/db/de/d18/d24/d72/f121 0 2024-05-27T09:29:51.062 INFO:tasks.workunit.client.1.smithi138.stdout:5/844: read d3/d12/d26/d53/f58 [164575,28406] 0 2024-05-27T09:29:51.066 INFO:tasks.workunit.client.1.smithi138.stdout:4/889: dread d7/df/f4d [0,4194304] 0 2024-05-27T09:29:51.069 INFO:tasks.workunit.client.1.smithi138.stdout:5/845: creat d3/d12/d32/d60/de6/d111/f118 x:0 0 0 2024-05-27T09:29:51.073 INFO:tasks.workunit.client.1.smithi138.stdout:5/846: mknod d3/d12/d16/d1d/d22/d2b/c119 0 2024-05-27T09:29:51.074 INFO:tasks.workunit.client.1.smithi138.stdout:5/847: creat d3/d12/d16/d1d/d22/db9/f11a x:0 0 0 2024-05-27T09:29:51.092 INFO:tasks.workunit.client.1.smithi138.stdout:5/848: rename d3/d12/d32/d87/d89/d10f to d3/d12/d32/d87/d11b 0 2024-05-27T09:29:51.120 INFO:tasks.workunit.client.1.smithi138.stdout:7/937: dwrite d0/d2b/daf/d98/fc9 [0,4194304] 0 2024-05-27T09:29:51.120 INFO:tasks.workunit.client.1.smithi138.stdout:5/849: dread d3/d12/d32/d60/fa3 [0,4194304] 0 2024-05-27T09:29:51.121 INFO:tasks.workunit.client.1.smithi138.stdout:5/850: mknod d3/c11c 0 2024-05-27T09:29:51.121 INFO:tasks.workunit.client.1.smithi138.stdout:5/851: write d3/d12/d32/d60/f82 [871794,62099] 0 2024-05-27T09:29:51.122 INFO:tasks.workunit.client.1.smithi138.stdout:7/938: mknod d0/d23/da8/d8e/c130 0 2024-05-27T09:29:51.122 INFO:tasks.workunit.client.1.smithi138.stdout:7/939: write d0/d23/d3c/d84/d9c/dc3/f122 [665326,83696] 0 2024-05-27T09:29:51.123 INFO:tasks.workunit.client.1.smithi138.stdout:9/839: dwrite d6/db/de/d18/d24/f65 [0,4194304] 0 2024-05-27T09:29:51.123 INFO:tasks.workunit.client.1.smithi138.stdout:9/840: write d6/da/d29/f6d [5673809,92923] 0 2024-05-27T09:29:51.125 INFO:tasks.workunit.client.1.smithi138.stdout:5/852: symlink d3/d12/d16/d1d/d22/l11d 0 2024-05-27T09:29:51.125 INFO:tasks.workunit.client.1.smithi138.stdout:7/940: creat d0/d23/dc4/ddf/f131 x:0 0 0 2024-05-27T09:29:51.129 INFO:tasks.workunit.client.1.smithi138.stdout:7/941: write d0/d23/d3a/f55 [588624,34154] 0 2024-05-27T09:29:51.129 INFO:tasks.workunit.client.1.smithi138.stdout:7/942: fsync d0/d2b/daf/f54 0 2024-05-27T09:29:51.129 INFO:tasks.workunit.client.1.smithi138.stdout:7/943: write d0/d23/dc4/ddf/d91/fb4 [330621,118912] 0 2024-05-27T09:29:51.133 INFO:tasks.workunit.client.1.smithi138.stdout:5/853: rename d3/dab/dc2 to d3/d12/dcf/d11e 0 2024-05-27T09:29:51.133 INFO:tasks.workunit.client.1.smithi138.stdout:1/828: dwrite d4/d48/d22/d65/f10a [0,4194304] 0 2024-05-27T09:29:51.133 INFO:tasks.workunit.client.1.smithi138.stdout:1/829: dread - d4/d1a/d2e/d4a/d7e/ff1 zero size 2024-05-27T09:29:51.149 INFO:tasks.workunit.client.1.smithi138.stdout:1/830: symlink d4/d1a/d2e/d4a/d7e/l111 0 2024-05-27T09:29:51.157 INFO:tasks.workunit.client.1.smithi138.stdout:1/831: link d4/d48/d39/d5f/d93/faf d4/d38/f112 0 2024-05-27T09:29:51.161 INFO:tasks.workunit.client.1.smithi138.stdout:1/832: link d4/d1a/d2e/d4a/c94 d4/d48/d39/c113 0 2024-05-27T09:29:51.166 INFO:tasks.workunit.client.1.smithi138.stdout:7/944: dread d0/f10 [0,4194304] 0 2024-05-27T09:29:51.166 INFO:tasks.workunit.client.1.smithi138.stdout:7/945: truncate d0/d2b/fe1 973975 0 2024-05-27T09:29:51.166 INFO:tasks.workunit.client.1.smithi138.stdout:7/946: truncate d0/f24 4807629 0 2024-05-27T09:29:51.166 INFO:tasks.workunit.client.1.smithi138.stdout:7/947: readlink d0/d2b/daf/d43/l121 0 2024-05-27T09:29:51.166 INFO:tasks.workunit.client.1.smithi138.stdout:7/948: stat d0/f45 0 2024-05-27T09:29:51.166 INFO:tasks.workunit.client.1.smithi138.stdout:1/833: stat d4/d48/l17 0 2024-05-27T09:29:51.185 INFO:tasks.workunit.client.1.smithi138.stdout:4/890: dwrite d7/df/d5b/f7c [0,4194304] 0 2024-05-27T09:29:51.185 INFO:tasks.workunit.client.1.smithi138.stdout:1/834: dread d4/d38/f44 [0,4194304] 0 2024-05-27T09:29:51.187 INFO:tasks.workunit.client.1.smithi138.stdout:5/854: dwrite d3/d12/dcf/d11e/feb [0,4194304] 0 2024-05-27T09:29:51.189 INFO:tasks.workunit.client.1.smithi138.stdout:9/841: dwrite d6/da/d29/d8f/db7/dbd/d10e/d28/d102/f11c [0,4194304] 0 2024-05-27T09:29:51.189 INFO:tasks.workunit.client.1.smithi138.stdout:9/842: fsync d6/db/de/d18/f64 0 2024-05-27T09:29:51.189 INFO:tasks.workunit.client.1.smithi138.stdout:9/843: fsync d6/da/f5d 0 2024-05-27T09:29:51.191 INFO:tasks.workunit.client.1.smithi138.stdout:1/835: symlink d4/d3c/l114 0 2024-05-27T09:29:51.192 INFO:tasks.workunit.client.1.smithi138.stdout:1/836: fdatasync d4/d48/d22/d65/f10a 0 2024-05-27T09:29:51.196 INFO:tasks.workunit.client.1.smithi138.stdout:4/891: link d7/df/d35/d4c/d59/d5d/d64/d6a/la8 d7/df/d35/d4c/l13d 0 2024-05-27T09:29:51.197 INFO:tasks.workunit.client.1.smithi138.stdout:9/844: dread d6/da/f9e [0,4194304] 0 2024-05-27T09:29:51.199 INFO:tasks.workunit.client.1.smithi138.stdout:5/855: truncate d3/d12/d16/d1d/f97 1097446 0 2024-05-27T09:29:51.200 INFO:tasks.workunit.client.1.smithi138.stdout:9/845: creat d6/da/d29/d59/dee/f122 x:0 0 0 2024-05-27T09:29:51.200 INFO:tasks.workunit.client.1.smithi138.stdout:9/846: chown d6/db/dc 1247 1 2024-05-27T09:29:51.200 INFO:tasks.workunit.client.1.smithi138.stdout:9/847: readlink d6/db/dc/dca/l109 0 2024-05-27T09:29:51.200 INFO:tasks.workunit.client.1.smithi138.stdout:9/848: chown d6/da/d29/d59/dee/d55/f3a 2 1 2024-05-27T09:29:51.203 INFO:tasks.workunit.client.1.smithi138.stdout:9/849: write d6/da/f9e [1294970,23421] 0 2024-05-27T09:29:51.203 INFO:tasks.workunit.client.1.smithi138.stdout:1/837: symlink d4/d48/d22/d65/l115 0 2024-05-27T09:29:51.208 INFO:tasks.workunit.client.1.smithi138.stdout:9/850: dread d6/f88 [0,4194304] 0 2024-05-27T09:29:51.210 INFO:tasks.workunit.client.1.smithi138.stdout:9/851: truncate d6/da/d2f/f8b 1705277 0 2024-05-27T09:29:51.210 INFO:tasks.workunit.client.1.smithi138.stdout:9/852: fdatasync d6/da/d29/d59/dee/d55/f6c 0 2024-05-27T09:29:51.218 INFO:tasks.workunit.client.1.smithi138.stdout:9/853: dread d6/da/d29/d61/f11a [0,4194304] 0 2024-05-27T09:29:51.218 INFO:tasks.workunit.client.1.smithi138.stdout:9/854: fdatasync d6/da/d29/d59/dee/d55/f3a 0 2024-05-27T09:29:51.219 INFO:tasks.workunit.client.1.smithi138.stdout:9/855: creat d6/da/d29/d59/dee/f123 x:0 0 0 2024-05-27T09:29:51.222 INFO:tasks.workunit.client.1.smithi138.stdout:7/949: dwrite d0/d2b/daf/f54 [0,4194304] 0 2024-05-27T09:29:51.222 INFO:tasks.workunit.client.1.smithi138.stdout:7/950: write d0/d23/f129 [481024,102640] 0 2024-05-27T09:29:51.222 INFO:tasks.workunit.client.1.smithi138.stdout:7/951: fsync d0/d23/f58 0 2024-05-27T09:29:51.223 INFO:tasks.workunit.client.1.smithi138.stdout:5/856: unlink d3/d12/d16/d1d/d23/f67 0 2024-05-27T09:29:51.224 INFO:tasks.workunit.client.1.smithi138.stdout:9/856: fsync d6/da/d2f/f8b 0 2024-05-27T09:29:51.224 INFO:tasks.workunit.client.1.smithi138.stdout:9/857: write d6/da/d29/d59/dee/f123 [566545,115232] 0 2024-05-27T09:29:51.225 INFO:tasks.workunit.client.1.smithi138.stdout:7/952: rmdir d0/d2b 39 2024-05-27T09:29:51.228 INFO:tasks.workunit.client.1.smithi138.stdout:9/858: creat d6/da/d29/d8f/db7/dbd/d10e/d28/f124 x:0 0 0 2024-05-27T09:29:51.228 INFO:tasks.workunit.client.1.smithi138.stdout:9/859: write d6/db/de/f31 [2177812,1577] 0 2024-05-27T09:29:51.231 INFO:tasks.workunit.client.1.smithi138.stdout:4/892: unlink d7/df/d35/d4c/d59/d5d/d64/d6a/l8d 0 2024-05-27T09:29:51.232 INFO:tasks.workunit.client.1.smithi138.stdout:7/953: mkdir d0/d2b/daf/d98/db6/d132 0 2024-05-27T09:29:51.232 INFO:tasks.workunit.client.1.smithi138.stdout:7/954: read - d0/d2b/daf/dbf/f118 zero size 2024-05-27T09:29:51.233 INFO:tasks.workunit.client.1.smithi138.stdout:9/860: link d6/db/dc/f120 d6/da/d29/d59/dee/f125 0 2024-05-27T09:29:51.234 INFO:tasks.workunit.client.1.smithi138.stdout:9/861: write d6/da/d29/d59/dee/fb1 [391161,96646] 0 2024-05-27T09:29:51.234 INFO:tasks.workunit.client.1.smithi138.stdout:7/955: symlink d0/d23/dc4/ddf/d91/l133 0 2024-05-27T09:29:51.234 INFO:tasks.workunit.client.1.smithi138.stdout:7/956: truncate d0/d23/d3c/d84/d9c/f11b 815473 0 2024-05-27T09:29:51.234 INFO:tasks.workunit.client.1.smithi138.stdout:7/957: fdatasync d0/d23/f4f 0 2024-05-27T09:29:51.235 INFO:tasks.workunit.client.1.smithi138.stdout:5/857: mknod d3/d12/dcf/d11e/dc6/c11f 0 2024-05-27T09:29:51.235 INFO:tasks.workunit.client.1.smithi138.stdout:5/858: write d3/d12/d16/d1d/d23/d68/fdd [480875,29284] 0 2024-05-27T09:29:51.235 INFO:tasks.workunit.client.1.smithi138.stdout:5/859: fdatasync d3/f1b 0 2024-05-27T09:29:51.236 INFO:tasks.workunit.client.1.smithi138.stdout:5/860: fsync d3/d12/d16/d1d/d22/d2b/d9e/db6/f10a 0 2024-05-27T09:29:51.236 INFO:tasks.workunit.client.1.smithi138.stdout:5/861: truncate d3/d12/f115 822359 0 2024-05-27T09:29:51.236 INFO:tasks.workunit.client.1.smithi138.stdout:9/862: creat d6/db/de/d6b/d104/d10a/f126 x:0 0 0 2024-05-27T09:29:51.238 INFO:tasks.workunit.client.1.smithi138.stdout:7/958: creat d0/df7/f134 x:0 0 0 2024-05-27T09:29:51.238 INFO:tasks.workunit.client.1.smithi138.stdout:7/959: chown d0/d23/d3a/f46 3 1 2024-05-27T09:29:51.238 INFO:tasks.workunit.client.1.smithi138.stdout:4/893: unlink d7/dba/dcf/d3a/c46 0 2024-05-27T09:29:51.238 INFO:tasks.workunit.client.1.smithi138.stdout:2/669: sync 2024-05-27T09:29:51.238 INFO:tasks.workunit.client.1.smithi138.stdout:9/863: creat d6/da/d2f/f127 x:0 0 0 2024-05-27T09:29:51.239 INFO:tasks.workunit.client.1.smithi138.stdout:7/960: mkdir d0/d2b/daf/d98/db6/d135 0 2024-05-27T09:29:51.240 INFO:tasks.workunit.client.1.smithi138.stdout:5/862: rmdir d3/d12/d32/d7e/df3 0 2024-05-27T09:29:51.240 INFO:tasks.workunit.client.1.smithi138.stdout:2/670: creat d1/d15/d2d/d17/d25/d4d/d6d/fe4 x:0 0 0 2024-05-27T09:29:51.240 INFO:tasks.workunit.client.1.smithi138.stdout:5/863: write d3/d12/d32/d60/de6/df2/f10e [211052,84567] 0 2024-05-27T09:29:51.242 INFO:tasks.workunit.client.1.smithi138.stdout:3/773: sync 2024-05-27T09:29:51.242 INFO:tasks.workunit.client.1.smithi138.stdout:9/864: rename d6/da/d29/d8f/db7/dbd/d46 to d6/da/d29/d8f/db7/dbd/d10e/d28/d102/d128 0 2024-05-27T09:29:51.242 INFO:tasks.workunit.client.1.smithi138.stdout:3/774: dread - d0/d3f/d50/fa6 zero size 2024-05-27T09:29:51.244 INFO:tasks.workunit.client.1.smithi138.stdout:9/865: link d6/db/d8d/fdc d6/db/de/d6b/d104/d10a/f129 0 2024-05-27T09:29:51.244 INFO:tasks.workunit.client.1.smithi138.stdout:9/866: fsync d6/fef 0 2024-05-27T09:29:51.246 INFO:tasks.workunit.client.1.smithi138.stdout:3/775: truncate d0/d2/da/df/d10/d2a/d48/f4d 2451873 0 2024-05-27T09:29:51.246 INFO:tasks.workunit.client.1.smithi138.stdout:3/776: dread - d0/d2/da/df/d10/d2a/db2/fed zero size 2024-05-27T09:29:51.246 INFO:tasks.workunit.client.1.smithi138.stdout:3/777: truncate d0/d2/da/df/d10/d2a/d48/d90/fde 857006 0 2024-05-27T09:29:51.246 INFO:tasks.workunit.client.1.smithi138.stdout:3/778: chown d0/d2/da/df/d10/f4a 19966 1 2024-05-27T09:29:51.246 INFO:tasks.workunit.client.1.smithi138.stdout:3/779: chown d0/d3f/d60/d93/de5/fd3 773572537 1 2024-05-27T09:29:51.246 INFO:tasks.workunit.client.1.smithi138.stdout:3/780: truncate d0/d2/da/df/d10/d2a/d61/de1/f101 64011 0 2024-05-27T09:29:51.250 INFO:tasks.workunit.client.1.smithi138.stdout:4/894: chown d7/lad 24975 1 2024-05-27T09:29:51.250 INFO:tasks.workunit.client.1.smithi138.stdout:4/895: dread - d7/df/d5b/da1/dbc/fe3 zero size 2024-05-27T09:29:51.250 INFO:tasks.workunit.client.1.smithi138.stdout:2/671: rmdir d1/d15/d2d/d2e 39 2024-05-27T09:29:51.251 INFO:tasks.workunit.client.1.smithi138.stdout:2/672: symlink d1/d10/d52/le5 0 2024-05-27T09:29:51.257 INFO:tasks.workunit.client.1.smithi138.stdout:5/864: unlink d3/dab/f116 0 2024-05-27T09:29:51.257 INFO:tasks.workunit.client.1.smithi138.stdout:4/896: dread d7/df/d35/d4c/d59/d5d/d64/df6/d10d/f117 [0,4194304] 0 2024-05-27T09:29:51.260 INFO:tasks.workunit.client.1.smithi138.stdout:4/897: truncate d7/dba/dcf/daa/f10f 254901 0 2024-05-27T09:29:51.260 INFO:tasks.workunit.client.1.smithi138.stdout:4/898: chown d7/f12a 7 1 2024-05-27T09:29:51.261 INFO:tasks.workunit.client.1.smithi138.stdout:7/961: dread d0/d2b/daf/f12 [0,4194304] 0 2024-05-27T09:29:51.262 INFO:tasks.workunit.client.1.smithi138.stdout:7/962: stat d0/d2b/daf/d60/d9d/cb0 0 2024-05-27T09:29:51.265 INFO:tasks.workunit.client.1.smithi138.stdout:9/867: rename d6/da/d29/d8f/db7/dbd/d10e/d28 to d6/db/de/d6b/d104/d12a 0 2024-05-27T09:29:51.269 INFO:tasks.workunit.client.1.smithi138.stdout:2/673: rename d1/d15/d2d/d17/d25/d9d/d4b/c79 to d1/d15/d2d/d17/ce6 0 2024-05-27T09:29:51.269 INFO:tasks.workunit.client.1.smithi138.stdout:2/674: dread - d1/d10/d52/d5d/f91 zero size 2024-05-27T09:29:51.270 INFO:tasks.workunit.client.1.smithi138.stdout:7/963: getdents d0/d23 0 2024-05-27T09:29:51.271 INFO:tasks.workunit.client.1.smithi138.stdout:9/868: mknod d6/db/de/d18/d24/c12b 0 2024-05-27T09:29:51.271 INFO:tasks.workunit.client.1.smithi138.stdout:9/869: dread - d6/db/de/d18/d24/d72/f113 zero size 2024-05-27T09:29:51.273 INFO:tasks.workunit.client.1.smithi138.stdout:5/865: dread d3/d12/d32/d6b/fd2 [0,4194304] 0 2024-05-27T09:29:51.276 INFO:tasks.workunit.client.1.smithi138.stdout:9/870: creat d6/da/d29/d8f/db7/dbd/da2/f12c x:0 0 0 2024-05-27T09:29:51.276 INFO:tasks.workunit.client.1.smithi138.stdout:9/871: chown c5 53911 1 2024-05-27T09:29:51.277 INFO:tasks.workunit.client.1.smithi138.stdout:5/866: creat d3/d12/d16/d1d/d23/d3b/dec/d100/f120 x:0 0 0 2024-05-27T09:29:51.280 INFO:tasks.workunit.client.1.smithi138.stdout:5/867: rename d3/l34 to d3/d12/d32/d87/d89/dc3/l121 0 2024-05-27T09:29:51.286 INFO:tasks.workunit.client.1.smithi138.stdout:5/868: rename d3/d12/d16/d1d/d22/d2b/c119 to d3/ddc/c122 0 2024-05-27T09:29:51.289 INFO:tasks.workunit.client.1.smithi138.stdout:5/869: creat d3/d12/d32/d87/d11b/f123 x:0 0 0 2024-05-27T09:29:51.295 INFO:tasks.workunit.client.1.smithi138.stdout:5/870: getdents d3/d12/d16/d1d/d30 0 2024-05-27T09:29:51.304 INFO:tasks.workunit.client.1.smithi138.stdout:5/871: link d3/c14 d3/d12/d16/d1d/d30/c124 0 2024-05-27T09:29:51.314 INFO:tasks.workunit.client.1.smithi138.stdout:5/872: rename d3/d12/d26/d53/c54 to d3/d12/d16/d1d/d23/d68/c125 0 2024-05-27T09:29:51.342 INFO:tasks.workunit.client.1.smithi138.stdout:9/872: dwrite d6/da/d29/d59/dee/d55/fc6 [0,4194304] 0 2024-05-27T09:29:51.342 INFO:tasks.workunit.client.1.smithi138.stdout:2/675: dwrite d1/d15/d2d/f9f [0,4194304] 0 2024-05-27T09:29:51.342 INFO:tasks.workunit.client.1.smithi138.stdout:2/676: chown d1/d10/d52/d5d/f88 428429 1 2024-05-27T09:29:51.342 INFO:tasks.workunit.client.1.smithi138.stdout:2/677: write d1/d15/d2d/f3e [1414712,54627] 0 2024-05-27T09:29:51.343 INFO:tasks.workunit.client.1.smithi138.stdout:2/678: write d1/d15/dd6/f43 [471383,85672] 0 2024-05-27T09:29:51.343 INFO:tasks.workunit.client.1.smithi138.stdout:2/679: dread - d1/d15/d2d/d17/f9b zero size 2024-05-27T09:29:51.345 INFO:tasks.workunit.client.1.smithi138.stdout:9/873: link d6/db/de/d18/ce0 d6/db/de/d6b/d104/d12a/d102/c12d 0 2024-05-27T09:29:51.346 INFO:tasks.workunit.client.1.smithi138.stdout:7/964: dwrite d0/d2b/daf/d43/ffb [0,4194304] 0 2024-05-27T09:29:51.347 INFO:tasks.workunit.client.1.smithi138.stdout:9/874: mknod d6/db/de/d85/c12e 0 2024-05-27T09:29:51.347 INFO:tasks.workunit.client.1.smithi138.stdout:4/899: dwrite d7/df/f4d [0,4194304] 0 2024-05-27T09:29:51.347 INFO:tasks.workunit.client.1.smithi138.stdout:9/875: stat d6/db/de/d6b/d104/d12a/d102/d128/lfb 0 2024-05-27T09:29:51.348 INFO:tasks.workunit.client.1.smithi138.stdout:7/965: creat d0/d23/f136 x:0 0 0 2024-05-27T09:29:51.349 INFO:tasks.workunit.client.1.smithi138.stdout:7/966: readlink d0/d23/d3c/l114 0 2024-05-27T09:29:51.349 INFO:tasks.workunit.client.1.smithi138.stdout:7/967: truncate d0/f77 1907128 0 2024-05-27T09:29:51.350 INFO:tasks.workunit.client.1.smithi138.stdout:4/900: symlink d7/dba/dcf/daa/def/l13e 0 2024-05-27T09:29:51.350 INFO:tasks.workunit.client.1.smithi138.stdout:4/901: readlink d7/dba/dcf/l2b 0 2024-05-27T09:29:51.361 INFO:tasks.workunit.client.1.smithi138.stdout:9/876: rename d6/da/d29/l60 to d6/db/d8d/d115/l12f 0 2024-05-27T09:29:51.361 INFO:tasks.workunit.client.1.smithi138.stdout:9/877: truncate d6/da/d29/d8f/db7/dbd/f108 4914496 0 2024-05-27T09:29:51.361 INFO:tasks.workunit.client.1.smithi138.stdout:9/878: fsync d6/da/d29/d8f/db7/dbd/d4c/fae 0 2024-05-27T09:29:51.361 INFO:tasks.workunit.client.1.smithi138.stdout:9/879: stat d6/da/d29/d59/dee/d55/d75/dd8 0 2024-05-27T09:29:51.363 INFO:tasks.workunit.client.1.smithi138.stdout:4/902: dread d7/df/f68 [0,4194304] 0 2024-05-27T09:29:51.366 INFO:tasks.workunit.client.1.smithi138.stdout:4/903: getdents d7/df/d5b 0 2024-05-27T09:29:51.366 INFO:tasks.workunit.client.1.smithi138.stdout:4/904: readlink d7/df/l84 0 2024-05-27T09:29:51.368 INFO:tasks.workunit.client.1.smithi138.stdout:0/739: sync 2024-05-27T09:29:51.368 INFO:tasks.workunit.client.1.smithi138.stdout:0/740: stat da/d2a/d8e/d1f/d42/d63/cd9 0 2024-05-27T09:29:51.370 INFO:tasks.workunit.client.1.smithi138.stdout:0/741: creat da/d2a/d8e/d1f/fee x:0 0 0 2024-05-27T09:29:51.370 INFO:tasks.workunit.client.1.smithi138.stdout:0/742: write f1 [2814509,98031] 0 2024-05-27T09:29:51.374 INFO:tasks.workunit.client.1.smithi138.stdout:0/743: mkdir da/d17/def 0 2024-05-27T09:29:51.376 INFO:tasks.workunit.client.1.smithi138.stdout:4/905: write d7/df/d35/d4c/d59/d5d/d64/df6/f11c [378570,111954] 0 2024-05-27T09:29:51.376 INFO:tasks.workunit.client.1.smithi138.stdout:4/906: write d7/df/d35/d4c/d59/f108 [7930,45661] 0 2024-05-27T09:29:51.381 INFO:tasks.workunit.client.1.smithi138.stdout:4/907: rename d7/df/d5b to d7/df/d35/d4c/d59/d5d/d64/df6/d13f 0 2024-05-27T09:29:51.381 INFO:tasks.workunit.client.1.smithi138.stdout:4/908: readlink d7/df/d35/d4c/d59/d5d/d64/df6/d13f/lb3 0 2024-05-27T09:29:51.381 INFO:tasks.workunit.client.1.smithi138.stdout:4/909: fsync d7/fdd 0 2024-05-27T09:29:51.381 INFO:tasks.workunit.client.1.smithi138.stdout:4/910: chown d7/df/d35/d4c/d59/d5d/d64/df6/d13f/f86 1 1 2024-05-27T09:29:51.386 INFO:tasks.workunit.client.1.smithi138.stdout:4/911: creat d7/df/d35/d4c/d59/d5d/d64/d6a/f140 x:0 0 0 2024-05-27T09:29:51.403 INFO:tasks.workunit.client.1.smithi138.stdout:9/880: write d6/db/dc/f4d [3438771,122798] 0 2024-05-27T09:29:51.405 INFO:tasks.workunit.client.1.smithi138.stdout:9/881: rename d6/db/fd4 to d6/db/de/d18/ddb/f130 0 2024-05-27T09:29:51.407 INFO:tasks.workunit.client.1.smithi138.stdout:9/882: rmdir d6/db/de 39 2024-05-27T09:29:51.408 INFO:tasks.workunit.client.1.smithi138.stdout:9/883: mkdir d6/db/de/d6b/d104/d12a/d131 0 2024-05-27T09:29:51.411 INFO:tasks.workunit.client.1.smithi138.stdout:7/968: dwrite d0/d23/dc4/ddf/f131 [0,4194304] 0 2024-05-27T09:29:51.413 INFO:tasks.workunit.client.1.smithi138.stdout:4/912: dread d7/dba/dcf/d3a/f8e [0,4194304] 0 2024-05-27T09:29:51.413 INFO:tasks.workunit.client.1.smithi138.stdout:4/913: chown d7/df/d35/d4c/d59/d5d/d64/df6 141 1 2024-05-27T09:29:51.413 INFO:tasks.workunit.client.1.smithi138.stdout:4/914: readlink d7/df/d35/d4c/d59/d5d/d64/l7a 0 2024-05-27T09:29:51.414 INFO:tasks.workunit.client.1.smithi138.stdout:9/884: link d6/db/de/d18/d24/c6f d6/db/de/d6b/d104/d12a/c132 0 2024-05-27T09:29:51.415 INFO:tasks.workunit.client.1.smithi138.stdout:4/915: rmdir d7/df/d35/d4c/d59/d5d/d64/dc5 39 2024-05-27T09:29:51.416 INFO:tasks.workunit.client.1.smithi138.stdout:9/885: mkdir d6/db/de/d18/d133 0 2024-05-27T09:29:51.421 INFO:tasks.workunit.client.1.smithi138.stdout:4/916: rmdir d7/df/d12f 0 2024-05-27T09:29:51.431 INFO:tasks.workunit.client.1.smithi138.stdout:7/969: dread d0/d2b/daf/d43/f67 [4194304,4194304] 0 2024-05-27T09:29:51.432 INFO:tasks.workunit.client.1.smithi138.stdout:9/886: dread d6/da/d29/d59/dee/d55/f8a [0,4194304] 0 2024-05-27T09:29:51.433 INFO:tasks.workunit.client.1.smithi138.stdout:0/744: dwrite da/d2a/d55/fdc [0,4194304] 0 2024-05-27T09:29:51.433 INFO:tasks.workunit.client.1.smithi138.stdout:0/745: chown da/d2a/d8e/d1f/f3e 349901 1 2024-05-27T09:29:51.433 INFO:tasks.workunit.client.1.smithi138.stdout:0/746: read da/d2a/d55/f61 [6477672,84217] 0 2024-05-27T09:29:51.433 INFO:tasks.workunit.client.1.smithi138.stdout:0/747: truncate da/d2a/d3f/f59 4534808 0 2024-05-27T09:29:51.434 INFO:tasks.workunit.client.1.smithi138.stdout:7/970: link d0/d2b/daf/f54 d0/d23/dc4/f137 0 2024-05-27T09:29:51.434 INFO:tasks.workunit.client.1.smithi138.stdout:9/887: mknod d6/da/d29/d8f/c134 0 2024-05-27T09:29:51.436 INFO:tasks.workunit.client.1.smithi138.stdout:9/888: getdents d6/da/d2f 0 2024-05-27T09:29:51.436 INFO:tasks.workunit.client.1.smithi138.stdout:9/889: read d6/da/f5d [454675,22036] 0 2024-05-27T09:29:51.438 INFO:tasks.workunit.client.1.smithi138.stdout:9/890: mkdir d6/da/d2f/d135 0 2024-05-27T09:29:51.438 INFO:tasks.workunit.client.1.smithi138.stdout:9/891: fsync d6/da/d29/d8f/db7/dbd/d4c/fd0 0 2024-05-27T09:29:51.439 INFO:tasks.workunit.client.1.smithi138.stdout:9/892: link d6/db/de/d18/f64 d6/db/de/d85/d119/f136 0 2024-05-27T09:29:51.440 INFO:tasks.workunit.client.1.smithi138.stdout:9/893: chown d6/db/de/d18/d24/d72/ccb 1 1 2024-05-27T09:29:51.440 INFO:tasks.workunit.client.1.smithi138.stdout:0/748: mknod da/d2a/d8e/dda/ddf/cf0 0 2024-05-27T09:29:51.444 INFO:tasks.workunit.client.1.smithi138.stdout:0/749: getdents da/d2a/d8e/d1f/dd5 0 2024-05-27T09:29:51.444 INFO:tasks.workunit.client.1.smithi138.stdout:4/917: write d7/d6e/df8/f107 [1647722,113870] 0 2024-05-27T09:29:51.444 INFO:tasks.workunit.client.1.smithi138.stdout:4/918: dread - d7/dba/dcf/d3a/f11f zero size 2024-05-27T09:29:51.444 INFO:tasks.workunit.client.1.smithi138.stdout:9/894: creat d6/db/d8d/f137 x:0 0 0 2024-05-27T09:29:51.444 INFO:tasks.workunit.client.1.smithi138.stdout:9/895: write d6/db/de/d18/d24/f73 [5028483,95123] 0 2024-05-27T09:29:51.444 INFO:tasks.workunit.client.1.smithi138.stdout:9/896: write d6/db/de/d6b/d104/d10a/f129 [261553,10748] 0 2024-05-27T09:29:51.444 INFO:tasks.workunit.client.1.smithi138.stdout:9/897: chown d6/db/de/c62 639240 1 2024-05-27T09:29:51.444 INFO:tasks.workunit.client.1.smithi138.stdout:4/919: creat d7/df/d35/d4c/d59/d5d/f141 x:0 0 0 2024-05-27T09:29:51.447 INFO:tasks.workunit.client.1.smithi138.stdout:4/920: creat d7/dba/dcf/daa/dda/f142 x:0 0 0 2024-05-27T09:29:51.447 INFO:tasks.workunit.client.1.smithi138.stdout:4/921: chown d7/dba/dcf/d3a 173991805 1 2024-05-27T09:29:51.449 INFO:tasks.workunit.client.1.smithi138.stdout:4/922: mknod d7/dba/dcf/daa/c143 0 2024-05-27T09:29:51.451 INFO:tasks.workunit.client.1.smithi138.stdout:4/923: link d7/df/d35/d4c/d59/d5d/d64/d6a/la8 d7/dba/dcf/daa/dda/d138/l144 0 2024-05-27T09:29:51.452 INFO:tasks.workunit.client.1.smithi138.stdout:4/924: write d7/dba/dcf/daa/fe0 [669287,128165] 0 2024-05-27T09:29:51.461 INFO:tasks.workunit.client.1.smithi138.stdout:9/898: truncate d6/db/de/d6b/d104/d10a/f129 138552 0 2024-05-27T09:29:51.463 INFO:tasks.workunit.client.1.smithi138.stdout:9/899: mkdir d6/da/d29/d8f/db7/dbd/d10e/d138 0 2024-05-27T09:29:51.505 INFO:tasks.workunit.client.1.smithi138.stdout:9/900: dread d6/da/d29/d59/dee/d55/f6c [0,4194304] 0 2024-05-27T09:29:51.507 INFO:tasks.workunit.client.1.smithi138.stdout:9/901: symlink d6/db/de/d85/l139 0 2024-05-27T09:29:51.507 INFO:tasks.workunit.client.1.smithi138.stdout:9/902: stat d6/fef 0 2024-05-27T09:29:51.508 INFO:tasks.workunit.client.1.smithi138.stdout:7/971: dwrite d0/d2b/daf/d98/db6/ff2 [0,4194304] 0 2024-05-27T09:29:51.510 INFO:tasks.workunit.client.1.smithi138.stdout:9/903: unlink d6/db/l56 0 2024-05-27T09:29:51.510 INFO:tasks.workunit.client.1.smithi138.stdout:9/904: chown d6/db/d8d/fdc 7477926 1 2024-05-27T09:29:51.516 INFO:tasks.workunit.client.1.smithi138.stdout:0/750: dwrite da/d2a/d8e/d1f/d42/d63/fb5 [0,4194304] 0 2024-05-27T09:29:51.533 INFO:tasks.workunit.client.1.smithi138.stdout:7/972: dread d0/d23/d3c/fb9 [0,4194304] 0 2024-05-27T09:29:51.533 INFO:tasks.workunit.client.1.smithi138.stdout:9/905: dread d6/da/fa9 [0,4194304] 0 2024-05-27T09:29:51.536 INFO:tasks.workunit.client.1.smithi138.stdout:9/906: write d6/da/f9e [486575,121561] 0 2024-05-27T09:29:51.537 INFO:tasks.workunit.client.1.smithi138.stdout:0/751: write da/d2a/d8e/d6a/fd8 [1026159,120180] 0 2024-05-27T09:29:51.538 INFO:tasks.workunit.client.1.smithi138.stdout:7/973: unlink d0/d23/dc4/ddf/d91/db8/fef 0 2024-05-27T09:29:51.539 INFO:tasks.workunit.client.1.smithi138.stdout:4/925: dwrite d7/df/d35/d4c/d59/d5d/d64/df6/d13f/da1/dbc/f124 [0,4194304] 0 2024-05-27T09:29:51.541 INFO:tasks.workunit.client.1.smithi138.stdout:9/907: getdents d6/da/d29/d59 0 2024-05-27T09:29:51.543 INFO:tasks.workunit.client.1.smithi138.stdout:0/752: mkdir da/d2a/d8e/d43/d74/d97/df1 0 2024-05-27T09:29:51.543 INFO:tasks.workunit.client.1.smithi138.stdout:9/908: mkdir d6/da/d29/d8f/db7/dbd/da2/d13a 0 2024-05-27T09:29:51.544 INFO:tasks.workunit.client.1.smithi138.stdout:9/909: creat d6/db/dc/f13b x:0 0 0 2024-05-27T09:29:51.545 INFO:tasks.workunit.client.1.smithi138.stdout:9/910: symlink d6/da/d29/d61/l13c 0 2024-05-27T09:29:51.545 INFO:tasks.workunit.client.1.smithi138.stdout:9/911: dread - d6/db/de/d18/ddb/f110 zero size 2024-05-27T09:29:51.545 INFO:tasks.workunit.client.1.smithi138.stdout:9/912: chown d6/db/d8d/fdc 63542340 1 2024-05-27T09:29:51.557 INFO:tasks.workunit.client.1.smithi138.stdout:0/753: unlink da/d2a/d2b/c69 0 2024-05-27T09:29:51.569 INFO:tasks.workunit.client.1.smithi138.stdout:0/754: symlink da/d2a/d8e/dc0/lf2 0 2024-05-27T09:29:51.569 INFO:tasks.workunit.client.1.smithi138.stdout:1/838: sync 2024-05-27T09:29:51.570 INFO:tasks.workunit.client.1.smithi138.stdout:4/926: dread d7/df/d35/d4c/f116 [4194304,4194304] 0 2024-05-27T09:29:51.574 INFO:tasks.workunit.client.1.smithi138.stdout:4/927: unlink d7/df/d35/d4c/d59/f9d 0 2024-05-27T09:29:51.579 INFO:tasks.workunit.client.1.smithi138.stdout:1/839: dread d4/d48/d22/d65/f6f [0,4194304] 0 2024-05-27T09:29:51.582 INFO:tasks.workunit.client.1.smithi138.stdout:0/755: read da/d17/f44 [5740276,54980] 0 2024-05-27T09:29:51.587 INFO:tasks.workunit.client.1.smithi138.stdout:1/840: rename d4/d48/d39/d5f/d4d/c92 to d4/d48/d39/d5f/d4d/c116 0 2024-05-27T09:29:51.587 INFO:tasks.workunit.client.1.smithi138.stdout:1/841: truncate d4/d48/d22/f10f 345305 0 2024-05-27T09:29:51.587 INFO:tasks.workunit.client.1.smithi138.stdout:1/842: write d4/d3c/fc8 [2019074,107823] 0 2024-05-27T09:29:51.587 INFO:tasks.workunit.client.1.smithi138.stdout:4/928: rename d7/dba/dcf/daa to d7/d145 0 2024-05-27T09:29:51.592 INFO:tasks.workunit.client.1.smithi138.stdout:0/756: getdents da/d2a/d8e/d6a/dc5 0 2024-05-27T09:29:51.592 INFO:tasks.workunit.client.1.smithi138.stdout:0/757: stat da/d2a/d8e/d1f/d42/d4a/d53/f87 0 2024-05-27T09:29:51.594 INFO:tasks.workunit.client.1.smithi138.stdout:4/929: mknod d7/df/d35/d4c/d59/d5d/d64/df6/d13f/c146 0 2024-05-27T09:29:51.594 INFO:tasks.workunit.client.1.smithi138.stdout:4/930: fdatasync d7/dba/f125 0 2024-05-27T09:29:51.598 INFO:tasks.workunit.client.1.smithi138.stdout:0/758: unlink da/d2a/d55/fb4 0 2024-05-27T09:29:51.599 INFO:tasks.workunit.client.1.smithi138.stdout:4/931: getdents d7/d6e/df8 0 2024-05-27T09:29:51.603 INFO:tasks.workunit.client.1.smithi138.stdout:0/759: symlink da/d2a/d8e/d1f/d60/dbd/lf3 0 2024-05-27T09:29:51.604 INFO:tasks.workunit.client.1.smithi138.stdout:0/760: chown da/d2a/d55/f5c 1271 1 2024-05-27T09:29:51.606 INFO:tasks.workunit.client.1.smithi138.stdout:4/932: unlink d7/dba/dcf/d3a/cb4 0 2024-05-27T09:29:51.606 INFO:tasks.workunit.client.1.smithi138.stdout:4/933: fsync d7/d6e/df8/f106 0 2024-05-27T09:29:51.611 INFO:tasks.workunit.client.1.smithi138.stdout:1/843: write d4/d1a/d2e/d32/d8c/dc6/f61 [3690684,127464] 0 2024-05-27T09:29:51.615 INFO:tasks.workunit.client.1.smithi138.stdout:0/761: mkdir da/df4 0 2024-05-27T09:29:51.615 INFO:tasks.workunit.client.1.smithi138.stdout:0/762: chown da/d2a/d8e/d1f/d71/f79 436940 1 2024-05-27T09:29:51.617 INFO:tasks.workunit.client.1.smithi138.stdout:0/763: creat da/d2a/d8e/d1f/ff5 x:0 0 0 2024-05-27T09:29:51.617 INFO:tasks.workunit.client.1.smithi138.stdout:0/764: write da/d2a/d55/f61 [9926245,70510] 0 2024-05-27T09:29:51.617 INFO:tasks.workunit.client.1.smithi138.stdout:0/765: dread - da/d2a/d8e/d43/fe5 zero size 2024-05-27T09:29:51.628 INFO:tasks.workunit.client.1.smithi138.stdout:4/934: link f3 d7/df/d35/d4c/d59/d5d/d64/df6/d10d/f147 0 2024-05-27T09:29:51.643 INFO:tasks.workunit.client.1.smithi138.stdout:0/766: read da/d2a/d8e/d1f/d42/d63/fb5 [4063985,103997] 0 2024-05-27T09:29:51.645 INFO:tasks.workunit.client.1.smithi138.stdout:0/767: getdents da/d2a/d3f 0 2024-05-27T09:29:51.649 INFO:tasks.workunit.client.1.smithi138.stdout:0/768: read da/d2a/d8e/d43/f6c [397139,7106] 0 2024-05-27T09:29:51.649 INFO:tasks.workunit.client.1.smithi138.stdout:0/769: truncate da/f10 5235787 0 2024-05-27T09:29:51.649 INFO:tasks.workunit.client.1.smithi138.stdout:7/974: dwrite d0/d2b/daf/f12 [0,4194304] 0 2024-05-27T09:29:51.653 INFO:tasks.workunit.client.1.smithi138.stdout:0/770: mknod da/d2a/d8e/d1f/d42/d63/d9d/cf6 0 2024-05-27T09:29:51.657 INFO:tasks.workunit.client.1.smithi138.stdout:9/913: dwrite d6/da/d29/d61/f11a [0,4194304] 0 2024-05-27T09:29:51.661 INFO:tasks.workunit.client.1.smithi138.stdout:9/914: creat d6/db/de/d6b/d104/d12a/d102/f13d x:0 0 0 2024-05-27T09:29:51.662 INFO:tasks.workunit.client.1.smithi138.stdout:1/844: dwrite d4/d48/d39/d5f/d63/fde [0,4194304] 0 2024-05-27T09:29:51.662 INFO:tasks.workunit.client.1.smithi138.stdout:4/935: dread d7/df/d35/d4c/d59/d5d/d64/df6/d13f/da1/db1/fe6 [4194304,4194304] 0 2024-05-27T09:29:51.663 INFO:tasks.workunit.client.1.smithi138.stdout:9/915: rename d6/db/dc/c87 to d6/da/d29/d59/dee/d55/c13e 0 2024-05-27T09:29:51.668 INFO:tasks.workunit.client.1.smithi138.stdout:4/936: creat d7/dba/dfa/f148 x:0 0 0 2024-05-27T09:29:51.668 INFO:tasks.workunit.client.1.smithi138.stdout:4/937: chown d7/df/d35/d4c/d59/d5d/d64/df6/d13f/da1/db1 1578 1 2024-05-27T09:29:51.668 INFO:tasks.workunit.client.1.smithi138.stdout:4/938: dread - d7/d6e/fc4 zero size 2024-05-27T09:29:51.676 INFO:tasks.workunit.client.1.smithi138.stdout:3/781: sync 2024-05-27T09:29:51.676 INFO:tasks.workunit.client.1.smithi138.stdout:1/845: getdents d4/d48 0 2024-05-27T09:29:51.676 INFO:tasks.workunit.client.1.smithi138.stdout:1/846: chown d4/d48/d39/f54 0 1 2024-05-27T09:29:51.676 INFO:tasks.workunit.client.1.smithi138.stdout:1/847: fsync d4/d1a/d2e/d4a/d7e/db7/dfb/f103 0 2024-05-27T09:29:51.676 INFO:tasks.workunit.client.1.smithi138.stdout:1/848: stat d4/d3c/fa0 0 2024-05-27T09:29:51.676 INFO:tasks.workunit.client.1.smithi138.stdout:4/939: symlink d7/dba/dfa/d127/d132/l149 0 2024-05-27T09:29:51.676 INFO:tasks.workunit.client.1.smithi138.stdout:4/940: stat d7/df/d35/d4c/d59/d5d/d64/df6/d13f/lb3 0 2024-05-27T09:29:51.676 INFO:tasks.workunit.client.1.smithi138.stdout:5/873: sync 2024-05-27T09:29:51.676 INFO:tasks.workunit.client.1.smithi138.stdout:2/680: sync 2024-05-27T09:29:51.676 INFO:tasks.workunit.client.1.smithi138.stdout:5/874: write d3/d12/d32/f4e [2682046,42860] 0 2024-05-27T09:29:51.684 INFO:tasks.workunit.client.1.smithi138.stdout:1/849: link d4/d48/d39/d5f/d4d/fac d4/d1a/d2e/d32/d8c/dc6/f117 0 2024-05-27T09:29:51.685 INFO:tasks.workunit.client.1.smithi138.stdout:4/941: symlink d7/dba/l14a 0 2024-05-27T09:29:51.694 INFO:tasks.workunit.client.1.smithi138.stdout:5/875: link d3/ddc/c122 d3/d12/d16/d1d/d22/d2b/d9e/c126 0 2024-05-27T09:29:51.694 INFO:tasks.workunit.client.1.smithi138.stdout:2/681: dread d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc/f33 [0,4194304] 0 2024-05-27T09:29:51.702 INFO:tasks.workunit.client.1.smithi138.stdout:3/782: dread d0/d2/da/df/d10/d2a/d61/de1/fc7 [0,4194304] 0 2024-05-27T09:29:51.704 INFO:tasks.workunit.client.1.smithi138.stdout:5/876: rmdir d3/d12/d32/d60/de6/d111 39 2024-05-27T09:29:51.704 INFO:tasks.workunit.client.1.smithi138.stdout:5/877: write d3/f4c [124205,123077] 0 2024-05-27T09:29:51.705 INFO:tasks.workunit.client.1.smithi138.stdout:0/771: dread da/d2a/d8e/d1f/d42/f9a [0,4194304] 0 2024-05-27T09:29:51.707 INFO:tasks.workunit.client.1.smithi138.stdout:2/682: creat d1/d15/d2d/d2e/fe7 x:0 0 0 2024-05-27T09:29:51.707 INFO:tasks.workunit.client.1.smithi138.stdout:1/850: dread d4/d48/d39/d5f/d4d/d2b/f6c [0,4194304] 0 2024-05-27T09:29:51.707 INFO:tasks.workunit.client.1.smithi138.stdout:3/783: mkdir d0/d2/d105 0 2024-05-27T09:29:51.708 INFO:tasks.workunit.client.1.smithi138.stdout:2/683: read d1/d15/d2d/d17/d25/d4d/d4e/f9e [1114375,122091] 0 2024-05-27T09:29:51.708 INFO:tasks.workunit.client.1.smithi138.stdout:2/684: dread - d1/d10/d52/fdb zero size 2024-05-27T09:29:51.708 INFO:tasks.workunit.client.1.smithi138.stdout:2/685: readlink d1/d15/d2d/d17/d25/d4d/d4e/d60/lc3 0 2024-05-27T09:29:51.708 INFO:tasks.workunit.client.1.smithi138.stdout:3/784: mkdir d0/dda/d106 0 2024-05-27T09:29:51.714 INFO:tasks.workunit.client.1.smithi138.stdout:3/785: symlink d0/d2/da/df/l107 0 2024-05-27T09:29:51.720 INFO:tasks.workunit.client.1.smithi138.stdout:7/975: dwrite d0/d2b/daf/d43/fa9 [0,4194304] 0 2024-05-27T09:29:51.721 INFO:tasks.workunit.client.1.smithi138.stdout:5/878: link d3/d12/d16/d1d/d23/l56 d3/d12/d16/d1d/d23/d3b/dec/d100/l127 0 2024-05-27T09:29:51.721 INFO:tasks.workunit.client.1.smithi138.stdout:5/879: write d3/d12/d32/d7e/f9f [4215098,114630] 0 2024-05-27T09:29:51.722 INFO:tasks.workunit.client.1.smithi138.stdout:0/772: rmdir da/d2a/d8e/dda 39 2024-05-27T09:29:51.727 INFO:tasks.workunit.client.1.smithi138.stdout:1/851: mkdir d4/d48/d22/d65/d100/d118 0 2024-05-27T09:29:51.730 INFO:tasks.workunit.client.1.smithi138.stdout:7/976: creat d0/d23/d3c/d123/f138 x:0 0 0 2024-05-27T09:29:51.731 INFO:tasks.workunit.client.1.smithi138.stdout:7/977: mknod d0/d2b/daf/dba/c139 0 2024-05-27T09:29:51.732 INFO:tasks.workunit.client.1.smithi138.stdout:7/978: dread - d0/d23/d3c/d84/d9c/dc3/de6/fe8 zero size 2024-05-27T09:29:51.736 INFO:tasks.workunit.client.1.smithi138.stdout:5/880: symlink d3/d12/d32/d60/d93/l128 0 2024-05-27T09:29:51.736 INFO:tasks.workunit.client.1.smithi138.stdout:5/881: creat d3/d12/d32/d87/d89/f129 x:0 0 0 2024-05-27T09:29:51.736 INFO:tasks.workunit.client.1.smithi138.stdout:5/882: chown d3/d12/d16/l96 50 1 2024-05-27T09:29:51.746 INFO:tasks.workunit.client.1.smithi138.stdout:5/883: link d3/d12/d16/c59 d3/d12/d16/d1d/d23/d3b/dec/d100/c12a 0 2024-05-27T09:29:51.746 INFO:tasks.workunit.client.1.smithi138.stdout:5/884: read - d3/d12/d32/d60/de6/d111/f118 zero size 2024-05-27T09:29:51.746 INFO:tasks.workunit.client.1.smithi138.stdout:5/885: creat d3/d75/f12b x:0 0 0 2024-05-27T09:29:51.759 INFO:tasks.workunit.client.1.smithi138.stdout:5/886: mknod d3/d12/dcf/d11e/c12c 0 2024-05-27T09:29:51.766 INFO:tasks.workunit.client.1.smithi138.stdout:3/786: dwrite d0/d2/da/df/d10/d2a/d61/de1/f101 [0,4194304] 0 2024-05-27T09:29:51.780 INFO:tasks.workunit.client.1.smithi138.stdout:1/852: dwrite d4/d1a/d2e/f79 [0,4194304] 0 2024-05-27T09:29:51.782 INFO:tasks.workunit.client.1.smithi138.stdout:1/853: creat d4/d1a/d2e/d4a/d7e/dc0/f119 x:0 0 0 2024-05-27T09:29:51.796 INFO:tasks.workunit.client.1.smithi138.stdout:5/887: dread d3/d12/d32/d6b/f78 [0,4194304] 0 2024-05-27T09:29:51.796 INFO:tasks.workunit.client.1.smithi138.stdout:5/888: fsync d3/d12/f79 0 2024-05-27T09:29:51.796 INFO:tasks.workunit.client.1.smithi138.stdout:5/889: write d3/d12/d16/d1d/d23/f62 [1783969,2705] 0 2024-05-27T09:29:51.808 INFO:tasks.workunit.client.1.smithi138.stdout:0/773: dwrite da/d2a/d8e/d1f/d42/fac [0,4194304] 0 2024-05-27T09:29:51.808 INFO:tasks.workunit.client.1.smithi138.stdout:0/774: stat da/d2a/d8e/d1f/d42/c92 0 2024-05-27T09:29:51.808 INFO:tasks.workunit.client.1.smithi138.stdout:7/979: dwrite d0/d2b/daf/dbf/fd2 [0,4194304] 0 2024-05-27T09:29:51.811 INFO:tasks.workunit.client.1.smithi138.stdout:7/980: mknod d0/d2b/daf/d98/db6/d132/c13a 0 2024-05-27T09:29:51.812 INFO:tasks.workunit.client.1.smithi138.stdout:7/981: mknod d0/d2b/daf/d98/db6/d132/c13b 0 2024-05-27T09:29:51.814 INFO:tasks.workunit.client.1.smithi138.stdout:7/982: creat d0/d23/da8/d8e/df4/f13c x:0 0 0 2024-05-27T09:29:51.814 INFO:tasks.workunit.client.1.smithi138.stdout:7/983: rename d0/d2b/daf to d0/d2b/daf/d43/d13d 22 2024-05-27T09:29:51.815 INFO:tasks.workunit.client.1.smithi138.stdout:0/775: mkdir da/d2a/d8e/d1f/dd5/df7 0 2024-05-27T09:29:51.815 INFO:tasks.workunit.client.1.smithi138.stdout:0/776: rename da to da/d2a/d8e/d1f/d42/d63/dea/df8 22 2024-05-27T09:29:51.815 INFO:tasks.workunit.client.1.smithi138.stdout:0/777: write da/d2a/d8e/d1f/fee [225331,87812] 0 2024-05-27T09:29:51.815 INFO:tasks.workunit.client.1.smithi138.stdout:0/778: read - da/d2a/d8e/d1f/d42/d4a/d53/f90 zero size 2024-05-27T09:29:51.815 INFO:tasks.workunit.client.1.smithi138.stdout:0/779: dread - da/d2a/d8e/d1f/d4f/fa8 zero size 2024-05-27T09:29:51.836 INFO:tasks.workunit.client.1.smithi138.stdout:0/780: rmdir da/d2a/d8e/d1f/d42 39 2024-05-27T09:29:51.854 INFO:tasks.workunit.client.1.smithi138.stdout:3/787: dwrite d0/d2/da/f6b [0,4194304] 0 2024-05-27T09:29:51.876 INFO:tasks.workunit.client.1.smithi138.stdout:7/984: dwrite d0/d2b/daf/dbf/fd2 [0,4194304] 0 2024-05-27T09:29:51.877 INFO:tasks.workunit.client.1.smithi138.stdout:7/985: fsync d0/d23/dc4/ddf/d91/d108/f12d 0 2024-05-27T09:29:51.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:51 smithi138 ceph-mon[34885]: Updating smithi138:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:29:51.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:51 smithi138 ceph-mon[34885]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-05-27T09:29:51.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:51 smithi138 ceph-mon[34885]: pgmap v23: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 138 MiB/s rd, 164 MiB/s wr, 269 op/s 2024-05-27T09:29:51.892 INFO:tasks.workunit.client.1.smithi138.stdout:5/890: dwrite d3/d12/d16/d1d/d23/fba [0,4194304] 0 2024-05-27T09:29:51.897 INFO:tasks.workunit.client.1.smithi138.stdout:5/891: mknod d3/d12/d16/d1d/d22/dd9/c12d 0 2024-05-27T09:29:51.897 INFO:tasks.workunit.client.1.smithi138.stdout:5/892: stat d3/d12/d32/d60/de6/d111/ffa 0 2024-05-27T09:29:51.897 INFO:tasks.workunit.client.1.smithi138.stdout:5/893: chown d3/d12/d32/d87/d11b 64164 1 2024-05-27T09:29:51.899 INFO:tasks.workunit.client.1.smithi138.stdout:5/894: mkdir d3/d12/d16/d1d/d12e 0 2024-05-27T09:29:51.905 INFO:tasks.workunit.client.1.smithi138.stdout:3/788: dwrite d0/dda/fea [0,4194304] 0 2024-05-27T09:29:51.905 INFO:tasks.workunit.client.1.smithi138.stdout:3/789: getdents d0/d2/da/df/d10/d32/d37 0 2024-05-27T09:29:51.905 INFO:tasks.workunit.client.1.smithi138.stdout:3/790: read - d0/d2/da/df/d10/d2a/d61/d73/deb/ffe zero size 2024-05-27T09:29:51.958 INFO:tasks.workunit.client.1.smithi138.stdout:0/781: dwrite da/d2a/d8e/d1f/d60/fc6 [4194304,4194304] 0 2024-05-27T09:29:51.958 INFO:tasks.workunit.client.1.smithi138.stdout:3/791: dwrite d0/d2/fc8 [0,4194304] 0 2024-05-27T09:29:51.962 INFO:tasks.workunit.client.1.smithi138.stdout:7/986: dwrite d0/f110 [0,4194304] 0 2024-05-27T09:29:51.964 INFO:tasks.workunit.client.1.smithi138.stdout:0/782: truncate da/d2a/d8e/d1f/d42/d63/d9d/fb6 1145254 0 2024-05-27T09:29:51.966 INFO:tasks.workunit.client.1.smithi138.stdout:7/987: rmdir d0/d2b/daf/d60/d127 39 2024-05-27T09:29:51.966 INFO:tasks.workunit.client.1.smithi138.stdout:7/988: stat d0/d23/d3a 0 2024-05-27T09:29:51.969 INFO:tasks.workunit.client.1.smithi138.stdout:0/783: unlink da/d2a/d55/f5c 0 2024-05-27T09:29:51.969 INFO:tasks.workunit.client.1.smithi138.stdout:7/989: unlink d0/d2b/daf/dba/f126 0 2024-05-27T09:29:51.970 INFO:tasks.workunit.client.1.smithi138.stdout:7/990: fsync d0/d23/da8/fa2 0 2024-05-27T09:29:51.970 INFO:tasks.workunit.client.1.smithi138.stdout:7/991: chown d0/d23/d3a/f46 242249048 1 2024-05-27T09:29:51.978 INFO:tasks.workunit.client.1.smithi138.stdout:0/784: dread da/d2a/d8e/d1f/fa2 [0,4194304] 0 2024-05-27T09:29:51.978 INFO:tasks.workunit.client.1.smithi138.stdout:0/785: stat da 0 2024-05-27T09:29:51.988 INFO:tasks.workunit.client.1.smithi138.stdout:5/895: dwrite d3/d12/d16/d1d/d23/fba [0,4194304] 0 2024-05-27T09:29:51.988 INFO:tasks.workunit.client.1.smithi138.stdout:5/896: creat d3/d12/d16/d1d/d22/db9/f12f x:0 0 0 2024-05-27T09:29:51.988 INFO:tasks.workunit.client.1.smithi138.stdout:5/897: creat d3/d12/d32/d60/de6/df2/f130 x:0 0 0 2024-05-27T09:29:51.990 INFO:tasks.workunit.client.1.smithi138.stdout:7/992: write d0/f32 [4427335,68638] 0 2024-05-27T09:29:51.991 INFO:tasks.workunit.client.1.smithi138.stdout:7/993: write d0/d2b/daf/d43/f10a [5960177,100104] 0 2024-05-27T09:29:51.992 INFO:tasks.workunit.client.1.smithi138.stdout:7/994: mkdir d0/d23/d3c/d84/d9c/d103/d13e 0 2024-05-27T09:29:51.993 INFO:tasks.workunit.client.1.smithi138.stdout:7/995: write d0/d2b/daf/d98/fa4 [663736,49423] 0 2024-05-27T09:29:51.993 INFO:tasks.workunit.client.1.smithi138.stdout:7/996: chown d0/d23/d3a/l8b 9 1 2024-05-27T09:29:51.995 INFO:tasks.workunit.client.1.smithi138.stdout:7/997: mkdir d0/d13f 0 2024-05-27T09:29:51.997 INFO:tasks.workunit.client.1.smithi138.stdout:7/998: symlink d0/d2b/daf/d60/d9d/l140 0 2024-05-27T09:29:51.997 INFO:tasks.workunit.client.1.smithi138.stdout:7/999: chown d0/d23/dc4/ddf/d91/db8 10859534 1 2024-05-27T09:29:51.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:51 smithi119 ceph-mon[29082]: Updating smithi138:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:29:51.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:51 smithi119 ceph-mon[29082]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-05-27T09:29:51.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:51 smithi119 ceph-mon[29082]: pgmap v23: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 138 MiB/s rd, 164 MiB/s wr, 269 op/s 2024-05-27T09:29:52.016 INFO:tasks.workunit.client.1.smithi138.stdout:0/786: dwrite da/d2a/d8e/d1f/fa2 [0,4194304] 0 2024-05-27T09:29:52.065 INFO:tasks.workunit.client.1.smithi138.stdout:0/787: truncate da/d17/f44 291478 0 2024-05-27T09:29:52.065 INFO:tasks.workunit.client.1.smithi138.stdout:0/788: chown da/d2a/d55/dc7 0 1 2024-05-27T09:29:52.065 INFO:tasks.workunit.client.1.smithi138.stdout:5/898: rename d3/d12/d16/d1d/d22/d2b/d9e/db6/f109 to d3/d12/d16/d1d/d30/f131 0 2024-05-27T09:29:52.065 INFO:tasks.workunit.client.1.smithi138.stdout:5/899: chown d3/d12/d32/d60/de6/d111/ffa 23953255 1 2024-05-27T09:29:52.066 INFO:tasks.workunit.client.1.smithi138.stdout:0/789: mknod da/d2a/d2b/cf9 0 2024-05-27T09:29:52.070 INFO:tasks.workunit.client.1.smithi138.stdout:0/790: creat da/d2a/d8e/d1f/ffa x:0 0 0 2024-05-27T09:29:52.070 INFO:tasks.workunit.client.1.smithi138.stdout:0/791: chown da/d2a/d8e/d1f/dd5/df7 1475869628 1 2024-05-27T09:29:52.070 INFO:tasks.workunit.client.1.smithi138.stdout:0/792: readlink da/d2a/d8e/d1f/l7a 0 2024-05-27T09:29:52.071 INFO:tasks.workunit.client.1.smithi138.stdout:9/916: sync 2024-05-27T09:29:52.105 INFO:tasks.workunit.client.1.smithi138.stdout:0/793: rmdir da 39 2024-05-27T09:29:52.114 INFO:tasks.workunit.client.1.smithi138.stdout:0/794: creat da/d2a/dc9/dca/ffb x:0 0 0 2024-05-27T09:29:52.117 INFO:tasks.workunit.client.1.smithi138.stdout:0/795: rename da/d2a/d55/ca4 to da/d2a/d8e/d1f/dd5/df7/cfc 0 2024-05-27T09:29:52.119 INFO:tasks.workunit.client.1.smithi138.stdout:0/796: creat da/d2a/d8e/d1f/d42/ffd x:0 0 0 2024-05-27T09:29:52.122 INFO:tasks.workunit.client.1.smithi138.stdout:0/797: symlink da/d2a/d8e/d1f/d4f/lfe 0 2024-05-27T09:29:52.122 INFO:tasks.workunit.client.1.smithi138.stdout:0/798: dread - da/d2a/d2b/fb7 zero size 2024-05-27T09:29:52.124 INFO:tasks.workunit.client.1.smithi138.stdout:0/799: creat da/d2a/d55/fff x:0 0 0 2024-05-27T09:29:52.129 INFO:tasks.workunit.client.1.smithi138.stdout:5/900: dwrite d3/d12/d16/d1d/d22/f2a [0,4194304] 0 2024-05-27T09:29:52.130 INFO:tasks.workunit.client.1.smithi138.stdout:9/917: dwrite d6/f88 [0,4194304] 0 2024-05-27T09:29:52.130 INFO:tasks.workunit.client.1.smithi138.stdout:9/918: write d6/da/d29/d59/dee/fb1 [1536341,66592] 0 2024-05-27T09:29:52.133 INFO:tasks.workunit.client.1.smithi138.stdout:5/901: truncate d3/d12/d26/d53/f114 482926 0 2024-05-27T09:29:52.133 INFO:tasks.workunit.client.1.smithi138.stdout:5/902: truncate d3/d12/d16/d1d/d22/d2b/d9e/fc7 182732 0 2024-05-27T09:29:52.133 INFO:tasks.workunit.client.1.smithi138.stdout:5/903: truncate d3/d12/d16/d1d/d22/d2b/d9e/db6/fe9 978747 0 2024-05-27T09:29:52.133 INFO:tasks.workunit.client.1.smithi138.stdout:5/904: chown d3/d12/ffe 1141 1 2024-05-27T09:29:52.133 INFO:tasks.workunit.client.1.smithi138.stdout:5/905: readlink d3/d12/d32/d60/lb4 0 2024-05-27T09:29:52.134 INFO:tasks.workunit.client.1.smithi138.stdout:5/906: fdatasync d3/d12/d32/d60/f91 0 2024-05-27T09:29:52.134 INFO:tasks.workunit.client.1.smithi138.stdout:5/907: write d3/d12/d16/d1d/d22/d2b/daf/db2/fc4 [475980,81207] 0 2024-05-27T09:29:52.134 INFO:tasks.workunit.client.1.smithi138.stdout:5/908: dread - d3/d12/dcf/d11e/ff9 zero size 2024-05-27T09:29:52.137 INFO:tasks.workunit.client.1.smithi138.stdout:9/919: mknod d6/db/de/d6b/c13f 0 2024-05-27T09:29:52.137 INFO:tasks.workunit.client.1.smithi138.stdout:9/920: read - d6/db/de/d18/ddb/f110 zero size 2024-05-27T09:29:52.138 INFO:tasks.workunit.client.1.smithi138.stdout:0/800: dread da/d2a/d8e/d6a/fd8 [0,4194304] 0 2024-05-27T09:29:52.141 INFO:tasks.workunit.client.1.smithi138.stdout:5/909: dread d3/d12/d16/d1d/d22/d2b/d9e/fc7 [0,4194304] 0 2024-05-27T09:29:52.144 INFO:tasks.workunit.client.1.smithi138.stdout:0/801: symlink da/d2a/d8e/d1f/d42/d4a/l100 0 2024-05-27T09:29:52.144 INFO:tasks.workunit.client.1.smithi138.stdout:0/802: write da/d2a/d8e/d1f/d42/d4a/d53/f90 [694779,30433] 0 2024-05-27T09:29:52.146 INFO:tasks.workunit.client.1.smithi138.stdout:5/910: mknod d3/d12/d16/d1d/c132 0 2024-05-27T09:29:52.154 INFO:tasks.workunit.client.1.smithi138.stdout:5/911: dread d3/f4c [0,4194304] 0 2024-05-27T09:29:52.155 INFO:tasks.workunit.client.1.smithi138.stdout:5/912: chown d3/d12/d32/d60/de6/lff 277087 1 2024-05-27T09:29:52.155 INFO:tasks.workunit.client.1.smithi138.stdout:9/921: dread d6/db/de/d6b/d104/d12a/f58 [4194304,4194304] 0 2024-05-27T09:29:52.155 INFO:tasks.workunit.client.1.smithi138.stdout:9/922: dread - d6/da/d29/d59/dee/f122 zero size 2024-05-27T09:29:52.155 INFO:tasks.workunit.client.1.smithi138.stdout:9/923: chown d6/db/de/d18/d133 2830809 1 2024-05-27T09:29:52.166 INFO:tasks.workunit.client.1.smithi138.stdout:2/686: sync 2024-05-27T09:29:52.169 INFO:tasks.workunit.client.1.smithi138.stdout:2/687: creat d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/fe8 x:0 0 0 2024-05-27T09:29:52.174 INFO:tasks.workunit.client.1.smithi138.stdout:2/688: link d1/d15/d2d/d17/d25/d9d/d4b/lc4 d1/d15/d2d/d17/d25/d4d/db6/le9 0 2024-05-27T09:29:52.174 INFO:tasks.workunit.client.1.smithi138.stdout:2/689: write d1/d15/d2d/d17/d25/d9d/fc7 [994541,15389] 0 2024-05-27T09:29:52.174 INFO:tasks.workunit.client.1.smithi138.stdout:1/854: sync 2024-05-27T09:29:52.174 INFO:tasks.workunit.client.1.smithi138.stdout:4/942: sync 2024-05-27T09:29:52.175 INFO:tasks.workunit.client.1.smithi138.stdout:4/943: dread - d7/df/d35/d4c/d59/d5d/d64/d6a/ff4 zero size 2024-05-27T09:29:52.176 INFO:tasks.workunit.client.1.smithi138.stdout:4/944: symlink d7/l14b 0 2024-05-27T09:29:52.176 INFO:tasks.workunit.client.1.smithi138.stdout:4/945: dread - d7/f12a zero size 2024-05-27T09:29:52.177 INFO:tasks.workunit.client.1.smithi138.stdout:2/690: symlink d1/d15/d2d/d2e/lea 0 2024-05-27T09:29:52.177 INFO:tasks.workunit.client.1.smithi138.stdout:2/691: chown d1/d15/f4f 142693 1 2024-05-27T09:29:52.177 INFO:tasks.workunit.client.1.smithi138.stdout:2/692: readlink d1/d10/d52/d5d/d7e/l8c 0 2024-05-27T09:29:52.177 INFO:tasks.workunit.client.1.smithi138.stdout:2/693: dread - d1/d15/d2d/d17/d25/d4d/d4e/d60/f92 zero size 2024-05-27T09:29:52.179 INFO:tasks.workunit.client.1.smithi138.stdout:2/694: rmdir d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc 39 2024-05-27T09:29:52.186 INFO:tasks.workunit.client.1.smithi138.stdout:9/924: dread d6/db/de/d18/f7b [0,4194304] 0 2024-05-27T09:29:52.188 INFO:tasks.workunit.client.1.smithi138.stdout:4/946: read d7/dba/dfa/f11a [269006,122395] 0 2024-05-27T09:29:52.191 INFO:tasks.workunit.client.1.smithi138.stdout:4/947: mkdir d7/df/d35/d4c/d59/d5d/d64/d14c 0 2024-05-27T09:29:52.197 INFO:tasks.workunit.client.1.smithi138.stdout:4/948: creat d7/d145/dda/dfc/f14d x:0 0 0 2024-05-27T09:29:52.201 INFO:tasks.workunit.client.1.smithi138.stdout:1/855: dread d4/d48/f14 [4194304,4194304] 0 2024-05-27T09:29:52.201 INFO:tasks.workunit.client.1.smithi138.stdout:1/856: truncate d4/d88/ff2 908530 0 2024-05-27T09:29:52.203 INFO:tasks.workunit.client.1.smithi138.stdout:1/857: creat d4/d48/d22/d65/d100/d118/f11a x:0 0 0 2024-05-27T09:29:52.205 INFO:tasks.workunit.client.1.smithi138.stdout:1/858: creat d4/dcc/f11b x:0 0 0 2024-05-27T09:29:52.207 INFO:tasks.workunit.client.1.smithi138.stdout:9/925: write d6/da/d29/d8f/db7/dbd/d4c/fae [4781851,7194] 0 2024-05-27T09:29:52.209 INFO:tasks.workunit.client.1.smithi138.stdout:1/859: mknod d4/d38/d97/dfe/c11c 0 2024-05-27T09:29:52.210 INFO:tasks.workunit.client.1.smithi138.stdout:1/860: mkdir d4/d1a/d2e/d4a/d7e/d11d 0 2024-05-27T09:29:52.210 INFO:tasks.workunit.client.1.smithi138.stdout:1/861: dread - d4/d1a/d2e/d32/f109 zero size 2024-05-27T09:29:52.210 INFO:tasks.workunit.client.1.smithi138.stdout:1/862: dread - d4/d3c/f108 zero size 2024-05-27T09:29:52.211 INFO:tasks.workunit.client.1.smithi138.stdout:1/863: creat d4/d1a/d2e/d32/d8c/f11e x:0 0 0 2024-05-27T09:29:52.212 INFO:tasks.workunit.client.1.smithi138.stdout:1/864: readlink d4/d38/l4f 0 2024-05-27T09:29:52.212 INFO:tasks.workunit.client.1.smithi138.stdout:1/865: readlink d4/d48/d39/d5f/l87 0 2024-05-27T09:29:52.226 INFO:tasks.workunit.client.1.smithi138.stdout:9/926: write d6/db/de/d18/d24/f5c [7788322,102341] 0 2024-05-27T09:29:52.226 INFO:tasks.workunit.client.1.smithi138.stdout:9/927: creat d6/da/d29/d8f/db7/dbd/da2/f140 x:0 0 0 2024-05-27T09:29:52.226 INFO:tasks.workunit.client.1.smithi138.stdout:5/913: dwrite d3/d12/d32/d60/fa3 [0,4194304] 0 2024-05-27T09:29:52.237 INFO:tasks.workunit.client.1.smithi138.stdout:9/928: creat d6/da/d29/d59/dee/d55/d75/d8c/f141 x:0 0 0 2024-05-27T09:29:52.237 INFO:tasks.workunit.client.1.smithi138.stdout:9/929: stat d6/db/dc/f2c 0 2024-05-27T09:29:52.241 INFO:tasks.workunit.client.1.smithi138.stdout:4/949: dwrite d7/df/d35/d4c/d59/d5d/d64/dc5/f131 [0,4194304] 0 2024-05-27T09:29:52.241 INFO:tasks.workunit.client.1.smithi138.stdout:4/950: creat d7/df/d35/d4c/dc0/dfb/f14e x:0 0 0 2024-05-27T09:29:52.242 INFO:tasks.workunit.client.1.smithi138.stdout:4/951: chown d7/dba/dcb/fed 6 1 2024-05-27T09:29:52.242 INFO:tasks.workunit.client.1.smithi138.stdout:4/952: dread - d7/df/d35/d4c/dc0/dfb/f14e zero size 2024-05-27T09:29:52.263 INFO:tasks.workunit.client.1.smithi138.stdout:9/930: write d6/da/d2f/f32 [1759304,62156] 0 2024-05-27T09:29:52.263 INFO:tasks.workunit.client.1.smithi138.stdout:9/931: chown d6/db/lce 190 1 2024-05-27T09:29:52.272 INFO:tasks.workunit.client.1.smithi138.stdout:3/792: sync 2024-05-27T09:29:52.274 INFO:tasks.workunit.client.1.smithi138.stdout:9/932: dread d6/fef [0,4194304] 0 2024-05-27T09:29:52.274 INFO:tasks.workunit.client.1.smithi138.stdout:9/933: creat d6/da/d29/d61/f142 x:0 0 0 2024-05-27T09:29:52.279 INFO:tasks.workunit.client.1.smithi138.stdout:1/866: dwrite d4/d1a/d2e/f62 [0,4194304] 0 2024-05-27T09:29:52.281 INFO:tasks.workunit.client.1.smithi138.stdout:3/793: read d0/d2/da/df/d10/f4a [1538791,40975] 0 2024-05-27T09:29:52.282 INFO:tasks.workunit.client.1.smithi138.stdout:1/867: write d4/fd [58918,118393] 0 2024-05-27T09:29:52.286 INFO:tasks.workunit.client.1.smithi138.stdout:3/794: unlink d0/d3f/d50/f76 0 2024-05-27T09:29:52.292 INFO:tasks.workunit.client.1.smithi138.stdout:5/914: dwrite d3/d12/d16/f108 [0,4194304] 0 2024-05-27T09:29:52.292 INFO:tasks.workunit.client.1.smithi138.stdout:4/953: dwrite d7/df/d35/d4c/d59/d5d/d64/d6a/fbe [0,4194304] 0 2024-05-27T09:29:52.292 INFO:tasks.workunit.client.1.smithi138.stdout:4/954: chown d7/d6e/l113 471 1 2024-05-27T09:29:52.299 INFO:tasks.workunit.client.1.smithi138.stdout:4/955: mknod d7/dba/dcb/c14f 0 2024-05-27T09:29:52.299 INFO:tasks.workunit.client.1.smithi138.stdout:4/956: write d7/df/f29 [309947,57539] 0 2024-05-27T09:29:52.299 INFO:tasks.workunit.client.1.smithi138.stdout:4/957: truncate d7/df/f130 255960 0 2024-05-27T09:29:52.301 INFO:tasks.workunit.client.1.smithi138.stdout:3/795: dread d0/d3f/d50/f97 [0,4194304] 0 2024-05-27T09:29:52.301 INFO:tasks.workunit.client.1.smithi138.stdout:3/796: fdatasync d0/d2/da/df/fa1 0 2024-05-27T09:29:52.302 INFO:tasks.workunit.client.1.smithi138.stdout:4/958: creat d7/df/d35/d4c/d59/d5d/d64/df6/d13f/f150 x:0 0 0 2024-05-27T09:29:52.303 INFO:tasks.workunit.client.1.smithi138.stdout:4/959: mkdir d7/dba/dfa/d136/d151 0 2024-05-27T09:29:52.304 INFO:tasks.workunit.client.1.smithi138.stdout:3/797: mknod d0/d2/da/df/d10/d2a/d48/c108 0 2024-05-27T09:29:52.305 INFO:tasks.workunit.client.1.smithi138.stdout:4/960: symlink d7/df/d35/d4c/d59/d5d/d64/df6/l152 0 2024-05-27T09:29:52.306 INFO:tasks.workunit.client.1.smithi138.stdout:4/961: mkdir d7/df/d35/d4c/d59/d5d/d64/df6/d10d/d153 0 2024-05-27T09:29:52.308 INFO:tasks.workunit.client.1.smithi138.stdout:3/798: mkdir d0/d2/da/df/d10/d2a/d61/d73/deb/d109 0 2024-05-27T09:29:52.310 INFO:tasks.workunit.client.1.smithi138.stdout:4/962: link d7/df/d35/d4c/d59/f103 d7/df/d35/d4c/d59/d5d/d64/d14c/f154 0 2024-05-27T09:29:52.310 INFO:tasks.workunit.client.1.smithi138.stdout:4/963: fsync d7/df/d35/d4c/d59/d5d/d64/d6a/f140 0 2024-05-27T09:29:52.312 INFO:tasks.workunit.client.1.smithi138.stdout:4/964: truncate d7/dba/dcf/f34 510064 0 2024-05-27T09:29:52.313 INFO:tasks.workunit.client.1.smithi138.stdout:4/965: truncate d7/df/d35/f115 487381 0 2024-05-27T09:29:52.316 INFO:tasks.workunit.client.1.smithi138.stdout:3/799: rename d0/d2/da/df/d10/d32/d37/l3e to d0/d2/l10a 0 2024-05-27T09:29:52.316 INFO:tasks.workunit.client.1.smithi138.stdout:3/800: creat d0/d2/da/df/d10/d2a/d61/d73/deb/f10b x:0 0 0 2024-05-27T09:29:52.332 INFO:tasks.workunit.client.1.smithi138.stdout:1/868: dwrite d4/d1a/d2e/d32/f109 [0,4194304] 0 2024-05-27T09:29:52.341 INFO:tasks.workunit.client.1.smithi138.stdout:1/869: mknod d4/d48/d39/d5f/d4d/d2b/c11f 0 2024-05-27T09:29:52.344 INFO:tasks.workunit.client.1.smithi138.stdout:1/870: creat d4/d1a/d2e/d4a/d7e/d11d/f120 x:0 0 0 2024-05-27T09:29:52.347 INFO:tasks.workunit.client.1.smithi138.stdout:3/801: dread d0/d3f/f98 [0,4194304] 0 2024-05-27T09:29:52.352 INFO:tasks.workunit.client.1.smithi138.stdout:1/871: rmdir d4/d38/d97 39 2024-05-27T09:29:52.352 INFO:tasks.workunit.client.1.smithi138.stdout:3/802: rename d0/fd to d0/dda/f10c 0 2024-05-27T09:29:52.352 INFO:tasks.workunit.client.1.smithi138.stdout:1/872: chown d4/d3c/l5d 1628088241 1 2024-05-27T09:29:52.352 INFO:tasks.workunit.client.1.smithi138.stdout:1/873: write d4/d3c/f108 [862613,51142] 0 2024-05-27T09:29:52.355 INFO:tasks.workunit.client.1.smithi138.stdout:1/874: mkdir d4/dc7/d121 0 2024-05-27T09:29:52.358 INFO:tasks.workunit.client.1.smithi138.stdout:1/875: rename d4/c91 to d4/d1a/d2e/c122 0 2024-05-27T09:29:52.359 INFO:tasks.workunit.client.1.smithi138.stdout:1/876: rmdir d4/d48/d39 39 2024-05-27T09:29:52.359 INFO:tasks.workunit.client.1.smithi138.stdout:1/877: creat d4/d1a/deb/f123 x:0 0 0 2024-05-27T09:29:52.360 INFO:tasks.workunit.client.1.smithi138.stdout:3/803: dread d0/d2/da/fcf [0,4194304] 0 2024-05-27T09:29:52.360 INFO:tasks.workunit.client.1.smithi138.stdout:3/804: chown d0/d3f/d60/d93/de5/dca/cf4 19 1 2024-05-27T09:29:52.360 INFO:tasks.workunit.client.1.smithi138.stdout:3/805: chown d0/d2/da/df/d10/l44 1038 1 2024-05-27T09:29:52.360 INFO:tasks.workunit.client.1.smithi138.stdout:3/806: chown d0/d2/da/df/c19 369 1 2024-05-27T09:29:52.360 INFO:tasks.workunit.client.1.smithi138.stdout:3/807: write d0/d3f/d50/f63 [4234324,81750] 0 2024-05-27T09:29:52.361 INFO:tasks.workunit.client.1.smithi138.stdout:3/808: fsync d0/d3f/f98 0 2024-05-27T09:29:52.361 INFO:tasks.workunit.client.1.smithi138.stdout:3/809: truncate d0/d2/da/df/d10/d2a/db2/dd1/fae 927621 0 2024-05-27T09:29:52.361 INFO:tasks.workunit.client.1.smithi138.stdout:1/878: truncate d4/d48/d39/f85 356735 0 2024-05-27T09:29:52.365 INFO:tasks.workunit.client.1.smithi138.stdout:3/810: creat d0/d2/d105/f10d x:0 0 0 2024-05-27T09:29:52.378 INFO:tasks.workunit.client.1.smithi138.stdout:9/934: dwrite d6/da/d29/d8f/db7/dbd/d10e/fd6 [0,4194304] 0 2024-05-27T09:29:52.378 INFO:tasks.workunit.client.1.smithi138.stdout:9/935: stat d6/da/d29/d59 0 2024-05-27T09:29:52.397 INFO:tasks.workunit.client.1.smithi138.stdout:5/915: dwrite d3/d12/dcf/d11e/ff9 [0,4194304] 0 2024-05-27T09:29:52.397 INFO:tasks.workunit.client.1.smithi138.stdout:9/936: rmdir d6/db/de/d6b/d104/d10a 39 2024-05-27T09:29:52.397 INFO:tasks.workunit.client.1.smithi138.stdout:9/937: chown d6/f9 54471 1 2024-05-27T09:29:52.400 INFO:tasks.workunit.client.1.smithi138.stdout:5/916: rename d3/d12/d16/d1d/d22/f2a to d3/d12/dcf/d11e/dc6/f133 0 2024-05-27T09:29:52.402 INFO:tasks.workunit.client.1.smithi138.stdout:5/917: truncate d3/d12/d26/fe8 850586 0 2024-05-27T09:29:52.405 INFO:tasks.workunit.client.1.smithi138.stdout:5/918: getdents d3 0 2024-05-27T09:29:52.410 INFO:tasks.workunit.client.1.smithi138.stdout:4/966: dwrite d7/df/d35/d4c/d59/d5d/d64/df6/d10d/f12e [0,4194304] 0 2024-05-27T09:29:52.435 INFO:tasks.workunit.client.1.smithi138.stdout:3/811: dwrite d0/d2/da/df/d10/d2a/d61/d73/deb/ffe [0,4194304] 0 2024-05-27T09:29:52.435 INFO:tasks.workunit.client.1.smithi138.stdout:3/812: chown d0/d3f/d50/d9b/d9f/da3/dc6 55 1 2024-05-27T09:29:52.440 INFO:tasks.workunit.client.1.smithi138.stdout:1/879: dwrite d4/d48/d39/d5f/d63/f98 [0,4194304] 0 2024-05-27T09:29:52.440 INFO:tasks.workunit.client.1.smithi138.stdout:1/880: chown d4/d48/d39/d5f/d4d/f8b 15 1 2024-05-27T09:29:52.472 INFO:tasks.workunit.client.1.smithi138.stdout:3/813: dread d0/d2/da/df/d10/d2a/d48/f5f [0,4194304] 0 2024-05-27T09:29:52.476 INFO:tasks.workunit.client.1.smithi138.stdout:3/814: truncate d0/d2/da/df/d10/d2a/f4f 2654284 0 2024-05-27T09:29:52.480 INFO:tasks.workunit.client.1.smithi138.stdout:3/815: rename d0/d2/da/lf1 to d0/d2/da/df/d10/d32/d37/l10e 0 2024-05-27T09:29:52.480 INFO:tasks.workunit.client.1.smithi138.stdout:3/816: fdatasync d0/d2/da/df/d10/d2a/d61/de1/fc7 0 2024-05-27T09:29:52.480 INFO:tasks.workunit.client.1.smithi138.stdout:3/817: chown d0/d2/da/df/c2c 2172353 1 2024-05-27T09:29:52.491 INFO:tasks.workunit.client.1.smithi138.stdout:5/919: dwrite d3/d12/d16/d1d/d22/db9/fcc [0,4194304] 0 2024-05-27T09:29:52.492 INFO:tasks.workunit.client.1.smithi138.stdout:5/920: creat d3/d12/d16/d1d/d22/d2b/f134 x:0 0 0 2024-05-27T09:29:52.498 INFO:tasks.workunit.client.1.smithi138.stdout:5/921: rmdir d3/d12/d32/d87/d89/dc3 39 2024-05-27T09:29:52.501 INFO:tasks.workunit.client.1.smithi138.stdout:5/922: rename d3/d12/d16/d1d/d23/d3b/dec/fe4 to d3/f135 0 2024-05-27T09:29:52.504 INFO:tasks.workunit.client.1.smithi138.stdout:5/923: symlink d3/d12/d32/d60/d93/l136 0 2024-05-27T09:29:52.504 INFO:tasks.workunit.client.1.smithi138.stdout:9/938: dwrite d6/db/de/d18/ddb/fe4 [0,4194304] 0 2024-05-27T09:29:52.509 INFO:tasks.workunit.client.1.smithi138.stdout:5/924: symlink d3/d12/d16/d1d/d23/d3b/dec/l137 0 2024-05-27T09:29:52.510 INFO:tasks.workunit.client.1.smithi138.stdout:9/939: rmdir d6/da/d29/d59 39 2024-05-27T09:29:52.514 INFO:tasks.workunit.client.1.smithi138.stdout:5/925: mkdir d3/d12/d32/d87/d11b/d138 0 2024-05-27T09:29:52.514 INFO:tasks.workunit.client.1.smithi138.stdout:5/926: fsync d3/d12/d16/d1d/f97 0 2024-05-27T09:29:52.514 INFO:tasks.workunit.client.1.smithi138.stdout:5/927: read - d3/d12/d32/d87/d89/f101 zero size 2024-05-27T09:29:52.518 INFO:tasks.workunit.client.1.smithi138.stdout:5/928: unlink d3/d12/d16/d1d/d22/d2b/c37 0 2024-05-27T09:29:52.519 INFO:tasks.workunit.client.1.smithi138.stdout:5/929: write d3/d12/d16/d1d/d30/f131 [978802,70463] 0 2024-05-27T09:29:52.524 INFO:tasks.workunit.client.1.smithi138.stdout:1/881: dwrite d4/d48/f58 [0,4194304] 0 2024-05-27T09:29:52.529 INFO:tasks.workunit.client.1.smithi138.stdout:9/940: dread d6/da/d29/d8f/db7/dbd/fc5 [0,4194304] 0 2024-05-27T09:29:52.540 INFO:tasks.workunit.client.1.smithi138.stdout:1/882: mkdir d4/d48/d39/d5f/dd3/d10b/d124 0 2024-05-27T09:29:52.542 INFO:tasks.workunit.client.1.smithi138.stdout:5/930: rename d3/d12/d32/d87/dc0 to d3/d12/d16/d1d/d22/d2b/d9e/d139 0 2024-05-27T09:29:52.543 INFO:tasks.workunit.client.1.smithi138.stdout:0/803: sync 2024-05-27T09:29:52.543 INFO:tasks.workunit.client.1.smithi138.stdout:2/695: sync 2024-05-27T09:29:52.548 INFO:tasks.workunit.client.1.smithi138.stdout:1/883: unlink d4/d48/d39/d5f/d63/fd2 0 2024-05-27T09:29:52.550 INFO:tasks.workunit.client.1.smithi138.stdout:5/931: symlink d3/d12/d32/d6b/l13a 0 2024-05-27T09:29:52.550 INFO:tasks.workunit.client.1.smithi138.stdout:5/932: chown d3/d12/d32/d87/d89/dc3/l112 13956 1 2024-05-27T09:29:52.550 INFO:tasks.workunit.client.1.smithi138.stdout:5/933: fsync d3/d12/d16/d1d/d22/d2b/d9e/db6/f43 0 2024-05-27T09:29:52.555 INFO:tasks.workunit.client.1.smithi138.stdout:0/804: truncate da/d2a/d8e/d1f/d42/d4a/d53/fbf 2863866 0 2024-05-27T09:29:52.561 INFO:tasks.workunit.client.1.smithi138.stdout:1/884: creat d4/d1a/d2e/d32/d8c/dc6/dd8/f125 x:0 0 0 2024-05-27T09:29:52.572 INFO:tasks.workunit.client.1.smithi138.stdout:5/934: dread d3/d12/dcf/d11e/ff9 [0,4194304] 0 2024-05-27T09:29:52.578 INFO:tasks.workunit.client.1.smithi138.stdout:5/935: creat d3/d12/d16/d1d/d22/d2b/daf/f13b x:0 0 0 2024-05-27T09:29:52.578 INFO:tasks.workunit.client.1.smithi138.stdout:5/936: write d3/d12/d16/d1d/d22/d2b/daf/db2/fc4 [938360,30562] 0 2024-05-27T09:29:52.584 INFO:tasks.workunit.client.1.smithi138.stdout:5/937: symlink d3/d12/d16/l13c 0 2024-05-27T09:29:52.584 INFO:tasks.workunit.client.1.smithi138.stdout:5/938: truncate d3/d12/d16/d1d/d23/d68/fdd 1471394 0 2024-05-27T09:29:52.585 INFO:tasks.workunit.client.1.smithi138.stdout:5/939: truncate d3/d12/d26/ff4 1919127 0 2024-05-27T09:29:52.589 INFO:tasks.workunit.client.1.smithi138.stdout:3/818: dwrite d0/d2/da/df/d10/f5d [0,4194304] 0 2024-05-27T09:29:52.594 INFO:tasks.workunit.client.1.smithi138.stdout:3/819: link d0/d3f/d60/d93/de5/fc3 d0/d2/da/df/d10/d2a/d61/d73/deb/f10f 0 2024-05-27T09:29:52.596 INFO:tasks.workunit.client.1.smithi138.stdout:3/820: truncate d0/d3f/d50/f63 2454360 0 2024-05-27T09:29:52.598 INFO:tasks.workunit.client.1.smithi138.stdout:3/821: link d0/d2/da/df/d10/d2a/d61/de1/fc7 d0/d2/da/df/d10/d2a/d61/d73/deb/f110 0 2024-05-27T09:29:52.599 INFO:tasks.workunit.client.1.smithi138.stdout:3/822: read d0/d2/fc [2061624,71367] 0 2024-05-27T09:29:52.599 INFO:tasks.workunit.client.1.smithi138.stdout:3/823: chown d0/d2/da/df/d10/d32/d37/cd0 20 1 2024-05-27T09:29:52.599 INFO:tasks.workunit.client.1.smithi138.stdout:3/824: creat d0/d2/da/df/d10/d2a/d61/de1/f111 x:0 0 0 2024-05-27T09:29:52.599 INFO:tasks.workunit.client.1.smithi138.stdout:3/825: readlink d0/d2/da/df/d10/d2a/l82 0 2024-05-27T09:29:52.599 INFO:tasks.workunit.client.1.smithi138.stdout:3/826: read - d0/d3f/d50/ff5 zero size 2024-05-27T09:29:52.650 INFO:tasks.workunit.client.1.smithi138.stdout:9/941: dwrite d6/db/de/d6b/d104/d12a/f124 [0,4194304] 0 2024-05-27T09:29:52.655 INFO:tasks.workunit.client.1.smithi138.stdout:9/942: rename d6/da/d2f/c83 to d6/da/d29/d61/c143 0 2024-05-27T09:29:52.656 INFO:tasks.workunit.client.1.smithi138.stdout:5/940: dread d3/d12/d16/d1d/d22/d2b/d9e/db6/f43 [0,4194304] 0 2024-05-27T09:29:52.663 INFO:tasks.workunit.client.1.smithi138.stdout:5/941: symlink d3/d12/d32/d7e/l13d 0 2024-05-27T09:29:52.663 INFO:tasks.workunit.client.1.smithi138.stdout:2/696: dwrite d1/d15/d2d/d2e/f50 [4194304,4194304] 0 2024-05-27T09:29:52.667 INFO:tasks.workunit.client.1.smithi138.stdout:9/943: link d6/ccf d6/db/de/c144 0 2024-05-27T09:29:52.667 INFO:tasks.workunit.client.1.smithi138.stdout:2/697: creat d1/d15/d2d/d17/d25/d4d/db6/feb x:0 0 0 2024-05-27T09:29:52.669 INFO:tasks.workunit.client.1.smithi138.stdout:2/698: mkdir d1/d15/dd6/dec 0 2024-05-27T09:29:52.669 INFO:tasks.workunit.client.1.smithi138.stdout:2/699: creat d1/d15/d2d/d17/d25/d4d/db6/fed x:0 0 0 2024-05-27T09:29:52.672 INFO:tasks.workunit.client.1.smithi138.stdout:2/700: mknod d1/d15/d2d/d2e/cee 0 2024-05-27T09:29:52.676 INFO:tasks.workunit.client.1.smithi138.stdout:3/827: dwrite d0/d2/da/df/fe2 [0,4194304] 0 2024-05-27T09:29:52.678 INFO:tasks.workunit.client.1.smithi138.stdout:3/828: link d0/d3f/d50/d9b/lfc d0/d3f/d60/dcb/l112 0 2024-05-27T09:29:52.678 INFO:tasks.workunit.client.1.smithi138.stdout:3/829: stat d0/d3f/d60/daf 0 2024-05-27T09:29:52.679 INFO:tasks.workunit.client.1.smithi138.stdout:3/830: getdents d0 0 2024-05-27T09:29:52.680 INFO:tasks.workunit.client.1.smithi138.stdout:9/944: dread d6/db/dc/f12 [0,4194304] 0 2024-05-27T09:29:52.695 INFO:tasks.workunit.client.1.smithi138.stdout:5/942: write d3/d12/d32/f4f [2506072,35660] 0 2024-05-27T09:29:52.695 INFO:tasks.workunit.client.1.smithi138.stdout:5/943: fsync d3/d75/f12b 0 2024-05-27T09:29:52.695 INFO:tasks.workunit.client.1.smithi138.stdout:3/831: unlink d0/d2/da/f8a 0 2024-05-27T09:29:52.695 INFO:tasks.workunit.client.1.smithi138.stdout:0/805: dwrite da/f10 [0,4194304] 0 2024-05-27T09:29:52.696 INFO:tasks.workunit.client.1.smithi138.stdout:9/945: truncate d6/da/d29/d59/dee/fb1 262082 0 2024-05-27T09:29:52.698 INFO:tasks.workunit.client.1.smithi138.stdout:5/944: rmdir d3/d12/d16/d1d/d22/d2b/d9e/d139 39 2024-05-27T09:29:52.698 INFO:tasks.workunit.client.1.smithi138.stdout:3/832: rmdir d0/d2/da/df/d10 39 2024-05-27T09:29:52.698 INFO:tasks.workunit.client.1.smithi138.stdout:0/806: creat da/d2a/d8e/d1f/d42/d4a/d53/f101 x:0 0 0 2024-05-27T09:29:52.707 INFO:tasks.workunit.client.1.smithi138.stdout:5/945: symlink d3/d12/d26/l13e 0 2024-05-27T09:29:52.708 INFO:tasks.workunit.client.1.smithi138.stdout:0/807: creat da/d2a/dc9/f102 x:0 0 0 2024-05-27T09:29:52.713 INFO:tasks.workunit.client.1.smithi138.stdout:3/833: dread d0/d2/f45 [0,4194304] 0 2024-05-27T09:29:52.714 INFO:tasks.workunit.client.1.smithi138.stdout:3/834: chown d0/l5 28873677 1 2024-05-27T09:29:52.714 INFO:tasks.workunit.client.1.smithi138.stdout:3/835: stat d0/d3f/d50/f7c 0 2024-05-27T09:29:52.714 INFO:tasks.workunit.client.1.smithi138.stdout:3/836: stat d0/d2/da/df/d10/d2a/d61/d73/l8b 0 2024-05-27T09:29:52.715 INFO:tasks.workunit.client.1.smithi138.stdout:3/837: truncate d0/f12 366598 0 2024-05-27T09:29:52.717 INFO:tasks.workunit.client.1.smithi138.stdout:3/838: mkdir d0/d80/dab/d113 0 2024-05-27T09:29:52.734 INFO:tasks.workunit.client.1.smithi138.stdout:5/946: dread d3/d12/d26/fa6 [0,4194304] 0 2024-05-27T09:29:52.734 INFO:tasks.workunit.client.1.smithi138.stdout:5/947: chown d3/c74 485297 1 2024-05-27T09:29:52.753 INFO:tasks.workunit.client.1.smithi138.stdout:2/701: dwrite d1/d15/f48 [4194304,4194304] 0 2024-05-27T09:29:52.753 INFO:tasks.workunit.client.1.smithi138.stdout:2/702: fsync d1/d15/d2d/f39 0 2024-05-27T09:29:52.764 INFO:tasks.workunit.client.1.smithi138.stdout:5/948: write d3/d12/d16/d1d/d23/fba [357502,103651] 0 2024-05-27T09:29:52.764 INFO:tasks.workunit.client.1.smithi138.stdout:5/949: chown d3/d12/d16/d1d/d22/d2b/daf/lbc 0 1 2024-05-27T09:29:52.765 INFO:tasks.workunit.client.1.smithi138.stdout:5/950: write d3/d12/d16/d1d/d22/d2b/d9e/db6/f43 [4952268,109973] 0 2024-05-27T09:29:52.766 INFO:tasks.workunit.client.1.smithi138.stdout:0/808: dwrite da/d2a/d8e/d6a/daf/dc4/fc8 [0,4194304] 0 2024-05-27T09:29:52.768 INFO:tasks.workunit.client.1.smithi138.stdout:0/809: mkdir da/d2a/d8e/d43/d74/d97/df1/d103 0 2024-05-27T09:29:52.770 INFO:tasks.workunit.client.1.smithi138.stdout:0/810: creat da/d2a/d8e/d1f/d60/dbd/f104 x:0 0 0 2024-05-27T09:29:52.773 INFO:tasks.workunit.client.1.smithi138.stdout:0/811: rename da/d2a/d8e/d1f/l83 to da/d2a/d3f/l105 0 2024-05-27T09:29:52.774 INFO:tasks.workunit.client.1.smithi138.stdout:9/946: dwrite d6/f2e [4194304,4194304] 0 2024-05-27T09:29:52.774 INFO:tasks.workunit.client.1.smithi138.stdout:9/947: read - d6/db/de/d18/ddb/f110 zero size 2024-05-27T09:29:52.776 INFO:tasks.workunit.client.1.smithi138.stdout:9/948: truncate d6/db/de/d6b/d104/d12a/d102/f11c 3640278 0 2024-05-27T09:29:52.779 INFO:tasks.workunit.client.1.smithi138.stdout:9/949: write d6/da/d29/d59/f90 [1988070,73326] 0 2024-05-27T09:29:52.779 INFO:tasks.workunit.client.1.smithi138.stdout:2/703: dread d1/d15/d2d/d17/d25/d9d/fb3 [0,4194304] 0 2024-05-27T09:29:52.779 INFO:tasks.workunit.client.1.smithi138.stdout:0/812: rename da/d2a/d8e/d1f/f7b to da/d2a/d55/dc7/f106 0 2024-05-27T09:29:52.781 INFO:tasks.workunit.client.1.smithi138.stdout:2/704: getdents d1/d15/d2d/d17 0 2024-05-27T09:29:52.786 INFO:tasks.workunit.client.1.smithi138.stdout:5/951: dread d3/d12/d32/d60/f82 [0,4194304] 0 2024-05-27T09:29:52.786 INFO:tasks.workunit.client.1.smithi138.stdout:5/952: truncate d3/d12/d26/d53/f114 1361611 0 2024-05-27T09:29:52.786 INFO:tasks.workunit.client.1.smithi138.stdout:5/953: dread - d3/d12/d16/d1d/f73 zero size 2024-05-27T09:29:52.786 INFO:tasks.workunit.client.1.smithi138.stdout:5/954: chown d3/d12/d32/ca0 3624 1 2024-05-27T09:29:52.786 INFO:tasks.workunit.client.1.smithi138.stdout:3/839: dwrite d0/d2/da/df/d10/f38 [0,4194304] 0 2024-05-27T09:29:52.786 INFO:tasks.workunit.client.1.smithi138.stdout:3/840: rmdir d0/d3f/d60/d93/de5 39 2024-05-27T09:29:52.787 INFO:tasks.workunit.client.1.smithi138.stdout:3/841: chown d0/d3f/f98 0 1 2024-05-27T09:29:52.787 INFO:tasks.workunit.client.1.smithi138.stdout:0/813: truncate da/d2a/d8e/dc0/fce 993190 0 2024-05-27T09:29:52.787 INFO:tasks.workunit.client.1.smithi138.stdout:0/814: stat da/d2a/d8e/d1f/d42/d63/dea/ced 0 2024-05-27T09:29:52.787 INFO:tasks.workunit.client.1.smithi138.stdout:0/815: readlink da/d2a/d2b/l36 0 2024-05-27T09:29:52.794 INFO:tasks.workunit.client.1.smithi138.stdout:5/955: creat d3/d12/d32/f13f x:0 0 0 2024-05-27T09:29:52.815 INFO:tasks.workunit.client.1.smithi138.stdout:0/816: creat da/d2a/d8e/dda/ddf/f107 x:0 0 0 2024-05-27T09:29:52.818 INFO:tasks.workunit.client.1.smithi138.stdout:5/956: mknod d3/d12/d32/d60/de6/c140 0 2024-05-27T09:29:52.823 INFO:tasks.workunit.client.1.smithi138.stdout:5/957: unlink d3/d12/d32/d60/de6/lff 0 2024-05-27T09:29:52.854 INFO:tasks.workunit.client.1.smithi138.stdout:0/817: write da/f10 [413522,37104] 0 2024-05-27T09:29:52.866 INFO:tasks.workunit.client.1.smithi138.stdout:2/705: dwrite d1/d15/d2d/d17/d25/f3a [0,4194304] 0 2024-05-27T09:29:52.866 INFO:tasks.workunit.client.1.smithi138.stdout:2/706: fsync d1/d15/d2d/d17/d25/d4d/d4e/ddf/f89 0 2024-05-27T09:29:52.869 INFO:tasks.workunit.client.1.smithi138.stdout:2/707: mkdir d1/d15/d2d/d17/def 0 2024-05-27T09:29:52.869 INFO:tasks.workunit.client.1.smithi138.stdout:2/708: stat d1/d15/d2d/d17/d25 0 2024-05-27T09:29:52.871 INFO:tasks.workunit.client.1.smithi138.stdout:2/709: rename d1/d15/d2d/d17/d25/d9d/d4b/fc5 to d1/d15/d2d/d17/d25/d4d/ff0 0 2024-05-27T09:29:52.871 INFO:tasks.workunit.client.1.smithi138.stdout:3/842: dwrite d0/d3f/d50/ff5 [0,4194304] 0 2024-05-27T09:29:52.875 INFO:tasks.workunit.client.1.smithi138.stdout:3/843: getdents d0/d2/da/df/d10/d32/d37 0 2024-05-27T09:29:52.877 INFO:tasks.workunit.client.1.smithi138.stdout:9/950: dwrite d6/da/d29/d61/f142 [0,4194304] 0 2024-05-27T09:29:52.877 INFO:tasks.workunit.client.1.smithi138.stdout:9/951: write d6/da/d29/d61/f11a [4493427,83789] 0 2024-05-27T09:29:52.879 INFO:tasks.workunit.client.1.smithi138.stdout:3/844: creat d0/d2/da/df/d10/d32/d37/f114 x:0 0 0 2024-05-27T09:29:52.883 INFO:tasks.workunit.client.1.smithi138.stdout:9/952: creat d6/dc8/f145 x:0 0 0 2024-05-27T09:29:52.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:52 smithi138 ceph-mon[34885]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-05-27T09:29:52.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:52 smithi138 ceph-mon[34885]: Updating smithi119:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T09:29:52.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:52 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:52.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:52 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:52.902 INFO:tasks.workunit.client.1.smithi138.stdout:0/818: write da/d2a/d3f/f59 [2439639,50460] 0 2024-05-27T09:29:52.905 INFO:tasks.workunit.client.1.smithi138.stdout:3/845: dread d0/d2/da/df/d10/d2a/f42 [0,4194304] 0 2024-05-27T09:29:52.912 INFO:tasks.workunit.client.1.smithi138.stdout:5/958: dwrite d3/d12/f31 [0,4194304] 0 2024-05-27T09:29:52.912 INFO:tasks.workunit.client.1.smithi138.stdout:2/710: rmdir d1/d15/d2d/d17/d25/d9d/d4b 39 2024-05-27T09:29:52.913 INFO:tasks.workunit.client.1.smithi138.stdout:2/711: creat d1/d10/d52/db9/ff1 x:0 0 0 2024-05-27T09:29:52.922 INFO:tasks.workunit.client.1.smithi138.stdout:5/959: symlink d3/dab/l141 0 2024-05-27T09:29:52.922 INFO:tasks.workunit.client.1.smithi138.stdout:5/960: stat d3/d12/d16/d1d/d23/d3b/f95 0 2024-05-27T09:29:52.923 INFO:tasks.workunit.client.1.smithi138.stdout:3/846: read d0/d2/da/df/d10/d2a/d61/de1/f101 [4001361,75903] 0 2024-05-27T09:29:52.925 INFO:tasks.workunit.client.1.smithi138.stdout:3/847: rename d0/d2/da/df/d10/d32/d37/f114 to d0/dda/f115 0 2024-05-27T09:29:52.925 INFO:tasks.workunit.client.1.smithi138.stdout:3/848: read - d0/d3f/d50/d9b/d9f/fc9 zero size 2024-05-27T09:29:52.927 INFO:tasks.workunit.client.1.smithi138.stdout:5/961: creat d3/d12/d16/d1d/d23/d3b/dec/f142 x:0 0 0 2024-05-27T09:29:52.944 INFO:tasks.workunit.client.1.smithi138.stdout:5/962: dread d3/d12/d16/d1d/d30/f9c [0,4194304] 0 2024-05-27T09:29:52.944 INFO:tasks.workunit.client.1.smithi138.stdout:5/963: chown d3/d12/d16/d1d/d22/db9 3 1 2024-05-27T09:29:52.946 INFO:tasks.workunit.client.1.smithi138.stdout:5/964: truncate d3/d12/d32/d60/de6/d111/ffa 1321074 0 2024-05-27T09:29:52.947 INFO:tasks.workunit.client.1.smithi138.stdout:5/965: mknod d3/d12/d16/d1d/d12e/c143 0 2024-05-27T09:29:52.947 INFO:tasks.workunit.client.1.smithi138.stdout:5/966: symlink d3/d12/d32/d60/d93/dd0/l144 0 2024-05-27T09:29:52.961 INFO:tasks.workunit.client.1.smithi138.stdout:5/967: write d3/d12/d16/f2e [7932501,107283] 0 2024-05-27T09:29:52.963 INFO:tasks.workunit.client.1.smithi138.stdout:5/968: symlink d3/d12/d16/d1d/l145 0 2024-05-27T09:29:52.967 INFO:tasks.workunit.client.1.smithi138.stdout:5/969: rmdir d3/d12/d16/d1d/d23/d68 39 2024-05-27T09:29:52.967 INFO:tasks.workunit.client.1.smithi138.stdout:5/970: stat d3/d12/d16/d1d/d23/f62 0 2024-05-27T09:29:52.974 INFO:tasks.workunit.client.1.smithi138.stdout:5/971: truncate d3/d12/d32/d60/f8e 431701 0 2024-05-27T09:29:52.977 INFO:tasks.workunit.client.1.smithi138.stdout:2/712: read d1/d15/d2d/d17/d25/d9d/fb3 [239129,23182] 0 2024-05-27T09:29:52.983 INFO:tasks.workunit.client.1.smithi138.stdout:2/713: getdents d1 0 2024-05-27T09:29:52.983 INFO:tasks.workunit.client.1.smithi138.stdout:2/714: stat d1/d15/d2d/f8e 0 2024-05-27T09:29:52.983 INFO:tasks.workunit.client.1.smithi138.stdout:2/715: dread - d1/d15/d2d/d17/f9b zero size 2024-05-27T09:29:52.985 INFO:tasks.workunit.client.1.smithi138.stdout:2/716: mkdir d1/d15/d2d/d17/dac/df2 0 2024-05-27T09:29:52.993 INFO:tasks.workunit.client.1.smithi138.stdout:0/819: dwrite da/d2a/d8e/d6a/daf/dc4/fc8 [0,4194304] 0 2024-05-27T09:29:52.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:52 smithi119 ceph-mon[29082]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-05-27T09:29:52.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:52 smithi119 ceph-mon[29082]: Updating smithi119:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T09:29:52.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:52 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:52.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:52 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:53.005 INFO:tasks.workunit.client.1.smithi138.stdout:0/820: symlink da/d2a/dc9/dca/l108 0 2024-05-27T09:29:53.005 INFO:tasks.workunit.client.1.smithi138.stdout:0/821: fsync da/d2a/d2b/f33 0 2024-05-27T09:29:53.007 INFO:tasks.workunit.client.1.smithi138.stdout:4/967: sync 2024-05-27T09:29:53.013 INFO:tasks.workunit.client.1.smithi138.stdout:0/822: rmdir da/d2a/d8e/d43/d74/d97/df1/d103 0 2024-05-27T09:29:53.015 INFO:tasks.workunit.client.1.smithi138.stdout:5/972: dwrite d3/d12/d16/d1d/d22/d2b/f134 [0,4194304] 0 2024-05-27T09:29:53.015 INFO:tasks.workunit.client.1.smithi138.stdout:5/973: stat d3/d12/d32/d60/cf7 0 2024-05-27T09:29:53.022 INFO:tasks.workunit.client.1.smithi138.stdout:4/968: mkdir d7/df/d35/d4c/d59/d5d/d64/d14c/d155 0 2024-05-27T09:29:53.022 INFO:tasks.workunit.client.1.smithi138.stdout:4/969: chown f3 1 1 2024-05-27T09:29:53.031 INFO:tasks.workunit.client.1.smithi138.stdout:5/974: mkdir d3/d12/d16/d1d/d22/d2b/daf/d146 0 2024-05-27T09:29:53.031 INFO:tasks.workunit.client.1.smithi138.stdout:1/885: sync 2024-05-27T09:29:53.031 INFO:tasks.workunit.client.1.smithi138.stdout:1/886: fdatasync d4/d48/d39/d5f/d4d/d2b/fdd 0 2024-05-27T09:29:53.035 INFO:tasks.workunit.client.1.smithi138.stdout:5/975: creat d3/d12/d16/d1d/d22/d2b/daf/d146/f147 x:0 0 0 2024-05-27T09:29:53.043 INFO:tasks.workunit.client.1.smithi138.stdout:1/887: rename d4/d48/d39/d5f/l87 to d4/daa/l126 0 2024-05-27T09:29:53.050 INFO:tasks.workunit.client.1.smithi138.stdout:1/888: mkdir d4/d1a/d2e/d4a/d7e/db7/dfb/d127 0 2024-05-27T09:29:53.050 INFO:tasks.workunit.client.1.smithi138.stdout:5/976: dread d3/d12/d16/d1d/d22/d2b/daf/db2/fc4 [0,4194304] 0 2024-05-27T09:29:53.050 INFO:tasks.workunit.client.1.smithi138.stdout:5/977: chown d3/fd 0 1 2024-05-27T09:29:53.051 INFO:tasks.workunit.client.1.smithi138.stdout:5/978: chown d3/d12/d26/d53/f114 4538 1 2024-05-27T09:29:53.051 INFO:tasks.workunit.client.1.smithi138.stdout:5/979: creat d3/d12/f148 x:0 0 0 2024-05-27T09:29:53.051 INFO:tasks.workunit.client.1.smithi138.stdout:5/980: dread - d3/d12/d16/d1d/d22/db9/f12f zero size 2024-05-27T09:29:53.051 INFO:tasks.workunit.client.1.smithi138.stdout:5/981: dread - d3/d12/d32/d60/de6/df2/f130 zero size 2024-05-27T09:29:53.051 INFO:tasks.workunit.client.1.smithi138.stdout:5/982: creat d3/d12/d16/d1d/d23/d3b/dec/d100/f149 x:0 0 0 2024-05-27T09:29:53.051 INFO:tasks.workunit.client.1.smithi138.stdout:5/983: chown d3/d12/d32/d7e/l13d 23527 1 2024-05-27T09:29:53.052 INFO:tasks.workunit.client.1.smithi138.stdout:2/717: dwrite d1/d15/d2d/d17/d25/d4d/d6d/d87/fbf [0,4194304] 0 2024-05-27T09:29:53.052 INFO:tasks.workunit.client.1.smithi138.stdout:2/718: chown d1/d10/d52/db9 1894 1 2024-05-27T09:29:53.058 INFO:tasks.workunit.client.1.smithi138.stdout:2/719: mkdir d1/d15/d2d/d17/df3 0 2024-05-27T09:29:53.058 INFO:tasks.workunit.client.1.smithi138.stdout:1/889: mknod d4/d1a/d2e/c128 0 2024-05-27T09:29:53.058 INFO:tasks.workunit.client.1.smithi138.stdout:1/890: creat d4/d1a/d2e/d32/f129 x:0 0 0 2024-05-27T09:29:53.058 INFO:tasks.workunit.client.1.smithi138.stdout:1/891: write d4/d1a/d2e/d32/d8c/dc6/f117 [1566463,74634] 0 2024-05-27T09:29:53.058 INFO:tasks.workunit.client.1.smithi138.stdout:1/892: creat d4/d1a/d2e/d32/f12a x:0 0 0 2024-05-27T09:29:53.058 INFO:tasks.workunit.client.1.smithi138.stdout:1/893: chown d4/d1a/c8d 194647 1 2024-05-27T09:29:53.058 INFO:tasks.workunit.client.1.smithi138.stdout:1/894: readlink d4/d48/l81 0 2024-05-27T09:29:53.061 INFO:tasks.workunit.client.1.smithi138.stdout:0/823: dwrite da/d2a/d8e/d1f/fde [0,4194304] 0 2024-05-27T09:29:53.065 INFO:tasks.workunit.client.1.smithi138.stdout:5/984: creat d3/d12/d16/d1d/d30/f14a x:0 0 0 2024-05-27T09:29:53.065 INFO:tasks.workunit.client.1.smithi138.stdout:5/985: chown d3/d12/d16/d1d/d23/d3b/dec/d100/f149 11 1 2024-05-27T09:29:53.068 INFO:tasks.workunit.client.1.smithi138.stdout:1/895: creat d4/d48/d39/d5f/f12b x:0 0 0 2024-05-27T09:29:53.068 INFO:tasks.workunit.client.1.smithi138.stdout:1/896: write d4/d48/d39/d5f/d4d/fcb [178287,25356] 0 2024-05-27T09:29:53.070 INFO:tasks.workunit.client.1.smithi138.stdout:4/970: dwrite d7/d6e/fc2 [0,4194304] 0 2024-05-27T09:29:53.070 INFO:tasks.workunit.client.1.smithi138.stdout:4/971: stat d7/df/d35/d4c/d59/d5d/d64/d14c/d155 0 2024-05-27T09:29:53.071 INFO:tasks.workunit.client.1.smithi138.stdout:4/972: write d7/df/d35/d4c/dc0/dfb/f12c [172284,46555] 0 2024-05-27T09:29:53.072 INFO:tasks.workunit.client.1.smithi138.stdout:0/824: rmdir da/d2a/d8e/d1f/d60 39 2024-05-27T09:29:53.072 INFO:tasks.workunit.client.1.smithi138.stdout:0/825: write da/d2a/d8e/d1f/ff5 [465970,24215] 0 2024-05-27T09:29:53.073 INFO:tasks.workunit.client.1.smithi138.stdout:5/986: mkdir d3/d12/d16/d1d/d12e/d14b 0 2024-05-27T09:29:53.073 INFO:tasks.workunit.client.1.smithi138.stdout:5/987: creat d3/d12/d16/d1d/f14c x:0 0 0 2024-05-27T09:29:53.077 INFO:tasks.workunit.client.1.smithi138.stdout:4/973: mknod d7/dba/dcf/d3a/c156 0 2024-05-27T09:29:53.078 INFO:tasks.workunit.client.1.smithi138.stdout:4/974: read d7/fdc [221988,26396] 0 2024-05-27T09:29:53.079 INFO:tasks.workunit.client.1.smithi138.stdout:0/826: symlink da/d2a/d8e/d1f/d42/l109 0 2024-05-27T09:29:53.086 INFO:tasks.workunit.client.1.smithi138.stdout:0/827: link da/fb8 da/d2a/dc9/f10a 0 2024-05-27T09:29:53.087 INFO:tasks.workunit.client.1.smithi138.stdout:1/897: dread d4/d1a/d2e/d4a/d7e/dc0/fd0 [0,4194304] 0 2024-05-27T09:29:53.087 INFO:tasks.workunit.client.1.smithi138.stdout:1/898: chown d4/d1a/d2e/d4a/d7e/db7/dfb 58843 1 2024-05-27T09:29:53.089 INFO:tasks.workunit.client.1.smithi138.stdout:0/828: creat da/d2a/d8e/d1f/d4f/f10b x:0 0 0 2024-05-27T09:29:53.092 INFO:tasks.workunit.client.1.smithi138.stdout:1/899: rmdir d4/d1a/d2e/d4a/d7e/d11d 39 2024-05-27T09:29:53.092 INFO:tasks.workunit.client.1.smithi138.stdout:1/900: truncate d4/d1a/d2e/d4a/d7e/dc0/f119 859942 0 2024-05-27T09:29:53.092 INFO:tasks.workunit.client.1.smithi138.stdout:1/901: chown d4/d1a/d2e/d4a/d7e/db7 39 1 2024-05-27T09:29:53.108 INFO:tasks.workunit.client.1.smithi138.stdout:0/829: dread da/f66 [0,4194304] 0 2024-05-27T09:29:53.115 INFO:tasks.workunit.client.1.smithi138.stdout:1/902: read d4/f78 [4583935,105573] 0 2024-05-27T09:29:53.117 INFO:tasks.workunit.client.1.smithi138.stdout:1/903: getdents d4/d48/d39/d5f/d86 0 2024-05-27T09:29:53.150 INFO:tasks.workunit.client.1.smithi138.stdout:5/988: dwrite d3/d12/d32/d87/f8f [0,4194304] 0 2024-05-27T09:29:53.160 INFO:tasks.workunit.client.1.smithi138.stdout:5/989: dread d3/d12/d16/fa9 [0,4194304] 0 2024-05-27T09:29:53.168 INFO:tasks.workunit.client.1.smithi138.stdout:1/904: dwrite d4/d1a/d2e/d32/d8c/dc6/f61 [0,4194304] 0 2024-05-27T09:29:53.170 INFO:tasks.workunit.client.1.smithi138.stdout:1/905: mknod d4/d48/d39/d5f/dd3/d10b/c12c 0 2024-05-27T09:29:53.177 INFO:tasks.workunit.client.1.smithi138.stdout:5/990: dread d3/f19 [0,4194304] 0 2024-05-27T09:29:53.178 INFO:tasks.workunit.client.1.smithi138.stdout:5/991: creat d3/d12/d16/d1d/d22/d2b/f14d x:0 0 0 2024-05-27T09:29:53.217 INFO:tasks.workunit.client.1.smithi138.stdout:5/992: dwrite d3/d12/d26/d53/f85 [0,4194304] 0 2024-05-27T09:29:53.265 INFO:tasks.workunit.client.1.smithi138.stdout:1/906: creat d4/d38/d97/dfe/f12d x:0 0 0 2024-05-27T09:29:53.274 INFO:tasks.workunit.client.1.smithi138.stdout:5/993: mknod d3/d12/d16/d1d/d23/d3b/dec/d100/c14e 0 2024-05-27T09:29:53.282 INFO:tasks.workunit.client.1.smithi138.stdout:5/994: mknod d3/d12/d32/d60/d93/c14f 0 2024-05-27T09:29:53.288 INFO:tasks.workunit.client.1.smithi138.stdout:1/907: dread d4/d38/f44 [0,4194304] 0 2024-05-27T09:29:53.288 INFO:tasks.workunit.client.1.smithi138.stdout:1/908: write d4/d88/ff2 [1311199,89889] 0 2024-05-27T09:29:53.294 INFO:tasks.workunit.client.1.smithi138.stdout:1/909: mknod d4/d1a/d2e/d32/d8c/dc6/c12e 0 2024-05-27T09:29:53.322 INFO:tasks.workunit.client.1.smithi138.stdout:9/953: sync 2024-05-27T09:29:53.322 INFO:tasks.workunit.client.1.smithi138.stdout:9/954: chown d6/da/d29/d8f/db7/dbd/da2/le6 26 1 2024-05-27T09:29:53.339 INFO:tasks.workunit.client.1.smithi138.stdout:9/955: rmdir d6 39 2024-05-27T09:29:53.339 INFO:tasks.workunit.client.1.smithi138.stdout:9/956: dread - d6/da/d29/d8f/da8/fdd zero size 2024-05-27T09:29:53.341 INFO:tasks.workunit.client.1.smithi138.stdout:9/957: rename d6/db/de/d18/d24 to d6/da/d29/d59/d146 0 2024-05-27T09:29:53.342 INFO:tasks.workunit.client.1.smithi138.stdout:9/958: mkdir d6/db/d147 0 2024-05-27T09:29:53.344 INFO:tasks.workunit.client.1.smithi138.stdout:9/959: unlink d6/da/d2f/l45 0 2024-05-27T09:29:53.344 INFO:tasks.workunit.client.1.smithi138.stdout:9/960: chown d6/da/l36 18 1 2024-05-27T09:29:53.346 INFO:tasks.workunit.client.1.smithi138.stdout:5/995: dwrite d3/d12/d32/d60/de6/df2/f10e [0,4194304] 0 2024-05-27T09:29:53.346 INFO:tasks.workunit.client.1.smithi138.stdout:9/961: link d6/da/d2f/le5 d6/da/d29/d61/l148 0 2024-05-27T09:29:53.348 INFO:tasks.workunit.client.1.smithi138.stdout:5/996: mknod d3/d12/d32/d60/de6/d111/c150 0 2024-05-27T09:29:53.348 INFO:tasks.workunit.client.1.smithi138.stdout:9/962: creat d6/da/d29/f149 x:0 0 0 2024-05-27T09:29:53.348 INFO:tasks.workunit.client.1.smithi138.stdout:9/963: chown d6/cfe 3711751 1 2024-05-27T09:29:53.350 INFO:tasks.workunit.client.1.smithi138.stdout:9/964: chown d6/da/d29/d59/d146/c12b 238747422 1 2024-05-27T09:29:53.350 INFO:tasks.workunit.client.1.smithi138.stdout:9/965: creat d6/da/d29/d8f/f14a x:0 0 0 2024-05-27T09:29:53.350 INFO:tasks.workunit.client.1.smithi138.stdout:9/966: fsync d6/db/de/d6b/ff5 0 2024-05-27T09:29:53.352 INFO:tasks.workunit.client.1.smithi138.stdout:9/967: mknod d6/db/c14b 0 2024-05-27T09:29:53.365 INFO:tasks.workunit.client.1.smithi138.stdout:5/997: dread d3/d12/d26/f57 [0,4194304] 0 2024-05-27T09:29:53.367 INFO:tasks.workunit.client.1.smithi138.stdout:5/998: symlink d3/d12/d16/d1d/d22/l151 0 2024-05-27T09:29:53.367 INFO:tasks.workunit.client.1.smithi138.stdout:5/999: stat d3/d12/d26/l48 0 2024-05-27T09:29:53.370 INFO:tasks.workunit.client.1.smithi138.stdout:1/910: dwrite d4/d1a/d2e/d4a/d7e/dc0/fd0 [0,4194304] 0 2024-05-27T09:29:53.370 INFO:tasks.workunit.client.1.smithi138.stdout:1/911: creat d4/d48/d39/d5f/f12f x:0 0 0 2024-05-27T09:29:53.375 INFO:tasks.workunit.client.1.smithi138.stdout:9/968: write d6/da/d29/d8f/db7/fb8 [4049955,116184] 0 2024-05-27T09:29:53.376 INFO:tasks.workunit.client.1.smithi138.stdout:1/912: rename d4/d48/d39/d5f/f2f to d4/d48/d39/d5f/d4d/d2b/f130 0 2024-05-27T09:29:53.378 INFO:tasks.workunit.client.1.smithi138.stdout:1/913: rename d4/d88/d9f/lc1 to d4/d48/d39/d5f/dd3/d10b/d124/l131 0 2024-05-27T09:29:53.379 INFO:tasks.workunit.client.1.smithi138.stdout:9/969: link d6/db/dc/dca/fec d6/db/de/d85/d119/f14c 0 2024-05-27T09:29:53.381 INFO:tasks.workunit.client.1.smithi138.stdout:1/914: link d4/d48/l36 d4/d1a/d2e/d32/l132 0 2024-05-27T09:29:53.381 INFO:tasks.workunit.client.1.smithi138.stdout:1/915: write d4/d3c/fa0 [1439860,59289] 0 2024-05-27T09:29:53.385 INFO:tasks.workunit.client.1.smithi138.stdout:9/970: mkdir d6/da/d29/d59/dee/d55/d14d 0 2024-05-27T09:29:53.387 INFO:tasks.workunit.client.1.smithi138.stdout:9/971: rmdir d6/da/d29/d59/d146 39 2024-05-27T09:29:53.388 INFO:tasks.workunit.client.1.smithi138.stdout:9/972: mknod d6/da/d29/d59/d146/c14e 0 2024-05-27T09:29:53.388 INFO:tasks.workunit.client.1.smithi138.stdout:9/973: write d6/da/d29/d59/d146/f10d [505987,63260] 0 2024-05-27T09:29:53.390 INFO:tasks.workunit.client.1.smithi138.stdout:9/974: mkdir d6/da/d29/d59/dee/d55/d75/d8c/d14f 0 2024-05-27T09:29:53.391 INFO:tasks.workunit.client.1.smithi138.stdout:9/975: chown d6/da/d29/d8f/db7/dbd/d4c 334071 1 2024-05-27T09:29:53.391 INFO:tasks.workunit.client.1.smithi138.stdout:9/976: truncate d6/db/dc/dca/fec 884524 0 2024-05-27T09:29:53.415 INFO:tasks.workunit.client.1.smithi138.stdout:3/849: sync 2024-05-27T09:29:53.416 INFO:tasks.workunit.client.1.smithi138.stdout:3/850: getdents d0 0 2024-05-27T09:29:53.424 INFO:tasks.workunit.client.1.smithi138.stdout:9/977: dwrite d6/da/d29/d59/dee/d55/d75/f99 [0,4194304] 0 2024-05-27T09:29:53.427 INFO:tasks.workunit.client.1.smithi138.stdout:9/978: creat d6/db/de/d6b/f150 x:0 0 0 2024-05-27T09:29:53.427 INFO:tasks.workunit.client.1.smithi138.stdout:9/979: fsync d6/db/de/d6b/d104/d12a/d102/f13d 0 2024-05-27T09:29:53.435 INFO:tasks.workunit.client.1.smithi138.stdout:9/980: write d6/db/de/d18/ddb/f130 [1204944,114960] 0 2024-05-27T09:29:53.452 INFO:tasks.workunit.client.1.smithi138.stdout:9/981: dread d6/db/f10 [0,4194304] 0 2024-05-27T09:29:53.452 INFO:tasks.workunit.client.1.smithi138.stdout:9/982: write d6/da/d29/d8f/db7/fa1 [353718,15195] 0 2024-05-27T09:29:53.453 INFO:tasks.workunit.client.1.smithi138.stdout:9/983: readlink d6/da/d29/d59/dee/d55/d75/d8c/lc3 0 2024-05-27T09:29:53.453 INFO:tasks.workunit.client.1.smithi138.stdout:9/984: chown d6/da/d29/d59/d146/c14e 1 1 2024-05-27T09:29:53.454 INFO:tasks.workunit.client.1.smithi138.stdout:9/985: mkdir d6/db/de/d6b/d104/d151 0 2024-05-27T09:29:53.477 INFO:tasks.workunit.client.1.smithi138.stdout:9/986: write d6/db/de/d18/fbe [701928,101127] 0 2024-05-27T09:29:53.479 INFO:tasks.workunit.client.1.smithi138.stdout:9/987: symlink d6/db/de/l152 0 2024-05-27T09:29:53.481 INFO:tasks.workunit.client.1.smithi138.stdout:9/988: getdents d6/da/d29/d8f/db7/dbd/da2 0 2024-05-27T09:29:53.483 INFO:tasks.workunit.client.1.smithi138.stdout:9/989: creat d6/da/d29/d59/d146/f153 x:0 0 0 2024-05-27T09:29:53.486 INFO:tasks.workunit.client.1.smithi138.stdout:3/851: dwrite d0/d3f/f84 [0,4194304] 0 2024-05-27T09:29:53.490 INFO:tasks.workunit.client.1.smithi138.stdout:3/852: mknod d0/d2/da/df/d100/c116 0 2024-05-27T09:29:53.492 INFO:tasks.workunit.client.1.smithi138.stdout:3/853: rename d0/d2/f13 to d0/d3f/d50/d9b/d9f/f117 0 2024-05-27T09:29:53.494 INFO:tasks.workunit.client.1.smithi138.stdout:3/854: truncate d0/d2/fc8 1672457 0 2024-05-27T09:29:53.501 INFO:tasks.workunit.client.1.smithi138.stdout:3/855: rename d0/d2/da/df/fa1 to d0/d2/da/df/d10/d2a/d61/f118 0 2024-05-27T09:29:53.525 INFO:tasks.workunit.client.1.smithi138.stdout:3/856: write d0/d2/da/df/d10/f38 [2333443,4669] 0 2024-05-27T09:29:53.525 INFO:tasks.workunit.client.1.smithi138.stdout:3/857: stat d0/dda 0 2024-05-27T09:29:53.527 INFO:tasks.workunit.client.1.smithi138.stdout:3/858: link d0/d2/da/c1c d0/dda/c119 0 2024-05-27T09:29:53.527 INFO:tasks.workunit.client.1.smithi138.stdout:3/859: write d0/d3f/d50/f7c [1329541,16813] 0 2024-05-27T09:29:53.529 INFO:tasks.workunit.client.1.smithi138.stdout:9/990: dwrite d6/da/d29/d8f/db7/dbd/d4c/fae [4194304,4194304] 0 2024-05-27T09:29:53.529 INFO:tasks.workunit.client.1.smithi138.stdout:3/860: rename d0/d2/da/df/d10/d2a/cb8 to d0/d3f/d50/c11a 0 2024-05-27T09:29:53.534 INFO:tasks.workunit.client.1.smithi138.stdout:9/991: link d6/db/l63 d6/db/de/d6b/d104/d12a/d102/l154 0 2024-05-27T09:29:53.538 INFO:tasks.workunit.client.1.smithi138.stdout:3/861: link d0/d2/c20 d0/d80/dab/c11b 0 2024-05-27T09:29:53.540 INFO:tasks.workunit.client.1.smithi138.stdout:3/862: truncate d0/d2/da/df/d10/d2a/d48/f79 1483081 0 2024-05-27T09:29:53.546 INFO:tasks.workunit.client.1.smithi138.stdout:4/975: sync 2024-05-27T09:29:53.546 INFO:tasks.workunit.client.1.smithi138.stdout:2/720: sync 2024-05-27T09:29:53.548 INFO:tasks.workunit.client.1.smithi138.stdout:2/721: rename d1/d15/d2d/d17/c70 to d1/cf4 0 2024-05-27T09:29:53.549 INFO:tasks.workunit.client.1.smithi138.stdout:2/722: link d1/d15/d2d/d17/d25/l26 d1/d15/d2d/d17/d25/d4d/d4e/lf5 0 2024-05-27T09:29:53.561 INFO:tasks.workunit.client.1.smithi138.stdout:4/976: mkdir d7/d157 0 2024-05-27T09:29:53.582 INFO:tasks.workunit.client.1.smithi138.stdout:4/977: dread d7/df/d35/d4c/d59/d5d/fe8 [0,4194304] 0 2024-05-27T09:29:53.607 INFO:tasks.workunit.client.1.smithi138.stdout:9/992: dwrite d6/da/d29/d8f/db7/fd3 [0,4194304] 0 2024-05-27T09:29:53.615 INFO:tasks.workunit.client.1.smithi138.stdout:4/978: dread d7/df/d35/d4c/d59/d5d/d64/df6/d13f/da1/db1/fe6 [0,4194304] 0 2024-05-27T09:29:53.615 INFO:tasks.workunit.client.1.smithi138.stdout:4/979: write d7/df/d35/f77 [388405,9262] 0 2024-05-27T09:29:53.617 INFO:tasks.workunit.client.1.smithi138.stdout:4/980: symlink d7/d6e/l158 0 2024-05-27T09:29:53.627 INFO:tasks.workunit.client.1.smithi138.stdout:9/993: dread d6/db/de/d6b/d104/d12a/fea [0,4194304] 0 2024-05-27T09:29:53.631 INFO:tasks.workunit.client.1.smithi138.stdout:9/994: truncate d6/da/d29/d59/d146/f73 1444753 0 2024-05-27T09:29:53.638 INFO:tasks.workunit.client.1.smithi138.stdout:3/863: dwrite d0/d3f/d60/d93/de5/dca/fce [0,4194304] 0 2024-05-27T09:29:53.638 INFO:tasks.workunit.client.1.smithi138.stdout:3/864: read - d0/d2/da/df/d10/d2a/d48/f8c zero size 2024-05-27T09:29:53.640 INFO:tasks.workunit.client.1.smithi138.stdout:4/981: write d7/d145/fc9 [1448021,118939] 0 2024-05-27T09:29:53.641 INFO:tasks.workunit.client.1.smithi138.stdout:4/982: stat d7/df/d35/d4c/d59/d5d/ff7 0 2024-05-27T09:29:53.641 INFO:tasks.workunit.client.1.smithi138.stdout:4/983: chown d7/d145/def/c10c 196108296 1 2024-05-27T09:29:53.674 INFO:tasks.workunit.client.1.smithi138.stdout:4/984: write d7/df/d35/d4c/d59/d5d/d64/dc5/f131 [634837,90178] 0 2024-05-27T09:29:53.674 INFO:tasks.workunit.client.1.smithi138.stdout:4/985: chown d7/dba/dcf/ce1 304018 1 2024-05-27T09:29:53.692 INFO:tasks.workunit.client.1.smithi138.stdout:3/865: dwrite d0/d3f/d60/d93/de5/fd3 [0,4194304] 0 2024-05-27T09:29:53.693 INFO:tasks.workunit.client.1.smithi138.stdout:3/866: rmdir d0/d3f/d60/d93/de5 39 2024-05-27T09:29:53.695 INFO:tasks.workunit.client.1.smithi138.stdout:3/867: symlink d0/d2/da/df/d10/d2a/d48/d90/l11c 0 2024-05-27T09:29:53.696 INFO:tasks.workunit.client.1.smithi138.stdout:3/868: mkdir d0/dda/d11d 0 2024-05-27T09:29:53.696 INFO:tasks.workunit.client.1.smithi138.stdout:3/869: write d0/d3f/d50/d9b/d9f/f117 [3210736,57174] 0 2024-05-27T09:29:53.700 INFO:tasks.workunit.client.1.smithi138.stdout:0/830: sync 2024-05-27T09:29:53.700 INFO:tasks.workunit.client.1.smithi138.stdout:4/986: dread d7/f120 [0,4194304] 0 2024-05-27T09:29:53.700 INFO:tasks.workunit.client.1.smithi138.stdout:4/987: write d7/d145/fe0 [1152938,610] 0 2024-05-27T09:29:53.700 INFO:tasks.workunit.client.1.smithi138.stdout:4/988: truncate d7/df/d35/d4c/d59/d5d/d64/fa7 204385 0 2024-05-27T09:29:53.700 INFO:tasks.workunit.client.1.smithi138.stdout:9/995: dwrite d6/f88 [0,4194304] 0 2024-05-27T09:29:53.703 INFO:tasks.workunit.client.1.smithi138.stdout:9/996: symlink d6/da/d2f/d135/l155 0 2024-05-27T09:29:53.703 INFO:tasks.workunit.client.1.smithi138.stdout:9/997: read d6/da/d29/ffd [1143339,119271] 0 2024-05-27T09:29:53.704 INFO:tasks.workunit.client.1.smithi138.stdout:0/831: creat da/d2a/d8e/d1f/d42/d63/d9d/f10c x:0 0 0 2024-05-27T09:29:53.704 INFO:tasks.workunit.client.1.smithi138.stdout:0/832: chown da/d2a/d8e/le6 12458682 1 2024-05-27T09:29:53.706 INFO:tasks.workunit.client.1.smithi138.stdout:0/833: creat da/d2a/dc9/f10d x:0 0 0 2024-05-27T09:29:53.706 INFO:tasks.workunit.client.1.smithi138.stdout:9/998: stat d6/da/la0 0 2024-05-27T09:29:53.725 INFO:tasks.workunit.client.1.smithi138.stdout:4/989: write d7/f18 [3682373,58375] 0 2024-05-27T09:29:53.725 INFO:tasks.workunit.client.1.smithi138.stdout:9/999: dread d6/da/d29/d8f/db7/dbd/d4c/fd0 [0,4194304] 0 2024-05-27T09:29:53.743 INFO:tasks.workunit.client.1.smithi138.stdout:0/834: write da/d2a/d8e/d6a/daf/dc4/fc8 [762804,76622] 0 2024-05-27T09:29:53.743 INFO:tasks.workunit.client.1.smithi138.stdout:0/835: readlink da/d2a/d8e/l23 0 2024-05-27T09:29:53.743 INFO:tasks.workunit.client.1.smithi138.stdout:3/870: dwrite d0/d3f/ff2 [0,4194304] 0 2024-05-27T09:29:53.744 INFO:tasks.workunit.client.1.smithi138.stdout:3/871: fdatasync d0/dda/f115 0 2024-05-27T09:29:53.744 INFO:tasks.workunit.client.1.smithi138.stdout:3/872: fsync d0/d3f/d50/d9b/d9f/fc9 0 2024-05-27T09:29:53.746 INFO:tasks.workunit.client.1.smithi138.stdout:3/873: symlink d0/d2/da/l11e 0 2024-05-27T09:29:53.748 INFO:tasks.workunit.client.1.smithi138.stdout:0/836: mknod da/d2a/d8e/d6a/c10e 0 2024-05-27T09:29:53.748 INFO:tasks.workunit.client.1.smithi138.stdout:0/837: creat da/d2a/d8e/d1f/d60/dbd/f10f x:0 0 0 2024-05-27T09:29:53.748 INFO:tasks.workunit.client.1.smithi138.stdout:3/874: creat d0/d3f/d60/d93/de5/dca/f11f x:0 0 0 2024-05-27T09:29:53.750 INFO:tasks.workunit.client.1.smithi138.stdout:3/875: getdents d0/d3f/d60/d93/de5 0 2024-05-27T09:29:53.750 INFO:tasks.workunit.client.1.smithi138.stdout:0/838: mknod da/d2a/d2b/c110 0 2024-05-27T09:29:53.752 INFO:tasks.workunit.client.1.smithi138.stdout:3/876: creat d0/d2/da/df/d10/d2a/d61/d73/f120 x:0 0 0 2024-05-27T09:29:53.761 INFO:tasks.workunit.client.1.smithi138.stdout:3/877: dread d0/d2/da/df/d10/d2a/f8f [0,4194304] 0 2024-05-27T09:29:53.764 INFO:tasks.workunit.client.1.smithi138.stdout:2/723: sync 2024-05-27T09:29:53.764 INFO:tasks.workunit.client.1.smithi138.stdout:1/916: sync 2024-05-27T09:29:53.764 INFO:tasks.workunit.client.1.smithi138.stdout:3/878: symlink d0/d3f/d50/d9b/d9f/l121 0 2024-05-27T09:29:53.765 INFO:tasks.workunit.client.1.smithi138.stdout:3/879: fsync d0/d2/da/df/d10/f77 0 2024-05-27T09:29:53.766 INFO:tasks.workunit.client.1.smithi138.stdout:3/880: link d0/d3f/d50/d9b/lef d0/dda/d106/l122 0 2024-05-27T09:29:53.769 INFO:tasks.workunit.client.1.smithi138.stdout:3/881: mkdir d0/d3f/d50/d9b/d123 0 2024-05-27T09:29:53.770 INFO:tasks.workunit.client.1.smithi138.stdout:1/917: mkdir d4/d48/d39/d5f/d93/d133 0 2024-05-27T09:29:53.770 INFO:tasks.workunit.client.1.smithi138.stdout:3/882: creat d0/d2/da/df/d10/f124 x:0 0 0 2024-05-27T09:29:53.772 INFO:tasks.workunit.client.1.smithi138.stdout:3/883: mknod d0/d3f/d60/d93/c125 0 2024-05-27T09:29:53.772 INFO:tasks.workunit.client.1.smithi138.stdout:3/884: fdatasync d0/d2/da/df/d10/d2a/d61/de1/f111 0 2024-05-27T09:29:53.772 INFO:tasks.workunit.client.1.smithi138.stdout:3/885: truncate d0/d2/da/df/d10/d2a/d61/f99 120496 0 2024-05-27T09:29:53.775 INFO:tasks.workunit.client.1.smithi138.stdout:3/886: rmdir d0/d2/da/dad 0 2024-05-27T09:29:53.777 INFO:tasks.workunit.client.1.smithi138.stdout:3/887: creat d0/d3f/d50/d9b/d9f/da3/f126 x:0 0 0 2024-05-27T09:29:53.782 INFO:tasks.workunit.client.1.smithi138.stdout:3/888: rename d0/d2/da/df/d10/d2a/db2/dd1/dd8/cdd to d0/d3f/d60/d93/de5/dca/c127 0 2024-05-27T09:29:53.784 INFO:tasks.workunit.client.1.smithi138.stdout:3/889: mkdir d0/d3f/d50/d9b/d123/d128 0 2024-05-27T09:29:53.784 INFO:tasks.workunit.client.1.smithi138.stdout:3/890: fdatasync d0/d2/da/f94 0 2024-05-27T09:29:53.804 INFO:tasks.workunit.client.1.smithi138.stdout:0/839: dwrite da/f66 [0,4194304] 0 2024-05-27T09:29:53.825 INFO:tasks.workunit.client.1.smithi138.stdout:1/918: dwrite d4/d48/d39/d5f/d63/fa3 [0,4194304] 0 2024-05-27T09:29:53.828 INFO:tasks.workunit.client.1.smithi138.stdout:0/840: dread da/fb8 [0,4194304] 0 2024-05-27T09:29:53.828 INFO:tasks.workunit.client.1.smithi138.stdout:0/841: truncate da/d2a/d8e/d1f/ff5 935697 0 2024-05-27T09:29:53.829 INFO:tasks.workunit.client.1.smithi138.stdout:0/842: truncate da/d2a/d8e/d1f/fee 1247969 0 2024-05-27T09:29:53.829 INFO:tasks.workunit.client.1.smithi138.stdout:0/843: fsync da/d2a/d8e/d43/d74/faa 0 2024-05-27T09:29:53.829 INFO:tasks.workunit.client.1.smithi138.stdout:0/844: dread - da/d2a/dc9/f102 zero size 2024-05-27T09:29:53.834 INFO:tasks.workunit.client.1.smithi138.stdout:2/724: dwrite d1/d15/d2d/d17/d25/d4d/d4e/f9e [0,4194304] 0 2024-05-27T09:29:53.840 INFO:tasks.workunit.client.1.smithi138.stdout:1/919: read d4/d48/d39/d5f/f5b [449301,17654] 0 2024-05-27T09:29:53.843 INFO:tasks.workunit.client.1.smithi138.stdout:0/845: rename da/d2a/d8e/d1f/f21 to da/d2a/d8e/d43/f111 0 2024-05-27T09:29:53.854 INFO:tasks.workunit.client.1.smithi138.stdout:3/891: dwrite d0/d2/fe0 [0,4194304] 0 2024-05-27T09:29:53.854 INFO:tasks.workunit.client.1.smithi138.stdout:3/892: fsync d0/d2/da/df/d10/d32/f9 0 2024-05-27T09:29:53.854 INFO:tasks.workunit.client.1.smithi138.stdout:3/893: dread - d0/d2/da/df/d10/d2a/d61/d73/f120 zero size 2024-05-27T09:29:53.854 INFO:tasks.workunit.client.1.smithi138.stdout:3/894: write d0/d2/da/df/d10/f4a [1735591,102450] 0 2024-05-27T09:29:53.859 INFO:tasks.workunit.client.1.smithi138.stdout:3/895: creat d0/d3f/d50/d9b/f129 x:0 0 0 2024-05-27T09:29:53.859 INFO:tasks.workunit.client.1.smithi138.stdout:3/896: write d0/d2/da/df/d10/d2a/db2/fed [409574,19443] 0 2024-05-27T09:29:53.861 INFO:tasks.workunit.client.1.smithi138.stdout:3/897: mknod d0/c12a 0 2024-05-27T09:29:53.864 INFO:tasks.workunit.client.1.smithi138.stdout:1/920: write d4/d48/d22/d65/f10a [1393882,70573] 0 2024-05-27T09:29:53.871 INFO:tasks.workunit.client.1.smithi138.stdout:0/846: dread da/d2a/d3f/f4c [4194304,4194304] 0 2024-05-27T09:29:53.871 INFO:tasks.workunit.client.1.smithi138.stdout:0/847: getdents da/d2a/d8e/d1f 0 2024-05-27T09:29:53.883 INFO:tasks.workunit.client.1.smithi138.stdout:3/898: write d0/d2/da/df/d10/d2a/d48/f4d [135330,55270] 0 2024-05-27T09:29:53.883 INFO:tasks.workunit.client.1.smithi138.stdout:3/899: creat d0/d3f/d50/d9b/f12b x:0 0 0 2024-05-27T09:29:53.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:53 smithi138 ceph-mon[34885]: Updating smithi138:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T09:29:53.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:53 smithi138 ceph-mon[34885]: pgmap v24: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 187 MiB/s rd, 223 MiB/s wr, 387 op/s 2024-05-27T09:29:53.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:53 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:53.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:53 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:53.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:53 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:53.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:53 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:53.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:53 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:53.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:53 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:53.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:53 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:53.892 INFO:tasks.workunit.client.1.smithi138.stdout:2/725: dwrite d1/d10/d52/d5d/fd8 [0,4194304] 0 2024-05-27T09:29:53.893 INFO:tasks.workunit.client.1.smithi138.stdout:0/848: write da/f20 [670014,37564] 0 2024-05-27T09:29:53.893 INFO:tasks.workunit.client.1.smithi138.stdout:0/849: chown da/d2a/d8e/d1f/d42/fac 139120612 1 2024-05-27T09:29:53.896 INFO:tasks.workunit.client.1.smithi138.stdout:0/850: unlink da/d2a/d8e/d1f/d42/d4a/d53/lb0 0 2024-05-27T09:29:53.904 INFO:tasks.workunit.client.1.smithi138.stdout:0/851: dread da/d2a/d8e/d43/f6c [0,4194304] 0 2024-05-27T09:29:53.908 INFO:tasks.workunit.client.1.smithi138.stdout:0/852: rename da/d2a/d8e/d6a to da/d17/d112 0 2024-05-27T09:29:53.908 INFO:tasks.workunit.client.1.smithi138.stdout:0/853: truncate da/d2a/d8e/d1f/d42/ffd 370285 0 2024-05-27T09:29:53.908 INFO:tasks.workunit.client.1.smithi138.stdout:0/854: truncate da/d2a/d8e/d1f/fad 803381 0 2024-05-27T09:29:53.908 INFO:tasks.workunit.client.1.smithi138.stdout:0/855: creat da/d2a/d55/f113 x:0 0 0 2024-05-27T09:29:53.953 INFO:tasks.workunit.client.1.smithi138.stdout:2/726: dwrite d1/d10/f76 [4194304,4194304] 0 2024-05-27T09:29:53.954 INFO:tasks.workunit.client.1.smithi138.stdout:2/727: fsync d1/d15/d2d/d17/d25/d4d/d6d/fb7 0 2024-05-27T09:29:53.954 INFO:tasks.workunit.client.1.smithi138.stdout:1/921: dwrite d4/d3c/f42 [0,4194304] 0 2024-05-27T09:29:53.955 INFO:tasks.workunit.client.1.smithi138.stdout:1/922: mkdir d4/d48/d39/d5f/d86/d134 0 2024-05-27T09:29:53.956 INFO:tasks.workunit.client.1.smithi138.stdout:2/728: mknod d1/d15/d2d/d17/d25/d4d/d6d/cf6 0 2024-05-27T09:29:53.957 INFO:tasks.workunit.client.1.smithi138.stdout:2/729: truncate d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc/f83 1268951 0 2024-05-27T09:29:53.957 INFO:tasks.workunit.client.1.smithi138.stdout:2/730: chown d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc 6111 1 2024-05-27T09:29:53.957 INFO:tasks.workunit.client.1.smithi138.stdout:2/731: dread - d1/d15/d2d/d2e/faf zero size 2024-05-27T09:29:53.970 INFO:tasks.workunit.client.1.smithi138.stdout:1/923: write d4/d48/d39/d5f/f5b [229830,9490] 0 2024-05-27T09:29:53.970 INFO:tasks.workunit.client.1.smithi138.stdout:1/924: creat d4/d1a/d2e/d32/d8c/f135 x:0 0 0 2024-05-27T09:29:53.972 INFO:tasks.workunit.client.1.smithi138.stdout:1/925: dread - d4/d1a/d2e/d4a/d7e/d11d/f120 zero size 2024-05-27T09:29:53.978 INFO:tasks.workunit.client.1.smithi138.stdout:1/926: write d4/d88/ff2 [721804,49857] 0 2024-05-27T09:29:53.985 INFO:tasks.workunit.client.1.smithi138.stdout:0/856: dwrite da/d2a/d8e/d1f/f3e [0,4194304] 0 2024-05-27T09:29:53.997 INFO:tasks.workunit.client.1.smithi138.stdout:0/857: symlink da/d2a/d8e/d1f/d4f/l114 0 2024-05-27T09:29:53.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:53 smithi119 ceph-mon[29082]: Updating smithi138:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T09:29:53.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:53 smithi119 ceph-mon[29082]: pgmap v24: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 187 MiB/s rd, 223 MiB/s wr, 387 op/s 2024-05-27T09:29:53.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:53 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:53.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:53 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:53.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:53 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:53.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:53 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:53.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:53 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:53.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:53 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:53.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:53 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:29:53.999 INFO:tasks.workunit.client.1.smithi138.stdout:2/732: dwrite d1/d15/d2d/d17/d25/d9d/f3b [0,4194304] 0 2024-05-27T09:29:53.999 INFO:tasks.workunit.client.1.smithi138.stdout:2/733: creat d1/dd4/ff7 x:0 0 0 2024-05-27T09:29:54.001 INFO:tasks.workunit.client.1.smithi138.stdout:2/734: creat d1/d15/d2d/d17/dac/df2/ff8 x:0 0 0 2024-05-27T09:29:54.001 INFO:tasks.workunit.client.1.smithi138.stdout:2/735: write d1/d10/d52/d5d/f88 [5128961,36953] 0 2024-05-27T09:29:54.013 INFO:tasks.workunit.client.1.smithi138.stdout:4/990: sync 2024-05-27T09:29:54.013 INFO:tasks.workunit.client.1.smithi138.stdout:4/991: dread - d7/df/d35/f133 zero size 2024-05-27T09:29:54.015 INFO:tasks.workunit.client.1.smithi138.stdout:4/992: unlink d7/dba/dfa/f148 0 2024-05-27T09:29:54.018 INFO:tasks.workunit.client.1.smithi138.stdout:4/993: mknod d7/df/d35/d4c/d59/d5d/d64/df6/d13f/da1/dbc/c159 0 2024-05-27T09:29:54.061 INFO:tasks.workunit.client.1.smithi138.stdout:4/994: dwrite d7/df/f29 [0,4194304] 0 2024-05-27T09:29:54.163 INFO:tasks.workunit.client.1.smithi138.stdout:4/995: symlink d7/df/l15a 0 2024-05-27T09:29:54.170 INFO:tasks.workunit.client.1.smithi138.stdout:4/996: rmdir d7/d145 39 2024-05-27T09:29:54.170 INFO:tasks.workunit.client.1.smithi138.stdout:4/997: fsync d7/df/d35/d4c/d59/d5d/d64/df6/d13f/fa0 0 2024-05-27T09:29:54.177 INFO:tasks.workunit.client.1.smithi138.stdout:4/998: creat d7/f15b x:0 0 0 2024-05-27T09:29:54.177 INFO:tasks.workunit.client.1.smithi138.stdout:4/999: creat d7/dba/dfa/f15c x:0 0 0 2024-05-27T09:29:54.217 INFO:tasks.workunit.client.1.smithi138.stdout:3/900: sync 2024-05-27T09:29:54.217 INFO:tasks.workunit.client.1.smithi138.stdout:3/901: fdatasync d0/d3f/d60/f95 0 2024-05-27T09:29:54.217 INFO:tasks.workunit.client.1.smithi138.stdout:3/902: write d0/d2/da/df/d10/d32/fa5 [158696,110756] 0 2024-05-27T09:29:54.219 INFO:tasks.workunit.client.1.smithi138.stdout:3/903: creat d0/d2/da/df/d10/d2a/db2/dd1/dd8/f12c x:0 0 0 2024-05-27T09:29:54.219 INFO:tasks.workunit.client.1.smithi138.stdout:3/904: fsync d0/d2/da/f94 0 2024-05-27T09:29:54.221 INFO:tasks.workunit.client.1.smithi138.stdout:3/905: mkdir d0/d2/da/df/d10/d32/d12d 0 2024-05-27T09:29:54.221 INFO:tasks.workunit.client.1.smithi138.stdout:3/906: symlink d0/d80/dab/d113/l12e 0 2024-05-27T09:29:54.223 INFO:tasks.workunit.client.1.smithi138.stdout:3/907: mkdir d0/d2/da/df/d10/d2a/d48/d12f 0 2024-05-27T09:29:54.223 INFO:tasks.workunit.client.1.smithi138.stdout:3/908: chown d0/d2/da/l16 266 1 2024-05-27T09:29:54.225 INFO:tasks.workunit.client.1.smithi138.stdout:3/909: rename d0/d2/fc to d0/d2/da/df/d10/d2a/f130 0 2024-05-27T09:29:54.227 INFO:tasks.workunit.client.1.smithi138.stdout:3/910: creat d0/d2/da/df/d10/d32/f131 x:0 0 0 2024-05-27T09:29:54.227 INFO:tasks.workunit.client.1.smithi138.stdout:3/911: stat d0/d2/c2b 0 2024-05-27T09:29:54.229 INFO:tasks.workunit.client.1.smithi138.stdout:3/912: unlink d0/f12 0 2024-05-27T09:29:54.229 INFO:tasks.workunit.client.1.smithi138.stdout:3/913: stat d0/d2/da/df/d10/d2a/f42 0 2024-05-27T09:29:54.272 INFO:tasks.workunit.client.1.smithi138.stdout:1/927: sync 2024-05-27T09:29:54.382 INFO:tasks.workunit.client.1.smithi138.stdout:2/736: sync 2024-05-27T09:29:54.382 INFO:tasks.workunit.client.1.smithi138.stdout:0/858: sync 2024-05-27T09:29:54.382 INFO:tasks.workunit.client.1.smithi138.stdout:0/859: fsync da/d2a/d8e/d1f/f38 0 2024-05-27T09:29:54.383 INFO:tasks.workunit.client.1.smithi138.stdout:2/737: dread - d1/d15/d2d/f8e zero size 2024-05-27T09:29:54.383 INFO:tasks.workunit.client.1.smithi138.stdout:2/738: dread - d1/d15/d2d/d2e/faf zero size 2024-05-27T09:29:54.385 INFO:tasks.workunit.client.1.smithi138.stdout:2/739: mkdir d1/d15/d2d/d17/d25/d9d/df9 0 2024-05-27T09:29:54.418 INFO:tasks.workunit.client.1.smithi138.stdout:2/740: dread d1/f1b [4194304,4194304] 0 2024-05-27T09:29:54.423 INFO:tasks.workunit.client.1.smithi138.stdout:2/741: creat d1/d15/d2d/d17/d25/d4d/d4e/ffa x:0 0 0 2024-05-27T09:29:54.490 INFO:tasks.workunit.client.1.smithi138.stdout:2/742: dwrite d1/d15/d2d/dd2/fe0 [0,4194304] 0 2024-05-27T09:29:54.494 INFO:tasks.workunit.client.1.smithi138.stdout:2/743: creat d1/ffb x:0 0 0 2024-05-27T09:29:54.497 INFO:tasks.workunit.client.1.smithi138.stdout:2/744: rename d1/d15/d2d/d2e/faf to d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/ffc 0 2024-05-27T09:29:54.500 INFO:tasks.workunit.client.1.smithi138.stdout:2/745: link d1/d10/d52/db9/lc2 d1/d15/d2d/d17/d25/d4d/d4e/d60/lfd 0 2024-05-27T09:29:54.506 INFO:tasks.workunit.client.1.smithi138.stdout:3/914: sync 2024-05-27T09:29:54.506 INFO:tasks.workunit.client.1.smithi138.stdout:1/928: sync 2024-05-27T09:29:54.506 INFO:tasks.workunit.client.1.smithi138.stdout:1/929: write d4/d48/f104 [935787,39446] 0 2024-05-27T09:29:54.508 INFO:tasks.workunit.client.1.smithi138.stdout:1/930: truncate d4/d1a/d2e/d32/d8c/dc6/f117 1531366 0 2024-05-27T09:29:54.515 INFO:tasks.workunit.client.1.smithi138.stdout:3/915: symlink d0/d2/d105/l132 0 2024-05-27T09:29:54.519 INFO:tasks.workunit.client.1.smithi138.stdout:3/916: mknod d0/d3f/d50/c133 0 2024-05-27T09:29:54.521 INFO:tasks.workunit.client.1.smithi138.stdout:3/917: truncate d0/d2/da/df/d10/d2a/d61/de1/fc7 7566024 0 2024-05-27T09:29:54.523 INFO:tasks.workunit.client.1.smithi138.stdout:3/918: unlink d0/d2/da/df/d10/d2a/f83 0 2024-05-27T09:29:54.525 INFO:tasks.workunit.client.1.smithi138.stdout:3/919: symlink d0/d3f/d50/d9b/d123/l134 0 2024-05-27T09:29:54.527 INFO:tasks.workunit.client.1.smithi138.stdout:3/920: symlink d0/d2/da/df/d10/d2a/d61/d73/l135 0 2024-05-27T09:29:54.532 INFO:tasks.workunit.client.1.smithi138.stdout:3/921: unlink d0/d2/fc8 0 2024-05-27T09:29:54.538 INFO:tasks.workunit.client.1.smithi138.stdout:3/922: creat d0/d3f/d60/d93/de5/f136 x:0 0 0 2024-05-27T09:29:54.553 INFO:tasks.workunit.client.1.smithi138.stdout:2/746: dwrite d1/d10/d52/fdb [0,4194304] 0 2024-05-27T09:29:54.553 INFO:tasks.workunit.client.1.smithi138.stdout:2/747: chown d1/d15/d2d/d17/d25/d9d/c77 187337 1 2024-05-27T09:29:54.560 INFO:tasks.workunit.client.1.smithi138.stdout:0/860: sync 2024-05-27T09:29:54.564 INFO:tasks.workunit.client.1.smithi138.stdout:0/861: rename da/d2a/d8e/d1f/fee to da/d2a/d55/f115 0 2024-05-27T09:29:54.564 INFO:tasks.workunit.client.1.smithi138.stdout:0/862: fsync da/d2a/d8e/d43/fec 0 2024-05-27T09:29:54.564 INFO:tasks.workunit.client.1.smithi138.stdout:0/863: dread - da/d2a/dc9/f102 zero size 2024-05-27T09:29:54.597 INFO:tasks.workunit.client.1.smithi138.stdout:3/923: dwrite d0/d3f/d50/f7c [4194304,4194304] 0 2024-05-27T09:29:54.603 INFO:tasks.workunit.client.1.smithi138.stdout:0/864: dread da/d2a/d55/f98 [0,4194304] 0 2024-05-27T09:29:54.610 INFO:tasks.workunit.client.1.smithi138.stdout:2/748: dwrite d1/d15/d2d/d17/d25/f2a [0,4194304] 0 2024-05-27T09:29:54.613 INFO:tasks.workunit.client.1.smithi138.stdout:2/749: rename d1/d15/d2d/d17/d25/d4d/d4e/d60/lc3 to d1/d15/lfe 0 2024-05-27T09:29:54.616 INFO:tasks.workunit.client.1.smithi138.stdout:2/750: unlink d1/d15/d2d/d17/d25/d9d/fb3 0 2024-05-27T09:29:54.618 INFO:tasks.workunit.client.1.smithi138.stdout:2/751: creat d1/d10/d52/fff x:0 0 0 2024-05-27T09:29:54.621 INFO:tasks.workunit.client.1.smithi138.stdout:2/752: unlink d1/d15/d2d/d17/d25/d4d/d4e/ddf/l82 0 2024-05-27T09:29:54.622 INFO:tasks.workunit.client.1.smithi138.stdout:2/753: mknod d1/d15/d2d/d17/def/c100 0 2024-05-27T09:29:54.626 INFO:tasks.workunit.client.1.smithi138.stdout:2/754: symlink d1/d10/d52/d5d/d7e/l101 0 2024-05-27T09:29:54.626 INFO:tasks.workunit.client.1.smithi138.stdout:2/755: write d1/d15/d2d/d17/d25/d9d/d4b/fbe [1679229,125623] 0 2024-05-27T09:29:54.626 INFO:tasks.workunit.client.1.smithi138.stdout:2/756: write d1/d15/d2d/f3e [409641,113485] 0 2024-05-27T09:29:54.626 INFO:tasks.workunit.client.1.smithi138.stdout:2/757: dread - d1/d10/d52/d5d/fbd zero size 2024-05-27T09:29:54.629 INFO:tasks.workunit.client.1.smithi138.stdout:2/758: mkdir d1/d15/d2d/d2e/d102 0 2024-05-27T09:29:54.629 INFO:tasks.workunit.client.1.smithi138.stdout:2/759: readlink d1/d15/d2d/d17/d25/d4d/db6/le1 0 2024-05-27T09:29:54.629 INFO:tasks.workunit.client.1.smithi138.stdout:2/760: write d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc/f7d [4592027,121569] 0 2024-05-27T09:29:54.646 INFO:tasks.workunit.client.1.smithi138.stdout:3/924: dwrite d0/dda/f10c [8388608,4194304] 0 2024-05-27T09:29:54.649 INFO:tasks.workunit.client.1.smithi138.stdout:0/865: dwrite da/d17/f8c [8388608,4194304] 0 2024-05-27T09:29:54.649 INFO:tasks.workunit.client.1.smithi138.stdout:0/866: readlink da/d2a/d8e/l2c 0 2024-05-27T09:29:54.654 INFO:tasks.workunit.client.1.smithi138.stdout:0/867: mkdir da/d2a/d8e/dc0/d116 0 2024-05-27T09:29:54.654 INFO:tasks.workunit.client.1.smithi138.stdout:0/868: fdatasync da/d2a/d55/f95 0 2024-05-27T09:29:54.665 INFO:tasks.workunit.client.1.smithi138.stdout:3/925: write d0/d3f/d50/d9b/d9f/da3/fc1 [3311789,60386] 0 2024-05-27T09:29:54.665 INFO:tasks.workunit.client.1.smithi138.stdout:3/926: chown d0/d3f/d50/f7c 47289 1 2024-05-27T09:29:54.668 INFO:tasks.workunit.client.1.smithi138.stdout:1/931: sync 2024-05-27T09:29:54.668 INFO:tasks.workunit.client.1.smithi138.stdout:1/932: read - d4/d1a/d2e/d4a/d7e/d11d/f120 zero size 2024-05-27T09:29:54.670 INFO:tasks.workunit.client.1.smithi138.stdout:1/933: stat d4/d48/l81 0 2024-05-27T09:29:54.670 INFO:tasks.workunit.client.1.smithi138.stdout:1/934: read d4/d1a/d2e/d32/d8c/dc6/f96 [30803,74066] 0 2024-05-27T09:29:54.670 INFO:tasks.workunit.client.1.smithi138.stdout:1/935: read - d4/d1a/d2e/d32/f129 zero size 2024-05-27T09:29:54.693 INFO:tasks.workunit.client.1.smithi138.stdout:3/927: dread d0/d2/da/f1b [0,4194304] 0 2024-05-27T09:29:54.693 INFO:tasks.workunit.client.1.smithi138.stdout:3/928: readlink d0/d3f/d50/d9b/d123/l134 0 2024-05-27T09:29:54.694 INFO:tasks.workunit.client.1.smithi138.stdout:1/936: dread d4/d48/d39/d5f/d63/f98 [0,4194304] 0 2024-05-27T09:29:54.698 INFO:tasks.workunit.client.1.smithi138.stdout:3/929: truncate d0/d2/da/df/d10/f5d 4155324 0 2024-05-27T09:29:54.698 INFO:tasks.workunit.client.1.smithi138.stdout:3/930: stat d0/d3f/d50/f7c 0 2024-05-27T09:29:54.698 INFO:tasks.workunit.client.1.smithi138.stdout:3/931: write d0/d3f/d60/d93/de5/dca/f11f [960086,40914] 0 2024-05-27T09:29:54.698 INFO:tasks.workunit.client.1.smithi138.stdout:1/937: unlink d4/d48/d39/f54 0 2024-05-27T09:29:54.699 INFO:tasks.workunit.client.1.smithi138.stdout:3/932: unlink d0/d3f/d60/d93/de5/ld4 0 2024-05-27T09:29:54.699 INFO:tasks.workunit.client.1.smithi138.stdout:3/933: write d0/d3f/d50/d9b/d9f/da3/fc1 [1675192,27503] 0 2024-05-27T09:29:54.700 INFO:tasks.workunit.client.1.smithi138.stdout:3/934: creat d0/d3f/d60/d93/de5/f137 x:0 0 0 2024-05-27T09:29:54.700 INFO:tasks.workunit.client.1.smithi138.stdout:1/938: getdents d4/d1a/d2e/d32/d8c 0 2024-05-27T09:29:54.702 INFO:tasks.workunit.client.1.smithi138.stdout:1/939: symlink d4/d1a/d2e/d4a/d7e/db7/dfb/l136 0 2024-05-27T09:29:54.706 INFO:tasks.workunit.client.1.smithi138.stdout:0/869: dwrite da/d2a/d8e/dda/ddf/fe4 [0,4194304] 0 2024-05-27T09:29:54.706 INFO:tasks.workunit.client.1.smithi138.stdout:1/940: read d4/d1a/d2e/d32/d8c/dc6/fbb [676643,10821] 0 2024-05-27T09:29:54.714 INFO:tasks.workunit.client.1.smithi138.stdout:0/870: link da/d2a/d8e/d1f/fad da/d2a/d8e/d1f/d42/d4a/f117 0 2024-05-27T09:29:54.719 INFO:tasks.workunit.client.1.smithi138.stdout:3/935: dread d0/d2/da/df/fe2 [0,4194304] 0 2024-05-27T09:29:54.719 INFO:tasks.workunit.client.1.smithi138.stdout:3/936: fdatasync d0/d2/da/df/d10/d2a/db2/fed 0 2024-05-27T09:29:54.719 INFO:tasks.workunit.client.1.smithi138.stdout:3/937: fsync d0/d2/d105/f10d 0 2024-05-27T09:29:54.719 INFO:tasks.workunit.client.1.smithi138.stdout:3/938: truncate d0/d2/da/df/d10/f5d 4194222 0 2024-05-27T09:29:54.719 INFO:tasks.workunit.client.1.smithi138.stdout:3/939: chown d0/d2/da/df/d10/d2a/d61/d73/l8b 7540 1 2024-05-27T09:29:54.719 INFO:tasks.workunit.client.1.smithi138.stdout:1/941: symlink d4/d1a/d2e/d32/d8c/l137 0 2024-05-27T09:29:54.723 INFO:tasks.workunit.client.1.smithi138.stdout:1/942: creat d4/d48/d39/d5f/d4d/d2b/f138 x:0 0 0 2024-05-27T09:29:54.723 INFO:tasks.workunit.client.1.smithi138.stdout:0/871: truncate da/d2a/d8e/d1f/d42/d4a/d53/f72 1928556 0 2024-05-27T09:29:54.728 INFO:tasks.workunit.client.1.smithi138.stdout:2/761: dwrite d1/d15/d2d/d17/d25/d9d/d4b/fbe [0,4194304] 0 2024-05-27T09:29:54.736 INFO:tasks.workunit.client.1.smithi138.stdout:2/762: mknod d1/d15/d2d/d17/d25/d4d/d4e/c103 0 2024-05-27T09:29:54.745 INFO:tasks.workunit.client.1.smithi138.stdout:2/763: rmdir d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc 39 2024-05-27T09:29:54.745 INFO:tasks.workunit.client.1.smithi138.stdout:2/764: readlink d1/d15/d2d/d17/d25/d4d/d4e/d60/l61 0 2024-05-27T09:29:54.745 INFO:tasks.workunit.client.1.smithi138.stdout:2/765: chown d1/d15/d2d/d17/dac/df2/ff8 1900 1 2024-05-27T09:29:54.757 INFO:tasks.workunit.client.1.smithi138.stdout:2/766: rename d1/d15/d2d/d17/d25/d4d/d4e/ddf to d1/d15/d2d/d17/dac/d104 0 2024-05-27T09:29:54.779 INFO:tasks.workunit.client.1.smithi138.stdout:1/943: dwrite d4/d1a/deb/f123 [0,4194304] 0 2024-05-27T09:29:54.785 INFO:tasks.workunit.client.1.smithi138.stdout:1/944: symlink d4/d48/d39/d5f/dd3/d10b/d124/l139 0 2024-05-27T09:29:54.787 INFO:tasks.workunit.client.1.smithi138.stdout:1/945: write d4/d1a/d2e/d4a/d7e/dc0/f119 [669527,65282] 0 2024-05-27T09:29:54.787 INFO:tasks.workunit.client.1.smithi138.stdout:1/946: fdatasync d4/d1a/deb/f101 0 2024-05-27T09:29:54.787 INFO:tasks.workunit.client.1.smithi138.stdout:1/947: readlink d4/d1a/d2e/d4a/d7e/lb3 0 2024-05-27T09:29:54.787 INFO:tasks.workunit.client.1.smithi138.stdout:1/948: getdents d4/d48/d39/d5f/d63/de6 0 2024-05-27T09:29:54.788 INFO:tasks.workunit.client.1.smithi138.stdout:3/940: dwrite d0/d3f/d50/f7c [0,4194304] 0 2024-05-27T09:29:54.788 INFO:tasks.workunit.client.1.smithi138.stdout:3/941: dread - d0/d3f/d60/ff6 zero size 2024-05-27T09:29:54.789 INFO:tasks.workunit.client.1.smithi138.stdout:1/949: link d4/d48/d39/d5f/d93/fb2 d4/d48/d39/d5f/d86/f13a 0 2024-05-27T09:29:54.794 INFO:tasks.workunit.client.1.smithi138.stdout:1/950: rename d4/d38/f112 to d4/d1a/d2e/d4a/d7e/db7/dfb/d127/f13b 0 2024-05-27T09:29:54.796 INFO:tasks.workunit.client.1.smithi138.stdout:1/951: symlink d4/d48/d39/d5f/dd3/l13c 0 2024-05-27T09:29:54.796 INFO:tasks.workunit.client.1.smithi138.stdout:1/952: fdatasync d4/d1a/d2e/d32/f129 0 2024-05-27T09:29:54.810 INFO:tasks.workunit.client.1.smithi138.stdout:0/872: dwrite da/d2a/d55/f95 [0,4194304] 0 2024-05-27T09:29:54.813 INFO:tasks.workunit.client.1.smithi138.stdout:0/873: unlink da/d17/d112/c7d 0 2024-05-27T09:29:54.815 INFO:tasks.workunit.client.1.smithi138.stdout:1/953: dread d4/d1a/d2e/d4a/d7e/db7/dfb/d127/f13b [0,4194304] 0 2024-05-27T09:29:54.816 INFO:tasks.workunit.client.1.smithi138.stdout:0/874: creat da/d2a/d8e/dc0/d116/f118 x:0 0 0 2024-05-27T09:29:54.818 INFO:tasks.workunit.client.1.smithi138.stdout:1/954: rename d4/d48/d39/d5f/d4d/d2b/f6c to d4/d1a/d2e/d4a/d7e/db7/dfb/f13d 0 2024-05-27T09:29:54.822 INFO:tasks.workunit.client.1.smithi138.stdout:0/875: rename da/d2a/d8e/d1f/ffa to da/d2a/d8e/d1f/f119 0 2024-05-27T09:29:54.823 INFO:tasks.workunit.client.1.smithi138.stdout:0/876: stat da/d2a/d8e/d1f/f26 0 2024-05-27T09:29:54.823 INFO:tasks.workunit.client.1.smithi138.stdout:0/877: fdatasync da/d17/f8c 0 2024-05-27T09:29:54.825 INFO:tasks.workunit.client.1.smithi138.stdout:1/955: rmdir d4/d48/d22/d65/d100/d118 39 2024-05-27T09:29:54.826 INFO:tasks.workunit.client.1.smithi138.stdout:1/956: creat d4/d48/d22/f13e x:0 0 0 2024-05-27T09:29:54.826 INFO:tasks.workunit.client.1.smithi138.stdout:1/957: readlink d4/daa/l126 0 2024-05-27T09:29:54.827 INFO:tasks.workunit.client.1.smithi138.stdout:1/958: truncate d4/d1a/d2e/d32/d8c/dc6/f61 1736890 0 2024-05-27T09:29:54.827 INFO:tasks.workunit.client.1.smithi138.stdout:1/959: write d4/f9b [2975527,28298] 0 2024-05-27T09:29:54.830 INFO:tasks.workunit.client.1.smithi138.stdout:1/960: mkdir d4/d1a/d2e/d4a/d7e/dc0/d13f 0 2024-05-27T09:29:54.833 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:54 smithi119 ceph-mon[29082]: Reconfiguring prometheus.smithi119 (dependencies changed)... 2024-05-27T09:29:54.833 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:54 smithi119 ceph-mon[29082]: Reconfiguring daemon prometheus.smithi119 on smithi119 2024-05-27T09:29:54.837 INFO:tasks.workunit.client.1.smithi138.stdout:2/767: dwrite d1/d15/d2d/d17/d25/d4d/d4e/d60/f66 [0,4194304] 0 2024-05-27T09:29:54.837 INFO:tasks.workunit.client.1.smithi138.stdout:2/768: truncate d1/d15/d2d/d17/d25/f3a 4959655 0 2024-05-27T09:29:54.839 INFO:tasks.workunit.client.1.smithi138.stdout:3/942: dwrite d0/d2/da/df/d10/d2a/d61/de1/f101 [0,4194304] 0 2024-05-27T09:29:54.839 INFO:tasks.workunit.client.1.smithi138.stdout:3/943: readlink d0/d2/da/df/la8 0 2024-05-27T09:29:54.839 INFO:tasks.workunit.client.1.smithi138.stdout:3/944: chown d0/d3f/d60/d93/de5/cbd 673214620 1 2024-05-27T09:29:54.840 INFO:tasks.workunit.client.1.smithi138.stdout:0/878: dread da/d2a/d8e/d1f/d42/d4a/d53/f72 [0,4194304] 0 2024-05-27T09:29:54.842 INFO:tasks.workunit.client.1.smithi138.stdout:0/879: rmdir da/df4 0 2024-05-27T09:29:54.842 INFO:tasks.workunit.client.1.smithi138.stdout:0/880: chown da/d2a/d8e/d1f/le9 51679676 1 2024-05-27T09:29:54.842 INFO:tasks.workunit.client.1.smithi138.stdout:0/881: fsync da/d2a/d8e/d1f/f119 0 2024-05-27T09:29:54.843 INFO:tasks.workunit.client.1.smithi138.stdout:2/769: creat d1/d15/dd6/dec/f105 x:0 0 0 2024-05-27T09:29:54.843 INFO:tasks.workunit.client.1.smithi138.stdout:0/882: getdents da/d2a/d8e/d43 0 2024-05-27T09:29:54.844 INFO:tasks.workunit.client.1.smithi138.stdout:2/770: symlink d1/d15/d2d/d17/dac/l106 0 2024-05-27T09:29:54.849 INFO:tasks.workunit.client.1.smithi138.stdout:0/883: mknod da/d2a/d55/c11a 0 2024-05-27T09:29:54.862 INFO:tasks.workunit.client.1.smithi138.stdout:3/945: dread d0/d2/da/df/d10/f5d [0,4194304] 0 2024-05-27T09:29:54.862 INFO:tasks.workunit.client.1.smithi138.stdout:3/946: fdatasync d0/d3f/d60/f96 0 2024-05-27T09:29:54.878 INFO:tasks.workunit.client.1.smithi138.stdout:3/947: dread d0/d2/da/df/d10/d2a/d48/f5f [0,4194304] 0 2024-05-27T09:29:54.882 INFO:tasks.workunit.client.1.smithi138.stdout:3/948: rmdir d0/d80/dab 39 2024-05-27T09:29:54.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:54 smithi138 ceph-mon[34885]: Reconfiguring prometheus.smithi119 (dependencies changed)... 2024-05-27T09:29:54.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:54 smithi138 ceph-mon[34885]: Reconfiguring daemon prometheus.smithi119 on smithi119 2024-05-27T09:29:54.888 INFO:tasks.workunit.client.1.smithi138.stdout:1/961: dwrite d4/d48/d39/d5f/d4d/d2b/fe5 [0,4194304] 0 2024-05-27T09:29:54.891 INFO:tasks.workunit.client.1.smithi138.stdout:1/962: rename d4/d48/d22/c28 to d4/d48/d39/d5f/d86/c140 0 2024-05-27T09:29:54.896 INFO:tasks.workunit.client.1.smithi138.stdout:2/771: dwrite d1/d10/fe [4194304,4194304] 0 2024-05-27T09:29:54.901 INFO:tasks.workunit.client.1.smithi138.stdout:2/772: link d1/c6a d1/d15/d2d/d17/c107 0 2024-05-27T09:29:54.943 INFO:tasks.workunit.client.1.smithi138.stdout:0/884: dwrite da/d2a/d55/fff [0,4194304] 0 2024-05-27T09:29:54.947 INFO:tasks.workunit.client.1.smithi138.stdout:0/885: mknod da/d17/d112/daf/dc4/c11b 0 2024-05-27T09:29:54.949 INFO:tasks.workunit.client.1.smithi138.stdout:0/886: write da/fcb [161996,78139] 0 2024-05-27T09:29:54.949 INFO:tasks.workunit.client.1.smithi138.stdout:0/887: dread - da/fe2 zero size 2024-05-27T09:29:54.951 INFO:tasks.workunit.client.1.smithi138.stdout:0/888: getdents da/d2a/d8e/d1f/d42/d4a 0 2024-05-27T09:29:54.951 INFO:tasks.workunit.client.1.smithi138.stdout:0/889: write da/d2a/dc9/f102 [941419,5312] 0 2024-05-27T09:29:54.957 INFO:tasks.workunit.client.1.smithi138.stdout:3/949: dwrite d0/d2/da/df/d10/d2a/db2/dd1/ffa [0,4194304] 0 2024-05-27T09:29:54.960 INFO:tasks.workunit.client.1.smithi138.stdout:1/963: dwrite d4/d1a/d2e/d32/d8c/f135 [0,4194304] 0 2024-05-27T09:29:54.962 INFO:tasks.workunit.client.1.smithi138.stdout:1/964: mkdir d4/d3c/d141 0 2024-05-27T09:29:54.975 INFO:tasks.workunit.client.1.smithi138.stdout:3/950: write d0/d2/da/f6b [539349,30687] 0 2024-05-27T09:29:54.977 INFO:tasks.workunit.client.1.smithi138.stdout:3/951: creat d0/dda/d11d/f138 x:0 0 0 2024-05-27T09:29:54.977 INFO:tasks.workunit.client.1.smithi138.stdout:3/952: write d0/d2/da/df/d10/d2a/d61/fb5 [926916,26640] 0 2024-05-27T09:29:54.979 INFO:tasks.workunit.client.1.smithi138.stdout:3/953: mkdir d0/d3f/d60/d93/de5/d139 0 2024-05-27T09:29:54.981 INFO:tasks.workunit.client.1.smithi138.stdout:3/954: mkdir d0/d2/da/df/d100/d13a 0 2024-05-27T09:29:54.993 INFO:tasks.workunit.client.1.smithi138.stdout:0/890: dwrite da/d2a/d8e/d1f/d42/d4a/d53/f87 [0,4194304] 0 2024-05-27T09:29:54.995 INFO:tasks.workunit.client.1.smithi138.stdout:0/891: mknod da/d2a/d8e/dda/ddf/c11c 0 2024-05-27T09:29:54.999 INFO:tasks.workunit.client.1.smithi138.stdout:0/892: symlink da/d2a/d8e/d1f/dd5/df7/l11d 0 2024-05-27T09:29:55.002 INFO:tasks.workunit.client.1.smithi138.stdout:0/893: creat da/d2a/d8e/d1f/dd5/df7/f11e x:0 0 0 2024-05-27T09:29:55.005 INFO:tasks.workunit.client.1.smithi138.stdout:0/894: mkdir da/d2a/d8e/d1f/d11f 0 2024-05-27T09:29:55.023 INFO:tasks.workunit.client.1.smithi138.stdout:0/895: write da/d2a/d55/fdc [3787514,79473] 0 2024-05-27T09:29:55.023 INFO:tasks.workunit.client.1.smithi138.stdout:0/896: write da/d2a/d8e/d1f/d42/d4a/fbb [148591,6098] 0 2024-05-27T09:29:55.024 INFO:tasks.workunit.client.1.smithi138.stdout:0/897: mkdir da/d2a/d8e/d1f/dd5/df7/d120 0 2024-05-27T09:29:55.026 INFO:tasks.workunit.client.1.smithi138.stdout:0/898: creat da/d17/def/f121 x:0 0 0 2024-05-27T09:29:55.026 INFO:tasks.workunit.client.1.smithi138.stdout:0/899: chown da/d2a/d8e/dda/ddf/fe4 10 1 2024-05-27T09:29:55.027 INFO:tasks.workunit.client.1.smithi138.stdout:0/900: link da/d2a/d8e/d1f/d4f/l114 da/d2a/d8e/d1f/d60/l122 0 2024-05-27T09:29:55.029 INFO:tasks.workunit.client.1.smithi138.stdout:0/901: read da/d2a/d8e/d1f/d71/f93 [90820,113431] 0 2024-05-27T09:29:55.031 INFO:tasks.workunit.client.1.smithi138.stdout:3/955: dwrite d0/d2/da/df/d10/f124 [0,4194304] 0 2024-05-27T09:29:55.031 INFO:tasks.workunit.client.1.smithi138.stdout:3/956: write d0/d2/da/df/fe2 [4639187,43771] 0 2024-05-27T09:29:55.031 INFO:tasks.workunit.client.1.smithi138.stdout:3/957: stat d0/d2/da/df/d10/d2a/d61/d73/deb/f110 0 2024-05-27T09:29:55.031 INFO:tasks.workunit.client.1.smithi138.stdout:0/902: creat da/d2a/d8e/d43/f123 x:0 0 0 2024-05-27T09:29:55.035 INFO:tasks.workunit.client.1.smithi138.stdout:3/958: creat d0/d2/da/df/f13b x:0 0 0 2024-05-27T09:29:55.035 INFO:tasks.workunit.client.1.smithi138.stdout:3/959: chown d0/d2/d105 1185 1 2024-05-27T09:29:55.037 INFO:tasks.workunit.client.1.smithi138.stdout:3/960: unlink d0/d2/da/df/d10/d2a/f53 0 2024-05-27T09:29:55.054 INFO:tasks.workunit.client.1.smithi138.stdout:0/903: read da/d2a/d8e/d1f/fde [1499043,37941] 0 2024-05-27T09:29:55.056 INFO:tasks.workunit.client.1.smithi138.stdout:0/904: creat da/d2a/d8e/d1f/d42/d63/dea/f124 x:0 0 0 2024-05-27T09:29:55.056 INFO:tasks.workunit.client.1.smithi138.stdout:0/905: stat da/d2a/d8e/d1f/d42/d63/dea/f124 0 2024-05-27T09:29:55.056 INFO:tasks.workunit.client.1.smithi138.stdout:0/906: truncate da/d2a/d8e/d1f/d42/d4a/fbb 441792 0 2024-05-27T09:29:55.056 INFO:tasks.workunit.client.1.smithi138.stdout:0/907: stat da/d2a/d8e/d1f/d42/d4a/d53/c6b 0 2024-05-27T09:29:55.057 INFO:tasks.workunit.client.1.smithi138.stdout:0/908: symlink da/d17/d112/daf/l125 0 2024-05-27T09:29:55.057 INFO:tasks.workunit.client.1.smithi138.stdout:0/909: write da/d2a/d55/fdc [4407943,63609] 0 2024-05-27T09:29:55.057 INFO:tasks.workunit.client.1.smithi138.stdout:0/910: dread - da/d2a/d55/f113 zero size 2024-05-27T09:29:55.059 INFO:tasks.workunit.client.1.smithi138.stdout:0/911: write da/d2a/d3f/f7c [495146,42718] 0 2024-05-27T09:29:55.059 INFO:tasks.workunit.client.1.smithi138.stdout:0/912: chown da/d2a/d3f 0 1 2024-05-27T09:29:55.063 INFO:tasks.workunit.client.1.smithi138.stdout:0/913: getdents da/d2a/dc9/dca 0 2024-05-27T09:29:55.070 INFO:tasks.workunit.client.1.smithi138.stdout:0/914: mkdir da/d2a/d8e/d1f/d4f/d126 0 2024-05-27T09:29:55.076 INFO:tasks.workunit.client.1.smithi138.stdout:0/915: mkdir da/d2a/d8e/d1f/d4f/d127 0 2024-05-27T09:29:55.076 INFO:tasks.workunit.client.1.smithi138.stdout:0/916: truncate da/d2a/d2b/fb7 421405 0 2024-05-27T09:29:55.086 INFO:tasks.workunit.client.1.smithi138.stdout:0/917: rmdir da/d17/d112/dc5 0 2024-05-27T09:29:55.086 INFO:tasks.workunit.client.1.smithi138.stdout:0/918: dread - da/d2a/dc9/dca/ffb zero size 2024-05-27T09:29:55.091 INFO:tasks.workunit.client.1.smithi138.stdout:3/961: dwrite d0/fa4 [0,4194304] 0 2024-05-27T09:29:55.091 INFO:tasks.workunit.client.1.smithi138.stdout:0/919: unlink da/d17/d112/d99/fb2 0 2024-05-27T09:29:55.091 INFO:tasks.workunit.client.1.smithi138.stdout:0/920: truncate da/d2a/d8e/d43/fe5 251032 0 2024-05-27T09:29:55.091 INFO:tasks.workunit.client.1.smithi138.stdout:0/921: chown da/d2a/d8e/d1f/d42/d4a/d53/f6e 252 1 2024-05-27T09:29:55.094 INFO:tasks.workunit.client.1.smithi138.stdout:3/962: rename d0/d3f/d60/d93/de5/fd3 to d0/d2/da/df/d10/d32/d37/f13c 0 2024-05-27T09:29:55.096 INFO:tasks.workunit.client.1.smithi138.stdout:0/922: creat da/d2a/d8e/d1f/d42/d63/f128 x:0 0 0 2024-05-27T09:29:55.096 INFO:tasks.workunit.client.1.smithi138.stdout:3/963: creat d0/d3f/f13d x:0 0 0 2024-05-27T09:29:55.098 INFO:tasks.workunit.client.1.smithi138.stdout:3/964: symlink d0/d2/da/df/d10/d2a/db2/dd1/l13e 0 2024-05-27T09:29:55.098 INFO:tasks.workunit.client.1.smithi138.stdout:3/965: fdatasync d0/d2/da/df/d10/fe9 0 2024-05-27T09:29:55.105 INFO:tasks.workunit.client.1.smithi138.stdout:0/923: truncate da/d2a/d8e/d1f/d42/f9a 656334 0 2024-05-27T09:29:55.106 INFO:tasks.workunit.client.1.smithi138.stdout:3/966: rename d0/d3f/f7f to d0/f13f 0 2024-05-27T09:29:55.107 INFO:tasks.workunit.client.1.smithi138.stdout:0/924: mkdir da/d2a/d8e/dc0/d116/d129 0 2024-05-27T09:29:55.115 INFO:tasks.workunit.client.1.smithi138.stdout:0/925: link da/d2a/d8e/d1f/d4f/ca5 da/d2a/d8e/d1f/dd5/df7/c12a 0 2024-05-27T09:29:55.119 INFO:tasks.workunit.client.1.smithi138.stdout:0/926: rename da/f66 to da/d2a/d8e/d1f/dd5/df7/f12b 0 2024-05-27T09:29:55.124 INFO:tasks.workunit.client.1.smithi138.stdout:0/927: rename da/d2a/d55/dc7/cd4 to da/d2a/d8e/d1f/d4f/d127/c12c 0 2024-05-27T09:29:55.131 INFO:tasks.workunit.client.1.smithi138.stdout:0/928: creat da/d2a/d55/dc7/f12d x:0 0 0 2024-05-27T09:29:55.131 INFO:tasks.workunit.client.1.smithi138.stdout:0/929: chown da/d2a/d8e/d1f/d42/d4a/c65 927 1 2024-05-27T09:29:55.135 INFO:tasks.workunit.client.1.smithi138.stdout:0/930: rename f1 to da/d17/d112/d99/f12e 0 2024-05-27T09:29:55.135 INFO:tasks.workunit.client.1.smithi138.stdout:0/931: stat da/d2a/d8e/d1f 0 2024-05-27T09:29:55.138 INFO:tasks.workunit.client.1.smithi138.stdout:0/932: chown c6 72 1 2024-05-27T09:29:55.146 INFO:tasks.workunit.client.1.smithi138.stdout:3/967: dwrite d0/d3f/d60/dcb/fdb [0,4194304] 0 2024-05-27T09:29:55.162 INFO:tasks.workunit.client.1.smithi138.stdout:3/968: write d0/d2/da/f1b [187239,5063] 0 2024-05-27T09:29:55.165 INFO:tasks.workunit.client.1.smithi138.stdout:3/969: symlink d0/d3f/d50/l140 0 2024-05-27T09:29:55.165 INFO:tasks.workunit.client.1.smithi138.stdout:3/970: chown d0/d3f/d50/f63 2744557 1 2024-05-27T09:29:55.167 INFO:tasks.workunit.client.1.smithi138.stdout:3/971: creat d0/d2/da/df/d10/d2a/db2/dd1/f141 x:0 0 0 2024-05-27T09:29:55.169 INFO:tasks.workunit.client.1.smithi138.stdout:3/972: symlink d0/d2/da/df/d100/d13a/l142 0 2024-05-27T09:29:55.171 INFO:tasks.workunit.client.1.smithi138.stdout:3/973: creat d0/d2/da/f143 x:0 0 0 2024-05-27T09:29:55.174 INFO:tasks.workunit.client.1.smithi138.stdout:3/974: rename d0/d2/da/c68 to d0/d3f/d60/d93/de5/c144 0 2024-05-27T09:29:55.174 INFO:tasks.workunit.client.1.smithi138.stdout:3/975: chown d0/d3f/d60/d93 44405721 1 2024-05-27T09:29:55.176 INFO:tasks.workunit.client.1.smithi138.stdout:3/976: rename d0/dda/d106 to d0/d2/da/df/d10/d2a/db2/dd1/dd8/d145 0 2024-05-27T09:29:55.180 INFO:tasks.workunit.client.1.smithi138.stdout:3/977: rename d0/d3f/d60/dcb/l112 to d0/d2/da/df/d10/d2a/d48/d90/l146 0 2024-05-27T09:29:55.183 INFO:tasks.workunit.client.1.smithi138.stdout:3/978: rename d0/d3f/d50/d9b/d9f/fc9 to d0/d3f/d50/d9b/d123/f147 0 2024-05-27T09:29:55.183 INFO:tasks.workunit.client.1.smithi138.stdout:3/979: creat d0/d2/da/df/d10/d2a/db2/dd1/dd8/f148 x:0 0 0 2024-05-27T09:29:55.189 INFO:tasks.workunit.client.1.smithi138.stdout:3/980: rmdir d0/d2/da/df/d10/d32 39 2024-05-27T09:29:55.189 INFO:tasks.workunit.client.1.smithi138.stdout:3/981: truncate d0/d2/da/df/f13b 726220 0 2024-05-27T09:29:55.189 INFO:tasks.workunit.client.1.smithi138.stdout:3/982: rename d0/d3f to d0/d3f/d60/d93/de5/dca/d149 22 2024-05-27T09:29:55.208 INFO:tasks.workunit.client.1.smithi138.stdout:3/983: read d0/fa4 [5859537,21755] 0 2024-05-27T09:29:55.285 INFO:tasks.workunit.client.1.smithi138.stdout:3/984: dwrite d0/d3f/d60/d93/de5/fe3 [0,4194304] 0 2024-05-27T09:29:55.285 INFO:tasks.workunit.client.1.smithi138.stdout:3/985: creat d0/d2/d105/f14a x:0 0 0 2024-05-27T09:29:55.287 INFO:tasks.workunit.client.1.smithi138.stdout:0/933: sync 2024-05-27T09:29:55.287 INFO:tasks.workunit.client.1.smithi138.stdout:2/773: sync 2024-05-27T09:29:55.288 INFO:tasks.workunit.client.1.smithi138.stdout:1/965: sync 2024-05-27T09:29:55.288 INFO:tasks.workunit.client.1.smithi138.stdout:0/934: chown da/d2a/d8e/d1f/d42/d63/l9f 170508019 1 2024-05-27T09:29:55.288 INFO:tasks.workunit.client.1.smithi138.stdout:2/774: chown d1/d15/d2d/d17/d25/d4d/d4e/d60 106718549 1 2024-05-27T09:29:55.289 INFO:tasks.workunit.client.1.smithi138.stdout:0/935: creat da/d2a/d8e/d1f/d42/d63/d9d/f12f x:0 0 0 2024-05-27T09:29:55.293 INFO:tasks.workunit.client.1.smithi138.stdout:3/986: truncate d0/d3f/f55 121856 0 2024-05-27T09:29:55.295 INFO:tasks.workunit.client.1.smithi138.stdout:0/936: write da/d2a/d2b/f4d [634565,35140] 0 2024-05-27T09:29:55.295 INFO:tasks.workunit.client.1.smithi138.stdout:0/937: fdatasync da/d2a/d8e/d1f/d42/d4a/d53/f6e 0 2024-05-27T09:29:55.297 INFO:tasks.workunit.client.1.smithi138.stdout:0/938: mknod da/d2a/d3f/c130 0 2024-05-27T09:29:55.301 INFO:tasks.workunit.client.1.smithi138.stdout:3/987: creat d0/d3f/d60/f14b x:0 0 0 2024-05-27T09:29:55.301 INFO:tasks.workunit.client.1.smithi138.stdout:0/939: symlink da/d2a/d8e/d1f/d42/d63/dea/l131 0 2024-05-27T09:29:55.305 INFO:tasks.workunit.client.1.smithi138.stdout:0/940: symlink da/d2a/d55/dc7/l132 0 2024-05-27T09:29:55.310 INFO:tasks.workunit.client.1.smithi138.stdout:2/775: dread d1/d10/d52/d5d/d7e/f84 [0,4194304] 0 2024-05-27T09:29:55.310 INFO:tasks.workunit.client.1.smithi138.stdout:2/776: creat d1/d10/d52/d5d/d7e/f108 x:0 0 0 2024-05-27T09:29:55.310 INFO:tasks.workunit.client.1.smithi138.stdout:0/941: truncate da/d2a/d8e/d1f/d42/d4a/d53/f90 291877 0 2024-05-27T09:29:55.310 INFO:tasks.workunit.client.1.smithi138.stdout:0/942: read - da/d2a/dc9/f10d zero size 2024-05-27T09:29:55.314 INFO:tasks.workunit.client.1.smithi138.stdout:0/943: mkdir da/d2a/d55/dc7/d133 0 2024-05-27T09:29:55.314 INFO:tasks.workunit.client.1.smithi138.stdout:0/944: chown da/d2a/d8e/d1f/d42/d63/d9d/ca3 655 1 2024-05-27T09:29:55.318 INFO:tasks.workunit.client.1.smithi138.stdout:2/777: creat d1/d15/d2d/d17/df3/f109 x:0 0 0 2024-05-27T09:29:55.323 INFO:tasks.workunit.client.1.smithi138.stdout:2/778: unlink d1/d10/d52/d5d/d7e/l101 0 2024-05-27T09:29:55.328 INFO:tasks.workunit.client.1.smithi138.stdout:2/779: symlink d1/d15/d2d/d17/d25/d4d/l10a 0 2024-05-27T09:29:55.339 INFO:tasks.workunit.client.1.smithi138.stdout:2/780: dread d1/d15/d2d/d17/dac/d104/f89 [0,4194304] 0 2024-05-27T09:29:55.348 INFO:tasks.workunit.client.1.smithi138.stdout:2/781: unlink d1/d15/d2d/d17/d25/d9d/ca2 0 2024-05-27T09:29:55.348 INFO:tasks.workunit.client.1.smithi138.stdout:2/782: chown d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc/c98 13626 1 2024-05-27T09:29:55.351 INFO:tasks.workunit.client.1.smithi138.stdout:2/783: unlink d1/l42 0 2024-05-27T09:29:55.351 INFO:tasks.workunit.client.1.smithi138.stdout:2/784: chown d1/d15/d2d/d17/d25/d9d/f5b 4075641 1 2024-05-27T09:29:55.365 INFO:tasks.workunit.client.1.smithi138.stdout:0/945: dwrite da/d2a/d8e/d1f/d42/d4a/d53/f90 [0,4194304] 0 2024-05-27T09:29:55.367 INFO:tasks.workunit.client.1.smithi138.stdout:0/946: getdents da/d2a/d8e/d43/d74/d97/df1 0 2024-05-27T09:29:55.369 INFO:tasks.workunit.client.1.smithi138.stdout:0/947: rename da/d2a/d8e/d1f/d60/c67 to da/d17/d112/daf/c134 0 2024-05-27T09:29:55.374 INFO:tasks.workunit.client.1.smithi138.stdout:2/785: read d1/d10/d52/d5d/fd8 [868443,127064] 0 2024-05-27T09:29:55.374 INFO:tasks.workunit.client.1.smithi138.stdout:2/786: dread - d1/d15/d2d/d17/f9b zero size 2024-05-27T09:29:55.376 INFO:tasks.workunit.client.1.smithi138.stdout:3/988: dwrite d0/d2/da/f33 [0,4194304] 0 2024-05-27T09:29:55.376 INFO:tasks.workunit.client.1.smithi138.stdout:3/989: fdatasync d0/dda/f115 0 2024-05-27T09:29:55.376 INFO:tasks.workunit.client.1.smithi138.stdout:3/990: chown d0/d3f/d50/l9c 52221 1 2024-05-27T09:29:55.376 INFO:tasks.workunit.client.1.smithi138.stdout:3/991: chown d0/d2/c6 982459 1 2024-05-27T09:29:55.376 INFO:tasks.workunit.client.1.smithi138.stdout:3/992: write d0/d3f/d60/d93/de5/f136 [162863,51939] 0 2024-05-27T09:29:55.376 INFO:tasks.workunit.client.1.smithi138.stdout:3/993: readlink d0/d2/da/df/d10/d2a/db2/dd1/dd8/d145/l122 0 2024-05-27T09:29:55.376 INFO:tasks.workunit.client.1.smithi138.stdout:2/787: link d1/d15/d2d/d17/d25/d4d/d4e/d60/lfd d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/l10b 0 2024-05-27T09:29:55.380 INFO:tasks.workunit.client.1.smithi138.stdout:2/788: unlink d1/d15/lfe 0 2024-05-27T09:29:55.381 INFO:tasks.workunit.client.1.smithi138.stdout:3/994: rmdir d0/d80 39 2024-05-27T09:29:55.381 INFO:tasks.workunit.client.1.smithi138.stdout:3/995: readlink d0/d2/da/df/d10/d2a/d48/d90/lb3 0 2024-05-27T09:29:55.384 INFO:tasks.workunit.client.1.smithi138.stdout:2/789: creat d1/d15/d2d/d17/d25/d4d/f10c x:0 0 0 2024-05-27T09:29:55.386 INFO:tasks.workunit.client.1.smithi138.stdout:3/996: creat d0/d3f/d50/d9b/d9f/f14c x:0 0 0 2024-05-27T09:29:55.389 INFO:tasks.workunit.client.1.smithi138.stdout:2/790: mknod d1/d15/d2d/d17/d25/d4d/d4e/d60/c10d 0 2024-05-27T09:29:55.389 INFO:tasks.workunit.client.1.smithi138.stdout:2/791: readlink d1/l18 0 2024-05-27T09:29:55.434 INFO:tasks.workunit.client.1.smithi138.stdout:2/792: dwrite d1/d15/d2d/d17/d25/d4d/ff0 [0,4194304] 0 2024-05-27T09:29:55.439 INFO:tasks.workunit.client.1.smithi138.stdout:2/793: rename d1/d15/d2d/d17/c107 to d1/d15/d2d/d17/d25/c10e 0 2024-05-27T09:29:55.439 INFO:tasks.workunit.client.1.smithi138.stdout:2/794: write d1/d15/d2d/d17/d25/d4d/d4e/ffa [49848,19349] 0 2024-05-27T09:29:55.441 INFO:tasks.workunit.client.1.smithi138.stdout:2/795: unlink d1/d15/d2d/d17/d25/d4d/d4e/d60/f92 0 2024-05-27T09:29:55.441 INFO:tasks.workunit.client.1.smithi138.stdout:2/796: stat d1/d10/d52/fdb 0 2024-05-27T09:29:55.449 INFO:tasks.workunit.client.1.smithi138.stdout:0/948: dwrite da/d2a/d3f/f4c [4194304,4194304] 0 2024-05-27T09:29:55.454 INFO:tasks.workunit.client.1.smithi138.stdout:3/997: dwrite d0/d3f/f98 [0,4194304] 0 2024-05-27T09:29:55.480 INFO:tasks.workunit.client.1.smithi138.stdout:3/998: dread d0/d3f/d50/d9b/d9f/f117 [0,4194304] 0 2024-05-27T09:29:55.526 INFO:tasks.workunit.client.1.smithi138.stdout:2/797: rename d1/d15/d2d/d17/d25/d4d/d4e/d60/lfd to d1/d15/d2d/d17/dac/d104/l10f 0 2024-05-27T09:29:55.526 INFO:tasks.workunit.client.1.smithi138.stdout:2/798: truncate d1/d15/d2d/d17/dac/d104/f62 4952644 0 2024-05-27T09:29:55.526 INFO:tasks.workunit.client.1.smithi138.stdout:2/799: truncate d1/d15/d2d/d17/d25/d4d/d6d/fb7 4961722 0 2024-05-27T09:29:55.527 INFO:tasks.workunit.client.1.smithi138.stdout:2/800: mkdir d1/d15/d2d/d17/dac/d110 0 2024-05-27T09:29:55.535 INFO:tasks.workunit.client.1.smithi138.stdout:0/949: rename da/d2a/d2b/c2f to da/d2a/d55/dc7/d133/c135 0 2024-05-27T09:29:55.535 INFO:tasks.workunit.client.1.smithi138.stdout:0/950: chown c6 62269 1 2024-05-27T09:29:55.555 INFO:tasks.workunit.client.1.smithi138.stdout:1/966: sync 2024-05-27T09:29:55.561 INFO:tasks.workunit.client.1.smithi138.stdout:1/967: unlink d4/d1a/d2e/l9e 0 2024-05-27T09:29:55.561 INFO:tasks.workunit.client.1.smithi138.stdout:1/968: write d4/d48/d39/d5f/d63/fa3 [4251940,128326] 0 2024-05-27T09:29:55.583 INFO:tasks.workunit.client.1.smithi138.stdout:1/969: read d4/d1a/d2e/d4a/f80 [256226,31702] 0 2024-05-27T09:29:55.588 INFO:tasks.workunit.client.1.smithi138.stdout:1/970: getdents d4/d1a/d2e/d32/d8c/dc6 0 2024-05-27T09:29:55.588 INFO:tasks.workunit.client.1.smithi138.stdout:1/971: chown d4/dcc/f11b 1311970147 1 2024-05-27T09:29:55.591 INFO:tasks.workunit.client.1.smithi138.stdout:1/972: rmdir d4/d48 39 2024-05-27T09:29:55.591 INFO:tasks.workunit.client.1.smithi138.stdout:1/973: fdatasync d4/d48/f104 0 2024-05-27T09:29:55.606 INFO:tasks.workunit.client.1.smithi138.stdout:1/974: dread d4/d1a/d2e/d4a/d7e/dc0/f119 [0,4194304] 0 2024-05-27T09:29:55.606 INFO:tasks.workunit.client.1.smithi138.stdout:1/975: readlink d4/l1d 0 2024-05-27T09:29:55.626 INFO:tasks.workunit.client.1.smithi138.stdout:2/801: dwrite d1/d15/d2d/d17/d25/d4d/d6d/d87/fc0 [0,4194304] 0 2024-05-27T09:29:55.626 INFO:tasks.workunit.client.1.smithi138.stdout:2/802: fsync d1/d10/d52/d5d/fbd 0 2024-05-27T09:29:55.631 INFO:tasks.workunit.client.1.smithi138.stdout:2/803: rename d1/d15/d2d/dd2/fdd to d1/d15/d2d/d17/d25/d4d/d6d/f111 0 2024-05-27T09:29:55.632 INFO:tasks.workunit.client.1.smithi138.stdout:2/804: symlink d1/d10/l112 0 2024-05-27T09:29:55.683 INFO:tasks.workunit.client.1.smithi138.stdout:1/976: dwrite d4/d88/ff2 [0,4194304] 0 2024-05-27T09:29:55.690 INFO:tasks.workunit.client.1.smithi138.stdout:1/977: creat d4/d88/f142 x:0 0 0 2024-05-27T09:29:55.690 INFO:tasks.workunit.client.1.smithi138.stdout:1/978: chown d4/d48/d39/d5f/d93/fb2 2045744 1 2024-05-27T09:29:55.690 INFO:tasks.workunit.client.1.smithi138.stdout:1/979: creat d4/d1a/d2e/d4a/d7e/db7/dfb/f143 x:0 0 0 2024-05-27T09:29:55.696 INFO:tasks.workunit.client.1.smithi138.stdout:1/980: mkdir d4/d48/d22/d65/d144 0 2024-05-27T09:29:55.696 INFO:tasks.workunit.client.1.smithi138.stdout:1/981: stat d4/dc7/cd1 0 2024-05-27T09:29:55.698 INFO:tasks.workunit.client.1.smithi138.stdout:2/805: dwrite d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc/f83 [0,4194304] 0 2024-05-27T09:29:55.701 INFO:tasks.workunit.client.1.smithi138.stdout:2/806: rename d1/d15/d2d/d17/d25/d4d/d6d/fba to d1/d15/d2d/d7b/f113 0 2024-05-27T09:29:55.702 INFO:tasks.workunit.client.1.smithi138.stdout:1/982: mkdir d4/d1a/d2e/d32/d8c/dc6/d145 0 2024-05-27T09:29:55.702 INFO:tasks.workunit.client.1.smithi138.stdout:1/983: readlink d4/d1a/d2e/d4a/d7e/lb3 0 2024-05-27T09:29:55.707 INFO:tasks.workunit.client.1.smithi138.stdout:1/984: rename d4/d48/l31 to d4/d48/d39/d5f/dd3/l146 0 2024-05-27T09:29:55.707 INFO:tasks.workunit.client.1.smithi138.stdout:1/985: readlink d4/d1a/d2e/d32/d8c/l137 0 2024-05-27T09:29:55.712 INFO:tasks.workunit.client.1.smithi138.stdout:2/807: dread d1/d15/d2d/d17/dac/d104/f89 [0,4194304] 0 2024-05-27T09:29:55.712 INFO:tasks.workunit.client.1.smithi138.stdout:2/808: stat d1/d15/d2d/d17/d25/d4d/d6d/d87/fbf 0 2024-05-27T09:29:55.712 INFO:tasks.workunit.client.1.smithi138.stdout:2/809: fsync d1/d15/d2d/d17/d25/f3a 0 2024-05-27T09:29:55.714 INFO:tasks.workunit.client.1.smithi138.stdout:2/810: unlink d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc/fb4 0 2024-05-27T09:29:55.739 INFO:tasks.workunit.client.1.smithi138.stdout:3/999: sync 2024-05-27T09:29:55.739 INFO:tasks.workunit.client.1.smithi138.stdout:0/951: sync 2024-05-27T09:29:55.743 INFO:tasks.workunit.client.1.smithi138.stdout:2/811: dread d1/d10/f6 [0,4194304] 0 2024-05-27T09:29:55.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:55 smithi119 ceph-mon[29082]: pgmap v25: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 101 MiB/s rd, 128 MiB/s wr, 239 op/s 2024-05-27T09:29:55.757 INFO:tasks.workunit.client.1.smithi138.stdout:1/986: dwrite d4/d48/d39/d5f/d4d/f3f [0,4194304] 0 2024-05-27T09:29:55.798 INFO:tasks.workunit.client.1.smithi138.stdout:0/952: dwrite da/d2a/d55/f95 [0,4194304] 0 2024-05-27T09:29:55.808 INFO:tasks.workunit.client.1.smithi138.stdout:2/812: dwrite d1/d15/d2d/d17/d25/d4d/d6d/d87/fbf [0,4194304] 0 2024-05-27T09:29:55.811 INFO:tasks.workunit.client.1.smithi138.stdout:2/813: link d1/d15/d2d/d17/def/c100 d1/d15/d2d/d17/d25/d4d/c114 0 2024-05-27T09:29:55.816 INFO:tasks.workunit.client.1.smithi138.stdout:1/987: dwrite d4/d1a/deb/f101 [0,4194304] 0 2024-05-27T09:29:55.821 INFO:tasks.workunit.client.1.smithi138.stdout:1/988: creat d4/d48/d39/d5f/d86/d134/f147 x:0 0 0 2024-05-27T09:29:55.823 INFO:tasks.workunit.client.1.smithi138.stdout:1/989: unlink d4/d1a/d2e/d32/d8c/dc6/fc2 0 2024-05-27T09:29:55.824 INFO:tasks.workunit.client.1.smithi138.stdout:1/990: dread - d4/d38/d97/dfe/f12d zero size 2024-05-27T09:29:55.862 INFO:tasks.workunit.client.1.smithi138.stdout:1/991: dread d4/d48/d39/d5f/f5b [0,4194304] 0 2024-05-27T09:29:55.864 INFO:tasks.workunit.client.1.smithi138.stdout:1/992: unlink d4/d88/f142 0 2024-05-27T09:29:55.864 INFO:tasks.workunit.client.1.smithi138.stdout:1/993: dread - d4/dc7/f106 zero size 2024-05-27T09:29:55.876 INFO:tasks.workunit.client.1.smithi138.stdout:2/814: dwrite d1/d15/d2d/d17/d25/d4d/ff0 [0,4194304] 0 2024-05-27T09:29:55.882 INFO:tasks.workunit.client.1.smithi138.stdout:1/994: read d4/d1a/d2e/f79 [882182,85759] 0 2024-05-27T09:29:55.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:55 smithi138 ceph-mon[34885]: pgmap v25: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 101 MiB/s rd, 128 MiB/s wr, 239 op/s 2024-05-27T09:29:55.886 INFO:tasks.workunit.client.1.smithi138.stdout:1/995: truncate d4/d1a/d2e/f79 70848 0 2024-05-27T09:29:55.887 INFO:tasks.workunit.client.1.smithi138.stdout:1/996: write d4/d38/fd4 [481719,93438] 0 2024-05-27T09:29:55.887 INFO:tasks.workunit.client.1.smithi138.stdout:2/815: rename d1/d10/d52/d5d to d1/d15/d2d/d17/dac/d104/d115 0 2024-05-27T09:29:55.891 INFO:tasks.workunit.client.1.smithi138.stdout:1/997: unlink d4/f78 0 2024-05-27T09:29:55.891 INFO:tasks.workunit.client.1.smithi138.stdout:1/998: write d4/d1a/d2e/d4a/d7e/db7/dfb/f103 [11381,57081] 0 2024-05-27T09:29:55.891 INFO:tasks.workunit.client.1.smithi138.stdout:1/999: chown d4/daa/cec 133363793 1 2024-05-27T09:29:55.926 INFO:tasks.workunit.client.1.smithi138.stdout:2/816: dwrite d1/d15/dd6/f99 [0,4194304] 0 2024-05-27T09:29:55.926 INFO:tasks.workunit.client.1.smithi138.stdout:2/817: chown d1/d15/d2d/d17/d25/d9d 1365360 1 2024-05-27T09:29:55.929 INFO:tasks.workunit.client.1.smithi138.stdout:2/818: creat d1/d15/f116 x:0 0 0 2024-05-27T09:29:55.932 INFO:tasks.workunit.client.1.smithi138.stdout:2/819: rename d1/d15/d2d/d17/def/c100 to d1/d15/d2d/d17/d25/d9d/d4b/c117 0 2024-05-27T09:29:55.936 INFO:tasks.workunit.client.1.smithi138.stdout:2/820: mkdir d1/d15/d2d/d17/df3/d118 0 2024-05-27T09:29:55.939 INFO:tasks.workunit.client.1.smithi138.stdout:2/821: link d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc/l8f d1/d15/d2d/d17/dac/d110/l119 0 2024-05-27T09:29:55.941 INFO:tasks.workunit.client.1.smithi138.stdout:2/822: creat d1/d15/d2d/d17/dac/d104/d115/f11a x:0 0 0 2024-05-27T09:29:55.941 INFO:tasks.workunit.client.1.smithi138.stdout:2/823: stat d1/d15/d2d/d17/d25/d4d/d4e/d60/l61 0 2024-05-27T09:29:55.941 INFO:tasks.workunit.client.1.smithi138.stdout:2/824: dread - d1/d15/d2d/d17/d25/d4d/f10c zero size 2024-05-27T09:29:56.009 INFO:tasks.workunit.client.1.smithi138.stdout:2/825: dwrite d1/d15/d2d/d17/dac/d104/d115/f91 [0,4194304] 0 2024-05-27T09:29:56.028 INFO:tasks.workunit.client.1.smithi138.stdout:0/953: sync 2024-05-27T09:29:56.032 INFO:tasks.workunit.client.1.smithi138.stdout:0/954: rename da/fb8 to da/d2a/d55/dc7/d133/f136 0 2024-05-27T09:29:56.032 INFO:tasks.workunit.client.1.smithi138.stdout:0/955: chown da/d2a/d8e/d1f/d71/f93 1260 1 2024-05-27T09:29:56.034 INFO:tasks.workunit.client.1.smithi138.stdout:0/956: creat da/d2a/d8e/d1f/d71/f137 x:0 0 0 2024-05-27T09:29:56.100 INFO:tasks.workunit.client.1.smithi138.stdout:0/957: dwrite da/d17/d112/daf/dc4/fc8 [0,4194304] 0 2024-05-27T09:29:56.100 INFO:tasks.workunit.client.1.smithi138.stdout:0/958: stat da/d17/d112/daf/dc4 0 2024-05-27T09:29:56.100 INFO:tasks.workunit.client.1.smithi138.stdout:0/959: truncate da/d2a/d8e/d1f/d71/f93 604815 0 2024-05-27T09:29:56.102 INFO:tasks.workunit.client.1.smithi138.stdout:0/960: link da/d2a/d8e/d1f/d42/d4a/d53/f57 da/d2a/dc9/dca/f138 0 2024-05-27T09:29:56.148 INFO:tasks.workunit.client.1.smithi138.stdout:0/961: dwrite da/d2a/d8e/dc0/fce [0,4194304] 0 2024-05-27T09:29:56.151 INFO:tasks.workunit.client.1.smithi138.stdout:0/962: getdents da/d2a/d8e/dda/ddf 0 2024-05-27T09:29:56.155 INFO:tasks.workunit.client.1.smithi138.stdout:0/963: rmdir da/d2a/d8e/dc0 39 2024-05-27T09:29:56.155 INFO:tasks.workunit.client.1.smithi138.stdout:0/964: fdatasync da/d2a/d2b/f33 0 2024-05-27T09:29:56.157 INFO:tasks.workunit.client.1.smithi138.stdout:0/965: truncate da/d2a/d8e/dda/ddf/fe4 3494229 0 2024-05-27T09:29:56.159 INFO:tasks.workunit.client.1.smithi138.stdout:0/966: mkdir da/d2a/d8e/d1f/d42/d139 0 2024-05-27T09:29:56.306 INFO:tasks.workunit.client.1.smithi138.stdout:2/826: sync 2024-05-27T09:29:56.306 INFO:tasks.workunit.client.1.smithi138.stdout:2/827: creat d1/d15/d2d/d17/dac/f11b x:0 0 0 2024-05-27T09:29:56.311 INFO:tasks.workunit.client.1.smithi138.stdout:2/828: creat d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/f11c x:0 0 0 2024-05-27T09:29:56.360 INFO:tasks.workunit.client.1.smithi138.stdout:2/829: read d1/d10/d52/fa6 [4716614,60748] 0 2024-05-27T09:29:56.365 INFO:tasks.workunit.client.1.smithi138.stdout:2/830: getdents d1/d15/d2d/d17/d25/d4d/d4e 0 2024-05-27T09:29:56.365 INFO:tasks.workunit.client.1.smithi138.stdout:2/831: stat d1/d15/dd6/fca 0 2024-05-27T09:29:56.377 INFO:tasks.workunit.client.1.smithi138.stdout:0/967: sync 2024-05-27T09:29:56.381 INFO:tasks.workunit.client.1.smithi138.stdout:0/968: link da/d2a/d8e/d1f/d42/d63/dea/l131 da/d17/def/l13a 0 2024-05-27T09:29:56.381 INFO:tasks.workunit.client.1.smithi138.stdout:0/969: write da/d2a/d8e/d1f/d42/d63/dea/f124 [355555,82155] 0 2024-05-27T09:29:56.385 INFO:tasks.workunit.client.1.smithi138.stdout:0/970: mkdir da/d2a/d8e/d1f/d42/d63/d9d/d13b 0 2024-05-27T09:29:56.459 INFO:tasks.workunit.client.1.smithi138.stdout:2/832: dwrite d1/d15/dd6/f43 [0,4194304] 0 2024-05-27T09:29:56.459 INFO:tasks.workunit.client.1.smithi138.stdout:2/833: chown d1/d15/d2d/d17/d25/d4d/cb5 0 1 2024-05-27T09:29:56.468 INFO:tasks.workunit.client.1.smithi138.stdout:2/834: rename d1/d15/d2d/d17/d25/d4d/d4e/lf5 to d1/d15/d2d/d17/def/l11d 0 2024-05-27T09:29:56.469 INFO:tasks.workunit.client.1.smithi138.stdout:0/971: dwrite da/d2a/d8e/d1f/d42/d4a/fbb [0,4194304] 0 2024-05-27T09:29:56.538 INFO:tasks.workunit.client.1.smithi138.stdout:2/835: dwrite d1/d15/d2d/d17/d25/d9d/f3b [0,4194304] 0 2024-05-27T09:29:56.538 INFO:tasks.workunit.client.1.smithi138.stdout:2/836: stat d1/d10/f64 0 2024-05-27T09:29:56.550 INFO:tasks.workunit.client.1.smithi138.stdout:0/972: dwrite da/d2a/d3f/f7c [0,4194304] 0 2024-05-27T09:29:56.569 INFO:tasks.workunit.client.1.smithi138.stdout:0/973: getdents da/d2a/d8e/d43/d74 0 2024-05-27T09:29:56.575 INFO:tasks.workunit.client.1.smithi138.stdout:0/974: creat da/d2a/d8e/d1f/d60/f13c x:0 0 0 2024-05-27T09:29:56.575 INFO:tasks.workunit.client.1.smithi138.stdout:0/975: write da/d2a/d8e/d1f/ff5 [1436939,65274] 0 2024-05-27T09:29:56.597 INFO:tasks.workunit.client.1.smithi138.stdout:2/837: dwrite d1/d10/d52/fa6 [0,4194304] 0 2024-05-27T09:29:56.597 INFO:tasks.workunit.client.1.smithi138.stdout:2/838: stat d1/d15/d2d/d17/d25/d9d/d4b 0 2024-05-27T09:29:56.597 INFO:tasks.workunit.client.1.smithi138.stdout:2/839: write d1/d15/d2d/d17/dac/d104/f89 [1378818,53614] 0 2024-05-27T09:29:56.600 INFO:tasks.workunit.client.1.smithi138.stdout:2/840: creat d1/d10/f11e x:0 0 0 2024-05-27T09:29:56.605 INFO:tasks.workunit.client.1.smithi138.stdout:0/976: dread da/d17/d112/d99/f12e [4194304,4194304] 0 2024-05-27T09:29:56.624 INFO:tasks.workunit.client.1.smithi138.stdout:2/841: dread d1/d15/d2d/d17/d25/d9d/d4b/fbe [0,4194304] 0 2024-05-27T09:29:56.653 INFO:tasks.workunit.client.1.smithi138.stdout:0/977: dwrite da/d2a/d8e/d43/f48 [0,4194304] 0 2024-05-27T09:29:56.663 INFO:tasks.workunit.client.1.smithi138.stdout:0/978: mkdir da/d2a/d8e/d1f/d42/d63/d9d/d13d 0 2024-05-27T09:29:56.663 INFO:tasks.workunit.client.1.smithi138.stdout:0/979: creat da/d2a/d8e/d1f/d42/d63/d9d/f13e x:0 0 0 2024-05-27T09:29:56.663 INFO:tasks.workunit.client.1.smithi138.stdout:0/980: fsync da/d2a/d8e/d1f/d42/d4a/f117 0 2024-05-27T09:29:56.684 INFO:tasks.workunit.client.1.smithi138.stdout:2/842: dwrite d1/d15/d2d/d17/d25/d4d/d6d/d87/fbf [4194304,4194304] 0 2024-05-27T09:29:56.689 INFO:tasks.workunit.client.1.smithi138.stdout:2/843: mkdir d1/d15/d2d/d2e/d102/d11f 0 2024-05-27T09:29:56.691 INFO:tasks.workunit.client.1.smithi138.stdout:2/844: mknod d1/d15/d2d/d17/d25/c120 0 2024-05-27T09:29:56.695 INFO:tasks.workunit.client.1.smithi138.stdout:2/845: link d1/d10/d52/lad d1/d10/l121 0 2024-05-27T09:29:56.697 INFO:tasks.workunit.client.1.smithi138.stdout:2/846: truncate d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc/f7d 4263183 0 2024-05-27T09:29:56.699 INFO:tasks.workunit.client.1.smithi138.stdout:2/847: getdents d1/d15/dd6/dec 0 2024-05-27T09:29:56.707 INFO:tasks.workunit.client.1.smithi138.stdout:2/848: rename d1/d15/d2d/d17/dac/d104/d115/d7e/f84 to d1/d15/d2d/d17/dac/f122 0 2024-05-27T09:29:56.757 INFO:tasks.workunit.client.1.smithi138.stdout:2/849: dwrite d1/d15/d2d/d17/d25/f74 [0,4194304] 0 2024-05-27T09:29:56.757 INFO:tasks.workunit.client.1.smithi138.stdout:2/850: chown d1/d15/d2d/l78 46955 1 2024-05-27T09:29:56.757 INFO:tasks.workunit.client.1.smithi138.stdout:2/851: dread - d1/d15/d2d/d7b/f113 zero size 2024-05-27T09:29:56.757 INFO:tasks.workunit.client.1.smithi138.stdout:2/852: stat d1/d15/d2d/d17/d25/d4d/d4e/f81 0 2024-05-27T09:29:56.760 INFO:tasks.workunit.client.1.smithi138.stdout:2/853: truncate d1/d15/d2d/d2e/f37 1307590 0 2024-05-27T09:29:56.765 INFO:tasks.workunit.client.1.smithi138.stdout:2/854: dread d1/d15/d2d/d17/d25/d4d/d4e/ffa [0,4194304] 0 2024-05-27T09:29:56.783 INFO:tasks.workunit.client.1.smithi138.stdout:2/855: dread d1/d15/d2d/d17/d25/f2a [0,4194304] 0 2024-05-27T09:29:56.797 INFO:tasks.workunit.client.1.smithi138.stdout:2/856: write d1/d15/d2d/d2e/f50 [3061620,34453] 0 2024-05-27T09:29:56.802 INFO:tasks.workunit.client.1.smithi138.stdout:2/857: dread d1/d15/d2d/f39 [4194304,4194304] 0 2024-05-27T09:29:56.802 INFO:tasks.workunit.client.1.smithi138.stdout:2/858: readlink d1/d10/l121 0 2024-05-27T09:29:56.844 INFO:tasks.workunit.client.1.smithi138.stdout:2/859: dwrite d1/d15/d2d/d17/d25/d4d/d6d/d87/fc0 [4194304,4194304] 0 2024-05-27T09:29:56.845 INFO:tasks.workunit.client.1.smithi138.stdout:2/860: mknod d1/d10/d52/db9/c123 0 2024-05-27T09:29:56.846 INFO:tasks.workunit.client.1.smithi138.stdout:2/861: mkdir d1/d15/d124 0 2024-05-27T09:29:56.847 INFO:tasks.workunit.client.1.smithi138.stdout:2/862: mknod d1/d15/d2d/d17/dac/c125 0 2024-05-27T09:29:56.864 INFO:tasks.workunit.client.1.smithi138.stdout:2/863: write d1/d15/dd6/f99 [1833659,105173] 0 2024-05-27T09:29:56.866 INFO:tasks.workunit.client.1.smithi138.stdout:2/864: creat d1/d15/d2d/d17/d25/d9d/d4b/f126 x:0 0 0 2024-05-27T09:29:56.868 INFO:tasks.workunit.client.1.smithi138.stdout:2/865: creat d1/d15/d2d/d2e/d102/f127 x:0 0 0 2024-05-27T09:29:56.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:56 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:29:56.888 INFO:tasks.workunit.client.1.smithi138.stdout:2/866: dread d1/d15/d2d/f9f [0,4194304] 0 2024-05-27T09:29:56.891 INFO:tasks.workunit.client.1.smithi138.stdout:2/867: rename d1/d15/d2d/d17/d25/d4d/d6d/d87/fbf to d1/d10/d52/f128 0 2024-05-27T09:29:56.893 INFO:tasks.workunit.client.1.smithi138.stdout:2/868: truncate d1/d15/d2d/d17/dac/d104/d115/f91 1374961 0 2024-05-27T09:29:56.895 INFO:tasks.workunit.client.1.smithi138.stdout:2/869: symlink d1/d15/l129 0 2024-05-27T09:29:56.895 INFO:tasks.workunit.client.1.smithi138.stdout:2/870: fdatasync d1/d15/d2d/d17/d25/f74 0 2024-05-27T09:29:56.910 INFO:tasks.workunit.client.1.smithi138.stdout:2/871: dread d1/d15/d2d/f9f [0,4194304] 0 2024-05-27T09:29:56.912 INFO:tasks.workunit.client.1.smithi138.stdout:2/872: symlink d1/d15/d2d/d17/dac/d104/d115/d7e/l12a 0 2024-05-27T09:29:56.925 INFO:tasks.workunit.client.1.smithi138.stdout:2/873: rmdir d1 39 2024-05-27T09:29:56.925 INFO:tasks.workunit.client.1.smithi138.stdout:2/874: chown d1/d15/d2d/d17/d25/d9d/f3b 19890 1 2024-05-27T09:29:56.933 INFO:tasks.workunit.client.1.smithi138.stdout:0/981: sync 2024-05-27T09:29:56.935 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:56 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:29:56.936 INFO:tasks.workunit.client.1.smithi138.stdout:0/982: getdents da/d2a/d55/dc7/d133 0 2024-05-27T09:29:56.936 INFO:tasks.workunit.client.1.smithi138.stdout:0/983: write da/d2a/dc9/f10d [196935,32451] 0 2024-05-27T09:29:56.991 INFO:tasks.workunit.client.1.smithi138.stdout:2/875: dwrite d1/d15/d2d/d7b/f97 [0,4194304] 0 2024-05-27T09:29:56.991 INFO:tasks.workunit.client.1.smithi138.stdout:2/876: fdatasync d1/d15/d2d/d17/d25/d9d/d4b/fa1 0 2024-05-27T09:29:56.992 INFO:tasks.workunit.client.1.smithi138.stdout:0/984: dwrite da/d17/d112/d99/f12e [4194304,4194304] 0 2024-05-27T09:29:56.996 INFO:tasks.workunit.client.1.smithi138.stdout:2/877: creat d1/d15/d2d/d17/df3/d118/f12b x:0 0 0 2024-05-27T09:29:56.996 INFO:tasks.workunit.client.1.smithi138.stdout:2/878: write d1/d10/fe [8784142,73889] 0 2024-05-27T09:29:56.998 INFO:tasks.workunit.client.1.smithi138.stdout:2/879: rename d1/d15/dd6/c5e to d1/d15/d2d/d17/dac/df2/c12c 0 2024-05-27T09:29:56.998 INFO:tasks.workunit.client.1.smithi138.stdout:2/880: read d1/d10/d52/db9/fcf [1441253,98820] 0 2024-05-27T09:29:57.001 INFO:tasks.workunit.client.1.smithi138.stdout:2/881: unlink d1/d15/f48 0 2024-05-27T09:29:57.002 INFO:tasks.workunit.client.1.smithi138.stdout:2/882: chown d1/d15/d2d/d17/d25/d4d/d4e/d60/c9c 0 1 2024-05-27T09:29:57.004 INFO:tasks.workunit.client.1.smithi138.stdout:2/883: rename d1/d15/d2d/d17/dac/d104 to d1/d15/d2d/d17/d25/d4d/d6d/d12d 0 2024-05-27T09:29:57.007 INFO:tasks.workunit.client.1.smithi138.stdout:2/884: rename d1/d15/d2d/d17/d25/ld5 to d1/d15/d2d/d2e/l12e 0 2024-05-27T09:29:57.009 INFO:tasks.workunit.client.1.smithi138.stdout:2/885: creat d1/f12f x:0 0 0 2024-05-27T09:29:57.011 INFO:tasks.workunit.client.1.smithi138.stdout:2/886: mknod d1/d15/d2d/d17/d25/d9d/df9/c130 0 2024-05-27T09:29:57.011 INFO:tasks.workunit.client.1.smithi138.stdout:2/887: dread - d1/d15/d2d/d17/d25/d4d/d6d/d12d/d115/d7e/f108 zero size 2024-05-27T09:29:57.011 INFO:tasks.workunit.client.1.smithi138.stdout:2/888: truncate d1/d15/d2d/d17/d25/d4d/d6d/d87/fae 327564 0 2024-05-27T09:29:57.030 INFO:tasks.workunit.client.1.smithi138.stdout:2/889: read d1/d15/d2d/d17/d25/d4d/d4e/d60/f66 [2349384,85539] 0 2024-05-27T09:29:57.049 INFO:tasks.workunit.client.1.smithi138.stdout:0/985: symlink da/d2a/d8e/d1f/d42/d139/l13f 0 2024-05-27T09:29:57.049 INFO:tasks.workunit.client.1.smithi138.stdout:0/986: write da/d2a/f8f [1140631,75635] 0 2024-05-27T09:29:57.050 INFO:tasks.workunit.client.1.smithi138.stdout:0/987: write da/d2a/d8e/d1f/d42/d63/f128 [525953,33612] 0 2024-05-27T09:29:57.050 INFO:tasks.workunit.client.1.smithi138.stdout:0/988: write da/f20 [1087673,17984] 0 2024-05-27T09:29:57.093 INFO:tasks.workunit.client.1.smithi138.stdout:2/890: dwrite d1/d10/f6 [0,4194304] 0 2024-05-27T09:29:57.093 INFO:tasks.workunit.client.1.smithi138.stdout:2/891: write d1/d10/fe [8886628,119460] 0 2024-05-27T09:29:57.095 INFO:tasks.workunit.client.1.smithi138.stdout:2/892: creat d1/d15/d2d/d2e/d102/f131 x:0 0 0 2024-05-27T09:29:57.096 INFO:tasks.workunit.client.1.smithi138.stdout:2/893: rmdir d1/d15/d2d/d17/df3 39 2024-05-27T09:29:57.096 INFO:tasks.workunit.client.1.smithi138.stdout:2/894: fsync d1/d15/d2d/d17/d25/d4d/d6d/fe4 0 2024-05-27T09:29:57.097 INFO:tasks.workunit.client.1.smithi138.stdout:2/895: readlink d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/l10b 0 2024-05-27T09:29:57.098 INFO:tasks.workunit.client.1.smithi138.stdout:2/896: creat d1/d15/d2d/d17/df3/f132 x:0 0 0 2024-05-27T09:29:57.099 INFO:tasks.workunit.client.1.smithi138.stdout:2/897: symlink d1/d15/d2d/d17/df3/d118/l133 0 2024-05-27T09:29:57.100 INFO:tasks.workunit.client.1.smithi138.stdout:2/898: creat d1/d15/d2d/d17/dac/f134 x:0 0 0 2024-05-27T09:29:57.101 INFO:tasks.workunit.client.1.smithi138.stdout:2/899: symlink d1/d15/d2d/d17/d25/d4d/d4e/d60/l135 0 2024-05-27T09:29:57.101 INFO:tasks.workunit.client.1.smithi138.stdout:2/900: write d1/ffb [787378,42696] 0 2024-05-27T09:29:57.106 INFO:tasks.workunit.client.1.smithi138.stdout:0/989: unlink da/d2a/d8e/d1f/d42/d63/l9f 0 2024-05-27T09:29:57.106 INFO:tasks.workunit.client.1.smithi138.stdout:0/990: chown da/d2a/dc9/dca/f138 101945 1 2024-05-27T09:29:57.107 INFO:tasks.workunit.client.1.smithi138.stdout:0/991: truncate da/d2a/d8e/d1f/ff5 143300 0 2024-05-27T09:29:57.123 INFO:tasks.workunit.client.1.smithi138.stdout:2/901: dread d1/d15/d2d/fa0 [0,4194304] 0 2024-05-27T09:29:57.125 INFO:tasks.workunit.client.1.smithi138.stdout:2/902: creat d1/d15/d2d/d17/d25/d4d/d6d/d87/f136 x:0 0 0 2024-05-27T09:29:57.126 INFO:tasks.workunit.client.1.smithi138.stdout:2/903: truncate d1/d15/f116 692358 0 2024-05-27T09:29:57.126 INFO:tasks.workunit.client.1.smithi138.stdout:2/904: fsync d1/d15/dd6/f94 0 2024-05-27T09:29:57.130 INFO:tasks.workunit.client.1.smithi138.stdout:0/992: dread da/f20 [0,4194304] 0 2024-05-27T09:29:57.130 INFO:tasks.workunit.client.1.smithi138.stdout:0/993: fsync da/d2a/d8e/d1f/d42/d4a/d53/f57 0 2024-05-27T09:29:57.172 INFO:tasks.workunit.client.1.smithi138.stdout:0/994: dwrite da/d2a/d8e/d1f/f119 [0,4194304] 0 2024-05-27T09:29:57.173 INFO:tasks.workunit.client.1.smithi138.stdout:0/995: chown da/d2a/d8e/d1f/d42/d4a/l54 2477560 1 2024-05-27T09:29:57.173 INFO:tasks.workunit.client.1.smithi138.stdout:0/996: readlink da/l8d 0 2024-05-27T09:29:57.186 INFO:tasks.workunit.client.1.smithi138.stdout:0/997: write da/d2a/d8e/d1f/f38 [1135103,72395] 0 2024-05-27T09:29:57.186 INFO:tasks.workunit.client.1.smithi138.stdout:0/998: write da/d2a/d8e/d1f/f119 [5009570,68212] 0 2024-05-27T09:29:57.256 INFO:tasks.workunit.client.1.smithi138.stdout:0/999: dwrite da/d2a/d8e/d1f/d42/f9a [0,4194304] 0 2024-05-27T09:29:57.398 INFO:tasks.workunit.client.1.smithi138.stdout:2/905: sync 2024-05-27T09:29:57.458 INFO:tasks.workunit.client.1.smithi138.stdout:2/906: read d1/d15/d2d/d17/f58 [1230596,67040] 0 2024-05-27T09:29:57.458 INFO:tasks.workunit.client.1.smithi138.stdout:2/907: fdatasync d1/d15/d2d/d2e/f50 0 2024-05-27T09:29:57.460 INFO:tasks.workunit.client.1.smithi138.stdout:2/908: rename d1/d15/d2d/dd2 to d1/d15/dd6/d137 0 2024-05-27T09:29:57.462 INFO:tasks.workunit.client.1.smithi138.stdout:2/909: link d1/d15/d2d/d17/d25/d9d/c75 d1/d15/d2d/d17/d25/d4d/d4e/c138 0 2024-05-27T09:29:57.464 INFO:tasks.workunit.client.1.smithi138.stdout:2/910: mknod d1/d15/d2d/d2e/c139 0 2024-05-27T09:29:57.500 INFO:tasks.workunit.client.1.smithi138.stdout:2/911: dread d1/d15/d2d/d17/d25/d4d/d6d/fab [0,4194304] 0 2024-05-27T09:29:57.599 INFO:tasks.workunit.client.1.smithi138.stdout:2/912: dwrite d1/d15/d2d/d17/d25/f28 [0,4194304] 0 2024-05-27T09:29:57.599 INFO:tasks.workunit.client.1.smithi138.stdout:2/913: write d1/d15/d2d/d17/d25/d4d/f10c [319194,10630] 0 2024-05-27T09:29:57.671 INFO:tasks.workunit.client.1.smithi138.stdout:2/914: dwrite d1/dd4/ff7 [0,4194304] 0 2024-05-27T09:29:57.673 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:57 smithi119 ceph-mon[29082]: pgmap v26: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 101 MiB/s rd, 128 MiB/s wr, 239 op/s 2024-05-27T09:29:57.675 INFO:tasks.workunit.client.1.smithi138.stdout:2/915: mkdir d1/d10/d52/d13a 0 2024-05-27T09:29:57.675 INFO:tasks.workunit.client.1.smithi138.stdout:2/916: fdatasync d1/d15/d2d/d17/d25/d4d/db6/fed 0 2024-05-27T09:29:57.675 INFO:tasks.workunit.client.1.smithi138.stdout:2/917: write d1/d15/d2d/d17/dac/f11b [349099,15431] 0 2024-05-27T09:29:57.676 INFO:tasks.workunit.client.1.smithi138.stdout:2/918: write d1/d10/fe [9419008,81875] 0 2024-05-27T09:29:57.676 INFO:tasks.workunit.client.1.smithi138.stdout:2/919: fdatasync d1/d15/d2d/d17/dac/f134 0 2024-05-27T09:29:57.680 INFO:tasks.workunit.client.1.smithi138.stdout:2/920: rename d1/d15/d2d/c56 to d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/c13b 0 2024-05-27T09:29:57.683 INFO:tasks.workunit.client.1.smithi138.stdout:2/921: link d1/d15/d2d/d17/d25/d4d/d4e/d60/l61 d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc/l13c 0 2024-05-27T09:29:57.701 INFO:tasks.workunit.client.1.smithi138.stdout:2/922: dread d1/d15/d2d/d17/d25/f3a [0,4194304] 0 2024-05-27T09:29:57.701 INFO:tasks.workunit.client.1.smithi138.stdout:2/923: creat d1/d15/d2d/d17/df3/d118/f13d x:0 0 0 2024-05-27T09:29:57.701 INFO:tasks.workunit.client.1.smithi138.stdout:2/924: fdatasync d1/d15/d2d/d17/f58 0 2024-05-27T09:29:57.770 INFO:tasks.workunit.client.1.smithi138.stdout:2/925: dwrite d1/d15/d2d/d17/d25/d4d/f10c [0,4194304] 0 2024-05-27T09:29:57.771 INFO:tasks.workunit.client.1.smithi138.stdout:2/926: mkdir d1/d15/d2d/d17/dac/d110/d13e 0 2024-05-27T09:29:57.774 INFO:tasks.workunit.client.1.smithi138.stdout:2/927: unlink d1/d15/d2d/d17/d25/d4d/d6d/ccb 0 2024-05-27T09:29:57.788 INFO:tasks.workunit.client.1.smithi138.stdout:2/928: write d1/d15/d2d/d17/d25/d9d/d4b/fbe [1219745,117616] 0 2024-05-27T09:29:57.788 INFO:tasks.workunit.client.1.smithi138.stdout:2/929: dread - d1/d15/d2d/d17/dac/df2/ff8 zero size 2024-05-27T09:29:57.790 INFO:tasks.workunit.client.1.smithi138.stdout:2/930: mkdir d1/dd4/d13f 0 2024-05-27T09:29:57.790 INFO:tasks.workunit.client.1.smithi138.stdout:2/931: fsync d1/d15/d2d/d17/d25/d4d/d6d/fab 0 2024-05-27T09:29:57.790 INFO:tasks.workunit.client.1.smithi138.stdout:2/932: dread - d1/d15/d2d/d7b/f113 zero size 2024-05-27T09:29:57.790 INFO:tasks.workunit.client.1.smithi138.stdout:2/933: write d1/d15/d2d/d17/d25/d4d/d6d/d12d/d115/f88 [6033371,3771] 0 2024-05-27T09:29:57.793 INFO:tasks.workunit.client.1.smithi138.stdout:2/934: mkdir d1/dd4/d140 0 2024-05-27T09:29:57.794 INFO:tasks.workunit.client.1.smithi138.stdout:2/935: symlink d1/d15/d2d/d17/d25/d4d/d6d/d12d/l141 0 2024-05-27T09:29:57.795 INFO:tasks.workunit.client.1.smithi138.stdout:2/936: write d1/d15/d2d/d2e/d102/f127 [617014,38570] 0 2024-05-27T09:29:57.795 INFO:tasks.workunit.client.1.smithi138.stdout:2/937: fdatasync d1/d15/d2d/d17/df3/f132 0 2024-05-27T09:29:57.797 INFO:tasks.workunit.client.1.smithi138.stdout:2/938: rmdir d1/d15/d2d/d17/d25/d9d/d4b 39 2024-05-27T09:29:57.797 INFO:tasks.workunit.client.1.smithi138.stdout:2/939: chown d1/d15/d2d/d17/d25/d4d/d4e/d60 89 1 2024-05-27T09:29:57.799 INFO:tasks.workunit.client.1.smithi138.stdout:2/940: symlink d1/d15/d2d/d17/dac/d110/l142 0 2024-05-27T09:29:57.800 INFO:tasks.workunit.client.1.smithi138.stdout:2/941: write d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/ffc [674612,28873] 0 2024-05-27T09:29:57.802 INFO:tasks.workunit.client.1.smithi138.stdout:2/942: chown d1/d10/d52/db9 1 1 2024-05-27T09:29:57.805 INFO:tasks.workunit.client.1.smithi138.stdout:2/943: read - d1/d10/d52/fff zero size 2024-05-27T09:29:57.878 INFO:tasks.workunit.client.1.smithi138.stdout:2/944: dwrite d1/d15/d2d/f8e [0,4194304] 0 2024-05-27T09:29:57.880 INFO:tasks.workunit.client.1.smithi138.stdout:2/945: mknod d1/d15/d2d/d17/df3/c143 0 2024-05-27T09:29:57.881 INFO:tasks.workunit.client.1.smithi138.stdout:2/946: rename d1/d15/d2d to d1/d15/d2d/d2e/d102/d144 22 2024-05-27T09:29:57.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:57 smithi138 ceph-mon[34885]: pgmap v26: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 101 MiB/s rd, 128 MiB/s wr, 239 op/s 2024-05-27T09:29:57.901 INFO:tasks.workunit.client.1.smithi138.stdout:2/947: write d1/d15/dd6/f43 [1675713,33711] 0 2024-05-27T09:29:57.962 INFO:tasks.workunit.client.1.smithi138.stdout:2/948: dwrite d1/d10/d52/fdb [0,4194304] 0 2024-05-27T09:29:57.964 INFO:tasks.workunit.client.1.smithi138.stdout:2/949: rmdir d1/d15/d2d/d17/d25/d4d 39 2024-05-27T09:29:57.964 INFO:tasks.workunit.client.1.smithi138.stdout:2/950: chown d1/d10 556292 1 2024-05-27T09:29:57.967 INFO:tasks.workunit.client.1.smithi138.stdout:2/951: mkdir d1/d15/d2d/d17/dac/df2/d145 0 2024-05-27T09:29:57.967 INFO:tasks.workunit.client.1.smithi138.stdout:2/952: chown d1/d10/d52/fff 1976577 1 2024-05-27T09:29:57.967 INFO:tasks.workunit.client.1.smithi138.stdout:2/953: creat d1/d15/d2d/d17/df3/d118/f146 x:0 0 0 2024-05-27T09:29:57.967 INFO:tasks.workunit.client.1.smithi138.stdout:2/954: write d1/d15/dd6/f99 [4535251,116936] 0 2024-05-27T09:29:57.973 INFO:tasks.workunit.client.1.smithi138.stdout:2/955: symlink d1/d15/d2d/d17/d25/d4d/d6d/d12d/d115/d7e/l147 0 2024-05-27T09:29:57.977 INFO:tasks.workunit.client.1.smithi138.stdout:2/956: link d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc/ld9 d1/d15/d2d/d17/df3/d118/l148 0 2024-05-27T09:29:57.977 INFO:tasks.workunit.client.1.smithi138.stdout:2/957: stat d1/d15/d2d/d17/df3/d118 0 2024-05-27T09:29:57.980 INFO:tasks.workunit.client.1.smithi138.stdout:2/958: creat d1/d15/d2d/f149 x:0 0 0 2024-05-27T09:29:58.357 INFO:tasks.workunit.client.1.smithi138.stdout:2/959: sync 2024-05-27T09:29:58.362 INFO:tasks.workunit.client.1.smithi138.stdout:2/960: link d1/l80 d1/d15/d2d/d17/d25/d4d/d6d/l14a 0 2024-05-27T09:29:58.364 INFO:tasks.workunit.client.1.smithi138.stdout:2/961: symlink d1/d15/d2d/d7b/l14b 0 2024-05-27T09:29:58.389 INFO:tasks.workunit.client.1.smithi138.stdout:2/962: write d1/d15/d2d/f8e [1090163,31889] 0 2024-05-27T09:29:58.396 INFO:tasks.workunit.client.1.smithi138.stdout:2/963: creat d1/d10/d52/db9/f14c x:0 0 0 2024-05-27T09:29:58.544 INFO:tasks.workunit.client.1.smithi138.stdout:2/964: sync 2024-05-27T09:29:58.544 INFO:tasks.workunit.client.1.smithi138.stdout:2/965: chown d1/d15/d2d/d17/d25/d4d/db6 146197 1 2024-05-27T09:29:58.545 INFO:tasks.workunit.client.1.smithi138.stdout:2/966: rmdir d1/d15/d2d/d17/d25/d4d/d6d/d12d/d115 39 2024-05-27T09:29:58.548 INFO:tasks.workunit.client.1.smithi138.stdout:2/967: creat d1/d15/d2d/d17/f14d x:0 0 0 2024-05-27T09:29:58.548 INFO:tasks.workunit.client.1.smithi138.stdout:2/968: write d1/d15/d2d/d17/d25/d4d/d6d/fe4 [496739,109841] 0 2024-05-27T09:29:58.549 INFO:tasks.workunit.client.1.smithi138.stdout:2/969: creat d1/f14e x:0 0 0 2024-05-27T09:29:58.549 INFO:tasks.workunit.client.1.smithi138.stdout:2/970: stat d1/d15/d2d 0 2024-05-27T09:29:58.549 INFO:tasks.workunit.client.1.smithi138.stdout:2/971: chown d1/d15/d2d/d7b/l14b 33069858 1 2024-05-27T09:29:58.628 INFO:tasks.workunit.client.1.smithi138.stdout:2/972: dwrite d1/d15/d2d/d17/d25/d4d/d6d/fab [0,4194304] 0 2024-05-27T09:29:58.628 INFO:tasks.workunit.client.1.smithi138.stdout:2/973: stat d1/d15/d2d/d2e/d102/f131 0 2024-05-27T09:29:58.630 INFO:tasks.workunit.client.1.smithi138.stdout:2/974: symlink d1/d15/d2d/l14f 0 2024-05-27T09:29:58.633 INFO:tasks.workunit.client.1.smithi138.stdout:2/975: mkdir d1/d15/d2d/d17/d150 0 2024-05-27T09:29:58.633 INFO:tasks.workunit.client.1.smithi138.stdout:2/976: fsync d1/d15/d2d/d17/dac/f134 0 2024-05-27T09:29:58.633 INFO:tasks.workunit.client.1.smithi138.stdout:2/977: readlink d1/d15/d2d/d17/d25/d9d/l5a 0 2024-05-27T09:29:58.633 INFO:tasks.workunit.client.1.smithi138.stdout:2/978: readlink d1/d15/d2d/d17/d25/d4d/d6d/d12d/d115/d7e/l147 0 2024-05-27T09:29:58.636 INFO:tasks.workunit.client.1.smithi138.stdout:2/979: mkdir d1/d15/d2d/d17/df3/d151 0 2024-05-27T09:29:58.636 INFO:tasks.workunit.client.1.smithi138.stdout:2/980: truncate d1/d15/dd6/f94 3994004 0 2024-05-27T09:29:58.638 INFO:tasks.workunit.client.1.smithi138.stdout:2/981: mkdir d1/d10/d52/d13a/d152 0 2024-05-27T09:29:58.638 INFO:tasks.workunit.client.1.smithi138.stdout:2/982: write d1/d15/d2d/d17/df3/d118/f13d [361574,61972] 0 2024-05-27T09:29:58.638 INFO:tasks.workunit.client.1.smithi138.stdout:2/983: stat d1/d15/d2d/d17/d25/d4d/d4e/d60/la7 0 2024-05-27T09:29:58.641 INFO:tasks.workunit.client.1.smithi138.stdout:2/984: creat d1/d15/d2d/d17/d25/f153 x:0 0 0 2024-05-27T09:29:58.642 INFO:tasks.workunit.client.1.smithi138.stdout:2/985: symlink d1/d10/d52/d13a/l154 0 2024-05-27T09:29:58.663 INFO:tasks.workunit.client.1.smithi138.stdout:2/986: dread d1/d15/d2d/f8e [0,4194304] 0 2024-05-27T09:29:58.667 INFO:tasks.workunit.client.1.smithi138.stdout:2/987: link d1/d15/d2d/d17/d25/d4d/d4e/d60/c9c d1/d15/d2d/d17/d25/d4d/d4e/d60/c155 0 2024-05-27T09:29:58.668 INFO:tasks.workunit.client.1.smithi138.stdout:2/988: chown d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/ce2 1 1 2024-05-27T09:29:58.731 INFO:tasks.workunit.client.1.smithi138.stdout:2/989: dwrite d1/d15/d2d/d17/d25/d4d/d6d/d12d/d115/f88 [0,4194304] 0 2024-05-27T09:29:58.736 INFO:tasks.workunit.client.1.smithi138.stdout:2/990: truncate d1/d15/d2d/d17/d25/d9d/d4b/fbe 1431780 0 2024-05-27T09:29:58.736 INFO:tasks.workunit.client.1.smithi138.stdout:2/991: chown d1/f7a 183031 1 2024-05-27T09:29:58.736 INFO:tasks.workunit.client.1.smithi138.stdout:2/992: stat d1/d15/d2d/d17/cd3 0 2024-05-27T09:29:58.737 INFO:tasks.workunit.client.1.smithi138.stdout:2/993: write d1/d10/f11e [327550,96838] 0 2024-05-27T09:29:58.737 INFO:tasks.workunit.client.1.smithi138.stdout:2/994: fsync d1/d15/d2d/d17/d25/d4d/d4e/d60/dc8/dcc/f33 0 2024-05-27T09:29:58.759 INFO:tasks.workunit.client.1.smithi138.stdout:2/995: dread d1/d15/d2d/d17/d25/d4d/ff0 [0,4194304] 0 2024-05-27T09:29:58.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:29:58 smithi138 ceph-mon[34885]: pgmap v27: 65 pgs: 65 active+clean; 3.2 GiB data, 9.9 GiB used, 527 GiB / 536 GiB avail; 126 MiB/s rd, 169 MiB/s wr, 331 op/s 2024-05-27T09:29:58.932 INFO:tasks.workunit.client.1.smithi138.stdout:2/996: sync 2024-05-27T09:29:58.932 INFO:tasks.workunit.client.1.smithi138.stdout:2/997: chown d1/d15/d2d/d17/dac/l106 48 1 2024-05-27T09:29:58.932 INFO:tasks.workunit.client.1.smithi138.stdout:2/998: fdatasync d1/f1b 0 2024-05-27T09:29:58.937 INFO:tasks.workunit.client.1.smithi138.stdout:2/999: link d1/d15/d2d/d17/d25/d4d/d6d/l14a d1/d15/d2d/d17/d25/d4d/d4e/d60/l156 0 2024-05-27T09:29:58.946 INFO:tasks.workunit.client.1.smithi138.stderr:+ rm -rf -- ./tmp.nUPTxQ7sni 2024-05-27T09:29:58.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:29:58 smithi119 ceph-mon[29082]: pgmap v27: 65 pgs: 65 active+clean; 3.2 GiB data, 9.9 GiB used, 527 GiB / 536 GiB avail; 126 MiB/s rd, 169 MiB/s wr, 331 op/s 2024-05-27T09:30:00.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:00 smithi138 ceph-mon[34885]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-27T09:30:00.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:00 smithi138 ceph-mon[34885]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:30:00.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:00 smithi138 ceph-mon[34885]: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:30:00.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:00 smithi119 ceph-mon[29082]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-27T09:30:00.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:00 smithi119 ceph-mon[29082]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:30:00.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:00 smithi119 ceph-mon[29082]: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:30:01.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:01 smithi119 ceph-mon[29082]: pgmap v28: 65 pgs: 65 active+clean; 3.2 GiB data, 9.9 GiB used, 527 GiB / 536 GiB avail; 73 MiB/s rd, 100 MiB/s wr, 210 op/s 2024-05-27T09:30:01.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:01 smithi138 ceph-mon[34885]: pgmap v28: 65 pgs: 65 active+clean; 3.2 GiB data, 9.9 GiB used, 527 GiB / 536 GiB avail; 73 MiB/s rd, 100 MiB/s wr, 210 op/s 2024-05-27T09:30:02.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:02 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:02.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:02 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:02.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:02 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-27T09:30:02.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:02 smithi138 ceph-mon[34885]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-27T09:30:02.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:02 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:30:02.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:02 smithi138 ceph-mon[34885]: Upgrade: Need to upgrade myself (mgr.smithi138.dxsuqf) 2024-05-27T09:30:02.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:02 smithi138 ceph-mon[34885]: Upgrade: Need to upgrade myself (mgr.smithi138.dxsuqf) 2024-05-27T09:30:02.449 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:02 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:02.450 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:02 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:02.450 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:02 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-27T09:30:02.450 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:02 smithi119 ceph-mon[29082]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-27T09:30:02.450 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:02 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:30:02.450 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:02 smithi119 ceph-mon[29082]: Upgrade: Need to upgrade myself (mgr.smithi138.dxsuqf) 2024-05-27T09:30:02.450 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:02 smithi119 ceph-mon[29082]: Upgrade: Need to upgrade myself (mgr.smithi138.dxsuqf) 2024-05-27T09:30:03.175 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-27T09:30:03.176 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-27T09:30:03.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:03 smithi119 ceph-mon[29082]: pgmap v29: 65 pgs: 65 active+clean; 2.4 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 79 MiB/s rd, 106 MiB/s wr, 288 op/s 2024-05-27T09:30:03.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:03 smithi138 ceph-mon[34885]: pgmap v29: 65 pgs: 65 active+clean; 2.4 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 79 MiB/s rd, 106 MiB/s wr, 288 op/s 2024-05-27T09:30:05.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:05 smithi138 ceph-mon[34885]: Upgrade: Updating mgr.smithi119.ncwmog 2024-05-27T09:30:05.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:05 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:05.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:05 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi119.ncwmog", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T09:30:05.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:05 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi119.ncwmog", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T09:30:05.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:05 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T09:30:05.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:05 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:30:05.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:05 smithi138 ceph-mon[34885]: Deploying daemon mgr.smithi119.ncwmog on smithi119 2024-05-27T09:30:05.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:05 smithi138 ceph-mon[34885]: pgmap v30: 65 pgs: 65 active+clean; 2.4 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 30 MiB/s rd, 46 MiB/s wr, 169 op/s 2024-05-27T09:30:05.441 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:05 smithi119 ceph-mon[29082]: Upgrade: Updating mgr.smithi119.ncwmog 2024-05-27T09:30:05.441 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:05 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:05.441 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:05 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi119.ncwmog", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T09:30:05.442 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:05 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi119.ncwmog", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T09:30:05.442 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:05 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T09:30:05.442 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:05 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:30:05.442 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:05 smithi119 ceph-mon[29082]: Deploying daemon mgr.smithi119.ncwmog on smithi119 2024-05-27T09:30:05.442 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:05 smithi119 ceph-mon[29082]: pgmap v30: 65 pgs: 65 active+clean; 2.4 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 30 MiB/s rd, 46 MiB/s wr, 169 op/s 2024-05-27T09:30:07.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:07 smithi119 ceph-mon[29082]: pgmap v31: 65 pgs: 65 active+clean; 2.4 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 30 MiB/s rd, 46 MiB/s wr, 169 op/s 2024-05-27T09:30:07.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:07 smithi138 ceph-mon[34885]: pgmap v31: 65 pgs: 65 active+clean; 2.4 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 30 MiB/s rd, 46 MiB/s wr, 169 op/s 2024-05-27T09:30:08.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:08 smithi138 ceph-mon[34885]: fsmap cephfs:2 {0=cephfs.smithi119.ppmzzk=up:active,1=cephfs.smithi138.ndfkhd=up:active} 2 up:standby 2024-05-27T09:30:08.472 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:08 smithi119 ceph-mon[29082]: fsmap cephfs:2 {0=cephfs.smithi119.ppmzzk=up:active,1=cephfs.smithi138.ndfkhd=up:active} 2 up:standby 2024-05-27T09:30:08.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.470+0000 7f2d92400700 1 -- 172.21.15.119:0/2990876408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c0737b0 msgr2=0x7f2d8c073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:08.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.470+0000 7f2d92400700 1 --2- 172.21.15.119:0/2990876408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c0737b0 0x7f2d8c073c20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009910 tx=0x7f2d74009c20 comp rx=0 tx=0).stop 2024-05-27T09:30:08.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.470+0000 7f2d92400700 1 -- 172.21.15.119:0/2990876408 shutdown_connections 2024-05-27T09:30:08.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.470+0000 7f2d92400700 1 --2- 172.21.15.119:0/2990876408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c0737b0 0x7f2d8c073c20 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:08.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.470+0000 7f2d92400700 1 --2- 172.21.15.119:0/2990876408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d8c074d80 0x7f2d8c0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:08.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.470+0000 7f2d92400700 1 -- 172.21.15.119:0/2990876408 >> 172.21.15.119:0/2990876408 conn(0x7f2d8c0fb6e0 msgr2=0x7f2d8c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:08.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.471+0000 7f2d92400700 1 -- 172.21.15.119:0/2990876408 shutdown_connections 2024-05-27T09:30:08.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.471+0000 7f2d92400700 1 -- 172.21.15.119:0/2990876408 wait complete. 2024-05-27T09:30:08.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.472+0000 7f2d92400700 1 Processor -- start 2024-05-27T09:30:08.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.473+0000 7f2d92400700 1 -- start start 2024-05-27T09:30:08.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.473+0000 7f2d92400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c0737b0 0x7f2d8c19bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:08.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.473+0000 7f2d92400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d8c074d80 0x7f2d8c19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:08.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.473+0000 7f2d92400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c19ca70 con 0x7f2d8c0737b0 2024-05-27T09:30:08.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.473+0000 7f2d92400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c19cbb0 con 0x7f2d8c074d80 2024-05-27T09:30:08.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.473+0000 7f2d8b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c0737b0 0x7f2d8c19bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:08.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.473+0000 7f2d8b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c0737b0 0x7f2d8c19bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60088/0 (socket says 172.21.15.119:60088) 2024-05-27T09:30:08.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.473+0000 7f2d8b400700 1 -- 172.21.15.119:0/3158845974 learned_addr learned my addr 172.21.15.119:0/3158845974 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:30:08.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.474+0000 7f2d8b400700 1 -- 172.21.15.119:0/3158845974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d8c074d80 msgr2=0x7f2d8c19c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:08.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.474+0000 7f2d8b400700 1 --2- 172.21.15.119:0/3158845974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d8c074d80 0x7f2d8c19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:08.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.474+0000 7f2d8b400700 1 -- 172.21.15.119:0/3158845974 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d740095f0 con 0x7f2d8c0737b0 2024-05-27T09:30:08.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.474+0000 7f2d8b400700 1 --2- 172.21.15.119:0/3158845974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c0737b0 0x7f2d8c19bf10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00c8a0 tx=0x7f2d7c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:08.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.475+0000 7f2d88c00700 1 -- 172.21.15.119:0/3158845974 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d7c00cea0 con 0x7f2d8c0737b0 2024-05-27T09:30:08.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.475+0000 7f2d92400700 1 -- 172.21.15.119:0/3158845974 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8c1a1660 con 0x7f2d8c0737b0 2024-05-27T09:30:08.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.475+0000 7f2d88c00700 1 -- 172.21.15.119:0/3158845974 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2d7c004830 con 0x7f2d8c0737b0 2024-05-27T09:30:08.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.475+0000 7f2d88c00700 1 -- 172.21.15.119:0/3158845974 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d7c0055b0 con 0x7f2d8c0737b0 2024-05-27T09:30:08.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.475+0000 7f2d92400700 1 -- 172.21.15.119:0/3158845974 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8c1a1bb0 con 0x7f2d8c0737b0 2024-05-27T09:30:08.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.477+0000 7f2d92400700 1 -- 172.21.15.119:0/3158845974 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d68005320 con 0x7f2d8c0737b0 2024-05-27T09:30:08.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.477+0000 7f2d88c00700 1 -- 172.21.15.119:0/3158845974 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) v1 ==== 95024+0+0 (secure 0 0 0) 0x7f2d7c007ad0 con 0x7f2d8c0737b0 2024-05-27T09:30:08.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.479+0000 7f2d88c00700 1 --2- 172.21.15.119:0/3158845974 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f2d700727d0 0x7f2d70074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:08.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.479+0000 7f2d88c00700 1 -- 172.21.15.119:0/3158845974 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f2d7c092f90 con 0x7f2d8c0737b0 2024-05-27T09:30:08.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.479+0000 7f2d83e00700 1 --2- 172.21.15.119:0/3158845974 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f2d700727d0 0x7f2d70074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:08.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.481+0000 7f2d83e00700 1 --2- 172.21.15.119:0/3158845974 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f2d700727d0 0x7f2d70074c80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2d74005090 tx=0x7f2d74005fb0 comp rx=0 tx=0).ready entity=mgr.14570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:08.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.484+0000 7f2d88c00700 1 -- 172.21.15.119:0/3158845974 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f2d7c05afa0 con 0x7f2d8c0737b0 2024-05-27T09:30:08.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.781+0000 7f2d92400700 1 -- 172.21.15.119:0/3158845974 --> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d68000bf0 con 0x7f2d700727d0 2024-05-27T09:30:08.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.785+0000 7f2d88c00700 1 -- 172.21.15.119:0/3158845974 <== mgr.14570 v2:172.21.15.138:6828/1384904888 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f2d68000bf0 con 0x7f2d700727d0 2024-05-27T09:30:08.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.788+0000 7f2d92400700 1 -- 172.21.15.119:0/3158845974 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f2d700727d0 msgr2=0x7f2d70074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:08.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.788+0000 7f2d92400700 1 --2- 172.21.15.119:0/3158845974 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f2d700727d0 0x7f2d70074c80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2d74005090 tx=0x7f2d74005fb0 comp rx=0 tx=0).stop 2024-05-27T09:30:08.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.788+0000 7f2d92400700 1 -- 172.21.15.119:0/3158845974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c0737b0 msgr2=0x7f2d8c19bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:08.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.788+0000 7f2d92400700 1 --2- 172.21.15.119:0/3158845974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c0737b0 0x7f2d8c19bf10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00c8a0 tx=0x7f2d7c00cc60 comp rx=0 tx=0).stop 2024-05-27T09:30:08.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.788+0000 7f2d92400700 1 -- 172.21.15.119:0/3158845974 shutdown_connections 2024-05-27T09:30:08.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.788+0000 7f2d92400700 1 --2- 172.21.15.119:0/3158845974 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f2d700727d0 0x7f2d70074c80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:08.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.788+0000 7f2d92400700 1 --2- 172.21.15.119:0/3158845974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d8c0737b0 0x7f2d8c19bf10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:08.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.788+0000 7f2d92400700 1 --2- 172.21.15.119:0/3158845974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d8c074d80 0x7f2d8c19c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:08.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.788+0000 7f2d92400700 1 -- 172.21.15.119:0/3158845974 >> 172.21.15.119:0/3158845974 conn(0x7f2d8c0fb6e0 msgr2=0x7f2d8c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:08.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.788+0000 7f2d92400700 1 -- 172.21.15.119:0/3158845974 shutdown_connections 2024-05-27T09:30:08.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.789+0000 7f2d92400700 1 -- 172.21.15.119:0/3158845974 wait complete. 2024-05-27T09:30:08.811 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:30:08.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.980+0000 7f95bb200700 1 -- 172.21.15.119:0/4021821458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b4103500 msgr2=0x7f95b41058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:08.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.980+0000 7f95bb200700 1 --2- 172.21.15.119:0/4021821458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b4103500 0x7f95b41058e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f95a0009910 tx=0x7f95a0009c20 comp rx=0 tx=0).stop 2024-05-27T09:30:08.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.981+0000 7f95bb200700 1 -- 172.21.15.119:0/4021821458 shutdown_connections 2024-05-27T09:30:08.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.981+0000 7f95bb200700 1 --2- 172.21.15.119:0/4021821458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b4103500 0x7f95b41058e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:08.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.981+0000 7f95bb200700 1 --2- 172.21.15.119:0/4021821458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95b4100be0 0x7f95b4102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:08.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.981+0000 7f95bb200700 1 -- 172.21.15.119:0/4021821458 >> 172.21.15.119:0/4021821458 conn(0x7f95b40fa5e0 msgr2=0x7f95b40fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:08.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.981+0000 7f95bb200700 1 -- 172.21.15.119:0/4021821458 shutdown_connections 2024-05-27T09:30:08.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.983+0000 7f95bb200700 1 -- 172.21.15.119:0/4021821458 wait complete. 2024-05-27T09:30:08.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.984+0000 7f95bb200700 1 Processor -- start 2024-05-27T09:30:08.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.984+0000 7f95bb200700 1 -- start start 2024-05-27T09:30:08.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.984+0000 7f95bb200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95b4100be0 0x7f95b4197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:08.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.984+0000 7f95bb200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b4103500 0x7f95b41980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:08.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.984+0000 7f95bb200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b41986c0 con 0x7f95b4103500 2024-05-27T09:30:08.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.984+0000 7f95bb200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b4198800 con 0x7f95b4100be0 2024-05-27T09:30:08.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.985+0000 7f95b3400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b4103500 0x7f95b41980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:08.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.985+0000 7f95b3400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b4103500 0x7f95b41980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42332/0 (socket says 172.21.15.119:42332) 2024-05-27T09:30:08.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.985+0000 7f95b3400700 1 -- 172.21.15.119:0/2730159863 learned_addr learned my addr 172.21.15.119:0/2730159863 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:30:08.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.985+0000 7f95b3e00700 1 --2- 172.21.15.119:0/2730159863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95b4100be0 0x7f95b4197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:08.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.986+0000 7f95b3400700 1 -- 172.21.15.119:0/2730159863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95b4100be0 msgr2=0x7f95b4197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:08.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.986+0000 7f95b3400700 1 --2- 172.21.15.119:0/2730159863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95b4100be0 0x7f95b4197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:08.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.986+0000 7f95b3400700 1 -- 172.21.15.119:0/2730159863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a00095f0 con 0x7f95b4103500 2024-05-27T09:30:08.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.986+0000 7f95b3400700 1 --2- 172.21.15.119:0/2730159863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b4103500 0x7f95b41980a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f95a0005b40 tx=0x7f95a000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:08.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.987+0000 7f95b0c00700 1 -- 172.21.15.119:0/2730159863 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95a001d070 con 0x7f95b4103500 2024-05-27T09:30:08.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.987+0000 7f95b0c00700 1 -- 172.21.15.119:0/2730159863 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f95a000f460 con 0x7f95b4103500 2024-05-27T09:30:08.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.987+0000 7f95b0c00700 1 -- 172.21.15.119:0/2730159863 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95a0021640 con 0x7f95b4103500 2024-05-27T09:30:08.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.987+0000 7f95bb200700 1 -- 172.21.15.119:0/2730159863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b419d250 con 0x7f95b4103500 2024-05-27T09:30:08.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.987+0000 7f95bb200700 1 -- 172.21.15.119:0/2730159863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b419d740 con 0x7f95b4103500 2024-05-27T09:30:08.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.988+0000 7f95bb200700 1 -- 172.21.15.119:0/2730159863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b40fc1c0 con 0x7f95b4103500 2024-05-27T09:30:08.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.990+0000 7f95b0c00700 1 -- 172.21.15.119:0/2730159863 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) v1 ==== 95024+0+0 (secure 0 0 0) 0x7f95a000fa90 con 0x7f95b4103500 2024-05-27T09:30:08.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.990+0000 7f95b0c00700 1 --2- 172.21.15.119:0/2730159863 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f9598072820 0x7f9598074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:08.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.991+0000 7f95b0c00700 1 -- 172.21.15.119:0/2730159863 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f95a0094650 con 0x7f95b4103500 2024-05-27T09:30:08.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.991+0000 7f95b3e00700 1 --2- 172.21.15.119:0/2730159863 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f9598072820 0x7f9598074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:08.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.993+0000 7f95b3e00700 1 --2- 172.21.15.119:0/2730159863 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f9598072820 0x7f9598074cd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f95a4005950 tx=0x7f95a40058e0 comp rx=0 tx=0).ready entity=mgr.14570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:08.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:08.997+0000 7f95b0c00700 1 -- 172.21.15.119:0/2730159863 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f95a005c660 con 0x7f95b4103500 2024-05-27T09:30:09.140 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:09 smithi119 ceph-mon[29082]: pgmap v32: 65 pgs: 65 active+clean; 1.6 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 30 MiB/s rd, 46 MiB/s wr, 253 op/s 2024-05-27T09:30:09.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.436+0000 7f95bb200700 1 -- 172.21.15.119:0/2730159863 --> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f95b4061190 con 0x7f9598072820 2024-05-27T09:30:09.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.439+0000 7f95b0c00700 1 -- 172.21.15.119:0/2730159863 <== mgr.14570 v2:172.21.15.138:6828/1384904888 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f95b4061190 con 0x7f9598072820 2024-05-27T09:30:09.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.442+0000 7f95bb200700 1 -- 172.21.15.119:0/2730159863 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f9598072820 msgr2=0x7f9598074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:09.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.442+0000 7f95bb200700 1 --2- 172.21.15.119:0/2730159863 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f9598072820 0x7f9598074cd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f95a4005950 tx=0x7f95a40058e0 comp rx=0 tx=0).stop 2024-05-27T09:30:09.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.442+0000 7f95bb200700 1 -- 172.21.15.119:0/2730159863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b4103500 msgr2=0x7f95b41980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:09.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.442+0000 7f95bb200700 1 --2- 172.21.15.119:0/2730159863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b4103500 0x7f95b41980a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f95a0005b40 tx=0x7f95a000be20 comp rx=0 tx=0).stop 2024-05-27T09:30:09.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.443+0000 7f95bb200700 1 -- 172.21.15.119:0/2730159863 shutdown_connections 2024-05-27T09:30:09.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.443+0000 7f95bb200700 1 --2- 172.21.15.119:0/2730159863 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f9598072820 0x7f9598074cd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:09.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.443+0000 7f95bb200700 1 --2- 172.21.15.119:0/2730159863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95b4100be0 0x7f95b4197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:09.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.443+0000 7f95bb200700 1 --2- 172.21.15.119:0/2730159863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b4103500 0x7f95b41980a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:09.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.443+0000 7f95bb200700 1 -- 172.21.15.119:0/2730159863 >> 172.21.15.119:0/2730159863 conn(0x7f95b40fa5e0 msgr2=0x7f95b40ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:09.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.444+0000 7f95bb200700 1 -- 172.21.15.119:0/2730159863 shutdown_connections 2024-05-27T09:30:09.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.444+0000 7f95bb200700 1 -- 172.21.15.119:0/2730159863 wait complete. 2024-05-27T09:30:09.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.630+0000 7f70b6600700 1 -- 172.21.15.119:0/762664119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b0073730 msgr2=0x7f70b0073ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:09.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.630+0000 7f70b6600700 1 --2- 172.21.15.119:0/762664119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b0073730 0x7f70b0073ba0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f709c009910 tx=0x7f709c009c20 comp rx=0 tx=0).stop 2024-05-27T09:30:09.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.630+0000 7f70b6600700 1 -- 172.21.15.119:0/762664119 shutdown_connections 2024-05-27T09:30:09.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.630+0000 7f70b6600700 1 --2- 172.21.15.119:0/762664119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b0073730 0x7f70b0073ba0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:09.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.630+0000 7f70b6600700 1 --2- 172.21.15.119:0/762664119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b0074d00 0x7f70b0073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:09.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.630+0000 7f70b6600700 1 -- 172.21.15.119:0/762664119 >> 172.21.15.119:0/762664119 conn(0x7f70b00fb570 msgr2=0x7f70b00fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:09.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.630+0000 7f70b6600700 1 -- 172.21.15.119:0/762664119 shutdown_connections 2024-05-27T09:30:09.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.630+0000 7f70b6600700 1 -- 172.21.15.119:0/762664119 wait complete. 2024-05-27T09:30:09.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.631+0000 7f70b6600700 1 Processor -- start 2024-05-27T09:30:09.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.631+0000 7f70b6600700 1 -- start start 2024-05-27T09:30:09.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.631+0000 7f70b6600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b0073730 0x7f70b01979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:09.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.631+0000 7f70b6600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b0074d00 0x7f70b0197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:09.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.631+0000 7f70b6600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b0198550 con 0x7f70b0074d00 2024-05-27T09:30:09.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.631+0000 7f70b6600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b0198690 con 0x7f70b0073730 2024-05-27T09:30:09.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.632+0000 7f70afe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b0074d00 0x7f70b0197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:09.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.632+0000 7f70afe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b0074d00 0x7f70b0197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42358/0 (socket says 172.21.15.119:42358) 2024-05-27T09:30:09.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.632+0000 7f70afe00700 1 -- 172.21.15.119:0/1194886655 learned_addr learned my addr 172.21.15.119:0/1194886655 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:30:09.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.632+0000 7f70b5200700 1 --2- 172.21.15.119:0/1194886655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b0073730 0x7f70b01979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:09.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.632+0000 7f70afe00700 1 -- 172.21.15.119:0/1194886655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b0073730 msgr2=0x7f70b01979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:09.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.632+0000 7f70afe00700 1 --2- 172.21.15.119:0/1194886655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b0073730 0x7f70b01979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:09.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.632+0000 7f70afe00700 1 -- 172.21.15.119:0/1194886655 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c0095f0 con 0x7f70b0074d00 2024-05-27T09:30:09.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.633+0000 7f70afe00700 1 --2- 172.21.15.119:0/1194886655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b0074d00 0x7f70b0197f30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f709c005b40 tx=0x7f709c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:09.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.633+0000 7f70ad600700 1 -- 172.21.15.119:0/1194886655 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f709c01d070 con 0x7f70b0074d00 2024-05-27T09:30:09.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.633+0000 7f70ad600700 1 -- 172.21.15.119:0/1194886655 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f709c022470 con 0x7f70b0074d00 2024-05-27T09:30:09.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.633+0000 7f70b6600700 1 -- 172.21.15.119:0/1194886655 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b019d0e0 con 0x7f70b0074d00 2024-05-27T09:30:09.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.633+0000 7f70ad600700 1 -- 172.21.15.119:0/1194886655 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f709c00f690 con 0x7f70b0074d00 2024-05-27T09:30:09.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:09 smithi138 ceph-mon[34885]: pgmap v32: 65 pgs: 65 active+clean; 1.6 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 30 MiB/s rd, 46 MiB/s wr, 253 op/s 2024-05-27T09:30:09.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.633+0000 7f70b6600700 1 -- 172.21.15.119:0/1194886655 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b019d650 con 0x7f70b0074d00 2024-05-27T09:30:09.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.635+0000 7f70b6600700 1 -- 172.21.15.119:0/1194886655 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b010aec0 con 0x7f70b0074d00 2024-05-27T09:30:09.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.636+0000 7f70ad600700 1 -- 172.21.15.119:0/1194886655 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) v1 ==== 95024+0+0 (secure 0 0 0) 0x7f709c0225e0 con 0x7f70b0074d00 2024-05-27T09:30:09.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.637+0000 7f70ad600700 1 --2- 172.21.15.119:0/1194886655 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f70980727d0 0x7f7098074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:09.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.637+0000 7f70ad600700 1 -- 172.21.15.119:0/1194886655 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f709c094ae0 con 0x7f70b0074d00 2024-05-27T09:30:09.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.637+0000 7f70b5200700 1 --2- 172.21.15.119:0/1194886655 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f70980727d0 0x7f7098074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:09.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.639+0000 7f70b5200700 1 --2- 172.21.15.119:0/1194886655 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f70980727d0 0x7f7098074c80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f70a4006fd0 tx=0x7f70a4008040 comp rx=0 tx=0).ready entity=mgr.14570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:09.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.643+0000 7f70ad600700 1 -- 172.21.15.119:0/1194886655 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f709c05caf0 con 0x7f70b0074d00 2024-05-27T09:30:09.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:09.994+0000 7f70b6600700 1 -- 172.21.15.119:0/1194886655 --> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f70b0107540 con 0x7f70980727d0 2024-05-27T09:30:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.015+0000 7f70ad600700 1 -- 172.21.15.119:0/1194886655 <== mgr.14570 v2:172.21.15.138:6828/1384904888 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f70b0107540 con 0x7f70980727d0 2024-05-27T09:30:10.016 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:30:10.016 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (9m) 35s ago 11m 13.9M - 0.25.0 c8568f914cd2 bc280230ab28 2024-05-27T09:30:10.017 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (11m) 35s ago 11m 7772k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:30:10.017 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (10m) 40s ago 10m 7629k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:30:10.017 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (11m) 35s ago 11m 6665k - 18.2.0 dc2bc1663786 e5fafde44183 2024-05-27T09:30:10.017 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (10m) 40s ago 10m 6660k - 18.2.0 dc2bc1663786 46956ab99bab 2024-05-27T09:30:10.017 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (9m) 35s ago 10m 74.5M - 9.4.7 954c08fa6188 16eeaded4295 2024-05-27T09:30:10.017 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (4m) 35s ago 4m 13.5M - 18.2.0 dc2bc1663786 93b21713fb1e 2024-05-27T09:30:10.017 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (4m) 35s ago 4m 132M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:30:10.017 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (4m) 40s ago 4m 13.5M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:30:10.017 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (4m) 40s ago 4m 127M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:30:10.017 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:9283,8765,8443 running (13m) 35s ago 13m 323M - 18.2.0 dc2bc1663786 abe5e83463f2 2024-05-27T09:30:10.017 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (75s) 40s ago 9m 572M - 19.0.0-3868-ga6507388 850a06a84d28 be5c0668306a 2024-05-27T09:30:10.017 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (13m) 35s ago 13m 51.5M 2048M 18.2.0 dc2bc1663786 ba998b789a1c 2024-05-27T09:30:10.018 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (9m) 40s ago 9m 41.4M 2048M 18.2.0 dc2bc1663786 60216fd67078 2024-05-27T09:30:10.018 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (11m) 35s ago 11m 8828k - 1.5.0 0da6a335fe13 79c36d4e5b04 2024-05-27T09:30:10.018 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (10m) 40s ago 10m 8086k - 1.5.0 0da6a335fe13 80ee890e8830 2024-05-27T09:30:10.018 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (8m) 35s ago 8m 175M 1216M 18.2.0 dc2bc1663786 825bd1392508 2024-05-27T09:30:10.018 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (8m) 35s ago 8m 227M 1216M 18.2.0 dc2bc1663786 fce9819903d9 2024-05-27T09:30:10.018 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (7m) 35s ago 7m 168M 1216M 18.2.0 dc2bc1663786 3955a6112c1c 2024-05-27T09:30:10.018 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (7m) 40s ago 7m 215M 2240M 18.2.0 dc2bc1663786 693c92220622 2024-05-27T09:30:10.018 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (6m) 40s ago 6m 186M 2240M 18.2.0 dc2bc1663786 7559e658e917 2024-05-27T09:30:10.018 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (5m) 40s ago 5m 213M 2240M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:30:10.018 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 starting - - - - 2024-05-27T09:30:10.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.019+0000 7f70b6600700 1 -- 172.21.15.119:0/1194886655 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f70980727d0 msgr2=0x7f7098074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:10.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.019+0000 7f70b6600700 1 --2- 172.21.15.119:0/1194886655 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f70980727d0 0x7f7098074c80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f70a4006fd0 tx=0x7f70a4008040 comp rx=0 tx=0).stop 2024-05-27T09:30:10.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.019+0000 7f70b6600700 1 -- 172.21.15.119:0/1194886655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b0074d00 msgr2=0x7f70b0197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:10.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.019+0000 7f70b6600700 1 --2- 172.21.15.119:0/1194886655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b0074d00 0x7f70b0197f30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f709c005b40 tx=0x7f709c00be20 comp rx=0 tx=0).stop 2024-05-27T09:30:10.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.019+0000 7f70b6600700 1 -- 172.21.15.119:0/1194886655 shutdown_connections 2024-05-27T09:30:10.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.019+0000 7f70b6600700 1 --2- 172.21.15.119:0/1194886655 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f70980727d0 0x7f7098074c80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:10.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.019+0000 7f70b6600700 1 --2- 172.21.15.119:0/1194886655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b0073730 0x7f70b01979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:10.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.019+0000 7f70b6600700 1 --2- 172.21.15.119:0/1194886655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b0074d00 0x7f70b0197f30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:10.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.019+0000 7f70b6600700 1 -- 172.21.15.119:0/1194886655 >> 172.21.15.119:0/1194886655 conn(0x7f70b00fb570 msgr2=0x7f70b0105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:10.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.020+0000 7f70b6600700 1 -- 172.21.15.119:0/1194886655 shutdown_connections 2024-05-27T09:30:10.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.020+0000 7f70b6600700 1 -- 172.21.15.119:0/1194886655 wait complete. 2024-05-27T09:30:10.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.203+0000 7fbc60400700 1 -- 172.21.15.119:0/3533207966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc58101350 msgr2=0x7fbc581017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:10.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.203+0000 7fbc60400700 1 --2- 172.21.15.119:0/3533207966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc58101350 0x7fbc581017a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009910 tx=0x7fbc44009c20 comp rx=0 tx=0).stop 2024-05-27T09:30:10.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.203+0000 7fbc60400700 1 -- 172.21.15.119:0/3533207966 shutdown_connections 2024-05-27T09:30:10.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.203+0000 7fbc60400700 1 --2- 172.21.15.119:0/3533207966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc58101350 0x7fbc581017a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:10.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.203+0000 7fbc60400700 1 --2- 172.21.15.119:0/3533207966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc58100150 0x7fbc58100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:10.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.203+0000 7fbc60400700 1 -- 172.21.15.119:0/3533207966 >> 172.21.15.119:0/3533207966 conn(0x7fbc580fb6e0 msgr2=0x7fbc580fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:10.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.204+0000 7fbc60400700 1 -- 172.21.15.119:0/3533207966 shutdown_connections 2024-05-27T09:30:10.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.204+0000 7fbc60400700 1 -- 172.21.15.119:0/3533207966 wait complete. 2024-05-27T09:30:10.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.205+0000 7fbc60400700 1 Processor -- start 2024-05-27T09:30:10.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.205+0000 7fbc60400700 1 -- start start 2024-05-27T09:30:10.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.206+0000 7fbc60400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc58100150 0x7fbc58197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:10.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.206+0000 7fbc60400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc58101350 0x7fbc581980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:10.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.206+0000 7fbc60400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc581986f0 con 0x7fbc58100150 2024-05-27T09:30:10.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.206+0000 7fbc60400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc58198830 con 0x7fbc58101350 2024-05-27T09:30:10.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.206+0000 7fbc5d000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc58101350 0x7fbc581980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:10.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.206+0000 7fbc5d000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc58101350 0x7fbc581980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.119:48516/0 (socket says 172.21.15.119:48516) 2024-05-27T09:30:10.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.206+0000 7fbc5d000700 1 -- 172.21.15.119:0/520229535 learned_addr learned my addr 172.21.15.119:0/520229535 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:30:10.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.207+0000 7fbc5da00700 1 --2- 172.21.15.119:0/520229535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc58100150 0x7fbc58197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:10.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.207+0000 7fbc5d000700 1 -- 172.21.15.119:0/520229535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc58100150 msgr2=0x7fbc58197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:10.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.207+0000 7fbc5d000700 1 --2- 172.21.15.119:0/520229535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc58100150 0x7fbc58197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:10.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.207+0000 7fbc5d000700 1 -- 172.21.15.119:0/520229535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc440095f0 con 0x7fbc58101350 2024-05-27T09:30:10.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.207+0000 7fbc5d000700 1 --2- 172.21.15.119:0/520229535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc58101350 0x7fbc581980d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbc44005b40 tx=0x7fbc4400bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:10.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.208+0000 7fbc56000700 1 -- 172.21.15.119:0/520229535 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc4401d070 con 0x7fbc58101350 2024-05-27T09:30:10.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.208+0000 7fbc56000700 1 -- 172.21.15.119:0/520229535 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbc4400f460 con 0x7fbc58101350 2024-05-27T09:30:10.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.208+0000 7fbc56000700 1 -- 172.21.15.119:0/520229535 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc44021620 con 0x7fbc58101350 2024-05-27T09:30:10.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.208+0000 7fbc60400700 1 -- 172.21.15.119:0/520229535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5819d280 con 0x7fbc58101350 2024-05-27T09:30:10.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.208+0000 7fbc60400700 1 -- 172.21.15.119:0/520229535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5819d720 con 0x7fbc58101350 2024-05-27T09:30:10.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.209+0000 7fbc60400700 1 -- 172.21.15.119:0/520229535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc58105430 con 0x7fbc58101350 2024-05-27T09:30:10.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.211+0000 7fbc56000700 1 -- 172.21.15.119:0/520229535 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) v1 ==== 95024+0+0 (secure 0 0 0) 0x7fbc4400fa40 con 0x7fbc58101350 2024-05-27T09:30:10.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.212+0000 7fbc56000700 1 --2- 172.21.15.119:0/520229535 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7fbc400727d0 0x7fbc40074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:10.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.212+0000 7fbc56000700 1 -- 172.21.15.119:0/520229535 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fbc440945f0 con 0x7fbc58101350 2024-05-27T09:30:10.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.213+0000 7fbc5da00700 1 --2- 172.21.15.119:0/520229535 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7fbc400727d0 0x7fbc40074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:10.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.214+0000 7fbc5da00700 1 --2- 172.21.15.119:0/520229535 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7fbc400727d0 0x7fbc40074c80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbc581011b0 tx=0x7fbc4c00b410 comp rx=0 tx=0).ready entity=mgr.14570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:10.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.221+0000 7fbc56000700 1 -- 172.21.15.119:0/520229535 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7fbc4405c5c0 con 0x7fbc58101350 2024-05-27T09:30:10.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:10 smithi119 ceph-mon[29082]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:30:10.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:10 smithi119 ceph-mon[29082]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:30:10.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.628+0000 7fbc60400700 1 -- 172.21.15.119:0/520229535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbc5802cdc0 con 0x7fbc58101350 2024-05-27T09:30:10.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.629+0000 7fbc56000700 1 -- 172.21.15.119:0/520229535 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7fbc4405bf60 con 0x7fbc58101350 2024-05-27T09:30:10.631 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:30:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:30:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-05-27T09:30:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:30:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:30:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-05-27T09:30:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 1 2024-05-27T09:30:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:30:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:30:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-05-27T09:30:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:30:10.631 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:30:10.632 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-05-27T09:30:10.632 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:30:10.632 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:30:10.632 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 13, 2024-05-27T09:30:10.632 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 1 2024-05-27T09:30:10.632 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:30:10.632 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:30:10.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.632+0000 7fbc60400700 1 -- 172.21.15.119:0/520229535 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7fbc400727d0 msgr2=0x7fbc40074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:10.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.632+0000 7fbc60400700 1 --2- 172.21.15.119:0/520229535 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7fbc400727d0 0x7fbc40074c80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbc581011b0 tx=0x7fbc4c00b410 comp rx=0 tx=0).stop 2024-05-27T09:30:10.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.632+0000 7fbc60400700 1 -- 172.21.15.119:0/520229535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc58101350 msgr2=0x7fbc581980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:10.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.632+0000 7fbc60400700 1 --2- 172.21.15.119:0/520229535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc58101350 0x7fbc581980d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbc44005b40 tx=0x7fbc4400bd70 comp rx=0 tx=0).stop 2024-05-27T09:30:10.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.632+0000 7fbc60400700 1 -- 172.21.15.119:0/520229535 shutdown_connections 2024-05-27T09:30:10.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.632+0000 7fbc60400700 1 --2- 172.21.15.119:0/520229535 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7fbc400727d0 0x7fbc40074c80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:10.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.632+0000 7fbc60400700 1 --2- 172.21.15.119:0/520229535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc58100150 0x7fbc58197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:10.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.632+0000 7fbc60400700 1 --2- 172.21.15.119:0/520229535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc58101350 0x7fbc581980d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:10.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.632+0000 7fbc60400700 1 -- 172.21.15.119:0/520229535 >> 172.21.15.119:0/520229535 conn(0x7fbc580fb6e0 msgr2=0x7fbc58102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:10.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.633+0000 7fbc60400700 1 -- 172.21.15.119:0/520229535 shutdown_connections 2024-05-27T09:30:10.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.633+0000 7fbc60400700 1 -- 172.21.15.119:0/520229535 wait complete. 2024-05-27T09:30:10.638 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:10 smithi138 ceph-mon[34885]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:30:10.638 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:10 smithi138 ceph-mon[34885]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:30:10.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.782+0000 7f2113a00700 1 -- 172.21.15.119:0/3197054380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f210c101350 msgr2=0x7f210c1017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:10.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.782+0000 7f2113a00700 1 --2- 172.21.15.119:0/3197054380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f210c101350 0x7f210c1017a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009910 tx=0x7f20f4009c20 comp rx=0 tx=0).stop 2024-05-27T09:30:10.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.782+0000 7f2113a00700 1 -- 172.21.15.119:0/3197054380 shutdown_connections 2024-05-27T09:30:10.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.782+0000 7f2113a00700 1 --2- 172.21.15.119:0/3197054380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f210c101350 0x7f210c1017a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:10.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.782+0000 7f2113a00700 1 --2- 172.21.15.119:0/3197054380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f210c100150 0x7f210c100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:10.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.782+0000 7f2113a00700 1 -- 172.21.15.119:0/3197054380 >> 172.21.15.119:0/3197054380 conn(0x7f210c0fb6e0 msgr2=0x7f210c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:10.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.782+0000 7f2113a00700 1 -- 172.21.15.119:0/3197054380 shutdown_connections 2024-05-27T09:30:10.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.782+0000 7f2113a00700 1 -- 172.21.15.119:0/3197054380 wait complete. 2024-05-27T09:30:10.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.783+0000 7f2113a00700 1 Processor -- start 2024-05-27T09:30:10.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.783+0000 7f2113a00700 1 -- start start 2024-05-27T09:30:10.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.784+0000 7f2113a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f210c100150 0x7f210c195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:10.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.784+0000 7f2113a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f210c101350 0x7f210c195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:10.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.784+0000 7f2113a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c196560 con 0x7f210c100150 2024-05-27T09:30:10.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.784+0000 7f2113a00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c1966a0 con 0x7f210c101350 2024-05-27T09:30:10.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.784+0000 7f2111000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f210c100150 0x7f210c195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:10.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.784+0000 7f2111000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f210c100150 0x7f210c195a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42392/0 (socket says 172.21.15.119:42392) 2024-05-27T09:30:10.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.784+0000 7f2111000700 1 -- 172.21.15.119:0/1257791782 learned_addr learned my addr 172.21.15.119:0/1257791782 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:30:10.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.784+0000 7f2103e00700 1 --2- 172.21.15.119:0/1257791782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f210c101350 0x7f210c195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:10.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.784+0000 7f2111000700 1 -- 172.21.15.119:0/1257791782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f210c101350 msgr2=0x7f210c195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:10.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.784+0000 7f2111000700 1 --2- 172.21.15.119:0/1257791782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f210c101350 0x7f210c195f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:10.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.784+0000 7f2111000700 1 -- 172.21.15.119:0/1257791782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f40095f0 con 0x7f210c100150 2024-05-27T09:30:10.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.785+0000 7f2111000700 1 --2- 172.21.15.119:0/1257791782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f210c100150 0x7f210c195a00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00b480 tx=0x7f20fc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:10.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.785+0000 7f210a000700 1 -- 172.21.15.119:0/1257791782 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20fc0118c0 con 0x7f210c100150 2024-05-27T09:30:10.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.785+0000 7f210a000700 1 -- 172.21.15.119:0/1257791782 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f20fc010380 con 0x7f210c100150 2024-05-27T09:30:10.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.785+0000 7f2113a00700 1 -- 172.21.15.119:0/1257791782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210c19b150 con 0x7f210c100150 2024-05-27T09:30:10.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.785+0000 7f210a000700 1 -- 172.21.15.119:0/1257791782 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20fc00f4f0 con 0x7f210c100150 2024-05-27T09:30:10.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.785+0000 7f2113a00700 1 -- 172.21.15.119:0/1257791782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210c19b720 con 0x7f210c100150 2024-05-27T09:30:10.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.786+0000 7f2113a00700 1 -- 172.21.15.119:0/1257791782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210c108d40 con 0x7f210c100150 2024-05-27T09:30:10.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.787+0000 7f210a000700 1 -- 172.21.15.119:0/1257791782 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) v1 ==== 95024+0+0 (secure 0 0 0) 0x7f20fc0104f0 con 0x7f210c100150 2024-05-27T09:30:10.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.788+0000 7f210a000700 1 --2- 172.21.15.119:0/1257791782 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f20f00727d0 0x7f20f0074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:10.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.788+0000 7f210a000700 1 -- 172.21.15.119:0/1257791782 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f20fc092e70 con 0x7f210c100150 2024-05-27T09:30:10.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.789+0000 7f2103e00700 1 --2- 172.21.15.119:0/1257791782 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f20f00727d0 0x7f20f0074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:10.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.790+0000 7f2103e00700 1 --2- 172.21.15.119:0/1257791782 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f20f00727d0 0x7f20f0074c80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f20f4006010 tx=0x7f20f400b540 comp rx=0 tx=0).ready entity=mgr.14570 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:10.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:10.793+0000 7f210a000700 1 -- 172.21.15.119:0/1257791782 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f20fc05af10 con 0x7f210c100150 2024-05-27T09:30:11.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:11.097+0000 7f2113a00700 1 -- 172.21.15.119:0/1257791782 --> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f210c105ca0 con 0x7f20f00727d0 2024-05-27T09:30:11.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:11.100+0000 7f210a000700 1 -- 172.21.15.119:0/1257791782 <== mgr.14570 v2:172.21.15.138:6828/1384904888 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f210c105ca0 con 0x7f20f00727d0 2024-05-27T09:30:11.100 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:30:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:30:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:30:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-27T09:30:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [], 2024-05-27T09:30:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "1/2 daemons upgraded", 2024-05-27T09:30:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "Currently upgrading mgr daemons", 2024-05-27T09:30:11.101 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:30:11.101 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:30:11.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:11.103+0000 7f2113a00700 1 -- 172.21.15.119:0/1257791782 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f20f00727d0 msgr2=0x7f20f0074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:11.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:11.103+0000 7f2113a00700 1 --2- 172.21.15.119:0/1257791782 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f20f00727d0 0x7f20f0074c80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f20f4006010 tx=0x7f20f400b540 comp rx=0 tx=0).stop 2024-05-27T09:30:11.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:11.103+0000 7f2113a00700 1 -- 172.21.15.119:0/1257791782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f210c100150 msgr2=0x7f210c195a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:11.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:11.103+0000 7f2113a00700 1 --2- 172.21.15.119:0/1257791782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f210c100150 0x7f210c195a00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00b480 tx=0x7f20fc00b840 comp rx=0 tx=0).stop 2024-05-27T09:30:11.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:11.103+0000 7f2113a00700 1 -- 172.21.15.119:0/1257791782 shutdown_connections 2024-05-27T09:30:11.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:11.103+0000 7f2113a00700 1 --2- 172.21.15.119:0/1257791782 >> [v2:172.21.15.138:6828/1384904888,v1:172.21.15.138:6829/1384904888] conn(0x7f20f00727d0 0x7f20f0074c80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:11.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:11.103+0000 7f2113a00700 1 --2- 172.21.15.119:0/1257791782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f210c100150 0x7f210c195a00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:11.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:11.103+0000 7f2113a00700 1 --2- 172.21.15.119:0/1257791782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f210c101350 0x7f210c195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:11.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:11.103+0000 7f2113a00700 1 -- 172.21.15.119:0/1257791782 >> 172.21.15.119:0/1257791782 conn(0x7f210c0fb6e0 msgr2=0x7f210c104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:11.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:11.103+0000 7f2113a00700 1 -- 172.21.15.119:0/1257791782 shutdown_connections 2024-05-27T09:30:11.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:11.103+0000 7f2113a00700 1 -- 172.21.15.119:0/1257791782 wait complete. 2024-05-27T09:30:11.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:11 smithi138 ceph-mon[34885]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:30:11.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:11 smithi138 ceph-mon[34885]: pgmap v33: 65 pgs: 65 active+clean; 1.6 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 161 op/s 2024-05-27T09:30:11.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:11 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:30:11.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:11 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/520229535' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:30:11.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:11 smithi119 ceph-mon[29082]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:30:11.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:11 smithi119 ceph-mon[29082]: pgmap v33: 65 pgs: 65 active+clean; 1.6 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 161 op/s 2024-05-27T09:30:11.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:11 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:30:11.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:11 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/520229535' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:30:12.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:12 smithi119 ceph-mon[29082]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:30:12.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:12 smithi138 ceph-mon[34885]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:30:13.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:13 smithi119 ceph-mon[29082]: pgmap v34: 65 pgs: 65 active+clean; 899 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 6.0 MiB/s rd, 6.6 MiB/s wr, 243 op/s 2024-05-27T09:30:13.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:13 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:13.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:13 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:13.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:13 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:30:13.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:13 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:13.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:13 smithi138 ceph-mon[34885]: pgmap v34: 65 pgs: 65 active+clean; 899 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 6.0 MiB/s rd, 6.6 MiB/s wr, 243 op/s 2024-05-27T09:30:13.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:13 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:13.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:13 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:13.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:13 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:30:13.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:13 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:14.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:14 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:14.962 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:14 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:15.623 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:15 smithi119 ceph-mon[29082]: pgmap v35: 65 pgs: 65 active+clean; 899 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-05-27T09:30:15.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:15 smithi138 ceph-mon[34885]: pgmap v35: 65 pgs: 65 active+clean; 899 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-05-27T09:30:17.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:17 smithi138 ceph-mon[34885]: pgmap v36: 65 pgs: 65 active+clean; 899 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-05-27T09:30:17.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:17 smithi119 ceph-mon[29082]: pgmap v36: 65 pgs: 65 active+clean; 899 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-05-27T09:30:19.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:19 smithi138 ceph-mon[34885]: pgmap v37: 65 pgs: 65 active+clean; 907 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 24 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-05-27T09:30:19.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:19 smithi119 ceph-mon[29082]: pgmap v37: 65 pgs: 65 active+clean; 907 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 24 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-05-27T09:30:21.169 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:20 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:21.169 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:20 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:21.169 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:20 smithi119 ceph-mon[29082]: pgmap v38: 65 pgs: 65 active+clean; 907 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 13 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-05-27T09:30:21.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:20 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:21.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:20 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:21.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:20 smithi138 ceph-mon[34885]: pgmap v38: 65 pgs: 65 active+clean; 907 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 13 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-05-27T09:30:23.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:23 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:23.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:23 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:23.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:23 smithi119 ceph-mon[29082]: pgmap v39: 65 pgs: 65 active+clean; 427 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-05-27T09:30:23.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:23 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:23.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:23 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:23.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:23 smithi138 ceph-mon[34885]: pgmap v39: 65 pgs: 65 active+clean; 427 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-05-27T09:30:24.494 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:24 smithi119 ceph-mon[29082]: Standby manager daemon smithi119.ncwmog restarted 2024-05-27T09:30:24.494 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:24 smithi119 ceph-mon[29082]: Standby manager daemon smithi119.ncwmog started 2024-05-27T09:30:24.494 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:24 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi119.ncwmog/crt"}]: dispatch 2024-05-27T09:30:24.494 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:24 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T09:30:24.495 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:24 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi119.ncwmog/key"}]: dispatch 2024-05-27T09:30:24.495 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:24 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T09:30:24.596 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-27T09:30:24.597 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-05-27T09:30:24.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:24 smithi138 ceph-mon[34885]: Standby manager daemon smithi119.ncwmog restarted 2024-05-27T09:30:24.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:24 smithi138 ceph-mon[34885]: Standby manager daemon smithi119.ncwmog started 2024-05-27T09:30:24.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:24 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi119.ncwmog/crt"}]: dispatch 2024-05-27T09:30:24.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:24 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T09:30:24.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:24 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi119.ncwmog/key"}]: dispatch 2024-05-27T09:30:24.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:24 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T09:30:25.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:25 smithi119 ceph-mon[29082]: mgrmap e26: smithi138.dxsuqf(active, since 73s), standbys: smithi119.ncwmog 2024-05-27T09:30:25.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:25 smithi119 ceph-mon[29082]: pgmap v40: 65 pgs: 65 active+clean; 427 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 12 KiB/s rd, 1.2 MiB/s wr, 111 op/s 2024-05-27T09:30:25.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:25 smithi138 ceph-mon[34885]: mgrmap e26: smithi138.dxsuqf(active, since 73s), standbys: smithi119.ncwmog 2024-05-27T09:30:25.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:25 smithi138 ceph-mon[34885]: pgmap v40: 65 pgs: 65 active+clean; 427 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 12 KiB/s rd, 1.2 MiB/s wr, 111 op/s 2024-05-27T09:30:26.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:26 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:30:26.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:26 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:30:27.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:27 smithi119 ceph-mon[29082]: pgmap v41: 65 pgs: 65 active+clean; 427 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 12 KiB/s rd, 1.2 MiB/s wr, 111 op/s 2024-05-27T09:30:27.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:27 smithi138 ceph-mon[34885]: pgmap v41: 65 pgs: 65 active+clean; 427 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 12 KiB/s rd, 1.2 MiB/s wr, 111 op/s 2024-05-27T09:30:28.401 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:28.401 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:28.401 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:30:28.401 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:30:28.401 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:28.402 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:30:28.402 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mgr fail", "who": "smithi138.dxsuqf"}]: dispatch 2024-05-27T09:30:28.402 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mgr fail", "who": "smithi138.dxsuqf"}]: dispatch 2024-05-27T09:30:28.402 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: osdmap e48: 6 total, 6 up, 6 in 2024-05-27T09:30:28.402 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' cmd='[{"prefix": "mgr fail", "who": "smithi138.dxsuqf"}]': finished 2024-05-27T09:30:28.402 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: mgrmap e27: smithi119.ncwmog(active, starting, since 0.0161224s) 2024-05-27T09:30:28.402 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi119"}]: dispatch 2024-05-27T09:30:28.402 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:30:28.402 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.ppmzzk"}]: dispatch 2024-05-27T09:30:28.402 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.nghcey"}]: dispatch 2024-05-27T09:30:28.403 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.ndfkhd"}]: dispatch 2024-05-27T09:30:28.403 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.alszpk"}]: dispatch 2024-05-27T09:30:28.403 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr metadata", "who": "smithi119.ncwmog", "id": "smithi119.ncwmog"}]: dispatch 2024-05-27T09:30:28.403 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T09:30:28.403 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T09:30:28.403 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T09:30:28.403 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T09:30:28.403 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T09:30:28.403 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T09:30:28.403 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T09:30:28.403 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T09:30:28.403 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T09:30:28.404 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: Manager daemon smithi119.ncwmog is now available 2024-05-27T09:30:28.404 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:30:28.404 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:30:28.404 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi119.ncwmog/mirror_snapshot_schedule"}]: dispatch 2024-05-27T09:30:28.404 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi119.ncwmog/trash_purge_schedule"}]: dispatch 2024-05-27T09:30:28.809 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:28.810 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:28.810 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:30:28.810 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:30:28.810 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' 2024-05-27T09:30:28.810 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:30:28.810 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14570 172.21.15.138:0/1466904297' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mgr fail", "who": "smithi138.dxsuqf"}]: dispatch 2024-05-27T09:30:28.810 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "mgr fail", "who": "smithi138.dxsuqf"}]: dispatch 2024-05-27T09:30:28.810 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: osdmap e48: 6 total, 6 up, 6 in 2024-05-27T09:30:28.810 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14570 ' entity='mgr.smithi138.dxsuqf' cmd='[{"prefix": "mgr fail", "who": "smithi138.dxsuqf"}]': finished 2024-05-27T09:30:28.810 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: mgrmap e27: smithi119.ncwmog(active, starting, since 0.0161224s) 2024-05-27T09:30:28.810 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi119"}]: dispatch 2024-05-27T09:30:28.810 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:30:28.811 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.ppmzzk"}]: dispatch 2024-05-27T09:30:28.811 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.nghcey"}]: dispatch 2024-05-27T09:30:28.811 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.ndfkhd"}]: dispatch 2024-05-27T09:30:28.811 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.alszpk"}]: dispatch 2024-05-27T09:30:28.811 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr metadata", "who": "smithi119.ncwmog", "id": "smithi119.ncwmog"}]: dispatch 2024-05-27T09:30:28.811 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T09:30:28.811 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T09:30:28.811 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T09:30:28.811 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T09:30:28.811 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T09:30:28.811 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T09:30:28.811 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T09:30:28.811 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T09:30:28.812 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T09:30:28.812 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: Manager daemon smithi119.ncwmog is now available 2024-05-27T09:30:28.812 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:30:28.812 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:30:28.812 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi119.ncwmog/mirror_snapshot_schedule"}]: dispatch 2024-05-27T09:30:28.812 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi119.ncwmog/trash_purge_schedule"}]: dispatch 2024-05-27T09:30:30.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:29 smithi138 ceph-mon[34885]: [27/May/2024:09:30:28] ENGINE Bus STARTING 2024-05-27T09:30:30.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:29 smithi138 ceph-mon[34885]: [27/May/2024:09:30:28] ENGINE Serving on https://172.21.15.119:7150 2024-05-27T09:30:30.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:29 smithi138 ceph-mon[34885]: [27/May/2024:09:30:28] ENGINE Serving on http://172.21.15.119:8765 2024-05-27T09:30:30.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:29 smithi138 ceph-mon[34885]: [27/May/2024:09:30:28] ENGINE Bus STARTED 2024-05-27T09:30:30.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:29 smithi138 ceph-mon[34885]: mgrmap e28: smithi119.ncwmog(active, since 1.02224s) 2024-05-27T09:30:30.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:29 smithi138 ceph-mon[34885]: pgmap v3: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:30:30.151 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:29 smithi119 ceph-mon[29082]: [27/May/2024:09:30:28] ENGINE Bus STARTING 2024-05-27T09:30:30.151 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:29 smithi119 ceph-mon[29082]: [27/May/2024:09:30:28] ENGINE Serving on https://172.21.15.119:7150 2024-05-27T09:30:30.151 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:29 smithi119 ceph-mon[29082]: [27/May/2024:09:30:28] ENGINE Serving on http://172.21.15.119:8765 2024-05-27T09:30:30.151 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:29 smithi119 ceph-mon[29082]: [27/May/2024:09:30:28] ENGINE Bus STARTED 2024-05-27T09:30:30.151 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:29 smithi119 ceph-mon[29082]: mgrmap e28: smithi119.ncwmog(active, since 1.02224s) 2024-05-27T09:30:30.151 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:29 smithi119 ceph-mon[29082]: pgmap v3: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:30:31.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:30 smithi138 ceph-mon[34885]: pgmap v4: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:30:31.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:30 smithi138 ceph-mon[34885]: mgrmap e29: smithi119.ncwmog(active, since 2s) 2024-05-27T09:30:31.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:30 smithi119 ceph-mon[29082]: pgmap v4: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:30:31.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:30 smithi119 ceph-mon[29082]: mgrmap e29: smithi119.ncwmog(active, since 2s) 2024-05-27T09:30:33.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:33 smithi119 ceph-mon[29082]: pgmap v5: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:30:33.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:33 smithi119 ceph-mon[29082]: mgrmap e30: smithi119.ncwmog(active, since 4s) 2024-05-27T09:30:33.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:33 smithi138 ceph-mon[34885]: pgmap v5: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:30:33.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:33 smithi138 ceph-mon[34885]: mgrmap e30: smithi119.ncwmog(active, since 4s) 2024-05-27T09:30:35.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:35 smithi138 ceph-mon[34885]: pgmap v6: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:30:35.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:35 smithi119 ceph-mon[29082]: pgmap v6: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:30:37.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:37 smithi119 ceph-mon[29082]: pgmap v7: 65 pgs: 65 active+clean; 337 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 57 KiB/s rd, 1.3 MiB/s wr, 351 op/s 2024-05-27T09:30:37.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:37 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:37 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:37 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:37 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:37.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:37 smithi138 ceph-mon[34885]: pgmap v7: 65 pgs: 65 active+clean; 337 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 57 KiB/s rd, 1.3 MiB/s wr, 351 op/s 2024-05-27T09:30:37.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:37 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:37.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:37 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:37.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:37 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:37.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:37 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:39.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:39 smithi119 ceph-mon[29082]: pgmap v8: 65 pgs: 65 active+clean; 337 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 45 KiB/s rd, 1.0 MiB/s wr, 273 op/s 2024-05-27T09:30:39.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:39 smithi138 ceph-mon[34885]: pgmap v8: 65 pgs: 65 active+clean; 337 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 45 KiB/s rd, 1.0 MiB/s wr, 273 op/s 2024-05-27T09:30:40.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:40 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:40.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:40 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:40.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:40 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:40.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:40 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:40.654 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:40 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:40.654 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:40 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:40.654 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:40 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:40.654 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:40 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:41.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.285+0000 7f8e5ee00700 1 -- 172.21.15.119:0/3042205388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e580737b0 msgr2=0x7f8e58073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:41.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.285+0000 7f8e5ee00700 1 --2- 172.21.15.119:0/3042205388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e580737b0 0x7f8e58073c20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8e40009940 tx=0x7f8e40009c50 comp rx=0 tx=0).stop 2024-05-27T09:30:41.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.285+0000 7f8e5ee00700 1 -- 172.21.15.119:0/3042205388 shutdown_connections 2024-05-27T09:30:41.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.285+0000 7f8e5ee00700 1 --2- 172.21.15.119:0/3042205388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e580737b0 0x7f8e58073c20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:41.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.285+0000 7f8e5ee00700 1 --2- 172.21.15.119:0/3042205388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e58074d80 0x7f8e580731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:41.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.285+0000 7f8e5ee00700 1 -- 172.21.15.119:0/3042205388 >> 172.21.15.119:0/3042205388 conn(0x7f8e580fb6e0 msgr2=0x7f8e580fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:41.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.285+0000 7f8e5ee00700 1 -- 172.21.15.119:0/3042205388 shutdown_connections 2024-05-27T09:30:41.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.285+0000 7f8e5ee00700 1 -- 172.21.15.119:0/3042205388 wait complete. 2024-05-27T09:30:41.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.287+0000 7f8e5ee00700 1 Processor -- start 2024-05-27T09:30:41.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.287+0000 7f8e5ee00700 1 -- start start 2024-05-27T09:30:41.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.287+0000 7f8e5ee00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e580737b0 0x7f8e58197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:41.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.287+0000 7f8e5ee00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e58074d80 0x7f8e581980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:41.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.287+0000 7f8e5ee00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e581986c0 con 0x7f8e580737b0 2024-05-27T09:30:41.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.287+0000 7f8e5ee00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e58198800 con 0x7f8e58074d80 2024-05-27T09:30:41.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.288+0000 7f8e4fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e580737b0 0x7f8e58197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:41.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.288+0000 7f8e4fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e580737b0 0x7f8e58197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35940/0 (socket says 172.21.15.119:35940) 2024-05-27T09:30:41.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.288+0000 7f8e4fe00700 1 -- 172.21.15.119:0/1824446578 learned_addr learned my addr 172.21.15.119:0/1824446578 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:30:41.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.288+0000 7f8e57e00700 1 --2- 172.21.15.119:0/1824446578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e58074d80 0x7f8e581980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:41.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.288+0000 7f8e4fe00700 1 -- 172.21.15.119:0/1824446578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e58074d80 msgr2=0x7f8e581980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:41.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.288+0000 7f8e4fe00700 1 --2- 172.21.15.119:0/1824446578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e58074d80 0x7f8e581980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:41.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.288+0000 7f8e4fe00700 1 -- 172.21.15.119:0/1824446578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e400095f0 con 0x7f8e580737b0 2024-05-27T09:30:41.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.288+0000 7f8e4fe00700 1 --2- 172.21.15.119:0/1824446578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e580737b0 0x7f8e58197b60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800d8d0 tx=0x7f8e4800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:41.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.289+0000 7f8e55600700 1 -- 172.21.15.119:0/1824446578 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e48009940 con 0x7f8e580737b0 2024-05-27T09:30:41.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.289+0000 7f8e55600700 1 -- 172.21.15.119:0/1824446578 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8e48010460 con 0x7f8e580737b0 2024-05-27T09:30:41.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.289+0000 7f8e55600700 1 -- 172.21.15.119:0/1824446578 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e4800f5f0 con 0x7f8e580737b0 2024-05-27T09:30:41.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.289+0000 7f8e5ee00700 1 -- 172.21.15.119:0/1824446578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5819d2b0 con 0x7f8e580737b0 2024-05-27T09:30:41.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.289+0000 7f8e5ee00700 1 -- 172.21.15.119:0/1824446578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5819d880 con 0x7f8e580737b0 2024-05-27T09:30:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.291+0000 7f8e55600700 1 -- 172.21.15.119:0/1824446578 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 30) v1 ==== 50363+0+0 (secure 0 0 0) 0x7f8e480105d0 con 0x7f8e580737b0 2024-05-27T09:30:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.291+0000 7f8e5ee00700 1 -- 172.21.15.119:0/1824446578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5810ae90 con 0x7f8e580737b0 2024-05-27T09:30:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.291+0000 7f8e55600700 1 --2- 172.21.15.119:0/1824446578 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f8e3c03e780 0x7f8e3c040c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.291+0000 7f8e55600700 1 -- 172.21.15.119:0/1824446578 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8e48053cf0 con 0x7f8e580737b0 2024-05-27T09:30:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.291+0000 7f8e57e00700 1 --2- 172.21.15.119:0/1824446578 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f8e3c03e780 0x7f8e3c040c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.292+0000 7f8e57e00700 1 --2- 172.21.15.119:0/1824446578 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f8e3c03e780 0x7f8e3c040c30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8e40009910 tx=0x7f8e40005fb0 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:41.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.301+0000 7f8e55600700 1 -- 172.21.15.119:0/1824446578 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f8e4805a080 con 0x7f8e580737b0 2024-05-27T09:30:41.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.406+0000 7f8e55600700 1 -- 172.21.15.119:0/1824446578 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 31) v1 ==== 100037+0+0 (secure 0 0 0) 0x7f8e4801ed50 con 0x7f8e580737b0 2024-05-27T09:30:41.538 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:41 smithi119 ceph-mon[29082]: pgmap v9: 65 pgs: 65 active+clean; 338 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 61 KiB/s rd, 2.0 MiB/s wr, 414 op/s 2024-05-27T09:30:41.538 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:41 smithi119 ceph-mon[29082]: Standby manager daemon smithi138.dxsuqf started 2024-05-27T09:30:41.538 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:41 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.138:0/2828616138' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.dxsuqf/crt"}]: dispatch 2024-05-27T09:30:41.538 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:41 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.138:0/2828616138' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T09:30:41.538 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:41 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.138:0/2828616138' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.dxsuqf/key"}]: dispatch 2024-05-27T09:30:41.538 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:41 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.138:0/2828616138' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T09:30:41.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.677+0000 7f8e5ee00700 1 -- 172.21.15.119:0/1824446578 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e58107650 con 0x7f8e3c03e780 2024-05-27T09:30:41.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.679+0000 7f8e55600700 1 -- 172.21.15.119:0/1824446578 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f8e58107650 con 0x7f8e3c03e780 2024-05-27T09:30:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.682+0000 7f8e5ee00700 1 -- 172.21.15.119:0/1824446578 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f8e3c03e780 msgr2=0x7f8e3c040c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.682+0000 7f8e5ee00700 1 --2- 172.21.15.119:0/1824446578 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f8e3c03e780 0x7f8e3c040c30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8e40009910 tx=0x7f8e40005fb0 comp rx=0 tx=0).stop 2024-05-27T09:30:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.682+0000 7f8e5ee00700 1 -- 172.21.15.119:0/1824446578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e580737b0 msgr2=0x7f8e58197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.682+0000 7f8e5ee00700 1 --2- 172.21.15.119:0/1824446578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e580737b0 0x7f8e58197b60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800d8d0 tx=0x7f8e4800dc90 comp rx=0 tx=0).stop 2024-05-27T09:30:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.682+0000 7f8e5ee00700 1 -- 172.21.15.119:0/1824446578 shutdown_connections 2024-05-27T09:30:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.682+0000 7f8e5ee00700 1 --2- 172.21.15.119:0/1824446578 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f8e3c03e780 0x7f8e3c040c30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.682+0000 7f8e5ee00700 1 --2- 172.21.15.119:0/1824446578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e580737b0 0x7f8e58197b60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.682+0000 7f8e5ee00700 1 --2- 172.21.15.119:0/1824446578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e58074d80 0x7f8e581980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:41.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.682+0000 7f8e5ee00700 1 -- 172.21.15.119:0/1824446578 >> 172.21.15.119:0/1824446578 conn(0x7f8e580fb6e0 msgr2=0x7f8e58105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:41.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.683+0000 7f8e5ee00700 1 -- 172.21.15.119:0/1824446578 shutdown_connections 2024-05-27T09:30:41.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.683+0000 7f8e5ee00700 1 -- 172.21.15.119:0/1824446578 wait complete. 2024-05-27T09:30:41.699 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:30:41.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:41 smithi138 ceph-mon[34885]: pgmap v9: 65 pgs: 65 active+clean; 338 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 61 KiB/s rd, 2.0 MiB/s wr, 414 op/s 2024-05-27T09:30:41.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:41 smithi138 ceph-mon[34885]: Standby manager daemon smithi138.dxsuqf started 2024-05-27T09:30:41.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:41 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.138:0/2828616138' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.dxsuqf/crt"}]: dispatch 2024-05-27T09:30:41.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:41 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.138:0/2828616138' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T09:30:41.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:41 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.138:0/2828616138' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.dxsuqf/key"}]: dispatch 2024-05-27T09:30:41.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:41 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.138:0/2828616138' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T09:30:41.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.890+0000 7fb010400700 1 -- 172.21.15.119:0/3916270456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb008103500 msgr2=0x7fb0081058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:41.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.890+0000 7fb010400700 1 --2- 172.21.15.119:0/3916270456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb008103500 0x7fb0081058e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7faff4009870 tx=0x7faff4009b80 comp rx=0 tx=0).stop 2024-05-27T09:30:41.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.890+0000 7fb010400700 1 -- 172.21.15.119:0/3916270456 shutdown_connections 2024-05-27T09:30:41.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.890+0000 7fb010400700 1 --2- 172.21.15.119:0/3916270456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb008103500 0x7fb0081058e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:41.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.890+0000 7fb010400700 1 --2- 172.21.15.119:0/3916270456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb008100be0 0x7fb008102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:41.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.890+0000 7fb010400700 1 -- 172.21.15.119:0/3916270456 >> 172.21.15.119:0/3916270456 conn(0x7fb0080fa5e0 msgr2=0x7fb0080fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:41.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.891+0000 7fb010400700 1 -- 172.21.15.119:0/3916270456 shutdown_connections 2024-05-27T09:30:41.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.891+0000 7fb010400700 1 -- 172.21.15.119:0/3916270456 wait complete. 2024-05-27T09:30:41.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.892+0000 7fb010400700 1 Processor -- start 2024-05-27T09:30:41.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.893+0000 7fb010400700 1 -- start start 2024-05-27T09:30:41.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.894+0000 7fb010400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb008100be0 0x7fb008100540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:41.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.894+0000 7fb010400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb008103500 0x7fb0080feb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:41.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.894+0000 7fb010400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080ff0d0 con 0x7fb008103500 2024-05-27T09:30:41.894 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.894+0000 7fb010400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080ff210 con 0x7fb008100be0 2024-05-27T09:30:41.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.894+0000 7fb00da00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb008100be0 0x7fb008100540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:41.895 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.894+0000 7fb00da00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb008100be0 0x7fb008100540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.119:33580/0 (socket says 172.21.15.119:33580) 2024-05-27T09:30:41.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.894+0000 7fb00da00700 1 -- 172.21.15.119:0/376105487 learned_addr learned my addr 172.21.15.119:0/376105487 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:30:41.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.895+0000 7fb00d000700 1 --2- 172.21.15.119:0/376105487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb008103500 0x7fb0080feb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:41.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.895+0000 7fb00d000700 1 -- 172.21.15.119:0/376105487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb008100be0 msgr2=0x7fb008100540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:41.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.895+0000 7fb00d000700 1 --2- 172.21.15.119:0/376105487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb008100be0 0x7fb008100540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:41.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.895+0000 7fb00d000700 1 -- 172.21.15.119:0/376105487 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff4009520 con 0x7fb008103500 2024-05-27T09:30:41.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.896+0000 7fb00d000700 1 --2- 172.21.15.119:0/376105487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb008103500 0x7fb0080feb90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7faff400b650 tx=0x7faff400f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:41.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.896+0000 7fb006000700 1 -- 172.21.15.119:0/376105487 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faff401c070 con 0x7fb008103500 2024-05-27T09:30:41.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.896+0000 7fb010400700 1 -- 172.21.15.119:0/376105487 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0080ff490 con 0x7fb008103500 2024-05-27T09:30:41.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.896+0000 7fb010400700 1 -- 172.21.15.119:0/376105487 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0080ff950 con 0x7fb008103500 2024-05-27T09:30:41.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.897+0000 7fb006000700 1 -- 172.21.15.119:0/376105487 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faff400fc80 con 0x7fb008103500 2024-05-27T09:30:41.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.897+0000 7fb006000700 1 -- 172.21.15.119:0/376105487 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faff4022cc0 con 0x7fb008103500 2024-05-27T09:30:41.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.898+0000 7fb010400700 1 -- 172.21.15.119:0/376105487 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafe8005320 con 0x7fb008103500 2024-05-27T09:30:41.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.901+0000 7fb006000700 1 -- 172.21.15.119:0/376105487 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 31) v1 ==== 100037+0+0 (secure 0 0 0) 0x7faff4020070 con 0x7fb008103500 2024-05-27T09:30:41.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.902+0000 7fb006000700 1 --2- 172.21.15.119:0/376105487 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7faff0078880 0x7faff007ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:41.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.903+0000 7fb006000700 1 -- 172.21.15.119:0/376105487 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7faff409c620 con 0x7fb008103500 2024-05-27T09:30:41.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.903+0000 7fb00da00700 1 --2- 172.21.15.119:0/376105487 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7faff0078880 0x7faff007ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:41.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.909+0000 7fb00da00700 1 --2- 172.21.15.119:0/376105487 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7faff0078880 0x7faff007ad30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faffc005950 tx=0x7faffc009310 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:41.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:41.910+0000 7fb006000700 1 -- 172.21.15.119:0/376105487 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7faff4064560 con 0x7fb008103500 2024-05-27T09:30:42.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.215+0000 7fb010400700 1 -- 172.21.15.119:0/376105487 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fafe8000bf0 con 0x7faff0078880 2024-05-27T09:30:42.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.216+0000 7fb006000700 1 -- 172.21.15.119:0/376105487 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fafe8000bf0 con 0x7faff0078880 2024-05-27T09:30:42.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.220+0000 7fb010400700 1 -- 172.21.15.119:0/376105487 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7faff0078880 msgr2=0x7faff007ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:42.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.220+0000 7fb010400700 1 --2- 172.21.15.119:0/376105487 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7faff0078880 0x7faff007ad30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faffc005950 tx=0x7faffc009310 comp rx=0 tx=0).stop 2024-05-27T09:30:42.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.220+0000 7fb010400700 1 -- 172.21.15.119:0/376105487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb008103500 msgr2=0x7fb0080feb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:42.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.220+0000 7fb010400700 1 --2- 172.21.15.119:0/376105487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb008103500 0x7fb0080feb90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7faff400b650 tx=0x7faff400f460 comp rx=0 tx=0).stop 2024-05-27T09:30:42.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.221+0000 7fb010400700 1 -- 172.21.15.119:0/376105487 shutdown_connections 2024-05-27T09:30:42.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.221+0000 7fb010400700 1 --2- 172.21.15.119:0/376105487 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7faff0078880 0x7faff007ad30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:42.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.221+0000 7fb010400700 1 --2- 172.21.15.119:0/376105487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb008100be0 0x7fb008100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:42.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.221+0000 7fb010400700 1 --2- 172.21.15.119:0/376105487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb008103500 0x7fb0080feb90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:42.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.221+0000 7fb010400700 1 -- 172.21.15.119:0/376105487 >> 172.21.15.119:0/376105487 conn(0x7fb0080fa5e0 msgr2=0x7fb0080fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:42.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.221+0000 7fb010400700 1 -- 172.21.15.119:0/376105487 shutdown_connections 2024-05-27T09:30:42.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.221+0000 7fb010400700 1 -- 172.21.15.119:0/376105487 wait complete. 2024-05-27T09:30:42.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.391+0000 7fdf31e00700 1 -- 172.21.15.119:0/186003876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf2c102380 msgr2=0x7fdf2c102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:42.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.391+0000 7fdf31e00700 1 --2- 172.21.15.119:0/186003876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf2c102380 0x7fdf2c102790 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009910 tx=0x7fdf1c009c20 comp rx=0 tx=0).stop 2024-05-27T09:30:42.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.391+0000 7fdf31e00700 1 -- 172.21.15.119:0/186003876 shutdown_connections 2024-05-27T09:30:42.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.391+0000 7fdf31e00700 1 --2- 172.21.15.119:0/186003876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf2c103580 0x7fdf2c1039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:42.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.391+0000 7fdf31e00700 1 --2- 172.21.15.119:0/186003876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf2c102380 0x7fdf2c102790 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:42.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.391+0000 7fdf31e00700 1 -- 172.21.15.119:0/186003876 >> 172.21.15.119:0/186003876 conn(0x7fdf2c0fd930 msgr2=0x7fdf2c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:42.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.391+0000 7fdf31e00700 1 -- 172.21.15.119:0/186003876 shutdown_connections 2024-05-27T09:30:42.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.392+0000 7fdf31e00700 1 -- 172.21.15.119:0/186003876 wait complete. 2024-05-27T09:30:42.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.392+0000 7fdf31e00700 1 Processor -- start 2024-05-27T09:30:42.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.393+0000 7fdf31e00700 1 -- start start 2024-05-27T09:30:42.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.393+0000 7fdf31e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf2c102380 0x7fdf2c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:42.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.393+0000 7fdf31e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf2c103580 0x7fdf2c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:42.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.393+0000 7fdf31e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c198720 con 0x7fdf2c103580 2024-05-27T09:30:42.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.393+0000 7fdf31e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c198860 con 0x7fdf2c102380 2024-05-27T09:30:42.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.394+0000 7fdf2aa00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf2c102380 0x7fdf2c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:42.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.394+0000 7fdf23e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf2c103580 0x7fdf2c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:42.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.394+0000 7fdf23e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf2c103580 0x7fdf2c198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35978/0 (socket says 172.21.15.119:35978) 2024-05-27T09:30:42.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.394+0000 7fdf23e00700 1 -- 172.21.15.119:0/259572965 learned_addr learned my addr 172.21.15.119:0/259572965 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:30:42.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.394+0000 7fdf23e00700 1 -- 172.21.15.119:0/259572965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf2c102380 msgr2=0x7fdf2c197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:42.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.394+0000 7fdf23e00700 1 --2- 172.21.15.119:0/259572965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf2c102380 0x7fdf2c197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:42.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.394+0000 7fdf23e00700 1 -- 172.21.15.119:0/259572965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c0095f0 con 0x7fdf2c103580 2024-05-27T09:30:42.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.394+0000 7fdf23e00700 1 --2- 172.21.15.119:0/259572965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf2c103580 0x7fdf2c198100 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400d900 tx=0x7fdf1400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:42.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.395+0000 7fdf30a00700 1 -- 172.21.15.119:0/259572965 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf14004a00 con 0x7fdf2c103580 2024-05-27T09:30:42.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.395+0000 7fdf30a00700 1 -- 172.21.15.119:0/259572965 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdf140054d0 con 0x7fdf2c103580 2024-05-27T09:30:42.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.395+0000 7fdf30a00700 1 -- 172.21.15.119:0/259572965 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf14009d00 con 0x7fdf2c103580 2024-05-27T09:30:42.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.395+0000 7fdf31e00700 1 -- 172.21.15.119:0/259572965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2c19d310 con 0x7fdf2c103580 2024-05-27T09:30:42.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.395+0000 7fdf31e00700 1 -- 172.21.15.119:0/259572965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2c19d860 con 0x7fdf2c103580 2024-05-27T09:30:42.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.396+0000 7fdf31e00700 1 -- 172.21.15.119:0/259572965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2c10aef0 con 0x7fdf2c103580 2024-05-27T09:30:42.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.398+0000 7fdf30a00700 1 -- 172.21.15.119:0/259572965 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 31) v1 ==== 100037+0+0 (secure 0 0 0) 0x7fdf14005000 con 0x7fdf2c103580 2024-05-27T09:30:42.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.399+0000 7fdf30a00700 1 --2- 172.21.15.119:0/259572965 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fdf10078830 0x7fdf1007ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:42.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.399+0000 7fdf30a00700 1 -- 172.21.15.119:0/259572965 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fdf1409adb0 con 0x7fdf2c103580 2024-05-27T09:30:42.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.400+0000 7fdf2aa00700 1 --2- 172.21.15.119:0/259572965 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fdf10078830 0x7fdf1007ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:42.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.401+0000 7fdf2aa00700 1 --2- 172.21.15.119:0/259572965 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fdf10078830 0x7fdf1007ace0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c006010 tx=0x7fdf1c00b560 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:42.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.402+0000 7fdf30a00700 1 -- 172.21.15.119:0/259572965 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7fdf14062cf0 con 0x7fdf2c103580 2024-05-27T09:30:42.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.720+0000 7fdf31e00700 1 -- 172.21.15.119:0/259572965 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf2c107ed0 con 0x7fdf10078830 2024-05-27T09:30:42.723 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:42 smithi119 ceph-mon[29082]: mgrmap e31: smithi119.ncwmog(active, since 13s), standbys: smithi138.dxsuqf 2024-05-27T09:30:42.723 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:42 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr metadata", "who": "smithi138.dxsuqf", "id": "smithi138.dxsuqf"}]: dispatch 2024-05-27T09:30:42.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.750+0000 7fdf30a00700 1 -- 172.21.15.119:0/259572965 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fdf2c107ed0 con 0x7fdf10078830 2024-05-27T09:30:42.752 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:30:42.752 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (9m) 5s ago 11m 13.9M - 0.25.0 c8568f914cd2 bc280230ab28 2024-05-27T09:30:42.752 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (12m) 5s ago 12m 7864k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:30:42.752 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (10m) 5s ago 10m 7742k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:30:42.753 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (12m) 5s ago 12m 6665k - 18.2.0 dc2bc1663786 e5fafde44183 2024-05-27T09:30:42.753 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (10m) 5s ago 10m 6660k - 18.2.0 dc2bc1663786 46956ab99bab 2024-05-27T09:30:42.753 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (9m) 5s ago 11m 75.3M - 9.4.7 954c08fa6188 16eeaded4295 2024-05-27T09:30:42.753 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (4m) 5s ago 4m 14.3M - 18.2.0 dc2bc1663786 93b21713fb1e 2024-05-27T09:30:42.753 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (4m) 5s ago 4m 205M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:30:42.753 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (4m) 5s ago 4m 14.3M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:30:42.753 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (4m) 5s ago 4m 163M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:30:42.753 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (30s) 5s ago 13m 561M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:30:42.753 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (107s) 5s ago 10m 139M - 19.0.0-3868-ga6507388 850a06a84d28 be5c0668306a 2024-05-27T09:30:42.753 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (13m) 5s ago 13m 54.5M 2048M 18.2.0 dc2bc1663786 ba998b789a1c 2024-05-27T09:30:42.753 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (10m) 5s ago 10m 44.8M 2048M 18.2.0 dc2bc1663786 60216fd67078 2024-05-27T09:30:42.753 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (12m) 5s ago 12m 8032k - 1.5.0 0da6a335fe13 79c36d4e5b04 2024-05-27T09:30:42.753 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (10m) 5s ago 10m 8019k - 1.5.0 0da6a335fe13 80ee890e8830 2024-05-27T09:30:42.754 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (9m) 5s ago 9m 201M 1216M 18.2.0 dc2bc1663786 825bd1392508 2024-05-27T09:30:42.754 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (8m) 5s ago 8m 222M 1216M 18.2.0 dc2bc1663786 fce9819903d9 2024-05-27T09:30:42.754 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (8m) 5s ago 8m 169M 1216M 18.2.0 dc2bc1663786 3955a6112c1c 2024-05-27T09:30:42.754 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (7m) 5s ago 7m 242M 2240M 18.2.0 dc2bc1663786 693c92220622 2024-05-27T09:30:42.754 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (7m) 5s ago 7m 215M 2240M 18.2.0 dc2bc1663786 7559e658e917 2024-05-27T09:30:42.754 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (6m) 5s ago 6m 213M 2240M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:30:42.754 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (42s) 5s ago 11m 48.7M - 2.43.0 a07b618ecd1d f2dc450b96cb 2024-05-27T09:30:42.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.754+0000 7fdf31e00700 1 -- 172.21.15.119:0/259572965 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fdf10078830 msgr2=0x7fdf1007ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:42.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.754+0000 7fdf31e00700 1 --2- 172.21.15.119:0/259572965 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fdf10078830 0x7fdf1007ace0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c006010 tx=0x7fdf1c00b560 comp rx=0 tx=0).stop 2024-05-27T09:30:42.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.754+0000 7fdf31e00700 1 -- 172.21.15.119:0/259572965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf2c103580 msgr2=0x7fdf2c198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:42.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.754+0000 7fdf31e00700 1 --2- 172.21.15.119:0/259572965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf2c103580 0x7fdf2c198100 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400d900 tx=0x7fdf1400dc10 comp rx=0 tx=0).stop 2024-05-27T09:30:42.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.755+0000 7fdf31e00700 1 -- 172.21.15.119:0/259572965 shutdown_connections 2024-05-27T09:30:42.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.755+0000 7fdf31e00700 1 --2- 172.21.15.119:0/259572965 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fdf10078830 0x7fdf1007ace0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:42.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.755+0000 7fdf31e00700 1 --2- 172.21.15.119:0/259572965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf2c102380 0x7fdf2c197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:42.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.755+0000 7fdf31e00700 1 --2- 172.21.15.119:0/259572965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf2c103580 0x7fdf2c198100 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:42.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.755+0000 7fdf31e00700 1 -- 172.21.15.119:0/259572965 >> 172.21.15.119:0/259572965 conn(0x7fdf2c0fd930 msgr2=0x7fdf2c1067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:42.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.756+0000 7fdf31e00700 1 -- 172.21.15.119:0/259572965 shutdown_connections 2024-05-27T09:30:42.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:42.756+0000 7fdf31e00700 1 -- 172.21.15.119:0/259572965 wait complete. 2024-05-27T09:30:42.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:42 smithi138 ceph-mon[34885]: mgrmap e31: smithi119.ncwmog(active, since 13s), standbys: smithi138.dxsuqf 2024-05-27T09:30:42.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:42 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr metadata", "who": "smithi138.dxsuqf", "id": "smithi138.dxsuqf"}]: dispatch 2024-05-27T09:30:43.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.011+0000 7f83ec400700 1 -- 172.21.15.119:0/97050132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e4101330 msgr2=0x7f83e41037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:43.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.011+0000 7f83ec400700 1 --2- 172.21.15.119:0/97050132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e4101330 0x7f83e41037b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009870 tx=0x7f83d0009b80 comp rx=0 tx=0).stop 2024-05-27T09:30:43.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.011+0000 7f83ec400700 1 -- 172.21.15.119:0/97050132 shutdown_connections 2024-05-27T09:30:43.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.011+0000 7f83ec400700 1 --2- 172.21.15.119:0/97050132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e4101330 0x7f83e41037b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:43.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.011+0000 7f83ec400700 1 --2- 172.21.15.119:0/97050132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83e40fe9d0 0x7f83e4100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:43.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.011+0000 7f83ec400700 1 -- 172.21.15.119:0/97050132 >> 172.21.15.119:0/97050132 conn(0x7f83e40fa5e0 msgr2=0x7f83e40fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:43.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.011+0000 7f83ec400700 1 -- 172.21.15.119:0/97050132 shutdown_connections 2024-05-27T09:30:43.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.012+0000 7f83ec400700 1 -- 172.21.15.119:0/97050132 wait complete. 2024-05-27T09:30:43.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.013+0000 7f83ec400700 1 Processor -- start 2024-05-27T09:30:43.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.013+0000 7f83ec400700 1 -- start start 2024-05-27T09:30:43.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.013+0000 7f83ec400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83e40fe9d0 0x7f83e419bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:43.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.013+0000 7f83ec400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e4101330 0x7f83e419c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:43.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.013+0000 7f83ec400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e419ca90 con 0x7f83e4101330 2024-05-27T09:30:43.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.013+0000 7f83ec400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e419cbd0 con 0x7f83e40fe9d0 2024-05-27T09:30:43.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.014+0000 7f83e9000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e4101330 0x7f83e419c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:43.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.014+0000 7f83e9000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e4101330 0x7f83e419c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36004/0 (socket says 172.21.15.119:36004) 2024-05-27T09:30:43.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.014+0000 7f83e9000700 1 -- 172.21.15.119:0/1557566835 learned_addr learned my addr 172.21.15.119:0/1557566835 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:30:43.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.014+0000 7f83e9a00700 1 --2- 172.21.15.119:0/1557566835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83e40fe9d0 0x7f83e419bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:43.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.014+0000 7f83e9000700 1 -- 172.21.15.119:0/1557566835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83e40fe9d0 msgr2=0x7f83e419bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:43.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.014+0000 7f83e9000700 1 --2- 172.21.15.119:0/1557566835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83e40fe9d0 0x7f83e419bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:43.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.015+0000 7f83e9000700 1 -- 172.21.15.119:0/1557566835 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d0009520 con 0x7f83e4101330 2024-05-27T09:30:43.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.015+0000 7f83e9000700 1 --2- 172.21.15.119:0/1557566835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e4101330 0x7f83e419c470 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009230 tx=0x7f83d000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:43.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.016+0000 7f83e2000700 1 -- 172.21.15.119:0/1557566835 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83d001c070 con 0x7f83e4101330 2024-05-27T09:30:43.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.016+0000 7f83e2000700 1 -- 172.21.15.119:0/1557566835 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f83d000fe40 con 0x7f83e4101330 2024-05-27T09:30:43.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.016+0000 7f83ec400700 1 -- 172.21.15.119:0/1557566835 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e41a1620 con 0x7f83e4101330 2024-05-27T09:30:43.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.016+0000 7f83e2000700 1 -- 172.21.15.119:0/1557566835 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83d00173f0 con 0x7f83e4101330 2024-05-27T09:30:43.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.016+0000 7f83ec400700 1 -- 172.21.15.119:0/1557566835 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e41a1b90 con 0x7f83e4101330 2024-05-27T09:30:43.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.019+0000 7f83ec400700 1 -- 172.21.15.119:0/1557566835 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c4005320 con 0x7f83e4101330 2024-05-27T09:30:43.022 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.019+0000 7f83e2000700 1 -- 172.21.15.119:0/1557566835 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 31) v1 ==== 100037+0+0 (secure 0 0 0) 0x7f83d0020070 con 0x7f83e4101330 2024-05-27T09:30:43.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.021+0000 7f83e2000700 1 --2- 172.21.15.119:0/1557566835 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f83cc078910 0x7f83cc07adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:43.023 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.021+0000 7f83e2000700 1 -- 172.21.15.119:0/1557566835 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f83d009c090 con 0x7f83e4101330 2024-05-27T09:30:43.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.022+0000 7f83e9a00700 1 --2- 172.21.15.119:0/1557566835 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f83cc078910 0x7f83cc07adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:43.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.025+0000 7f83e9a00700 1 --2- 172.21.15.119:0/1557566835 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f83cc078910 0x7f83cc07adc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009380 tx=0x7f83d800b040 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:43.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.031+0000 7f83e2000700 1 -- 172.21.15.119:0/1557566835 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f83d0064bb0 con 0x7f83e4101330 2024-05-27T09:30:43.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.507+0000 7f83ec400700 1 -- 172.21.15.119:0/1557566835 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f83c4005cc0 con 0x7f83e4101330 2024-05-27T09:30:43.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.508+0000 7f83e2000700 1 -- 172.21.15.119:0/1557566835 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f83d0064550 con 0x7f83e4101330 2024-05-27T09:30:43.511 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:30:43.511 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:30:43.511 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-05-27T09:30:43.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:30:43.511 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:30:43.511 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:30:43.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:30:43.511 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:30:43.511 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-05-27T09:30:43.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:30:43.511 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:30:43.511 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-05-27T09:30:43.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:30:43.512 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:30:43.512 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-05-27T09:30:43.512 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:30:43.512 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:30:43.512 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:30:43.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.512+0000 7f83ec400700 1 -- 172.21.15.119:0/1557566835 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f83cc078910 msgr2=0x7f83cc07adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:43.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.512+0000 7f83ec400700 1 --2- 172.21.15.119:0/1557566835 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f83cc078910 0x7f83cc07adc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009380 tx=0x7f83d800b040 comp rx=0 tx=0).stop 2024-05-27T09:30:43.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.512+0000 7f83ec400700 1 -- 172.21.15.119:0/1557566835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e4101330 msgr2=0x7f83e419c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:43.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.512+0000 7f83ec400700 1 --2- 172.21.15.119:0/1557566835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e4101330 0x7f83e419c470 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009230 tx=0x7f83d000f740 comp rx=0 tx=0).stop 2024-05-27T09:30:43.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.512+0000 7f83ec400700 1 -- 172.21.15.119:0/1557566835 shutdown_connections 2024-05-27T09:30:43.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.512+0000 7f83ec400700 1 --2- 172.21.15.119:0/1557566835 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f83cc078910 0x7f83cc07adc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:43.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.512+0000 7f83ec400700 1 --2- 172.21.15.119:0/1557566835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83e40fe9d0 0x7f83e419bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:43.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.512+0000 7f83ec400700 1 --2- 172.21.15.119:0/1557566835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e4101330 0x7f83e419c470 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:43.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.513+0000 7f83ec400700 1 -- 172.21.15.119:0/1557566835 >> 172.21.15.119:0/1557566835 conn(0x7f83e40fa5e0 msgr2=0x7f83e40fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:43.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.513+0000 7f83ec400700 1 -- 172.21.15.119:0/1557566835 shutdown_connections 2024-05-27T09:30:43.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.513+0000 7f83ec400700 1 -- 172.21.15.119:0/1557566835 wait complete. 2024-05-27T09:30:43.702 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:43 smithi119 ceph-mon[29082]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:30:43.703 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:43 smithi119 ceph-mon[29082]: pgmap v10: 65 pgs: 65 active+clean; 338 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 56 KiB/s rd, 1.8 MiB/s wr, 379 op/s 2024-05-27T09:30:43.703 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:43 smithi119 ceph-mon[29082]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:30:43.703 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:43 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:30:43.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.743+0000 7f2e20200700 1 -- 172.21.15.119:0/2059131934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e180725d0 msgr2=0x7f2e18107d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:43.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.743+0000 7f2e20200700 1 --2- 172.21.15.119:0/2059131934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e180725d0 0x7f2e18107d50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009910 tx=0x7f2e0c009c20 comp rx=0 tx=0).stop 2024-05-27T09:30:43.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.743+0000 7f2e20200700 1 -- 172.21.15.119:0/2059131934 shutdown_connections 2024-05-27T09:30:43.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.743+0000 7f2e20200700 1 --2- 172.21.15.119:0/2059131934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e180725d0 0x7f2e18107d50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:43.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.743+0000 7f2e20200700 1 --2- 172.21.15.119:0/2059131934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e18071c80 0x7f2e18072090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:43.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.743+0000 7f2e20200700 1 -- 172.21.15.119:0/2059131934 >> 172.21.15.119:0/2059131934 conn(0x7f2e1806d050 msgr2=0x7f2e1806f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:43.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.743+0000 7f2e20200700 1 -- 172.21.15.119:0/2059131934 shutdown_connections 2024-05-27T09:30:43.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.744+0000 7f2e20200700 1 -- 172.21.15.119:0/2059131934 wait complete. 2024-05-27T09:30:43.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.745+0000 7f2e20200700 1 Processor -- start 2024-05-27T09:30:43.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.746+0000 7f2e20200700 1 -- start start 2024-05-27T09:30:43.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.747+0000 7f2e20200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e18071c80 0x7f2e1819bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:43.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.747+0000 7f2e20200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e180725d0 0x7f2e1819c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:43.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.747+0000 7f2e20200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1819cb00 con 0x7f2e180725d0 2024-05-27T09:30:43.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.747+0000 7f2e20200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e181b2560 con 0x7f2e18071c80 2024-05-27T09:30:43.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.747+0000 7f2e1ce00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e180725d0 0x7f2e1819c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:43.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.748+0000 7f2e1d800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e18071c80 0x7f2e1819bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:43.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.748+0000 7f2e1d800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e18071c80 0x7f2e1819bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.119:33638/0 (socket says 172.21.15.119:33638) 2024-05-27T09:30:43.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.748+0000 7f2e1d800700 1 -- 172.21.15.119:0/2664405008 learned_addr learned my addr 172.21.15.119:0/2664405008 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:30:43.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.748+0000 7f2e1d800700 1 -- 172.21.15.119:0/2664405008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e180725d0 msgr2=0x7f2e1819c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:43.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.748+0000 7f2e1d800700 1 --2- 172.21.15.119:0/2664405008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e180725d0 0x7f2e1819c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:43.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.748+0000 7f2e1d800700 1 -- 172.21.15.119:0/2664405008 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0c0095f0 con 0x7f2e18071c80 2024-05-27T09:30:43.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.749+0000 7f2e1d800700 1 --2- 172.21.15.119:0/2664405008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e18071c80 0x7f2e1819bfa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400e920 tx=0x7f2e0400ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:43.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.750+0000 7f2e16000700 1 -- 172.21.15.119:0/2664405008 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e0400c9d0 con 0x7f2e18071c80 2024-05-27T09:30:43.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.750+0000 7f2e16000700 1 -- 172.21.15.119:0/2664405008 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2e04010430 con 0x7f2e18071c80 2024-05-27T09:30:43.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.750+0000 7f2e16000700 1 -- 172.21.15.119:0/2664405008 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e04018650 con 0x7f2e18071c80 2024-05-27T09:30:43.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.750+0000 7f2e20200700 1 -- 172.21.15.119:0/2664405008 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e181b2760 con 0x7f2e18071c80 2024-05-27T09:30:43.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.751+0000 7f2e20200700 1 -- 172.21.15.119:0/2664405008 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e181b2c30 con 0x7f2e18071c80 2024-05-27T09:30:43.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.754+0000 7f2e20200700 1 -- 172.21.15.119:0/2664405008 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e1810f220 con 0x7f2e18071c80 2024-05-27T09:30:43.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.754+0000 7f2e16000700 1 -- 172.21.15.119:0/2664405008 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) v1 ==== 100037+0+0 (secure 0 0 0) 0x7f2e040105a0 con 0x7f2e18071c80 2024-05-27T09:30:43.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.756+0000 7f2e16000700 1 --2- 172.21.15.119:0/2664405008 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f2e000787f0 0x7f2e0007aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:30:43.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.756+0000 7f2e16000700 1 -- 172.21.15.119:0/2664405008 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2e04014070 con 0x7f2e18071c80 2024-05-27T09:30:43.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.757+0000 7f2e1ce00700 1 --2- 172.21.15.119:0/2664405008 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f2e000787f0 0x7f2e0007aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:30:43.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.758+0000 7f2e1ce00700 1 --2- 172.21.15.119:0/2664405008 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f2e000787f0 0x7f2e0007aca0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00b6f0 tx=0x7f2e0c005fd0 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:30:43.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:43.767+0000 7f2e16000700 1 -- 172.21.15.119:0/2664405008 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f2e04054a30 con 0x7f2e18071c80 2024-05-27T09:30:43.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:43 smithi138 ceph-mon[34885]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:30:43.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:43 smithi138 ceph-mon[34885]: pgmap v10: 65 pgs: 65 active+clean; 338 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 56 KiB/s rd, 1.8 MiB/s wr, 379 op/s 2024-05-27T09:30:43.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:43 smithi138 ceph-mon[34885]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:30:43.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:43 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:30:44.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:44.194+0000 7f2e20200700 1 -- 172.21.15.119:0/2664405008 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e1810e750 con 0x7f2e000787f0 2024-05-27T09:30:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:44.196+0000 7f2e16000700 1 -- 172.21.15.119:0/2664405008 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f2e1810e750 con 0x7f2e000787f0 2024-05-27T09:30:44.197 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:30:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:30:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:30:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-27T09:30:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [ 2024-05-27T09:30:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr" 2024-05-27T09:30:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-05-27T09:30:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "2/2 daemons upgraded", 2024-05-27T09:30:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "", 2024-05-27T09:30:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:30:44.198 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:30:44.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:44.198+0000 7f2e20200700 1 -- 172.21.15.119:0/2664405008 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f2e000787f0 msgr2=0x7f2e0007aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:44.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:44.198+0000 7f2e20200700 1 --2- 172.21.15.119:0/2664405008 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f2e000787f0 0x7f2e0007aca0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00b6f0 tx=0x7f2e0c005fd0 comp rx=0 tx=0).stop 2024-05-27T09:30:44.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:44.198+0000 7f2e20200700 1 -- 172.21.15.119:0/2664405008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e18071c80 msgr2=0x7f2e1819bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:30:44.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:44.198+0000 7f2e20200700 1 --2- 172.21.15.119:0/2664405008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e18071c80 0x7f2e1819bfa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400e920 tx=0x7f2e0400ec30 comp rx=0 tx=0).stop 2024-05-27T09:30:44.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:44.199+0000 7f2e20200700 1 -- 172.21.15.119:0/2664405008 shutdown_connections 2024-05-27T09:30:44.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:44.199+0000 7f2e20200700 1 --2- 172.21.15.119:0/2664405008 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f2e000787f0 0x7f2e0007aca0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:44.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:44.199+0000 7f2e20200700 1 --2- 172.21.15.119:0/2664405008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e18071c80 0x7f2e1819bfa0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:44.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:44.199+0000 7f2e20200700 1 --2- 172.21.15.119:0/2664405008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e180725d0 0x7f2e1819c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:30:44.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:44.199+0000 7f2e20200700 1 -- 172.21.15.119:0/2664405008 >> 172.21.15.119:0/2664405008 conn(0x7f2e1806d050 msgr2=0x7f2e1810d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:30:44.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:44.199+0000 7f2e20200700 1 -- 172.21.15.119:0/2664405008 shutdown_connections 2024-05-27T09:30:44.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:30:44.199+0000 7f2e20200700 1 -- 172.21.15.119:0/2664405008 wait complete. 2024-05-27T09:30:44.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:44 smithi119 ceph-mon[29082]: from='client.14680 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:30:44.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:44 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/1557566835' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:30:44.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:44 smithi138 ceph-mon[34885]: from='client.14680 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:30:44.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:44 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/1557566835' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:30:45.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:45 smithi119 ceph-mon[29082]: pgmap v11: 65 pgs: 65 active+clean; 338 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 56 KiB/s rd, 1.8 MiB/s wr, 379 op/s 2024-05-27T09:30:45.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:45 smithi119 ceph-mon[29082]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:30:45.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:45 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:45.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:45 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:45.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:45 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:30:45.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:45 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:30:45.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:45 smithi138 ceph-mon[34885]: pgmap v11: 65 pgs: 65 active+clean; 338 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 56 KiB/s rd, 1.8 MiB/s wr, 379 op/s 2024-05-27T09:30:45.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:45 smithi138 ceph-mon[34885]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:30:45.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:45 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:45.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:45 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:45.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:45 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:30:45.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:45 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:30:46.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:46 smithi119 ceph-mon[29082]: Updating smithi119:/etc/ceph/ceph.conf 2024-05-27T09:30:46.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:46 smithi119 ceph-mon[29082]: Updating smithi138:/etc/ceph/ceph.conf 2024-05-27T09:30:46.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:46 smithi119 ceph-mon[29082]: Updating smithi119:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:30:46.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:46 smithi119 ceph-mon[29082]: Updating smithi138:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:30:46.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:46 smithi138 ceph-mon[34885]: Updating smithi119:/etc/ceph/ceph.conf 2024-05-27T09:30:46.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:46 smithi138 ceph-mon[34885]: Updating smithi138:/etc/ceph/ceph.conf 2024-05-27T09:30:46.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:46 smithi138 ceph-mon[34885]: Updating smithi119:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:30:46.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:46 smithi138 ceph-mon[34885]: Updating smithi138:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:30:47.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:47 smithi119 ceph-mon[29082]: pgmap v12: 65 pgs: 65 active+clean; 338 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 78 KiB/s rd, 2.8 MiB/s wr, 610 op/s 2024-05-27T09:30:47.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:47 smithi119 ceph-mon[29082]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-05-27T09:30:47.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:47 smithi119 ceph-mon[29082]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-05-27T09:30:47.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:47 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:47.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:47 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:47.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:47 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:47.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:47 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:47.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:47 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:47.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:47 smithi138 ceph-mon[34885]: pgmap v12: 65 pgs: 65 active+clean; 338 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 78 KiB/s rd, 2.8 MiB/s wr, 610 op/s 2024-05-27T09:30:47.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:47 smithi138 ceph-mon[34885]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-05-27T09:30:47.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:47 smithi138 ceph-mon[34885]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-05-27T09:30:47.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:47 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:47.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:47 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:47.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:47 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:47.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:47 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:47.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:47 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:48.718 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:48 smithi119 ceph-mon[29082]: Updating smithi119:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T09:30:48.719 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:48 smithi119 ceph-mon[29082]: Updating smithi138:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T09:30:48.719 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:48 smithi119 ceph-mon[29082]: Reconfiguring prometheus.smithi119 (dependencies changed)... 2024-05-27T09:30:48.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:48 smithi138 ceph-mon[34885]: Updating smithi119:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T09:30:48.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:48 smithi138 ceph-mon[34885]: Updating smithi138:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T09:30:48.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:48 smithi138 ceph-mon[34885]: Reconfiguring prometheus.smithi119 (dependencies changed)... 2024-05-27T09:30:49.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:49 smithi119 ceph-mon[29082]: Reconfiguring daemon prometheus.smithi119 on smithi119 2024-05-27T09:30:49.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:49 smithi119 ceph-mon[29082]: pgmap v13: 65 pgs: 65 active+clean; 338 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45 KiB/s rd, 2.1 MiB/s wr, 406 op/s 2024-05-27T09:30:49.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:49 smithi138 ceph-mon[34885]: Reconfiguring daemon prometheus.smithi119 on smithi119 2024-05-27T09:30:49.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:49 smithi138 ceph-mon[34885]: pgmap v13: 65 pgs: 65 active+clean; 338 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45 KiB/s rd, 2.1 MiB/s wr, 406 op/s 2024-05-27T09:30:50.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:50 smithi119 ceph-mon[29082]: pgmap v14: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 62 KiB/s rd, 3.1 MiB/s wr, 600 op/s 2024-05-27T09:30:50.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:50 smithi138 ceph-mon[34885]: pgmap v14: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 62 KiB/s rd, 3.1 MiB/s wr, 600 op/s 2024-05-27T09:30:53.427 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:53 smithi119 ceph-mon[29082]: pgmap v15: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 39 KiB/s rd, 2.1 MiB/s wr, 425 op/s 2024-05-27T09:30:53.607 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:53 smithi138 ceph-mon[34885]: pgmap v15: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 39 KiB/s rd, 2.1 MiB/s wr, 425 op/s 2024-05-27T09:30:55.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:55 smithi119 ceph-mon[29082]: pgmap v16: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 39 KiB/s rd, 2.1 MiB/s wr, 425 op/s 2024-05-27T09:30:55.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:55 smithi138 ceph-mon[34885]: pgmap v16: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 39 KiB/s rd, 2.1 MiB/s wr, 425 op/s 2024-05-27T09:30:57.368 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:57 smithi119 ceph-mon[29082]: pgmap v17: 65 pgs: 65 active+clean; 336 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 2.9 MiB/s wr, 561 op/s 2024-05-27T09:30:57.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:57 smithi138 ceph-mon[34885]: pgmap v17: 65 pgs: 65 active+clean; 336 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 2.9 MiB/s wr, 561 op/s 2024-05-27T09:30:58.493 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:58 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:30:58.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:58 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:30:59.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:59 smithi119 ceph-mon[29082]: pgmap v18: 65 pgs: 65 active+clean; 336 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.8 MiB/s wr, 329 op/s 2024-05-27T09:30:59.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:59 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:59.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:59 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:59.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:59 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-27T09:30:59.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:30:59 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:30:59.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:59 smithi138 ceph-mon[34885]: pgmap v18: 65 pgs: 65 active+clean; 336 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.8 MiB/s wr, 329 op/s 2024-05-27T09:30:59.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:59 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:59.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:59 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:30:59.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:59 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-27T09:30:59.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:30:59 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:31:00.366 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:00 smithi138 ceph-mon[34885]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-27T09:31:00.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:00 smithi119 ceph-mon[29082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-27T09:31:01.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:01 smithi119 ceph-mon[29082]: pgmap v19: 65 pgs: 65 active+clean; 336 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 2.3 MiB/s wr, 446 op/s 2024-05-27T09:31:01.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:01 smithi138 ceph-mon[34885]: pgmap v19: 65 pgs: 65 active+clean; 336 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 2.3 MiB/s wr, 446 op/s 2024-05-27T09:31:03.450 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:03 smithi119 ceph-mon[29082]: pgmap v20: 65 pgs: 65 active+clean; 336 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 252 op/s 2024-05-27T09:31:03.451 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:03 smithi119 ceph-mon[29082]: Upgrade: Updating mgr.smithi138.dxsuqf 2024-05-27T09:31:03.451 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:03 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.dxsuqf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T09:31:03.451 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:03 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T09:31:03.451 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:03 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:31:03.451 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:03 smithi119 ceph-mon[29082]: Deploying daemon mgr.smithi138.dxsuqf on smithi138 2024-05-27T09:31:03.590 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:03 smithi138 ceph-mon[34885]: pgmap v20: 65 pgs: 65 active+clean; 336 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 252 op/s 2024-05-27T09:31:03.590 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:03 smithi138 ceph-mon[34885]: Upgrade: Updating mgr.smithi138.dxsuqf 2024-05-27T09:31:03.590 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:03 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.dxsuqf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T09:31:03.590 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:03 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T09:31:03.590 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:03 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:31:03.590 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:03 smithi138 ceph-mon[34885]: Deploying daemon mgr.smithi138.dxsuqf on smithi138 2024-05-27T09:31:05.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:05 smithi119 ceph-mon[29082]: pgmap v21: 65 pgs: 65 active+clean; 336 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 252 op/s 2024-05-27T09:31:05.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:05 smithi138 ceph-mon[34885]: pgmap v21: 65 pgs: 65 active+clean; 336 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 252 op/s 2024-05-27T09:31:07.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:07 smithi119 ceph-mon[29082]: pgmap v22: 65 pgs: 65 active+clean; 338 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.7 MiB/s wr, 340 op/s 2024-05-27T09:31:07.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:07 smithi138 ceph-mon[34885]: pgmap v22: 65 pgs: 65 active+clean; 338 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.7 MiB/s wr, 340 op/s 2024-05-27T09:31:09.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:09 smithi119 ceph-mon[29082]: pgmap v23: 65 pgs: 65 active+clean; 338 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 992 KiB/s wr, 205 op/s 2024-05-27T09:31:09.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:09 smithi138 ceph-mon[34885]: pgmap v23: 65 pgs: 65 active+clean; 338 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 992 KiB/s wr, 205 op/s 2024-05-27T09:31:11.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:11 smithi119 ceph-mon[29082]: pgmap v24: 65 pgs: 65 active+clean; 342 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.6 MiB/s wr, 330 op/s 2024-05-27T09:31:11.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:11 smithi138 ceph-mon[34885]: pgmap v24: 65 pgs: 65 active+clean; 342 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.6 MiB/s wr, 330 op/s 2024-05-27T09:31:13.477 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:13 smithi119 ceph-mon[29082]: pgmap v25: 65 pgs: 65 active+clean; 342 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 1.1 MiB/s wr, 213 op/s 2024-05-27T09:31:13.477 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:13 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:31:13.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:13 smithi138 ceph-mon[34885]: pgmap v25: 65 pgs: 65 active+clean; 342 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 1.1 MiB/s wr, 213 op/s 2024-05-27T09:31:13.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:13 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:31:14.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.424+0000 7f5baa600700 1 -- 172.21.15.119:0/163185178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba4075700 msgr2=0x7f5ba4075b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:14.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.424+0000 7f5baa600700 1 --2- 172.21.15.119:0/163185178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba4075700 0x7f5ba4075b10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009870 tx=0x7f5b94009b80 comp rx=0 tx=0).stop 2024-05-27T09:31:14.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.424+0000 7f5baa600700 1 -- 172.21.15.119:0/163185178 shutdown_connections 2024-05-27T09:31:14.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.424+0000 7f5baa600700 1 --2- 172.21.15.119:0/163185178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ba4076950 0x7f5ba4076dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:14.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.424+0000 7f5baa600700 1 --2- 172.21.15.119:0/163185178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba4075700 0x7f5ba4075b10 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:14.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.424+0000 7f5baa600700 1 -- 172.21.15.119:0/163185178 >> 172.21.15.119:0/163185178 conn(0x7f5ba40fd960 msgr2=0x7f5ba40ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:14.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.424+0000 7f5baa600700 1 -- 172.21.15.119:0/163185178 shutdown_connections 2024-05-27T09:31:14.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.424+0000 7f5baa600700 1 -- 172.21.15.119:0/163185178 wait complete. 2024-05-27T09:31:14.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.425+0000 7f5baa600700 1 Processor -- start 2024-05-27T09:31:14.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.426+0000 7f5baa600700 1 -- start start 2024-05-27T09:31:14.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.426+0000 7f5baa600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ba4075700 0x7f5ba419bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:14.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.426+0000 7f5baa600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba4076950 0x7f5ba419c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:14.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.426+0000 7f5baa600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba419cab0 con 0x7f5ba4076950 2024-05-27T09:31:14.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.426+0000 7f5baa600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba419cbf0 con 0x7f5ba4075700 2024-05-27T09:31:14.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.426+0000 7f5ba3400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ba4075700 0x7f5ba419bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:14.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.426+0000 7f5ba2a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba4076950 0x7f5ba419c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:14.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.426+0000 7f5ba3400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ba4075700 0x7f5ba419bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.119:40228/0 (socket says 172.21.15.119:40228) 2024-05-27T09:31:14.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.426+0000 7f5ba3400700 1 -- 172.21.15.119:0/4245174393 learned_addr learned my addr 172.21.15.119:0/4245174393 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:31:14.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.427+0000 7f5ba2a00700 1 -- 172.21.15.119:0/4245174393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ba4075700 msgr2=0x7f5ba419bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:14.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.427+0000 7f5ba2a00700 1 --2- 172.21.15.119:0/4245174393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ba4075700 0x7f5ba419bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:14.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.427+0000 7f5ba2a00700 1 -- 172.21.15.119:0/4245174393 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b94009520 con 0x7f5ba4076950 2024-05-27T09:31:14.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.427+0000 7f5ba3400700 1 --2- 172.21.15.119:0/4245174393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ba4075700 0x7f5ba419bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:31:14.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.427+0000 7f5ba2a00700 1 --2- 172.21.15.119:0/4245174393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba4076950 0x7f5ba419c490 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5b9000e940 tx=0x7f5b9000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:14.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.428+0000 7f5b9be00700 1 -- 172.21.15.119:0/4245174393 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b9000c990 con 0x7f5ba4076950 2024-05-27T09:31:14.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.428+0000 7f5b9be00700 1 -- 172.21.15.119:0/4245174393 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5b90010430 con 0x7f5ba4076950 2024-05-27T09:31:14.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.428+0000 7f5b9be00700 1 -- 172.21.15.119:0/4245174393 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b90018610 con 0x7f5ba4076950 2024-05-27T09:31:14.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.428+0000 7f5baa600700 1 -- 172.21.15.119:0/4245174393 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba41a16a0 con 0x7f5ba4076950 2024-05-27T09:31:14.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.428+0000 7f5baa600700 1 -- 172.21.15.119:0/4245174393 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba41a1c70 con 0x7f5ba4076950 2024-05-27T09:31:14.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.430+0000 7f5baa600700 1 -- 172.21.15.119:0/4245174393 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba410f300 con 0x7f5ba4076950 2024-05-27T09:31:14.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.432+0000 7f5b9be00700 1 -- 172.21.15.119:0/4245174393 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 31) v1 ==== 100037+0+0 (secure 0 0 0) 0x7f5b900105a0 con 0x7f5ba4076950 2024-05-27T09:31:14.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.434+0000 7f5b9be00700 1 --2- 172.21.15.119:0/4245174393 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f5b88078830 0x7f5b8807ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:14.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.434+0000 7f5b9be00700 1 -- 172.21.15.119:0/4245174393 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5b90014070 con 0x7f5ba4076950 2024-05-27T09:31:14.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.435+0000 7f5ba3400700 1 --2- 172.21.15.119:0/4245174393 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f5b88078830 0x7f5b8807ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:14.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.437+0000 7f5ba3400700 1 --2- 172.21.15.119:0/4245174393 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f5b88078830 0x7f5b8807ace0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400b6f0 tx=0x7f5b94019040 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:14.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.442+0000 7f5b9be00700 1 -- 172.21.15.119:0/4245174393 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f5b90054a30 con 0x7f5ba4076950 2024-05-27T09:31:14.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.737+0000 7f5baa600700 1 -- 172.21.15.119:0/4245174393 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ba410dd30 con 0x7f5b88078830 2024-05-27T09:31:14.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.739+0000 7f5b9be00700 1 -- 172.21.15.119:0/4245174393 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f5ba410dd30 con 0x7f5b88078830 2024-05-27T09:31:14.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.742+0000 7f5baa600700 1 -- 172.21.15.119:0/4245174393 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f5b88078830 msgr2=0x7f5b8807ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:14.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.742+0000 7f5baa600700 1 --2- 172.21.15.119:0/4245174393 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f5b88078830 0x7f5b8807ace0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400b6f0 tx=0x7f5b94019040 comp rx=0 tx=0).stop 2024-05-27T09:31:14.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.742+0000 7f5baa600700 1 -- 172.21.15.119:0/4245174393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba4076950 msgr2=0x7f5ba419c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:14.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.742+0000 7f5baa600700 1 --2- 172.21.15.119:0/4245174393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba4076950 0x7f5ba419c490 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5b9000e940 tx=0x7f5b9000ec50 comp rx=0 tx=0).stop 2024-05-27T09:31:14.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.742+0000 7f5baa600700 1 -- 172.21.15.119:0/4245174393 shutdown_connections 2024-05-27T09:31:14.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.742+0000 7f5baa600700 1 --2- 172.21.15.119:0/4245174393 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f5b88078830 0x7f5b8807ace0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:14.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.742+0000 7f5baa600700 1 --2- 172.21.15.119:0/4245174393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ba4075700 0x7f5ba419bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:14.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.742+0000 7f5baa600700 1 --2- 172.21.15.119:0/4245174393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ba4076950 0x7f5ba419c490 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:14.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.742+0000 7f5baa600700 1 -- 172.21.15.119:0/4245174393 >> 172.21.15.119:0/4245174393 conn(0x7f5ba40fd960 msgr2=0x7f5ba410c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:14.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.743+0000 7f5baa600700 1 -- 172.21.15.119:0/4245174393 shutdown_connections 2024-05-27T09:31:14.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.743+0000 7f5baa600700 1 -- 172.21.15.119:0/4245174393 wait complete. 2024-05-27T09:31:14.761 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:31:14.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.925+0000 7f4c7bc00700 1 -- 172.21.15.119:0/3417927321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c740737b0 msgr2=0x7f4c74073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:14.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.925+0000 7f4c7bc00700 1 --2- 172.21.15.119:0/3417927321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c740737b0 0x7f4c74073c20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009940 tx=0x7f4c60009c50 comp rx=0 tx=0).stop 2024-05-27T09:31:14.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.925+0000 7f4c7bc00700 1 -- 172.21.15.119:0/3417927321 shutdown_connections 2024-05-27T09:31:14.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.925+0000 7f4c7bc00700 1 --2- 172.21.15.119:0/3417927321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c740737b0 0x7f4c74073c20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:14.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.925+0000 7f4c7bc00700 1 --2- 172.21.15.119:0/3417927321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c74074d80 0x7f4c740731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:14.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.925+0000 7f4c7bc00700 1 -- 172.21.15.119:0/3417927321 >> 172.21.15.119:0/3417927321 conn(0x7f4c740fb6e0 msgr2=0x7f4c740fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:14.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.925+0000 7f4c7bc00700 1 -- 172.21.15.119:0/3417927321 shutdown_connections 2024-05-27T09:31:14.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.925+0000 7f4c7bc00700 1 -- 172.21.15.119:0/3417927321 wait complete. 2024-05-27T09:31:14.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.926+0000 7f4c7bc00700 1 Processor -- start 2024-05-27T09:31:14.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.927+0000 7f4c7bc00700 1 -- start start 2024-05-27T09:31:14.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.927+0000 7f4c7bc00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c740737b0 0x7f4c74197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:14.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.927+0000 7f4c7bc00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c74074d80 0x7f4c741980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:14.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.927+0000 7f4c7bc00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c741986c0 con 0x7f4c74074d80 2024-05-27T09:31:14.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.927+0000 7f4c7bc00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c74198800 con 0x7f4c740737b0 2024-05-27T09:31:14.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.927+0000 7f4c73e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c74074d80 0x7f4c741980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:14.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.927+0000 7f4c73e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c74074d80 0x7f4c741980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47278/0 (socket says 172.21.15.119:47278) 2024-05-27T09:31:14.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.927+0000 7f4c79200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c740737b0 0x7f4c74197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:14.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.927+0000 7f4c73e00700 1 -- 172.21.15.119:0/3728331013 learned_addr learned my addr 172.21.15.119:0/3728331013 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:31:14.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.928+0000 7f4c73e00700 1 -- 172.21.15.119:0/3728331013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c740737b0 msgr2=0x7f4c74197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:14.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.928+0000 7f4c73e00700 1 --2- 172.21.15.119:0/3728331013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c740737b0 0x7f4c74197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:14.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.928+0000 7f4c73e00700 1 -- 172.21.15.119:0/3728331013 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c600095f0 con 0x7f4c74074d80 2024-05-27T09:31:14.929 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.928+0000 7f4c73e00700 1 --2- 172.21.15.119:0/3728331013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c74074d80 0x7f4c741980a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4c600055f0 tx=0x7f4c60005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:14.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.929+0000 7f4c71600700 1 -- 172.21.15.119:0/3728331013 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c6001d070 con 0x7f4c74074d80 2024-05-27T09:31:14.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.929+0000 7f4c71600700 1 -- 172.21.15.119:0/3728331013 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4c6000f460 con 0x7f4c74074d80 2024-05-27T09:31:14.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.929+0000 7f4c7bc00700 1 -- 172.21.15.119:0/3728331013 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c7419d250 con 0x7f4c74074d80 2024-05-27T09:31:14.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.929+0000 7f4c71600700 1 -- 172.21.15.119:0/3728331013 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c60021640 con 0x7f4c74074d80 2024-05-27T09:31:14.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.929+0000 7f4c7bc00700 1 -- 172.21.15.119:0/3728331013 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7419d7c0 con 0x7f4c74074d80 2024-05-27T09:31:14.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.931+0000 7f4c7bc00700 1 -- 172.21.15.119:0/3728331013 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c7410ae90 con 0x7f4c74074d80 2024-05-27T09:31:14.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.933+0000 7f4c71600700 1 -- 172.21.15.119:0/3728331013 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 31) v1 ==== 100037+0+0 (secure 0 0 0) 0x7f4c6000fab0 con 0x7f4c74074d80 2024-05-27T09:31:14.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.934+0000 7f4c71600700 1 --2- 172.21.15.119:0/3728331013 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f4c5c0787e0 0x7f4c5c07ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:14.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.934+0000 7f4c71600700 1 -- 172.21.15.119:0/3728331013 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4c6009bc80 con 0x7f4c74074d80 2024-05-27T09:31:14.935 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.935+0000 7f4c79200700 1 --2- 172.21.15.119:0/3728331013 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f4c5c0787e0 0x7f4c5c07ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:14.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.936+0000 7f4c79200700 1 --2- 172.21.15.119:0/3728331013 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f4c5c0787e0 0x7f4c5c07ac90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4c68005950 tx=0x7f4c6800b410 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:14.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:14.942+0000 7f4c71600700 1 -- 172.21.15.119:0/3728331013 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f4c600a1020 con 0x7f4c74074d80 2024-05-27T09:31:15.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.353+0000 7f4c7bc00700 1 -- 172.21.15.119:0/3728331013 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c74107650 con 0x7f4c5c0787e0 2024-05-27T09:31:15.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.355+0000 7f4c71600700 1 -- 172.21.15.119:0/3728331013 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f4c74107650 con 0x7f4c5c0787e0 2024-05-27T09:31:15.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.361+0000 7f4c7bc00700 1 -- 172.21.15.119:0/3728331013 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f4c5c0787e0 msgr2=0x7f4c5c07ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:15.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.361+0000 7f4c7bc00700 1 --2- 172.21.15.119:0/3728331013 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f4c5c0787e0 0x7f4c5c07ac90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4c68005950 tx=0x7f4c6800b410 comp rx=0 tx=0).stop 2024-05-27T09:31:15.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.361+0000 7f4c7bc00700 1 -- 172.21.15.119:0/3728331013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c74074d80 msgr2=0x7f4c741980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:15.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.361+0000 7f4c7bc00700 1 --2- 172.21.15.119:0/3728331013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c74074d80 0x7f4c741980a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4c600055f0 tx=0x7f4c60005620 comp rx=0 tx=0).stop 2024-05-27T09:31:15.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.362+0000 7f4c7bc00700 1 -- 172.21.15.119:0/3728331013 shutdown_connections 2024-05-27T09:31:15.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.362+0000 7f4c7bc00700 1 --2- 172.21.15.119:0/3728331013 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f4c5c0787e0 0x7f4c5c07ac90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:15.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.362+0000 7f4c7bc00700 1 --2- 172.21.15.119:0/3728331013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c740737b0 0x7f4c74197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:15.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.362+0000 7f4c7bc00700 1 --2- 172.21.15.119:0/3728331013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c74074d80 0x7f4c741980a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:15.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.362+0000 7f4c7bc00700 1 -- 172.21.15.119:0/3728331013 >> 172.21.15.119:0/3728331013 conn(0x7f4c740fb6e0 msgr2=0x7f4c74105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:15.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.362+0000 7f4c7bc00700 1 -- 172.21.15.119:0/3728331013 shutdown_connections 2024-05-27T09:31:15.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.362+0000 7f4c7bc00700 1 -- 172.21.15.119:0/3728331013 wait complete. 2024-05-27T09:31:15.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:15 smithi138 ceph-mon[34885]: pgmap v26: 65 pgs: 65 active+clean; 344 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.4 MiB/s wr, 248 op/s 2024-05-27T09:31:15.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:15 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:15.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:15 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:15.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:15 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:31:15.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:15 smithi119 ceph-mon[29082]: pgmap v26: 65 pgs: 65 active+clean; 344 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.4 MiB/s wr, 248 op/s 2024-05-27T09:31:15.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:15 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:15.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:15 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:15.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:15 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:31:15.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.553+0000 7f8aee200700 1 -- 172.21.15.119:0/1174001856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ae8101350 msgr2=0x7f8ae81017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:15.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.553+0000 7f8aee200700 1 --2- 172.21.15.119:0/1174001856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ae8101350 0x7f8ae81017a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009910 tx=0x7f8ad8009c20 comp rx=0 tx=0).stop 2024-05-27T09:31:15.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.554+0000 7f8aee200700 1 -- 172.21.15.119:0/1174001856 shutdown_connections 2024-05-27T09:31:15.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.554+0000 7f8aee200700 1 --2- 172.21.15.119:0/1174001856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ae8101350 0x7f8ae81017a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:15.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.554+0000 7f8aee200700 1 --2- 172.21.15.119:0/1174001856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ae8100150 0x7f8ae8100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:15.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.554+0000 7f8aee200700 1 -- 172.21.15.119:0/1174001856 >> 172.21.15.119:0/1174001856 conn(0x7f8ae80fb6e0 msgr2=0x7f8ae80fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:15.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.554+0000 7f8aee200700 1 -- 172.21.15.119:0/1174001856 shutdown_connections 2024-05-27T09:31:15.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.554+0000 7f8aee200700 1 -- 172.21.15.119:0/1174001856 wait complete. 2024-05-27T09:31:15.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.555+0000 7f8aee200700 1 Processor -- start 2024-05-27T09:31:15.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.555+0000 7f8aee200700 1 -- start start 2024-05-27T09:31:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.555+0000 7f8aee200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ae8100150 0x7f8ae8197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.555+0000 7f8aee200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ae8101350 0x7f8ae81980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.555+0000 7f8aee200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae81986f0 con 0x7f8ae8100150 2024-05-27T09:31:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.555+0000 7f8aee200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8198830 con 0x7f8ae8101350 2024-05-27T09:31:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.556+0000 7f8ae7400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ae8100150 0x7f8ae8197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.556+0000 7f8ae7400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ae8100150 0x7f8ae8197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47290/0 (socket says 172.21.15.119:47290) 2024-05-27T09:31:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.556+0000 7f8ae7400700 1 -- 172.21.15.119:0/797166872 learned_addr learned my addr 172.21.15.119:0/797166872 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:31:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.556+0000 7f8ae6a00700 1 --2- 172.21.15.119:0/797166872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ae8101350 0x7f8ae81980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:15.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.556+0000 7f8ae7400700 1 -- 172.21.15.119:0/797166872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ae8101350 msgr2=0x7f8ae81980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:15.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.556+0000 7f8ae7400700 1 --2- 172.21.15.119:0/797166872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ae8101350 0x7f8ae81980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:15.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.556+0000 7f8ae7400700 1 -- 172.21.15.119:0/797166872 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad80095f0 con 0x7f8ae8100150 2024-05-27T09:31:15.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.556+0000 7f8ae6a00700 1 --2- 172.21.15.119:0/797166872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ae8101350 0x7f8ae81980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:31:15.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.557+0000 7f8ae7400700 1 --2- 172.21.15.119:0/797166872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ae8100150 0x7f8ae8197b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000e8d0 tx=0x7f8ad000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:15.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.557+0000 7f8adfe00700 1 -- 172.21.15.119:0/797166872 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ad0009770 con 0x7f8ae8100150 2024-05-27T09:31:15.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.558+0000 7f8adfe00700 1 -- 172.21.15.119:0/797166872 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8ad000eed0 con 0x7f8ae8100150 2024-05-27T09:31:15.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.558+0000 7f8aee200700 1 -- 172.21.15.119:0/797166872 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae819d2e0 con 0x7f8ae8100150 2024-05-27T09:31:15.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.558+0000 7f8aee200700 1 -- 172.21.15.119:0/797166872 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae819d830 con 0x7f8ae8100150 2024-05-27T09:31:15.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.558+0000 7f8adfe00700 1 -- 172.21.15.119:0/797166872 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ad0003e40 con 0x7f8ae8100150 2024-05-27T09:31:15.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.559+0000 7f8aee200700 1 -- 172.21.15.119:0/797166872 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac4005320 con 0x7f8ae8100150 2024-05-27T09:31:15.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.560+0000 7f8adfe00700 1 -- 172.21.15.119:0/797166872 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 31) v1 ==== 100037+0+0 (secure 0 0 0) 0x7f8ad000cba0 con 0x7f8ae8100150 2024-05-27T09:31:15.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.562+0000 7f8adfe00700 1 --2- 172.21.15.119:0/797166872 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f8acc078830 0x7f8acc07ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:15.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.562+0000 7f8adfe00700 1 -- 172.21.15.119:0/797166872 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8ad0014070 con 0x7f8ae8100150 2024-05-27T09:31:15.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.563+0000 7f8ae6a00700 1 --2- 172.21.15.119:0/797166872 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f8acc078830 0x7f8acc07ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:15.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.564+0000 7f8ae6a00700 1 --2- 172.21.15.119:0/797166872 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f8acc078830 0x7f8acc07ace0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8006010 tx=0x7f8ad8005a90 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:15.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.569+0000 7f8adfe00700 1 -- 172.21.15.119:0/797166872 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f8ad0062c20 con 0x7f8ae8100150 2024-05-27T09:31:15.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.889+0000 7f8aee200700 1 -- 172.21.15.119:0/797166872 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ac4000bf0 con 0x7f8acc078830 2024-05-27T09:31:15.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.906+0000 7f8adfe00700 1 -- 172.21.15.119:0/797166872 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8ac4000bf0 con 0x7f8acc078830 2024-05-27T09:31:15.907 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:31:15.907 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (10m) 38s ago 12m 13.9M - 0.25.0 c8568f914cd2 bc280230ab28 2024-05-27T09:31:15.907 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (12m) 38s ago 12m 7864k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:31:15.907 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (11m) 38s ago 11m 7742k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:31:15.907 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (12m) 38s ago 12m 6665k - 18.2.0 dc2bc1663786 e5fafde44183 2024-05-27T09:31:15.907 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (11m) 38s ago 11m 6660k - 18.2.0 dc2bc1663786 46956ab99bab 2024-05-27T09:31:15.907 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (10m) 38s ago 12m 75.3M - 9.4.7 954c08fa6188 16eeaded4295 2024-05-27T09:31:15.907 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (5m) 38s ago 5m 14.3M - 18.2.0 dc2bc1663786 93b21713fb1e 2024-05-27T09:31:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (5m) 38s ago 5m 205M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:31:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (5m) 38s ago 5m 14.3M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:31:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (5m) 38s ago 5m 163M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:31:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (63s) 38s ago 14m 561M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:31:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 starting - - - - 2024-05-27T09:31:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (14m) 38s ago 14m 54.5M 2048M 18.2.0 dc2bc1663786 ba998b789a1c 2024-05-27T09:31:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (10m) 38s ago 10m 44.8M 2048M 18.2.0 dc2bc1663786 60216fd67078 2024-05-27T09:31:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (12m) 38s ago 12m 8032k - 1.5.0 0da6a335fe13 79c36d4e5b04 2024-05-27T09:31:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (11m) 38s ago 11m 8019k - 1.5.0 0da6a335fe13 80ee890e8830 2024-05-27T09:31:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (10m) 38s ago 10m 201M 1216M 18.2.0 dc2bc1663786 825bd1392508 2024-05-27T09:31:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (9m) 38s ago 9m 222M 1216M 18.2.0 dc2bc1663786 fce9819903d9 2024-05-27T09:31:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (8m) 38s ago 8m 169M 1216M 18.2.0 dc2bc1663786 3955a6112c1c 2024-05-27T09:31:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (8m) 38s ago 8m 242M 2240M 18.2.0 dc2bc1663786 693c92220622 2024-05-27T09:31:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (7m) 38s ago 7m 215M 2240M 18.2.0 dc2bc1663786 7559e658e917 2024-05-27T09:31:15.908 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (7m) 38s ago 7m 213M 2240M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:31:15.909 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 starting - - - - 2024-05-27T09:31:15.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.910+0000 7f8aee200700 1 -- 172.21.15.119:0/797166872 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f8acc078830 msgr2=0x7f8acc07ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:15.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.910+0000 7f8aee200700 1 --2- 172.21.15.119:0/797166872 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f8acc078830 0x7f8acc07ace0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8006010 tx=0x7f8ad8005a90 comp rx=0 tx=0).stop 2024-05-27T09:31:15.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.910+0000 7f8aee200700 1 -- 172.21.15.119:0/797166872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ae8100150 msgr2=0x7f8ae8197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:15.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.910+0000 7f8aee200700 1 --2- 172.21.15.119:0/797166872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ae8100150 0x7f8ae8197b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000e8d0 tx=0x7f8ad000ec90 comp rx=0 tx=0).stop 2024-05-27T09:31:15.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.910+0000 7f8aee200700 1 -- 172.21.15.119:0/797166872 shutdown_connections 2024-05-27T09:31:15.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.910+0000 7f8aee200700 1 --2- 172.21.15.119:0/797166872 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f8acc078830 0x7f8acc07ace0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:15.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.910+0000 7f8aee200700 1 --2- 172.21.15.119:0/797166872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ae8100150 0x7f8ae8197b90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:15.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.910+0000 7f8aee200700 1 --2- 172.21.15.119:0/797166872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ae8101350 0x7f8ae81980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:15.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.910+0000 7f8aee200700 1 -- 172.21.15.119:0/797166872 >> 172.21.15.119:0/797166872 conn(0x7f8ae80fb6e0 msgr2=0x7f8ae8102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:15.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.911+0000 7f8aee200700 1 -- 172.21.15.119:0/797166872 shutdown_connections 2024-05-27T09:31:15.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:15.911+0000 7f8aee200700 1 -- 172.21.15.119:0/797166872 wait complete. 2024-05-27T09:31:16.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.099+0000 7f4919400700 1 -- 172.21.15.119:0/2888090297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4914075700 msgr2=0x7f4914075b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:16.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.099+0000 7f4919400700 1 --2- 172.21.15.119:0/2888090297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4914075700 0x7f4914075b10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4904009910 tx=0x7f4904009c20 comp rx=0 tx=0).stop 2024-05-27T09:31:16.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.099+0000 7f4919400700 1 -- 172.21.15.119:0/2888090297 shutdown_connections 2024-05-27T09:31:16.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.099+0000 7f4919400700 1 --2- 172.21.15.119:0/2888090297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4914076950 0x7f4914076dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:16.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.099+0000 7f4919400700 1 --2- 172.21.15.119:0/2888090297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4914075700 0x7f4914075b10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:16.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.099+0000 7f4919400700 1 -- 172.21.15.119:0/2888090297 >> 172.21.15.119:0/2888090297 conn(0x7f49140fd960 msgr2=0x7f49140ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:16.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.100+0000 7f4919400700 1 -- 172.21.15.119:0/2888090297 shutdown_connections 2024-05-27T09:31:16.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.100+0000 7f4919400700 1 -- 172.21.15.119:0/2888090297 wait complete. 2024-05-27T09:31:16.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.101+0000 7f4919400700 1 Processor -- start 2024-05-27T09:31:16.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.102+0000 7f4919400700 1 -- start start 2024-05-27T09:31:16.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.102+0000 7f4919400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4914075700 0x7f491410d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:16.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.102+0000 7f4919400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4914076950 0x7f491410d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:16.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.102+0000 7f4919400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491410df60 con 0x7f4914076950 2024-05-27T09:31:16.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.102+0000 7f4919400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49141a9d30 con 0x7f4914075700 2024-05-27T09:31:16.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.103+0000 7f490be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4914076950 0x7f491410d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:16.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.103+0000 7f490be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4914076950 0x7f491410d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47308/0 (socket says 172.21.15.119:47308) 2024-05-27T09:31:16.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.103+0000 7f490be00700 1 -- 172.21.15.119:0/148480684 learned_addr learned my addr 172.21.15.119:0/148480684 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:31:16.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.104+0000 7f4912800700 1 --2- 172.21.15.119:0/148480684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4914075700 0x7f491410d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:16.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.104+0000 7f4912800700 1 -- 172.21.15.119:0/148480684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4914076950 msgr2=0x7f491410d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:16.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.105+0000 7f4912800700 1 --2- 172.21.15.119:0/148480684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4914076950 0x7f491410d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:16.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.105+0000 7f4912800700 1 -- 172.21.15.119:0/148480684 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49040095f0 con 0x7f4914075700 2024-05-27T09:31:16.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.105+0000 7f490be00700 1 --2- 172.21.15.119:0/148480684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4914076950 0x7f491410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:31:16.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.106+0000 7f4912800700 1 --2- 172.21.15.119:0/148480684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4914075700 0x7f491410d400 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4904000c00 tx=0x7f490400bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.107+0000 7f490b400700 1 -- 172.21.15.119:0/148480684 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f490401d070 con 0x7f4914075700 2024-05-27T09:31:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.107+0000 7f490b400700 1 -- 172.21.15.119:0/148480684 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4904005dc0 con 0x7f4914075700 2024-05-27T09:31:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.107+0000 7f490b400700 1 -- 172.21.15.119:0/148480684 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4904021410 con 0x7f4914075700 2024-05-27T09:31:16.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.108+0000 7f4919400700 1 -- 172.21.15.119:0/148480684 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49141a9ed0 con 0x7f4914075700 2024-05-27T09:31:16.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.109+0000 7f4919400700 1 -- 172.21.15.119:0/148480684 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49141aa2e0 con 0x7f4914075700 2024-05-27T09:31:16.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.110+0000 7f4919400700 1 -- 172.21.15.119:0/148480684 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4914106cc0 con 0x7f4914075700 2024-05-27T09:31:16.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.112+0000 7f490b400700 1 -- 172.21.15.119:0/148480684 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) v1 ==== 100037+0+0 (secure 0 0 0) 0x7f490400f7d0 con 0x7f4914075700 2024-05-27T09:31:16.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.113+0000 7f490b400700 1 --2- 172.21.15.119:0/148480684 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f48f8078830 0x7f48f807ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:16.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.114+0000 7f490b400700 1 -- 172.21.15.119:0/148480684 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f490409bbf0 con 0x7f4914075700 2024-05-27T09:31:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.114+0000 7f490be00700 1 --2- 172.21.15.119:0/148480684 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f48f8078830 0x7f48f807ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:16.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.115+0000 7f490be00700 1 --2- 172.21.15.119:0/148480684 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f48f8078830 0x7f48f807ace0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0104e0 tx=0x7f48fc007480 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:16.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.122+0000 7f490b400700 1 -- 172.21.15.119:0/148480684 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f4904063b30 con 0x7f4914075700 2024-05-27T09:31:16.465 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:16 smithi138 ceph-mon[34885]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:31:16.465 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:16 smithi138 ceph-mon[34885]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:31:16.488 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:16 smithi119 ceph-mon[29082]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:31:16.489 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:16 smithi119 ceph-mon[29082]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:31:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.691+0000 7f4919400700 1 -- 172.21.15.119:0/148480684 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f491402ce40 con 0x7f4914075700 2024-05-27T09:31:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.692+0000 7f490b400700 1 -- 172.21.15.119:0/148480684 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f490400f460 con 0x7f4914075700 2024-05-27T09:31:16.693 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:31:16.693 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:31:16.693 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-05-27T09:31:16.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:31:16.693 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:31:16.693 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:31:16.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:31:16.693 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:31:16.693 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-05-27T09:31:16.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:31:16.693 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:31:16.693 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-05-27T09:31:16.693 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:31:16.694 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:31:16.694 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-05-27T09:31:16.694 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:31:16.694 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:31:16.694 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:31:16.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.695+0000 7f4919400700 1 -- 172.21.15.119:0/148480684 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f48f8078830 msgr2=0x7f48f807ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:16.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.695+0000 7f4919400700 1 --2- 172.21.15.119:0/148480684 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f48f8078830 0x7f48f807ace0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0104e0 tx=0x7f48fc007480 comp rx=0 tx=0).stop 2024-05-27T09:31:16.696 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.695+0000 7f4919400700 1 -- 172.21.15.119:0/148480684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4914075700 msgr2=0x7f491410d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:16.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.695+0000 7f4919400700 1 --2- 172.21.15.119:0/148480684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4914075700 0x7f491410d400 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4904000c00 tx=0x7f490400bd70 comp rx=0 tx=0).stop 2024-05-27T09:31:16.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.695+0000 7f4919400700 1 -- 172.21.15.119:0/148480684 shutdown_connections 2024-05-27T09:31:16.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.695+0000 7f4919400700 1 --2- 172.21.15.119:0/148480684 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f48f8078830 0x7f48f807ace0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:16.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.695+0000 7f4919400700 1 --2- 172.21.15.119:0/148480684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4914075700 0x7f491410d400 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:16.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.695+0000 7f4919400700 1 --2- 172.21.15.119:0/148480684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4914076950 0x7f491410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:16.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.696+0000 7f4919400700 1 -- 172.21.15.119:0/148480684 >> 172.21.15.119:0/148480684 conn(0x7f49140fd960 msgr2=0x7f4914103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:16.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.696+0000 7f4919400700 1 -- 172.21.15.119:0/148480684 shutdown_connections 2024-05-27T09:31:16.698 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.696+0000 7f4919400700 1 -- 172.21.15.119:0/148480684 wait complete. 2024-05-27T09:31:16.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.882+0000 7f06f6c00700 1 -- 172.21.15.119:0/2640834793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f0101350 msgr2=0x7f06f01017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:16.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.882+0000 7f06f6c00700 1 --2- 172.21.15.119:0/2640834793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f0101350 0x7f06f01017a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f06dc009910 tx=0x7f06dc009c20 comp rx=0 tx=0).stop 2024-05-27T09:31:16.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.882+0000 7f06f6c00700 1 -- 172.21.15.119:0/2640834793 shutdown_connections 2024-05-27T09:31:16.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.882+0000 7f06f6c00700 1 --2- 172.21.15.119:0/2640834793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f0101350 0x7f06f01017a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:16.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.882+0000 7f06f6c00700 1 --2- 172.21.15.119:0/2640834793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06f0100150 0x7f06f0100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:16.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.882+0000 7f06f6c00700 1 -- 172.21.15.119:0/2640834793 >> 172.21.15.119:0/2640834793 conn(0x7f06f00fb6e0 msgr2=0x7f06f00fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:16.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.882+0000 7f06f6c00700 1 -- 172.21.15.119:0/2640834793 shutdown_connections 2024-05-27T09:31:16.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.882+0000 7f06f6c00700 1 -- 172.21.15.119:0/2640834793 wait complete. 2024-05-27T09:31:16.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.883+0000 7f06f6c00700 1 Processor -- start 2024-05-27T09:31:16.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.883+0000 7f06f6c00700 1 -- start start 2024-05-27T09:31:16.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.884+0000 7f06f6c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06f0100150 0x7f06f0074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:16.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.884+0000 7f06f6c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f0101350 0x7f06f0073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:16.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.884+0000 7f06f6c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f0073680 con 0x7f06f0101350 2024-05-27T09:31:16.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.884+0000 7f06f6c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f00737c0 con 0x7f06f0100150 2024-05-27T09:31:16.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.884+0000 7f06ef400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f0101350 0x7f06f0073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:16.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.884+0000 7f06ef400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f0101350 0x7f06f0073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47318/0 (socket says 172.21.15.119:47318) 2024-05-27T09:31:16.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.884+0000 7f06ef400700 1 -- 172.21.15.119:0/1205500715 learned_addr learned my addr 172.21.15.119:0/1205500715 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:31:16.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.884+0000 7f06efe00700 1 --2- 172.21.15.119:0/1205500715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06f0100150 0x7f06f0074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:16.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.884+0000 7f06ef400700 1 -- 172.21.15.119:0/1205500715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06f0100150 msgr2=0x7f06f0074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:16.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.884+0000 7f06ef400700 1 --2- 172.21.15.119:0/1205500715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06f0100150 0x7f06f0074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:16.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.884+0000 7f06ef400700 1 -- 172.21.15.119:0/1205500715 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06dc0095f0 con 0x7f06f0101350 2024-05-27T09:31:16.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.885+0000 7f06ef400700 1 --2- 172.21.15.119:0/1205500715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f0101350 0x7f06f0073140 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f06dc005b40 tx=0x7f06dc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:16.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.885+0000 7f06ecc00700 1 -- 172.21.15.119:0/1205500715 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06dc01d070 con 0x7f06f0101350 2024-05-27T09:31:16.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.885+0000 7f06ecc00700 1 -- 172.21.15.119:0/1205500715 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f06dc022470 con 0x7f06f0101350 2024-05-27T09:31:16.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.885+0000 7f06ecc00700 1 -- 172.21.15.119:0/1205500715 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06dc00f690 con 0x7f06f0101350 2024-05-27T09:31:16.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.885+0000 7f06f6c00700 1 -- 172.21.15.119:0/1205500715 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f0073a40 con 0x7f06f0101350 2024-05-27T09:31:16.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.885+0000 7f06f6c00700 1 -- 172.21.15.119:0/1205500715 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f0073f30 con 0x7f06f0101350 2024-05-27T09:31:16.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.887+0000 7f06f6c00700 1 -- 172.21.15.119:0/1205500715 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06f0108ca0 con 0x7f06f0101350 2024-05-27T09:31:16.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.889+0000 7f06ecc00700 1 -- 172.21.15.119:0/1205500715 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 31) v1 ==== 100037+0+0 (secure 0 0 0) 0x7f06dc0225e0 con 0x7f06f0101350 2024-05-27T09:31:16.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.890+0000 7f06ecc00700 1 --2- 172.21.15.119:0/1205500715 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f06d40787e0 0x7f06d407ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:16.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.890+0000 7f06ecc00700 1 -- 172.21.15.119:0/1205500715 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f06dc09c230 con 0x7f06f0101350 2024-05-27T09:31:16.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.890+0000 7f06efe00700 1 --2- 172.21.15.119:0/1205500715 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f06d40787e0 0x7f06d407ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:16.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.891+0000 7f06efe00700 1 --2- 172.21.15.119:0/1205500715 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f06d40787e0 0x7f06d407ac90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f06e0005fd0 tx=0x7f06e0005dd0 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:16.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:16.897+0000 7f06ecc00700 1 -- 172.21.15.119:0/1205500715 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f06dc0641b0 con 0x7f06f0101350 2024-05-27T09:31:17.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:17.253+0000 7f06f6c00700 1 -- 172.21.15.119:0/1205500715 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f06f0105ca0 con 0x7f06d40787e0 2024-05-27T09:31:17.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:17.255+0000 7f06ecc00700 1 -- 172.21.15.119:0/1205500715 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f06f0105ca0 con 0x7f06d40787e0 2024-05-27T09:31:17.255 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:31:17.256 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:31:17.256 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:31:17.256 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-27T09:31:17.256 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [], 2024-05-27T09:31:17.256 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "", 2024-05-27T09:31:17.256 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "Currently upgrading mgr daemons", 2024-05-27T09:31:17.256 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:31:17.256 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:31:17.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:17.257+0000 7f06f6c00700 1 -- 172.21.15.119:0/1205500715 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f06d40787e0 msgr2=0x7f06d407ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:17.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:17.257+0000 7f06f6c00700 1 --2- 172.21.15.119:0/1205500715 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f06d40787e0 0x7f06d407ac90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f06e0005fd0 tx=0x7f06e0005dd0 comp rx=0 tx=0).stop 2024-05-27T09:31:17.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:17.257+0000 7f06f6c00700 1 -- 172.21.15.119:0/1205500715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f0101350 msgr2=0x7f06f0073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:17.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:17.257+0000 7f06f6c00700 1 --2- 172.21.15.119:0/1205500715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f0101350 0x7f06f0073140 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f06dc005b40 tx=0x7f06dc00be20 comp rx=0 tx=0).stop 2024-05-27T09:31:17.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:17.257+0000 7f06f6c00700 1 -- 172.21.15.119:0/1205500715 shutdown_connections 2024-05-27T09:31:17.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:17.257+0000 7f06f6c00700 1 --2- 172.21.15.119:0/1205500715 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f06d40787e0 0x7f06d407ac90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:17.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:17.257+0000 7f06f6c00700 1 --2- 172.21.15.119:0/1205500715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06f0100150 0x7f06f0074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:17.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:17.257+0000 7f06f6c00700 1 --2- 172.21.15.119:0/1205500715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f0101350 0x7f06f0073140 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:17.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:17.257+0000 7f06f6c00700 1 -- 172.21.15.119:0/1205500715 >> 172.21.15.119:0/1205500715 conn(0x7f06f00fb6e0 msgr2=0x7f06f0104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:17.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:17.258+0000 7f06f6c00700 1 -- 172.21.15.119:0/1205500715 shutdown_connections 2024-05-27T09:31:17.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:17.258+0000 7f06f6c00700 1 -- 172.21.15.119:0/1205500715 wait complete. 2024-05-27T09:31:17.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:17 smithi119 ceph-mon[29082]: pgmap v27: 65 pgs: 65 active+clean; 341 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.7 MiB/s wr, 350 op/s 2024-05-27T09:31:17.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:17 smithi119 ceph-mon[29082]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:31:17.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:17 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/148480684' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:17.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:17 smithi138 ceph-mon[34885]: pgmap v27: 65 pgs: 65 active+clean; 341 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.7 MiB/s wr, 350 op/s 2024-05-27T09:31:17.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:17 smithi138 ceph-mon[34885]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:31:17.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:17 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/148480684' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:18.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:18 smithi119 ceph-mon[29082]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:31:18.592 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:18 smithi138 ceph-mon[34885]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:31:19.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:19 smithi119 ceph-mon[29082]: pgmap v28: 65 pgs: 65 active+clean; 341 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 262 op/s 2024-05-27T09:31:19.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:19 smithi138 ceph-mon[34885]: pgmap v28: 65 pgs: 65 active+clean; 341 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 262 op/s 2024-05-27T09:31:21.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:21 smithi138 ceph-mon[34885]: pgmap v29: 65 pgs: 65 active+clean; 338 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 2.0 MiB/s wr, 440 op/s 2024-05-27T09:31:21.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:21 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:21.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:21 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:21.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:21 smithi119 ceph-mon[29082]: pgmap v29: 65 pgs: 65 active+clean; 338 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 2.0 MiB/s wr, 440 op/s 2024-05-27T09:31:21.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:21 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:21.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:21 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:23.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:23 smithi119 ceph-mon[29082]: pgmap v30: 65 pgs: 65 active+clean; 338 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.4 MiB/s wr, 315 op/s 2024-05-27T09:31:23.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:23 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:23.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:23 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:23.286 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:23 smithi138 ceph-mon[34885]: pgmap v30: 65 pgs: 65 active+clean; 338 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.4 MiB/s wr, 315 op/s 2024-05-27T09:31:23.286 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:23 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:23.287 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:23 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:25.044 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:24 smithi138 ceph-mon[34885]: pgmap v31: 65 pgs: 65 active+clean; 337 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 39 KiB/s rd, 1.4 MiB/s wr, 343 op/s 2024-05-27T09:31:25.044 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:24 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:25.044 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:24 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:25.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:24 smithi119 ceph-mon[29082]: pgmap v31: 65 pgs: 65 active+clean; 337 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 39 KiB/s rd, 1.4 MiB/s wr, 343 op/s 2024-05-27T09:31:25.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:24 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:25.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:24 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:25.436 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-05-27T09:31:25.437 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-27T09:31:26.745 DEBUG:teuthology.parallel:result is None 2024-05-27T09:31:27.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:27 smithi138 ceph-mon[34885]: pgmap v32: 65 pgs: 65 active+clean; 338 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 1.9 MiB/s wr, 460 op/s 2024-05-27T09:31:27.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:27 smithi119 ceph-mon[29082]: pgmap v32: 65 pgs: 65 active+clean; 338 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 1.9 MiB/s wr, 460 op/s 2024-05-27T09:31:28.325 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-05-27T09:31:28.326 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-05-27T09:31:28.458 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: Standby manager daemon smithi138.dxsuqf restarted 2024-05-27T09:31:28.459 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: Standby manager daemon smithi138.dxsuqf started 2024-05-27T09:31:28.459 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.138:0/2294394339' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.dxsuqf/crt"}]: dispatch 2024-05-27T09:31:28.459 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.138:0/2294394339' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T09:31:28.459 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.138:0/2294394339' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.dxsuqf/key"}]: dispatch 2024-05-27T09:31:28.459 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.138:0/2294394339' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T09:31:28.459 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:28.459 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:28.459 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:31:28.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:31:28.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:28.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:31:28.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:28.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:28.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi119.ncwmog"}]: dispatch 2024-05-27T09:31:28.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi119.ncwmog"}]': finished 2024-05-27T09:31:28.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi138.dxsuqf"}]: dispatch 2024-05-27T09:31:28.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi138.dxsuqf"}]': finished 2024-05-27T09:31:28.460 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:31:28.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:28.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:28.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:28.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:28.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:28.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:28.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:28.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:28.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:28.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:28.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:28.461 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:28.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: Standby manager daemon smithi138.dxsuqf restarted 2024-05-27T09:31:28.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: Standby manager daemon smithi138.dxsuqf started 2024-05-27T09:31:28.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.138:0/2294394339' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.dxsuqf/crt"}]: dispatch 2024-05-27T09:31:28.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.138:0/2294394339' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T09:31:28.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.138:0/2294394339' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.dxsuqf/key"}]: dispatch 2024-05-27T09:31:28.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.? 172.21.15.138:0/2294394339' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T09:31:28.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:28.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:28.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:31:28.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:31:28.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:28.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:31:28.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:28.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:28.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi119.ncwmog"}]: dispatch 2024-05-27T09:31:28.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi119.ncwmog"}]': finished 2024-05-27T09:31:28.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi138.dxsuqf"}]: dispatch 2024-05-27T09:31:28.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi138.dxsuqf"}]': finished 2024-05-27T09:31:28.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:31:28.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:28.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:28.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:28.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:28.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:28.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:28.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:28.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:28.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:28.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:28.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:28.500 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:29.337 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:29 smithi119 ceph-mon[29082]: pgmap v33: 65 pgs: 65 active+clean; 338 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.6 MiB/s wr, 358 op/s 2024-05-27T09:31:29.337 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:29 smithi119 ceph-mon[29082]: Upgrade: Setting container_image for all mgr 2024-05-27T09:31:29.337 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:29 smithi119 ceph-mon[29082]: Upgrade: Setting container_image for all rgw 2024-05-27T09:31:29.337 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:29 smithi119 ceph-mon[29082]: Upgrade: Setting container_image for all rbd-mirror 2024-05-27T09:31:29.337 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:29 smithi119 ceph-mon[29082]: Upgrade: Setting container_image for all cephfs-mirror 2024-05-27T09:31:29.337 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:29 smithi119 ceph-mon[29082]: Upgrade: Setting container_image for all iscsi 2024-05-27T09:31:29.337 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:29 smithi119 ceph-mon[29082]: Upgrade: Setting container_image for all nfs 2024-05-27T09:31:29.337 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:29 smithi119 ceph-mon[29082]: Upgrade: Setting container_image for all nvmeof 2024-05-27T09:31:29.338 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:29 smithi119 ceph-mon[29082]: mgrmap e32: smithi119.ncwmog(active, since 60s), standbys: smithi138.dxsuqf 2024-05-27T09:31:29.502 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:29 smithi138 ceph-mon[34885]: pgmap v33: 65 pgs: 65 active+clean; 338 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.6 MiB/s wr, 358 op/s 2024-05-27T09:31:29.502 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:29 smithi138 ceph-mon[34885]: Upgrade: Setting container_image for all mgr 2024-05-27T09:31:29.502 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:29 smithi138 ceph-mon[34885]: Upgrade: Setting container_image for all rgw 2024-05-27T09:31:29.502 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:29 smithi138 ceph-mon[34885]: Upgrade: Setting container_image for all rbd-mirror 2024-05-27T09:31:29.502 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:29 smithi138 ceph-mon[34885]: Upgrade: Setting container_image for all cephfs-mirror 2024-05-27T09:31:29.502 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:29 smithi138 ceph-mon[34885]: Upgrade: Setting container_image for all iscsi 2024-05-27T09:31:29.503 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:29 smithi138 ceph-mon[34885]: Upgrade: Setting container_image for all nfs 2024-05-27T09:31:29.503 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:29 smithi138 ceph-mon[34885]: Upgrade: Setting container_image for all nvmeof 2024-05-27T09:31:29.503 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:29 smithi138 ceph-mon[34885]: mgrmap e32: smithi119.ncwmog(active, since 60s), standbys: smithi138.dxsuqf 2024-05-27T09:31:29.505 DEBUG:teuthology.parallel:result is None 2024-05-27T09:31:29.505 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-27T09:31:29.663 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-27T09:31:29.663 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-05-27T09:31:29.755 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-05-27T09:31:29.755 DEBUG:teuthology.parallel:result is None 2024-05-27T09:31:31.487 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:31 smithi119 ceph-mon[29082]: pgmap v34: 65 pgs: 65 active+clean; 331 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 49 KiB/s rd, 2.1 MiB/s wr, 498 op/s 2024-05-27T09:31:31.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:31 smithi138 ceph-mon[34885]: pgmap v34: 65 pgs: 65 active+clean; 331 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 49 KiB/s rd, 2.1 MiB/s wr, 498 op/s 2024-05-27T09:31:32.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:32 smithi119 ceph-mon[29082]: Upgrade: Updating node-exporter.smithi119 (1/2) 2024-05-27T09:31:32.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:32 smithi119 ceph-mon[29082]: Deploying daemon node-exporter.smithi119 on smithi119 2024-05-27T09:31:32.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:32 smithi138 ceph-mon[34885]: Upgrade: Updating node-exporter.smithi119 (1/2) 2024-05-27T09:31:32.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:32 smithi138 ceph-mon[34885]: Deploying daemon node-exporter.smithi119 on smithi119 2024-05-27T09:31:33.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:33 smithi119 ceph-mon[29082]: pgmap v35: 65 pgs: 65 active+clean; 331 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.3 MiB/s wr, 321 op/s 2024-05-27T09:31:33.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:33 smithi138 ceph-mon[34885]: pgmap v35: 65 pgs: 65 active+clean; 331 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.3 MiB/s wr, 321 op/s 2024-05-27T09:31:35.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:35 smithi119 ceph-mon[29082]: pgmap v36: 65 pgs: 65 active+clean; 331 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.3 MiB/s wr, 321 op/s 2024-05-27T09:31:35.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:35 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:35.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:35 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:35.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:35 smithi138 ceph-mon[34885]: pgmap v36: 65 pgs: 65 active+clean; 331 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.3 MiB/s wr, 321 op/s 2024-05-27T09:31:35.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:35 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:35.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:35 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:37.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:37 smithi138 ceph-mon[34885]: pgmap v37: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 1.3 MiB/s wr, 295 op/s 2024-05-27T09:31:37.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:37 smithi119 ceph-mon[29082]: pgmap v37: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 1.3 MiB/s wr, 295 op/s 2024-05-27T09:31:38.380 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:38 smithi138 ceph-mon[34885]: Upgrade: Updating node-exporter.smithi138 (2/2) 2024-05-27T09:31:38.381 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:38 smithi138 ceph-mon[34885]: Deploying daemon node-exporter.smithi138 on smithi138 2024-05-27T09:31:38.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:38 smithi119 ceph-mon[29082]: Upgrade: Updating node-exporter.smithi138 (2/2) 2024-05-27T09:31:38.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:38 smithi119 ceph-mon[29082]: Deploying daemon node-exporter.smithi138 on smithi138 2024-05-27T09:31:39.300 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:39 smithi138 ceph-mon[34885]: pgmap v38: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 542 KiB/s wr, 142 op/s 2024-05-27T09:31:39.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:39 smithi119 ceph-mon[29082]: pgmap v38: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 542 KiB/s wr, 142 op/s 2024-05-27T09:31:41.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:41 smithi138 ceph-mon[34885]: pgmap v39: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 543 KiB/s wr, 143 op/s 2024-05-27T09:31:41.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:41 smithi119 ceph-mon[29082]: pgmap v39: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 14 KiB/s rd, 543 KiB/s wr, 143 op/s 2024-05-27T09:31:42.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:42 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:42.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:42 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:42.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:42 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:31:42.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:42 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:42.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:42 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:42.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:42 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:31:43.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:43 smithi138 ceph-mon[34885]: pgmap v40: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 341 B/s rd, 21 KiB/s wr, 2 op/s 2024-05-27T09:31:43.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:43 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:31:43.650 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:43 smithi119 ceph-mon[29082]: pgmap v40: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 341 B/s rd, 21 KiB/s wr, 2 op/s 2024-05-27T09:31:43.650 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:43 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:31:45.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:45 smithi119 ceph-mon[29082]: pgmap v41: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 341 B/s rd, 21 KiB/s wr, 2 op/s 2024-05-27T09:31:45.578 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:45 smithi138 ceph-mon[34885]: pgmap v41: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 341 B/s rd, 21 KiB/s wr, 2 op/s 2024-05-27T09:31:47.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.451+0000 7f0c66000700 1 -- 172.21.15.119:0/3850567729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c601079a0 msgr2=0x7f0c60107df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:47.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.451+0000 7f0c66000700 1 --2- 172.21.15.119:0/3850567729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c601079a0 0x7f0c60107df0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009910 tx=0x7f0c4c009c20 comp rx=0 tx=0).stop 2024-05-27T09:31:47.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.452+0000 7f0c66000700 1 -- 172.21.15.119:0/3850567729 shutdown_connections 2024-05-27T09:31:47.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.452+0000 7f0c66000700 1 --2- 172.21.15.119:0/3850567729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c601079a0 0x7f0c60107df0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:47.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.452+0000 7f0c66000700 1 --2- 172.21.15.119:0/3850567729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c601067a0 0x7f0c60106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:47.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.452+0000 7f0c66000700 1 -- 172.21.15.119:0/3850567729 >> 172.21.15.119:0/3850567729 conn(0x7f0c60075910 msgr2=0x7f0c60077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:47.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.452+0000 7f0c66000700 1 -- 172.21.15.119:0/3850567729 shutdown_connections 2024-05-27T09:31:47.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.452+0000 7f0c66000700 1 -- 172.21.15.119:0/3850567729 wait complete. 2024-05-27T09:31:47.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.453+0000 7f0c66000700 1 Processor -- start 2024-05-27T09:31:47.458 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.453+0000 7f0c66000700 1 -- start start 2024-05-27T09:31:47.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.453+0000 7f0c66000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c601067a0 0x7f0c6019bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:47.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.453+0000 7f0c66000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c601079a0 0x7f0c6019c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:47.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.454+0000 7f0c66000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6019cb00 con 0x7f0c601067a0 2024-05-27T09:31:47.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.454+0000 7f0c66000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6019cc40 con 0x7f0c601079a0 2024-05-27T09:31:47.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.454+0000 7f0c5f400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c601067a0 0x7f0c6019bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:47.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.454+0000 7f0c5f400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c601067a0 0x7f0c6019bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48450/0 (socket says 172.21.15.119:48450) 2024-05-27T09:31:47.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.454+0000 7f0c5f400700 1 -- 172.21.15.119:0/2658641512 learned_addr learned my addr 172.21.15.119:0/2658641512 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:31:47.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.454+0000 7f0c5ea00700 1 --2- 172.21.15.119:0/2658641512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c601079a0 0x7f0c6019c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:47.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.454+0000 7f0c5f400700 1 -- 172.21.15.119:0/2658641512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c601079a0 msgr2=0x7f0c6019c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:47.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.454+0000 7f0c5f400700 1 --2- 172.21.15.119:0/2658641512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c601079a0 0x7f0c6019c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:47.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.454+0000 7f0c5f400700 1 -- 172.21.15.119:0/2658641512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4c0095f0 con 0x7f0c601067a0 2024-05-27T09:31:47.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.455+0000 7f0c5f400700 1 --2- 172.21.15.119:0/2658641512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c601067a0 0x7f0c6019bfa0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000d900 tx=0x7f0c5000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:47.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.455+0000 7f0c57e00700 1 -- 172.21.15.119:0/2658641512 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c500041f0 con 0x7f0c601067a0 2024-05-27T09:31:47.459 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.455+0000 7f0c57e00700 1 -- 172.21.15.119:0/2658641512 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0c50004350 con 0x7f0c601067a0 2024-05-27T09:31:47.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.455+0000 7f0c57e00700 1 -- 172.21.15.119:0/2658641512 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c50003da0 con 0x7f0c601067a0 2024-05-27T09:31:47.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.455+0000 7f0c66000700 1 -- 172.21.15.119:0/2658641512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c601a16f0 con 0x7f0c601067a0 2024-05-27T09:31:47.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.455+0000 7f0c66000700 1 -- 172.21.15.119:0/2658641512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c601a1bf0 con 0x7f0c601067a0 2024-05-27T09:31:47.460 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.456+0000 7f0c66000700 1 -- 172.21.15.119:0/2658641512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c6010f2f0 con 0x7f0c601067a0 2024-05-27T09:31:47.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.457+0000 7f0c57e00700 1 -- 172.21.15.119:0/2658641512 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 100037+0+0 (secure 0 0 0) 0x7f0c50009730 con 0x7f0c601067a0 2024-05-27T09:31:47.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.458+0000 7f0c57e00700 1 --2- 172.21.15.119:0/2658641512 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f0c440787f0 0x7f0c4407aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:47.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.459+0000 7f0c57e00700 1 -- 172.21.15.119:0/2658641512 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0c5009ac30 con 0x7f0c601067a0 2024-05-27T09:31:47.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.459+0000 7f0c5ea00700 1 --2- 172.21.15.119:0/2658641512 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f0c440787f0 0x7f0c4407aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:47.461 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.460+0000 7f0c5ea00700 1 --2- 172.21.15.119:0/2658641512 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f0c440787f0 0x7f0c4407aca0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c005b40 tx=0x7f0c4c005a90 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:47.464 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.464+0000 7f0c57e00700 1 -- 172.21.15.119:0/2658641512 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f0c50062a70 con 0x7f0c601067a0 2024-05-27T09:31:47.547 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:47 smithi119 ceph-mon[29082]: pgmap v42: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 21 KiB/s wr, 2 op/s 2024-05-27T09:31:47.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:47 smithi138 ceph-mon[34885]: pgmap v42: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 21 KiB/s wr, 2 op/s 2024-05-27T09:31:47.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.725+0000 7f0c66000700 1 -- 172.21.15.119:0/2658641512 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c6010c2f0 con 0x7f0c440787f0 2024-05-27T09:31:47.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.727+0000 7f0c57e00700 1 -- 172.21.15.119:0/2658641512 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f0c6010c2f0 con 0x7f0c440787f0 2024-05-27T09:31:47.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.730+0000 7f0c66000700 1 -- 172.21.15.119:0/2658641512 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f0c440787f0 msgr2=0x7f0c4407aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:47.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.730+0000 7f0c66000700 1 --2- 172.21.15.119:0/2658641512 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f0c440787f0 0x7f0c4407aca0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c005b40 tx=0x7f0c4c005a90 comp rx=0 tx=0).stop 2024-05-27T09:31:47.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.730+0000 7f0c66000700 1 -- 172.21.15.119:0/2658641512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c601067a0 msgr2=0x7f0c6019bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:47.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.730+0000 7f0c66000700 1 --2- 172.21.15.119:0/2658641512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c601067a0 0x7f0c6019bfa0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000d900 tx=0x7f0c5000dcc0 comp rx=0 tx=0).stop 2024-05-27T09:31:47.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.731+0000 7f0c66000700 1 -- 172.21.15.119:0/2658641512 shutdown_connections 2024-05-27T09:31:47.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.731+0000 7f0c66000700 1 --2- 172.21.15.119:0/2658641512 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f0c440787f0 0x7f0c4407aca0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:47.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.731+0000 7f0c66000700 1 --2- 172.21.15.119:0/2658641512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c601067a0 0x7f0c6019bfa0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:47.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.731+0000 7f0c66000700 1 --2- 172.21.15.119:0/2658641512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c601079a0 0x7f0c6019c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:47.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.731+0000 7f0c66000700 1 -- 172.21.15.119:0/2658641512 >> 172.21.15.119:0/2658641512 conn(0x7f0c60075910 msgr2=0x7f0c6010abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:47.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.731+0000 7f0c66000700 1 -- 172.21.15.119:0/2658641512 shutdown_connections 2024-05-27T09:31:47.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.731+0000 7f0c66000700 1 -- 172.21.15.119:0/2658641512 wait complete. 2024-05-27T09:31:47.750 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:31:47.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.897+0000 7ff64b000700 1 -- 172.21.15.119:0/2476026245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff644073750 msgr2=0x7ff644073bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:47.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.897+0000 7ff64b000700 1 --2- 172.21.15.119:0/2476026245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff644073750 0x7ff644073bc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff630009910 tx=0x7ff630009c20 comp rx=0 tx=0).stop 2024-05-27T09:31:47.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.897+0000 7ff64b000700 1 -- 172.21.15.119:0/2476026245 shutdown_connections 2024-05-27T09:31:47.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.897+0000 7ff64b000700 1 --2- 172.21.15.119:0/2476026245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff644073750 0x7ff644073bc0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:47.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.897+0000 7ff64b000700 1 --2- 172.21.15.119:0/2476026245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff644074d20 0x7ff644073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:47.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.897+0000 7ff64b000700 1 -- 172.21.15.119:0/2476026245 >> 172.21.15.119:0/2476026245 conn(0x7ff6440fb6e0 msgr2=0x7ff6440fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:47.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.898+0000 7ff64b000700 1 -- 172.21.15.119:0/2476026245 shutdown_connections 2024-05-27T09:31:47.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.898+0000 7ff64b000700 1 -- 172.21.15.119:0/2476026245 wait complete. 2024-05-27T09:31:47.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.899+0000 7ff64b000700 1 Processor -- start 2024-05-27T09:31:47.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.899+0000 7ff64b000700 1 -- start start 2024-05-27T09:31:47.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.900+0000 7ff64b000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff644073750 0x7ff644197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:47.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.900+0000 7ff64b000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff644074d20 0x7ff644198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:47.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.900+0000 7ff64b000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6441986b0 con 0x7ff644073750 2024-05-27T09:31:47.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.900+0000 7ff64b000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6441987f0 con 0x7ff644074d20 2024-05-27T09:31:47.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.900+0000 7ff643e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff644073750 0x7ff644197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:47.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.900+0000 7ff643e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff644073750 0x7ff644197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48470/0 (socket says 172.21.15.119:48470) 2024-05-27T09:31:47.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.900+0000 7ff643e00700 1 -- 172.21.15.119:0/2898603234 learned_addr learned my addr 172.21.15.119:0/2898603234 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:31:47.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.900+0000 7ff643400700 1 --2- 172.21.15.119:0/2898603234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff644074d20 0x7ff644198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:47.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.901+0000 7ff643e00700 1 -- 172.21.15.119:0/2898603234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff644074d20 msgr2=0x7ff644198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:47.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.901+0000 7ff643e00700 1 --2- 172.21.15.119:0/2898603234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff644074d20 0x7ff644198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:47.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.901+0000 7ff643e00700 1 -- 172.21.15.119:0/2898603234 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6300095f0 con 0x7ff644073750 2024-05-27T09:31:47.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.901+0000 7ff643e00700 1 --2- 172.21.15.119:0/2898603234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff644073750 0x7ff644197b50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff63400d8d0 tx=0x7ff63400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:47.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.901+0000 7ff640c00700 1 -- 172.21.15.119:0/2898603234 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff634009940 con 0x7ff644073750 2024-05-27T09:31:47.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.901+0000 7ff640c00700 1 -- 172.21.15.119:0/2898603234 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff634010460 con 0x7ff644073750 2024-05-27T09:31:47.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.901+0000 7ff64b000700 1 -- 172.21.15.119:0/2898603234 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64419d2a0 con 0x7ff644073750 2024-05-27T09:31:47.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.901+0000 7ff640c00700 1 -- 172.21.15.119:0/2898603234 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff63400f5f0 con 0x7ff644073750 2024-05-27T09:31:47.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.901+0000 7ff64b000700 1 -- 172.21.15.119:0/2898603234 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64419d7f0 con 0x7ff644073750 2024-05-27T09:31:47.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.903+0000 7ff64b000700 1 -- 172.21.15.119:0/2898603234 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff64410ae90 con 0x7ff644073750 2024-05-27T09:31:47.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.904+0000 7ff640c00700 1 -- 172.21.15.119:0/2898603234 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 100037+0+0 (secure 0 0 0) 0x7ff6340105d0 con 0x7ff644073750 2024-05-27T09:31:47.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.905+0000 7ff640c00700 1 --2- 172.21.15.119:0/2898603234 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7ff6280787e0 0x7ff62807ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:47.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.905+0000 7ff640c00700 1 -- 172.21.15.119:0/2898603234 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff63409aa20 con 0x7ff644073750 2024-05-27T09:31:47.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.906+0000 7ff643400700 1 --2- 172.21.15.119:0/2898603234 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7ff6280787e0 0x7ff62807ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:47.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.906+0000 7ff643400700 1 --2- 172.21.15.119:0/2898603234 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7ff6280787e0 0x7ff62807ac90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff630006010 tx=0x7ff63000b540 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:47.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:47.910+0000 7ff640c00700 1 -- 172.21.15.119:0/2898603234 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7ff6340628e0 con 0x7ff644073750 2024-05-27T09:31:48.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.163+0000 7ff64b000700 1 -- 172.21.15.119:0/2898603234 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6441076b0 con 0x7ff6280787e0 2024-05-27T09:31:48.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.165+0000 7ff640c00700 1 -- 172.21.15.119:0/2898603234 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7ff6441076b0 con 0x7ff6280787e0 2024-05-27T09:31:48.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.168+0000 7ff64b000700 1 -- 172.21.15.119:0/2898603234 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7ff6280787e0 msgr2=0x7ff62807ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:48.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.168+0000 7ff64b000700 1 --2- 172.21.15.119:0/2898603234 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7ff6280787e0 0x7ff62807ac90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff630006010 tx=0x7ff63000b540 comp rx=0 tx=0).stop 2024-05-27T09:31:48.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.168+0000 7ff64b000700 1 -- 172.21.15.119:0/2898603234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff644073750 msgr2=0x7ff644197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:48.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.168+0000 7ff64b000700 1 --2- 172.21.15.119:0/2898603234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff644073750 0x7ff644197b50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff63400d8d0 tx=0x7ff63400dc90 comp rx=0 tx=0).stop 2024-05-27T09:31:48.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.168+0000 7ff64b000700 1 -- 172.21.15.119:0/2898603234 shutdown_connections 2024-05-27T09:31:48.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.168+0000 7ff64b000700 1 --2- 172.21.15.119:0/2898603234 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7ff6280787e0 0x7ff62807ac90 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:48.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.168+0000 7ff64b000700 1 --2- 172.21.15.119:0/2898603234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff644073750 0x7ff644197b50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:48.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.168+0000 7ff64b000700 1 --2- 172.21.15.119:0/2898603234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff644074d20 0x7ff644198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:48.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.168+0000 7ff64b000700 1 -- 172.21.15.119:0/2898603234 >> 172.21.15.119:0/2898603234 conn(0x7ff6440fb6e0 msgr2=0x7ff644105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:48.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.168+0000 7ff64b000700 1 -- 172.21.15.119:0/2898603234 shutdown_connections 2024-05-27T09:31:48.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.168+0000 7ff64b000700 1 -- 172.21.15.119:0/2898603234 wait complete. 2024-05-27T09:31:48.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.332+0000 7f2e05200700 1 -- 172.21.15.119:0/1522610670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e00100150 msgr2=0x7f2e00100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:48.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.332+0000 7f2e05200700 1 --2- 172.21.15.119:0/1522610670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e00100150 0x7f2e00100560 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009910 tx=0x7f2df0009c20 comp rx=0 tx=0).stop 2024-05-27T09:31:48.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.332+0000 7f2e05200700 1 -- 172.21.15.119:0/1522610670 shutdown_connections 2024-05-27T09:31:48.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.332+0000 7f2e05200700 1 --2- 172.21.15.119:0/1522610670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e00101350 0x7f2e001017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:48.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.332+0000 7f2e05200700 1 --2- 172.21.15.119:0/1522610670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e00100150 0x7f2e00100560 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:48.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.332+0000 7f2e05200700 1 -- 172.21.15.119:0/1522610670 >> 172.21.15.119:0/1522610670 conn(0x7f2e000fb6e0 msgr2=0x7f2e000fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:48.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.332+0000 7f2e05200700 1 -- 172.21.15.119:0/1522610670 shutdown_connections 2024-05-27T09:31:48.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.332+0000 7f2e05200700 1 -- 172.21.15.119:0/1522610670 wait complete. 2024-05-27T09:31:48.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.333+0000 7f2e05200700 1 Processor -- start 2024-05-27T09:31:48.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.333+0000 7f2e05200700 1 -- start start 2024-05-27T09:31:48.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.333+0000 7f2e05200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e00100150 0x7f2e00197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:48.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.334+0000 7f2e05200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e00101350 0x7f2e001980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:48.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.334+0000 7f2e05200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e00198700 con 0x7f2e00100150 2024-05-27T09:31:48.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.334+0000 7f2e05200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e00198840 con 0x7f2e00101350 2024-05-27T09:31:48.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.334+0000 7f2dfde00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e00100150 0x7f2e00197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:48.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.334+0000 7f2dfde00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e00100150 0x7f2e00197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48478/0 (socket says 172.21.15.119:48478) 2024-05-27T09:31:48.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.334+0000 7f2dfde00700 1 -- 172.21.15.119:0/1178679868 learned_addr learned my addr 172.21.15.119:0/1178679868 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:31:48.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.334+0000 7f2df7e00700 1 --2- 172.21.15.119:0/1178679868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e00101350 0x7f2e001980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:48.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.334+0000 7f2dfde00700 1 -- 172.21.15.119:0/1178679868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e00101350 msgr2=0x7f2e001980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:48.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.334+0000 7f2dfde00700 1 --2- 172.21.15.119:0/1178679868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e00101350 0x7f2e001980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:48.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.334+0000 7f2dfde00700 1 -- 172.21.15.119:0/1178679868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df00095f0 con 0x7f2e00100150 2024-05-27T09:31:48.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.335+0000 7f2dfde00700 1 --2- 172.21.15.119:0/1178679868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e00100150 0x7f2e00197ba0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2df000bda0 tx=0x7f2df000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:48.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.335+0000 7f2df6a00700 1 -- 172.21.15.119:0/1178679868 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2df001d070 con 0x7f2e00100150 2024-05-27T09:31:48.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.335+0000 7f2df6a00700 1 -- 172.21.15.119:0/1178679868 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2df000f460 con 0x7f2e00100150 2024-05-27T09:31:48.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.335+0000 7f2e05200700 1 -- 172.21.15.119:0/1178679868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e0019d290 con 0x7f2e00100150 2024-05-27T09:31:48.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.335+0000 7f2df6a00700 1 -- 172.21.15.119:0/1178679868 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2df0021640 con 0x7f2e00100150 2024-05-27T09:31:48.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.335+0000 7f2e05200700 1 -- 172.21.15.119:0/1178679868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e0019d780 con 0x7f2e00100150 2024-05-27T09:31:48.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.336+0000 7f2e05200700 1 -- 172.21.15.119:0/1178679868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e00105430 con 0x7f2e00100150 2024-05-27T09:31:48.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.338+0000 7f2df6a00700 1 -- 172.21.15.119:0/1178679868 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 100037+0+0 (secure 0 0 0) 0x7f2df000f5d0 con 0x7f2e00100150 2024-05-27T09:31:48.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.339+0000 7f2df6a00700 1 --2- 172.21.15.119:0/1178679868 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f2de40787f0 0x7f2de407aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:48.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.339+0000 7f2df6a00700 1 -- 172.21.15.119:0/1178679868 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2df009bad0 con 0x7f2e00100150 2024-05-27T09:31:48.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.339+0000 7f2df7e00700 1 --2- 172.21.15.119:0/1178679868 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f2de40787f0 0x7f2de407aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:48.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.340+0000 7f2df7e00700 1 --2- 172.21.15.119:0/1178679868 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f2de40787f0 0x7f2de407aca0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2de8005950 tx=0x7f2de800b410 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:48.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.345+0000 7f2df6a00700 1 -- 172.21.15.119:0/1178679868 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f2df0063a60 con 0x7f2e00100150 2024-05-27T09:31:48.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.591+0000 7f2e05200700 1 -- 172.21.15.119:0/1178679868 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e00061190 con 0x7f2de40787f0 2024-05-27T09:31:48.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.622+0000 7f2df6a00700 1 -- 172.21.15.119:0/1178679868 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f2e00061190 con 0x7f2de40787f0 2024-05-27T09:31:48.623 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:31:48.623 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (11m) 0s ago 12m 14.0M - 0.25.0 c8568f914cd2 bc280230ab28 2024-05-27T09:31:48.623 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (13m) 0s ago 13m 7960k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:31:48.623 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (11m) 28s ago 11m 7801k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:31:48.623 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (13m) 0s ago 13m 6665k - 18.2.0 dc2bc1663786 e5fafde44183 2024-05-27T09:31:48.623 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (11m) 28s ago 11m 6660k - 18.2.0 dc2bc1663786 46956ab99bab 2024-05-27T09:31:48.623 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (10m) 0s ago 12m 74.8M - 9.4.7 954c08fa6188 16eeaded4295 2024-05-27T09:31:48.623 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (5m) 0s ago 5m 14.9M - 18.2.0 dc2bc1663786 93b21713fb1e 2024-05-27T09:31:48.623 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (5m) 0s ago 5m 130M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:31:48.624 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (5m) 28s ago 5m 14.8M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:31:48.624 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (5m) 28s ago 5m 142M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:31:48.624 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (96s) 0s ago 14m 580M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:31:48.624 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (34s) 28s ago 11m 121M - 19.0.0-3868-ga6507388 850a06a84d28 2fc3a80696fa 2024-05-27T09:31:48.624 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (14m) 0s ago 14m 57.6M 2048M 18.2.0 dc2bc1663786 ba998b789a1c 2024-05-27T09:31:48.624 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (11m) 28s ago 11m 45.5M 2048M 18.2.0 dc2bc1663786 60216fd67078 2024-05-27T09:31:48.624 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (14s) 0s ago 13m 2819k - 1.5.0 0da6a335fe13 709c37815cff 2024-05-27T09:31:48.624 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 starting - - - - 2024-05-27T09:31:48.624 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (10m) 0s ago 10m 224M 1216M 18.2.0 dc2bc1663786 825bd1392508 2024-05-27T09:31:48.624 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (9m) 0s ago 9m 250M 1216M 18.2.0 dc2bc1663786 fce9819903d9 2024-05-27T09:31:48.624 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (9m) 0s ago 9m 189M 1216M 18.2.0 dc2bc1663786 3955a6112c1c 2024-05-27T09:31:48.624 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (8m) 28s ago 8m 261M 2240M 18.2.0 dc2bc1663786 693c92220622 2024-05-27T09:31:48.625 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (8m) 28s ago 8m 234M 2240M 18.2.0 dc2bc1663786 7559e658e917 2024-05-27T09:31:48.625 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (7m) 28s ago 7m 226M 2240M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:31:48.625 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (50s) 0s ago 12m 56.7M - 2.43.0 a07b618ecd1d 74722ed1e9c2 2024-05-27T09:31:48.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.626+0000 7f2e05200700 1 -- 172.21.15.119:0/1178679868 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f2de40787f0 msgr2=0x7f2de407aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:48.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.626+0000 7f2e05200700 1 --2- 172.21.15.119:0/1178679868 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f2de40787f0 0x7f2de407aca0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2de8005950 tx=0x7f2de800b410 comp rx=0 tx=0).stop 2024-05-27T09:31:48.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.626+0000 7f2e05200700 1 -- 172.21.15.119:0/1178679868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e00100150 msgr2=0x7f2e00197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:48.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.626+0000 7f2e05200700 1 --2- 172.21.15.119:0/1178679868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e00100150 0x7f2e00197ba0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2df000bda0 tx=0x7f2df000bdd0 comp rx=0 tx=0).stop 2024-05-27T09:31:48.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.626+0000 7f2e05200700 1 -- 172.21.15.119:0/1178679868 shutdown_connections 2024-05-27T09:31:48.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.626+0000 7f2e05200700 1 --2- 172.21.15.119:0/1178679868 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f2de40787f0 0x7f2de407aca0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:48.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.626+0000 7f2e05200700 1 --2- 172.21.15.119:0/1178679868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e00100150 0x7f2e00197ba0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:48.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.626+0000 7f2e05200700 1 --2- 172.21.15.119:0/1178679868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e00101350 0x7f2e001980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:48.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.626+0000 7f2e05200700 1 -- 172.21.15.119:0/1178679868 >> 172.21.15.119:0/1178679868 conn(0x7f2e000fb6e0 msgr2=0x7f2e00102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:48.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.626+0000 7f2e05200700 1 -- 172.21.15.119:0/1178679868 shutdown_connections 2024-05-27T09:31:48.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.626+0000 7f2e05200700 1 -- 172.21.15.119:0/1178679868 wait complete. 2024-05-27T09:31:48.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.779+0000 7fe866400700 1 -- 172.21.15.119:0/4053748366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8600737b0 msgr2=0x7fe860073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:48.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.779+0000 7fe866400700 1 --2- 172.21.15.119:0/4053748366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8600737b0 0x7fe860073c20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009940 tx=0x7fe84c009c50 comp rx=0 tx=0).stop 2024-05-27T09:31:48.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.779+0000 7fe866400700 1 -- 172.21.15.119:0/4053748366 shutdown_connections 2024-05-27T09:31:48.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.779+0000 7fe866400700 1 --2- 172.21.15.119:0/4053748366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8600737b0 0x7fe860073c20 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:48.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.779+0000 7fe866400700 1 --2- 172.21.15.119:0/4053748366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe860074d80 0x7fe8600731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:48.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.779+0000 7fe866400700 1 -- 172.21.15.119:0/4053748366 >> 172.21.15.119:0/4053748366 conn(0x7fe8600fb6e0 msgr2=0x7fe8600fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:48.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.779+0000 7fe866400700 1 -- 172.21.15.119:0/4053748366 shutdown_connections 2024-05-27T09:31:48.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.779+0000 7fe866400700 1 -- 172.21.15.119:0/4053748366 wait complete. 2024-05-27T09:31:48.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.780+0000 7fe866400700 1 Processor -- start 2024-05-27T09:31:48.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.780+0000 7fe866400700 1 -- start start 2024-05-27T09:31:48.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.781+0000 7fe866400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8600737b0 0x7fe860197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:48.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.781+0000 7fe866400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe860074d80 0x7fe8601980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:48.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.781+0000 7fe866400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8601986c0 con 0x7fe8600737b0 2024-05-27T09:31:48.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.781+0000 7fe866400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe860198800 con 0x7fe860074d80 2024-05-27T09:31:48.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.781+0000 7fe85f400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8600737b0 0x7fe860197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:48.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.781+0000 7fe85f400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8600737b0 0x7fe860197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42364/0 (socket says 172.21.15.119:42364) 2024-05-27T09:31:48.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.781+0000 7fe85f400700 1 -- 172.21.15.119:0/261468191 learned_addr learned my addr 172.21.15.119:0/261468191 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:31:48.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.781+0000 7fe85ea00700 1 --2- 172.21.15.119:0/261468191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe860074d80 0x7fe8601980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:48.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.782+0000 7fe85f400700 1 -- 172.21.15.119:0/261468191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe860074d80 msgr2=0x7fe8601980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:48.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.782+0000 7fe85f400700 1 --2- 172.21.15.119:0/261468191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe860074d80 0x7fe8601980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:48.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.782+0000 7fe85f400700 1 -- 172.21.15.119:0/261468191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84c0095f0 con 0x7fe8600737b0 2024-05-27T09:31:48.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.782+0000 7fe85ea00700 1 --2- 172.21.15.119:0/261468191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe860074d80 0x7fe8601980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:31:48.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.782+0000 7fe85f400700 1 --2- 172.21.15.119:0/261468191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8600737b0 0x7fe860197b60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe85000d8d0 tx=0x7fe85000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:48.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.782+0000 7fe857e00700 1 -- 172.21.15.119:0/261468191 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe850009940 con 0x7fe8600737b0 2024-05-27T09:31:48.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.782+0000 7fe857e00700 1 -- 172.21.15.119:0/261468191 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe850010460 con 0x7fe8600737b0 2024-05-27T09:31:48.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.782+0000 7fe866400700 1 -- 172.21.15.119:0/261468191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86019d2b0 con 0x7fe8600737b0 2024-05-27T09:31:48.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.782+0000 7fe866400700 1 -- 172.21.15.119:0/261468191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe86019d880 con 0x7fe8600737b0 2024-05-27T09:31:48.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.782+0000 7fe857e00700 1 -- 172.21.15.119:0/261468191 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe85000f5f0 con 0x7fe8600737b0 2024-05-27T09:31:48.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.784+0000 7fe866400700 1 -- 172.21.15.119:0/261468191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe840005320 con 0x7fe8600737b0 2024-05-27T09:31:48.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.786+0000 7fe857e00700 1 -- 172.21.15.119:0/261468191 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 100037+0+0 (secure 0 0 0) 0x7fe850010ad0 con 0x7fe8600737b0 2024-05-27T09:31:48.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.787+0000 7fe857e00700 1 --2- 172.21.15.119:0/261468191 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe8440787e0 0x7fe84407ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:48.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.787+0000 7fe857e00700 1 -- 172.21.15.119:0/261468191 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe85009a9b0 con 0x7fe8600737b0 2024-05-27T09:31:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.787+0000 7fe85ea00700 1 --2- 172.21.15.119:0/261468191 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe8440787e0 0x7fe84407ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.788+0000 7fe85ea00700 1 --2- 172.21.15.119:0/261468191 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe8440787e0 0x7fe84407ac90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe84c005b40 tx=0x7fe84c005ab0 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:48.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:48.792+0000 7fe857e00700 1 -- 172.21.15.119:0/261468191 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7fe850062870 con 0x7fe8600737b0 2024-05-27T09:31:49.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.186+0000 7fe866400700 1 -- 172.21.15.119:0/261468191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe840005cc0 con 0x7fe8600737b0 2024-05-27T09:31:49.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.188+0000 7fe857e00700 1 -- 172.21.15.119:0/261468191 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fe850062210 con 0x7fe8600737b0 2024-05-27T09:31:49.189 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:31:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:31:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-05-27T09:31:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:31:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:31:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:31:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:31:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:31:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-05-27T09:31:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:31:49.189 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:31:49.190 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-05-27T09:31:49.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:31:49.190 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:31:49.190 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-05-27T09:31:49.190 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:31:49.190 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:31:49.190 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:31:49.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.191+0000 7fe866400700 1 -- 172.21.15.119:0/261468191 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe8440787e0 msgr2=0x7fe84407ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:49.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.191+0000 7fe866400700 1 --2- 172.21.15.119:0/261468191 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe8440787e0 0x7fe84407ac90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe84c005b40 tx=0x7fe84c005ab0 comp rx=0 tx=0).stop 2024-05-27T09:31:49.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.191+0000 7fe866400700 1 -- 172.21.15.119:0/261468191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8600737b0 msgr2=0x7fe860197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:49.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.191+0000 7fe866400700 1 --2- 172.21.15.119:0/261468191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8600737b0 0x7fe860197b60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe85000d8d0 tx=0x7fe85000dc90 comp rx=0 tx=0).stop 2024-05-27T09:31:49.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.191+0000 7fe866400700 1 -- 172.21.15.119:0/261468191 shutdown_connections 2024-05-27T09:31:49.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.191+0000 7fe866400700 1 --2- 172.21.15.119:0/261468191 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe8440787e0 0x7fe84407ac90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:49.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.191+0000 7fe866400700 1 --2- 172.21.15.119:0/261468191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8600737b0 0x7fe860197b60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:49.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.191+0000 7fe866400700 1 --2- 172.21.15.119:0/261468191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe860074d80 0x7fe8601980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:49.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.191+0000 7fe866400700 1 -- 172.21.15.119:0/261468191 >> 172.21.15.119:0/261468191 conn(0x7fe8600fb6e0 msgr2=0x7fe860105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:49.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.191+0000 7fe866400700 1 -- 172.21.15.119:0/261468191 shutdown_connections 2024-05-27T09:31:49.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.191+0000 7fe866400700 1 -- 172.21.15.119:0/261468191 wait complete. 2024-05-27T09:31:49.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.356+0000 7f8617000700 1 -- 172.21.15.119:0/1794841919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8610101350 msgr2=0x7f86101017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:49.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.356+0000 7f8617000700 1 --2- 172.21.15.119:0/1794841919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8610101350 0x7f86101017a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009910 tx=0x7f85fc009c20 comp rx=0 tx=0).stop 2024-05-27T09:31:49.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.356+0000 7f8617000700 1 -- 172.21.15.119:0/1794841919 shutdown_connections 2024-05-27T09:31:49.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.356+0000 7f8617000700 1 --2- 172.21.15.119:0/1794841919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8610101350 0x7f86101017a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:49.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.356+0000 7f8617000700 1 --2- 172.21.15.119:0/1794841919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8610100150 0x7f8610100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:49.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.356+0000 7f8617000700 1 -- 172.21.15.119:0/1794841919 >> 172.21.15.119:0/1794841919 conn(0x7f86100fb6e0 msgr2=0x7f86100fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:49.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.356+0000 7f8617000700 1 -- 172.21.15.119:0/1794841919 shutdown_connections 2024-05-27T09:31:49.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.356+0000 7f8617000700 1 -- 172.21.15.119:0/1794841919 wait complete. 2024-05-27T09:31:49.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.358+0000 7f8617000700 1 Processor -- start 2024-05-27T09:31:49.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.358+0000 7f8617000700 1 -- start start 2024-05-27T09:31:49.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.359+0000 7f8617000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8610100150 0x7f8610195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:49.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.359+0000 7f8617000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8610101350 0x7f8610195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:49.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.359+0000 7f8617000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86101964c0 con 0x7f8610100150 2024-05-27T09:31:49.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.359+0000 7f8617000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8610196600 con 0x7f8610101350 2024-05-27T09:31:49.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.359+0000 7f860fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8610100150 0x7f8610195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:49.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.359+0000 7f860fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8610100150 0x7f8610195960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42390/0 (socket says 172.21.15.119:42390) 2024-05-27T09:31:49.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.359+0000 7f860fe00700 1 -- 172.21.15.119:0/2541521536 learned_addr learned my addr 172.21.15.119:0/2541521536 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:31:49.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.359+0000 7f860f400700 1 --2- 172.21.15.119:0/2541521536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8610101350 0x7f8610195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:49.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.359+0000 7f860fe00700 1 -- 172.21.15.119:0/2541521536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8610101350 msgr2=0x7f8610195ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:49.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.359+0000 7f860fe00700 1 --2- 172.21.15.119:0/2541521536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8610101350 0x7f8610195ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:49.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.359+0000 7f860fe00700 1 -- 172.21.15.119:0/2541521536 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85fc0095f0 con 0x7f8610100150 2024-05-27T09:31:49.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.359+0000 7f860f400700 1 --2- 172.21.15.119:0/2541521536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8610101350 0x7f8610195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:31:49.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.360+0000 7f860fe00700 1 --2- 172.21.15.119:0/2541521536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8610100150 0x7f8610195960 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f860000e8f0 tx=0x7f860000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:49.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.360+0000 7f860cc00700 1 -- 172.21.15.119:0/2541521536 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8600009790 con 0x7f8610100150 2024-05-27T09:31:49.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.360+0000 7f860cc00700 1 -- 172.21.15.119:0/2541521536 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f86000044d0 con 0x7f8610100150 2024-05-27T09:31:49.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.360+0000 7f8617000700 1 -- 172.21.15.119:0/2541521536 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f861019b0b0 con 0x7f8610100150 2024-05-27T09:31:49.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.360+0000 7f860cc00700 1 -- 172.21.15.119:0/2541521536 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8600003da0 con 0x7f8610100150 2024-05-27T09:31:49.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.360+0000 7f8617000700 1 -- 172.21.15.119:0/2541521536 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f861019b600 con 0x7f8610100150 2024-05-27T09:31:49.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.361+0000 7f8617000700 1 -- 172.21.15.119:0/2541521536 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8610108ca0 con 0x7f8610100150 2024-05-27T09:31:49.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.363+0000 7f860cc00700 1 -- 172.21.15.119:0/2541521536 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 100037+0+0 (secure 0 0 0) 0x7f860000cba0 con 0x7f8610100150 2024-05-27T09:31:49.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.364+0000 7f860cc00700 1 --2- 172.21.15.119:0/2541521536 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f85f40787e0 0x7f85f407ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:31:49.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.364+0000 7f860cc00700 1 -- 172.21.15.119:0/2541521536 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8600014070 con 0x7f8610100150 2024-05-27T09:31:49.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.365+0000 7f860f400700 1 --2- 172.21.15.119:0/2541521536 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f85f40787e0 0x7f85f407ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:31:49.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.365+0000 7f860f400700 1 --2- 172.21.15.119:0/2541521536 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f85f40787e0 0x7f85f407ac90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009fd0 tx=0x7f85fc005a90 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:31:49.367 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:49 smithi119 ceph-mon[29082]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:31:49.367 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:49 smithi119 ceph-mon[29082]: pgmap v43: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 682 B/s wr, 0 op/s 2024-05-27T09:31:49.367 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:49 smithi119 ceph-mon[29082]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:31:49.367 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:49 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:49.367 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:49 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:49.367 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:49 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:49.367 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:49 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:49.367 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:49 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/261468191' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:49.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.369+0000 7f860cc00700 1 -- 172.21.15.119:0/2541521536 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f8600062d40 con 0x7f8610100150 2024-05-27T09:31:49.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:49 smithi138 ceph-mon[34885]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:31:49.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:49 smithi138 ceph-mon[34885]: pgmap v43: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 682 B/s wr, 0 op/s 2024-05-27T09:31:49.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:49 smithi138 ceph-mon[34885]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:31:49.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:49 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:49.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:49 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:49.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:49 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:49.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:49 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:49.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:49 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/261468191' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:49.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.639+0000 7f8617000700 1 -- 172.21.15.119:0/2541521536 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8610105ca0 con 0x7f85f40787e0 2024-05-27T09:31:49.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.642+0000 7f860cc00700 1 -- 172.21.15.119:0/2541521536 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f8610105ca0 con 0x7f85f40787e0 2024-05-27T09:31:49.643 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:31:49.643 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:31:49.643 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:31:49.643 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-27T09:31:49.643 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [ 2024-05-27T09:31:49.643 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr" 2024-05-27T09:31:49.643 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-05-27T09:31:49.643 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "2/2 daemons upgraded", 2024-05-27T09:31:49.643 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "Currently upgrading node-exporter daemons", 2024-05-27T09:31:49.643 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:31:49.644 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:31:49.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.645+0000 7f8617000700 1 -- 172.21.15.119:0/2541521536 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f85f40787e0 msgr2=0x7f85f407ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:49.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.645+0000 7f8617000700 1 --2- 172.21.15.119:0/2541521536 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f85f40787e0 0x7f85f407ac90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009fd0 tx=0x7f85fc005a90 comp rx=0 tx=0).stop 2024-05-27T09:31:49.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.645+0000 7f8617000700 1 -- 172.21.15.119:0/2541521536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8610100150 msgr2=0x7f8610195960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:31:49.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.645+0000 7f8617000700 1 --2- 172.21.15.119:0/2541521536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8610100150 0x7f8610195960 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f860000e8f0 tx=0x7f860000ecb0 comp rx=0 tx=0).stop 2024-05-27T09:31:49.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.645+0000 7f8617000700 1 -- 172.21.15.119:0/2541521536 shutdown_connections 2024-05-27T09:31:49.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.645+0000 7f8617000700 1 --2- 172.21.15.119:0/2541521536 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f85f40787e0 0x7f85f407ac90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:49.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.645+0000 7f8617000700 1 --2- 172.21.15.119:0/2541521536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8610100150 0x7f8610195960 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:49.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.645+0000 7f8617000700 1 --2- 172.21.15.119:0/2541521536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8610101350 0x7f8610195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:31:49.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.645+0000 7f8617000700 1 -- 172.21.15.119:0/2541521536 >> 172.21.15.119:0/2541521536 conn(0x7f86100fb6e0 msgr2=0x7f8610104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:31:49.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.645+0000 7f8617000700 1 -- 172.21.15.119:0/2541521536 shutdown_connections 2024-05-27T09:31:49.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:31:49.646+0000 7f8617000700 1 -- 172.21.15.119:0/2541521536 wait complete. 2024-05-27T09:31:50.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:50 smithi138 ceph-mon[34885]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:31:50.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:50 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:50.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:50 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:50.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:50 smithi119 ceph-mon[29082]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:31:50.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:50 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:50.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:50 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:51.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:51 smithi119 ceph-mon[29082]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:31:51.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:51 smithi119 ceph-mon[29082]: pgmap v44: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 682 B/s wr, 0 op/s 2024-05-27T09:31:51.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:51 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:51.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:51 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:51.639 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:51 smithi138 ceph-mon[34885]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:31:51.639 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:51 smithi138 ceph-mon[34885]: pgmap v44: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 682 B/s wr, 0 op/s 2024-05-27T09:31:51.639 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:51 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:51.639 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:51 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:53.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:53 smithi138 ceph-mon[34885]: pgmap v45: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:31:53.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:53 smithi119 ceph-mon[29082]: pgmap v45: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:31:55.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:55 smithi119 ceph-mon[29082]: pgmap v46: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:31:55.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:55 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:55.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:55 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:55.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:55 smithi138 ceph-mon[34885]: pgmap v46: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:31:55.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:55 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:55.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:55 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:56.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:56 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:56.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:56 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:56.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:56 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:31:56.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:56 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:31:56.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:56 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:56.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:56 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:31:56.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:56 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:56.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:56 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:56.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:56 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:56.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:56 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:56.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:56 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:56.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:56 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:56.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:56 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:56.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:56 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:56.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:56 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:56.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:56 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:56.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:56 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:31:56.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:56 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:31:56.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:56 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:31:56.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:56 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:31:56.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:56 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:56.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:56 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:56.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:56 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:56.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:56 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:56.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:56 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:56.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:56 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:56.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:56 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:56.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:56 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:31:57.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:57 smithi119 ceph-mon[29082]: pgmap v47: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:31:57.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:57 smithi138 ceph-mon[34885]: pgmap v47: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:31:58.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:58 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:31:58.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:58 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:31:59.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:59 smithi119 ceph-mon[29082]: pgmap v48: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:31:59.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:59 smithi119 ceph-mon[29082]: Upgrade: Updating prometheus.smithi119 2024-05-27T09:31:59.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:31:59 smithi119 ceph-mon[29082]: Deploying daemon prometheus.smithi119 on smithi119 2024-05-27T09:31:59.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:59 smithi138 ceph-mon[34885]: pgmap v48: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:31:59.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:59 smithi138 ceph-mon[34885]: Upgrade: Updating prometheus.smithi119 2024-05-27T09:31:59.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:31:59 smithi138 ceph-mon[34885]: Deploying daemon prometheus.smithi119 on smithi119 2024-05-27T09:32:00.714 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:00 smithi119 ceph-mon[29082]: mgrmap e33: smithi119.ncwmog(active, since 91s), standbys: smithi138.dxsuqf 2024-05-27T09:32:00.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:00 smithi138 ceph-mon[34885]: mgrmap e33: smithi119.ncwmog(active, since 91s), standbys: smithi138.dxsuqf 2024-05-27T09:32:01.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:01 smithi119 ceph-mon[29082]: pgmap v49: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:32:01.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:01 smithi138 ceph-mon[34885]: pgmap v49: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:32:03.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:03 smithi119 ceph-mon[29082]: pgmap v50: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:32:03.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:03 smithi138 ceph-mon[34885]: pgmap v50: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:32:04.760 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:04 smithi119 ceph-mon[29082]: pgmap v51: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:32:04.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:04 smithi138 ceph-mon[34885]: pgmap v51: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:32:07.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:06 smithi119 ceph-mon[29082]: pgmap v52: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:32:07.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:06 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:07.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:06 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:07.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:06 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:32:07.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:06 smithi138 ceph-mon[34885]: pgmap v52: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:32:07.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:06 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:07.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:06 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:07.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:06 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:32:09.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:08 smithi119 ceph-mon[29082]: pgmap v53: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:32:09.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:08 smithi138 ceph-mon[34885]: pgmap v53: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:32:11.232 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:10 smithi119 ceph-mon[29082]: pgmap v54: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:32:11.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:10 smithi138 ceph-mon[34885]: pgmap v54: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:32:13.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:13 smithi119 ceph-mon[29082]: pgmap v55: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-05-27T09:32:13.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:13 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:32:13.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:13 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:13.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:13 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:13.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:13 smithi138 ceph-mon[34885]: pgmap v55: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-05-27T09:32:13.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:13 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:32:13.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:13 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:13.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:13 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:15.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:15 smithi119 ceph-mon[29082]: pgmap v56: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-05-27T09:32:15.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:15 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:15.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:15 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:15.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:15 smithi138 ceph-mon[34885]: pgmap v56: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-05-27T09:32:15.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:15 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:15.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:15 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:17.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:17 smithi119 ceph-mon[29082]: pgmap v57: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-05-27T09:32:17.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:17 smithi138 ceph-mon[34885]: pgmap v57: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-05-27T09:32:19.487 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:19 smithi119 ceph-mon[29082]: pgmap v58: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:19.638 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:19 smithi138 ceph-mon[34885]: pgmap v58: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:19.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.819+0000 7f4136800700 1 -- 172.21.15.119:0/2735775713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4130076950 msgr2=0x7f4130076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:19.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.819+0000 7f4136800700 1 --2- 172.21.15.119:0/2735775713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4130076950 0x7f4130076dc0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f411c009910 tx=0x7f411c009c20 comp rx=0 tx=0).stop 2024-05-27T09:32:19.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.819+0000 7f4136800700 1 -- 172.21.15.119:0/2735775713 shutdown_connections 2024-05-27T09:32:19.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.819+0000 7f4136800700 1 --2- 172.21.15.119:0/2735775713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4130076950 0x7f4130076dc0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:19.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.819+0000 7f4136800700 1 --2- 172.21.15.119:0/2735775713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4130075700 0x7f4130075b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:19.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.819+0000 7f4136800700 1 -- 172.21.15.119:0/2735775713 >> 172.21.15.119:0/2735775713 conn(0x7f41300fd930 msgr2=0x7f41300ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:19.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.820+0000 7f4136800700 1 -- 172.21.15.119:0/2735775713 shutdown_connections 2024-05-27T09:32:19.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.820+0000 7f4136800700 1 -- 172.21.15.119:0/2735775713 wait complete. 2024-05-27T09:32:19.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.821+0000 7f4136800700 1 Processor -- start 2024-05-27T09:32:19.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.821+0000 7f4136800700 1 -- start start 2024-05-27T09:32:19.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.821+0000 7f4136800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4130075700 0x7f413019bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:19.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.821+0000 7f4136800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4130076950 0x7f413019c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:19.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.821+0000 7f4136800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413019caa0 con 0x7f4130076950 2024-05-27T09:32:19.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.821+0000 7f4136800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413019cbe0 con 0x7f4130075700 2024-05-27T09:32:19.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.822+0000 7f412ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4130076950 0x7f413019c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:19.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.822+0000 7f412ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4130076950 0x7f413019c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35768/0 (socket says 172.21.15.119:35768) 2024-05-27T09:32:19.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.822+0000 7f412ea00700 1 -- 172.21.15.119:0/3737978590 learned_addr learned my addr 172.21.15.119:0/3737978590 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:32:19.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.822+0000 7f412f400700 1 --2- 172.21.15.119:0/3737978590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4130075700 0x7f413019bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:19.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.822+0000 7f412ea00700 1 -- 172.21.15.119:0/3737978590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4130075700 msgr2=0x7f413019bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:19.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.822+0000 7f412ea00700 1 --2- 172.21.15.119:0/3737978590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4130075700 0x7f413019bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:19.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.822+0000 7f412ea00700 1 -- 172.21.15.119:0/3737978590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c0095f0 con 0x7f4130076950 2024-05-27T09:32:19.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.823+0000 7f412ea00700 1 --2- 172.21.15.119:0/3737978590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4130076950 0x7f413019c480 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f411c005b40 tx=0x7f411c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:19.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.823+0000 7f4127e00700 1 -- 172.21.15.119:0/3737978590 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f411c01d070 con 0x7f4130076950 2024-05-27T09:32:19.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.823+0000 7f4127e00700 1 -- 172.21.15.119:0/3737978590 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f411c00f460 con 0x7f4130076950 2024-05-27T09:32:19.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.823+0000 7f4136800700 1 -- 172.21.15.119:0/3737978590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41301a1630 con 0x7f4130076950 2024-05-27T09:32:19.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.823+0000 7f4127e00700 1 -- 172.21.15.119:0/3737978590 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f411c021620 con 0x7f4130076950 2024-05-27T09:32:19.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.823+0000 7f4136800700 1 -- 172.21.15.119:0/3737978590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41301a1b20 con 0x7f4130076950 2024-05-27T09:32:19.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.824+0000 7f4136800700 1 -- 172.21.15.119:0/3737978590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f413010f2d0 con 0x7f4130076950 2024-05-27T09:32:19.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.825+0000 7f4127e00700 1 -- 172.21.15.119:0/3737978590 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7f411c00fa40 con 0x7f4130076950 2024-05-27T09:32:19.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.826+0000 7f4127e00700 1 --2- 172.21.15.119:0/3737978590 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f41140786c0 0x7f411407ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:19.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.827+0000 7f4127e00700 1 -- 172.21.15.119:0/3737978590 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f411c09bb80 con 0x7f4130076950 2024-05-27T09:32:19.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.827+0000 7f412f400700 1 --2- 172.21.15.119:0/3737978590 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f41140786c0 0x7f411407ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:19.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.828+0000 7f412f400700 1 --2- 172.21.15.119:0/3737978590 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f41140786c0 0x7f411407ab70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4120005950 tx=0x7f412000b410 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:19.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:19.832+0000 7f4127e00700 1 -- 172.21.15.119:0/3737978590 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f411c063be0 con 0x7f4130076950 2024-05-27T09:32:20.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.105+0000 7f4136800700 1 -- 172.21.15.119:0/3737978590 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f41301a2220 con 0x7f41140786c0 2024-05-27T09:32:20.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.107+0000 7f4127e00700 1 -- 172.21.15.119:0/3737978590 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f41301a2220 con 0x7f41140786c0 2024-05-27T09:32:20.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.110+0000 7f4136800700 1 -- 172.21.15.119:0/3737978590 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f41140786c0 msgr2=0x7f411407ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:20.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.110+0000 7f4136800700 1 --2- 172.21.15.119:0/3737978590 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f41140786c0 0x7f411407ab70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4120005950 tx=0x7f412000b410 comp rx=0 tx=0).stop 2024-05-27T09:32:20.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.110+0000 7f4136800700 1 -- 172.21.15.119:0/3737978590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4130076950 msgr2=0x7f413019c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:20.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.110+0000 7f4136800700 1 --2- 172.21.15.119:0/3737978590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4130076950 0x7f413019c480 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f411c005b40 tx=0x7f411c00bd70 comp rx=0 tx=0).stop 2024-05-27T09:32:20.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.110+0000 7f4136800700 1 -- 172.21.15.119:0/3737978590 shutdown_connections 2024-05-27T09:32:20.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.110+0000 7f4136800700 1 --2- 172.21.15.119:0/3737978590 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f41140786c0 0x7f411407ab70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:20.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.110+0000 7f4136800700 1 --2- 172.21.15.119:0/3737978590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4130075700 0x7f413019bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:20.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.110+0000 7f4136800700 1 --2- 172.21.15.119:0/3737978590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4130076950 0x7f413019c480 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:20.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.110+0000 7f4136800700 1 -- 172.21.15.119:0/3737978590 >> 172.21.15.119:0/3737978590 conn(0x7f41300fd930 msgr2=0x7f413010c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:20.111 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.110+0000 7f4136800700 1 -- 172.21.15.119:0/3737978590 shutdown_connections 2024-05-27T09:32:20.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.110+0000 7f4136800700 1 -- 172.21.15.119:0/3737978590 wait complete. 2024-05-27T09:32:20.127 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:32:20.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.267+0000 7f965d600700 1 -- 172.21.15.119:0/3259144646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96580fff90 msgr2=0x7f96581003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:20.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.267+0000 7f965d600700 1 --2- 172.21.15.119:0/3259144646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96580fff90 0x7f96581003a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9648009910 tx=0x7f9648009c20 comp rx=0 tx=0).stop 2024-05-27T09:32:20.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.267+0000 7f965d600700 1 -- 172.21.15.119:0/3259144646 shutdown_connections 2024-05-27T09:32:20.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.267+0000 7f965d600700 1 --2- 172.21.15.119:0/3259144646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9658101190 0x7f96581015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:20.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.267+0000 7f965d600700 1 --2- 172.21.15.119:0/3259144646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96580fff90 0x7f96581003a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:20.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.267+0000 7f965d600700 1 -- 172.21.15.119:0/3259144646 >> 172.21.15.119:0/3259144646 conn(0x7f96580fb500 msgr2=0x7f96580fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:20.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.267+0000 7f965d600700 1 -- 172.21.15.119:0/3259144646 shutdown_connections 2024-05-27T09:32:20.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.267+0000 7f965d600700 1 -- 172.21.15.119:0/3259144646 wait complete. 2024-05-27T09:32:20.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.268+0000 7f965d600700 1 Processor -- start 2024-05-27T09:32:20.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.268+0000 7f965d600700 1 -- start start 2024-05-27T09:32:20.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.268+0000 7f965d600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96580fff90 0x7f9658195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:20.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.268+0000 7f965d600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9658101190 0x7f9658195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:20.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.268+0000 7f965d600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96581963b0 con 0x7f9658101190 2024-05-27T09:32:20.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.268+0000 7f965d600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96581964f0 con 0x7f96580fff90 2024-05-27T09:32:20.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.269+0000 7f964f400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9658101190 0x7f9658195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:20.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.269+0000 7f964f400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9658101190 0x7f9658195d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35776/0 (socket says 172.21.15.119:35776) 2024-05-27T09:32:20.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.269+0000 7f964f400700 1 -- 172.21.15.119:0/1388802490 learned_addr learned my addr 172.21.15.119:0/1388802490 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:32:20.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.269+0000 7f9657e00700 1 --2- 172.21.15.119:0/1388802490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96580fff90 0x7f9658195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:20.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.269+0000 7f964f400700 1 -- 172.21.15.119:0/1388802490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96580fff90 msgr2=0x7f9658195850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:20.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.269+0000 7f964f400700 1 --2- 172.21.15.119:0/1388802490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96580fff90 0x7f9658195850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:20.270 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.269+0000 7f964f400700 1 -- 172.21.15.119:0/1388802490 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96480095f0 con 0x7f9658101190 2024-05-27T09:32:20.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.270+0000 7f964f400700 1 --2- 172.21.15.119:0/1388802490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9658101190 0x7f9658195d90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f964000b4b0 tx=0x7f964000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:20.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.270+0000 7f9655600700 1 -- 172.21.15.119:0/1388802490 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9640011800 con 0x7f9658101190 2024-05-27T09:32:20.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.270+0000 7f9655600700 1 -- 172.21.15.119:0/1388802490 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9640011e10 con 0x7f9658101190 2024-05-27T09:32:20.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.270+0000 7f9655600700 1 -- 172.21.15.119:0/1388802490 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f964000f2e0 con 0x7f9658101190 2024-05-27T09:32:20.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.270+0000 7f965d600700 1 -- 172.21.15.119:0/1388802490 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f965819afa0 con 0x7f9658101190 2024-05-27T09:32:20.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.270+0000 7f965d600700 1 -- 172.21.15.119:0/1388802490 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f965819b540 con 0x7f9658101190 2024-05-27T09:32:20.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.271+0000 7f965d600700 1 -- 172.21.15.119:0/1388802490 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9658108b60 con 0x7f9658101190 2024-05-27T09:32:20.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.274+0000 7f9655600700 1 -- 172.21.15.119:0/1388802490 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7f9640010380 con 0x7f9658101190 2024-05-27T09:32:20.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.274+0000 7f9655600700 1 --2- 172.21.15.119:0/1388802490 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f963c0786c0 0x7f963c07ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:20.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.275+0000 7f9655600700 1 -- 172.21.15.119:0/1388802490 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f964009a3a0 con 0x7f9658101190 2024-05-27T09:32:20.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.275+0000 7f9657e00700 1 --2- 172.21.15.119:0/1388802490 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f963c0786c0 0x7f963c07ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:20.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.276+0000 7f9657e00700 1 --2- 172.21.15.119:0/1388802490 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f963c0786c0 0x7f963c07ab70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9648006010 tx=0x7f964800b540 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:20.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.280+0000 7f9655600700 1 -- 172.21.15.119:0/1388802490 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f9640062480 con 0x7f9658101190 2024-05-27T09:32:20.547 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.546+0000 7f965d600700 1 -- 172.21.15.119:0/1388802490 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9658105ae0 con 0x7f963c0786c0 2024-05-27T09:32:20.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.548+0000 7f9655600700 1 -- 172.21.15.119:0/1388802490 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f9658105ae0 con 0x7f963c0786c0 2024-05-27T09:32:20.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.551+0000 7f965d600700 1 -- 172.21.15.119:0/1388802490 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f963c0786c0 msgr2=0x7f963c07ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:20.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.551+0000 7f965d600700 1 --2- 172.21.15.119:0/1388802490 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f963c0786c0 0x7f963c07ab70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9648006010 tx=0x7f964800b540 comp rx=0 tx=0).stop 2024-05-27T09:32:20.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.551+0000 7f965d600700 1 -- 172.21.15.119:0/1388802490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9658101190 msgr2=0x7f9658195d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:20.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.551+0000 7f965d600700 1 --2- 172.21.15.119:0/1388802490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9658101190 0x7f9658195d90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f964000b4b0 tx=0x7f964000b870 comp rx=0 tx=0).stop 2024-05-27T09:32:20.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.552+0000 7f965d600700 1 -- 172.21.15.119:0/1388802490 shutdown_connections 2024-05-27T09:32:20.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.552+0000 7f965d600700 1 --2- 172.21.15.119:0/1388802490 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f963c0786c0 0x7f963c07ab70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:20.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.552+0000 7f965d600700 1 --2- 172.21.15.119:0/1388802490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96580fff90 0x7f9658195850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:20.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.552+0000 7f965d600700 1 --2- 172.21.15.119:0/1388802490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9658101190 0x7f9658195d90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:20.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.552+0000 7f965d600700 1 -- 172.21.15.119:0/1388802490 >> 172.21.15.119:0/1388802490 conn(0x7f96580fb500 msgr2=0x7f96581043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:20.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.552+0000 7f965d600700 1 -- 172.21.15.119:0/1388802490 shutdown_connections 2024-05-27T09:32:20.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.552+0000 7f965d600700 1 -- 172.21.15.119:0/1388802490 wait complete. 2024-05-27T09:32:20.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.709+0000 7f5f01a00700 1 -- 172.21.15.119:0/3395461249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5efc100be0 msgr2=0x7f5efc102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:20.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.709+0000 7f5f01a00700 1 --2- 172.21.15.119:0/3395461249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5efc100be0 0x7f5efc102fc0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009910 tx=0x7f5eec009c20 comp rx=0 tx=0).stop 2024-05-27T09:32:20.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.709+0000 7f5f01a00700 1 -- 172.21.15.119:0/3395461249 shutdown_connections 2024-05-27T09:32:20.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.709+0000 7f5f01a00700 1 --2- 172.21.15.119:0/3395461249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5efc103500 0x7f5efc1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:20.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.709+0000 7f5f01a00700 1 --2- 172.21.15.119:0/3395461249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5efc100be0 0x7f5efc102fc0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:20.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.709+0000 7f5f01a00700 1 -- 172.21.15.119:0/3395461249 >> 172.21.15.119:0/3395461249 conn(0x7f5efc0fa5e0 msgr2=0x7f5efc0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:20.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.709+0000 7f5f01a00700 1 -- 172.21.15.119:0/3395461249 shutdown_connections 2024-05-27T09:32:20.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.709+0000 7f5f01a00700 1 -- 172.21.15.119:0/3395461249 wait complete. 2024-05-27T09:32:20.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.710+0000 7f5f01a00700 1 Processor -- start 2024-05-27T09:32:20.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.711+0000 7f5f01a00700 1 -- start start 2024-05-27T09:32:20.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.711+0000 7f5f01a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5efc100be0 0x7f5efc1959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:20.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.711+0000 7f5f01a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5efc103500 0x7f5efc195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:20.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.711+0000 7f5f01a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc196500 con 0x7f5efc103500 2024-05-27T09:32:20.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.711+0000 7f5f01a00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc196640 con 0x7f5efc100be0 2024-05-27T09:32:20.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.711+0000 7f5efa000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5efc103500 0x7f5efc195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:20.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.711+0000 7f5efa000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5efc103500 0x7f5efc195ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35792/0 (socket says 172.21.15.119:35792) 2024-05-27T09:32:20.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.711+0000 7f5efa000700 1 -- 172.21.15.119:0/1341523320 learned_addr learned my addr 172.21.15.119:0/1341523320 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:32:20.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.712+0000 7f5efaa00700 1 --2- 172.21.15.119:0/1341523320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5efc100be0 0x7f5efc1959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:20.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.712+0000 7f5efa000700 1 -- 172.21.15.119:0/1341523320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5efc100be0 msgr2=0x7f5efc1959a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:20.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.712+0000 7f5efa000700 1 --2- 172.21.15.119:0/1341523320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5efc100be0 0x7f5efc1959a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:20.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.712+0000 7f5efa000700 1 -- 172.21.15.119:0/1341523320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec0095f0 con 0x7f5efc103500 2024-05-27T09:32:20.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.712+0000 7f5efa000700 1 --2- 172.21.15.119:0/1341523320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5efc103500 0x7f5efc195ee0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800b4b0 tx=0x7f5ee800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:20.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.712+0000 7f5ef3400700 1 -- 172.21.15.119:0/1341523320 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ee8011800 con 0x7f5efc103500 2024-05-27T09:32:20.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.712+0000 7f5ef3400700 1 -- 172.21.15.119:0/1341523320 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5ee8011e10 con 0x7f5efc103500 2024-05-27T09:32:20.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.712+0000 7f5f01a00700 1 -- 172.21.15.119:0/1341523320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5efc19b0f0 con 0x7f5efc103500 2024-05-27T09:32:20.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.713+0000 7f5f01a00700 1 -- 172.21.15.119:0/1341523320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5efc19b6c0 con 0x7f5efc103500 2024-05-27T09:32:20.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.713+0000 7f5ef3400700 1 -- 172.21.15.119:0/1341523320 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ee800f2e0 con 0x7f5efc103500 2024-05-27T09:32:20.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.714+0000 7f5f01a00700 1 -- 172.21.15.119:0/1341523320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5edc005320 con 0x7f5efc103500 2024-05-27T09:32:20.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.715+0000 7f5ef3400700 1 -- 172.21.15.119:0/1341523320 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7f5ee8011960 con 0x7f5efc103500 2024-05-27T09:32:20.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.717+0000 7f5ef3400700 1 --2- 172.21.15.119:0/1341523320 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f5ee00786c0 0x7f5ee007ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:20.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.717+0000 7f5ef3400700 1 -- 172.21.15.119:0/1341523320 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5ee809a400 con 0x7f5efc103500 2024-05-27T09:32:20.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.717+0000 7f5efaa00700 1 --2- 172.21.15.119:0/1341523320 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f5ee00786c0 0x7f5ee007ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:20.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.718+0000 7f5efaa00700 1 --2- 172.21.15.119:0/1341523320 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f5ee00786c0 0x7f5ee007ab70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5eec006010 tx=0x7f5eec00b580 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:20.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.722+0000 7f5ef3400700 1 -- 172.21.15.119:0/1341523320 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f5ee8062460 con 0x7f5efc103500 2024-05-27T09:32:20.812 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:20 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:20.812 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:20 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:20.812 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:20 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:32:20.812 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:20 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:32:20.812 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:20 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:20.812 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:20 smithi119 ceph-mon[29082]: pgmap v59: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:20.812 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:20 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-27T09:32:20.813 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:20 smithi119 ceph-mon[29082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-27T09:32:20.813 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:20 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:32:20.813 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:20 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:20.813 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:20 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:20.813 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:20 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:20.813 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:20 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:20.813 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:20 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:20.813 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:20 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:20.813 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:20 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:20.813 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:20 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:20.813 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:20 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:20.813 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:20 smithi119 ceph-mon[29082]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:32:20.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:20 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:20.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:20 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:20.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:20 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:32:20.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:20 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:32:20.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:20 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:20.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:20 smithi138 ceph-mon[34885]: pgmap v59: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:20.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:20 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-27T09:32:20.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:20 smithi138 ceph-mon[34885]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-27T09:32:20.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:20 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:32:20.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:20 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:20.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:20 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:20.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:20 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:20.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:20 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:20.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:20 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:20.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:20 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:20.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:20 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:20.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:20 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:20.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:20 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:20.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:20 smithi138 ceph-mon[34885]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:32:20.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.974+0000 7f5f01a00700 1 -- 172.21.15.119:0/1341523320 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5edc000bf0 con 0x7f5ee00786c0 2024-05-27T09:32:20.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.989+0000 7f5ef3400700 1 -- 172.21.15.119:0/1341523320 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5edc000bf0 con 0x7f5ee00786c0 2024-05-27T09:32:20.991 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:32:20.991 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (11m) 8s ago 13m 14.2M - 0.25.0 c8568f914cd2 bc280230ab28 2024-05-27T09:32:20.991 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (13m) 8s ago 13m 7986k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:32:20.991 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (12m) 32s ago 12m 7834k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:32:20.991 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (13m) 8s ago 13m 6665k - 18.2.0 dc2bc1663786 e5fafde44183 2024-05-27T09:32:20.991 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (12m) 32s ago 12m 6660k - 18.2.0 dc2bc1663786 46956ab99bab 2024-05-27T09:32:20.991 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (11m) 8s ago 13m 73.0M - 9.4.7 954c08fa6188 16eeaded4295 2024-05-27T09:32:20.991 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (6m) 8s ago 6m 15.1M - 18.2.0 dc2bc1663786 93b21713fb1e 2024-05-27T09:32:20.991 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (6m) 8s ago 6m 130M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:32:20.991 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (6m) 32s ago 6m 15.0M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:32:20.991 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (6m) 32s ago 6m 128M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:32:20.991 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (2m) 8s ago 15m 585M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:32:20.991 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (66s) 32s ago 12m 529M - 19.0.0-3868-ga6507388 850a06a84d28 2fc3a80696fa 2024-05-27T09:32:20.992 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (15m) 8s ago 15m 58.7M 2048M 18.2.0 dc2bc1663786 ba998b789a1c 2024-05-27T09:32:20.992 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (12m) 32s ago 12m 47.8M 2048M 18.2.0 dc2bc1663786 60216fd67078 2024-05-27T09:32:20.992 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (46s) 8s ago 13m 7470k - 1.5.0 0da6a335fe13 709c37815cff 2024-05-27T09:32:20.992 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (40s) 32s ago 12m 2889k - 1.5.0 0da6a335fe13 999047a1a5bf 2024-05-27T09:32:20.992 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (11m) 8s ago 11m 224M 1216M 18.2.0 dc2bc1663786 825bd1392508 2024-05-27T09:32:20.992 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (10m) 8s ago 10m 251M 1216M 18.2.0 dc2bc1663786 fce9819903d9 2024-05-27T09:32:20.992 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (9m) 8s ago 9m 190M 1216M 18.2.0 dc2bc1663786 3955a6112c1c 2024-05-27T09:32:20.992 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (9m) 32s ago 9m 268M 2240M 18.2.0 dc2bc1663786 693c92220622 2024-05-27T09:32:20.992 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (8m) 32s ago 8m 246M 2240M 18.2.0 dc2bc1663786 7559e658e917 2024-05-27T09:32:20.992 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (8m) 32s ago 8m 229M 2240M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:32:20.992 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (15s) 8s ago 12m 44.9M - 2.43.0 a07b618ecd1d cba50454ed21 2024-05-27T09:32:20.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.992+0000 7f5f01a00700 1 -- 172.21.15.119:0/1341523320 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f5ee00786c0 msgr2=0x7f5ee007ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:20.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.992+0000 7f5f01a00700 1 --2- 172.21.15.119:0/1341523320 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f5ee00786c0 0x7f5ee007ab70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5eec006010 tx=0x7f5eec00b580 comp rx=0 tx=0).stop 2024-05-27T09:32:20.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.992+0000 7f5f01a00700 1 -- 172.21.15.119:0/1341523320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5efc103500 msgr2=0x7f5efc195ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:20.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.992+0000 7f5f01a00700 1 --2- 172.21.15.119:0/1341523320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5efc103500 0x7f5efc195ee0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800b4b0 tx=0x7f5ee800b870 comp rx=0 tx=0).stop 2024-05-27T09:32:20.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.993+0000 7f5f01a00700 1 -- 172.21.15.119:0/1341523320 shutdown_connections 2024-05-27T09:32:20.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.993+0000 7f5f01a00700 1 --2- 172.21.15.119:0/1341523320 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f5ee00786c0 0x7f5ee007ab70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:20.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.993+0000 7f5f01a00700 1 --2- 172.21.15.119:0/1341523320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5efc100be0 0x7f5efc1959a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:20.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.993+0000 7f5f01a00700 1 --2- 172.21.15.119:0/1341523320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5efc103500 0x7f5efc195ee0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:20.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.993+0000 7f5f01a00700 1 -- 172.21.15.119:0/1341523320 >> 172.21.15.119:0/1341523320 conn(0x7f5efc0fa5e0 msgr2=0x7f5efc0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:20.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.993+0000 7f5f01a00700 1 -- 172.21.15.119:0/1341523320 shutdown_connections 2024-05-27T09:32:20.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:20.993+0000 7f5f01a00700 1 -- 172.21.15.119:0/1341523320 wait complete. 2024-05-27T09:32:21.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.143+0000 7f6e2f800700 1 -- 172.21.15.119:0/169135799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e28103500 msgr2=0x7f6e281058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:21.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.143+0000 7f6e2f800700 1 --2- 172.21.15.119:0/169135799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e28103500 0x7f6e281058e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6e14009910 tx=0x7f6e14009c20 comp rx=0 tx=0).stop 2024-05-27T09:32:21.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.143+0000 7f6e2f800700 1 -- 172.21.15.119:0/169135799 shutdown_connections 2024-05-27T09:32:21.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.143+0000 7f6e2f800700 1 --2- 172.21.15.119:0/169135799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e28103500 0x7f6e281058e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.143+0000 7f6e2f800700 1 --2- 172.21.15.119:0/169135799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e28100be0 0x7f6e28102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.143+0000 7f6e2f800700 1 -- 172.21.15.119:0/169135799 >> 172.21.15.119:0/169135799 conn(0x7f6e280fa5e0 msgr2=0x7f6e280fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:21.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.144+0000 7f6e2f800700 1 -- 172.21.15.119:0/169135799 shutdown_connections 2024-05-27T09:32:21.145 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.144+0000 7f6e2f800700 1 -- 172.21.15.119:0/169135799 wait complete. 2024-05-27T09:32:21.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.145+0000 7f6e2f800700 1 Processor -- start 2024-05-27T09:32:21.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.145+0000 7f6e2f800700 1 -- start start 2024-05-27T09:32:21.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.145+0000 7f6e2f800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e28100be0 0x7f6e28195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:21.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.145+0000 7f6e2f800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e28103500 0x7f6e28195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:21.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.145+0000 7f6e2f800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e28196490 con 0x7f6e28103500 2024-05-27T09:32:21.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.145+0000 7f6e2f800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e281965d0 con 0x7f6e28100be0 2024-05-27T09:32:21.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.145+0000 7f6e27e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e28103500 0x7f6e28195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:21.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.145+0000 7f6e27e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e28103500 0x7f6e28195e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35810/0 (socket says 172.21.15.119:35810) 2024-05-27T09:32:21.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.145+0000 7f6e27e00700 1 -- 172.21.15.119:0/2357210082 learned_addr learned my addr 172.21.15.119:0/2357210082 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:32:21.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.146+0000 7f6e2ce00700 1 --2- 172.21.15.119:0/2357210082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e28100be0 0x7f6e28195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:21.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.146+0000 7f6e27e00700 1 -- 172.21.15.119:0/2357210082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e28100be0 msgr2=0x7f6e28195930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:21.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.146+0000 7f6e27e00700 1 --2- 172.21.15.119:0/2357210082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e28100be0 0x7f6e28195930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.147 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.146+0000 7f6e27e00700 1 -- 172.21.15.119:0/2357210082 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e140095f0 con 0x7f6e28103500 2024-05-27T09:32:21.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.146+0000 7f6e27e00700 1 --2- 172.21.15.119:0/2357210082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e28103500 0x7f6e28195e70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6e14005b40 tx=0x7f6e1400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:21.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.147+0000 7f6e25600700 1 -- 172.21.15.119:0/2357210082 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e1401d070 con 0x7f6e28103500 2024-05-27T09:32:21.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.147+0000 7f6e25600700 1 -- 172.21.15.119:0/2357210082 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6e14022470 con 0x7f6e28103500 2024-05-27T09:32:21.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.147+0000 7f6e25600700 1 -- 172.21.15.119:0/2357210082 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e1400f690 con 0x7f6e28103500 2024-05-27T09:32:21.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.147+0000 7f6e2f800700 1 -- 172.21.15.119:0/2357210082 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e2819b020 con 0x7f6e28103500 2024-05-27T09:32:21.148 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.147+0000 7f6e2f800700 1 -- 172.21.15.119:0/2357210082 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e2819b510 con 0x7f6e28103500 2024-05-27T09:32:21.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.148+0000 7f6e2f800700 1 -- 172.21.15.119:0/2357210082 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e2818fb70 con 0x7f6e28103500 2024-05-27T09:32:21.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.149+0000 7f6e25600700 1 -- 172.21.15.119:0/2357210082 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7f6e140225e0 con 0x7f6e28103500 2024-05-27T09:32:21.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.150+0000 7f6e25600700 1 --2- 172.21.15.119:0/2357210082 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f6e100786c0 0x7f6e1007ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:21.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.150+0000 7f6e25600700 1 -- 172.21.15.119:0/2357210082 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6e1409bfd0 con 0x7f6e28103500 2024-05-27T09:32:21.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.151+0000 7f6e2ce00700 1 --2- 172.21.15.119:0/2357210082 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f6e100786c0 0x7f6e1007ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:21.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.151+0000 7f6e2ce00700 1 --2- 172.21.15.119:0/2357210082 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f6e100786c0 0x7f6e1007ab70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c005fd0 tx=0x7f6e1c00a560 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:21.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.155+0000 7f6e25600700 1 -- 172.21.15.119:0/2357210082 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f6e140d1a30 con 0x7f6e28103500 2024-05-27T09:32:21.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.491+0000 7f6e2f800700 1 -- 172.21.15.119:0/2357210082 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6e2802cc30 con 0x7f6e28103500 2024-05-27T09:32:21.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.492+0000 7f6e25600700 1 -- 172.21.15.119:0/2357210082 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f6e14064130 con 0x7f6e28103500 2024-05-27T09:32:21.494 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:32:21.494 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:32:21.494 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-05-27T09:32:21.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:32:21.494 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:32:21.494 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:32:21.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:32:21.494 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:32:21.494 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-05-27T09:32:21.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:32:21.494 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:32:21.494 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-05-27T09:32:21.495 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:32:21.495 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:32:21.495 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-05-27T09:32:21.495 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:32:21.495 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:32:21.495 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:32:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.496+0000 7f6e2f800700 1 -- 172.21.15.119:0/2357210082 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f6e100786c0 msgr2=0x7f6e1007ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.496+0000 7f6e2f800700 1 --2- 172.21.15.119:0/2357210082 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f6e100786c0 0x7f6e1007ab70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c005fd0 tx=0x7f6e1c00a560 comp rx=0 tx=0).stop 2024-05-27T09:32:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.496+0000 7f6e2f800700 1 -- 172.21.15.119:0/2357210082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e28103500 msgr2=0x7f6e28195e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.496+0000 7f6e2f800700 1 --2- 172.21.15.119:0/2357210082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e28103500 0x7f6e28195e70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6e14005b40 tx=0x7f6e1400be20 comp rx=0 tx=0).stop 2024-05-27T09:32:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.496+0000 7f6e2f800700 1 -- 172.21.15.119:0/2357210082 shutdown_connections 2024-05-27T09:32:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.496+0000 7f6e2f800700 1 --2- 172.21.15.119:0/2357210082 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f6e100786c0 0x7f6e1007ab70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.496+0000 7f6e2f800700 1 --2- 172.21.15.119:0/2357210082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e28100be0 0x7f6e28195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.496+0000 7f6e2f800700 1 --2- 172.21.15.119:0/2357210082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e28103500 0x7f6e28195e70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.496+0000 7f6e2f800700 1 -- 172.21.15.119:0/2357210082 >> 172.21.15.119:0/2357210082 conn(0x7f6e280fa5e0 msgr2=0x7f6e280fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.496+0000 7f6e2f800700 1 -- 172.21.15.119:0/2357210082 shutdown_connections 2024-05-27T09:32:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.496+0000 7f6e2f800700 1 -- 172.21.15.119:0/2357210082 wait complete. 2024-05-27T09:32:21.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.654+0000 7f68bc600700 1 -- 172.21.15.119:0/2476272030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b40737b0 msgr2=0x7f68b4073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:21.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.654+0000 7f68bc600700 1 --2- 172.21.15.119:0/2476272030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b40737b0 0x7f68b4073c20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009940 tx=0x7f68a0009c50 comp rx=0 tx=0).stop 2024-05-27T09:32:21.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.654+0000 7f68bc600700 1 -- 172.21.15.119:0/2476272030 shutdown_connections 2024-05-27T09:32:21.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.654+0000 7f68bc600700 1 --2- 172.21.15.119:0/2476272030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b40737b0 0x7f68b4073c20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.654+0000 7f68bc600700 1 --2- 172.21.15.119:0/2476272030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68b4074d80 0x7f68b40731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.654+0000 7f68bc600700 1 -- 172.21.15.119:0/2476272030 >> 172.21.15.119:0/2476272030 conn(0x7f68b40fb6e0 msgr2=0x7f68b40fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:21.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.654+0000 7f68bc600700 1 -- 172.21.15.119:0/2476272030 shutdown_connections 2024-05-27T09:32:21.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.655+0000 7f68bc600700 1 -- 172.21.15.119:0/2476272030 wait complete. 2024-05-27T09:32:21.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.655+0000 7f68bc600700 1 Processor -- start 2024-05-27T09:32:21.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.656+0000 7f68bc600700 1 -- start start 2024-05-27T09:32:21.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.656+0000 7f68bc600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68b40737b0 0x7f68b4197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:21.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.656+0000 7f68bc600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b4074d80 0x7f68b4198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:21.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.656+0000 7f68bc600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b4198690 con 0x7f68b4074d80 2024-05-27T09:32:21.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.656+0000 7f68bc600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b41987d0 con 0x7f68b40737b0 2024-05-27T09:32:21.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.656+0000 7f68b9200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b4074d80 0x7f68b4198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:21.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.656+0000 7f68b9200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b4074d80 0x7f68b4198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35828/0 (socket says 172.21.15.119:35828) 2024-05-27T09:32:21.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.656+0000 7f68b9200700 1 -- 172.21.15.119:0/3729361238 learned_addr learned my addr 172.21.15.119:0/3729361238 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:32:21.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.657+0000 7f68b9c00700 1 --2- 172.21.15.119:0/3729361238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68b40737b0 0x7f68b4197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:21.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.657+0000 7f68b9200700 1 -- 172.21.15.119:0/3729361238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68b40737b0 msgr2=0x7f68b4197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:21.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.657+0000 7f68b9200700 1 --2- 172.21.15.119:0/3729361238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68b40737b0 0x7f68b4197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.657+0000 7f68b9200700 1 -- 172.21.15.119:0/3729361238 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a00095f0 con 0x7f68b4074d80 2024-05-27T09:32:21.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.657+0000 7f68b9c00700 1 --2- 172.21.15.119:0/3729361238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68b40737b0 0x7f68b4197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:32:21.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.657+0000 7f68b9200700 1 --2- 172.21.15.119:0/3729361238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b4074d80 0x7f68b4198070 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f68a00055f0 tx=0x7f68a00056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:21.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.658+0000 7f68b2000700 1 -- 172.21.15.119:0/3729361238 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68a001d070 con 0x7f68b4074d80 2024-05-27T09:32:21.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.658+0000 7f68b2000700 1 -- 172.21.15.119:0/3729361238 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f68a000f460 con 0x7f68b4074d80 2024-05-27T09:32:21.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.658+0000 7f68bc600700 1 -- 172.21.15.119:0/3729361238 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b419d220 con 0x7f68b4074d80 2024-05-27T09:32:21.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.658+0000 7f68b2000700 1 -- 172.21.15.119:0/3729361238 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68a0021640 con 0x7f68b4074d80 2024-05-27T09:32:21.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.658+0000 7f68bc600700 1 -- 172.21.15.119:0/3729361238 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b419d790 con 0x7f68b4074d80 2024-05-27T09:32:21.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.659+0000 7f68bc600700 1 -- 172.21.15.119:0/3729361238 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b410aeb0 con 0x7f68b4074d80 2024-05-27T09:32:21.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.660+0000 7f68b2000700 1 -- 172.21.15.119:0/3729361238 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7f68a000fad0 con 0x7f68b4074d80 2024-05-27T09:32:21.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.661+0000 7f68b2000700 1 --2- 172.21.15.119:0/3729361238 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f689c0786c0 0x7f689c07ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:21.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.661+0000 7f68b2000700 1 -- 172.21.15.119:0/3729361238 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f68a009bb80 con 0x7f68b4074d80 2024-05-27T09:32:21.662 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.662+0000 7f68b9c00700 1 --2- 172.21.15.119:0/3729361238 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f689c0786c0 0x7f689c07ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:21.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.663+0000 7f68b9c00700 1 --2- 172.21.15.119:0/3729361238 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f689c0786c0 0x7f689c07ab70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f68a8005fd0 tx=0x7f68a8005ee0 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:21.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.669+0000 7f68b2000700 1 -- 172.21.15.119:0/3729361238 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f68a0063be0 con 0x7f68b4074d80 2024-05-27T09:32:21.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.924+0000 7f68bc600700 1 -- 172.21.15.119:0/3729361238 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68b4107670 con 0x7f689c0786c0 2024-05-27T09:32:21.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.927+0000 7f68b2000700 1 -- 172.21.15.119:0/3729361238 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f68b4107670 con 0x7f689c0786c0 2024-05-27T09:32:21.928 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:32:21.928 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:32:21.928 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:32:21.928 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-27T09:32:21.928 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [ 2024-05-27T09:32:21.928 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr" 2024-05-27T09:32:21.928 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-05-27T09:32:21.928 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "2/2 daemons upgraded", 2024-05-27T09:32:21.928 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "Currently upgrading alertmanager daemons", 2024-05-27T09:32:21.928 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:32:21.928 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:32:21.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.930+0000 7f68bc600700 1 -- 172.21.15.119:0/3729361238 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f689c0786c0 msgr2=0x7f689c07ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:21.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.930+0000 7f68bc600700 1 --2- 172.21.15.119:0/3729361238 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f689c0786c0 0x7f689c07ab70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f68a8005fd0 tx=0x7f68a8005ee0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.930+0000 7f68bc600700 1 -- 172.21.15.119:0/3729361238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b4074d80 msgr2=0x7f68b4198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:21.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.930+0000 7f68bc600700 1 --2- 172.21.15.119:0/3729361238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b4074d80 0x7f68b4198070 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f68a00055f0 tx=0x7f68a00056d0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.930+0000 7f68bc600700 1 -- 172.21.15.119:0/3729361238 shutdown_connections 2024-05-27T09:32:21.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.930+0000 7f68bc600700 1 --2- 172.21.15.119:0/3729361238 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f689c0786c0 0x7f689c07ab70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.930+0000 7f68bc600700 1 --2- 172.21.15.119:0/3729361238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68b40737b0 0x7f68b4197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.930+0000 7f68bc600700 1 --2- 172.21.15.119:0/3729361238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68b4074d80 0x7f68b4198070 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:21.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.930+0000 7f68bc600700 1 -- 172.21.15.119:0/3729361238 >> 172.21.15.119:0/3729361238 conn(0x7f68b40fb6e0 msgr2=0x7f68b4105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:21.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.930+0000 7f68bc600700 1 -- 172.21.15.119:0/3729361238 shutdown_connections 2024-05-27T09:32:21.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:21.930+0000 7f68bc600700 1 -- 172.21.15.119:0/3729361238 wait complete. 2024-05-27T09:32:22.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:22 smithi119 ceph-mon[29082]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:32:22.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:22 smithi119 ceph-mon[29082]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:32:22.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:22 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/2357210082' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:22.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:22 smithi138 ceph-mon[34885]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:32:22.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:22 smithi138 ceph-mon[34885]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:32:22.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:22 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/2357210082' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:23.450 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:23 smithi119 ceph-mon[29082]: pgmap v60: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:23.451 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:23 smithi119 ceph-mon[29082]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:32:23.451 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:23 smithi119 ceph-mon[29082]: Upgrade: Updating alertmanager.smithi119 2024-05-27T09:32:23.451 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:23 smithi119 ceph-mon[29082]: Deploying daemon alertmanager.smithi119 on smithi119 2024-05-27T09:32:23.538 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:23 smithi138 ceph-mon[34885]: pgmap v60: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:23.538 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:23 smithi138 ceph-mon[34885]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:32:23.538 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:23 smithi138 ceph-mon[34885]: Upgrade: Updating alertmanager.smithi119 2024-05-27T09:32:23.538 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:23 smithi138 ceph-mon[34885]: Deploying daemon alertmanager.smithi119 on smithi119 2024-05-27T09:32:25.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:25 smithi119 ceph-mon[29082]: pgmap v61: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:25.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:25 smithi138 ceph-mon[34885]: pgmap v61: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:27.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:27 smithi119 ceph-mon[29082]: pgmap v62: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:27.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:27 smithi138 ceph-mon[34885]: pgmap v62: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:28.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:32:28.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:32:29.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:29 smithi119 ceph-mon[29082]: pgmap v63: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:29.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:29 smithi138 ceph-mon[34885]: pgmap v63: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:31.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:31 smithi138 ceph-mon[34885]: pgmap v64: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:31.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:31 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:31.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:31 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:31.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:31 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:32:31.398 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:31 smithi119 ceph-mon[29082]: pgmap v64: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:31.398 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:31 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:31.398 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:31 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:31.398 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:31 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:32:33.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:33 smithi119 ceph-mon[29082]: pgmap v65: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:33.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:33 smithi138 ceph-mon[34885]: pgmap v65: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:35.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:35 smithi119 ceph-mon[29082]: pgmap v66: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:35.569 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:35 smithi138 ceph-mon[34885]: pgmap v66: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:37 smithi119 ceph-mon[29082]: pgmap v67: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-05-27T09:32:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:37 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:37 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:37.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:37 smithi138 ceph-mon[34885]: pgmap v67: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-05-27T09:32:37.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:37 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:37.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:37 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:39.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:39 smithi119 ceph-mon[29082]: pgmap v68: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-05-27T09:32:39.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:39 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:39.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:39 smithi138 ceph-mon[34885]: pgmap v68: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-05-27T09:32:39.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:39 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:40.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:40 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:40.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:40 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:41.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:41 smithi119 ceph-mon[29082]: pgmap v69: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-05-27T09:32:41.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:41 smithi138 ceph-mon[34885]: pgmap v69: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-05-27T09:32:43.453 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:43 smithi119 ceph-mon[29082]: pgmap v70: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-05-27T09:32:43.453 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:43 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:32:43.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:43 smithi138 ceph-mon[34885]: pgmap v70: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-05-27T09:32:43.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:43 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:32:44.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:44 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:44.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:44 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:44.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:44 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:32:44.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:44 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:32:44.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:44 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:44.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:44 smithi138 ceph-mon[34885]: pgmap v71: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-05-27T09:32:44.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:44 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-27T09:32:44.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:44 smithi138 ceph-mon[34885]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-27T09:32:44.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:44 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:32:44.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:44 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:44.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:44 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:44.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:44 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:44.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:44 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:44.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:44 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:44.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:44 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:44.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:44 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:44.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:44 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:44.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:44 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:44.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:44 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:44.911 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:44 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:44.911 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:44 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:44.911 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:44 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:32:44.911 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:44 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:32:44.911 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:44 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:32:44.911 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:44 smithi119 ceph-mon[29082]: pgmap v71: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-05-27T09:32:44.911 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:44 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-27T09:32:44.911 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:44 smithi119 ceph-mon[29082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-27T09:32:44.912 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:44 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:32:44.912 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:44 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:44.912 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:44 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:44.912 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:44 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:44.912 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:44 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:44.912 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:44 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:44.912 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:44 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:44.912 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:44 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:44.912 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:44 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:44.912 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:44 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:44.912 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:44 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:47.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:47 smithi119 ceph-mon[29082]: pgmap v72: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-05-27T09:32:47.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:47 smithi119 ceph-mon[29082]: Upgrade: Updating grafana.smithi119 2024-05-27T09:32:47.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:47 smithi119 ceph-mon[29082]: Deploying daemon grafana.smithi119 on smithi119 2024-05-27T09:32:47.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:47 smithi138 ceph-mon[34885]: pgmap v72: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-05-27T09:32:47.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:47 smithi138 ceph-mon[34885]: Upgrade: Updating grafana.smithi119 2024-05-27T09:32:47.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:47 smithi138 ceph-mon[34885]: Deploying daemon grafana.smithi119 on smithi119 2024-05-27T09:32:49.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:49 smithi119 ceph-mon[29082]: pgmap v73: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:49.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:49 smithi138 ceph-mon[34885]: pgmap v73: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:51.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:51 smithi119 ceph-mon[29082]: pgmap v74: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:51.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:51 smithi138 ceph-mon[34885]: pgmap v74: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:52.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.098+0000 7f410f800700 1 -- 172.21.15.119:0/2686866086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4108100190 msgr2=0x7f41081005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:52.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.098+0000 7f410f800700 1 --2- 172.21.15.119:0/2686866086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4108100190 0x7f41081005a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f40f8009910 tx=0x7f40f8009c20 comp rx=0 tx=0).stop 2024-05-27T09:32:52.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.099+0000 7f410f800700 1 -- 172.21.15.119:0/2686866086 shutdown_connections 2024-05-27T09:32:52.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.099+0000 7f410f800700 1 --2- 172.21.15.119:0/2686866086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4108101390 0x7f41081017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.099+0000 7f410f800700 1 --2- 172.21.15.119:0/2686866086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4108100190 0x7f41081005a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.099+0000 7f410f800700 1 -- 172.21.15.119:0/2686866086 >> 172.21.15.119:0/2686866086 conn(0x7f41080fb720 msgr2=0x7f41080fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:52.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.099+0000 7f410f800700 1 -- 172.21.15.119:0/2686866086 shutdown_connections 2024-05-27T09:32:52.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.099+0000 7f410f800700 1 -- 172.21.15.119:0/2686866086 wait complete. 2024-05-27T09:32:52.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.100+0000 7f410f800700 1 Processor -- start 2024-05-27T09:32:52.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.100+0000 7f410f800700 1 -- start start 2024-05-27T09:32:52.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.101+0000 7f410f800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4108100190 0x7f4108197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:52.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.101+0000 7f410f800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4108101390 0x7f41081980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:52.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.101+0000 7f410f800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4108198680 con 0x7f4108101390 2024-05-27T09:32:52.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.101+0000 7f410f800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41081987c0 con 0x7f4108100190 2024-05-27T09:32:52.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.101+0000 7f4107e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4108101390 0x7f41081980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:52.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.101+0000 7f4107e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4108101390 0x7f41081980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35960/0 (socket says 172.21.15.119:35960) 2024-05-27T09:32:52.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.101+0000 7f4107e00700 1 -- 172.21.15.119:0/2781591156 learned_addr learned my addr 172.21.15.119:0/2781591156 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:32:52.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.101+0000 7f410ce00700 1 --2- 172.21.15.119:0/2781591156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4108100190 0x7f4108197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:52.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.102+0000 7f4107e00700 1 -- 172.21.15.119:0/2781591156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4108100190 msgr2=0x7f4108197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:52.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.102+0000 7f4107e00700 1 --2- 172.21.15.119:0/2781591156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4108100190 0x7f4108197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.102+0000 7f4107e00700 1 -- 172.21.15.119:0/2781591156 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f80095f0 con 0x7f4108101390 2024-05-27T09:32:52.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.102+0000 7f4107e00700 1 --2- 172.21.15.119:0/2781591156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4108101390 0x7f41081980f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f40f400c930 tx=0x7f40f400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:52.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.102+0000 7f4105600700 1 -- 172.21.15.119:0/2781591156 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40f4007ad0 con 0x7f4108101390 2024-05-27T09:32:52.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.102+0000 7f4105600700 1 -- 172.21.15.119:0/2781591156 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f40f400ce80 con 0x7f4108101390 2024-05-27T09:32:52.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.103+0000 7f4105600700 1 -- 172.21.15.119:0/2781591156 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40f4018630 con 0x7f4108101390 2024-05-27T09:32:52.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.103+0000 7f410f800700 1 -- 172.21.15.119:0/2781591156 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f410819d280 con 0x7f4108101390 2024-05-27T09:32:52.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.103+0000 7f410f800700 1 -- 172.21.15.119:0/2781591156 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410819d7a0 con 0x7f4108101390 2024-05-27T09:32:52.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.104+0000 7f410f800700 1 -- 172.21.15.119:0/2781591156 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4108105470 con 0x7f4108101390 2024-05-27T09:32:52.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.105+0000 7f4105600700 1 -- 172.21.15.119:0/2781591156 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7f40f400f450 con 0x7f4108101390 2024-05-27T09:32:52.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.106+0000 7f4105600700 1 --2- 172.21.15.119:0/2781591156 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f40ec0786c0 0x7f40ec07ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:52.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.107+0000 7f4105600700 1 -- 172.21.15.119:0/2781591156 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f40f409a6a0 con 0x7f4108101390 2024-05-27T09:32:52.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.107+0000 7f410ce00700 1 --2- 172.21.15.119:0/2781591156 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f40ec0786c0 0x7f40ec07ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:52.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.108+0000 7f410ce00700 1 --2- 172.21.15.119:0/2781591156 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f40ec0786c0 0x7f40ec07ab70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f40f8006010 tx=0x7f40f800b560 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:52.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.112+0000 7f4105600700 1 -- 172.21.15.119:0/2781591156 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f40f4062820 con 0x7f4108101390 2024-05-27T09:32:52.421 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.419+0000 7f410f800700 1 -- 172.21.15.119:0/2781591156 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4108061190 con 0x7f40ec0786c0 2024-05-27T09:32:52.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.421+0000 7f4105600700 1 -- 172.21.15.119:0/2781591156 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f4108061190 con 0x7f40ec0786c0 2024-05-27T09:32:52.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.425+0000 7f410f800700 1 -- 172.21.15.119:0/2781591156 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f40ec0786c0 msgr2=0x7f40ec07ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:52.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.425+0000 7f410f800700 1 --2- 172.21.15.119:0/2781591156 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f40ec0786c0 0x7f40ec07ab70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f40f8006010 tx=0x7f40f800b560 comp rx=0 tx=0).stop 2024-05-27T09:32:52.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.425+0000 7f410f800700 1 -- 172.21.15.119:0/2781591156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4108101390 msgr2=0x7f41081980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:52.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.425+0000 7f410f800700 1 --2- 172.21.15.119:0/2781591156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4108101390 0x7f41081980f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f40f400c930 tx=0x7f40f400cc40 comp rx=0 tx=0).stop 2024-05-27T09:32:52.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.425+0000 7f410f800700 1 -- 172.21.15.119:0/2781591156 shutdown_connections 2024-05-27T09:32:52.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.425+0000 7f410f800700 1 --2- 172.21.15.119:0/2781591156 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f40ec0786c0 0x7f40ec07ab70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.425+0000 7f410f800700 1 --2- 172.21.15.119:0/2781591156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4108100190 0x7f4108197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.425+0000 7f410f800700 1 --2- 172.21.15.119:0/2781591156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4108101390 0x7f41081980f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.425+0000 7f410f800700 1 -- 172.21.15.119:0/2781591156 >> 172.21.15.119:0/2781591156 conn(0x7f41080fb720 msgr2=0x7f41081025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:52.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.425+0000 7f410f800700 1 -- 172.21.15.119:0/2781591156 shutdown_connections 2024-05-27T09:32:52.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.425+0000 7f410f800700 1 -- 172.21.15.119:0/2781591156 wait complete. 2024-05-27T09:32:52.443 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:32:52.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.608+0000 7fa3f6600700 1 -- 172.21.15.119:0/3774507626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3f0101350 msgr2=0x7fa3f01017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:52.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.608+0000 7fa3f6600700 1 --2- 172.21.15.119:0/3774507626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3f0101350 0x7fa3f01017a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc009910 tx=0x7fa3dc009c20 comp rx=0 tx=0).stop 2024-05-27T09:32:52.610 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.608+0000 7fa3f6600700 1 -- 172.21.15.119:0/3774507626 shutdown_connections 2024-05-27T09:32:52.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.608+0000 7fa3f6600700 1 --2- 172.21.15.119:0/3774507626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3f0101350 0x7fa3f01017a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.608+0000 7fa3f6600700 1 --2- 172.21.15.119:0/3774507626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3f0100150 0x7fa3f0100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.608+0000 7fa3f6600700 1 -- 172.21.15.119:0/3774507626 >> 172.21.15.119:0/3774507626 conn(0x7fa3f00fb6e0 msgr2=0x7fa3f00fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:52.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.608+0000 7fa3f6600700 1 -- 172.21.15.119:0/3774507626 shutdown_connections 2024-05-27T09:32:52.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.608+0000 7fa3f6600700 1 -- 172.21.15.119:0/3774507626 wait complete. 2024-05-27T09:32:52.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.609+0000 7fa3f6600700 1 Processor -- start 2024-05-27T09:32:52.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.609+0000 7fa3f6600700 1 -- start start 2024-05-27T09:32:52.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.610+0000 7fa3f6600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3f0100150 0x7fa3f0195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:52.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.610+0000 7fa3f6600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3f0101350 0x7fa3f0195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:52.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.610+0000 7fa3f6600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f01964c0 con 0x7fa3f0101350 2024-05-27T09:32:52.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.610+0000 7fa3f6600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f0196600 con 0x7fa3f0100150 2024-05-27T09:32:52.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.610+0000 7fa3eea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3f0101350 0x7fa3f0195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:52.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.610+0000 7fa3eea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3f0101350 0x7fa3f0195ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35988/0 (socket says 172.21.15.119:35988) 2024-05-27T09:32:52.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.610+0000 7fa3eea00700 1 -- 172.21.15.119:0/3397539723 learned_addr learned my addr 172.21.15.119:0/3397539723 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:32:52.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.610+0000 7fa3ef400700 1 --2- 172.21.15.119:0/3397539723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3f0100150 0x7fa3f0195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:52.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.610+0000 7fa3eea00700 1 -- 172.21.15.119:0/3397539723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3f0100150 msgr2=0x7fa3f0195960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:52.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.610+0000 7fa3eea00700 1 --2- 172.21.15.119:0/3397539723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3f0100150 0x7fa3f0195960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.610+0000 7fa3eea00700 1 -- 172.21.15.119:0/3397539723 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc0095f0 con 0x7fa3f0101350 2024-05-27T09:32:52.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.611+0000 7fa3eea00700 1 --2- 172.21.15.119:0/3397539723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3f0101350 0x7fa3f0195ea0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc005b40 tx=0x7fa3dc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:52.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.611+0000 7fa3e7e00700 1 -- 172.21.15.119:0/3397539723 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3dc01d070 con 0x7fa3f0101350 2024-05-27T09:32:52.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.611+0000 7fa3e7e00700 1 -- 172.21.15.119:0/3397539723 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa3dc022470 con 0x7fa3f0101350 2024-05-27T09:32:52.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.611+0000 7fa3e7e00700 1 -- 172.21.15.119:0/3397539723 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3dc00f690 con 0x7fa3f0101350 2024-05-27T09:32:52.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.611+0000 7fa3f6600700 1 -- 172.21.15.119:0/3397539723 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f019b050 con 0x7fa3f0101350 2024-05-27T09:32:52.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.611+0000 7fa3f6600700 1 -- 172.21.15.119:0/3397539723 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f019b540 con 0x7fa3f0101350 2024-05-27T09:32:52.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.613+0000 7fa3f6600700 1 -- 172.21.15.119:0/3397539723 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d0005320 con 0x7fa3f0101350 2024-05-27T09:32:52.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.614+0000 7fa3e7e00700 1 -- 172.21.15.119:0/3397539723 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7fa3dc004b70 con 0x7fa3f0101350 2024-05-27T09:32:52.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.616+0000 7fa3e7e00700 1 --2- 172.21.15.119:0/3397539723 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fa3d40786c0 0x7fa3d407ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:52.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.616+0000 7fa3e7e00700 1 -- 172.21.15.119:0/3397539723 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa3dc09bf60 con 0x7fa3f0101350 2024-05-27T09:32:52.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.616+0000 7fa3ef400700 1 --2- 172.21.15.119:0/3397539723 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fa3d40786c0 0x7fa3d407ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:52.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.617+0000 7fa3ef400700 1 --2- 172.21.15.119:0/3397539723 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fa3d40786c0 0x7fa3d407ab70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0005fd0 tx=0x7fa3e0005dd0 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:52.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.621+0000 7fa3e7e00700 1 -- 172.21.15.119:0/3397539723 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7fa3dc0640c0 con 0x7fa3f0101350 2024-05-27T09:32:52.944 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.942+0000 7fa3f6600700 1 -- 172.21.15.119:0/3397539723 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3d0000bf0 con 0x7fa3d40786c0 2024-05-27T09:32:52.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.944+0000 7fa3e7e00700 1 -- 172.21.15.119:0/3397539723 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fa3d0000bf0 con 0x7fa3d40786c0 2024-05-27T09:32:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.947+0000 7fa3f6600700 1 -- 172.21.15.119:0/3397539723 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fa3d40786c0 msgr2=0x7fa3d407ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.947+0000 7fa3f6600700 1 --2- 172.21.15.119:0/3397539723 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fa3d40786c0 0x7fa3d407ab70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0005fd0 tx=0x7fa3e0005dd0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.947+0000 7fa3f6600700 1 -- 172.21.15.119:0/3397539723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3f0101350 msgr2=0x7fa3f0195ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.947+0000 7fa3f6600700 1 --2- 172.21.15.119:0/3397539723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3f0101350 0x7fa3f0195ea0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc005b40 tx=0x7fa3dc00be20 comp rx=0 tx=0).stop 2024-05-27T09:32:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.947+0000 7fa3f6600700 1 -- 172.21.15.119:0/3397539723 shutdown_connections 2024-05-27T09:32:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.947+0000 7fa3f6600700 1 --2- 172.21.15.119:0/3397539723 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fa3d40786c0 0x7fa3d407ab70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.947+0000 7fa3f6600700 1 --2- 172.21.15.119:0/3397539723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3f0100150 0x7fa3f0195960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.947+0000 7fa3f6600700 1 --2- 172.21.15.119:0/3397539723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3f0101350 0x7fa3f0195ea0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:52.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.947+0000 7fa3f6600700 1 -- 172.21.15.119:0/3397539723 >> 172.21.15.119:0/3397539723 conn(0x7fa3f00fb6e0 msgr2=0x7fa3f0104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:52.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.947+0000 7fa3f6600700 1 -- 172.21.15.119:0/3397539723 shutdown_connections 2024-05-27T09:32:52.950 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:52.947+0000 7fa3f6600700 1 -- 172.21.15.119:0/3397539723 wait complete. 2024-05-27T09:32:53.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.114+0000 7f3fa6000700 1 -- 172.21.15.119:0/3918823561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fa0103560 msgr2=0x7f3fa0105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:53.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.114+0000 7f3fa6000700 1 --2- 172.21.15.119:0/3918823561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fa0103560 0x7f3fa0105940 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009910 tx=0x7f3f8c009c20 comp rx=0 tx=0).stop 2024-05-27T09:32:53.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.114+0000 7f3fa6000700 1 -- 172.21.15.119:0/3918823561 shutdown_connections 2024-05-27T09:32:53.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.114+0000 7f3fa6000700 1 --2- 172.21.15.119:0/3918823561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fa0103560 0x7f3fa0105940 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:53.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.114+0000 7f3fa6000700 1 --2- 172.21.15.119:0/3918823561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fa0100c40 0x7f3fa0103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:53.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.114+0000 7f3fa6000700 1 -- 172.21.15.119:0/3918823561 >> 172.21.15.119:0/3918823561 conn(0x7f3fa00fa5e0 msgr2=0x7f3fa00fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:53.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.115+0000 7f3fa6000700 1 -- 172.21.15.119:0/3918823561 shutdown_connections 2024-05-27T09:32:53.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.115+0000 7f3fa6000700 1 -- 172.21.15.119:0/3918823561 wait complete. 2024-05-27T09:32:53.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.116+0000 7f3fa6000700 1 Processor -- start 2024-05-27T09:32:53.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.116+0000 7f3fa6000700 1 -- start start 2024-05-27T09:32:53.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.116+0000 7f3fa6000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fa0100c40 0x7f3fa019bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:53.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.116+0000 7f3fa6000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fa0103560 0x7f3fa019c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:53.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.117+0000 7f3fa6000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa019cab0 con 0x7f3fa0100c40 2024-05-27T09:32:53.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.117+0000 7f3fa6000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa019cbf0 con 0x7f3fa0103560 2024-05-27T09:32:53.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.117+0000 7f3f9f400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fa0100c40 0x7f3fa019bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:53.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.117+0000 7f3f9f400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fa0100c40 0x7f3fa019bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36006/0 (socket says 172.21.15.119:36006) 2024-05-27T09:32:53.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.117+0000 7f3f9f400700 1 -- 172.21.15.119:0/2936960025 learned_addr learned my addr 172.21.15.119:0/2936960025 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:32:53.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.117+0000 7f3f9f400700 1 -- 172.21.15.119:0/2936960025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fa0103560 msgr2=0x7f3fa019c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:53.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.117+0000 7f3f9f400700 1 --2- 172.21.15.119:0/2936960025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fa0103560 0x7f3fa019c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:53.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.117+0000 7f3f9f400700 1 -- 172.21.15.119:0/2936960025 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8c0095f0 con 0x7f3fa0100c40 2024-05-27T09:32:53.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.118+0000 7f3f9f400700 1 --2- 172.21.15.119:0/2936960025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fa0100c40 0x7f3fa019bf50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000d900 tx=0x7f3f9000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:53.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.118+0000 7f3f97e00700 1 -- 172.21.15.119:0/2936960025 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f90004a00 con 0x7f3fa0100c40 2024-05-27T09:32:53.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.119+0000 7f3f97e00700 1 -- 172.21.15.119:0/2936960025 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3f900054d0 con 0x7f3fa0100c40 2024-05-27T09:32:53.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.119+0000 7f3f97e00700 1 -- 172.21.15.119:0/2936960025 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f90009d60 con 0x7f3fa0100c40 2024-05-27T09:32:53.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.119+0000 7f3fa6000700 1 -- 172.21.15.119:0/2936960025 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa01a16a0 con 0x7f3fa0100c40 2024-05-27T09:32:53.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.119+0000 7f3fa6000700 1 -- 172.21.15.119:0/2936960025 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa01a1bf0 con 0x7f3fa0100c40 2024-05-27T09:32:53.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.120+0000 7f3fa6000700 1 -- 172.21.15.119:0/2936960025 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa00fc1c0 con 0x7f3fa0100c40 2024-05-27T09:32:53.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.122+0000 7f3f97e00700 1 -- 172.21.15.119:0/2936960025 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7f3f90005020 con 0x7f3fa0100c40 2024-05-27T09:32:53.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.123+0000 7f3f97e00700 1 --2- 172.21.15.119:0/2936960025 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f3f840786c0 0x7f3f8407ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:53.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.123+0000 7f3f97e00700 1 -- 172.21.15.119:0/2936960025 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3f9009a760 con 0x7f3fa0100c40 2024-05-27T09:32:53.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.123+0000 7f3f9ea00700 1 --2- 172.21.15.119:0/2936960025 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f3f840786c0 0x7f3f8407ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:53.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.124+0000 7f3f9ea00700 1 --2- 172.21.15.119:0/2936960025 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f3f840786c0 0x7f3f8407ab70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c005090 tx=0x7f3f8c005fb0 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:53.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.128+0000 7f3f97e00700 1 -- 172.21.15.119:0/2936960025 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f3f90062840 con 0x7f3fa0100c40 2024-05-27T09:32:53.422 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.420+0000 7f3fa6000700 1 -- 172.21.15.119:0/2936960025 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fa0061190 con 0x7f3f840786c0 2024-05-27T09:32:53.422 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:53 smithi119 ceph-mon[29082]: pgmap v75: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:53.422 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:53 smithi119 ceph-mon[29082]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:32:53.439 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:32:53.439 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (24s) 16s ago 13m 13.8M - 0.25.0 c8568f914cd2 6b90564dcef7 2024-05-27T09:32:53.439 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (14m) 16s ago 14m 8036k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:32:53.439 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (12m) 64s ago 12m 7834k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:32:53.439 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (14m) 16s ago 14m 6665k - 18.2.0 dc2bc1663786 e5fafde44183 2024-05-27T09:32:53.439 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (12m) 64s ago 12m 6660k - 18.2.0 dc2bc1663786 46956ab99bab 2024-05-27T09:32:53.439 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (11m) 16s ago 13m 72.9M - 9.4.7 954c08fa6188 16eeaded4295 2024-05-27T09:32:53.439 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (6m) 16s ago 6m 15.3M - 18.2.0 dc2bc1663786 93b21713fb1e 2024-05-27T09:32:53.440 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (6m) 16s ago 6m 130M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:32:53.440 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (6m) 64s ago 6m 15.0M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:32:53.440 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (6m) 64s ago 6m 128M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:32:53.440 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (2m) 16s ago 15m 586M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:32:53.440 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (99s) 64s ago 12m 529M - 19.0.0-3868-ga6507388 850a06a84d28 2fc3a80696fa 2024-05-27T09:32:53.440 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (15m) 16s ago 15m 58.7M 2048M 18.2.0 dc2bc1663786 ba998b789a1c 2024-05-27T09:32:53.440 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (12m) 64s ago 12m 47.8M 2048M 18.2.0 dc2bc1663786 60216fd67078 2024-05-27T09:32:53.440 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (79s) 16s ago 14m 7772k - 1.5.0 0da6a335fe13 709c37815cff 2024-05-27T09:32:53.440 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (72s) 64s ago 12m 2889k - 1.5.0 0da6a335fe13 999047a1a5bf 2024-05-27T09:32:53.440 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (11m) 16s ago 11m 224M 1216M 18.2.0 dc2bc1663786 825bd1392508 2024-05-27T09:32:53.440 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (11m) 16s ago 11m 250M 1216M 18.2.0 dc2bc1663786 fce9819903d9 2024-05-27T09:32:53.440 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (10m) 16s ago 10m 189M 1216M 18.2.0 dc2bc1663786 3955a6112c1c 2024-05-27T09:32:53.440 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (9m) 64s ago 9m 268M 2240M 18.2.0 dc2bc1663786 693c92220622 2024-05-27T09:32:53.440 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (9m) 64s ago 9m 246M 2240M 18.2.0 dc2bc1663786 7559e658e917 2024-05-27T09:32:53.440 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (8m) 64s ago 8m 229M 2240M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:32:53.441 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (48s) 16s ago 13m 51.4M - 2.43.0 a07b618ecd1d cba50454ed21 2024-05-27T09:32:53.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.436+0000 7f3f97e00700 1 -- 172.21.15.119:0/2936960025 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f3fa0061190 con 0x7f3f840786c0 2024-05-27T09:32:53.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.439+0000 7f3fa6000700 1 -- 172.21.15.119:0/2936960025 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f3f840786c0 msgr2=0x7f3f8407ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:53.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.439+0000 7f3fa6000700 1 --2- 172.21.15.119:0/2936960025 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f3f840786c0 0x7f3f8407ab70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c005090 tx=0x7f3f8c005fb0 comp rx=0 tx=0).stop 2024-05-27T09:32:53.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.439+0000 7f3fa6000700 1 -- 172.21.15.119:0/2936960025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fa0100c40 msgr2=0x7f3fa019bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:53.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.439+0000 7f3fa6000700 1 --2- 172.21.15.119:0/2936960025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fa0100c40 0x7f3fa019bf50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000d900 tx=0x7f3f9000dcc0 comp rx=0 tx=0).stop 2024-05-27T09:32:53.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.440+0000 7f3fa6000700 1 -- 172.21.15.119:0/2936960025 shutdown_connections 2024-05-27T09:32:53.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.440+0000 7f3fa6000700 1 --2- 172.21.15.119:0/2936960025 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f3f840786c0 0x7f3f8407ab70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:53.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.440+0000 7f3fa6000700 1 --2- 172.21.15.119:0/2936960025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fa0100c40 0x7f3fa019bf50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:53.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.440+0000 7f3fa6000700 1 --2- 172.21.15.119:0/2936960025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fa0103560 0x7f3fa019c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:53.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.440+0000 7f3fa6000700 1 -- 172.21.15.119:0/2936960025 >> 172.21.15.119:0/2936960025 conn(0x7f3fa00fa5e0 msgr2=0x7f3fa00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:53.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.440+0000 7f3fa6000700 1 -- 172.21.15.119:0/2936960025 shutdown_connections 2024-05-27T09:32:53.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.440+0000 7f3fa6000700 1 -- 172.21.15.119:0/2936960025 wait complete. 2024-05-27T09:32:53.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.621+0000 7f174d400700 1 -- 172.21.15.119:0/896569724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17480ff050 msgr2=0x7f17480ff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:53.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.621+0000 7f174d400700 1 --2- 172.21.15.119:0/896569724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17480ff050 0x7f17480ff460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1738009910 tx=0x7f1738009c20 comp rx=0 tx=0).stop 2024-05-27T09:32:53.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.622+0000 7f174d400700 1 -- 172.21.15.119:0/896569724 shutdown_connections 2024-05-27T09:32:53.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.622+0000 7f174d400700 1 --2- 172.21.15.119:0/896569724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1748100250 0x7f17481006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:53.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.622+0000 7f174d400700 1 --2- 172.21.15.119:0/896569724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17480ff050 0x7f17480ff460 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:53.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.622+0000 7f174d400700 1 -- 172.21.15.119:0/896569724 >> 172.21.15.119:0/896569724 conn(0x7f17480fa5e0 msgr2=0x7f17480fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:53.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.622+0000 7f174d400700 1 -- 172.21.15.119:0/896569724 shutdown_connections 2024-05-27T09:32:53.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.622+0000 7f174d400700 1 -- 172.21.15.119:0/896569724 wait complete. 2024-05-27T09:32:53.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.623+0000 7f174d400700 1 Processor -- start 2024-05-27T09:32:53.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.623+0000 7f174d400700 1 -- start start 2024-05-27T09:32:53.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.623+0000 7f174d400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17480ff050 0x7f1748197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:53.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.623+0000 7f174d400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1748100250 0x7f17481980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:53.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.623+0000 7f174d400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17481986f0 con 0x7f1748100250 2024-05-27T09:32:53.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.623+0000 7f174d400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1748198830 con 0x7f17480ff050 2024-05-27T09:32:53.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.624+0000 7f1745e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1748100250 0x7f17481980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:53.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.624+0000 7f1745e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1748100250 0x7f17481980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36022/0 (socket says 172.21.15.119:36022) 2024-05-27T09:32:53.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.624+0000 7f1745e00700 1 -- 172.21.15.119:0/3569378473 learned_addr learned my addr 172.21.15.119:0/3569378473 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:32:53.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.624+0000 7f1746800700 1 --2- 172.21.15.119:0/3569378473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17480ff050 0x7f1748197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:53.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.624+0000 7f1745e00700 1 -- 172.21.15.119:0/3569378473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17480ff050 msgr2=0x7f1748197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:53.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.624+0000 7f1745e00700 1 --2- 172.21.15.119:0/3569378473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17480ff050 0x7f1748197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:53.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.624+0000 7f1745e00700 1 -- 172.21.15.119:0/3569378473 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17380095f0 con 0x7f1748100250 2024-05-27T09:32:53.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.625+0000 7f1745e00700 1 --2- 172.21.15.119:0/3569378473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1748100250 0x7f17481980d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f173400d8d0 tx=0x7f173400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:53.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.625+0000 7f173f400700 1 -- 172.21.15.119:0/3569378473 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1734009940 con 0x7f1748100250 2024-05-27T09:32:53.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.625+0000 7f173f400700 1 -- 172.21.15.119:0/3569378473 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1734010460 con 0x7f1748100250 2024-05-27T09:32:53.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.625+0000 7f174d400700 1 -- 172.21.15.119:0/3569378473 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174819d2e0 con 0x7f1748100250 2024-05-27T09:32:53.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.625+0000 7f173f400700 1 -- 172.21.15.119:0/3569378473 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f173400f5f0 con 0x7f1748100250 2024-05-27T09:32:53.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.625+0000 7f174d400700 1 -- 172.21.15.119:0/3569378473 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174819d830 con 0x7f1748100250 2024-05-27T09:32:53.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.627+0000 7f174d400700 1 -- 172.21.15.119:0/3569378473 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1748107ba0 con 0x7f1748100250 2024-05-27T09:32:53.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.628+0000 7f173f400700 1 -- 172.21.15.119:0/3569378473 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7f17340105d0 con 0x7f1748100250 2024-05-27T09:32:53.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.629+0000 7f173f400700 1 --2- 172.21.15.119:0/3569378473 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f172c0786c0 0x7f172c07ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:53.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.630+0000 7f173f400700 1 -- 172.21.15.119:0/3569378473 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f173409a860 con 0x7f1748100250 2024-05-27T09:32:53.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.630+0000 7f1746800700 1 --2- 172.21.15.119:0/3569378473 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f172c0786c0 0x7f172c07ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:53.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.631+0000 7f1746800700 1 --2- 172.21.15.119:0/3569378473 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f172c0786c0 0x7f172c07ab70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1738006010 tx=0x7f173800b540 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:53.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:53 smithi138 ceph-mon[34885]: pgmap v75: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:53.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:53 smithi138 ceph-mon[34885]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:32:53.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:53.635+0000 7f173f400700 1 -- 172.21.15.119:0/3569378473 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f17340628c0 con 0x7f1748100250 2024-05-27T09:32:54.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.110+0000 7f174d400700 1 -- 172.21.15.119:0/3569378473 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1748107d60 con 0x7f1748100250 2024-05-27T09:32:54.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.111+0000 7f173f400700 1 -- 172.21.15.119:0/3569378473 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f1734062260 con 0x7f1748100250 2024-05-27T09:32:54.113 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:32:54.113 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:32:54.114 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-05-27T09:32:54.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:32:54.114 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:32:54.114 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:32:54.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:32:54.114 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:32:54.114 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-05-27T09:32:54.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:32:54.114 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:32:54.114 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-05-27T09:32:54.114 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:32:54.114 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:32:54.115 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-05-27T09:32:54.115 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:32:54.115 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:32:54.115 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:32:54.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.115+0000 7f174d400700 1 -- 172.21.15.119:0/3569378473 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f172c0786c0 msgr2=0x7f172c07ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:54.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.115+0000 7f174d400700 1 --2- 172.21.15.119:0/3569378473 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f172c0786c0 0x7f172c07ab70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1738006010 tx=0x7f173800b540 comp rx=0 tx=0).stop 2024-05-27T09:32:54.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.115+0000 7f174d400700 1 -- 172.21.15.119:0/3569378473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1748100250 msgr2=0x7f17481980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:54.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.115+0000 7f174d400700 1 --2- 172.21.15.119:0/3569378473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1748100250 0x7f17481980d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f173400d8d0 tx=0x7f173400dc90 comp rx=0 tx=0).stop 2024-05-27T09:32:54.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.115+0000 7f174d400700 1 -- 172.21.15.119:0/3569378473 shutdown_connections 2024-05-27T09:32:54.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.115+0000 7f174d400700 1 --2- 172.21.15.119:0/3569378473 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f172c0786c0 0x7f172c07ab70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:54.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.115+0000 7f174d400700 1 --2- 172.21.15.119:0/3569378473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17480ff050 0x7f1748197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:54.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.115+0000 7f174d400700 1 --2- 172.21.15.119:0/3569378473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1748100250 0x7f17481980d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:54.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.115+0000 7f174d400700 1 -- 172.21.15.119:0/3569378473 >> 172.21.15.119:0/3569378473 conn(0x7f17480fa5e0 msgr2=0x7f17480683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:54.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.115+0000 7f174d400700 1 -- 172.21.15.119:0/3569378473 shutdown_connections 2024-05-27T09:32:54.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.115+0000 7f174d400700 1 -- 172.21.15.119:0/3569378473 wait complete. 2024-05-27T09:32:54.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.322+0000 7f1158000700 1 -- 172.21.15.119:0/1858315778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1150073750 msgr2=0x7f1150073bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:54.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.322+0000 7f1158000700 1 --2- 172.21.15.119:0/1858315778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1150073750 0x7f1150073bc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f113c009910 tx=0x7f113c009c20 comp rx=0 tx=0).stop 2024-05-27T09:32:54.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.322+0000 7f1158000700 1 -- 172.21.15.119:0/1858315778 shutdown_connections 2024-05-27T09:32:54.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.322+0000 7f1158000700 1 --2- 172.21.15.119:0/1858315778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1150073750 0x7f1150073bc0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:54.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.322+0000 7f1158000700 1 --2- 172.21.15.119:0/1858315778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1150074d20 0x7f1150073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:54.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.322+0000 7f1158000700 1 -- 172.21.15.119:0/1858315778 >> 172.21.15.119:0/1858315778 conn(0x7f11500fb6e0 msgr2=0x7f11500fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:54.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.323+0000 7f1158000700 1 -- 172.21.15.119:0/1858315778 shutdown_connections 2024-05-27T09:32:54.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.323+0000 7f1158000700 1 -- 172.21.15.119:0/1858315778 wait complete. 2024-05-27T09:32:54.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.324+0000 7f1158000700 1 Processor -- start 2024-05-27T09:32:54.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.324+0000 7f1158000700 1 -- start start 2024-05-27T09:32:54.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.324+0000 7f1158000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1150073750 0x7f1150197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:54.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.324+0000 7f1158000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1150074d20 0x7f11501980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:54.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.324+0000 7f1158000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11501986c0 con 0x7f1150074d20 2024-05-27T09:32:54.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.324+0000 7f1158000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1150198800 con 0x7f1150073750 2024-05-27T09:32:54.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.325+0000 7f1154c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1150074d20 0x7f11501980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:54.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.325+0000 7f1154c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1150074d20 0x7f11501980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36036/0 (socket says 172.21.15.119:36036) 2024-05-27T09:32:54.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.325+0000 7f1154c00700 1 -- 172.21.15.119:0/1330904329 learned_addr learned my addr 172.21.15.119:0/1330904329 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:32:54.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.325+0000 7f1155600700 1 --2- 172.21.15.119:0/1330904329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1150073750 0x7f1150197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:54.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.325+0000 7f1154c00700 1 -- 172.21.15.119:0/1330904329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1150073750 msgr2=0x7f1150197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:54.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.325+0000 7f1154c00700 1 --2- 172.21.15.119:0/1330904329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1150073750 0x7f1150197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:54.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.325+0000 7f1154c00700 1 -- 172.21.15.119:0/1330904329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113c0095f0 con 0x7f1150074d20 2024-05-27T09:32:54.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.326+0000 7f1154c00700 1 --2- 172.21.15.119:0/1330904329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1150074d20 0x7f11501980a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f113c005b40 tx=0x7f113c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:54.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.326+0000 7f114e000700 1 -- 172.21.15.119:0/1330904329 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f113c01d070 con 0x7f1150074d20 2024-05-27T09:32:54.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.326+0000 7f114e000700 1 -- 172.21.15.119:0/1330904329 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f113c00f460 con 0x7f1150074d20 2024-05-27T09:32:54.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.326+0000 7f114e000700 1 -- 172.21.15.119:0/1330904329 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f113c021640 con 0x7f1150074d20 2024-05-27T09:32:54.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.326+0000 7f1158000700 1 -- 172.21.15.119:0/1330904329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115019d250 con 0x7f1150074d20 2024-05-27T09:32:54.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.326+0000 7f1158000700 1 -- 172.21.15.119:0/1330904329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115019d740 con 0x7f1150074d20 2024-05-27T09:32:54.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.327+0000 7f1158000700 1 -- 172.21.15.119:0/1330904329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f115010aeb0 con 0x7f1150074d20 2024-05-27T09:32:54.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.329+0000 7f114e000700 1 -- 172.21.15.119:0/1330904329 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7f113c00fa90 con 0x7f1150074d20 2024-05-27T09:32:54.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.330+0000 7f114e000700 1 --2- 172.21.15.119:0/1330904329 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f11380786c0 0x7f113807ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:32:54.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.331+0000 7f114e000700 1 -- 172.21.15.119:0/1330904329 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f113c09bf40 con 0x7f1150074d20 2024-05-27T09:32:54.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.331+0000 7f1155600700 1 --2- 172.21.15.119:0/1330904329 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f11380786c0 0x7f113807ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:32:54.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.332+0000 7f1155600700 1 --2- 172.21.15.119:0/1330904329 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f11380786c0 0x7f113807ab70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1144005950 tx=0x7f11440058e0 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:32:54.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.338+0000 7f114e000700 1 -- 172.21.15.119:0/1330904329 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f113c0640a0 con 0x7f1150074d20 2024-05-27T09:32:54.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:54 smithi119 ceph-mon[29082]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:32:54.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:54 smithi119 ceph-mon[29082]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:32:54.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:54 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/3569378473' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:54.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:54 smithi138 ceph-mon[34885]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:32:54.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:54 smithi138 ceph-mon[34885]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:32:54.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:54 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/3569378473' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:32:54.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.647+0000 7f1158000700 1 -- 172.21.15.119:0/1330904329 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f11501076d0 con 0x7f11380786c0 2024-05-27T09:32:54.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.649+0000 7f114e000700 1 -- 172.21.15.119:0/1330904329 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f11501076d0 con 0x7f11380786c0 2024-05-27T09:32:54.650 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:32:54.650 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:32:54.650 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:32:54.650 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-05-27T09:32:54.650 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [ 2024-05-27T09:32:54.650 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr" 2024-05-27T09:32:54.650 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-05-27T09:32:54.650 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "2/2 daemons upgraded", 2024-05-27T09:32:54.651 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "Currently upgrading grafana daemons", 2024-05-27T09:32:54.651 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:32:54.651 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:32:54.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.652+0000 7f1158000700 1 -- 172.21.15.119:0/1330904329 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f11380786c0 msgr2=0x7f113807ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:54.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.652+0000 7f1158000700 1 --2- 172.21.15.119:0/1330904329 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f11380786c0 0x7f113807ab70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1144005950 tx=0x7f11440058e0 comp rx=0 tx=0).stop 2024-05-27T09:32:54.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.652+0000 7f1158000700 1 -- 172.21.15.119:0/1330904329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1150074d20 msgr2=0x7f11501980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:32:54.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.652+0000 7f1158000700 1 --2- 172.21.15.119:0/1330904329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1150074d20 0x7f11501980a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f113c005b40 tx=0x7f113c00be20 comp rx=0 tx=0).stop 2024-05-27T09:32:54.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.652+0000 7f1158000700 1 -- 172.21.15.119:0/1330904329 shutdown_connections 2024-05-27T09:32:54.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.652+0000 7f1158000700 1 --2- 172.21.15.119:0/1330904329 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f11380786c0 0x7f113807ab70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:54.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.652+0000 7f1158000700 1 --2- 172.21.15.119:0/1330904329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1150073750 0x7f1150197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:54.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.652+0000 7f1158000700 1 --2- 172.21.15.119:0/1330904329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1150074d20 0x7f11501980a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:32:54.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.652+0000 7f1158000700 1 -- 172.21.15.119:0/1330904329 >> 172.21.15.119:0/1330904329 conn(0x7f11500fb6e0 msgr2=0x7f1150105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:32:54.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.653+0000 7f1158000700 1 -- 172.21.15.119:0/1330904329 shutdown_connections 2024-05-27T09:32:54.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:32:54.653+0000 7f1158000700 1 -- 172.21.15.119:0/1330904329 wait complete. 2024-05-27T09:32:55.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:55 smithi119 ceph-mon[29082]: pgmap v76: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:55.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:55 smithi138 ceph-mon[34885]: pgmap v76: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:56.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:56 smithi119 ceph-mon[29082]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:32:56.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:56 smithi138 ceph-mon[34885]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:32:57.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:57 smithi119 ceph-mon[29082]: pgmap v77: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:57.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:57 smithi138 ceph-mon[34885]: pgmap v77: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:58.316 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:58 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:32:58.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:58 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:32:59.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:32:59 smithi119 ceph-mon[29082]: pgmap v78: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:32:59.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:32:59 smithi138 ceph-mon[34885]: pgmap v78: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:01.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:01 smithi138 ceph-mon[34885]: pgmap v79: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:01.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:01 smithi119 ceph-mon[29082]: pgmap v79: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:03.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:03 smithi119 ceph-mon[29082]: pgmap v80: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:03.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:03 smithi138 ceph-mon[34885]: pgmap v80: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:05.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:05 smithi119 ceph-mon[29082]: pgmap v81: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:05.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:05 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:05.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:05 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:05.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:05 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:33:05.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:05 smithi138 ceph-mon[34885]: pgmap v81: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:05.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:05 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:05.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:05 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:05.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:05 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:33:07.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:07 smithi119 ceph-mon[29082]: pgmap v82: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:07.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:07 smithi138 ceph-mon[34885]: pgmap v82: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:09.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:09 smithi119 ceph-mon[29082]: pgmap v83: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:09.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:09 smithi138 ceph-mon[34885]: pgmap v83: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:11.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:11 smithi138 ceph-mon[34885]: pgmap v84: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:11.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:11 smithi119 ceph-mon[29082]: pgmap v84: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:12 smithi119 ceph-mon[29082]: pgmap v85: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:12 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:12 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:12 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:33:13.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:12 smithi138 ceph-mon[34885]: pgmap v85: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:13.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:12 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:13.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:12 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:13.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:12 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:33:15.312 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:15 smithi119 ceph-mon[29082]: pgmap v86: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:15.313 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:15 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:15.313 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:15 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:15.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:15 smithi138 ceph-mon[34885]: pgmap v86: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:15.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:15 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:15.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:15 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:17.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:17 smithi119 ceph-mon[29082]: pgmap v87: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:17.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:17 smithi138 ceph-mon[34885]: pgmap v87: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:19.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: pgmap v88: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:19.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:19.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:19.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:33:19.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:33:19.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:19.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-27T09:33:19.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:33:19.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:19.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-27T09:33:19.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-27T09:33:19.751 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:33:19.751 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-27T09:33:19.751 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-27T09:33:19.751 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-27T09:33:19.751 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-27T09:33:19.752 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-27T09:33:19.752 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-27T09:33:19.752 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-27T09:33:19.752 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-27T09:33:19.752 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:33:19.752 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-27T09:33:19.752 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-27T09:33:19.752 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-27T09:33:19.752 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-27T09:33:19.752 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-27T09:33:19.752 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-27T09:33:19.752 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-27T09:33:19.752 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:33:19.753 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:33:19.753 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:33:19.753 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:33:19.753 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:33:19.753 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:33:19.753 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-27T09:33:19.753 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-27T09:33:19.753 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:33:19.753 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:33:19.753 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:33:19.753 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:19.753 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:33:19.753 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:33:19.753 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:33:19.754 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:19 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:19.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: pgmap v88: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:19.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:19.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:19.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:33:19.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:33:19.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:19.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-27T09:33:19.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:33:19.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:19.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:19.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-27T09:33:19.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-27T09:33:19.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:33:19.889 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-27T09:33:19.889 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-27T09:33:19.889 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-27T09:33:19.889 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-27T09:33:19.889 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-27T09:33:19.889 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-27T09:33:19.890 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-27T09:33:19.890 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-27T09:33:19.890 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:33:19.890 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-27T09:33:19.890 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-27T09:33:19.890 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-27T09:33:19.890 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-27T09:33:19.890 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-27T09:33:19.890 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-27T09:33:19.890 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-27T09:33:19.890 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:33:19.890 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:33:19.890 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:33:19.890 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:33:19.891 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:33:19.891 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:33:19.891 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-27T09:33:19.891 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-27T09:33:19.891 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:33:19.891 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:33:19.891 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:33:19.891 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:19.891 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:33:19.891 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:33:19.891 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:33:19.891 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:19 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:20.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:20 smithi119 ceph-mon[29082]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-27T09:33:20.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:20 smithi119 ceph-mon[29082]: Upgrade: Finalizing container_image settings 2024-05-27T09:33:20.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:20 smithi119 ceph-mon[29082]: Upgrade: Complete! 2024-05-27T09:33:20.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:20 smithi138 ceph-mon[34885]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-27T09:33:20.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:20 smithi138 ceph-mon[34885]: Upgrade: Finalizing container_image settings 2024-05-27T09:33:20.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:20 smithi138 ceph-mon[34885]: Upgrade: Complete! 2024-05-27T09:33:21.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:21 smithi119 ceph-mon[29082]: pgmap v89: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:21.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:21 smithi138 ceph-mon[34885]: pgmap v89: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:23.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:23 smithi119 ceph-mon[29082]: pgmap v90: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:23.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:23 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:23.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:23 smithi138 ceph-mon[34885]: pgmap v90: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:23.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:23 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:24.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.824+0000 7fb3a1200700 1 -- 172.21.15.119:0/926388729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb39c0ff050 msgr2=0x7fb39c0ff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:24.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.824+0000 7fb3a1200700 1 --2- 172.21.15.119:0/926388729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb39c0ff050 0x7fb39c0ff460 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb384009910 tx=0x7fb384009c20 comp rx=0 tx=0).stop 2024-05-27T09:33:24.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.825+0000 7fb3a1200700 1 -- 172.21.15.119:0/926388729 shutdown_connections 2024-05-27T09:33:24.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.825+0000 7fb3a1200700 1 --2- 172.21.15.119:0/926388729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb39c100250 0x7fb39c1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:24.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.825+0000 7fb3a1200700 1 --2- 172.21.15.119:0/926388729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb39c0ff050 0x7fb39c0ff460 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:24.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.825+0000 7fb3a1200700 1 -- 172.21.15.119:0/926388729 >> 172.21.15.119:0/926388729 conn(0x7fb39c0fa5e0 msgr2=0x7fb39c0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:24.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.825+0000 7fb3a1200700 1 -- 172.21.15.119:0/926388729 shutdown_connections 2024-05-27T09:33:24.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.825+0000 7fb3a1200700 1 -- 172.21.15.119:0/926388729 wait complete. 2024-05-27T09:33:24.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.826+0000 7fb3a1200700 1 Processor -- start 2024-05-27T09:33:24.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.826+0000 7fb3a1200700 1 -- start start 2024-05-27T09:33:24.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.826+0000 7fb3a1200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb39c0ff050 0x7fb39c197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:24.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.826+0000 7fb3a1200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb39c100250 0x7fb39c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:24.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.826+0000 7fb3a1200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c1987f0 con 0x7fb39c0ff050 2024-05-27T09:33:24.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.826+0000 7fb3a1200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c198960 con 0x7fb39c100250 2024-05-27T09:33:24.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.827+0000 7fb399e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb39c0ff050 0x7fb39c197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:24.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.827+0000 7fb399e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb39c0ff050 0x7fb39c197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40346/0 (socket says 172.21.15.119:40346) 2024-05-27T09:33:24.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.827+0000 7fb399e00700 1 -- 172.21.15.119:0/592417108 learned_addr learned my addr 172.21.15.119:0/592417108 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:33:24.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.827+0000 7fb399e00700 1 -- 172.21.15.119:0/592417108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb39c100250 msgr2=0x7fb39c1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:24.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.827+0000 7fb399400700 1 --2- 172.21.15.119:0/592417108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb39c100250 0x7fb39c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:24.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.827+0000 7fb399e00700 1 --2- 172.21.15.119:0/592417108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb39c100250 0x7fb39c1981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:24.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.827+0000 7fb399e00700 1 -- 172.21.15.119:0/592417108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3840095f0 con 0x7fb39c0ff050 2024-05-27T09:33:24.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.828+0000 7fb399e00700 1 --2- 172.21.15.119:0/592417108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb39c0ff050 0x7fb39c197cb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb38400bd70 tx=0x7fb38400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:24.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.828+0000 7fb392a00700 1 -- 172.21.15.119:0/592417108 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb38401d070 con 0x7fb39c0ff050 2024-05-27T09:33:24.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.828+0000 7fb392a00700 1 -- 172.21.15.119:0/592417108 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb38400f460 con 0x7fb39c0ff050 2024-05-27T09:33:24.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.828+0000 7fb392a00700 1 -- 172.21.15.119:0/592417108 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb384021620 con 0x7fb39c0ff050 2024-05-27T09:33:24.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.828+0000 7fb3a1200700 1 -- 172.21.15.119:0/592417108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb39c19d390 con 0x7fb39c0ff050 2024-05-27T09:33:24.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.828+0000 7fb3a1200700 1 -- 172.21.15.119:0/592417108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39c19d900 con 0x7fb39c0ff050 2024-05-27T09:33:24.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.829+0000 7fb3a1200700 1 -- 172.21.15.119:0/592417108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb39c107c20 con 0x7fb39c0ff050 2024-05-27T09:33:24.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.831+0000 7fb392a00700 1 -- 172.21.15.119:0/592417108 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7fb38400fa70 con 0x7fb39c0ff050 2024-05-27T09:33:24.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.832+0000 7fb392a00700 1 --2- 172.21.15.119:0/592417108 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fb3800786c0 0x7fb38007ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:24.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.833+0000 7fb392a00700 1 -- 172.21.15.119:0/592417108 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb38409bf50 con 0x7fb39c0ff050 2024-05-27T09:33:24.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.833+0000 7fb399400700 1 --2- 172.21.15.119:0/592417108 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fb3800786c0 0x7fb38007ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:24.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.833+0000 7fb399400700 1 --2- 172.21.15.119:0/592417108 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fb3800786c0 0x7fb38007ab70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb38c006fd0 tx=0x7fb38c008040 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:24.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:24.837+0000 7fb392a00700 1 -- 172.21.15.119:0/592417108 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7fb3840640b0 con 0x7fb39c0ff050 2024-05-27T09:33:25.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:25.090+0000 7fb3a1200700 1 -- 172.21.15.119:0/592417108 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb39c068ae0 con 0x7fb3800786c0 2024-05-27T09:33:25.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:25.093+0000 7fb392a00700 1 -- 172.21.15.119:0/592417108 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fb39c068ae0 con 0x7fb3800786c0 2024-05-27T09:33:25.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:25.096+0000 7fb3a1200700 1 -- 172.21.15.119:0/592417108 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fb3800786c0 msgr2=0x7fb38007ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:25.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:25.096+0000 7fb3a1200700 1 --2- 172.21.15.119:0/592417108 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fb3800786c0 0x7fb38007ab70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb38c006fd0 tx=0x7fb38c008040 comp rx=0 tx=0).stop 2024-05-27T09:33:25.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:25.096+0000 7fb3a1200700 1 -- 172.21.15.119:0/592417108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb39c0ff050 msgr2=0x7fb39c197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:25.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:25.096+0000 7fb3a1200700 1 --2- 172.21.15.119:0/592417108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb39c0ff050 0x7fb39c197cb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb38400bd70 tx=0x7fb38400be50 comp rx=0 tx=0).stop 2024-05-27T09:33:25.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:25.096+0000 7fb3a1200700 1 -- 172.21.15.119:0/592417108 shutdown_connections 2024-05-27T09:33:25.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:25.096+0000 7fb3a1200700 1 --2- 172.21.15.119:0/592417108 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fb3800786c0 0x7fb38007ab70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:25.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:25.096+0000 7fb3a1200700 1 --2- 172.21.15.119:0/592417108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb39c0ff050 0x7fb39c197cb0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:25.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:25.096+0000 7fb3a1200700 1 --2- 172.21.15.119:0/592417108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb39c100250 0x7fb39c1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:25.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:25.097+0000 7fb3a1200700 1 -- 172.21.15.119:0/592417108 >> 172.21.15.119:0/592417108 conn(0x7fb39c0fa5e0 msgr2=0x7fb39c0683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:25.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:25.097+0000 7fb3a1200700 1 -- 172.21.15.119:0/592417108 shutdown_connections 2024-05-27T09:33:25.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:25.097+0000 7fb3a1200700 1 -- 172.21.15.119:0/592417108 wait complete. 2024-05-27T09:33:25.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:25 smithi119 ceph-mon[29082]: pgmap v91: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:25.866 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-05-27T09:33:25.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:25 smithi138 ceph-mon[34885]: pgmap v91: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:26.330 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:33:26.712 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:26 smithi119 ceph-mon[29082]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:33:26.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:26 smithi138 ceph-mon[34885]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:33:27.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:27 smithi119 ceph-mon[29082]: pgmap v92: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:27.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:27 smithi138 ceph-mon[34885]: pgmap v92: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:28.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.504+0000 7f4b4ca00700 1 -- 172.21.15.119:0/4049884428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b48106b80 msgr2=0x7f4b48106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:28.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.504+0000 7f4b4ca00700 1 --2- 172.21.15.119:0/4049884428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b48106b80 0x7f4b48106ff0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009910 tx=0x7f4b30009c20 comp rx=0 tx=0).stop 2024-05-27T09:33:28.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.504+0000 7f4b4ca00700 1 -- 172.21.15.119:0/4049884428 shutdown_connections 2024-05-27T09:33:28.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.504+0000 7f4b4ca00700 1 --2- 172.21.15.119:0/4049884428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b48106b80 0x7f4b48106ff0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:28.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.504+0000 7f4b4ca00700 1 --2- 172.21.15.119:0/4049884428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4810b610 0x7f4b4810b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:28.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.504+0000 7f4b4ca00700 1 -- 172.21.15.119:0/4049884428 >> 172.21.15.119:0/4049884428 conn(0x7f4b48075df0 msgr2=0x7f4b48078200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:28.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.505+0000 7f4b4ca00700 1 -- 172.21.15.119:0/4049884428 shutdown_connections 2024-05-27T09:33:28.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.505+0000 7f4b4ca00700 1 -- 172.21.15.119:0/4049884428 wait complete. 2024-05-27T09:33:28.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.506+0000 7f4b4ca00700 1 Processor -- start 2024-05-27T09:33:28.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.506+0000 7f4b4ca00700 1 -- start start 2024-05-27T09:33:28.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.507+0000 7f4b4ca00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b48106b80 0x7f4b4819c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:28.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.507+0000 7f4b4ca00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b4810b610 0x7f4b4819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:28.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.507+0000 7f4b4ca00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4819ce80 con 0x7f4b4810b610 2024-05-27T09:33:28.508 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.507+0000 7f4b4ca00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b481a0c10 con 0x7f4b48106b80 2024-05-27T09:33:28.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.507+0000 7f4b3fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b4810b610 0x7f4b4819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:28.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.507+0000 7f4b3fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b4810b610 0x7f4b4819c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40356/0 (socket says 172.21.15.119:40356) 2024-05-27T09:33:28.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.507+0000 7f4b3fe00700 1 -- 172.21.15.119:0/2204918684 learned_addr learned my addr 172.21.15.119:0/2204918684 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:33:28.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.507+0000 7f4b46800700 1 --2- 172.21.15.119:0/2204918684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b48106b80 0x7f4b4819c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:28.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.507+0000 7f4b3fe00700 1 -- 172.21.15.119:0/2204918684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b48106b80 msgr2=0x7f4b4819c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:28.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.507+0000 7f4b3fe00700 1 --2- 172.21.15.119:0/2204918684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b48106b80 0x7f4b4819c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:28.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.508+0000 7f4b3fe00700 1 -- 172.21.15.119:0/2204918684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b300095f0 con 0x7f4b4810b610 2024-05-27T09:33:28.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.508+0000 7f4b3fe00700 1 --2- 172.21.15.119:0/2204918684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b4810b610 0x7f4b4819c7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000b5c0 tx=0x7f4b3000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:28.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.508+0000 7f4b3f400700 1 -- 172.21.15.119:0/2204918684 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b3001d070 con 0x7f4b4810b610 2024-05-27T09:33:28.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.508+0000 7f4b3f400700 1 -- 172.21.15.119:0/2204918684 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b3000f460 con 0x7f4b4810b610 2024-05-27T09:33:28.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.508+0000 7f4b4ca00700 1 -- 172.21.15.119:0/2204918684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b481a0e90 con 0x7f4b4810b610 2024-05-27T09:33:28.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.508+0000 7f4b3f400700 1 -- 172.21.15.119:0/2204918684 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b30017610 con 0x7f4b4810b610 2024-05-27T09:33:28.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.508+0000 7f4b4ca00700 1 -- 172.21.15.119:0/2204918684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b481a1400 con 0x7f4b4810b610 2024-05-27T09:33:28.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.510+0000 7f4b4ca00700 1 -- 172.21.15.119:0/2204918684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b4810ea00 con 0x7f4b4810b610 2024-05-27T09:33:28.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.511+0000 7f4b3f400700 1 -- 172.21.15.119:0/2204918684 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7f4b3000f5d0 con 0x7f4b4810b610 2024-05-27T09:33:28.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.512+0000 7f4b3f400700 1 --2- 172.21.15.119:0/2204918684 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f4b2c0786c0 0x7f4b2c07ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:28.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.512+0000 7f4b3f400700 1 -- 172.21.15.119:0/2204918684 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4b3009bda0 con 0x7f4b4810b610 2024-05-27T09:33:28.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.512+0000 7f4b46800700 1 --2- 172.21.15.119:0/2204918684 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f4b2c0786c0 0x7f4b2c07ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:28.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.513+0000 7f4b46800700 1 --2- 172.21.15.119:0/2204918684 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f4b2c0786c0 0x7f4b2c07ab70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4b38006fd0 tx=0x7f4b38008040 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:28.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.517+0000 7f4b3f400700 1 -- 172.21.15.119:0/2204918684 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f4b30063e80 con 0x7f4b4810b610 2024-05-27T09:33:28.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:28 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:33:28.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.851+0000 7f4b4ca00700 1 -- 172.21.15.119:0/2204918684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4b4804f2a0 con 0x7f4b4810b610 2024-05-27T09:33:28.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.852+0000 7f4b3f400700 1 -- 172.21.15.119:0/2204918684 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f4b30063820 con 0x7f4b4810b610 2024-05-27T09:33:28.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.854+0000 7f4b4ca00700 1 -- 172.21.15.119:0/2204918684 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f4b2c0786c0 msgr2=0x7f4b2c07ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:28.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.854+0000 7f4b4ca00700 1 --2- 172.21.15.119:0/2204918684 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f4b2c0786c0 0x7f4b2c07ab70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4b38006fd0 tx=0x7f4b38008040 comp rx=0 tx=0).stop 2024-05-27T09:33:28.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.855+0000 7f4b4ca00700 1 -- 172.21.15.119:0/2204918684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b4810b610 msgr2=0x7f4b4819c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:28.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.855+0000 7f4b4ca00700 1 --2- 172.21.15.119:0/2204918684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b4810b610 0x7f4b4819c7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000b5c0 tx=0x7f4b3000be00 comp rx=0 tx=0).stop 2024-05-27T09:33:28.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.855+0000 7f4b4ca00700 1 -- 172.21.15.119:0/2204918684 shutdown_connections 2024-05-27T09:33:28.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.855+0000 7f4b4ca00700 1 --2- 172.21.15.119:0/2204918684 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f4b2c0786c0 0x7f4b2c07ab70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:28.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.855+0000 7f4b4ca00700 1 --2- 172.21.15.119:0/2204918684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b48106b80 0x7f4b4819c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:28.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.855+0000 7f4b4ca00700 1 --2- 172.21.15.119:0/2204918684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b4810b610 0x7f4b4819c7a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:28.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.855+0000 7f4b4ca00700 1 -- 172.21.15.119:0/2204918684 >> 172.21.15.119:0/2204918684 conn(0x7f4b48075df0 msgr2=0x7f4b480777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:28.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.855+0000 7f4b4ca00700 1 -- 172.21.15.119:0/2204918684 shutdown_connections 2024-05-27T09:33:28.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:28.855+0000 7f4b4ca00700 1 -- 172.21.15.119:0/2204918684 wait complete. 2024-05-27T09:33:28.872 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:33:28.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:28 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:33:29.603 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-05-27T09:33:29.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:29 smithi119 ceph-mon[29082]: pgmap v93: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:29.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:29 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/2204918684' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:29.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:29 smithi138 ceph-mon[34885]: pgmap v93: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:29.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:29 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/2204918684' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:30.004 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:33:31.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:31 smithi119 ceph-mon[29082]: pgmap v94: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:31.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:31 smithi138 ceph-mon[34885]: pgmap v94: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:32.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.083+0000 7f372c000700 1 -- 172.21.15.119:0/4263149362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3724073a00 msgr2=0x7f3724110c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:32.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.083+0000 7f372c000700 1 --2- 172.21.15.119:0/4263149362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3724073a00 0x7f3724110c20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3710009910 tx=0x7f3710009c20 comp rx=0 tx=0).stop 2024-05-27T09:33:32.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.083+0000 7f372c000700 1 -- 172.21.15.119:0/4263149362 shutdown_connections 2024-05-27T09:33:32.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.083+0000 7f372c000700 1 --2- 172.21.15.119:0/4263149362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3724073a00 0x7f3724110c20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:32.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.083+0000 7f372c000700 1 --2- 172.21.15.119:0/4263149362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37240730f0 0x7f37240734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:32.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.083+0000 7f372c000700 1 -- 172.21.15.119:0/4263149362 >> 172.21.15.119:0/4263149362 conn(0x7f37240fbc30 msgr2=0x7f37240fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:32.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.083+0000 7f372c000700 1 -- 172.21.15.119:0/4263149362 shutdown_connections 2024-05-27T09:33:32.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.083+0000 7f372c000700 1 -- 172.21.15.119:0/4263149362 wait complete. 2024-05-27T09:33:32.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.085+0000 7f372c000700 1 Processor -- start 2024-05-27T09:33:32.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.085+0000 7f372c000700 1 -- start start 2024-05-27T09:33:32.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.085+0000 7f372c000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37240730f0 0x7f372419c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:32.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.085+0000 7f372c000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3724073a00 0x7f372419c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:32.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.085+0000 7f372c000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372419cf20 con 0x7f3724073a00 2024-05-27T09:33:32.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.085+0000 7f372c000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37241a0cb0 con 0x7f37240730f0 2024-05-27T09:33:32.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.086+0000 7f3728c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3724073a00 0x7f372419c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:32.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.086+0000 7f3728c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3724073a00 0x7f372419c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43648/0 (socket says 172.21.15.119:43648) 2024-05-27T09:33:32.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.086+0000 7f3728c00700 1 -- 172.21.15.119:0/321854523 learned_addr learned my addr 172.21.15.119:0/321854523 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:33:32.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.086+0000 7f3728c00700 1 -- 172.21.15.119:0/321854523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37240730f0 msgr2=0x7f372419c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:32.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.086+0000 7f3729600700 1 --2- 172.21.15.119:0/321854523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37240730f0 0x7f372419c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:32.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.086+0000 7f3728c00700 1 --2- 172.21.15.119:0/321854523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37240730f0 0x7f372419c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:32.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.086+0000 7f3728c00700 1 -- 172.21.15.119:0/321854523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37100095f0 con 0x7f3724073a00 2024-05-27T09:33:32.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.086+0000 7f3729600700 1 --2- 172.21.15.119:0/321854523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37240730f0 0x7f372419c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:33:32.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.087+0000 7f3728c00700 1 --2- 172.21.15.119:0/321854523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3724073a00 0x7f372419c840 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f371000b5c0 tx=0x7f371000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:32.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.087+0000 7f3722000700 1 -- 172.21.15.119:0/321854523 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f371001d070 con 0x7f3724073a00 2024-05-27T09:33:32.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.087+0000 7f372c000700 1 -- 172.21.15.119:0/321854523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37241a0f90 con 0x7f3724073a00 2024-05-27T09:33:32.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.087+0000 7f3722000700 1 -- 172.21.15.119:0/321854523 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f371000f460 con 0x7f3724073a00 2024-05-27T09:33:32.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.087+0000 7f3722000700 1 -- 172.21.15.119:0/321854523 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3710017630 con 0x7f3724073a00 2024-05-27T09:33:32.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.087+0000 7f372c000700 1 -- 172.21.15.119:0/321854523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37241a14e0 con 0x7f3724073a00 2024-05-27T09:33:32.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.088+0000 7f372c000700 1 -- 172.21.15.119:0/321854523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f372410e3a0 con 0x7f3724073a00 2024-05-27T09:33:32.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.090+0000 7f3722000700 1 -- 172.21.15.119:0/321854523 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7f371000f5d0 con 0x7f3724073a00 2024-05-27T09:33:32.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.090+0000 7f3722000700 1 --2- 172.21.15.119:0/321854523 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f370c0786c0 0x7f370c07ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:32.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.091+0000 7f3722000700 1 -- 172.21.15.119:0/321854523 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f371009bd40 con 0x7f3724073a00 2024-05-27T09:33:32.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.091+0000 7f3729600700 1 --2- 172.21.15.119:0/321854523 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f370c0786c0 0x7f370c07ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:32.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.092+0000 7f3729600700 1 --2- 172.21.15.119:0/321854523 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f370c0786c0 0x7f370c07ab70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3718005fd0 tx=0x7f3718005dc0 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:32.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.096+0000 7f3722000700 1 -- 172.21.15.119:0/321854523 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f3710063e40 con 0x7f3724073a00 2024-05-27T09:33:32.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.431+0000 7f372c000700 1 -- 172.21.15.119:0/321854523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f372404ea50 con 0x7f3724073a00 2024-05-27T09:33:32.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.432+0000 7f3722000700 1 -- 172.21.15.119:0/321854523 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f37100637e0 con 0x7f3724073a00 2024-05-27T09:33:32.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.435+0000 7f372c000700 1 -- 172.21.15.119:0/321854523 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f370c0786c0 msgr2=0x7f370c07ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:32.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.435+0000 7f372c000700 1 --2- 172.21.15.119:0/321854523 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f370c0786c0 0x7f370c07ab70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3718005fd0 tx=0x7f3718005dc0 comp rx=0 tx=0).stop 2024-05-27T09:33:32.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.435+0000 7f372c000700 1 -- 172.21.15.119:0/321854523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3724073a00 msgr2=0x7f372419c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:32.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.435+0000 7f372c000700 1 --2- 172.21.15.119:0/321854523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3724073a00 0x7f372419c840 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f371000b5c0 tx=0x7f371000be00 comp rx=0 tx=0).stop 2024-05-27T09:33:32.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.435+0000 7f372c000700 1 -- 172.21.15.119:0/321854523 shutdown_connections 2024-05-27T09:33:32.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.435+0000 7f372c000700 1 --2- 172.21.15.119:0/321854523 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f370c0786c0 0x7f370c07ab70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:32.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.435+0000 7f372c000700 1 --2- 172.21.15.119:0/321854523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37240730f0 0x7f372419c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:32.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.435+0000 7f372c000700 1 --2- 172.21.15.119:0/321854523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3724073a00 0x7f372419c840 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:32.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.435+0000 7f372c000700 1 -- 172.21.15.119:0/321854523 >> 172.21.15.119:0/321854523 conn(0x7f37240fbc30 msgr2=0x7f3724102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:32.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.436+0000 7f372c000700 1 -- 172.21.15.119:0/321854523 shutdown_connections 2024-05-27T09:33:32.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:32.436+0000 7f372c000700 1 -- 172.21.15.119:0/321854523 wait complete. 2024-05-27T09:33:32.454 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)" 2024-05-27T09:33:32.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:32 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/321854523' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:32.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:32 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/321854523' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:33.176 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-05-27T09:33:33.557 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:33:33.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:33 smithi119 ceph-mon[29082]: pgmap v95: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:33.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:33 smithi138 ceph-mon[34885]: pgmap v95: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:34.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:34 smithi119 ceph-mon[29082]: pgmap v96: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:34.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:34 smithi138 ceph-mon[34885]: pgmap v96: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:35.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.717+0000 7f413ce00700 1 -- 172.21.15.119:0/2120795110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4138073a00 msgr2=0x7f4138110a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:35.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.717+0000 7f413ce00700 1 --2- 172.21.15.119:0/2120795110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4138073a00 0x7f4138110a80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4124009910 tx=0x7f4124009c20 comp rx=0 tx=0).stop 2024-05-27T09:33:35.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.717+0000 7f413ce00700 1 -- 172.21.15.119:0/2120795110 shutdown_connections 2024-05-27T09:33:35.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.717+0000 7f413ce00700 1 --2- 172.21.15.119:0/2120795110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4138073a00 0x7f4138110a80 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:35.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.717+0000 7f413ce00700 1 --2- 172.21.15.119:0/2120795110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41380730f0 0x7f41380734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:35.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.717+0000 7f413ce00700 1 -- 172.21.15.119:0/2120795110 >> 172.21.15.119:0/2120795110 conn(0x7f41380fbaf0 msgr2=0x7f41380fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:35.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.717+0000 7f413ce00700 1 -- 172.21.15.119:0/2120795110 shutdown_connections 2024-05-27T09:33:35.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.717+0000 7f413ce00700 1 -- 172.21.15.119:0/2120795110 wait complete. 2024-05-27T09:33:35.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.718+0000 7f413ce00700 1 Processor -- start 2024-05-27T09:33:35.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.718+0000 7f413ce00700 1 -- start start 2024-05-27T09:33:35.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.719+0000 7f413ce00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41380730f0 0x7f413819c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:35.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.719+0000 7f413ce00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4138073a00 0x7f413819c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:35.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.719+0000 7f413ce00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413819ce10 con 0x7f4138073a00 2024-05-27T09:33:35.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.719+0000 7f413ce00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41381a0ba0 con 0x7f41380730f0 2024-05-27T09:33:35.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.719+0000 7f4136a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4138073a00 0x7f413819c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:35.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.719+0000 7f4136a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4138073a00 0x7f413819c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43666/0 (socket says 172.21.15.119:43666) 2024-05-27T09:33:35.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.719+0000 7f4136a00700 1 -- 172.21.15.119:0/3879221048 learned_addr learned my addr 172.21.15.119:0/3879221048 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:33:35.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.719+0000 7f4137400700 1 --2- 172.21.15.119:0/3879221048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41380730f0 0x7f413819c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:35.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.719+0000 7f4136a00700 1 -- 172.21.15.119:0/3879221048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41380730f0 msgr2=0x7f413819c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:35.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.719+0000 7f4136a00700 1 --2- 172.21.15.119:0/3879221048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41380730f0 0x7f413819c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:35.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.719+0000 7f4136a00700 1 -- 172.21.15.119:0/3879221048 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41240095f0 con 0x7f4138073a00 2024-05-27T09:33:35.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.720+0000 7f4136a00700 1 --2- 172.21.15.119:0/3879221048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4138073a00 0x7f413819c730 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f412400bda0 tx=0x7f412400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:35.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.720+0000 7f412fe00700 1 -- 172.21.15.119:0/3879221048 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f412401d070 con 0x7f4138073a00 2024-05-27T09:33:35.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.720+0000 7f412fe00700 1 -- 172.21.15.119:0/3879221048 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f412400f460 con 0x7f4138073a00 2024-05-27T09:33:35.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.720+0000 7f412fe00700 1 -- 172.21.15.119:0/3879221048 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4124017630 con 0x7f4138073a00 2024-05-27T09:33:35.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.720+0000 7f413ce00700 1 -- 172.21.15.119:0/3879221048 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41381a0e80 con 0x7f4138073a00 2024-05-27T09:33:35.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.720+0000 7f413ce00700 1 -- 172.21.15.119:0/3879221048 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41381a13d0 con 0x7f4138073a00 2024-05-27T09:33:35.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.722+0000 7f413ce00700 1 -- 172.21.15.119:0/3879221048 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f413810e200 con 0x7f4138073a00 2024-05-27T09:33:35.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.723+0000 7f412fe00700 1 -- 172.21.15.119:0/3879221048 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7f412400fa60 con 0x7f4138073a00 2024-05-27T09:33:35.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.724+0000 7f412fe00700 1 --2- 172.21.15.119:0/3879221048 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f411c0786c0 0x7f411c07ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:35.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.724+0000 7f412fe00700 1 -- 172.21.15.119:0/3879221048 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f412409bc00 con 0x7f4138073a00 2024-05-27T09:33:35.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.724+0000 7f4137400700 1 --2- 172.21.15.119:0/3879221048 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f411c0786c0 0x7f411c07ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:35.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.725+0000 7f4137400700 1 --2- 172.21.15.119:0/3879221048 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f411c0786c0 0x7f411c07ab70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4128006fd0 tx=0x7f4128008040 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:35.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:35.729+0000 7f412fe00700 1 -- 172.21.15.119:0/3879221048 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f4124063ce0 con 0x7f4138073a00 2024-05-27T09:33:36.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:36.056+0000 7f413ce00700 1 -- 172.21.15.119:0/3879221048 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f41381a1620 con 0x7f4138073a00 2024-05-27T09:33:36.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:36.057+0000 7f412fe00700 1 -- 172.21.15.119:0/3879221048 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f4124063680 con 0x7f4138073a00 2024-05-27T09:33:36.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:36.059+0000 7f413ce00700 1 -- 172.21.15.119:0/3879221048 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f411c0786c0 msgr2=0x7f411c07ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:36.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:36.059+0000 7f413ce00700 1 --2- 172.21.15.119:0/3879221048 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f411c0786c0 0x7f411c07ab70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4128006fd0 tx=0x7f4128008040 comp rx=0 tx=0).stop 2024-05-27T09:33:36.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:36.060+0000 7f413ce00700 1 -- 172.21.15.119:0/3879221048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4138073a00 msgr2=0x7f413819c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:36.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:36.060+0000 7f413ce00700 1 --2- 172.21.15.119:0/3879221048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4138073a00 0x7f413819c730 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f412400bda0 tx=0x7f412400be80 comp rx=0 tx=0).stop 2024-05-27T09:33:36.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:36.060+0000 7f413ce00700 1 -- 172.21.15.119:0/3879221048 shutdown_connections 2024-05-27T09:33:36.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:36.060+0000 7f413ce00700 1 --2- 172.21.15.119:0/3879221048 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f411c0786c0 0x7f411c07ab70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:36.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:36.060+0000 7f413ce00700 1 --2- 172.21.15.119:0/3879221048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41380730f0 0x7f413819c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:36.061 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:36.060+0000 7f413ce00700 1 --2- 172.21.15.119:0/3879221048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4138073a00 0x7f413819c730 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:36.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:36.060+0000 7f413ce00700 1 -- 172.21.15.119:0/3879221048 >> 172.21.15.119:0/3879221048 conn(0x7f41380fbaf0 msgr2=0x7f41381025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:36.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:36.060+0000 7f413ce00700 1 -- 172.21.15.119:0/3879221048 shutdown_connections 2024-05-27T09:33:36.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:36.060+0000 7f413ce00700 1 -- 172.21.15.119:0/3879221048 wait complete. 2024-05-27T09:33:36.075 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:33:36.893 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-05-27T09:33:37.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:36 smithi119 ceph-mon[29082]: pgmap v97: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:37.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:36 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/3879221048' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:37.305 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:33:37.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:36 smithi138 ceph-mon[34885]: pgmap v97: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:37.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:36 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/3879221048' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:33:39.091 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:38 smithi119 ceph-mon[29082]: pgmap v98: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:39.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:38 smithi138 ceph-mon[34885]: pgmap v98: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:39.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.634+0000 7fe57a400700 1 -- 172.21.15.119:0/3035956983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe574102fd0 msgr2=0x7fe574106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:39.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.634+0000 7fe57a400700 1 --2- 172.21.15.119:0/3035956983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe574102fd0 0x7fe574106fa0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe560009910 tx=0x7fe560009c20 comp rx=0 tx=0).stop 2024-05-27T09:33:39.635 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.634+0000 7fe57a400700 1 -- 172.21.15.119:0/3035956983 shutdown_connections 2024-05-27T09:33:39.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.634+0000 7fe57a400700 1 --2- 172.21.15.119:0/3035956983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe574102fd0 0x7fe574106fa0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:39.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.634+0000 7fe57a400700 1 --2- 172.21.15.119:0/3035956983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe574102630 0x7fe574102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:39.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.634+0000 7fe57a400700 1 -- 172.21.15.119:0/3035956983 >> 172.21.15.119:0/3035956983 conn(0x7fe5740fdef0 msgr2=0x7fe574100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:39.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.634+0000 7fe57a400700 1 -- 172.21.15.119:0/3035956983 shutdown_connections 2024-05-27T09:33:39.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.634+0000 7fe57a400700 1 -- 172.21.15.119:0/3035956983 wait complete. 2024-05-27T09:33:39.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.636+0000 7fe57a400700 1 Processor -- start 2024-05-27T09:33:39.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.636+0000 7fe57a400700 1 -- start start 2024-05-27T09:33:39.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.636+0000 7fe57a400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe574102630 0x7fe574197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:39.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.636+0000 7fe57a400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe574102fd0 0x7fe574198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:39.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.636+0000 7fe57a400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe574198b70 con 0x7fe574102fd0 2024-05-27T09:33:39.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.636+0000 7fe57a400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe574198cb0 con 0x7fe574102630 2024-05-27T09:33:39.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.637+0000 7fe572a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe574102fd0 0x7fe574198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:39.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.637+0000 7fe572a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe574102fd0 0x7fe574198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47338/0 (socket says 172.21.15.119:47338) 2024-05-27T09:33:39.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.637+0000 7fe572a00700 1 -- 172.21.15.119:0/2240169149 learned_addr learned my addr 172.21.15.119:0/2240169149 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:33:39.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.637+0000 7fe573400700 1 --2- 172.21.15.119:0/2240169149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe574102630 0x7fe574197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:39.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.637+0000 7fe572a00700 1 -- 172.21.15.119:0/2240169149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe574102630 msgr2=0x7fe574197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:39.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.637+0000 7fe572a00700 1 --2- 172.21.15.119:0/2240169149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe574102630 0x7fe574197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:39.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.637+0000 7fe572a00700 1 -- 172.21.15.119:0/2240169149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5600095f0 con 0x7fe574102fd0 2024-05-27T09:33:39.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.638+0000 7fe572a00700 1 --2- 172.21.15.119:0/2240169149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe574102fd0 0x7fe574198520 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe56000b5c0 tx=0x7fe56000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:39.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.638+0000 7fe56be00700 1 -- 172.21.15.119:0/2240169149 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe56001d070 con 0x7fe574102fd0 2024-05-27T09:33:39.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.638+0000 7fe56be00700 1 -- 172.21.15.119:0/2240169149 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe56000f460 con 0x7fe574102fd0 2024-05-27T09:33:39.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.638+0000 7fe57a400700 1 -- 172.21.15.119:0/2240169149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe57419caa0 con 0x7fe574102fd0 2024-05-27T09:33:39.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.638+0000 7fe56be00700 1 -- 172.21.15.119:0/2240169149 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe560017610 con 0x7fe574102fd0 2024-05-27T09:33:39.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.638+0000 7fe57a400700 1 -- 172.21.15.119:0/2240169149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe57419cf90 con 0x7fe574102fd0 2024-05-27T09:33:39.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.639+0000 7fe57a400700 1 -- 172.21.15.119:0/2240169149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57404ea50 con 0x7fe574102fd0 2024-05-27T09:33:39.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.641+0000 7fe56be00700 1 -- 172.21.15.119:0/2240169149 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7fe56000f5d0 con 0x7fe574102fd0 2024-05-27T09:33:39.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.642+0000 7fe56be00700 1 --2- 172.21.15.119:0/2240169149 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe5580786c0 0x7fe55807ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:39.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.642+0000 7fe56be00700 1 -- 172.21.15.119:0/2240169149 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe56009be50 con 0x7fe574102fd0 2024-05-27T09:33:39.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.642+0000 7fe573400700 1 --2- 172.21.15.119:0/2240169149 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe5580786c0 0x7fe55807ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:39.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.643+0000 7fe573400700 1 --2- 172.21.15.119:0/2240169149 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe5580786c0 0x7fe55807ab70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe564005ea0 tx=0x7fe564005e30 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:39.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.647+0000 7fe56be00700 1 -- 172.21.15.119:0/2240169149 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7fe560063eb0 con 0x7fe574102fd0 2024-05-27T09:33:39.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:39.910+0000 7fe57a400700 1 -- 172.21.15.119:0/2240169149 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d", "target": ["mon-mgr", ""]}) v1 -- 0x7fe57419d510 con 0x7fe5580786c0 2024-05-27T09:33:41.239 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:40 smithi119 ceph-mon[29082]: pgmap v99: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:41.240 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:40 smithi119 ceph-mon[29082]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:33:41.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:40 smithi138 ceph-mon[34885]: pgmap v99: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:41.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:40 smithi138 ceph-mon[34885]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:33:42.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:42.838+0000 7fe56be00700 1 -- 172.21.15.119:0/2240169149 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5391 (secure 0 0 0) 0x7fe57419d510 con 0x7fe5580786c0 2024-05-27T09:33:42.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:42.841+0000 7fe57a400700 1 -- 172.21.15.119:0/2240169149 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe5580786c0 msgr2=0x7fe55807ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:42.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:42.841+0000 7fe57a400700 1 --2- 172.21.15.119:0/2240169149 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe5580786c0 0x7fe55807ab70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe564005ea0 tx=0x7fe564005e30 comp rx=0 tx=0).stop 2024-05-27T09:33:42.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:42.841+0000 7fe57a400700 1 -- 172.21.15.119:0/2240169149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe574102fd0 msgr2=0x7fe574198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:42.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:42.841+0000 7fe57a400700 1 --2- 172.21.15.119:0/2240169149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe574102fd0 0x7fe574198520 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe56000b5c0 tx=0x7fe56000be20 comp rx=0 tx=0).stop 2024-05-27T09:33:42.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:42.842+0000 7fe57a400700 1 -- 172.21.15.119:0/2240169149 shutdown_connections 2024-05-27T09:33:42.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:42.842+0000 7fe57a400700 1 --2- 172.21.15.119:0/2240169149 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe5580786c0 0x7fe55807ab70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:42.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:42.842+0000 7fe57a400700 1 --2- 172.21.15.119:0/2240169149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe574102630 0x7fe574197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:42.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:42.842+0000 7fe57a400700 1 --2- 172.21.15.119:0/2240169149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe574102fd0 0x7fe574198520 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:42.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:42.842+0000 7fe57a400700 1 -- 172.21.15.119:0/2240169149 >> 172.21.15.119:0/2240169149 conn(0x7fe5740fdef0 msgr2=0x7fe5740ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:42.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:42.842+0000 7fe57a400700 1 -- 172.21.15.119:0/2240169149 shutdown_connections 2024-05-27T09:33:42.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:42.842+0000 7fe57a400700 1 -- 172.21.15.119:0/2240169149 wait complete. 2024-05-27T09:33:42.860 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:33:43.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:43 smithi119 ceph-mon[29082]: pgmap v100: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:43.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:43 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:33:43.609 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'ceph orch ps' 2024-05-27T09:33:43.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:43 smithi138 ceph-mon[34885]: pgmap v100: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:43.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:43 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:33:44.016 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:33:45.343 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:45 smithi119 ceph-mon[29082]: pgmap v101: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:45.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:45 smithi138 ceph-mon[34885]: pgmap v101: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:46.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.150+0000 7fa013e00700 1 -- 172.21.15.119:0/2689142049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014106b80 msgr2=0x7fa014106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:46.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.150+0000 7fa013e00700 1 --2- 172.21.15.119:0/2689142049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014106b80 0x7fa014106ff0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa000009910 tx=0x7fa000009c20 comp rx=0 tx=0).stop 2024-05-27T09:33:46.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.150+0000 7fa013e00700 1 -- 172.21.15.119:0/2689142049 shutdown_connections 2024-05-27T09:33:46.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.150+0000 7fa013e00700 1 --2- 172.21.15.119:0/2689142049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014106b80 0x7fa014106ff0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:46.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.150+0000 7fa013e00700 1 --2- 172.21.15.119:0/2689142049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa01410b610 0x7fa01410b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:46.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.150+0000 7fa013e00700 1 -- 172.21.15.119:0/2689142049 >> 172.21.15.119:0/2689142049 conn(0x7fa014075df0 msgr2=0x7fa014078200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:46.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.150+0000 7fa013e00700 1 -- 172.21.15.119:0/2689142049 shutdown_connections 2024-05-27T09:33:46.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.150+0000 7fa013e00700 1 -- 172.21.15.119:0/2689142049 wait complete. 2024-05-27T09:33:46.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.151+0000 7fa013e00700 1 Processor -- start 2024-05-27T09:33:46.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.152+0000 7fa013e00700 1 -- start start 2024-05-27T09:33:46.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.152+0000 7fa013e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014106b80 0x7fa01419c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:46.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.152+0000 7fa013e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa01410b610 0x7fa01419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:46.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.152+0000 7fa013e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01419cc80 con 0x7fa014106b80 2024-05-27T09:33:46.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.152+0000 7fa013e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01419cdc0 con 0x7fa01410b610 2024-05-27T09:33:46.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.152+0000 7fa012a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014106b80 0x7fa01419c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:46.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.152+0000 7fa012a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014106b80 0x7fa01419c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47368/0 (socket says 172.21.15.119:47368) 2024-05-27T09:33:46.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.152+0000 7fa012a00700 1 -- 172.21.15.119:0/2353729845 learned_addr learned my addr 172.21.15.119:0/2353729845 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:33:46.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.153+0000 7fa012000700 1 --2- 172.21.15.119:0/2353729845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa01410b610 0x7fa01419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:46.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.153+0000 7fa012a00700 1 -- 172.21.15.119:0/2353729845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa01410b610 msgr2=0x7fa01419c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:46.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.153+0000 7fa012a00700 1 --2- 172.21.15.119:0/2353729845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa01410b610 0x7fa01419c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:46.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.153+0000 7fa012a00700 1 -- 172.21.15.119:0/2353729845 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0000095f0 con 0x7fa014106b80 2024-05-27T09:33:46.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.153+0000 7fa012a00700 1 --2- 172.21.15.119:0/2353729845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014106b80 0x7fa01419c0f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa00400c930 tx=0x7fa00400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:46.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.154+0000 7fa00b400700 1 -- 172.21.15.119:0/2353729845 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa004007ad0 con 0x7fa014106b80 2024-05-27T09:33:46.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.154+0000 7fa00b400700 1 -- 172.21.15.119:0/2353729845 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa004007c30 con 0x7fa014106b80 2024-05-27T09:33:46.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.154+0000 7fa00b400700 1 -- 172.21.15.119:0/2353729845 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa004018610 con 0x7fa014106b80 2024-05-27T09:33:46.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.154+0000 7fa013e00700 1 -- 172.21.15.119:0/2353729845 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0141a0c10 con 0x7fa014106b80 2024-05-27T09:33:46.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.154+0000 7fa013e00700 1 -- 172.21.15.119:0/2353729845 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0141a1140 con 0x7fa014106b80 2024-05-27T09:33:46.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.155+0000 7fa013e00700 1 -- 172.21.15.119:0/2353729845 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa01404f2a0 con 0x7fa014106b80 2024-05-27T09:33:46.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.156+0000 7fa00b400700 1 -- 172.21.15.119:0/2353729845 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7fa00400f450 con 0x7fa014106b80 2024-05-27T09:33:46.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.157+0000 7fa00b400700 1 --2- 172.21.15.119:0/2353729845 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f9ff80786c0 0x7f9ff807ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:46.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.157+0000 7fa00b400700 1 -- 172.21.15.119:0/2353729845 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa00409aa80 con 0x7fa014106b80 2024-05-27T09:33:46.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.158+0000 7fa012000700 1 --2- 172.21.15.119:0/2353729845 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f9ff80786c0 0x7f9ff807ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:46.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.158+0000 7fa012000700 1 --2- 172.21.15.119:0/2353729845 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f9ff80786c0 0x7f9ff807ab70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa000000c00 tx=0x7fa000005fd0 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:46.164 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.163+0000 7fa00b400700 1 -- 172.21.15.119:0/2353729845 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7fa004062ae0 con 0x7fa014106b80 2024-05-27T09:33:46.411 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.410+0000 7fa013e00700 1 -- 172.21.15.119:0/2353729845 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0141a16f0 con 0x7f9ff80786c0 2024-05-27T09:33:46.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.429+0000 7fa00b400700 1 -- 172.21.15.119:0/2353729845 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fa0141a16f0 con 0x7f9ff80786c0 2024-05-27T09:33:46.431 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:33:46.431 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (77s) 34s ago 14m 13.8M - 0.25.0 c8568f914cd2 6b90564dcef7 2024-05-27T09:33:46.431 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (15m) 34s ago 15m 8082k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:33:46.431 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (13m) 117s ago 13m 7834k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:33:46.431 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (15m) 34s ago 15m 6665k - 18.2.0 dc2bc1663786 e5fafde44183 2024-05-27T09:33:46.431 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (13m) 117s ago 13m 6660k - 18.2.0 dc2bc1663786 46956ab99bab 2024-05-27T09:33:46.431 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (42s) 34s ago 14m 51.1M - 9.4.12 2bacad6d85d8 b3abdab08db4 2024-05-27T09:33:46.431 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (7m) 34s ago 7m 15.5M - 18.2.0 dc2bc1663786 93b21713fb1e 2024-05-27T09:33:46.431 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (7m) 34s ago 7m 129M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:33:46.432 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (7m) 117s ago 7m 15.0M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:33:46.432 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (7m) 117s ago 7m 128M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:33:46.432 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (3m) 34s ago 16m 594M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:33:46.432 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (2m) 117s ago 13m 529M - 19.0.0-3868-ga6507388 850a06a84d28 2fc3a80696fa 2024-05-27T09:33:46.432 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (16m) 34s ago 16m 59.8M 2048M 18.2.0 dc2bc1663786 ba998b789a1c 2024-05-27T09:33:46.432 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (13m) 117s ago 13m 47.8M 2048M 18.2.0 dc2bc1663786 60216fd67078 2024-05-27T09:33:46.432 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (2m) 34s ago 15m 7495k - 1.5.0 0da6a335fe13 709c37815cff 2024-05-27T09:33:46.432 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (2m) 117s ago 13m 2889k - 1.5.0 0da6a335fe13 999047a1a5bf 2024-05-27T09:33:46.432 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (12m) 34s ago 12m 224M 1216M 18.2.0 dc2bc1663786 825bd1392508 2024-05-27T09:33:46.432 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (11m) 34s ago 11m 251M 1216M 18.2.0 dc2bc1663786 fce9819903d9 2024-05-27T09:33:46.432 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (11m) 34s ago 11m 190M 1216M 18.2.0 dc2bc1663786 3955a6112c1c 2024-05-27T09:33:46.432 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (10m) 117s ago 10m 268M 2240M 18.2.0 dc2bc1663786 693c92220622 2024-05-27T09:33:46.432 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (10m) 117s ago 10m 246M 2240M 18.2.0 dc2bc1663786 7559e658e917 2024-05-27T09:33:46.432 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (9m) 117s ago 9m 229M 2240M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:33:46.433 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (101s) 34s ago 14m 57.1M - 2.43.0 a07b618ecd1d cba50454ed21 2024-05-27T09:33:46.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.432+0000 7fa013e00700 1 -- 172.21.15.119:0/2353729845 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f9ff80786c0 msgr2=0x7f9ff807ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:46.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.432+0000 7fa013e00700 1 --2- 172.21.15.119:0/2353729845 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f9ff80786c0 0x7f9ff807ab70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa000000c00 tx=0x7fa000005fd0 comp rx=0 tx=0).stop 2024-05-27T09:33:46.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.432+0000 7fa013e00700 1 -- 172.21.15.119:0/2353729845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014106b80 msgr2=0x7fa01419c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:46.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.432+0000 7fa013e00700 1 --2- 172.21.15.119:0/2353729845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014106b80 0x7fa01419c0f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa00400c930 tx=0x7fa00400ccf0 comp rx=0 tx=0).stop 2024-05-27T09:33:46.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.432+0000 7fa013e00700 1 -- 172.21.15.119:0/2353729845 shutdown_connections 2024-05-27T09:33:46.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.432+0000 7fa013e00700 1 --2- 172.21.15.119:0/2353729845 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f9ff80786c0 0x7f9ff807ab70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:46.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.432+0000 7fa013e00700 1 --2- 172.21.15.119:0/2353729845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa014106b80 0x7fa01419c0f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:46.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.432+0000 7fa013e00700 1 --2- 172.21.15.119:0/2353729845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa01410b610 0x7fa01419c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:46.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.432+0000 7fa013e00700 1 -- 172.21.15.119:0/2353729845 >> 172.21.15.119:0/2353729845 conn(0x7fa014075df0 msgr2=0x7fa0140776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:46.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.432+0000 7fa013e00700 1 -- 172.21.15.119:0/2353729845 shutdown_connections 2024-05-27T09:33:46.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:46.433+0000 7fa013e00700 1 -- 172.21.15.119:0/2353729845 wait complete. 2024-05-27T09:33:47.171 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-27T09:33:47.172 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi119.front.sepia.ceph.com 2024-05-27T09:33:47.172 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-05-27T09:33:47.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:47 smithi119 ceph-mon[29082]: pgmap v102: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:47.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:47 smithi119 ceph-mon[29082]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:33:47.568 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:33:47.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:47 smithi138 ceph-mon[34885]: pgmap v102: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:47.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:47 smithi138 ceph-mon[34885]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:33:48.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:48 smithi138 ceph-mon[34885]: fsmap cephfs:2 {0=cephfs.smithi119.ppmzzk=up:active,1=cephfs.smithi138.ndfkhd=up:active} 2 up:standby 2024-05-27T09:33:48.443 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:48 smithi119 ceph-mon[29082]: fsmap cephfs:2 {0=cephfs.smithi119.ppmzzk=up:active,1=cephfs.smithi138.ndfkhd=up:active} 2 up:standby 2024-05-27T09:33:49.449 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:49 smithi119 ceph-mon[29082]: pgmap v103: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:49.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:49 smithi138 ceph-mon[34885]: pgmap v103: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:49.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.734+0000 7f01f8c00700 1 -- 172.21.15.119:0/2230972972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f41003b0 msgr2=0x7f01f4100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:49.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.734+0000 7f01f8c00700 1 --2- 172.21.15.119:0/2230972972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f41003b0 0x7f01f4100780 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009910 tx=0x7f01e4009c20 comp rx=0 tx=0).stop 2024-05-27T09:33:49.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.734+0000 7f01f8c00700 1 -- 172.21.15.119:0/2230972972 shutdown_connections 2024-05-27T09:33:49.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.734+0000 7f01f8c00700 1 --2- 172.21.15.119:0/2230972972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01f4100d50 0x7f01f4104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:49.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.734+0000 7f01f8c00700 1 --2- 172.21.15.119:0/2230972972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f41003b0 0x7f01f4100780 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:49.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.734+0000 7f01f8c00700 1 -- 172.21.15.119:0/2230972972 >> 172.21.15.119:0/2230972972 conn(0x7f01f40fbc30 msgr2=0x7f01f40fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:49.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.734+0000 7f01f8c00700 1 -- 172.21.15.119:0/2230972972 shutdown_connections 2024-05-27T09:33:49.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.734+0000 7f01f8c00700 1 -- 172.21.15.119:0/2230972972 wait complete. 2024-05-27T09:33:49.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.735+0000 7f01f8c00700 1 Processor -- start 2024-05-27T09:33:49.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.735+0000 7f01f8c00700 1 -- start start 2024-05-27T09:33:49.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.735+0000 7f01f8c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f41003b0 0x7f01f4198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:49.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.735+0000 7f01f8c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01f4100d50 0x7f01f4198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:49.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.735+0000 7f01f8c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4198c20 con 0x7f01f41003b0 2024-05-27T09:33:49.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.735+0000 7f01f8c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f419c9b0 con 0x7f01f4100d50 2024-05-27T09:33:49.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.736+0000 7f01f1e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f41003b0 0x7f01f4198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:49.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.736+0000 7f01f1e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f41003b0 0x7f01f4198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54936/0 (socket says 172.21.15.119:54936) 2024-05-27T09:33:49.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.736+0000 7f01f1e00700 1 -- 172.21.15.119:0/1632984416 learned_addr learned my addr 172.21.15.119:0/1632984416 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:33:49.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.736+0000 7f01f1e00700 1 -- 172.21.15.119:0/1632984416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01f4100d50 msgr2=0x7f01f4198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:49.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.736+0000 7f01f1e00700 1 --2- 172.21.15.119:0/1632984416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01f4100d50 0x7f01f4198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:49.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.736+0000 7f01f1e00700 1 -- 172.21.15.119:0/1632984416 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e40095f0 con 0x7f01f41003b0 2024-05-27T09:33:49.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.736+0000 7f01f1e00700 1 --2- 172.21.15.119:0/1632984416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f41003b0 0x7f01f4198000 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f01e4006010 tx=0x7f01e400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:49.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.737+0000 7f01eaa00700 1 -- 172.21.15.119:0/1632984416 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01e401d070 con 0x7f01f41003b0 2024-05-27T09:33:49.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.737+0000 7f01eaa00700 1 -- 172.21.15.119:0/1632984416 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01e4004fb0 con 0x7f01f41003b0 2024-05-27T09:33:49.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.737+0000 7f01eaa00700 1 -- 172.21.15.119:0/1632984416 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01e400f690 con 0x7f01f41003b0 2024-05-27T09:33:49.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.737+0000 7f01f8c00700 1 -- 172.21.15.119:0/1632984416 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f419cc30 con 0x7f01f41003b0 2024-05-27T09:33:49.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.737+0000 7f01f8c00700 1 -- 172.21.15.119:0/1632984416 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f419d120 con 0x7f01f41003b0 2024-05-27T09:33:49.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.738+0000 7f01f8c00700 1 -- 172.21.15.119:0/1632984416 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f41044a0 con 0x7f01f41003b0 2024-05-27T09:33:49.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.739+0000 7f01eaa00700 1 -- 172.21.15.119:0/1632984416 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7f01e40228a0 con 0x7f01f41003b0 2024-05-27T09:33:49.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.741+0000 7f01eaa00700 1 --2- 172.21.15.119:0/1632984416 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f01d80786c0 0x7f01d807ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:49.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.741+0000 7f01eaa00700 1 -- 172.21.15.119:0/1632984416 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f01e409bff0 con 0x7f01f41003b0 2024-05-27T09:33:49.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.741+0000 7f01f1400700 1 --2- 172.21.15.119:0/1632984416 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f01d80786c0 0x7f01d807ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:49.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.742+0000 7f01f1400700 1 --2- 172.21.15.119:0/1632984416 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f01d80786c0 0x7f01d807ab70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f01f4199620 tx=0x7f01e0006d20 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:49.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.746+0000 7f01eaa00700 1 -- 172.21.15.119:0/1632984416 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f01e40640f0 con 0x7f01f41003b0 2024-05-27T09:33:49.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.978+0000 7f01f8c00700 1 -- 172.21.15.119:0/1632984416 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f01f404ea50 con 0x7f01f41003b0 2024-05-27T09:33:49.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.989+0000 7f01eaa00700 1 -- 172.21.15.119:0/1632984416 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v44)=0 v44) v1 ==== 125+0+0 (secure 0 0 0) 0x7f01e4063a90 con 0x7f01f41003b0 2024-05-27T09:33:49.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.992+0000 7f01f8c00700 1 -- 172.21.15.119:0/1632984416 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f01d80786c0 msgr2=0x7f01d807ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:49.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.992+0000 7f01f8c00700 1 --2- 172.21.15.119:0/1632984416 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f01d80786c0 0x7f01d807ab70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f01f4199620 tx=0x7f01e0006d20 comp rx=0 tx=0).stop 2024-05-27T09:33:49.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.992+0000 7f01f8c00700 1 -- 172.21.15.119:0/1632984416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f41003b0 msgr2=0x7f01f4198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:49.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.992+0000 7f01f8c00700 1 --2- 172.21.15.119:0/1632984416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f41003b0 0x7f01f4198000 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f01e4006010 tx=0x7f01e400be20 comp rx=0 tx=0).stop 2024-05-27T09:33:49.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.992+0000 7f01f8c00700 1 -- 172.21.15.119:0/1632984416 shutdown_connections 2024-05-27T09:33:49.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.992+0000 7f01f8c00700 1 --2- 172.21.15.119:0/1632984416 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f01d80786c0 0x7f01d807ab70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:49.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.992+0000 7f01f8c00700 1 --2- 172.21.15.119:0/1632984416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f41003b0 0x7f01f4198000 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:49.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.992+0000 7f01f8c00700 1 --2- 172.21.15.119:0/1632984416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01f4100d50 0x7f01f4198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:49.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.992+0000 7f01f8c00700 1 -- 172.21.15.119:0/1632984416 >> 172.21.15.119:0/1632984416 conn(0x7f01f40fbc30 msgr2=0x7f01f40fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:49.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.992+0000 7f01f8c00700 1 -- 172.21.15.119:0/1632984416 shutdown_connections 2024-05-27T09:33:49.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:49.992+0000 7f01f8c00700 1 -- 172.21.15.119:0/1632984416 wait complete. 2024-05-27T09:33:50.735 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-27T09:33:50.735 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi119.front.sepia.ceph.com 2024-05-27T09:33:50.735 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-05-27T09:33:51.132 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:33:51.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:50 smithi119 ceph-mon[29082]: pgmap v104: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:51.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:50 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/1632984416' entity='client.admin' 2024-05-27T09:33:51.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:50 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:33:51.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:50 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:33:51.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:50 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:33:51.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:50 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:51.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:50 smithi138 ceph-mon[34885]: pgmap v104: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:51.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:50 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/1632984416' entity='client.admin' 2024-05-27T09:33:51.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:50 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:33:51.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:50 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:33:51.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:50 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:33:51.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:50 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:33:53.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.337+0000 7fd35b000700 1 -- 172.21.15.119:0/4240842297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd354102630 msgr2=0x7fd354102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:53.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.337+0000 7fd35b000700 1 --2- 172.21.15.119:0/4240842297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd354102630 0x7fd354102a00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd344009910 tx=0x7fd344009c20 comp rx=0 tx=0).stop 2024-05-27T09:33:53.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.337+0000 7fd35b000700 1 -- 172.21.15.119:0/4240842297 shutdown_connections 2024-05-27T09:33:53.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.337+0000 7fd35b000700 1 --2- 172.21.15.119:0/4240842297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd354102fd0 0x7fd354106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:53.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.337+0000 7fd35b000700 1 --2- 172.21.15.119:0/4240842297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd354102630 0x7fd354102a00 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:53.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.337+0000 7fd35b000700 1 -- 172.21.15.119:0/4240842297 >> 172.21.15.119:0/4240842297 conn(0x7fd3540fdef0 msgr2=0x7fd354100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:53.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.337+0000 7fd35b000700 1 -- 172.21.15.119:0/4240842297 shutdown_connections 2024-05-27T09:33:53.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.338+0000 7fd35b000700 1 -- 172.21.15.119:0/4240842297 wait complete. 2024-05-27T09:33:53.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.339+0000 7fd35b000700 1 Processor -- start 2024-05-27T09:33:53.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.340+0000 7fd35b000700 1 -- start start 2024-05-27T09:33:53.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.340+0000 7fd35b000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd354102630 0x7fd354198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:53.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.340+0000 7fd35b000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd354102fd0 0x7fd354198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:53.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.340+0000 7fd35b000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd354198bb0 con 0x7fd354102fd0 2024-05-27T09:33:53.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.340+0000 7fd35b000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd354198cf0 con 0x7fd354102630 2024-05-27T09:33:53.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.340+0000 7fd353400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd354102fd0 0x7fd354198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:53.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.341+0000 7fd353400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd354102fd0 0x7fd354198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54950/0 (socket says 172.21.15.119:54950) 2024-05-27T09:33:53.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.341+0000 7fd353400700 1 -- 172.21.15.119:0/2438995816 learned_addr learned my addr 172.21.15.119:0/2438995816 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:33:53.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.341+0000 7fd353e00700 1 --2- 172.21.15.119:0/2438995816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd354102630 0x7fd354198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:53.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.341+0000 7fd353400700 1 -- 172.21.15.119:0/2438995816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd354102630 msgr2=0x7fd354198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:53.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.341+0000 7fd353400700 1 --2- 172.21.15.119:0/2438995816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd354102630 0x7fd354198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:53.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.341+0000 7fd353400700 1 -- 172.21.15.119:0/2438995816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3440095f0 con 0x7fd354102fd0 2024-05-27T09:33:53.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.341+0000 7fd353400700 1 --2- 172.21.15.119:0/2438995816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd354102fd0 0x7fd354198560 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00c960 tx=0x7fd33c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:53.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.342+0000 7fd350c00700 1 -- 172.21.15.119:0/2438995816 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd33c007a70 con 0x7fd354102fd0 2024-05-27T09:33:53.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.342+0000 7fd350c00700 1 -- 172.21.15.119:0/2438995816 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd33c007bd0 con 0x7fd354102fd0 2024-05-27T09:33:53.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.342+0000 7fd35b000700 1 -- 172.21.15.119:0/2438995816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35419cb40 con 0x7fd354102fd0 2024-05-27T09:33:53.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.342+0000 7fd350c00700 1 -- 172.21.15.119:0/2438995816 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd33c0186c0 con 0x7fd354102fd0 2024-05-27T09:33:53.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.342+0000 7fd35b000700 1 -- 172.21.15.119:0/2438995816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35419d090 con 0x7fd354102fd0 2024-05-27T09:33:53.343 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.343+0000 7fd35b000700 1 -- 172.21.15.119:0/2438995816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd35410a940 con 0x7fd354102fd0 2024-05-27T09:33:53.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.344+0000 7fd350c00700 1 -- 172.21.15.119:0/2438995816 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7fd33c00faa0 con 0x7fd354102fd0 2024-05-27T09:33:53.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.345+0000 7fd350c00700 1 --2- 172.21.15.119:0/2438995816 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fd3380786c0 0x7fd33807ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:53.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.345+0000 7fd350c00700 1 -- 172.21.15.119:0/2438995816 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd33c09a7e0 con 0x7fd354102fd0 2024-05-27T09:33:53.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.346+0000 7fd353e00700 1 --2- 172.21.15.119:0/2438995816 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fd3380786c0 0x7fd33807ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:53.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.346+0000 7fd353e00700 1 --2- 172.21.15.119:0/2438995816 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fd3380786c0 0x7fd33807ab70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd34400b5c0 tx=0x7fd34401a040 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:53.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.350+0000 7fd350c00700 1 -- 172.21.15.119:0/2438995816 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7fd33c062440 con 0x7fd354102fd0 2024-05-27T09:33:53.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:53 smithi119 ceph-mon[29082]: pgmap v105: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:53.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.590+0000 7fd35b000700 1 -- 172.21.15.119:0/2438995816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fd35404ea50 con 0x7fd354102fd0 2024-05-27T09:33:53.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.590+0000 7fd350c00700 1 -- 172.21.15.119:0/2438995816 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v44)=0 v44) v1 ==== 155+0+0 (secure 0 0 0) 0x7fd33c062260 con 0x7fd354102fd0 2024-05-27T09:33:53.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.593+0000 7fd35b000700 1 -- 172.21.15.119:0/2438995816 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fd3380786c0 msgr2=0x7fd33807ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:53.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.593+0000 7fd35b000700 1 --2- 172.21.15.119:0/2438995816 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fd3380786c0 0x7fd33807ab70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd34400b5c0 tx=0x7fd34401a040 comp rx=0 tx=0).stop 2024-05-27T09:33:53.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.593+0000 7fd35b000700 1 -- 172.21.15.119:0/2438995816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd354102fd0 msgr2=0x7fd354198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:53.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.593+0000 7fd35b000700 1 --2- 172.21.15.119:0/2438995816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd354102fd0 0x7fd354198560 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00c960 tx=0x7fd33c00cd20 comp rx=0 tx=0).stop 2024-05-27T09:33:53.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.593+0000 7fd35b000700 1 -- 172.21.15.119:0/2438995816 shutdown_connections 2024-05-27T09:33:53.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.593+0000 7fd35b000700 1 --2- 172.21.15.119:0/2438995816 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fd3380786c0 0x7fd33807ab70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:53.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.593+0000 7fd35b000700 1 --2- 172.21.15.119:0/2438995816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd354102630 0x7fd354198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:53.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.593+0000 7fd35b000700 1 --2- 172.21.15.119:0/2438995816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd354102fd0 0x7fd354198560 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:53.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.593+0000 7fd35b000700 1 -- 172.21.15.119:0/2438995816 >> 172.21.15.119:0/2438995816 conn(0x7fd3540fdef0 msgr2=0x7fd3541002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:53.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.594+0000 7fd35b000700 1 -- 172.21.15.119:0/2438995816 shutdown_connections 2024-05-27T09:33:53.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:53.594+0000 7fd35b000700 1 -- 172.21.15.119:0/2438995816 wait complete. 2024-05-27T09:33:53.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:53 smithi138 ceph-mon[34885]: pgmap v105: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:54.316 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-05-27T09:33:54.722 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:33:55.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:55 smithi119 ceph-mon[29082]: pgmap v106: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:55.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:55 smithi138 ceph-mon[34885]: pgmap v106: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:56.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.884+0000 7fb65c400700 1 -- 172.21.15.119:0/2443545857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6541003b0 msgr2=0x7fb654100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:56.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.884+0000 7fb65c400700 1 --2- 172.21.15.119:0/2443545857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6541003b0 0x7fb654100780 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb648009910 tx=0x7fb648009c20 comp rx=0 tx=0).stop 2024-05-27T09:33:56.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.884+0000 7fb65c400700 1 -- 172.21.15.119:0/2443545857 shutdown_connections 2024-05-27T09:33:56.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.884+0000 7fb65c400700 1 --2- 172.21.15.119:0/2443545857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb654100d50 0x7fb654104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:56.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.884+0000 7fb65c400700 1 --2- 172.21.15.119:0/2443545857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6541003b0 0x7fb654100780 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:56.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.884+0000 7fb65c400700 1 -- 172.21.15.119:0/2443545857 >> 172.21.15.119:0/2443545857 conn(0x7fb6540fbc30 msgr2=0x7fb6540fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:56.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.884+0000 7fb65c400700 1 -- 172.21.15.119:0/2443545857 shutdown_connections 2024-05-27T09:33:56.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.884+0000 7fb65c400700 1 -- 172.21.15.119:0/2443545857 wait complete. 2024-05-27T09:33:56.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.885+0000 7fb65c400700 1 Processor -- start 2024-05-27T09:33:56.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.886+0000 7fb65c400700 1 -- start start 2024-05-27T09:33:56.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.886+0000 7fb65c400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6541003b0 0x7fb654195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:56.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.886+0000 7fb65c400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb654100d50 0x7fb654196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:56.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.886+0000 7fb65c400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb654196930 con 0x7fb6541003b0 2024-05-27T09:33:56.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.886+0000 7fb65c400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65419a6c0 con 0x7fb654100d50 2024-05-27T09:33:56.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.886+0000 7fb659a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6541003b0 0x7fb654195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:56.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.886+0000 7fb659a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6541003b0 0x7fb654195d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54964/0 (socket says 172.21.15.119:54964) 2024-05-27T09:33:56.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.886+0000 7fb659a00700 1 -- 172.21.15.119:0/665736309 learned_addr learned my addr 172.21.15.119:0/665736309 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:33:56.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.887+0000 7fb659000700 1 --2- 172.21.15.119:0/665736309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb654100d50 0x7fb654196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:56.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.887+0000 7fb659a00700 1 -- 172.21.15.119:0/665736309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb654100d50 msgr2=0x7fb654196250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:56.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.887+0000 7fb659a00700 1 --2- 172.21.15.119:0/665736309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb654100d50 0x7fb654196250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:56.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.887+0000 7fb659a00700 1 -- 172.21.15.119:0/665736309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6480095f0 con 0x7fb6541003b0 2024-05-27T09:33:56.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.887+0000 7fb659000700 1 --2- 172.21.15.119:0/665736309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb654100d50 0x7fb654196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:33:56.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.887+0000 7fb659a00700 1 --2- 172.21.15.119:0/665736309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6541003b0 0x7fb654195d10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb64800b5c0 tx=0x7fb64800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:56.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.888+0000 7fb652000700 1 -- 172.21.15.119:0/665736309 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb64801d070 con 0x7fb6541003b0 2024-05-27T09:33:56.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.888+0000 7fb65c400700 1 -- 172.21.15.119:0/665736309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65419a940 con 0x7fb6541003b0 2024-05-27T09:33:56.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.888+0000 7fb652000700 1 -- 172.21.15.119:0/665736309 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb64800f460 con 0x7fb6541003b0 2024-05-27T09:33:56.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.888+0000 7fb652000700 1 -- 172.21.15.119:0/665736309 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb648017610 con 0x7fb6541003b0 2024-05-27T09:33:56.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.888+0000 7fb65c400700 1 -- 172.21.15.119:0/665736309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65419ae30 con 0x7fb6541003b0 2024-05-27T09:33:56.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.889+0000 7fb65c400700 1 -- 172.21.15.119:0/665736309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6541086c0 con 0x7fb6541003b0 2024-05-27T09:33:56.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.890+0000 7fb652000700 1 -- 172.21.15.119:0/665736309 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7fb64800f5d0 con 0x7fb6541003b0 2024-05-27T09:33:56.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.892+0000 7fb652000700 1 --2- 172.21.15.119:0/665736309 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fb63c078710 0x7fb63c07abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:33:56.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.892+0000 7fb652000700 1 -- 172.21.15.119:0/665736309 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb64809bd90 con 0x7fb6541003b0 2024-05-27T09:33:56.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.892+0000 7fb659000700 1 --2- 172.21.15.119:0/665736309 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fb63c078710 0x7fb63c07abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:33:56.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.893+0000 7fb659000700 1 --2- 172.21.15.119:0/665736309 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fb63c078710 0x7fb63c07abc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb640005fd0 tx=0x7fb640005dc0 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:33:56.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:56.897+0000 7fb652000700 1 -- 172.21.15.119:0/665736309 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7fb648063e70 con 0x7fb6541003b0 2024-05-27T09:33:57.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:57.130+0000 7fb65c400700 1 -- 172.21.15.119:0/665736309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fb65402cc30 con 0x7fb6541003b0 2024-05-27T09:33:57.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:57.131+0000 7fb652000700 1 -- 172.21.15.119:0/665736309 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v44)=0 v44) v1 ==== 163+0+0 (secure 0 0 0) 0x7fb648063810 con 0x7fb6541003b0 2024-05-27T09:33:57.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:57.135+0000 7fb65c400700 1 -- 172.21.15.119:0/665736309 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fb63c078710 msgr2=0x7fb63c07abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:57.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:57.135+0000 7fb65c400700 1 --2- 172.21.15.119:0/665736309 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fb63c078710 0x7fb63c07abc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb640005fd0 tx=0x7fb640005dc0 comp rx=0 tx=0).stop 2024-05-27T09:33:57.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:57.135+0000 7fb65c400700 1 -- 172.21.15.119:0/665736309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6541003b0 msgr2=0x7fb654195d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:33:57.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:57.135+0000 7fb65c400700 1 --2- 172.21.15.119:0/665736309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6541003b0 0x7fb654195d10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb64800b5c0 tx=0x7fb64800be00 comp rx=0 tx=0).stop 2024-05-27T09:33:57.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:57.135+0000 7fb65c400700 1 -- 172.21.15.119:0/665736309 shutdown_connections 2024-05-27T09:33:57.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:57.135+0000 7fb65c400700 1 --2- 172.21.15.119:0/665736309 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fb63c078710 0x7fb63c07abc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:57.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:57.135+0000 7fb65c400700 1 --2- 172.21.15.119:0/665736309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6541003b0 0x7fb654195d10 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:57.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:57.135+0000 7fb65c400700 1 --2- 172.21.15.119:0/665736309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb654100d50 0x7fb654196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:33:57.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:57.135+0000 7fb65c400700 1 -- 172.21.15.119:0/665736309 >> 172.21.15.119:0/665736309 conn(0x7fb6540fbc30 msgr2=0x7fb6540fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:33:57.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:57.135+0000 7fb65c400700 1 -- 172.21.15.119:0/665736309 shutdown_connections 2024-05-27T09:33:57.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:33:57.136+0000 7fb65c400700 1 -- 172.21.15.119:0/665736309 wait complete. 2024-05-27T09:33:57.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:57 smithi119 ceph-mon[29082]: pgmap v107: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:57.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:57 smithi138 ceph-mon[34885]: pgmap v107: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:57.873 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'ceph config set global log_to_journald false --force' 2024-05-27T09:33:58.274 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:33:58.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:58 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:33:58.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:58 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:33:59.522 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:33:59 smithi119 ceph-mon[29082]: pgmap v108: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:33:59.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:33:59 smithi138 ceph-mon[34885]: pgmap v108: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:00.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.613+0000 7fbd06800700 1 -- 172.21.15.119:0/2517235221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd000ffc30 msgr2=0x7fbd0010a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:00.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.613+0000 7fbd06800700 1 --2- 172.21.15.119:0/2517235221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd000ffc30 0x7fbd0010a6a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbcec009940 tx=0x7fbcec009c50 comp rx=0 tx=0).stop 2024-05-27T09:34:00.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.614+0000 7fbd06800700 1 -- 172.21.15.119:0/2517235221 shutdown_connections 2024-05-27T09:34:00.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.614+0000 7fbd06800700 1 --2- 172.21.15.119:0/2517235221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd000ffc30 0x7fbd0010a6a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:00.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.614+0000 7fbd06800700 1 --2- 172.21.15.119:0/2517235221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd000ff320 0x7fbd000ff6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:00.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.614+0000 7fbd06800700 1 -- 172.21.15.119:0/2517235221 >> 172.21.15.119:0/2517235221 conn(0x7fbd000748b0 msgr2=0x7fbd00074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:00.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.614+0000 7fbd06800700 1 -- 172.21.15.119:0/2517235221 shutdown_connections 2024-05-27T09:34:00.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.614+0000 7fbd06800700 1 -- 172.21.15.119:0/2517235221 wait complete. 2024-05-27T09:34:00.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.615+0000 7fbd06800700 1 Processor -- start 2024-05-27T09:34:00.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.616+0000 7fbd06800700 1 -- start start 2024-05-27T09:34:00.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.616+0000 7fbd06800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd000ff320 0x7fbd00100500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:00.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.616+0000 7fbd06800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd000ffc30 0x7fbd00100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:00.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.616+0000 7fbd06800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd00100f80 con 0x7fbd000ffc30 2024-05-27T09:34:00.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.616+0000 7fbd06800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd001010c0 con 0x7fbd000ff320 2024-05-27T09:34:00.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.617+0000 7fbcfea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd000ffc30 0x7fbd00100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:00.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.617+0000 7fbcfea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd000ffc30 0x7fbd00100a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33592/0 (socket says 172.21.15.119:33592) 2024-05-27T09:34:00.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.617+0000 7fbcfea00700 1 -- 172.21.15.119:0/4112760652 learned_addr learned my addr 172.21.15.119:0/4112760652 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:34:00.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.617+0000 7fbcff400700 1 --2- 172.21.15.119:0/4112760652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd000ff320 0x7fbd00100500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:00.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.617+0000 7fbcfea00700 1 -- 172.21.15.119:0/4112760652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd000ff320 msgr2=0x7fbd00100500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:00.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.617+0000 7fbcfea00700 1 --2- 172.21.15.119:0/4112760652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd000ff320 0x7fbd00100500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:00.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.617+0000 7fbcfea00700 1 -- 172.21.15.119:0/4112760652 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec0095f0 con 0x7fbd000ffc30 2024-05-27T09:34:00.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.618+0000 7fbcfea00700 1 --2- 172.21.15.119:0/4112760652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd000ffc30 0x7fbd00100a40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbcec0055f0 tx=0x7fbcec0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:00.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.618+0000 7fbcf7e00700 1 -- 172.21.15.119:0/4112760652 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcec01d070 con 0x7fbd000ffc30 2024-05-27T09:34:00.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.618+0000 7fbcf7e00700 1 -- 172.21.15.119:0/4112760652 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbcec00f460 con 0x7fbd000ffc30 2024-05-27T09:34:00.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.618+0000 7fbcf7e00700 1 -- 172.21.15.119:0/4112760652 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcec021640 con 0x7fbd000ffc30 2024-05-27T09:34:00.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.618+0000 7fbd06800700 1 -- 172.21.15.119:0/4112760652 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd001a1e10 con 0x7fbd000ffc30 2024-05-27T09:34:00.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.618+0000 7fbd06800700 1 -- 172.21.15.119:0/4112760652 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd001a2170 con 0x7fbd000ffc30 2024-05-27T09:34:00.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.619+0000 7fbd06800700 1 -- 172.21.15.119:0/4112760652 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd00107da0 con 0x7fbd000ffc30 2024-05-27T09:34:00.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.621+0000 7fbcf7e00700 1 -- 172.21.15.119:0/4112760652 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7fbcec00fa20 con 0x7fbd000ffc30 2024-05-27T09:34:00.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.622+0000 7fbcf7e00700 1 --2- 172.21.15.119:0/4112760652 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fbce40786c0 0x7fbce407ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:00.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.622+0000 7fbcf7e00700 1 -- 172.21.15.119:0/4112760652 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbcec09ba60 con 0x7fbd000ffc30 2024-05-27T09:34:00.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.622+0000 7fbcff400700 1 --2- 172.21.15.119:0/4112760652 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fbce40786c0 0x7fbce407ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:00.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.623+0000 7fbcff400700 1 --2- 172.21.15.119:0/4112760652 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fbce40786c0 0x7fbce407ab70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0005950 tx=0x7fbcf00058e0 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:00.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.627+0000 7fbcf7e00700 1 -- 172.21.15.119:0/4112760652 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7fbcec063b40 con 0x7fbd000ffc30 2024-05-27T09:34:00.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.860+0000 7fbd06800700 1 -- 172.21.15.119:0/4112760652 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fbd0002cca0 con 0x7fbd000ffc30 2024-05-27T09:34:00.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.861+0000 7fbcf7e00700 1 -- 172.21.15.119:0/4112760652 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v44)=0 v44) v1 ==== 135+0+0 (secure 0 0 0) 0x7fbcec0634e0 con 0x7fbd000ffc30 2024-05-27T09:34:00.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.864+0000 7fbd06800700 1 -- 172.21.15.119:0/4112760652 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fbce40786c0 msgr2=0x7fbce407ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:00.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.864+0000 7fbd06800700 1 --2- 172.21.15.119:0/4112760652 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fbce40786c0 0x7fbce407ab70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0005950 tx=0x7fbcf00058e0 comp rx=0 tx=0).stop 2024-05-27T09:34:00.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.864+0000 7fbd06800700 1 -- 172.21.15.119:0/4112760652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd000ffc30 msgr2=0x7fbd00100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:00.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.864+0000 7fbd06800700 1 --2- 172.21.15.119:0/4112760652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd000ffc30 0x7fbd00100a40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbcec0055f0 tx=0x7fbcec0056d0 comp rx=0 tx=0).stop 2024-05-27T09:34:00.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.864+0000 7fbd06800700 1 -- 172.21.15.119:0/4112760652 shutdown_connections 2024-05-27T09:34:00.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.864+0000 7fbd06800700 1 --2- 172.21.15.119:0/4112760652 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fbce40786c0 0x7fbce407ab70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:00.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.864+0000 7fbd06800700 1 --2- 172.21.15.119:0/4112760652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd000ff320 0x7fbd00100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:00.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.864+0000 7fbd06800700 1 --2- 172.21.15.119:0/4112760652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd000ffc30 0x7fbd00100a40 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:00.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.864+0000 7fbd06800700 1 -- 172.21.15.119:0/4112760652 >> 172.21.15.119:0/4112760652 conn(0x7fbd000748b0 msgr2=0x7fbd00104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:00.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.864+0000 7fbd06800700 1 -- 172.21.15.119:0/4112760652 shutdown_connections 2024-05-27T09:34:00.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:00.864+0000 7fbd06800700 1 -- 172.21.15.119:0/4112760652 wait complete. 2024-05-27T09:34:01.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:01 smithi119 ceph-mon[29082]: pgmap v109: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:01.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:01 smithi138 ceph-mon[34885]: pgmap v109: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:01.643 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-05-27T09:34:02.046 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:34:02.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:02 smithi119 ceph-mon[29082]: fsmap cephfs:2 {0=cephfs.smithi119.ppmzzk=up:active,1=cephfs.smithi138.ndfkhd=up:active} 2 up:standby 2024-05-27T09:34:02.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:02 smithi138 ceph-mon[34885]: fsmap cephfs:2 {0=cephfs.smithi119.ppmzzk=up:active,1=cephfs.smithi138.ndfkhd=up:active} 2 up:standby 2024-05-27T09:34:03.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:03 smithi119 ceph-mon[29082]: pgmap v110: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:03.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:03 smithi138 ceph-mon[34885]: pgmap v110: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:04.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.135+0000 7f8bf4a00700 1 -- 172.21.15.119:0/3933608275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf0068630 msgr2=0x7f8bf0068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:04.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.135+0000 7f8bf4a00700 1 --2- 172.21.15.119:0/3933608275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf0068630 0x7f8bf0068a00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009910 tx=0x7f8be0009c20 comp rx=0 tx=0).stop 2024-05-27T09:34:04.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.135+0000 7f8bf4a00700 1 -- 172.21.15.119:0/3933608275 shutdown_connections 2024-05-27T09:34:04.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.135+0000 7f8bf4a00700 1 --2- 172.21.15.119:0/3933608275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bf0068fd0 0x7f8bf0104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:04.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.135+0000 7f8bf4a00700 1 --2- 172.21.15.119:0/3933608275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf0068630 0x7f8bf0068a00 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:04.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.135+0000 7f8bf4a00700 1 -- 172.21.15.119:0/3933608275 >> 172.21.15.119:0/3933608275 conn(0x7f8bf0074f50 msgr2=0x7f8bf0075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:04.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.135+0000 7f8bf4a00700 1 -- 172.21.15.119:0/3933608275 shutdown_connections 2024-05-27T09:34:04.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.135+0000 7f8bf4a00700 1 -- 172.21.15.119:0/3933608275 wait complete. 2024-05-27T09:34:04.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.136+0000 7f8bf4a00700 1 Processor -- start 2024-05-27T09:34:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.137+0000 7f8bf4a00700 1 -- start start 2024-05-27T09:34:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.137+0000 7f8bf4a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bf0068630 0x7f8bf0198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.137+0000 7f8bf4a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf0068fd0 0x7f8bf0198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.137+0000 7f8bf4a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf0198c40 con 0x7f8bf0068fd0 2024-05-27T09:34:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.137+0000 7f8bf4a00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf019c9d0 con 0x7f8bf0068630 2024-05-27T09:34:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.137+0000 7f8bed400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf0068fd0 0x7f8bf0198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.137+0000 7f8bed400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf0068fd0 0x7f8bf0198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33622/0 (socket says 172.21.15.119:33622) 2024-05-27T09:34:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.137+0000 7f8bed400700 1 -- 172.21.15.119:0/2046895519 learned_addr learned my addr 172.21.15.119:0/2046895519 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:34:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.138+0000 7f8bede00700 1 --2- 172.21.15.119:0/2046895519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bf0068630 0x7f8bf0198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.138+0000 7f8bed400700 1 -- 172.21.15.119:0/2046895519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bf0068630 msgr2=0x7f8bf0198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.138+0000 7f8bed400700 1 --2- 172.21.15.119:0/2046895519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bf0068630 0x7f8bf0198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:04.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.138+0000 7f8bed400700 1 -- 172.21.15.119:0/2046895519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be00095f0 con 0x7f8bf0068fd0 2024-05-27T09:34:04.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.138+0000 7f8bed400700 1 --2- 172.21.15.119:0/2046895519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf0068fd0 0x7f8bf0198560 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00b480 tx=0x7f8bdc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:04.140 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.138+0000 7f8be6a00700 1 -- 172.21.15.119:0/2046895519 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bdc0118c0 con 0x7f8bf0068fd0 2024-05-27T09:34:04.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.138+0000 7f8be6a00700 1 -- 172.21.15.119:0/2046895519 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8bdc010380 con 0x7f8bf0068fd0 2024-05-27T09:34:04.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.139+0000 7f8bf4a00700 1 -- 172.21.15.119:0/2046895519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf019ccb0 con 0x7f8bf0068fd0 2024-05-27T09:34:04.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.139+0000 7f8be6a00700 1 -- 172.21.15.119:0/2046895519 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bdc00f4f0 con 0x7f8bf0068fd0 2024-05-27T09:34:04.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.139+0000 7f8bf4a00700 1 -- 172.21.15.119:0/2046895519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf019d200 con 0x7f8bf0068fd0 2024-05-27T09:34:04.141 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.140+0000 7f8bf4a00700 1 -- 172.21.15.119:0/2046895519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf0104570 con 0x7f8bf0068fd0 2024-05-27T09:34:04.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.141+0000 7f8be6a00700 1 -- 172.21.15.119:0/2046895519 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7f8bdc0104f0 con 0x7f8bf0068fd0 2024-05-27T09:34:04.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.142+0000 7f8be6a00700 1 --2- 172.21.15.119:0/2046895519 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f8bd40786c0 0x7f8bd407ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:04.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.142+0000 7f8be6a00700 1 -- 172.21.15.119:0/2046895519 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8bdc09a3c0 con 0x7f8bf0068fd0 2024-05-27T09:34:04.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.143+0000 7f8bede00700 1 --2- 172.21.15.119:0/2046895519 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f8bd40786c0 0x7f8bd407ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:04.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.143+0000 7f8bede00700 1 --2- 172.21.15.119:0/2046895519 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f8bd40786c0 0x7f8bd407ab70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8be0006010 tx=0x7f8be001a040 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:04.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.149+0000 7f8be6a00700 1 -- 172.21.15.119:0/2046895519 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f8bdc0624a0 con 0x7f8bf0068fd0 2024-05-27T09:34:04.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.418+0000 7f8bf4a00700 1 -- 172.21.15.119:0/2046895519 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d", "target": ["mon-mgr", ""]}) v1 -- 0x7f8bf019d610 con 0x7f8bd40786c0 2024-05-27T09:34:04.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.429+0000 7f8be6a00700 1 -- 172.21.15.119:0/2046895519 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f8bf019d610 con 0x7f8bd40786c0 2024-05-27T09:34:04.430 INFO:teuthology.orchestra.run.smithi119.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:34:04.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.432+0000 7f8bf4a00700 1 -- 172.21.15.119:0/2046895519 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f8bd40786c0 msgr2=0x7f8bd407ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:04.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.432+0000 7f8bf4a00700 1 --2- 172.21.15.119:0/2046895519 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f8bd40786c0 0x7f8bd407ab70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8be0006010 tx=0x7f8be001a040 comp rx=0 tx=0).stop 2024-05-27T09:34:04.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.432+0000 7f8bf4a00700 1 -- 172.21.15.119:0/2046895519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf0068fd0 msgr2=0x7f8bf0198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:04.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.432+0000 7f8bf4a00700 1 --2- 172.21.15.119:0/2046895519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf0068fd0 0x7f8bf0198560 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00b480 tx=0x7f8bdc00b840 comp rx=0 tx=0).stop 2024-05-27T09:34:04.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.432+0000 7f8bf4a00700 1 -- 172.21.15.119:0/2046895519 shutdown_connections 2024-05-27T09:34:04.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.432+0000 7f8bf4a00700 1 --2- 172.21.15.119:0/2046895519 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f8bd40786c0 0x7f8bd407ab70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:04.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.432+0000 7f8bf4a00700 1 --2- 172.21.15.119:0/2046895519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bf0068630 0x7f8bf0198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:04.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.432+0000 7f8bf4a00700 1 --2- 172.21.15.119:0/2046895519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bf0068fd0 0x7f8bf0198560 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:04.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.432+0000 7f8bf4a00700 1 -- 172.21.15.119:0/2046895519 >> 172.21.15.119:0/2046895519 conn(0x7f8bf0074f50 msgr2=0x7f8bf00fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:04.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.432+0000 7f8bf4a00700 1 -- 172.21.15.119:0/2046895519 shutdown_connections 2024-05-27T09:34:04.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:04.433+0000 7f8bf4a00700 1 -- 172.21.15.119:0/2046895519 wait complete. 2024-05-27T09:34:05.272 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-27T09:34:05.273 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi119.front.sepia.ceph.com 2024-05-27T09:34:05.273 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-05-27T09:34:05.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:05 smithi119 ceph-mon[29082]: pgmap v111: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:05.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:05 smithi119 ceph-mon[29082]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:34:05.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:05 smithi119 ceph-mon[29082]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:34:05.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:05 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:05.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:05 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:34:05.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:05 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:34:05.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:05 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:34:05.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:05 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:05.663 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:34:05.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:05 smithi138 ceph-mon[34885]: pgmap v111: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:05.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:05 smithi138 ceph-mon[34885]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:34:05.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:05 smithi138 ceph-mon[34885]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:34:05.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:05 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:05.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:05 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:34:05.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:05 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:34:05.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:05 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:34:05.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:05 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:06.696 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:06 smithi119 ceph-mon[29082]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:34:06.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:06 smithi138 ceph-mon[34885]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:a6507388dd5057528934822c0163b0c347ef1d5d 2024-05-27T09:34:07.722 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:07 smithi119 ceph-mon[29082]: pgmap v112: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:07.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:07 smithi138 ceph-mon[34885]: pgmap v112: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:08.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.233+0000 7fe846000700 1 -- 172.21.15.119:0/2247435915 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe84010b700 msgr2=0x7fe84010bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:08.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.233+0000 7fe846000700 1 --2- 172.21.15.119:0/2247435915 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe84010b700 0x7fe84010bad0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe830009910 tx=0x7fe830009c20 comp rx=0 tx=0).stop 2024-05-27T09:34:08.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.233+0000 7fe846000700 1 -- 172.21.15.119:0/2247435915 shutdown_connections 2024-05-27T09:34:08.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.233+0000 7fe846000700 1 --2- 172.21.15.119:0/2247435915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe84010c010 0x7fe840106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:08.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.233+0000 7fe846000700 1 --2- 172.21.15.119:0/2247435915 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe84010b700 0x7fe84010bad0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:08.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.233+0000 7fe846000700 1 -- 172.21.15.119:0/2247435915 >> 172.21.15.119:0/2247435915 conn(0x7fe840076000 msgr2=0x7fe840078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:08.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.233+0000 7fe846000700 1 -- 172.21.15.119:0/2247435915 shutdown_connections 2024-05-27T09:34:08.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.233+0000 7fe846000700 1 -- 172.21.15.119:0/2247435915 wait complete. 2024-05-27T09:34:08.366 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.234+0000 7fe846000700 1 Processor -- start 2024-05-27T09:34:08.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.234+0000 7fe846000700 1 -- start start 2024-05-27T09:34:08.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.235+0000 7fe846000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe84010b700 0x7fe84019c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:08.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.235+0000 7fe846000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe84010c010 0x7fe84019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:08.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.235+0000 7fe846000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84019ced0 con 0x7fe84010b700 2024-05-27T09:34:08.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.235+0000 7fe846000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8401a0c60 con 0x7fe84010c010 2024-05-27T09:34:08.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.235+0000 7fe83f400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe84010b700 0x7fe84019c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:08.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.235+0000 7fe83f400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe84010b700 0x7fe84019c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33640/0 (socket says 172.21.15.119:33640) 2024-05-27T09:34:08.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.235+0000 7fe83f400700 1 -- 172.21.15.119:0/2384160169 learned_addr learned my addr 172.21.15.119:0/2384160169 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:34:08.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.236+0000 7fe83f400700 1 -- 172.21.15.119:0/2384160169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe84010c010 msgr2=0x7fe84019c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:08.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.236+0000 7fe83ea00700 1 --2- 172.21.15.119:0/2384160169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe84010c010 0x7fe84019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:08.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.236+0000 7fe83f400700 1 --2- 172.21.15.119:0/2384160169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe84010c010 0x7fe84019c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:08.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.236+0000 7fe83f400700 1 -- 172.21.15.119:0/2384160169 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8300095f0 con 0x7fe84010b700 2024-05-27T09:34:08.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.236+0000 7fe83ea00700 1 --2- 172.21.15.119:0/2384160169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe84010c010 0x7fe84019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:34:08.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.236+0000 7fe83f400700 1 --2- 172.21.15.119:0/2384160169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe84010b700 0x7fe84019c2b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe830006010 tx=0x7fe83000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:08.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.236+0000 7fe837e00700 1 -- 172.21.15.119:0/2384160169 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe83001d070 con 0x7fe84010b700 2024-05-27T09:34:08.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.236+0000 7fe837e00700 1 -- 172.21.15.119:0/2384160169 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe830004fb0 con 0x7fe84010b700 2024-05-27T09:34:08.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.236+0000 7fe837e00700 1 -- 172.21.15.119:0/2384160169 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe830022ea0 con 0x7fe84010b700 2024-05-27T09:34:08.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.236+0000 7fe846000700 1 -- 172.21.15.119:0/2384160169 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8401a0ee0 con 0x7fe84010b700 2024-05-27T09:34:08.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.236+0000 7fe846000700 1 -- 172.21.15.119:0/2384160169 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8401a1450 con 0x7fe84010b700 2024-05-27T09:34:08.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.238+0000 7fe846000700 1 -- 172.21.15.119:0/2384160169 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe84004ea50 con 0x7fe84010b700 2024-05-27T09:34:08.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.239+0000 7fe837e00700 1 -- 172.21.15.119:0/2384160169 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7fe830022470 con 0x7fe84010b700 2024-05-27T09:34:08.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.240+0000 7fe837e00700 1 --2- 172.21.15.119:0/2384160169 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe8240786c0 0x7fe82407ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:08.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.240+0000 7fe837e00700 1 -- 172.21.15.119:0/2384160169 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe83009bcf0 con 0x7fe84010b700 2024-05-27T09:34:08.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.240+0000 7fe83ea00700 1 --2- 172.21.15.119:0/2384160169 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe8240786c0 0x7fe82407ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:08.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.241+0000 7fe83ea00700 1 --2- 172.21.15.119:0/2384160169 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe8240786c0 0x7fe82407ab70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe82c005fd0 tx=0x7fe82c005dc0 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:08.369 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.246+0000 7fe837e00700 1 -- 172.21.15.119:0/2384160169 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7fe830063e80 con 0x7fe84010b700 2024-05-27T09:34:08.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.503+0000 7fe846000700 1 -- 172.21.15.119:0/2384160169 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8401a1980 con 0x7fe8240786c0 2024-05-27T09:34:08.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.505+0000 7fe837e00700 1 -- 172.21.15.119:0/2384160169 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fe8401a1980 con 0x7fe8240786c0 2024-05-27T09:34:08.509 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.508+0000 7fe846000700 1 -- 172.21.15.119:0/2384160169 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe8240786c0 msgr2=0x7fe82407ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:08.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.508+0000 7fe846000700 1 --2- 172.21.15.119:0/2384160169 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe8240786c0 0x7fe82407ab70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe82c005fd0 tx=0x7fe82c005dc0 comp rx=0 tx=0).stop 2024-05-27T09:34:08.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.508+0000 7fe846000700 1 -- 172.21.15.119:0/2384160169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe84010b700 msgr2=0x7fe84019c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:08.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.508+0000 7fe846000700 1 --2- 172.21.15.119:0/2384160169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe84010b700 0x7fe84019c2b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe830006010 tx=0x7fe83000be20 comp rx=0 tx=0).stop 2024-05-27T09:34:08.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.509+0000 7fe846000700 1 -- 172.21.15.119:0/2384160169 shutdown_connections 2024-05-27T09:34:08.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.509+0000 7fe846000700 1 --2- 172.21.15.119:0/2384160169 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe8240786c0 0x7fe82407ab70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:08.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.509+0000 7fe846000700 1 --2- 172.21.15.119:0/2384160169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe84010b700 0x7fe84019c2b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:08.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.509+0000 7fe846000700 1 --2- 172.21.15.119:0/2384160169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe84010c010 0x7fe84019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:08.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.509+0000 7fe846000700 1 -- 172.21.15.119:0/2384160169 >> 172.21.15.119:0/2384160169 conn(0x7fe840076000 msgr2=0x7fe840076880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:08.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.509+0000 7fe846000700 1 -- 172.21.15.119:0/2384160169 shutdown_connections 2024-05-27T09:34:08.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.509+0000 7fe846000700 1 -- 172.21.15.119:0/2384160169 wait complete. 2024-05-27T09:34:08.527 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:34:08.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.672+0000 7f739fe00700 1 -- 172.21.15.119:0/364137094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a00ff1d0 msgr2=0x7f73a00ff5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:08.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.672+0000 7f739fe00700 1 --2- 172.21.15.119:0/364137094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a00ff1d0 0x7f73a00ff5a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7390009910 tx=0x7f7390009c20 comp rx=0 tx=0).stop 2024-05-27T09:34:08.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.672+0000 7f739fe00700 1 -- 172.21.15.119:0/364137094 shutdown_connections 2024-05-27T09:34:08.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.672+0000 7f739fe00700 1 --2- 172.21.15.119:0/364137094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73a00ffb70 0x7f73a0103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:08.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.672+0000 7f739fe00700 1 --2- 172.21.15.119:0/364137094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a00ff1d0 0x7f73a00ff5a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:08.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.672+0000 7f739fe00700 1 -- 172.21.15.119:0/364137094 >> 172.21.15.119:0/364137094 conn(0x7f73a0074d00 msgr2=0x7f73a0075100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:08.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.672+0000 7f739fe00700 1 -- 172.21.15.119:0/364137094 shutdown_connections 2024-05-27T09:34:08.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.672+0000 7f739fe00700 1 -- 172.21.15.119:0/364137094 wait complete. 2024-05-27T09:34:08.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.673+0000 7f739fe00700 1 Processor -- start 2024-05-27T09:34:08.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.673+0000 7f739fe00700 1 -- start start 2024-05-27T09:34:08.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.674+0000 7f739fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a00ff1d0 0x7f73a0100b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:08.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.674+0000 7f739fe00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73a00ffb70 0x7f73a0101050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:08.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.674+0000 7f739fe00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a0101590 con 0x7f73a00ff1d0 2024-05-27T09:34:08.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.674+0000 7f739fe00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a01016d0 con 0x7f73a00ffb70 2024-05-27T09:34:08.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.674+0000 7f739ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a00ff1d0 0x7f73a0100b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:08.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.674+0000 7f739ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a00ff1d0 0x7f73a0100b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33664/0 (socket says 172.21.15.119:33664) 2024-05-27T09:34:08.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.674+0000 7f739ea00700 1 -- 172.21.15.119:0/1409948841 learned_addr learned my addr 172.21.15.119:0/1409948841 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:34:08.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.674+0000 7f739e000700 1 --2- 172.21.15.119:0/1409948841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73a00ffb70 0x7f73a0101050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:08.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.675+0000 7f739ea00700 1 -- 172.21.15.119:0/1409948841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73a00ffb70 msgr2=0x7f73a0101050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:08.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.675+0000 7f739ea00700 1 --2- 172.21.15.119:0/1409948841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73a00ffb70 0x7f73a0101050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:08.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.675+0000 7f739ea00700 1 -- 172.21.15.119:0/1409948841 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73900095f0 con 0x7f73a00ff1d0 2024-05-27T09:34:08.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.675+0000 7f739ea00700 1 --2- 172.21.15.119:0/1409948841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a00ff1d0 0x7f73a0100b10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f739000b5c0 tx=0x7f739000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:08.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.675+0000 7f73a4a00700 1 -- 172.21.15.119:0/1409948841 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f739001d070 con 0x7f73a00ff1d0 2024-05-27T09:34:08.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.675+0000 7f73a4a00700 1 -- 172.21.15.119:0/1409948841 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f739000f460 con 0x7f73a00ff1d0 2024-05-27T09:34:08.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.675+0000 7f739fe00700 1 -- 172.21.15.119:0/1409948841 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a0101950 con 0x7f73a00ff1d0 2024-05-27T09:34:08.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.675+0000 7f73a4a00700 1 -- 172.21.15.119:0/1409948841 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7390017610 con 0x7f73a00ff1d0 2024-05-27T09:34:08.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.676+0000 7f739fe00700 1 -- 172.21.15.119:0/1409948841 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a01a2060 con 0x7f73a00ff1d0 2024-05-27T09:34:08.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.677+0000 7f739fe00700 1 -- 172.21.15.119:0/1409948841 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a004f2a0 con 0x7f73a00ff1d0 2024-05-27T09:34:08.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.678+0000 7f73a4a00700 1 -- 172.21.15.119:0/1409948841 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7f739000f5d0 con 0x7f73a00ff1d0 2024-05-27T09:34:08.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.679+0000 7f73a4a00700 1 --2- 172.21.15.119:0/1409948841 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f73840786c0 0x7f738407ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:08.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.680+0000 7f73a4a00700 1 -- 172.21.15.119:0/1409948841 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f739009bd90 con 0x7f73a00ff1d0 2024-05-27T09:34:08.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.680+0000 7f739e000700 1 --2- 172.21.15.119:0/1409948841 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f73840786c0 0x7f738407ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:08.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.680+0000 7f739e000700 1 --2- 172.21.15.119:0/1409948841 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f73840786c0 0x7f738407ab70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f738c005fd0 tx=0x7f738c00a560 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:08.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.684+0000 7f73a4a00700 1 -- 172.21.15.119:0/1409948841 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f7390063e70 con 0x7f73a00ff1d0 2024-05-27T09:34:08.940 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.938+0000 7f739fe00700 1 -- 172.21.15.119:0/1409948841 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f73a00fc9b0 con 0x7f73840786c0 2024-05-27T09:34:08.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.941+0000 7f73a4a00700 1 -- 172.21.15.119:0/1409948841 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f73a00fc9b0 con 0x7f73840786c0 2024-05-27T09:34:08.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.943+0000 7f739fe00700 1 -- 172.21.15.119:0/1409948841 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f73840786c0 msgr2=0x7f738407ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:08.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.943+0000 7f739fe00700 1 --2- 172.21.15.119:0/1409948841 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f73840786c0 0x7f738407ab70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f738c005fd0 tx=0x7f738c00a560 comp rx=0 tx=0).stop 2024-05-27T09:34:08.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.943+0000 7f739fe00700 1 -- 172.21.15.119:0/1409948841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a00ff1d0 msgr2=0x7f73a0100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:08.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.943+0000 7f739fe00700 1 --2- 172.21.15.119:0/1409948841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a00ff1d0 0x7f73a0100b10 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f739000b5c0 tx=0x7f739000be00 comp rx=0 tx=0).stop 2024-05-27T09:34:08.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.943+0000 7f739fe00700 1 -- 172.21.15.119:0/1409948841 shutdown_connections 2024-05-27T09:34:08.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.943+0000 7f739fe00700 1 --2- 172.21.15.119:0/1409948841 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f73840786c0 0x7f738407ab70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:08.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.943+0000 7f739fe00700 1 --2- 172.21.15.119:0/1409948841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a00ff1d0 0x7f73a0100b10 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:08.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.943+0000 7f739fe00700 1 --2- 172.21.15.119:0/1409948841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73a00ffb70 0x7f73a0101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:08.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.943+0000 7f739fe00700 1 -- 172.21.15.119:0/1409948841 >> 172.21.15.119:0/1409948841 conn(0x7f73a0074d00 msgr2=0x7f73a0103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:08.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.944+0000 7f739fe00700 1 -- 172.21.15.119:0/1409948841 shutdown_connections 2024-05-27T09:34:08.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:08.944+0000 7f739fe00700 1 -- 172.21.15.119:0/1409948841 wait complete. 2024-05-27T09:34:09.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.113+0000 7fe04ea00700 1 -- 172.21.15.119:0/1046792231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe048073a00 msgr2=0x7fe048110c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:09.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.113+0000 7fe04ea00700 1 --2- 172.21.15.119:0/1046792231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe048073a00 0x7fe048110c20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe034009910 tx=0x7fe034009c20 comp rx=0 tx=0).stop 2024-05-27T09:34:09.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.113+0000 7fe04ea00700 1 -- 172.21.15.119:0/1046792231 shutdown_connections 2024-05-27T09:34:09.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.113+0000 7fe04ea00700 1 --2- 172.21.15.119:0/1046792231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe048073a00 0x7fe048110c20 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:09.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.113+0000 7fe04ea00700 1 --2- 172.21.15.119:0/1046792231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0480730f0 0x7fe0480734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:09.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.113+0000 7fe04ea00700 1 -- 172.21.15.119:0/1046792231 >> 172.21.15.119:0/1046792231 conn(0x7fe0480fbc30 msgr2=0x7fe0480fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:09.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.113+0000 7fe04ea00700 1 -- 172.21.15.119:0/1046792231 shutdown_connections 2024-05-27T09:34:09.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.113+0000 7fe04ea00700 1 -- 172.21.15.119:0/1046792231 wait complete. 2024-05-27T09:34:09.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.114+0000 7fe04ea00700 1 Processor -- start 2024-05-27T09:34:09.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.115+0000 7fe04ea00700 1 -- start start 2024-05-27T09:34:09.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.115+0000 7fe04ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0480730f0 0x7fe04819c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:09.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.115+0000 7fe04ea00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe048073a00 0x7fe04819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:09.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.115+0000 7fe04ea00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04819d040 con 0x7fe0480730f0 2024-05-27T09:34:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.115+0000 7fe04ea00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0481a0dd0 con 0x7fe048073a00 2024-05-27T09:34:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.116+0000 7fe047e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0480730f0 0x7fe04819c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.116+0000 7fe047e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0480730f0 0x7fe04819c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40120/0 (socket says 172.21.15.119:40120) 2024-05-27T09:34:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.116+0000 7fe047e00700 1 -- 172.21.15.119:0/3097489007 learned_addr learned my addr 172.21.15.119:0/3097489007 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:34:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.116+0000 7fe047400700 1 --2- 172.21.15.119:0/3097489007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe048073a00 0x7fe04819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.116+0000 7fe047e00700 1 -- 172.21.15.119:0/3097489007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe048073a00 msgr2=0x7fe04819c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.116+0000 7fe047e00700 1 --2- 172.21.15.119:0/3097489007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe048073a00 0x7fe04819c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.116+0000 7fe047e00700 1 -- 172.21.15.119:0/3097489007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0340095f0 con 0x7fe0480730f0 2024-05-27T09:34:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.116+0000 7fe047400700 1 --2- 172.21.15.119:0/3097489007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe048073a00 0x7fe04819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:34:09.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.117+0000 7fe047e00700 1 --2- 172.21.15.119:0/3097489007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0480730f0 0x7fe04819c420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe03800b4b0 tx=0x7fe03800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:09.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.121+0000 7fe044c00700 1 -- 172.21.15.119:0/3097489007 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe038011800 con 0x7fe0480730f0 2024-05-27T09:34:09.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.121+0000 7fe044c00700 1 -- 172.21.15.119:0/3097489007 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe038011e20 con 0x7fe0480730f0 2024-05-27T09:34:09.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.121+0000 7fe044c00700 1 -- 172.21.15.119:0/3097489007 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe03800f2e0 con 0x7fe0480730f0 2024-05-27T09:34:09.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.122+0000 7fe04ea00700 1 -- 172.21.15.119:0/3097489007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0481a10b0 con 0x7fe0480730f0 2024-05-27T09:34:09.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.122+0000 7fe04ea00700 1 -- 172.21.15.119:0/3097489007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0481a1600 con 0x7fe0480730f0 2024-05-27T09:34:09.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.123+0000 7fe04ea00700 1 -- 172.21.15.119:0/3097489007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe04810e3a0 con 0x7fe0480730f0 2024-05-27T09:34:09.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.125+0000 7fe044c00700 1 -- 172.21.15.119:0/3097489007 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7fe038010380 con 0x7fe0480730f0 2024-05-27T09:34:09.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.126+0000 7fe044c00700 1 --2- 172.21.15.119:0/3097489007 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe02c078720 0x7fe02c07abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:09.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.126+0000 7fe044c00700 1 -- 172.21.15.119:0/3097489007 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe03809a220 con 0x7fe0480730f0 2024-05-27T09:34:09.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.126+0000 7fe047400700 1 --2- 172.21.15.119:0/3097489007 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe02c078720 0x7fe02c07abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:09.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.127+0000 7fe047400700 1 --2- 172.21.15.119:0/3097489007 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe02c078720 0x7fe02c07abd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe034006010 tx=0x7fe03400b540 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:09.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.131+0000 7fe044c00700 1 -- 172.21.15.119:0/3097489007 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7fe038062430 con 0x7fe0480730f0 2024-05-27T09:34:09.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.369+0000 7fe04ea00700 1 -- 172.21.15.119:0/3097489007 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0481a1a10 con 0x7fe02c078720 2024-05-27T09:34:09.383 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.382+0000 7fe044c00700 1 -- 172.21.15.119:0/3097489007 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fe0481a1a10 con 0x7fe02c078720 2024-05-27T09:34:09.384 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:34:09.384 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (100s) 57s ago 15m 13.8M - 0.25.0 c8568f914cd2 6b90564dcef7 2024-05-27T09:34:09.384 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (15m) 57s ago 15m 8082k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:34:09.384 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (14m) 2m ago 14m 7834k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:34:09.384 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (15m) 57s ago 15m 6665k - 18.2.0 dc2bc1663786 e5fafde44183 2024-05-27T09:34:09.384 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (14m) 2m ago 14m 6660k - 18.2.0 dc2bc1663786 46956ab99bab 2024-05-27T09:34:09.384 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (65s) 57s ago 14m 51.1M - 9.4.12 2bacad6d85d8 b3abdab08db4 2024-05-27T09:34:09.384 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (8m) 57s ago 8m 15.5M - 18.2.0 dc2bc1663786 93b21713fb1e 2024-05-27T09:34:09.385 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (8m) 57s ago 8m 129M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:34:09.385 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (8m) 2m ago 8m 15.0M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:34:09.385 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (8m) 2m ago 8m 128M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:34:09.385 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (3m) 57s ago 17m 594M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:34:09.385 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (2m) 2m ago 13m 529M - 19.0.0-3868-ga6507388 850a06a84d28 2fc3a80696fa 2024-05-27T09:34:09.385 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (17m) 57s ago 17m 59.8M 2048M 18.2.0 dc2bc1663786 ba998b789a1c 2024-05-27T09:34:09.385 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (13m) 2m ago 13m 47.8M 2048M 18.2.0 dc2bc1663786 60216fd67078 2024-05-27T09:34:09.385 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (2m) 57s ago 15m 7495k - 1.5.0 0da6a335fe13 709c37815cff 2024-05-27T09:34:09.385 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (2m) 2m ago 14m 2889k - 1.5.0 0da6a335fe13 999047a1a5bf 2024-05-27T09:34:09.385 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (12m) 57s ago 12m 224M 1216M 18.2.0 dc2bc1663786 825bd1392508 2024-05-27T09:34:09.385 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (12m) 57s ago 12m 251M 1216M 18.2.0 dc2bc1663786 fce9819903d9 2024-05-27T09:34:09.385 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (11m) 57s ago 11m 190M 1216M 18.2.0 dc2bc1663786 3955a6112c1c 2024-05-27T09:34:09.385 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (11m) 2m ago 11m 268M 2240M 18.2.0 dc2bc1663786 693c92220622 2024-05-27T09:34:09.385 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (10m) 2m ago 10m 246M 2240M 18.2.0 dc2bc1663786 7559e658e917 2024-05-27T09:34:09.386 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (9m) 2m ago 9m 229M 2240M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:34:09.386 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (2m) 57s ago 14m 57.1M - 2.43.0 a07b618ecd1d cba50454ed21 2024-05-27T09:34:09.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.385+0000 7fe04ea00700 1 -- 172.21.15.119:0/3097489007 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe02c078720 msgr2=0x7fe02c07abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:09.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.385+0000 7fe04ea00700 1 --2- 172.21.15.119:0/3097489007 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe02c078720 0x7fe02c07abd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe034006010 tx=0x7fe03400b540 comp rx=0 tx=0).stop 2024-05-27T09:34:09.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.385+0000 7fe04ea00700 1 -- 172.21.15.119:0/3097489007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0480730f0 msgr2=0x7fe04819c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:09.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.385+0000 7fe04ea00700 1 --2- 172.21.15.119:0/3097489007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0480730f0 0x7fe04819c420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe03800b4b0 tx=0x7fe03800b870 comp rx=0 tx=0).stop 2024-05-27T09:34:09.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.385+0000 7fe04ea00700 1 -- 172.21.15.119:0/3097489007 shutdown_connections 2024-05-27T09:34:09.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.385+0000 7fe04ea00700 1 --2- 172.21.15.119:0/3097489007 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe02c078720 0x7fe02c07abd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:09.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.385+0000 7fe04ea00700 1 --2- 172.21.15.119:0/3097489007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0480730f0 0x7fe04819c420 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:09.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.385+0000 7fe04ea00700 1 --2- 172.21.15.119:0/3097489007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe048073a00 0x7fe04819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:09.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.385+0000 7fe04ea00700 1 -- 172.21.15.119:0/3097489007 >> 172.21.15.119:0/3097489007 conn(0x7fe0480fbc30 msgr2=0x7fe048102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:09.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.385+0000 7fe04ea00700 1 -- 172.21.15.119:0/3097489007 shutdown_connections 2024-05-27T09:34:09.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.385+0000 7fe04ea00700 1 -- 172.21.15.119:0/3097489007 wait complete. 2024-05-27T09:34:09.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.546+0000 7fa741e00700 1 -- 172.21.15.119:0/2391140506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa73c073a00 msgr2=0x7fa73c110c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:09.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.546+0000 7fa741e00700 1 --2- 172.21.15.119:0/2391140506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa73c073a00 0x7fa73c110c20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa728009910 tx=0x7fa728009c20 comp rx=0 tx=0).stop 2024-05-27T09:34:09.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.546+0000 7fa741e00700 1 -- 172.21.15.119:0/2391140506 shutdown_connections 2024-05-27T09:34:09.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.546+0000 7fa741e00700 1 --2- 172.21.15.119:0/2391140506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa73c073a00 0x7fa73c110c20 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:09.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.546+0000 7fa741e00700 1 --2- 172.21.15.119:0/2391140506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa73c0730f0 0x7fa73c0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:09.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.546+0000 7fa741e00700 1 -- 172.21.15.119:0/2391140506 >> 172.21.15.119:0/2391140506 conn(0x7fa73c0fbc30 msgr2=0x7fa73c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:09.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.546+0000 7fa741e00700 1 -- 172.21.15.119:0/2391140506 shutdown_connections 2024-05-27T09:34:09.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.546+0000 7fa741e00700 1 -- 172.21.15.119:0/2391140506 wait complete. 2024-05-27T09:34:09.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.548+0000 7fa741e00700 1 Processor -- start 2024-05-27T09:34:09.549 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.548+0000 7fa741e00700 1 -- start start 2024-05-27T09:34:09.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.548+0000 7fa741e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa73c0730f0 0x7fa73c19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:09.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.548+0000 7fa741e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa73c073a00 0x7fa73c19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:09.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.548+0000 7fa741e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c19cf20 con 0x7fa73c0730f0 2024-05-27T09:34:09.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.548+0000 7fa741e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c1a0cb0 con 0x7fa73c073a00 2024-05-27T09:34:09.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.549+0000 7fa73aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa73c0730f0 0x7fa73c19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:09.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.549+0000 7fa73aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa73c0730f0 0x7fa73c19c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40132/0 (socket says 172.21.15.119:40132) 2024-05-27T09:34:09.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.549+0000 7fa73aa00700 1 -- 172.21.15.119:0/4018479372 learned_addr learned my addr 172.21.15.119:0/4018479372 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:34:09.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.549+0000 7fa73a000700 1 --2- 172.21.15.119:0/4018479372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa73c073a00 0x7fa73c19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:09.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.549+0000 7fa73aa00700 1 -- 172.21.15.119:0/4018479372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa73c073a00 msgr2=0x7fa73c19c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:09.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.549+0000 7fa73aa00700 1 --2- 172.21.15.119:0/4018479372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa73c073a00 0x7fa73c19c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:09.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.549+0000 7fa73aa00700 1 -- 172.21.15.119:0/4018479372 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7280095f0 con 0x7fa73c0730f0 2024-05-27T09:34:09.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.549+0000 7fa73aa00700 1 --2- 172.21.15.119:0/4018479372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa73c0730f0 0x7fa73c19c300 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00d8d0 tx=0x7fa72c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:09.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.550+0000 7fa740a00700 1 -- 172.21.15.119:0/4018479372 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa72c009940 con 0x7fa73c0730f0 2024-05-27T09:34:09.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.550+0000 7fa741e00700 1 -- 172.21.15.119:0/4018479372 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73c1a0f90 con 0x7fa73c0730f0 2024-05-27T09:34:09.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.550+0000 7fa740a00700 1 -- 172.21.15.119:0/4018479372 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa72c010460 con 0x7fa73c0730f0 2024-05-27T09:34:09.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.550+0000 7fa740a00700 1 -- 172.21.15.119:0/4018479372 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa72c00f5f0 con 0x7fa73c0730f0 2024-05-27T09:34:09.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.550+0000 7fa741e00700 1 -- 172.21.15.119:0/4018479372 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73c1a14e0 con 0x7fa73c0730f0 2024-05-27T09:34:09.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.551+0000 7fa741e00700 1 -- 172.21.15.119:0/4018479372 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa73c10e3a0 con 0x7fa73c0730f0 2024-05-27T09:34:09.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.553+0000 7fa740a00700 1 -- 172.21.15.119:0/4018479372 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7fa72c0105d0 con 0x7fa73c0730f0 2024-05-27T09:34:09.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.553+0000 7fa740a00700 1 --2- 172.21.15.119:0/4018479372 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fa7200786c0 0x7fa72007ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:09.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.554+0000 7fa740a00700 1 -- 172.21.15.119:0/4018479372 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa72c09a880 con 0x7fa73c0730f0 2024-05-27T09:34:09.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.554+0000 7fa73a000700 1 --2- 172.21.15.119:0/4018479372 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fa7200786c0 0x7fa72007ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:09.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.555+0000 7fa73a000700 1 --2- 172.21.15.119:0/4018479372 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fa7200786c0 0x7fa72007ab70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa72800b5c0 tx=0x7fa728005960 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:09.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.557+0000 7fa740a00700 1 -- 172.21.15.119:0/4018479372 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7fa72c062a10 con 0x7fa73c0730f0 2024-05-27T09:34:09.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:09 smithi119 ceph-mon[29082]: pgmap v113: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:09.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:09 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:09.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:09 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:34:09.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:09 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:34:09.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:09 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:09.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:09 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-27T09:34:09.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:09 smithi138 ceph-mon[34885]: pgmap v113: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:09.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:09 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:09.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:09 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:34:09.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:09 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:34:09.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:09 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:09.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:09 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-27T09:34:09.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.891+0000 7fa741e00700 1 -- 172.21.15.119:0/4018479372 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa73c1a1800 con 0x7fa73c0730f0 2024-05-27T09:34:09.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.892+0000 7fa740a00700 1 -- 172.21.15.119:0/4018479372 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fa72c0623b0 con 0x7fa73c0730f0 2024-05-27T09:34:09.894 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:34:09.894 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:34:09.894 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-05-27T09:34:09.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:34:09.894 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:34:09.894 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:34:09.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:34:09.894 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:34:09.894 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-05-27T09:34:09.894 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:34:09.894 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:34:09.895 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-05-27T09:34:09.895 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:34:09.895 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:34:09.895 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-05-27T09:34:09.895 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:34:09.895 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:34:09.895 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:34:09.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.894+0000 7fa741e00700 1 -- 172.21.15.119:0/4018479372 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fa7200786c0 msgr2=0x7fa72007ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:09.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.894+0000 7fa741e00700 1 --2- 172.21.15.119:0/4018479372 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fa7200786c0 0x7fa72007ab70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa72800b5c0 tx=0x7fa728005960 comp rx=0 tx=0).stop 2024-05-27T09:34:09.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.894+0000 7fa741e00700 1 -- 172.21.15.119:0/4018479372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa73c0730f0 msgr2=0x7fa73c19c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:09.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.894+0000 7fa741e00700 1 --2- 172.21.15.119:0/4018479372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa73c0730f0 0x7fa73c19c300 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00d8d0 tx=0x7fa72c00dc90 comp rx=0 tx=0).stop 2024-05-27T09:34:09.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.895+0000 7fa741e00700 1 -- 172.21.15.119:0/4018479372 shutdown_connections 2024-05-27T09:34:09.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.895+0000 7fa741e00700 1 --2- 172.21.15.119:0/4018479372 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fa7200786c0 0x7fa72007ab70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:09.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.895+0000 7fa741e00700 1 --2- 172.21.15.119:0/4018479372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa73c0730f0 0x7fa73c19c300 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:09.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.895+0000 7fa741e00700 1 --2- 172.21.15.119:0/4018479372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa73c073a00 0x7fa73c19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:09.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.895+0000 7fa741e00700 1 -- 172.21.15.119:0/4018479372 >> 172.21.15.119:0/4018479372 conn(0x7fa73c0fbc30 msgr2=0x7fa73c102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:09.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.895+0000 7fa741e00700 1 -- 172.21.15.119:0/4018479372 shutdown_connections 2024-05-27T09:34:09.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:09.895+0000 7fa741e00700 1 -- 172.21.15.119:0/4018479372 wait complete. 2024-05-27T09:34:10.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.044+0000 7fd3af800700 1 -- 172.21.15.119:0/3040431032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3a8101c30 msgr2=0x7fd3a810a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:10.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.044+0000 7fd3af800700 1 --2- 172.21.15.119:0/3040431032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3a8101c30 0x7fd3a810a130 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd394009910 tx=0x7fd394009c20 comp rx=0 tx=0).stop 2024-05-27T09:34:10.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.044+0000 7fd3af800700 1 -- 172.21.15.119:0/3040431032 shutdown_connections 2024-05-27T09:34:10.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.044+0000 7fd3af800700 1 --2- 172.21.15.119:0/3040431032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3a8101c30 0x7fd3a810a130 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:10.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.044+0000 7fd3af800700 1 --2- 172.21.15.119:0/3040431032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3a8101320 0x7fd3a81016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:10.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.044+0000 7fd3af800700 1 -- 172.21.15.119:0/3040431032 >> 172.21.15.119:0/3040431032 conn(0x7fd3a80fab80 msgr2=0x7fd3a80fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:10.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.044+0000 7fd3af800700 1 -- 172.21.15.119:0/3040431032 shutdown_connections 2024-05-27T09:34:10.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.044+0000 7fd3af800700 1 -- 172.21.15.119:0/3040431032 wait complete. 2024-05-27T09:34:10.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.045+0000 7fd3af800700 1 Processor -- start 2024-05-27T09:34:10.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.045+0000 7fd3af800700 1 -- start start 2024-05-27T09:34:10.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.046+0000 7fd3af800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3a8101320 0x7fd3a819bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:10.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.046+0000 7fd3af800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3a8101c30 0x7fd3a819bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:10.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.046+0000 7fd3af800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a819c680 con 0x7fd3a8101320 2024-05-27T09:34:10.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.046+0000 7fd3af800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a8195b30 con 0x7fd3a8101c30 2024-05-27T09:34:10.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.046+0000 7fd3ace00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3a8101320 0x7fd3a819bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:10.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.046+0000 7fd3ace00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3a8101320 0x7fd3a819bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40154/0 (socket says 172.21.15.119:40154) 2024-05-27T09:34:10.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.046+0000 7fd3ace00700 1 -- 172.21.15.119:0/1007947460 learned_addr learned my addr 172.21.15.119:0/1007947460 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:34:10.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.046+0000 7fd3a7e00700 1 --2- 172.21.15.119:0/1007947460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3a8101c30 0x7fd3a819bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:10.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.046+0000 7fd3ace00700 1 -- 172.21.15.119:0/1007947460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3a8101c30 msgr2=0x7fd3a819bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:10.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.046+0000 7fd3ace00700 1 --2- 172.21.15.119:0/1007947460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3a8101c30 0x7fd3a819bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:10.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.046+0000 7fd3ace00700 1 -- 172.21.15.119:0/1007947460 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3940095f0 con 0x7fd3a8101320 2024-05-27T09:34:10.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.046+0000 7fd3a7e00700 1 --2- 172.21.15.119:0/1007947460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3a8101c30 0x7fd3a819bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:34:10.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.047+0000 7fd3ace00700 1 --2- 172.21.15.119:0/1007947460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3a8101320 0x7fd3a819bab0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00d8d0 tx=0x7fd39c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:10.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.047+0000 7fd3a5600700 1 -- 172.21.15.119:0/1007947460 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd39c009940 con 0x7fd3a8101320 2024-05-27T09:34:10.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.047+0000 7fd3af800700 1 -- 172.21.15.119:0/1007947460 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a8195e10 con 0x7fd3a8101320 2024-05-27T09:34:10.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.047+0000 7fd3a5600700 1 -- 172.21.15.119:0/1007947460 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd39c010460 con 0x7fd3a8101320 2024-05-27T09:34:10.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.047+0000 7fd3a5600700 1 -- 172.21.15.119:0/1007947460 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd39c00f5f0 con 0x7fd3a8101320 2024-05-27T09:34:10.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.047+0000 7fd3af800700 1 -- 172.21.15.119:0/1007947460 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a8196360 con 0x7fd3a8101320 2024-05-27T09:34:10.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.048+0000 7fd3af800700 1 -- 172.21.15.119:0/1007947460 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a81077a0 con 0x7fd3a8101320 2024-05-27T09:34:10.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.049+0000 7fd3a5600700 1 -- 172.21.15.119:0/1007947460 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7fd39c0105d0 con 0x7fd3a8101320 2024-05-27T09:34:10.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.050+0000 7fd3a5600700 1 --2- 172.21.15.119:0/1007947460 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fd3900786c0 0x7fd39007ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:10.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.050+0000 7fd3a5600700 1 -- 172.21.15.119:0/1007947460 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd39c09a8b0 con 0x7fd3a8101320 2024-05-27T09:34:10.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.051+0000 7fd3a7e00700 1 --2- 172.21.15.119:0/1007947460 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fd3900786c0 0x7fd39007ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:10.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.052+0000 7fd3a7e00700 1 --2- 172.21.15.119:0/1007947460 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fd3900786c0 0x7fd39007ab70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd39400b5c0 tx=0x7fd3940053a0 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:10.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.055+0000 7fd3a5600700 1 -- 172.21.15.119:0/1007947460 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7fd39c062910 con 0x7fd3a8101320 2024-05-27T09:34:10.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.332+0000 7fd3af800700 1 -- 172.21.15.119:0/1007947460 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd3a8196640 con 0x7fd3a8101320 2024-05-27T09:34:10.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.333+0000 7fd3a5600700 1 -- 172.21.15.119:0/1007947460 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1836 (secure 0 0 0) 0x7fd39c0622b0 con 0x7fd3a8101320 2024-05-27T09:34:10.335 INFO:teuthology.orchestra.run.smithi119.stdout:e22 2024-05-27T09:34:10.335 INFO:teuthology.orchestra.run.smithi119.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T09:34:10.335 INFO:teuthology.orchestra.run.smithi119.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:34:10.335 INFO:teuthology.orchestra.run.smithi119.stdout:legacy client fscid: 1 2024-05-27T09:34:10.335 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:34:10.335 INFO:teuthology.orchestra.run.smithi119.stdout:Filesystem 'cephfs' (1) 2024-05-27T09:34:10.335 INFO:teuthology.orchestra.run.smithi119.stdout:fs_name cephfs 2024-05-27T09:34:10.335 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 22 2024-05-27T09:34:10.335 INFO:teuthology.orchestra.run.smithi119.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-27T09:34:10.335 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-05-27T09:25:53.387388+0000 2024-05-27T09:34:10.336 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2024-05-27T09:34:01.250304+0000 2024-05-27T09:34:10.336 INFO:teuthology.orchestra.run.smithi119.stdout:tableserver 0 2024-05-27T09:34:10.336 INFO:teuthology.orchestra.run.smithi119.stdout:root 0 2024-05-27T09:34:10.336 INFO:teuthology.orchestra.run.smithi119.stdout:session_timeout 60 2024-05-27T09:34:10.336 INFO:teuthology.orchestra.run.smithi119.stdout:session_autoclose 300 2024-05-27T09:34:10.336 INFO:teuthology.orchestra.run.smithi119.stdout:max_file_size 1099511627776 2024-05-27T09:34:10.336 INFO:teuthology.orchestra.run.smithi119.stdout:required_client_features {} 2024-05-27T09:34:10.336 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure 0 2024-05-27T09:34:10.336 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure_osd_epoch 46 2024-05-27T09:34:10.336 INFO:teuthology.orchestra.run.smithi119.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:34:10.336 INFO:teuthology.orchestra.run.smithi119.stdout:max_mds 2 2024-05-27T09:34:10.336 INFO:teuthology.orchestra.run.smithi119.stdout:in 0,1 2024-05-27T09:34:10.337 INFO:teuthology.orchestra.run.smithi119.stdout:up {0=14472,1=24241} 2024-05-27T09:34:10.337 INFO:teuthology.orchestra.run.smithi119.stdout:failed 2024-05-27T09:34:10.337 INFO:teuthology.orchestra.run.smithi119.stdout:damaged 2024-05-27T09:34:10.337 INFO:teuthology.orchestra.run.smithi119.stdout:stopped 2024-05-27T09:34:10.337 INFO:teuthology.orchestra.run.smithi119.stdout:data_pools [3] 2024-05-27T09:34:10.337 INFO:teuthology.orchestra.run.smithi119.stdout:metadata_pool 2 2024-05-27T09:34:10.337 INFO:teuthology.orchestra.run.smithi119.stdout:inline_data enabled 2024-05-27T09:34:10.337 INFO:teuthology.orchestra.run.smithi119.stdout:balancer 2024-05-27T09:34:10.337 INFO:teuthology.orchestra.run.smithi119.stdout:bal_rank_mask -1 2024-05-27T09:34:10.337 INFO:teuthology.orchestra.run.smithi119.stdout:standby_count_wanted 1 2024-05-27T09:34:10.338 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.ppmzzk{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:34:10.338 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.ndfkhd{1:24241} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/897058594,v1:172.21.15.138:6825/897058594] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:34:10.338 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:34:10.338 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:34:10.338 INFO:teuthology.orchestra.run.smithi119.stdout:Standby daemons: 2024-05-27T09:34:10.338 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:34:10.338 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.nghcey{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.119:6826/413736191,v1:172.21.15.119:6827/413736191] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:34:10.338 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.alszpk{-1:24253} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/2345607053,v1:172.21.15.138:6827/2345607053] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:34:10.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.337+0000 7fd3af800700 1 -- 172.21.15.119:0/1007947460 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fd3900786c0 msgr2=0x7fd39007ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:10.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.337+0000 7fd3af800700 1 --2- 172.21.15.119:0/1007947460 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fd3900786c0 0x7fd39007ab70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd39400b5c0 tx=0x7fd3940053a0 comp rx=0 tx=0).stop 2024-05-27T09:34:10.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.337+0000 7fd3af800700 1 -- 172.21.15.119:0/1007947460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3a8101320 msgr2=0x7fd3a819bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:10.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.337+0000 7fd3af800700 1 --2- 172.21.15.119:0/1007947460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3a8101320 0x7fd3a819bab0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00d8d0 tx=0x7fd39c00dc90 comp rx=0 tx=0).stop 2024-05-27T09:34:10.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.337+0000 7fd3af800700 1 -- 172.21.15.119:0/1007947460 shutdown_connections 2024-05-27T09:34:10.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.337+0000 7fd3af800700 1 --2- 172.21.15.119:0/1007947460 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fd3900786c0 0x7fd39007ab70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:10.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.337+0000 7fd3af800700 1 --2- 172.21.15.119:0/1007947460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3a8101320 0x7fd3a819bab0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:10.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.337+0000 7fd3af800700 1 --2- 172.21.15.119:0/1007947460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3a8101c30 0x7fd3a819bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:10.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.337+0000 7fd3af800700 1 -- 172.21.15.119:0/1007947460 >> 172.21.15.119:0/1007947460 conn(0x7fd3a80fab80 msgr2=0x7fd3a81048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:10.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.337+0000 7fd3af800700 1 -- 172.21.15.119:0/1007947460 shutdown_connections 2024-05-27T09:34:10.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.337+0000 7fd3af800700 1 -- 172.21.15.119:0/1007947460 wait complete. 2024-05-27T09:34:10.340 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 22 2024-05-27T09:34:10.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.494+0000 7f8813200700 1 -- 172.21.15.119:0/2173768003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f880c10c010 msgr2=0x7f880c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:10.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.494+0000 7f8813200700 1 --2- 172.21.15.119:0/2173768003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f880c10c010 0x7f880c106b80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009910 tx=0x7f87f8009c20 comp rx=0 tx=0).stop 2024-05-27T09:34:10.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.495+0000 7f8813200700 1 -- 172.21.15.119:0/2173768003 shutdown_connections 2024-05-27T09:34:10.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.495+0000 7f8813200700 1 --2- 172.21.15.119:0/2173768003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f880c10c010 0x7f880c106b80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:10.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.495+0000 7f8813200700 1 --2- 172.21.15.119:0/2173768003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f880c10b700 0x7f880c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:10.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.495+0000 7f8813200700 1 -- 172.21.15.119:0/2173768003 >> 172.21.15.119:0/2173768003 conn(0x7f880c076000 msgr2=0x7f880c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:10.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.495+0000 7f8813200700 1 -- 172.21.15.119:0/2173768003 shutdown_connections 2024-05-27T09:34:10.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.495+0000 7f8813200700 1 -- 172.21.15.119:0/2173768003 wait complete. 2024-05-27T09:34:10.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.496+0000 7f8813200700 1 Processor -- start 2024-05-27T09:34:10.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.496+0000 7f8813200700 1 -- start start 2024-05-27T09:34:10.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.496+0000 7f8813200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f880c10b700 0x7f880c19c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:10.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.496+0000 7f8813200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f880c10c010 0x7f880c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:10.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.496+0000 7f8813200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880c19cfd0 con 0x7f880c10c010 2024-05-27T09:34:10.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.496+0000 7f8813200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880c1a0d60 con 0x7f880c10b700 2024-05-27T09:34:10.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.497+0000 7f880b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f880c10c010 0x7f880c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:10.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.497+0000 7f880b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f880c10c010 0x7f880c19c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40174/0 (socket says 172.21.15.119:40174) 2024-05-27T09:34:10.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.497+0000 7f880b400700 1 -- 172.21.15.119:0/3759034882 learned_addr learned my addr 172.21.15.119:0/3759034882 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:34:10.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.497+0000 7f880be00700 1 --2- 172.21.15.119:0/3759034882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f880c10b700 0x7f880c19c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:10.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.497+0000 7f880b400700 1 -- 172.21.15.119:0/3759034882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f880c10b700 msgr2=0x7f880c19c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:10.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.497+0000 7f880b400700 1 --2- 172.21.15.119:0/3759034882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f880c10b700 0x7f880c19c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:10.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.497+0000 7f880b400700 1 -- 172.21.15.119:0/3759034882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f80095f0 con 0x7f880c10c010 2024-05-27T09:34:10.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.498+0000 7f880b400700 1 --2- 172.21.15.119:0/3759034882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f880c10c010 0x7f880c19c8f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f87f800bda0 tx=0x7f87f800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:10.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.498+0000 7f8808c00700 1 -- 172.21.15.119:0/3759034882 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87f801d070 con 0x7f880c10c010 2024-05-27T09:34:10.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.498+0000 7f8808c00700 1 -- 172.21.15.119:0/3759034882 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87f8004fb0 con 0x7f880c10c010 2024-05-27T09:34:10.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.498+0000 7f8813200700 1 -- 172.21.15.119:0/3759034882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880c1a0fe0 con 0x7f880c10c010 2024-05-27T09:34:10.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.498+0000 7f8808c00700 1 -- 172.21.15.119:0/3759034882 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87f800f460 con 0x7f880c10c010 2024-05-27T09:34:10.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.498+0000 7f8813200700 1 -- 172.21.15.119:0/3759034882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880c1a1550 con 0x7f880c10c010 2024-05-27T09:34:10.501 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.499+0000 7f8813200700 1 -- 172.21.15.119:0/3759034882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f880c10ecf0 con 0x7f880c10c010 2024-05-27T09:34:10.502 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.500+0000 7f8808c00700 1 -- 172.21.15.119:0/3759034882 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7f87f8022800 con 0x7f880c10c010 2024-05-27T09:34:10.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.501+0000 7f8808c00700 1 --2- 172.21.15.119:0/3759034882 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f87f00786c0 0x7f87f007ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:10.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.502+0000 7f8808c00700 1 -- 172.21.15.119:0/3759034882 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f87f809bd90 con 0x7f880c10c010 2024-05-27T09:34:10.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.502+0000 7f880be00700 1 --2- 172.21.15.119:0/3759034882 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f87f00786c0 0x7f87f007ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:10.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.502+0000 7f880be00700 1 --2- 172.21.15.119:0/3759034882 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f87f00786c0 0x7f87f007ab70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f87fc006fd0 tx=0x7f87fc008040 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:10.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.506+0000 7f8808c00700 1 -- 172.21.15.119:0/3759034882 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7f87f8063ea0 con 0x7f880c10c010 2024-05-27T09:34:10.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:10 smithi119 ceph-mon[29082]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:34:10.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:10 smithi119 ceph-mon[29082]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:34:10.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:10 smithi119 ceph-mon[29082]: Upgrade: Target is version 19.0.0-3868-ga6507388 (squid) 2024-05-27T09:34:10.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:10 smithi119 ceph-mon[29082]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, digests ['quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6'] 2024-05-27T09:34:10.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:10 smithi119 ceph-mon[29082]: Upgrade: Setting container_image for all mgr 2024-05-27T09:34:10.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:10 smithi119 ceph-mon[29082]: from='client.14828 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:34:10.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:10 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/4018479372' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:34:10.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:10 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/1007947460' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:34:10.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.766+0000 7f8813200700 1 -- 172.21.15.119:0/3759034882 --> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f880c1a1940 con 0x7f87f00786c0 2024-05-27T09:34:10.771 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:34:10.771 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:34:10.771 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:34:10.771 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-27T09:34:10.771 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [ 2024-05-27T09:34:10.771 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr" 2024-05-27T09:34:10.771 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-05-27T09:34:10.771 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "2/23 daemons upgraded", 2024-05-27T09:34:10.772 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "Currently upgrading mon daemons", 2024-05-27T09:34:10.772 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:34:10.772 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:34:10.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.769+0000 7f8808c00700 1 -- 172.21.15.119:0/3759034882 <== mgr.14640 v2:172.21.15.119:6800/1378367028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f880c1a1940 con 0x7f87f00786c0 2024-05-27T09:34:10.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.772+0000 7f8813200700 1 -- 172.21.15.119:0/3759034882 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f87f00786c0 msgr2=0x7f87f007ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:10.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.772+0000 7f8813200700 1 --2- 172.21.15.119:0/3759034882 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f87f00786c0 0x7f87f007ab70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f87fc006fd0 tx=0x7f87fc008040 comp rx=0 tx=0).stop 2024-05-27T09:34:10.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.772+0000 7f8813200700 1 -- 172.21.15.119:0/3759034882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f880c10c010 msgr2=0x7f880c19c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:10.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.772+0000 7f8813200700 1 --2- 172.21.15.119:0/3759034882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f880c10c010 0x7f880c19c8f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f87f800bda0 tx=0x7f87f800be80 comp rx=0 tx=0).stop 2024-05-27T09:34:10.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.772+0000 7f8813200700 1 -- 172.21.15.119:0/3759034882 shutdown_connections 2024-05-27T09:34:10.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.772+0000 7f8813200700 1 --2- 172.21.15.119:0/3759034882 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7f87f00786c0 0x7f87f007ab70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:10.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.772+0000 7f8813200700 1 --2- 172.21.15.119:0/3759034882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f880c10b700 0x7f880c19c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:10.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.772+0000 7f8813200700 1 --2- 172.21.15.119:0/3759034882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f880c10c010 0x7f880c19c8f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:10.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.772+0000 7f8813200700 1 -- 172.21.15.119:0/3759034882 >> 172.21.15.119:0/3759034882 conn(0x7f880c076000 msgr2=0x7f880c076a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:10.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.773+0000 7f8813200700 1 -- 172.21.15.119:0/3759034882 shutdown_connections 2024-05-27T09:34:10.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.773+0000 7f8813200700 1 -- 172.21.15.119:0/3759034882 wait complete. 2024-05-27T09:34:10.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:10 smithi138 ceph-mon[34885]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:34:10.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:10 smithi138 ceph-mon[34885]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:34:10.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:10 smithi138 ceph-mon[34885]: Upgrade: Target is version 19.0.0-3868-ga6507388 (squid) 2024-05-27T09:34:10.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:10 smithi138 ceph-mon[34885]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, digests ['quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6'] 2024-05-27T09:34:10.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:10 smithi138 ceph-mon[34885]: Upgrade: Setting container_image for all mgr 2024-05-27T09:34:10.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:10 smithi138 ceph-mon[34885]: from='client.14828 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:34:10.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:10 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/4018479372' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:34:10.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:10 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/1007947460' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:34:10.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.918+0000 7fe04e400700 1 -- 172.21.15.119:0/1061876790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe048101ae0 msgr2=0x7fe048109fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:10.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.918+0000 7fe04e400700 1 --2- 172.21.15.119:0/1061876790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe048101ae0 0x7fe048109fe0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe034009910 tx=0x7fe034009c20 comp rx=0 tx=0).stop 2024-05-27T09:34:10.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.918+0000 7fe04e400700 1 -- 172.21.15.119:0/1061876790 shutdown_connections 2024-05-27T09:34:10.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.918+0000 7fe04e400700 1 --2- 172.21.15.119:0/1061876790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe048101ae0 0x7fe048109fe0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:10.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.918+0000 7fe04e400700 1 --2- 172.21.15.119:0/1061876790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0481011d0 0x7fe0481015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:10.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.918+0000 7fe04e400700 1 -- 172.21.15.119:0/1061876790 >> 172.21.15.119:0/1061876790 conn(0x7fe0480fa9d0 msgr2=0x7fe0480fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:10.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.918+0000 7fe04e400700 1 -- 172.21.15.119:0/1061876790 shutdown_connections 2024-05-27T09:34:10.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.918+0000 7fe04e400700 1 -- 172.21.15.119:0/1061876790 wait complete. 2024-05-27T09:34:10.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.919+0000 7fe04e400700 1 Processor -- start 2024-05-27T09:34:10.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.919+0000 7fe04e400700 1 -- start start 2024-05-27T09:34:10.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.919+0000 7fe04e400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0481011d0 0x7fe0480fee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:10.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.919+0000 7fe04e400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe048101ae0 0x7fe0480ff370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:10.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.919+0000 7fe04e400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0480ff8b0 con 0x7fe0481011d0 2024-05-27T09:34:10.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.919+0000 7fe04e400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0480ff9f0 con 0x7fe048101ae0 2024-05-27T09:34:10.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.920+0000 7fe04d000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0481011d0 0x7fe0480fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:10.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.920+0000 7fe04d000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0481011d0 0x7fe0480fee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40190/0 (socket says 172.21.15.119:40190) 2024-05-27T09:34:10.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.920+0000 7fe04d000700 1 -- 172.21.15.119:0/3185414939 learned_addr learned my addr 172.21.15.119:0/3185414939 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:34:10.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.920+0000 7fe047e00700 1 --2- 172.21.15.119:0/3185414939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe048101ae0 0x7fe0480ff370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:10.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.920+0000 7fe04d000700 1 -- 172.21.15.119:0/3185414939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe048101ae0 msgr2=0x7fe0480ff370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:10.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.920+0000 7fe04d000700 1 --2- 172.21.15.119:0/3185414939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe048101ae0 0x7fe0480ff370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:10.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.920+0000 7fe04d000700 1 -- 172.21.15.119:0/3185414939 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0340095f0 con 0x7fe0481011d0 2024-05-27T09:34:10.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.920+0000 7fe047e00700 1 --2- 172.21.15.119:0/3185414939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe048101ae0 0x7fe0480ff370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:34:10.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.920+0000 7fe04d000700 1 --2- 172.21.15.119:0/3185414939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0481011d0 0x7fe0480fee30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe03c00b4b0 tx=0x7fe03c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:10.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.921+0000 7fe045600700 1 -- 172.21.15.119:0/3185414939 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe03c011800 con 0x7fe0481011d0 2024-05-27T09:34:10.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.921+0000 7fe045600700 1 -- 172.21.15.119:0/3185414939 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe03c011e20 con 0x7fe0481011d0 2024-05-27T09:34:10.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.921+0000 7fe04e400700 1 -- 172.21.15.119:0/3185414939 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0481a1e60 con 0x7fe0481011d0 2024-05-27T09:34:10.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.921+0000 7fe045600700 1 -- 172.21.15.119:0/3185414939 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe03c00f2e0 con 0x7fe0481011d0 2024-05-27T09:34:10.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.921+0000 7fe04e400700 1 -- 172.21.15.119:0/3185414939 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0481a2270 con 0x7fe0481011d0 2024-05-27T09:34:10.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.922+0000 7fe04e400700 1 -- 172.21.15.119:0/3185414939 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe048107650 con 0x7fe0481011d0 2024-05-27T09:34:10.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.923+0000 7fe045600700 1 -- 172.21.15.119:0/3185414939 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 99935+0+0 (secure 0 0 0) 0x7fe03c010380 con 0x7fe0481011d0 2024-05-27T09:34:10.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.925+0000 7fe045600700 1 --2- 172.21.15.119:0/3185414939 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe0300786c0 0x7fe03007ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:10.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.925+0000 7fe045600700 1 -- 172.21.15.119:0/3185414939 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe03c09a640 con 0x7fe0481011d0 2024-05-27T09:34:10.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.925+0000 7fe047e00700 1 --2- 172.21.15.119:0/3185414939 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe0300786c0 0x7fe03007ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:10.928 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.926+0000 7fe047e00700 1 --2- 172.21.15.119:0/3185414939 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe0300786c0 0x7fe03007ab70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe034006010 tx=0x7fe03400b540 comp rx=0 tx=0).ready entity=mgr.14640 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:10.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:10.929+0000 7fe045600700 1 -- 172.21.15.119:0/3185414939 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+190996 (secure 0 0 0) 0x7fe03c062860 con 0x7fe0481011d0 2024-05-27T09:34:11.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:11.246+0000 7fe04e400700 1 -- 172.21.15.119:0/3185414939 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe04804f2a0 con 0x7fe0481011d0 2024-05-27T09:34:11.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:11.247+0000 7fe045600700 1 -- 172.21.15.119:0/3185414939 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fe03c062200 con 0x7fe0481011d0 2024-05-27T09:34:11.253 INFO:teuthology.orchestra.run.smithi119.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-27T09:34:11.254 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:34:11.254 INFO:teuthology.orchestra.run.smithi119.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:34:11.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:11.251+0000 7fe04e400700 1 -- 172.21.15.119:0/3185414939 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe0300786c0 msgr2=0x7fe03007ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:11.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:11.251+0000 7fe04e400700 1 --2- 172.21.15.119:0/3185414939 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe0300786c0 0x7fe03007ab70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe034006010 tx=0x7fe03400b540 comp rx=0 tx=0).stop 2024-05-27T09:34:11.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:11.251+0000 7fe04e400700 1 -- 172.21.15.119:0/3185414939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0481011d0 msgr2=0x7fe0480fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:11.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:11.251+0000 7fe04e400700 1 --2- 172.21.15.119:0/3185414939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0481011d0 0x7fe0480fee30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe03c00b4b0 tx=0x7fe03c00b870 comp rx=0 tx=0).stop 2024-05-27T09:34:11.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:11.251+0000 7fe04e400700 1 -- 172.21.15.119:0/3185414939 shutdown_connections 2024-05-27T09:34:11.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:11.251+0000 7fe04e400700 1 --2- 172.21.15.119:0/3185414939 >> [v2:172.21.15.119:6800/1378367028,v1:172.21.15.119:6801/1378367028] conn(0x7fe0300786c0 0x7fe03007ab70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:11.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:11.251+0000 7fe04e400700 1 --2- 172.21.15.119:0/3185414939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0481011d0 0x7fe0480fee30 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:11.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:11.251+0000 7fe04e400700 1 --2- 172.21.15.119:0/3185414939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe048101ae0 0x7fe0480ff370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:11.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:11.251+0000 7fe04e400700 1 -- 172.21.15.119:0/3185414939 >> 172.21.15.119:0/3185414939 conn(0x7fe0480fa9d0 msgr2=0x7fe048104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:11.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:11.251+0000 7fe04e400700 1 -- 172.21.15.119:0/3185414939 shutdown_connections 2024-05-27T09:34:11.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:11.251+0000 7fe04e400700 1 -- 172.21.15.119:0/3185414939 wait complete. 2024-05-27T09:34:11.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:11 smithi119 ceph-mon[29082]: pgmap v114: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:11.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:11 smithi119 ceph-mon[29082]: from='client.? 172.21.15.119:0/3185414939' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:34:11.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:11 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:11.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:11 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T09:34:11.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:11 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T09:34:11.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:11 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:34:11.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:11 smithi138 ceph-mon[34885]: pgmap v114: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:11.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:11 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/3185414939' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:34:11.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:11 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:11.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:11 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T09:34:11.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:11 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T09:34:11.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:11 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:34:12.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:12 smithi119 ceph-mon[29082]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:34:12.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:12 smithi119 ceph-mon[29082]: Upgrade: Updating mon.smithi119 2024-05-27T09:34:12.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:12 smithi119 ceph-mon[29082]: Deploying daemon mon.smithi119 on smithi119 2024-05-27T09:34:12.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:12 smithi138 ceph-mon[34885]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:34:12.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:12 smithi138 ceph-mon[34885]: Upgrade: Updating mon.smithi119 2024-05-27T09:34:12.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:12 smithi138 ceph-mon[34885]: Deploying daemon mon.smithi119 on smithi119 2024-05-27T09:34:13.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:13 smithi119 ceph-mon[29082]: pgmap v115: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:13.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:13 smithi119 ceph-mon[29082]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:34:13.812 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:13 smithi138 ceph-mon[34885]: pgmap v115: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:13.813 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:13 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:34:14.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:13 smithi119 systemd[1]: Stopping Ceph mon.smithi119 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:34:14.556 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:14 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119[29078]: 2024-05-27T09:34:14.410+0000 7f1b4e600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi119 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T09:34:14.556 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:14 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119[29078]: 2024-05-27T09:34:14.410+0000 7f1b4e600700 -1 mon.smithi119@0(leader) e2 *** Got Signal Terminated *** 2024-05-27T09:34:14.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:14 smithi119 podman[88102]: 2024-05-27 09:34:14.689506318 +0000 UTC m=+0.408951040 container died ba998b789a1c708b9fa6ba9962892e24c8488192c708f59682be39479f0dd0c8 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, io.buildah.version=1.29.1) 2024-05-27T09:34:15.360 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:15 smithi119 podman[88102]: 2024-05-27 09:34:15.149939928 +0000 UTC m=+0.869384683 container cleanup ba998b789a1c708b9fa6ba9962892e24c8488192c708f59682be39479f0dd0c8 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308) 2024-05-27T09:34:15.360 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:15 smithi119 bash[88102]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119 2024-05-27T09:34:15.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:15 smithi119 podman[88125]: 2024-05-27 09:34:15.359148306 +0000 UTC m=+0.655443185 container remove ba998b789a1c708b9fa6ba9962892e24c8488192c708f59682be39479f0dd0c8 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, GIT_CLEAN=True, io.buildah.version=1.29.1, RELEASE=HEAD, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-27T09:34:16.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:15 smithi119 systemd[1]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@mon.smithi119.service: Deactivated successfully. 2024-05-27T09:34:16.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:15 smithi119 systemd[1]: Stopped Ceph mon.smithi119 for b966b06e-1c09-11ef-bc9b-c7b262605968. 2024-05-27T09:34:16.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:15 smithi119 systemd[1]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@mon.smithi119.service: Consumed 32.261s CPU time. 2024-05-27T09:34:16.598 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:16 smithi119 systemd[1]: Starting Ceph mon.smithi119 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:34:16.941 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:16 smithi119 podman[88251]: 2024-05-27 09:34:16.501735244 +0000 UTC m=+0.032706928 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:34:16.941 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:16 smithi119 podman[88251]: 2024-05-27 09:34:16.627203105 +0000 UTC m=+0.158174826 container create 3e5c7a9ad372ff49da57e9334c4528c3069c1e1b46c4d00d1e8cbac016bf7e52 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-27T09:34:16.942 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:16 smithi119 podman[88251]: 2024-05-27 09:34:16.939780195 +0000 UTC m=+0.470751986 container init 3e5c7a9ad372ff49da57e9334c4528c3069c1e1b46c4d00d1e8cbac016bf7e52 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-27T09:34:17.251 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:16 smithi119 podman[88251]: 2024-05-27 09:34:16.951281161 +0000 UTC m=+0.482252868 container start 3e5c7a9ad372ff49da57e9334c4528c3069c1e1b46c4d00d1e8cbac016bf7e52 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T09:34:17.251 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 bash[88251]: 3e5c7a9ad372ff49da57e9334c4528c3069c1e1b46c4d00d1e8cbac016bf7e52 2024-05-27T09:34:17.251 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 systemd[1]: Started Ceph mon.smithi119 for b966b06e-1c09-11ef-bc9b-c7b262605968. 2024-05-27T09:34:17.251 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: set uid:gid to 167:167 (ceph:ceph) 2024-05-27T09:34:17.251 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev), process ceph-mon, pid 2 2024-05-27T09:34:17.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: pidfile_write: ignore empty --pid-file 2024-05-27T09:34:17.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: load: jerasure load: lrc 2024-05-27T09:34:17.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: RocksDB version: 7.9.2 2024-05-27T09:34:17.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Git sha 0 2024-05-27T09:34:17.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Compile date 2024-05-21 11:39:56 2024-05-27T09:34:17.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: DB SUMMARY 2024-05-27T09:34:17.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: DB Session ID: 4R2K9EZ3MVBI2YCB4F68 2024-05-27T09:34:17.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: CURRENT file: CURRENT 2024-05-27T09:34:17.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: IDENTITY file: IDENTITY 2024-05-27T09:34:17.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1746 Bytes 2024-05-27T09:34:17.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi119/store.db dir, Total Num: 1, files: 000035.sst 2024-05-27T09:34:17.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi119/store.db: 000033.log size: 465851 ; 2024-05-27T09:34:17.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.error_if_exists: 0 2024-05-27T09:34:17.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.create_if_missing: 0 2024-05-27T09:34:17.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.paranoid_checks: 1 2024-05-27T09:34:17.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-27T09:34:17.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-27T09:34:17.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-27T09:34:17.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.env: 0x55a5b3fdc500 2024-05-27T09:34:17.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.fs: PosixFileSystem 2024-05-27T09:34:17.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.info_log: 0x55a5c99bdee0 2024-05-27T09:34:17.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_file_opening_threads: 16 2024-05-27T09:34:17.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.statistics: (nil) 2024-05-27T09:34:17.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.use_fsync: 0 2024-05-27T09:34:17.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_log_file_size: 0 2024-05-27T09:34:17.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-27T09:34:17.254 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-27T09:34:17.254 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.keep_log_file_num: 1000 2024-05-27T09:34:17.254 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.recycle_log_file_num: 0 2024-05-27T09:34:17.254 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.allow_fallocate: 1 2024-05-27T09:34:17.254 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.allow_mmap_reads: 0 2024-05-27T09:34:17.254 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.allow_mmap_writes: 0 2024-05-27T09:34:17.254 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.use_direct_reads: 0 2024-05-27T09:34:17.254 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-27T09:34:17.255 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.create_missing_column_families: 0 2024-05-27T09:34:17.255 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.db_log_dir: 2024-05-27T09:34:17.255 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.wal_dir: 2024-05-27T09:34:17.255 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-27T09:34:17.255 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-27T09:34:17.255 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-27T09:34:17.255 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-27T09:34:17.255 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-27T09:34:17.255 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-27T09:34:17.255 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.advise_random_on_open: 1 2024-05-27T09:34:17.255 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.db_write_buffer_size: 0 2024-05-27T09:34:17.255 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.write_buffer_manager: 0x55a5c998e8c0 2024-05-27T09:34:17.255 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-27T09:34:17.256 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-27T09:34:17.256 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-27T09:34:17.256 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.rate_limiter: (nil) 2024-05-27T09:34:17.256 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-27T09:34:17.256 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.wal_recovery_mode: 2 2024-05-27T09:34:17.256 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.enable_thread_tracking: 0 2024-05-27T09:34:17.256 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.enable_pipelined_write: 0 2024-05-27T09:34:17.256 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.unordered_write: 0 2024-05-27T09:34:17.256 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-27T09:34:17.256 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-27T09:34:17.256 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-27T09:34:17.256 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-27T09:34:17.256 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.row_cache: None 2024-05-27T09:34:17.256 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.wal_filter: None 2024-05-27T09:34:17.257 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-27T09:34:17.257 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.allow_ingest_behind: 0 2024-05-27T09:34:17.257 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.two_write_queues: 0 2024-05-27T09:34:17.257 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.manual_wal_flush: 0 2024-05-27T09:34:17.257 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.wal_compression: 0 2024-05-27T09:34:17.257 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.atomic_flush: 0 2024-05-27T09:34:17.257 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-27T09:34:17.258 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-27T09:34:17.258 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-27T09:34:17.258 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.log_readahead_size: 0 2024-05-27T09:34:17.258 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-27T09:34:17.258 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.best_efforts_recovery: 0 2024-05-27T09:34:17.258 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-27T09:34:17.258 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-27T09:34:17.258 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.allow_data_in_errors: 0 2024-05-27T09:34:17.258 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.db_host_id: __hostname__ 2024-05-27T09:34:17.258 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-27T09:34:17.258 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_background_jobs: 2 2024-05-27T09:34:17.258 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_background_compactions: -1 2024-05-27T09:34:17.259 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_subcompactions: 1 2024-05-27T09:34:17.259 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-27T09:34:17.259 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-27T09:34:17.259 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-27T09:34:17.259 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_total_wal_size: 0 2024-05-27T09:34:17.259 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-27T09:34:17.259 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-27T09:34:17.259 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-27T09:34:17.259 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-27T09:34:17.259 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_open_files: -1 2024-05-27T09:34:17.259 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.bytes_per_sync: 0 2024-05-27T09:34:17.259 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-27T09:34:17.260 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-27T09:34:17.260 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compaction_readahead_size: 0 2024-05-27T09:34:17.260 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_background_flushes: -1 2024-05-27T09:34:17.260 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Compression algorithms supported: 2024-05-27T09:34:17.260 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-27T09:34:17.260 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: kZSTD supported: 0 2024-05-27T09:34:17.260 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: kXpressCompression supported: 0 2024-05-27T09:34:17.260 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: kLZ4HCCompression supported: 1 2024-05-27T09:34:17.260 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: kZlibCompression supported: 1 2024-05-27T09:34:17.260 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: kSnappyCompression supported: 1 2024-05-27T09:34:17.260 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: kLZ4Compression supported: 1 2024-05-27T09:34:17.260 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: kBZip2Compression supported: 0 2024-05-27T09:34:17.260 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-27T09:34:17.261 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-27T09:34:17.261 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi119/store.db/MANIFEST-000015 2024-05-27T09:34:17.261 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-27T09:34:17.261 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-27T09:34:17.261 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.merge_operator: 2024-05-27T09:34:17.261 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compaction_filter: None 2024-05-27T09:34:17.261 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compaction_filter_factory: None 2024-05-27T09:34:17.261 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.sst_partitioner_factory: None 2024-05-27T09:34:17.261 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-27T09:34:17.261 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-27T09:34:17.261 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a5c9ac40c0) 2024-05-27T09:34:17.261 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: cache_index_and_filter_blocks: 1 2024-05-27T09:34:17.261 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-27T09:34:17.261 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-27T09:34:17.262 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: pin_top_level_index_and_filter: 1 2024-05-27T09:34:17.262 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: index_type: 0 2024-05-27T09:34:17.262 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: data_block_index_type: 0 2024-05-27T09:34:17.262 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: index_shortening: 1 2024-05-27T09:34:17.262 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-27T09:34:17.262 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: checksum: 4 2024-05-27T09:34:17.262 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: no_block_cache: 0 2024-05-27T09:34:17.262 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: block_cache: 0x55a5c99c11f0 2024-05-27T09:34:17.262 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: block_cache_name: BinnedLRUCache 2024-05-27T09:34:17.262 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: block_cache_options: 2024-05-27T09:34:17.262 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: capacity : 536870912 2024-05-27T09:34:17.262 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: num_shard_bits : 4 2024-05-27T09:34:17.262 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: strict_capacity_limit : 0 2024-05-27T09:34:17.263 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: high_pri_pool_ratio: 0.000 2024-05-27T09:34:17.263 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: block_cache_compressed: (nil) 2024-05-27T09:34:17.263 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: persistent_cache: (nil) 2024-05-27T09:34:17.263 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: block_size: 4096 2024-05-27T09:34:17.263 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: block_size_deviation: 10 2024-05-27T09:34:17.263 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: block_restart_interval: 16 2024-05-27T09:34:17.263 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: index_block_restart_interval: 1 2024-05-27T09:34:17.263 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: metadata_block_size: 4096 2024-05-27T09:34:17.263 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: partition_filters: 0 2024-05-27T09:34:17.263 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: use_delta_encoding: 1 2024-05-27T09:34:17.263 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: filter_policy: bloomfilter 2024-05-27T09:34:17.263 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: whole_key_filtering: 1 2024-05-27T09:34:17.263 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: verify_compression: 0 2024-05-27T09:34:17.263 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: read_amp_bytes_per_bit: 0 2024-05-27T09:34:17.263 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: format_version: 5 2024-05-27T09:34:17.264 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: enable_index_compression: 1 2024-05-27T09:34:17.264 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: block_align: 0 2024-05-27T09:34:17.264 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: max_auto_readahead_size: 262144 2024-05-27T09:34:17.264 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: prepopulate_block_cache: 0 2024-05-27T09:34:17.264 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: initial_auto_readahead_size: 8192 2024-05-27T09:34:17.264 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: num_file_reads_for_auto_readahead: 2 2024-05-27T09:34:17.264 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.write_buffer_size: 33554432 2024-05-27T09:34:17.264 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_write_buffer_number: 2 2024-05-27T09:34:17.264 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compression: NoCompression 2024-05-27T09:34:17.264 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.bottommost_compression: Disabled 2024-05-27T09:34:17.264 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.prefix_extractor: nullptr 2024-05-27T09:34:17.265 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-27T09:34:17.265 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.num_levels: 7 2024-05-27T09:34:17.265 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-27T09:34:17.266 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-27T09:34:17.266 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-27T09:34:17.266 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-27T09:34:17.266 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-27T09:34:17.266 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-27T09:34:17.266 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-27T09:34:17.266 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-27T09:34:17.266 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-27T09:34:17.266 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-27T09:34:17.266 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-27T09:34:17.266 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-27T09:34:17.266 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-27T09:34:17.267 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compression_opts.level: 32767 2024-05-27T09:34:17.267 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compression_opts.strategy: 0 2024-05-27T09:34:17.267 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-27T09:34:17.267 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-27T09:34:17.267 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-27T09:34:17.267 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-27T09:34:17.267 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compression_opts.enabled: false 2024-05-27T09:34:17.267 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-27T09:34:17.267 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-27T09:34:17.267 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-27T09:34:17.267 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-27T09:34:17.267 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.target_file_size_base: 67108864 2024-05-27T09:34:17.267 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-27T09:34:17.267 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-27T09:34:17.268 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-27T09:34:17.268 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-27T09:34:17.268 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-27T09:34:17.268 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-27T09:34:17.268 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-27T09:34:17.268 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-27T09:34:17.268 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-27T09:34:17.268 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-27T09:34:17.268 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-27T09:34:17.268 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-27T09:34:17.268 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-27T09:34:17.268 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-27T09:34:17.268 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.arena_block_size: 1048576 2024-05-27T09:34:17.269 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-27T09:34:17.269 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-27T09:34:17.269 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.disable_auto_compactions: 0 2024-05-27T09:34:17.269 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-27T09:34:17.269 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-27T09:34:17.269 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-27T09:34:17.269 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-27T09:34:17.269 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-27T09:34:17.269 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-27T09:34:17.269 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-27T09:34:17.269 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-27T09:34:17.269 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-27T09:34:17.269 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-27T09:34:17.270 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-27T09:34:17.270 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.inplace_update_support: 0 2024-05-27T09:34:17.270 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-27T09:34:17.270 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-27T09:34:17.270 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-27T09:34:17.270 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-27T09:34:17.270 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.bloom_locality: 0 2024-05-27T09:34:17.270 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.max_successive_merges: 0 2024-05-27T09:34:17.270 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-27T09:34:17.270 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.paranoid_file_checks: 0 2024-05-27T09:34:17.270 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.force_consistency_checks: 1 2024-05-27T09:34:17.270 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.report_bg_io_stats: 0 2024-05-27T09:34:17.270 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.ttl: 2592000 2024-05-27T09:34:17.271 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-27T09:34:17.271 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-27T09:34:17.271 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-27T09:34:17.271 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.enable_blob_files: false 2024-05-27T09:34:17.271 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.min_blob_size: 0 2024-05-27T09:34:17.271 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.blob_file_size: 268435456 2024-05-27T09:34:17.271 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-27T09:34:17.271 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-27T09:34:17.271 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-27T09:34:17.271 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-27T09:34:17.271 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-27T09:34:17.271 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.blob_file_starting_level: 0 2024-05-27T09:34:17.271 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-27T09:34:17.272 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-27T09:34:17.272 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi119/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 37, last_sequence is 16528, log_number is 33,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 33 2024-05-27T09:34:17.272 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 33 2024-05-27T09:34:17.272 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: da0f174e-a9ee-4728-a3bb-037f69355945 2024-05-27T09:34:17.272 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716802457082327, "job": 1, "event": "recovery_started", "wal_files": [33]} 2024-05-27T09:34:17.272 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #33 mode 2 2024-05-27T09:34:17.272 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716802457086987, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 38, "file_size": 451438, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 16529, "largest_seqno": 17363, "table_properties": {"data_size": 447645, "index_size": 1793, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 965, "raw_key_size": 9114, "raw_average_key_size": 24, "raw_value_size": 440086, "raw_average_value_size": 1192, "num_data_blocks": 81, "num_entries": 369, "num_filter_entries": 369, "num_deletions": 6, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716802457, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "da0f174e-a9ee-4728-a3bb-037f69355945", "db_session_id": "4R2K9EZ3MVBI2YCB4F68", "orig_file_number": 38, "seqno_to_time_mapping": "N/A"}} 2024-05-27T09:34:17.272 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716802457087159, "job": 1, "event": "recovery_finished"} 2024-05-27T09:34:17.272 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: [db/version_set.cc:5047] Creating manifest 40 2024-05-27T09:34:17.272 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-27T09:34:17.272 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi119/store.db/000033.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-27T09:34:17.272 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a5c99a5c00 2024-05-27T09:34:17.272 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: DB pointer 0x55a5ce798000 2024-05-27T09:34:17.273 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-27T09:34:17.273 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-27T09:34:17.273 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: ** DB Stats ** 2024-05-27T09:34:17.273 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T09:34:17.273 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-27T09:34:17.273 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T09:34:17.273 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T09:34:17.273 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-27T09:34:17.273 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T09:34:17.273 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T09:34:17.273 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: 2024-05-27T09:34:17.273 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: ** Compaction Stats [default] ** 2024-05-27T09:34:17.273 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T09:34:17.273 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-27T09:34:17.274 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: L0 1/0 440.86 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 175.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-27T09:34:17.274 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: L6 1/0 8.27 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-05-27T09:34:17.274 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Sum 2/0 8.70 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 175.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-27T09:34:17.274 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 175.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-27T09:34:17.274 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: 2024-05-27T09:34:17.274 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: ** Compaction Stats [default] ** 2024-05-27T09:34:17.274 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T09:34:17.274 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-27T09:34:17.274 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 175.3 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-27T09:34:17.274 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: 2024-05-27T09:34:17.274 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-27T09:34:17.274 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: 2024-05-27T09:34:17.274 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T09:34:17.274 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-27T09:34:17.275 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-27T09:34:17.275 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-27T09:34:17.275 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-27T09:34:17.275 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-27T09:34:17.275 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Cumulative compaction: 0.00 GB write, 46.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T09:34:17.275 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Interval compaction: 0.00 GB write, 46.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T09:34:17.275 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-27T09:34:17.275 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Block cache BinnedLRUCache@0x55a5c99c11f0#2 capacity: 512.00 MB usage: 3.12 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-05-27T09:34:17.275 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.04 KB,0.000198185%) IndexBlock(1,2.08 KB,0.000396371%) Misc(1,0.00 KB,0%) 2024-05-27T09:34:17.275 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: 2024-05-27T09:34:17.275 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-27T09:34:17.275 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: starting mon.smithi119 rank 0 at public addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] at bind addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi119 fsid b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:34:17.275 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: mon.smithi119@-1(???) e2 preinit fsid b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:34:17.276 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: mon.smithi119@-1(???).mds e22 new map 2024-05-27T09:34:17.276 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: mon.smithi119@-1(???).mds e22 print_map 2024-05-27T09:34:17.276 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: e22 2024-05-27T09:34:17.276 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T09:34:17.276 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:34:17.276 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: legacy client fscid: 1 2024-05-27T09:34:17.276 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: 2024-05-27T09:34:17.276 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Filesystem 'cephfs' (1) 2024-05-27T09:34:17.276 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: fs_name cephfs 2024-05-27T09:34:17.276 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: epoch 22 2024-05-27T09:34:17.276 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-27T09:34:17.276 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: created 2024-05-27T09:25:53.387388+0000 2024-05-27T09:34:17.276 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: modified 2024-05-27T09:34:01.250304+0000 2024-05-27T09:34:17.277 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: tableserver 0 2024-05-27T09:34:17.277 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: root 0 2024-05-27T09:34:17.277 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: session_timeout 60 2024-05-27T09:34:17.277 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: session_autoclose 300 2024-05-27T09:34:17.277 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: max_file_size 1099511627776 2024-05-27T09:34:17.277 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: max_xattr_size 65536 2024-05-27T09:34:17.277 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: required_client_features {} 2024-05-27T09:34:17.277 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: last_failure 0 2024-05-27T09:34:17.277 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: last_failure_osd_epoch 46 2024-05-27T09:34:17.277 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:34:17.277 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: max_mds 2 2024-05-27T09:34:17.277 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: in 0,1 2024-05-27T09:34:17.277 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: up {0=14472,1=24241} 2024-05-27T09:34:17.278 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: failed 2024-05-27T09:34:17.278 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: damaged 2024-05-27T09:34:17.278 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: stopped 2024-05-27T09:34:17.278 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: data_pools [3] 2024-05-27T09:34:17.278 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: metadata_pool 2 2024-05-27T09:34:17.278 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: inline_data enabled 2024-05-27T09:34:17.278 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: balancer 2024-05-27T09:34:17.278 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: bal_rank_mask -1 2024-05-27T09:34:17.278 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: standby_count_wanted 1 2024-05-27T09:34:17.278 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: qdb_cluster leader: 0 members: 2024-05-27T09:34:17.278 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: [mds.cephfs.smithi119.ppmzzk{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:34:17.278 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: [mds.cephfs.smithi138.ndfkhd{1:24241} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/897058594,v1:172.21.15.138:6825/897058594] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:34:17.278 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: 2024-05-27T09:34:17.279 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: 2024-05-27T09:34:17.279 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: Standby daemons: 2024-05-27T09:34:17.279 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: 2024-05-27T09:34:17.279 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: [mds.cephfs.smithi119.nghcey{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.119:6826/413736191,v1:172.21.15.119:6827/413736191] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:34:17.279 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout: [mds.cephfs.smithi138.alszpk{-1:24253} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/2345607053,v1:172.21.15.138:6827/2345607053] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:34:17.279 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: mon.smithi119@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-05-27T09:34:17.279 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: mon.smithi119@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-05-27T09:34:17.279 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: mon.smithi119@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-05-27T09:34:17.279 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: mon.smithi119@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-05-27T09:34:17.279 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: mon.smithi119@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-05-27T09:34:18.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:17 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi119"}]: dispatch 2024-05-27T09:34:18.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:17 smithi138 ceph-mon[34885]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:34:18.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:17 smithi138 ceph-mon[34885]: pgmap v117: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:18.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:17 smithi138 ceph-mon[34885]: mon.smithi119 calling monitor election 2024-05-27T09:34:18.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:17 smithi138 ceph-mon[34885]: mon.smithi119 is new leader, mons smithi119,smithi138 in quorum (ranks 0,1) 2024-05-27T09:34:18.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:17 smithi138 ceph-mon[34885]: monmap epoch 2 2024-05-27T09:34:18.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:17 smithi138 ceph-mon[34885]: fsid b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:34:18.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:17 smithi138 ceph-mon[34885]: last_changed 2024-05-27T09:20:20.043858+0000 2024-05-27T09:34:18.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:17 smithi138 ceph-mon[34885]: created 2024-05-27T09:16:53.014813+0000 2024-05-27T09:34:18.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:17 smithi138 ceph-mon[34885]: min_mon_release 18 (reef) 2024-05-27T09:34:18.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:17 smithi138 ceph-mon[34885]: election_strategy: 1 2024-05-27T09:34:18.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:17 smithi138 ceph-mon[34885]: 0: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.smithi119 2024-05-27T09:34:18.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:17 smithi138 ceph-mon[34885]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.smithi138 2024-05-27T09:34:18.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:17 smithi138 ceph-mon[34885]: fsmap cephfs:2 {0=cephfs.smithi119.ppmzzk=up:active,1=cephfs.smithi138.ndfkhd=up:active} 2 up:standby 2024-05-27T09:34:18.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:17 smithi138 ceph-mon[34885]: osdmap e48: 6 total, 6 up, 6 in 2024-05-27T09:34:18.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:17 smithi138 ceph-mon[34885]: mgrmap e33: smithi119.ncwmog(active, since 3m), standbys: smithi138.dxsuqf 2024-05-27T09:34:18.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:17 smithi138 ceph-mon[34885]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-27T09:34:18.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:17 smithi138 ceph-mon[34885]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:34:18.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:17 smithi138 ceph-mon[34885]: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:34:18.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:17 smithi138 ceph-mon[34885]: from='mgr.14640 ' entity='' 2024-05-27T09:34:18.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:17 smithi138 ceph-mon[34885]: mgrmap e34: smithi119.ncwmog(active, since 3m), standbys: smithi138.dxsuqf 2024-05-27T09:34:18.189 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi119"}]: dispatch 2024-05-27T09:34:18.189 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: from='mgr.14640 172.21.15.119:0/323881929' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:34:18.189 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: pgmap v117: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:18.189 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: mon.smithi119 calling monitor election 2024-05-27T09:34:18.189 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: mon.smithi119 is new leader, mons smithi119,smithi138 in quorum (ranks 0,1) 2024-05-27T09:34:18.189 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: monmap epoch 2 2024-05-27T09:34:18.189 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: fsid b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:34:18.190 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: last_changed 2024-05-27T09:20:20.043858+0000 2024-05-27T09:34:18.190 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: created 2024-05-27T09:16:53.014813+0000 2024-05-27T09:34:18.190 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: min_mon_release 18 (reef) 2024-05-27T09:34:18.190 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: election_strategy: 1 2024-05-27T09:34:18.190 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: 0: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.smithi119 2024-05-27T09:34:18.190 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.smithi138 2024-05-27T09:34:18.190 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: fsmap cephfs:2 {0=cephfs.smithi119.ppmzzk=up:active,1=cephfs.smithi138.ndfkhd=up:active} 2 up:standby 2024-05-27T09:34:18.190 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: osdmap e48: 6 total, 6 up, 6 in 2024-05-27T09:34:18.190 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: mgrmap e33: smithi119.ncwmog(active, since 3m), standbys: smithi138.dxsuqf 2024-05-27T09:34:18.190 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-27T09:34:18.190 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:34:18.190 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:34:18.190 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: from='mgr.14640 ' entity='' 2024-05-27T09:34:18.190 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:17 smithi119 ceph-mon[88269]: mgrmap e34: smithi119.ncwmog(active, since 3m), standbys: smithi138.dxsuqf 2024-05-27T09:34:27.474 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:27 smithi138 ceph-mon[34885]: Standby manager daemon smithi138.dxsuqf restarted 2024-05-27T09:34:27.474 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:27 smithi138 ceph-mon[34885]: Standby manager daemon smithi138.dxsuqf started 2024-05-27T09:34:27.474 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:27 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.138:0/2154210011' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.dxsuqf/crt"}]: dispatch 2024-05-27T09:34:27.474 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:27 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.138:0/2154210011' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T09:34:27.474 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:27 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.138:0/2154210011' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.dxsuqf/key"}]: dispatch 2024-05-27T09:34:27.474 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:27 smithi138 ceph-mon[34885]: from='mgr.? 172.21.15.138:0/2154210011' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T09:34:27.474 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:27 smithi138 ceph-mon[34885]: Active manager daemon smithi119.ncwmog restarted 2024-05-27T09:34:27.474 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:27 smithi138 ceph-mon[34885]: Activating manager daemon smithi119.ncwmog 2024-05-27T09:34:27.480 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:27 smithi119 ceph-mon[88269]: Standby manager daemon smithi138.dxsuqf restarted 2024-05-27T09:34:27.480 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:27 smithi119 ceph-mon[88269]: Standby manager daemon smithi138.dxsuqf started 2024-05-27T09:34:27.480 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:27 smithi119 ceph-mon[88269]: from='mgr.? 172.21.15.138:0/2154210011' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.dxsuqf/crt"}]: dispatch 2024-05-27T09:34:27.481 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:27 smithi119 ceph-mon[88269]: from='mgr.? 172.21.15.138:0/2154210011' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T09:34:27.481 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:27 smithi119 ceph-mon[88269]: from='mgr.? 172.21.15.138:0/2154210011' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.dxsuqf/key"}]: dispatch 2024-05-27T09:34:27.481 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:27 smithi119 ceph-mon[88269]: from='mgr.? 172.21.15.138:0/2154210011' entity='mgr.smithi138.dxsuqf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T09:34:27.481 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:27 smithi119 ceph-mon[88269]: Active manager daemon smithi119.ncwmog restarted 2024-05-27T09:34:27.481 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:27 smithi119 ceph-mon[88269]: Activating manager daemon smithi119.ncwmog 2024-05-27T09:34:28.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: osdmap e49: 6 total, 6 up, 6 in 2024-05-27T09:34:28.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: mgrmap e35: smithi119.ncwmog(active, starting, since 0.0215294s), standbys: smithi138.dxsuqf 2024-05-27T09:34:28.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi119"}]: dispatch 2024-05-27T09:34:28.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:34:28.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.ppmzzk"}]: dispatch 2024-05-27T09:34:28.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.nghcey"}]: dispatch 2024-05-27T09:34:28.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.ndfkhd"}]: dispatch 2024-05-27T09:34:28.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.alszpk"}]: dispatch 2024-05-27T09:34:28.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr metadata", "who": "smithi119.ncwmog", "id": "smithi119.ncwmog"}]: dispatch 2024-05-27T09:34:28.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr metadata", "who": "smithi138.dxsuqf", "id": "smithi138.dxsuqf"}]: dispatch 2024-05-27T09:34:28.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T09:34:28.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T09:34:28.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T09:34:28.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T09:34:28.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T09:34:28.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T09:34:28.638 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T09:34:28.638 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T09:34:28.638 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T09:34:28.638 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: Manager daemon smithi119.ncwmog is now available 2024-05-27T09:34:28.638 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:34:28.638 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:34:28.638 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi119.ncwmog/mirror_snapshot_schedule"}]: dispatch 2024-05-27T09:34:28.638 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi119.ncwmog/mirror_snapshot_schedule"}]: dispatch 2024-05-27T09:34:28.638 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi119.ncwmog/trash_purge_schedule"}]: dispatch 2024-05-27T09:34:28.638 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:28 smithi138 ceph-mon[34885]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi119.ncwmog/trash_purge_schedule"}]: dispatch 2024-05-27T09:34:28.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: osdmap e49: 6 total, 6 up, 6 in 2024-05-27T09:34:28.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: mgrmap e35: smithi119.ncwmog(active, starting, since 0.0215294s), standbys: smithi138.dxsuqf 2024-05-27T09:34:28.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi119"}]: dispatch 2024-05-27T09:34:28.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:34:28.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.ppmzzk"}]: dispatch 2024-05-27T09:34:28.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.nghcey"}]: dispatch 2024-05-27T09:34:28.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.ndfkhd"}]: dispatch 2024-05-27T09:34:28.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.alszpk"}]: dispatch 2024-05-27T09:34:28.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr metadata", "who": "smithi119.ncwmog", "id": "smithi119.ncwmog"}]: dispatch 2024-05-27T09:34:28.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr metadata", "who": "smithi138.dxsuqf", "id": "smithi138.dxsuqf"}]: dispatch 2024-05-27T09:34:28.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T09:34:28.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T09:34:28.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T09:34:28.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T09:34:28.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T09:34:28.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T09:34:28.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T09:34:28.750 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T09:34:28.751 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T09:34:28.751 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: Manager daemon smithi119.ncwmog is now available 2024-05-27T09:34:28.751 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:34:28.751 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:34:28.751 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi119.ncwmog/mirror_snapshot_schedule"}]: dispatch 2024-05-27T09:34:28.751 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi119.ncwmog/mirror_snapshot_schedule"}]: dispatch 2024-05-27T09:34:28.751 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi119.ncwmog/trash_purge_schedule"}]: dispatch 2024-05-27T09:34:28.751 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:28 smithi119 ceph-mon[88269]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi119.ncwmog/trash_purge_schedule"}]: dispatch 2024-05-27T09:34:29.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:29 smithi138 ceph-mon[34885]: mgrmap e36: smithi119.ncwmog(active, since 1.04614s), standbys: smithi138.dxsuqf 2024-05-27T09:34:29.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:29 smithi138 ceph-mon[34885]: [27/May/2024:09:34:28] ENGINE Bus STARTING 2024-05-27T09:34:29.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:29 smithi119 ceph-mon[88269]: mgrmap e36: smithi119.ncwmog(active, since 1.04614s), standbys: smithi138.dxsuqf 2024-05-27T09:34:29.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:29 smithi119 ceph-mon[88269]: [27/May/2024:09:34:28] ENGINE Bus STARTING 2024-05-27T09:34:30.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:30 smithi138 ceph-mon[34885]: [27/May/2024:09:34:29] ENGINE Serving on https://172.21.15.119:7150 2024-05-27T09:34:30.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:30 smithi138 ceph-mon[34885]: [27/May/2024:09:34:29] ENGINE Serving on http://172.21.15.119:8765 2024-05-27T09:34:30.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:30 smithi138 ceph-mon[34885]: [27/May/2024:09:34:29] ENGINE Bus STARTED 2024-05-27T09:34:30.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:30 smithi138 ceph-mon[34885]: pgmap v4: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:30.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:30 smithi138 ceph-mon[34885]: mgrmap e37: smithi119.ncwmog(active, since 2s), standbys: smithi138.dxsuqf 2024-05-27T09:34:30.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:30 smithi119 ceph-mon[88269]: [27/May/2024:09:34:29] ENGINE Serving on https://172.21.15.119:7150 2024-05-27T09:34:30.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:30 smithi119 ceph-mon[88269]: [27/May/2024:09:34:29] ENGINE Serving on http://172.21.15.119:8765 2024-05-27T09:34:30.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:30 smithi119 ceph-mon[88269]: [27/May/2024:09:34:29] ENGINE Bus STARTED 2024-05-27T09:34:30.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:30 smithi119 ceph-mon[88269]: pgmap v4: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:30.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:30 smithi119 ceph-mon[88269]: mgrmap e37: smithi119.ncwmog(active, since 2s), standbys: smithi138.dxsuqf 2024-05-27T09:34:32.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:32 smithi119 ceph-mon[88269]: pgmap v5: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:32.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:32 smithi119 ceph-mon[88269]: mgrmap e38: smithi119.ncwmog(active, since 4s), standbys: smithi138.dxsuqf 2024-05-27T09:34:32.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:32 smithi119 ceph-mon[88269]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:32.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:32 smithi119 ceph-mon[88269]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:32.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:32 smithi138 ceph-mon[34885]: pgmap v5: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:32.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:32 smithi138 ceph-mon[34885]: mgrmap e38: smithi119.ncwmog(active, since 4s), standbys: smithi138.dxsuqf 2024-05-27T09:34:32.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:32 smithi138 ceph-mon[34885]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:32.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:32 smithi138 ceph-mon[34885]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:34.581 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:34 smithi119 ceph-mon[88269]: pgmap v6: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:34.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:34 smithi138 ceph-mon[34885]: pgmap v6: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:35.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:35 smithi119 ceph-mon[88269]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:35.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:35 smithi119 ceph-mon[88269]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:35.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:35 smithi119 ceph-mon[88269]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:35.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:35 smithi119 ceph-mon[88269]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:35.573 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:35 smithi138 ceph-mon[34885]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:35.573 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:35 smithi138 ceph-mon[34885]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:35.574 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:35 smithi138 ceph-mon[34885]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:35.574 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:35 smithi138 ceph-mon[34885]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:37.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:36 smithi119 ceph-mon[88269]: pgmap v7: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-27T09:34:37.300 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:36 smithi138 ceph-mon[34885]: pgmap v7: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-27T09:34:38.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:38 smithi138 ceph-mon[34885]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:38.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:38 smithi138 ceph-mon[34885]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:38.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:38 smithi138 ceph-mon[34885]: pgmap v8: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-27T09:34:38.440 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:38 smithi119 ceph-mon[88269]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:38.440 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:38 smithi119 ceph-mon[88269]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:38.441 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:38 smithi119 ceph-mon[88269]: pgmap v8: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-27T09:34:41.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:40 smithi119 ceph-mon[88269]: pgmap v9: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 16 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-27T09:34:41.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:41 smithi119 ceph-mon[88269]: Detected new or changed devices on smithi138 2024-05-27T09:34:41.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:41 smithi119 ceph-mon[88269]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:41.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:41 smithi119 ceph-mon[88269]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:41.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:41 smithi138 ceph-mon[34885]: pgmap v9: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 16 KiB/s rd, 0 B/s wr, 6 op/s 2024-05-27T09:34:41.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:41 smithi138 ceph-mon[34885]: Detected new or changed devices on smithi138 2024-05-27T09:34:41.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:41 smithi138 ceph-mon[34885]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:41.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:41 smithi138 ceph-mon[34885]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:41.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.428+0000 7f8d5c200700 1 -- 172.21.15.119:0/3970874314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d54102f90 msgr2=0x7f8d54106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:41.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.428+0000 7f8d5c200700 1 --2- 172.21.15.119:0/3970874314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d54102f90 0x7f8d54106f60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009910 tx=0x7f8d40009c20 comp rx=0 tx=0).stop 2024-05-27T09:34:41.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.428+0000 7f8d5c200700 1 -- 172.21.15.119:0/3970874314 shutdown_connections 2024-05-27T09:34:41.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.428+0000 7f8d5c200700 1 --2- 172.21.15.119:0/3970874314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d54102f90 0x7f8d54106f60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:41.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.428+0000 7f8d5c200700 1 --2- 172.21.15.119:0/3970874314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d541025f0 0x7f8d541029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:41.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.428+0000 7f8d5c200700 1 -- 172.21.15.119:0/3970874314 >> 172.21.15.119:0/3970874314 conn(0x7f8d540fde90 msgr2=0x7f8d541002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:41.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.428+0000 7f8d5c200700 1 -- 172.21.15.119:0/3970874314 shutdown_connections 2024-05-27T09:34:41.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.428+0000 7f8d5c200700 1 -- 172.21.15.119:0/3970874314 wait complete. 2024-05-27T09:34:41.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.430+0000 7f8d5c200700 1 Processor -- start 2024-05-27T09:34:41.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.430+0000 7f8d5c200700 1 -- start start 2024-05-27T09:34:41.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.430+0000 7f8d5c200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d541025f0 0x7f8d54197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:41.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.430+0000 7f8d5c200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d54102f90 0x7f8d54198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:41.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.430+0000 7f8d5c200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d54198c00 con 0x7f8d541025f0 2024-05-27T09:34:41.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.430+0000 7f8d5c200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5419c990 con 0x7f8d54102f90 2024-05-27T09:34:41.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.431+0000 7f8d59800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d541025f0 0x7f8d54197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:41.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.431+0000 7f8d59800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d541025f0 0x7f8d54197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43316/0 (socket says 172.21.15.119:43316) 2024-05-27T09:34:41.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.431+0000 7f8d59800700 1 -- 172.21.15.119:0/2344776642 learned_addr learned my addr 172.21.15.119:0/2344776642 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:34:41.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.431+0000 7f8d58e00700 1 --2- 172.21.15.119:0/2344776642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d54102f90 0x7f8d54198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:41.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.432+0000 7f8d59800700 1 -- 172.21.15.119:0/2344776642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d54102f90 msgr2=0x7f8d54198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:41.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.432+0000 7f8d59800700 1 --2- 172.21.15.119:0/2344776642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d54102f90 0x7f8d54198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:41.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.432+0000 7f8d59800700 1 -- 172.21.15.119:0/2344776642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d400095f0 con 0x7f8d541025f0 2024-05-27T09:34:41.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.432+0000 7f8d59800700 1 --2- 172.21.15.119:0/2344776642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d541025f0 0x7f8d54197fe0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800c8a0 tx=0x7f8d4800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:41.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.433+0000 7f8d52000700 1 -- 172.21.15.119:0/2344776642 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d480078e0 con 0x7f8d541025f0 2024-05-27T09:34:41.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.433+0000 7f8d52000700 1 -- 172.21.15.119:0/2344776642 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d4800f450 con 0x7f8d541025f0 2024-05-27T09:34:41.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.433+0000 7f8d52000700 1 -- 172.21.15.119:0/2344776642 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d4800e5e0 con 0x7f8d541025f0 2024-05-27T09:34:41.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.433+0000 7f8d5c200700 1 -- 172.21.15.119:0/2344776642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5419cb90 con 0x7f8d541025f0 2024-05-27T09:34:41.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.433+0000 7f8d5c200700 1 -- 172.21.15.119:0/2344776642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5419d0e0 con 0x7f8d541025f0 2024-05-27T09:34:41.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.434+0000 7f8d5c200700 1 -- 172.21.15.119:0/2344776642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d34005320 con 0x7f8d541025f0 2024-05-27T09:34:41.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.436+0000 7f8d52000700 1 -- 172.21.15.119:0/2344776642 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 38) v1 ==== 100184+0+0 (secure 0 0 0) 0x7f8d4800f5c0 con 0x7f8d541025f0 2024-05-27T09:34:41.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.437+0000 7f8d52000700 1 --2- 172.21.15.119:0/2344776642 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8d3c078b60 0x7f8d3c07b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:41.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.437+0000 7f8d52000700 1 -- 172.21.15.119:0/2344776642 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8d4809ae80 con 0x7f8d541025f0 2024-05-27T09:34:41.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.437+0000 7f8d58e00700 1 --2- 172.21.15.119:0/2344776642 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8d3c078b60 0x7f8d3c07b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:41.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.438+0000 7f8d58e00700 1 --2- 172.21.15.119:0/2344776642 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8d3c078b60 0x7f8d3c07b010 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8d54199600 tx=0x7f8d4001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:41.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.444+0000 7f8d52000700 1 -- 172.21.15.119:0/2344776642 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f8d480628c0 con 0x7f8d541025f0 2024-05-27T09:34:41.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.710+0000 7f8d5c200700 1 -- 172.21.15.119:0/2344776642 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d34000bf0 con 0x7f8d3c078b60 2024-05-27T09:34:41.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.713+0000 7f8d52000700 1 -- 172.21.15.119:0/2344776642 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f8d34000bf0 con 0x7f8d3c078b60 2024-05-27T09:34:41.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.716+0000 7f8d5c200700 1 -- 172.21.15.119:0/2344776642 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8d3c078b60 msgr2=0x7f8d3c07b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:41.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.716+0000 7f8d5c200700 1 --2- 172.21.15.119:0/2344776642 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8d3c078b60 0x7f8d3c07b010 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8d54199600 tx=0x7f8d4001a040 comp rx=0 tx=0).stop 2024-05-27T09:34:41.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.716+0000 7f8d5c200700 1 -- 172.21.15.119:0/2344776642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d541025f0 msgr2=0x7f8d54197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:41.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.716+0000 7f8d5c200700 1 --2- 172.21.15.119:0/2344776642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d541025f0 0x7f8d54197fe0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800c8a0 tx=0x7f8d4800cbb0 comp rx=0 tx=0).stop 2024-05-27T09:34:41.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.716+0000 7f8d5c200700 1 -- 172.21.15.119:0/2344776642 shutdown_connections 2024-05-27T09:34:41.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.716+0000 7f8d5c200700 1 --2- 172.21.15.119:0/2344776642 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8d3c078b60 0x7f8d3c07b010 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:41.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.716+0000 7f8d5c200700 1 --2- 172.21.15.119:0/2344776642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d541025f0 0x7f8d54197fe0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:41.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.716+0000 7f8d5c200700 1 --2- 172.21.15.119:0/2344776642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d54102f90 0x7f8d54198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:41.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.716+0000 7f8d5c200700 1 -- 172.21.15.119:0/2344776642 >> 172.21.15.119:0/2344776642 conn(0x7f8d540fde90 msgr2=0x7f8d540ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:41.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.716+0000 7f8d5c200700 1 -- 172.21.15.119:0/2344776642 shutdown_connections 2024-05-27T09:34:41.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.716+0000 7f8d5c200700 1 -- 172.21.15.119:0/2344776642 wait complete. 2024-05-27T09:34:41.734 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:34:41.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.876+0000 7ff22ce00700 1 -- 172.21.15.119:0/1177282128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff228068fd0 msgr2=0x7ff228104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:41.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.876+0000 7ff22ce00700 1 --2- 172.21.15.119:0/1177282128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff228068fd0 0x7ff228104df0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff210009910 tx=0x7ff210009c20 comp rx=0 tx=0).stop 2024-05-27T09:34:41.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.876+0000 7ff22ce00700 1 -- 172.21.15.119:0/1177282128 shutdown_connections 2024-05-27T09:34:41.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.876+0000 7ff22ce00700 1 --2- 172.21.15.119:0/1177282128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff228068fd0 0x7ff228104df0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:41.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.876+0000 7ff22ce00700 1 --2- 172.21.15.119:0/1177282128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff228068630 0x7ff228068a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:41.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.876+0000 7ff22ce00700 1 -- 172.21.15.119:0/1177282128 >> 172.21.15.119:0/1177282128 conn(0x7ff228074f50 msgr2=0x7ff228075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:41.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.876+0000 7ff22ce00700 1 -- 172.21.15.119:0/1177282128 shutdown_connections 2024-05-27T09:34:41.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.876+0000 7ff22ce00700 1 -- 172.21.15.119:0/1177282128 wait complete. 2024-05-27T09:34:41.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.877+0000 7ff22ce00700 1 Processor -- start 2024-05-27T09:34:41.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.878+0000 7ff22ce00700 1 -- start start 2024-05-27T09:34:41.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.878+0000 7ff22ce00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff228068630 0x7ff2280ff200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:41.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.878+0000 7ff22ce00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff228068fd0 0x7ff2280ff740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:41.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.878+0000 7ff22ce00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2280ffc80 con 0x7ff228068630 2024-05-27T09:34:41.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.878+0000 7ff22ce00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2280ffdc0 con 0x7ff228068fd0 2024-05-27T09:34:41.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.878+0000 7ff225e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff228068630 0x7ff2280ff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:41.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.878+0000 7ff225e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff228068630 0x7ff2280ff200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43332/0 (socket says 172.21.15.119:43332) 2024-05-27T09:34:41.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.878+0000 7ff225e00700 1 -- 172.21.15.119:0/661026645 learned_addr learned my addr 172.21.15.119:0/661026645 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:34:41.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.879+0000 7ff225e00700 1 -- 172.21.15.119:0/661026645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff228068fd0 msgr2=0x7ff2280ff740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:41.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.879+0000 7ff225e00700 1 --2- 172.21.15.119:0/661026645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff228068fd0 0x7ff2280ff740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:41.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.879+0000 7ff225e00700 1 -- 172.21.15.119:0/661026645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2100095f0 con 0x7ff228068630 2024-05-27T09:34:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.879+0000 7ff225e00700 1 --2- 172.21.15.119:0/661026645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff228068630 0x7ff2280ff200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff21800d8d0 tx=0x7ff21800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.879+0000 7ff21ea00700 1 -- 172.21.15.119:0/661026645 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff218009940 con 0x7ff228068630 2024-05-27T09:34:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.879+0000 7ff21ea00700 1 -- 172.21.15.119:0/661026645 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff218010460 con 0x7ff228068630 2024-05-27T09:34:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.880+0000 7ff22ce00700 1 -- 172.21.15.119:0/661026645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2281a1ff0 con 0x7ff228068630 2024-05-27T09:34:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.880+0000 7ff21ea00700 1 -- 172.21.15.119:0/661026645 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff21800f5f0 con 0x7ff228068630 2024-05-27T09:34:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.880+0000 7ff22ce00700 1 -- 172.21.15.119:0/661026645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2281a2400 con 0x7ff228068630 2024-05-27T09:34:41.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.881+0000 7ff22ce00700 1 -- 172.21.15.119:0/661026645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2281086d0 con 0x7ff228068630 2024-05-27T09:34:41.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.882+0000 7ff21ea00700 1 -- 172.21.15.119:0/661026645 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 38) v1 ==== 100184+0+0 (secure 0 0 0) 0x7ff2180105d0 con 0x7ff228068630 2024-05-27T09:34:41.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.883+0000 7ff21ea00700 1 --2- 172.21.15.119:0/661026645 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff20c078b70 0x7ff20c07b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:41.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.883+0000 7ff21ea00700 1 -- 172.21.15.119:0/661026645 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff21809af10 con 0x7ff228068630 2024-05-27T09:34:41.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.883+0000 7ff21fe00700 1 --2- 172.21.15.119:0/661026645 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff20c078b70 0x7ff20c07b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:41.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.884+0000 7ff21fe00700 1 --2- 172.21.15.119:0/661026645 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff20c078b70 0x7ff20c07b020 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff21000b5c0 tx=0x7ff210005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:41.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:41.889+0000 7ff21ea00700 1 -- 172.21.15.119:0/661026645 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7ff218062950 con 0x7ff228068630 2024-05-27T09:34:42.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.160+0000 7ff22ce00700 1 -- 172.21.15.119:0/661026645 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff22802cca0 con 0x7ff20c078b70 2024-05-27T09:34:42.163 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.162+0000 7ff21ea00700 1 -- 172.21.15.119:0/661026645 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7ff22802cca0 con 0x7ff20c078b70 2024-05-27T09:34:42.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.165+0000 7ff22ce00700 1 -- 172.21.15.119:0/661026645 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff20c078b70 msgr2=0x7ff20c07b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:42.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.165+0000 7ff22ce00700 1 --2- 172.21.15.119:0/661026645 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff20c078b70 0x7ff20c07b020 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff21000b5c0 tx=0x7ff210005960 comp rx=0 tx=0).stop 2024-05-27T09:34:42.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.165+0000 7ff22ce00700 1 -- 172.21.15.119:0/661026645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff228068630 msgr2=0x7ff2280ff200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:42.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.165+0000 7ff22ce00700 1 --2- 172.21.15.119:0/661026645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff228068630 0x7ff2280ff200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff21800d8d0 tx=0x7ff21800dc90 comp rx=0 tx=0).stop 2024-05-27T09:34:42.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.165+0000 7ff22ce00700 1 -- 172.21.15.119:0/661026645 shutdown_connections 2024-05-27T09:34:42.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.165+0000 7ff22ce00700 1 --2- 172.21.15.119:0/661026645 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff20c078b70 0x7ff20c07b020 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:42.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.165+0000 7ff22ce00700 1 --2- 172.21.15.119:0/661026645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff228068630 0x7ff2280ff200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:42.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.165+0000 7ff22ce00700 1 --2- 172.21.15.119:0/661026645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff228068fd0 0x7ff2280ff740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:42.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.165+0000 7ff22ce00700 1 -- 172.21.15.119:0/661026645 >> 172.21.15.119:0/661026645 conn(0x7ff228074f50 msgr2=0x7ff2280fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:42.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.165+0000 7ff22ce00700 1 -- 172.21.15.119:0/661026645 shutdown_connections 2024-05-27T09:34:42.167 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.165+0000 7ff22ce00700 1 -- 172.21.15.119:0/661026645 wait complete. 2024-05-27T09:34:42.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.319+0000 7ff18b200700 1 -- 172.21.15.119:0/2345934481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff18410b610 msgr2=0x7ff18410b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.319+0000 7ff18b200700 1 --2- 172.21.15.119:0/2345934481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff18410b610 0x7ff18410b9e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff178009910 tx=0x7ff178009c20 comp rx=0 tx=0).stop 2024-05-27T09:34:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.319+0000 7ff18b200700 1 -- 172.21.15.119:0/2345934481 shutdown_connections 2024-05-27T09:34:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.319+0000 7ff18b200700 1 --2- 172.21.15.119:0/2345934481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff184106b80 0x7ff184106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.319+0000 7ff18b200700 1 --2- 172.21.15.119:0/2345934481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff18410b610 0x7ff18410b9e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.319+0000 7ff18b200700 1 -- 172.21.15.119:0/2345934481 >> 172.21.15.119:0/2345934481 conn(0x7ff184075df0 msgr2=0x7ff184078200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.319+0000 7ff18b200700 1 -- 172.21.15.119:0/2345934481 shutdown_connections 2024-05-27T09:34:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.320+0000 7ff18b200700 1 -- 172.21.15.119:0/2345934481 wait complete. 2024-05-27T09:34:42.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.320+0000 7ff18b200700 1 Processor -- start 2024-05-27T09:34:42.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.321+0000 7ff18b200700 1 -- start start 2024-05-27T09:34:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.321+0000 7ff18b200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff184106b80 0x7ff18419c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.321+0000 7ff18b200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff18410b610 0x7ff18419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.321+0000 7ff18b200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18419cd10 con 0x7ff184106b80 2024-05-27T09:34:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.321+0000 7ff18b200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1841a0aa0 con 0x7ff18410b610 2024-05-27T09:34:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.321+0000 7ff189e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff184106b80 0x7ff18419c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.321+0000 7ff189e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff184106b80 0x7ff18419c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43356/0 (socket says 172.21.15.119:43356) 2024-05-27T09:34:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.321+0000 7ff189e00700 1 -- 172.21.15.119:0/1784153308 learned_addr learned my addr 172.21.15.119:0/1784153308 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:34:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.322+0000 7ff189400700 1 --2- 172.21.15.119:0/1784153308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff18410b610 0x7ff18419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.322+0000 7ff189e00700 1 -- 172.21.15.119:0/1784153308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff18410b610 msgr2=0x7ff18419c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.322+0000 7ff189e00700 1 --2- 172.21.15.119:0/1784153308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff18410b610 0x7ff18419c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.322+0000 7ff189e00700 1 -- 172.21.15.119:0/1784153308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1780095f0 con 0x7ff184106b80 2024-05-27T09:34:42.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.322+0000 7ff189e00700 1 --2- 172.21.15.119:0/1784153308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff184106b80 0x7ff18419c0f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff17800f690 tx=0x7ff17800f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.322+0000 7ff17ea00700 1 -- 172.21.15.119:0/1784153308 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff17801d070 con 0x7ff184106b80 2024-05-27T09:34:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.322+0000 7ff17ea00700 1 -- 172.21.15.119:0/1784153308 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1780050d0 con 0x7ff184106b80 2024-05-27T09:34:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.322+0000 7ff18b200700 1 -- 172.21.15.119:0/1784153308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1841a0d20 con 0x7ff184106b80 2024-05-27T09:34:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.322+0000 7ff17ea00700 1 -- 172.21.15.119:0/1784153308 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff178022470 con 0x7ff184106b80 2024-05-27T09:34:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.323+0000 7ff18b200700 1 -- 172.21.15.119:0/1784153308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1841a1290 con 0x7ff184106b80 2024-05-27T09:34:42.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.324+0000 7ff18b200700 1 -- 172.21.15.119:0/1784153308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff18404f2a0 con 0x7ff184106b80 2024-05-27T09:34:42.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.325+0000 7ff17ea00700 1 -- 172.21.15.119:0/1784153308 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 38) v1 ==== 100184+0+0 (secure 0 0 0) 0x7ff178003680 con 0x7ff184106b80 2024-05-27T09:34:42.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.326+0000 7ff17ea00700 1 --2- 172.21.15.119:0/1784153308 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff16c078b60 0x7ff16c07b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:42.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.326+0000 7ff17ea00700 1 -- 172.21.15.119:0/1784153308 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff17809c440 con 0x7ff184106b80 2024-05-27T09:34:42.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.326+0000 7ff189400700 1 --2- 172.21.15.119:0/1784153308 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff16c078b60 0x7ff16c07b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:42.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.327+0000 7ff189400700 1 --2- 172.21.15.119:0/1784153308 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff16c078b60 0x7ff16c07b010 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff18419d710 tx=0x7ff174009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:42.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.331+0000 7ff17ea00700 1 -- 172.21.15.119:0/1784153308 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7ff178063f00 con 0x7ff184106b80 2024-05-27T09:34:42.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.592+0000 7ff18b200700 1 -- 172.21.15.119:0/1784153308 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1841a1860 con 0x7ff16c078b60 2024-05-27T09:34:42.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.614+0000 7ff17ea00700 1 -- 172.21.15.119:0/1784153308 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff1841a1860 con 0x7ff16c078b60 2024-05-27T09:34:42.615 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:34:42.615 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (2m) 7s ago 15m 13.9M - 0.25.0 c8568f914cd2 6b90564dcef7 2024-05-27T09:34:42.615 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (16m) 7s ago 16m 8190k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:34:42.616 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (14m) 10s ago 14m 8048k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:34:42.616 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (16m) 7s ago 16m 6665k - 18.2.0 dc2bc1663786 e5fafde44183 2024-05-27T09:34:42.616 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (14m) 10s ago 14m 6660k - 18.2.0 dc2bc1663786 46956ab99bab 2024-05-27T09:34:42.616 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (98s) 7s ago 15m 74.7M - 9.4.12 2bacad6d85d8 b3abdab08db4 2024-05-27T09:34:42.616 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (8m) 7s ago 8m 16.6M - 18.2.0 dc2bc1663786 93b21713fb1e 2024-05-27T09:34:42.616 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (8m) 7s ago 8m 130M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:34:42.616 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (8m) 10s ago 8m 16.2M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:34:42.616 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (8m) 10s ago 8m 129M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:34:42.616 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (4m) 7s ago 17m 558M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:34:42.616 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (3m) 10s ago 14m 529M - 19.0.0-3868-ga6507388 850a06a84d28 2fc3a80696fa 2024-05-27T09:34:42.616 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (26s) 7s ago 17m 96.6M 2048M 19.0.0-3868-ga6507388 850a06a84d28 3e5c7a9ad372 2024-05-27T09:34:42.616 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (14m) 10s ago 14m 51.9M 2048M 18.2.0 dc2bc1663786 60216fd67078 2024-05-27T09:34:42.617 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (3m) 7s ago 15m 7482k - 1.5.0 0da6a335fe13 709c37815cff 2024-05-27T09:34:42.617 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (3m) 10s ago 14m 7884k - 1.5.0 0da6a335fe13 999047a1a5bf 2024-05-27T09:34:42.617 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (13m) 7s ago 13m 225M 1216M 18.2.0 dc2bc1663786 825bd1392508 2024-05-27T09:34:42.617 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (12m) 7s ago 12m 253M 1216M 18.2.0 dc2bc1663786 fce9819903d9 2024-05-27T09:34:42.617 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (12m) 7s ago 12m 190M 1216M 18.2.0 dc2bc1663786 3955a6112c1c 2024-05-27T09:34:42.617 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (11m) 10s ago 11m 271M 2240M 18.2.0 dc2bc1663786 693c92220622 2024-05-27T09:34:42.617 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (11m) 10s ago 11m 247M 2240M 18.2.0 dc2bc1663786 7559e658e917 2024-05-27T09:34:42.617 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (10m) 10s ago 10m 231M 2240M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:34:42.617 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (2m) 7s ago 15m 62.3M - 2.43.0 a07b618ecd1d cba50454ed21 2024-05-27T09:34:42.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.617+0000 7ff18b200700 1 -- 172.21.15.119:0/1784153308 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff16c078b60 msgr2=0x7ff16c07b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:42.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.617+0000 7ff18b200700 1 --2- 172.21.15.119:0/1784153308 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff16c078b60 0x7ff16c07b010 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff18419d710 tx=0x7ff174009260 comp rx=0 tx=0).stop 2024-05-27T09:34:42.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.617+0000 7ff18b200700 1 -- 172.21.15.119:0/1784153308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff184106b80 msgr2=0x7ff18419c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:42.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.617+0000 7ff18b200700 1 --2- 172.21.15.119:0/1784153308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff184106b80 0x7ff18419c0f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff17800f690 tx=0x7ff17800f770 comp rx=0 tx=0).stop 2024-05-27T09:34:42.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.617+0000 7ff18b200700 1 -- 172.21.15.119:0/1784153308 shutdown_connections 2024-05-27T09:34:42.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.617+0000 7ff18b200700 1 --2- 172.21.15.119:0/1784153308 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff16c078b60 0x7ff16c07b010 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:42.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.617+0000 7ff18b200700 1 --2- 172.21.15.119:0/1784153308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff184106b80 0x7ff18419c0f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:42.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.617+0000 7ff18b200700 1 --2- 172.21.15.119:0/1784153308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff18410b610 0x7ff18419c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:42.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.617+0000 7ff18b200700 1 -- 172.21.15.119:0/1784153308 >> 172.21.15.119:0/1784153308 conn(0x7ff184075df0 msgr2=0x7ff184077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:42.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.618+0000 7ff18b200700 1 -- 172.21.15.119:0/1784153308 shutdown_connections 2024-05-27T09:34:42.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.618+0000 7ff18b200700 1 -- 172.21.15.119:0/1784153308 wait complete. 2024-05-27T09:34:42.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.785+0000 7f0f32800700 1 -- 172.21.15.119:0/1495229195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f2c100d50 msgr2=0x7f0f2c104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:42.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.785+0000 7f0f32800700 1 --2- 172.21.15.119:0/1495229195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f2c100d50 0x7f0f2c104d20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009910 tx=0x7f0f18009c20 comp rx=0 tx=0).stop 2024-05-27T09:34:42.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.785+0000 7f0f32800700 1 -- 172.21.15.119:0/1495229195 shutdown_connections 2024-05-27T09:34:42.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.785+0000 7f0f32800700 1 --2- 172.21.15.119:0/1495229195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f2c100d50 0x7f0f2c104d20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:42.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.785+0000 7f0f32800700 1 --2- 172.21.15.119:0/1495229195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f2c1003b0 0x7f0f2c100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:42.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.785+0000 7f0f32800700 1 -- 172.21.15.119:0/1495229195 >> 172.21.15.119:0/1495229195 conn(0x7f0f2c0fbc30 msgr2=0x7f0f2c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:42.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.785+0000 7f0f32800700 1 -- 172.21.15.119:0/1495229195 shutdown_connections 2024-05-27T09:34:42.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.786+0000 7f0f32800700 1 -- 172.21.15.119:0/1495229195 wait complete. 2024-05-27T09:34:42.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.787+0000 7f0f32800700 1 Processor -- start 2024-05-27T09:34:42.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.787+0000 7f0f32800700 1 -- start start 2024-05-27T09:34:42.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.787+0000 7f0f32800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f2c1003b0 0x7f0f2c197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:42.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.787+0000 7f0f32800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f2c100d50 0x7f0f2c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:42.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.787+0000 7f0f32800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c198b70 con 0x7f0f2c100d50 2024-05-27T09:34:42.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.787+0000 7f0f32800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c19c900 con 0x7f0f2c1003b0 2024-05-27T09:34:42.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.788+0000 7f0f2aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f2c100d50 0x7f0f2c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:42.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.788+0000 7f0f2aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f2c100d50 0x7f0f2c198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43368/0 (socket says 172.21.15.119:43368) 2024-05-27T09:34:42.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.788+0000 7f0f2aa00700 1 -- 172.21.15.119:0/664249629 learned_addr learned my addr 172.21.15.119:0/664249629 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:34:42.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.788+0000 7f0f2b400700 1 --2- 172.21.15.119:0/664249629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f2c1003b0 0x7f0f2c197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:42.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.788+0000 7f0f2aa00700 1 -- 172.21.15.119:0/664249629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f2c1003b0 msgr2=0x7f0f2c197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:42.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.788+0000 7f0f2aa00700 1 --2- 172.21.15.119:0/664249629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f2c1003b0 0x7f0f2c197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:42.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.788+0000 7f0f2aa00700 1 -- 172.21.15.119:0/664249629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f180095f0 con 0x7f0f2c100d50 2024-05-27T09:34:42.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.789+0000 7f0f2aa00700 1 --2- 172.21.15.119:0/664249629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f2c100d50 0x7f0f2c198490 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800bda0 tx=0x7f0f1800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:42.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.789+0000 7f0f23e00700 1 -- 172.21.15.119:0/664249629 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f1801d070 con 0x7f0f2c100d50 2024-05-27T09:34:42.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.789+0000 7f0f23e00700 1 -- 172.21.15.119:0/664249629 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f18005dc0 con 0x7f0f2c100d50 2024-05-27T09:34:42.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.789+0000 7f0f23e00700 1 -- 172.21.15.119:0/664249629 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f18017400 con 0x7f0f2c100d50 2024-05-27T09:34:42.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.789+0000 7f0f32800700 1 -- 172.21.15.119:0/664249629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f2c19cb80 con 0x7f0f2c100d50 2024-05-27T09:34:42.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.789+0000 7f0f32800700 1 -- 172.21.15.119:0/664249629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2c19d070 con 0x7f0f2c100d50 2024-05-27T09:34:42.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.790+0000 7f0f32800700 1 -- 172.21.15.119:0/664249629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f2c1044a0 con 0x7f0f2c100d50 2024-05-27T09:34:42.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.793+0000 7f0f23e00700 1 -- 172.21.15.119:0/664249629 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 38) v1 ==== 100184+0+0 (secure 0 0 0) 0x7f0f1800f6f0 con 0x7f0f2c100d50 2024-05-27T09:34:42.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.794+0000 7f0f23e00700 1 --2- 172.21.15.119:0/664249629 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0f10078b60 0x7f0f1007b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:42.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.794+0000 7f0f23e00700 1 -- 172.21.15.119:0/664249629 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0f1809c2c0 con 0x7f0f2c100d50 2024-05-27T09:34:42.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.794+0000 7f0f2b400700 1 --2- 172.21.15.119:0/664249629 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0f10078b60 0x7f0f1007b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:42.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.795+0000 7f0f2b400700 1 --2- 172.21.15.119:0/664249629 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0f10078b60 0x7f0f1007b010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c005950 tx=0x7f0f1c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:42.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:42.800+0000 7f0f23e00700 1 -- 172.21.15.119:0/664249629 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f0f18063d00 con 0x7f0f2c100d50 2024-05-27T09:34:43.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.169+0000 7f0f32800700 1 -- 172.21.15.119:0/664249629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0f2c04ea50 con 0x7f0f2c100d50 2024-05-27T09:34:43.170 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:43 smithi119 ceph-mon[88269]: pgmap v10: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-27T09:34:43.170 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:43 smithi119 ceph-mon[88269]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:34:43.171 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:43 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:34:43.171 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:43 smithi119 ceph-mon[88269]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:43.171 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:43 smithi119 ceph-mon[88269]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:43.171 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:43 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:34:43.171 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:43 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:34:43.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.170+0000 7f0f23e00700 1 -- 172.21.15.119:0/664249629 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+779 (secure 0 0 0) 0x7f0f180636a0 con 0x7f0f2c100d50 2024-05-27T09:34:43.171 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:34:43.171 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:34:43.172 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-05-27T09:34:43.172 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 1 2024-05-27T09:34:43.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:34:43.172 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:34:43.172 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:34:43.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:34:43.172 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:34:43.172 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-05-27T09:34:43.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:34:43.172 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:34:43.172 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-05-27T09:34:43.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:34:43.173 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:34:43.173 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 11, 2024-05-27T09:34:43.173 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 3 2024-05-27T09:34:43.173 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:34:43.173 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:34:43.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.174+0000 7f0f32800700 1 -- 172.21.15.119:0/664249629 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0f10078b60 msgr2=0x7f0f1007b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:43.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.174+0000 7f0f32800700 1 --2- 172.21.15.119:0/664249629 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0f10078b60 0x7f0f1007b010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c005950 tx=0x7f0f1c00b410 comp rx=0 tx=0).stop 2024-05-27T09:34:43.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.174+0000 7f0f32800700 1 -- 172.21.15.119:0/664249629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f2c100d50 msgr2=0x7f0f2c198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:43.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.174+0000 7f0f32800700 1 --2- 172.21.15.119:0/664249629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f2c100d50 0x7f0f2c198490 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800bda0 tx=0x7f0f1800bdd0 comp rx=0 tx=0).stop 2024-05-27T09:34:43.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.174+0000 7f0f32800700 1 -- 172.21.15.119:0/664249629 shutdown_connections 2024-05-27T09:34:43.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.174+0000 7f0f32800700 1 --2- 172.21.15.119:0/664249629 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0f10078b60 0x7f0f1007b010 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:43.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.174+0000 7f0f32800700 1 --2- 172.21.15.119:0/664249629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f2c1003b0 0x7f0f2c197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:43.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.174+0000 7f0f32800700 1 --2- 172.21.15.119:0/664249629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f2c100d50 0x7f0f2c198490 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:43.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.174+0000 7f0f32800700 1 -- 172.21.15.119:0/664249629 >> 172.21.15.119:0/664249629 conn(0x7f0f2c0fbc30 msgr2=0x7f0f2c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:43.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.174+0000 7f0f32800700 1 -- 172.21.15.119:0/664249629 shutdown_connections 2024-05-27T09:34:43.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.174+0000 7f0f32800700 1 -- 172.21.15.119:0/664249629 wait complete. 2024-05-27T09:34:43.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.321+0000 7fa91e400700 1 -- 172.21.15.119:0/699489873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa918100d50 msgr2=0x7fa918104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:43.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.321+0000 7fa91e400700 1 --2- 172.21.15.119:0/699489873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa918100d50 0x7fa918104d20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa904009910 tx=0x7fa904009c20 comp rx=0 tx=0).stop 2024-05-27T09:34:43.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.321+0000 7fa91e400700 1 -- 172.21.15.119:0/699489873 shutdown_connections 2024-05-27T09:34:43.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.321+0000 7fa91e400700 1 --2- 172.21.15.119:0/699489873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa918100d50 0x7fa918104d20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:43.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.321+0000 7fa91e400700 1 --2- 172.21.15.119:0/699489873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9181003b0 0x7fa918100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:43.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.321+0000 7fa91e400700 1 -- 172.21.15.119:0/699489873 >> 172.21.15.119:0/699489873 conn(0x7fa9180fbc30 msgr2=0x7fa9180fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:43.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.321+0000 7fa91e400700 1 -- 172.21.15.119:0/699489873 shutdown_connections 2024-05-27T09:34:43.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.322+0000 7fa91e400700 1 -- 172.21.15.119:0/699489873 wait complete. 2024-05-27T09:34:43.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.323+0000 7fa91e400700 1 Processor -- start 2024-05-27T09:34:43.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.323+0000 7fa91e400700 1 -- start start 2024-05-27T09:34:43.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.323+0000 7fa91e400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9181003b0 0x7fa918197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:43.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.323+0000 7fa91e400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa918100d50 0x7fa918198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:43.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.323+0000 7fa91e400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa918198be0 con 0x7fa918100d50 2024-05-27T09:34:43.324 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.323+0000 7fa91e400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91819c970 con 0x7fa9181003b0 2024-05-27T09:34:43.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.323+0000 7fa916a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa918100d50 0x7fa918198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:43.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.323+0000 7fa916a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa918100d50 0x7fa918198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43392/0 (socket says 172.21.15.119:43392) 2024-05-27T09:34:43.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.323+0000 7fa916a00700 1 -- 172.21.15.119:0/4098100438 learned_addr learned my addr 172.21.15.119:0/4098100438 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:34:43.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.324+0000 7fa917400700 1 --2- 172.21.15.119:0/4098100438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9181003b0 0x7fa918197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:43.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.324+0000 7fa916a00700 1 -- 172.21.15.119:0/4098100438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9181003b0 msgr2=0x7fa918197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:43.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.324+0000 7fa916a00700 1 --2- 172.21.15.119:0/4098100438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9181003b0 0x7fa918197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:43.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.324+0000 7fa916a00700 1 -- 172.21.15.119:0/4098100438 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9040095f0 con 0x7fa918100d50 2024-05-27T09:34:43.325 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.324+0000 7fa916a00700 1 --2- 172.21.15.119:0/4098100438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa918100d50 0x7fa918198500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa904005b40 tx=0x7fa90400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:43.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.325+0000 7fa90fe00700 1 -- 172.21.15.119:0/4098100438 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa90401d070 con 0x7fa918100d50 2024-05-27T09:34:43.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.325+0000 7fa90fe00700 1 -- 172.21.15.119:0/4098100438 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa90400bed0 con 0x7fa918100d50 2024-05-27T09:34:43.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.325+0000 7fa91e400700 1 -- 172.21.15.119:0/4098100438 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91819cbf0 con 0x7fa918100d50 2024-05-27T09:34:43.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.325+0000 7fa90fe00700 1 -- 172.21.15.119:0/4098100438 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa904003e70 con 0x7fa918100d50 2024-05-27T09:34:43.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.325+0000 7fa91e400700 1 -- 172.21.15.119:0/4098100438 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91819d0e0 con 0x7fa918100d50 2024-05-27T09:34:43.327 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.326+0000 7fa91e400700 1 -- 172.21.15.119:0/4098100438 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9181044a0 con 0x7fa918100d50 2024-05-27T09:34:43.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.328+0000 7fa90fe00700 1 -- 172.21.15.119:0/4098100438 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 38) v1 ==== 100184+0+0 (secure 0 0 0) 0x7fa9040050d0 con 0x7fa918100d50 2024-05-27T09:34:43.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.329+0000 7fa90fe00700 1 --2- 172.21.15.119:0/4098100438 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fa8fc078b60 0x7fa8fc07b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:43.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.329+0000 7fa90fe00700 1 -- 172.21.15.119:0/4098100438 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa90409c340 con 0x7fa918100d50 2024-05-27T09:34:43.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.329+0000 7fa917400700 1 --2- 172.21.15.119:0/4098100438 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fa8fc078b60 0x7fa8fc07b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:43.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.330+0000 7fa917400700 1 --2- 172.21.15.119:0/4098100438 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fa8fc078b60 0x7fa8fc07b010 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa908009c60 tx=0x7fa908009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:43.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.334+0000 7fa90fe00700 1 -- 172.21.15.119:0/4098100438 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fa904063d00 con 0x7fa918100d50 2024-05-27T09:34:43.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:43 smithi138 ceph-mon[34885]: pgmap v10: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-27T09:34:43.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:43 smithi138 ceph-mon[34885]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:34:43.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:43 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:34:43.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:43 smithi138 ceph-mon[34885]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:43.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:43 smithi138 ceph-mon[34885]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:43.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:43 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:34:43.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:43 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:34:43.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.624+0000 7fa91e400700 1 -- 172.21.15.119:0/4098100438 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa91804ea50 con 0x7fa918100d50 2024-05-27T09:34:43.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.625+0000 7fa90fe00700 1 -- 172.21.15.119:0/4098100438 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1889 (secure 0 0 0) 0x7fa9040636a0 con 0x7fa918100d50 2024-05-27T09:34:43.626 INFO:teuthology.orchestra.run.smithi119.stdout:e22 2024-05-27T09:34:43.626 INFO:teuthology.orchestra.run.smithi119.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T09:34:43.626 INFO:teuthology.orchestra.run.smithi119.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:34:43.626 INFO:teuthology.orchestra.run.smithi119.stdout:legacy client fscid: 1 2024-05-27T09:34:43.626 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:34:43.626 INFO:teuthology.orchestra.run.smithi119.stdout:Filesystem 'cephfs' (1) 2024-05-27T09:34:43.626 INFO:teuthology.orchestra.run.smithi119.stdout:fs_name cephfs 2024-05-27T09:34:43.626 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 22 2024-05-27T09:34:43.627 INFO:teuthology.orchestra.run.smithi119.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-27T09:34:43.627 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-05-27T09:25:53.387388+0000 2024-05-27T09:34:43.627 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2024-05-27T09:34:01.250304+0000 2024-05-27T09:34:43.627 INFO:teuthology.orchestra.run.smithi119.stdout:tableserver 0 2024-05-27T09:34:43.627 INFO:teuthology.orchestra.run.smithi119.stdout:root 0 2024-05-27T09:34:43.627 INFO:teuthology.orchestra.run.smithi119.stdout:session_timeout 60 2024-05-27T09:34:43.627 INFO:teuthology.orchestra.run.smithi119.stdout:session_autoclose 300 2024-05-27T09:34:43.627 INFO:teuthology.orchestra.run.smithi119.stdout:max_file_size 1099511627776 2024-05-27T09:34:43.627 INFO:teuthology.orchestra.run.smithi119.stdout:max_xattr_size 65536 2024-05-27T09:34:43.627 INFO:teuthology.orchestra.run.smithi119.stdout:required_client_features {} 2024-05-27T09:34:43.627 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure 0 2024-05-27T09:34:43.627 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure_osd_epoch 46 2024-05-27T09:34:43.627 INFO:teuthology.orchestra.run.smithi119.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:34:43.628 INFO:teuthology.orchestra.run.smithi119.stdout:max_mds 2 2024-05-27T09:34:43.628 INFO:teuthology.orchestra.run.smithi119.stdout:in 0,1 2024-05-27T09:34:43.628 INFO:teuthology.orchestra.run.smithi119.stdout:up {0=14472,1=24241} 2024-05-27T09:34:43.628 INFO:teuthology.orchestra.run.smithi119.stdout:failed 2024-05-27T09:34:43.628 INFO:teuthology.orchestra.run.smithi119.stdout:damaged 2024-05-27T09:34:43.628 INFO:teuthology.orchestra.run.smithi119.stdout:stopped 2024-05-27T09:34:43.628 INFO:teuthology.orchestra.run.smithi119.stdout:data_pools [3] 2024-05-27T09:34:43.628 INFO:teuthology.orchestra.run.smithi119.stdout:metadata_pool 2 2024-05-27T09:34:43.628 INFO:teuthology.orchestra.run.smithi119.stdout:inline_data enabled 2024-05-27T09:34:43.628 INFO:teuthology.orchestra.run.smithi119.stdout:balancer 2024-05-27T09:34:43.628 INFO:teuthology.orchestra.run.smithi119.stdout:bal_rank_mask -1 2024-05-27T09:34:43.629 INFO:teuthology.orchestra.run.smithi119.stdout:standby_count_wanted 1 2024-05-27T09:34:43.629 INFO:teuthology.orchestra.run.smithi119.stdout:qdb_cluster leader: 0 members: 2024-05-27T09:34:43.629 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.ppmzzk{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:34:43.629 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.ndfkhd{1:24241} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/897058594,v1:172.21.15.138:6825/897058594] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:34:43.629 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:34:43.629 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:34:43.629 INFO:teuthology.orchestra.run.smithi119.stdout:Standby daemons: 2024-05-27T09:34:43.629 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:34:43.629 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.nghcey{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.119:6826/413736191,v1:172.21.15.119:6827/413736191] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:34:43.629 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.alszpk{-1:24253} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/2345607053,v1:172.21.15.138:6827/2345607053] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:34:43.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.627+0000 7fa91e400700 1 -- 172.21.15.119:0/4098100438 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fa8fc078b60 msgr2=0x7fa8fc07b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:43.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.627+0000 7fa91e400700 1 --2- 172.21.15.119:0/4098100438 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fa8fc078b60 0x7fa8fc07b010 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa908009c60 tx=0x7fa908009260 comp rx=0 tx=0).stop 2024-05-27T09:34:43.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.627+0000 7fa91e400700 1 -- 172.21.15.119:0/4098100438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa918100d50 msgr2=0x7fa918198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:43.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.627+0000 7fa91e400700 1 --2- 172.21.15.119:0/4098100438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa918100d50 0x7fa918198500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa904005b40 tx=0x7fa90400f690 comp rx=0 tx=0).stop 2024-05-27T09:34:43.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.627+0000 7fa91e400700 1 -- 172.21.15.119:0/4098100438 shutdown_connections 2024-05-27T09:34:43.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.627+0000 7fa91e400700 1 --2- 172.21.15.119:0/4098100438 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fa8fc078b60 0x7fa8fc07b010 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:43.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.627+0000 7fa91e400700 1 --2- 172.21.15.119:0/4098100438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9181003b0 0x7fa918197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:43.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.627+0000 7fa91e400700 1 --2- 172.21.15.119:0/4098100438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa918100d50 0x7fa918198500 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:43.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.627+0000 7fa91e400700 1 -- 172.21.15.119:0/4098100438 >> 172.21.15.119:0/4098100438 conn(0x7fa9180fbc30 msgr2=0x7fa9180fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:43.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.628+0000 7fa91e400700 1 -- 172.21.15.119:0/4098100438 shutdown_connections 2024-05-27T09:34:43.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.628+0000 7fa91e400700 1 -- 172.21.15.119:0/4098100438 wait complete. 2024-05-27T09:34:43.631 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 22 2024-05-27T09:34:43.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.777+0000 7f6c80000700 1 -- 172.21.15.119:0/1695718516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c78068630 msgr2=0x7f6c78068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:43.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.777+0000 7f6c80000700 1 --2- 172.21.15.119:0/1695718516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c78068630 0x7f6c78068a00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009910 tx=0x7f6c6c009c20 comp rx=0 tx=0).stop 2024-05-27T09:34:43.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.777+0000 7f6c80000700 1 -- 172.21.15.119:0/1695718516 shutdown_connections 2024-05-27T09:34:43.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.777+0000 7f6c80000700 1 --2- 172.21.15.119:0/1695718516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c78068fd0 0x7f6c78104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:43.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.777+0000 7f6c80000700 1 --2- 172.21.15.119:0/1695718516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c78068630 0x7f6c78068a00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:43.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.777+0000 7f6c80000700 1 -- 172.21.15.119:0/1695718516 >> 172.21.15.119:0/1695718516 conn(0x7f6c78074f50 msgr2=0x7f6c78075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:43.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.777+0000 7f6c80000700 1 -- 172.21.15.119:0/1695718516 shutdown_connections 2024-05-27T09:34:43.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.777+0000 7f6c80000700 1 -- 172.21.15.119:0/1695718516 wait complete. 2024-05-27T09:34:43.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.778+0000 7f6c80000700 1 Processor -- start 2024-05-27T09:34:43.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.778+0000 7f6c80000700 1 -- start start 2024-05-27T09:34:43.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.779+0000 7f6c80000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c78068630 0x7f6c780ff200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:43.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.779+0000 7f6c80000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c78068fd0 0x7f6c780ff740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:43.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.779+0000 7f6c80000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780ffc80 con 0x7f6c78068630 2024-05-27T09:34:43.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.779+0000 7f6c80000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780ffdc0 con 0x7f6c78068fd0 2024-05-27T09:34:43.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.779+0000 7f6c7d600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c78068630 0x7f6c780ff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:43.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.779+0000 7f6c7d600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c78068630 0x7f6c780ff200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43410/0 (socket says 172.21.15.119:43410) 2024-05-27T09:34:43.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.779+0000 7f6c7d600700 1 -- 172.21.15.119:0/2888422104 learned_addr learned my addr 172.21.15.119:0/2888422104 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:34:43.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.779+0000 7f6c7cc00700 1 --2- 172.21.15.119:0/2888422104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c78068fd0 0x7f6c780ff740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:43.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.780+0000 7f6c7d600700 1 -- 172.21.15.119:0/2888422104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c78068fd0 msgr2=0x7f6c780ff740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:43.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.780+0000 7f6c7d600700 1 --2- 172.21.15.119:0/2888422104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c78068fd0 0x7f6c780ff740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:43.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.780+0000 7f6c7d600700 1 -- 172.21.15.119:0/2888422104 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c0095f0 con 0x7f6c78068630 2024-05-27T09:34:43.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.780+0000 7f6c7d600700 1 --2- 172.21.15.119:0/2888422104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c78068630 0x7f6c780ff200 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009fd0 tx=0x7f6c6c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:43.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.780+0000 7f6c76000700 1 -- 172.21.15.119:0/2888422104 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c6c01d070 con 0x7f6c78068630 2024-05-27T09:34:43.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.780+0000 7f6c76000700 1 -- 172.21.15.119:0/2888422104 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c6c0050d0 con 0x7f6c78068630 2024-05-27T09:34:43.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.780+0000 7f6c76000700 1 -- 172.21.15.119:0/2888422104 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c6c017400 con 0x7f6c78068630 2024-05-27T09:34:43.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.780+0000 7f6c80000700 1 -- 172.21.15.119:0/2888422104 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c78100040 con 0x7f6c78068630 2024-05-27T09:34:43.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.780+0000 7f6c80000700 1 -- 172.21.15.119:0/2888422104 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c781a2230 con 0x7f6c78068630 2024-05-27T09:34:43.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.781+0000 7f6c80000700 1 -- 172.21.15.119:0/2888422104 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c781086d0 con 0x7f6c78068630 2024-05-27T09:34:43.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.783+0000 7f6c76000700 1 -- 172.21.15.119:0/2888422104 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 38) v1 ==== 100184+0+0 (secure 0 0 0) 0x7f6c6c003780 con 0x7f6c78068630 2024-05-27T09:34:43.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.785+0000 7f6c76000700 1 --2- 172.21.15.119:0/2888422104 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f6c60078b70 0x7f6c6007b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:43.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.785+0000 7f6c76000700 1 -- 172.21.15.119:0/2888422104 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6c6c09c2f0 con 0x7f6c78068630 2024-05-27T09:34:43.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.785+0000 7f6c7cc00700 1 --2- 172.21.15.119:0/2888422104 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f6c60078b70 0x7f6c6007b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:43.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.786+0000 7f6c7cc00700 1 --2- 172.21.15.119:0/2888422104 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f6c60078b70 0x7f6c6007b020 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6c781009f0 tx=0x7f6c64009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:43.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:43.790+0000 7f6c76000700 1 -- 172.21.15.119:0/2888422104 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f6c6c063d30 con 0x7f6c78068630 2024-05-27T09:34:44.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.049+0000 7f6c80000700 1 -- 172.21.15.119:0/2888422104 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c7802cda0 con 0x7f6c60078b70 2024-05-27T09:34:44.050 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:44 smithi119 ceph-mon[88269]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:34:44.050 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:44 smithi119 ceph-mon[88269]: Detected new or changed devices on smithi119 2024-05-27T09:34:44.051 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:44 smithi119 ceph-mon[88269]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:34:44.051 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:44 smithi119 ceph-mon[88269]: Updating smithi119:/etc/ceph/ceph.conf 2024-05-27T09:34:44.051 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:44 smithi119 ceph-mon[88269]: Updating smithi138:/etc/ceph/ceph.conf 2024-05-27T09:34:44.051 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:44 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/664249629' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:34:44.051 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:44 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/4098100438' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:34:44.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.051+0000 7f6c76000700 1 -- 172.21.15.119:0/2888422104 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f6c7802cda0 con 0x7f6c60078b70 2024-05-27T09:34:44.052 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:34:44.052 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:34:44.053 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:34:44.053 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-27T09:34:44.053 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [ 2024-05-27T09:34:44.053 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr" 2024-05-27T09:34:44.053 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-05-27T09:34:44.053 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "3/23 daemons upgraded", 2024-05-27T09:34:44.053 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "", 2024-05-27T09:34:44.053 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:34:44.053 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:34:44.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.055+0000 7f6c80000700 1 -- 172.21.15.119:0/2888422104 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f6c60078b70 msgr2=0x7f6c6007b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:44.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.055+0000 7f6c80000700 1 --2- 172.21.15.119:0/2888422104 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f6c60078b70 0x7f6c6007b020 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6c781009f0 tx=0x7f6c64009260 comp rx=0 tx=0).stop 2024-05-27T09:34:44.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.055+0000 7f6c80000700 1 -- 172.21.15.119:0/2888422104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c78068630 msgr2=0x7f6c780ff200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:44.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.055+0000 7f6c80000700 1 --2- 172.21.15.119:0/2888422104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c78068630 0x7f6c780ff200 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009fd0 tx=0x7f6c6c00bfd0 comp rx=0 tx=0).stop 2024-05-27T09:34:44.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.055+0000 7f6c80000700 1 -- 172.21.15.119:0/2888422104 shutdown_connections 2024-05-27T09:34:44.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.055+0000 7f6c80000700 1 --2- 172.21.15.119:0/2888422104 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f6c60078b70 0x7f6c6007b020 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:44.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.055+0000 7f6c80000700 1 --2- 172.21.15.119:0/2888422104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c78068630 0x7f6c780ff200 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:44.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.055+0000 7f6c80000700 1 --2- 172.21.15.119:0/2888422104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c78068fd0 0x7f6c780ff740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:44.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.055+0000 7f6c80000700 1 -- 172.21.15.119:0/2888422104 >> 172.21.15.119:0/2888422104 conn(0x7f6c78074f50 msgr2=0x7f6c780fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:44.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.055+0000 7f6c80000700 1 -- 172.21.15.119:0/2888422104 shutdown_connections 2024-05-27T09:34:44.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.055+0000 7f6c80000700 1 -- 172.21.15.119:0/2888422104 wait complete. 2024-05-27T09:34:44.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.223+0000 7f56de200700 1 -- 172.21.15.119:0/3712811131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d81003b0 msgr2=0x7f56d8100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:44.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.223+0000 7f56de200700 1 --2- 172.21.15.119:0/3712811131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d81003b0 0x7f56d8100780 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009910 tx=0x7f56c8009c20 comp rx=0 tx=0).stop 2024-05-27T09:34:44.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.224+0000 7f56de200700 1 -- 172.21.15.119:0/3712811131 shutdown_connections 2024-05-27T09:34:44.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.224+0000 7f56de200700 1 --2- 172.21.15.119:0/3712811131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56d8100d50 0x7f56d8104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:44.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.224+0000 7f56de200700 1 --2- 172.21.15.119:0/3712811131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d81003b0 0x7f56d8100780 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:44.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.224+0000 7f56de200700 1 -- 172.21.15.119:0/3712811131 >> 172.21.15.119:0/3712811131 conn(0x7f56d80fbc30 msgr2=0x7f56d80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:44.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.224+0000 7f56de200700 1 -- 172.21.15.119:0/3712811131 shutdown_connections 2024-05-27T09:34:44.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.224+0000 7f56de200700 1 -- 172.21.15.119:0/3712811131 wait complete. 2024-05-27T09:34:44.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.225+0000 7f56de200700 1 Processor -- start 2024-05-27T09:34:44.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.225+0000 7f56de200700 1 -- start start 2024-05-27T09:34:44.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.225+0000 7f56de200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d81003b0 0x7f56d8197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:44.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.225+0000 7f56de200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56d8100d50 0x7f56d81984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:44.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.225+0000 7f56de200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d8198bc0 con 0x7f56d81003b0 2024-05-27T09:34:44.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.225+0000 7f56de200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d819c950 con 0x7f56d8100d50 2024-05-27T09:34:44.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.226+0000 7f56d7400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d81003b0 0x7f56d8197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:44.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.226+0000 7f56d7400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d81003b0 0x7f56d8197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43428/0 (socket says 172.21.15.119:43428) 2024-05-27T09:34:44.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.226+0000 7f56d7400700 1 -- 172.21.15.119:0/3482670338 learned_addr learned my addr 172.21.15.119:0/3482670338 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:34:44.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.226+0000 7f56d6a00700 1 --2- 172.21.15.119:0/3482670338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56d8100d50 0x7f56d81984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:44.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.226+0000 7f56d7400700 1 -- 172.21.15.119:0/3482670338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56d8100d50 msgr2=0x7f56d81984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:44.227 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.226+0000 7f56d7400700 1 --2- 172.21.15.119:0/3482670338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56d8100d50 0x7f56d81984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:44.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.226+0000 7f56d7400700 1 -- 172.21.15.119:0/3482670338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c80095f0 con 0x7f56d81003b0 2024-05-27T09:34:44.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.227+0000 7f56d7400700 1 --2- 172.21.15.119:0/3482670338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d81003b0 0x7f56d8197fa0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f56c800b5c0 tx=0x7f56c800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:44.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.227+0000 7f56cfe00700 1 -- 172.21.15.119:0/3482670338 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56c801d070 con 0x7f56d81003b0 2024-05-27T09:34:44.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.227+0000 7f56de200700 1 -- 172.21.15.119:0/3482670338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d819cbd0 con 0x7f56d81003b0 2024-05-27T09:34:44.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.227+0000 7f56cfe00700 1 -- 172.21.15.119:0/3482670338 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f56c8005e00 con 0x7f56d81003b0 2024-05-27T09:34:44.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.227+0000 7f56cfe00700 1 -- 172.21.15.119:0/3482670338 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56c8017400 con 0x7f56d81003b0 2024-05-27T09:34:44.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.227+0000 7f56de200700 1 -- 172.21.15.119:0/3482670338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d819d0c0 con 0x7f56d81003b0 2024-05-27T09:34:44.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.228+0000 7f56de200700 1 -- 172.21.15.119:0/3482670338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d81044a0 con 0x7f56d81003b0 2024-05-27T09:34:44.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.230+0000 7f56cfe00700 1 -- 172.21.15.119:0/3482670338 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 38) v1 ==== 100184+0+0 (secure 0 0 0) 0x7f56c800f460 con 0x7f56d81003b0 2024-05-27T09:34:44.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.231+0000 7f56cfe00700 1 --2- 172.21.15.119:0/3482670338 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f56bc078b60 0x7f56bc07b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:34:44.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.231+0000 7f56cfe00700 1 -- 172.21.15.119:0/3482670338 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f56c809c770 con 0x7f56d81003b0 2024-05-27T09:34:44.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.231+0000 7f56d6a00700 1 --2- 172.21.15.119:0/3482670338 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f56bc078b60 0x7f56bc07b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:34:44.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.232+0000 7f56d6a00700 1 --2- 172.21.15.119:0/3482670338 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f56bc078b60 0x7f56bc07b010 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f56c4005950 tx=0x7f56c40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:34:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.236+0000 7f56cfe00700 1 -- 172.21.15.119:0/3482670338 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f56c80641b0 con 0x7f56d81003b0 2024-05-27T09:34:44.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:44 smithi138 ceph-mon[34885]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:34:44.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:44 smithi138 ceph-mon[34885]: Detected new or changed devices on smithi119 2024-05-27T09:34:44.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:44 smithi138 ceph-mon[34885]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:34:44.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:44 smithi138 ceph-mon[34885]: Updating smithi119:/etc/ceph/ceph.conf 2024-05-27T09:34:44.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:44 smithi138 ceph-mon[34885]: Updating smithi138:/etc/ceph/ceph.conf 2024-05-27T09:34:44.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:44 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/664249629' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:34:44.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:44 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/4098100438' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:34:44.570 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.570+0000 7f56de200700 1 -- 172.21.15.119:0/3482670338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f56d804ea50 con 0x7f56d81003b0 2024-05-27T09:34:44.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.570+0000 7f56cfe00700 1 -- 172.21.15.119:0/3482670338 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f56c8063b50 con 0x7f56d81003b0 2024-05-27T09:34:44.571 INFO:teuthology.orchestra.run.smithi119.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-27T09:34:44.571 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:34:44.571 INFO:teuthology.orchestra.run.smithi119.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:34:44.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.573+0000 7f56de200700 1 -- 172.21.15.119:0/3482670338 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f56bc078b60 msgr2=0x7f56bc07b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:44.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.573+0000 7f56de200700 1 --2- 172.21.15.119:0/3482670338 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f56bc078b60 0x7f56bc07b010 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f56c4005950 tx=0x7f56c40058e0 comp rx=0 tx=0).stop 2024-05-27T09:34:44.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.573+0000 7f56de200700 1 -- 172.21.15.119:0/3482670338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d81003b0 msgr2=0x7f56d8197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:34:44.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.573+0000 7f56de200700 1 --2- 172.21.15.119:0/3482670338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d81003b0 0x7f56d8197fa0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f56c800b5c0 tx=0x7f56c800be20 comp rx=0 tx=0).stop 2024-05-27T09:34:44.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.573+0000 7f56de200700 1 -- 172.21.15.119:0/3482670338 shutdown_connections 2024-05-27T09:34:44.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.573+0000 7f56de200700 1 --2- 172.21.15.119:0/3482670338 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f56bc078b60 0x7f56bc07b010 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:44.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.573+0000 7f56de200700 1 --2- 172.21.15.119:0/3482670338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d81003b0 0x7f56d8197fa0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:44.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.573+0000 7f56de200700 1 --2- 172.21.15.119:0/3482670338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56d8100d50 0x7f56d81984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:34:44.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.573+0000 7f56de200700 1 -- 172.21.15.119:0/3482670338 >> 172.21.15.119:0/3482670338 conn(0x7f56d80fbc30 msgr2=0x7f56d80fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:34:44.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.573+0000 7f56de200700 1 -- 172.21.15.119:0/3482670338 shutdown_connections 2024-05-27T09:34:44.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:34:44.573+0000 7f56de200700 1 -- 172.21.15.119:0/3482670338 wait complete. 2024-05-27T09:34:45.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:45 smithi138 ceph-mon[34885]: Updating smithi119:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:34:45.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:45 smithi138 ceph-mon[34885]: pgmap v11: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-27T09:34:45.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:45 smithi138 ceph-mon[34885]: Updating smithi138:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:34:45.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:45 smithi138 ceph-mon[34885]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-05-27T09:34:45.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:45 smithi138 ceph-mon[34885]: from='client.? 172.21.15.119:0/3482670338' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:34:45.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:45 smithi138 ceph-mon[34885]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:45.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:45 smithi119 ceph-mon[88269]: Updating smithi119:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:34:45.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:45 smithi119 ceph-mon[88269]: pgmap v11: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-27T09:34:45.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:45 smithi119 ceph-mon[88269]: Updating smithi138:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T09:34:45.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:45 smithi119 ceph-mon[88269]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-05-27T09:34:45.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:45 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/3482670338' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:34:45.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:45 smithi119 ceph-mon[88269]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:46.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:46 smithi138 ceph-mon[34885]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:34:46.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:46 smithi138 ceph-mon[34885]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-05-27T09:34:46.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:46 smithi138 ceph-mon[34885]: Updating smithi119:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T09:34:46.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:46 smithi138 ceph-mon[34885]: Updating smithi138:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T09:34:46.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:46 smithi138 ceph-mon[34885]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:46.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:46 smithi138 ceph-mon[34885]: pgmap v12: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-27T09:34:46.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:46 smithi138 ceph-mon[34885]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:46.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:46 smithi138 ceph-mon[34885]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:46.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:46 smithi138 ceph-mon[34885]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:46.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:46 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:34:46.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:46 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:34:46.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:46 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-27T09:34:46.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:46 smithi119 ceph-mon[88269]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:34:46.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:46 smithi119 ceph-mon[88269]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-05-27T09:34:46.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:46 smithi119 ceph-mon[88269]: Updating smithi119:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T09:34:46.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:46 smithi119 ceph-mon[88269]: Updating smithi138:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T09:34:46.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:46 smithi119 ceph-mon[88269]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:46.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:46 smithi119 ceph-mon[88269]: pgmap v12: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-27T09:34:46.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:46 smithi119 ceph-mon[88269]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:46.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:46 smithi119 ceph-mon[88269]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:46.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:46 smithi119 ceph-mon[88269]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:46.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:46 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:34:46.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:46 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:34:46.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:46 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-27T09:34:48.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:48 smithi138 ceph-mon[34885]: pgmap v13: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:48.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:48 smithi119 ceph-mon[88269]: pgmap v13: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:49.759 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:49 smithi138 ceph-mon[34885]: Upgrade: Updating mon.smithi138 2024-05-27T09:34:49.759 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:49 smithi138 ceph-mon[34885]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:49.759 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:49 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T09:34:49.759 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:49 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T09:34:49.759 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:49 smithi138 ceph-mon[34885]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:34:49.759 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:49 smithi138 ceph-mon[34885]: Deploying daemon mon.smithi138 on smithi138 2024-05-27T09:34:49.811 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:49 smithi119 ceph-mon[88269]: Upgrade: Updating mon.smithi138 2024-05-27T09:34:49.811 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:49 smithi119 ceph-mon[88269]: from='mgr.34104 ' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:49.811 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:49 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T09:34:49.811 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:49 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T09:34:49.811 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:49 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:34:49.811 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:49 smithi119 ceph-mon[88269]: Deploying daemon mon.smithi138 on smithi138 2024-05-27T09:34:50.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:50 smithi138 ceph-mon[34885]: pgmap v14: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:50.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:50 smithi119 ceph-mon[88269]: pgmap v14: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:51.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:51 smithi138 systemd[1]: Stopping Ceph mon.smithi138 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:34:52.081 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:51 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi138[34878]: 2024-05-27T09:34:51.681+0000 7fc5b1a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi138 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T09:34:52.081 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:51 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi138[34878]: 2024-05-27T09:34:51.681+0000 7fc5b1a00700 -1 mon.smithi138@1(peon) e2 *** Got Signal Terminated *** 2024-05-27T09:34:52.081 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:51 smithi138 podman[69843]: 2024-05-27 09:34:51.764912174 +0000 UTC m=+0.245134133 container died 60216fd6707805237264b6996f23ce90dca1fe616ecf272aa322f90c92d16471 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi138, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20231212, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, CEPH_POINT_RELEASE=-18.2.0) 2024-05-27T09:34:52.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:52 smithi138 podman[69843]: 2024-05-27 09:34:52.207458753 +0000 UTC m=+0.687680682 container cleanup 60216fd6707805237264b6996f23ce90dca1fe616ecf272aa322f90c92d16471 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi138, ceph=True, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-27T09:34:52.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:52 smithi138 bash[69843]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi138 2024-05-27T09:34:52.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:52 smithi138 podman[69854]: 2024-05-27 09:34:52.550484337 +0000 UTC m=+0.766721532 container remove 60216fd6707805237264b6996f23ce90dca1fe616ecf272aa322f90c92d16471 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi138, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0, GIT_CLEAN=True, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T09:34:53.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:53 smithi138 systemd[1]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@mon.smithi138.service: Deactivated successfully. 2024-05-27T09:34:53.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:53 smithi138 systemd[1]: Stopped Ceph mon.smithi138 for b966b06e-1c09-11ef-bc9b-c7b262605968. 2024-05-27T09:34:53.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:53 smithi138 systemd[1]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@mon.smithi138.service: Consumed 19.929s CPU time. 2024-05-27T09:34:53.790 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:53 smithi138 systemd[1]: Starting Ceph mon.smithi138 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:34:54.106 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:53 smithi138 podman[69986]: 2024-05-27 09:34:53.69934768 +0000 UTC m=+0.040905103 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:34:54.107 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:53 smithi138 podman[69986]: 2024-05-27 09:34:53.862127991 +0000 UTC m=+0.203685427 container create e4d471274cf172a54ea627d0e92c17ec3d31a6434d446bcc9f835f7ac4d853f3 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi138, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-27T09:34:54.107 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 podman[69986]: 2024-05-27 09:34:54.106033937 +0000 UTC m=+0.447591416 container init e4d471274cf172a54ea627d0e92c17ec3d31a6434d446bcc9f835f7ac4d853f3 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi138, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True) 2024-05-27T09:34:54.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 podman[69986]: 2024-05-27 09:34:54.1158443 +0000 UTC m=+0.457401723 container start e4d471274cf172a54ea627d0e92c17ec3d31a6434d446bcc9f835f7ac4d853f3 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi138, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-27T09:34:54.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: set uid:gid to 167:167 (ceph:ceph) 2024-05-27T09:34:54.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev), process ceph-mon, pid 2 2024-05-27T09:34:54.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: pidfile_write: ignore empty --pid-file 2024-05-27T09:34:54.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 bash[69986]: e4d471274cf172a54ea627d0e92c17ec3d31a6434d446bcc9f835f7ac4d853f3 2024-05-27T09:34:54.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: load: jerasure load: lrc 2024-05-27T09:34:54.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: RocksDB version: 7.9.2 2024-05-27T09:34:54.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Git sha 0 2024-05-27T09:34:54.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Compile date 2024-05-21 11:39:56 2024-05-27T09:34:54.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: DB SUMMARY 2024-05-27T09:34:54.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: DB Session ID: T5LMHE54OMMRNUZQIN09 2024-05-27T09:34:54.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: CURRENT file: CURRENT 2024-05-27T09:34:54.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: IDENTITY file: IDENTITY 2024-05-27T09:34:54.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1640 Bytes 2024-05-27T09:34:54.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi138/store.db dir, Total Num: 1, files: 000030.sst 2024-05-27T09:34:54.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi138/store.db: 000028.log size: 3620092 ; 2024-05-27T09:34:54.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.error_if_exists: 0 2024-05-27T09:34:54.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.create_if_missing: 0 2024-05-27T09:34:54.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.paranoid_checks: 1 2024-05-27T09:34:54.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-27T09:34:54.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-27T09:34:54.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-27T09:34:54.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.env: 0x55ec71737500 2024-05-27T09:34:54.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.fs: PosixFileSystem 2024-05-27T09:34:54.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.info_log: 0x55ec8e05fee0 2024-05-27T09:34:54.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_file_opening_threads: 16 2024-05-27T09:34:54.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.statistics: (nil) 2024-05-27T09:34:54.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.use_fsync: 0 2024-05-27T09:34:54.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_log_file_size: 0 2024-05-27T09:34:54.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-27T09:34:54.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-27T09:34:54.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.keep_log_file_num: 1000 2024-05-27T09:34:54.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.recycle_log_file_num: 0 2024-05-27T09:34:54.390 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.allow_fallocate: 1 2024-05-27T09:34:54.390 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.allow_mmap_reads: 0 2024-05-27T09:34:54.390 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.allow_mmap_writes: 0 2024-05-27T09:34:54.390 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.use_direct_reads: 0 2024-05-27T09:34:54.390 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-27T09:34:54.390 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.create_missing_column_families: 0 2024-05-27T09:34:54.390 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.db_log_dir: 2024-05-27T09:34:54.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.wal_dir: 2024-05-27T09:34:54.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-27T09:34:54.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-27T09:34:54.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-27T09:34:54.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-27T09:34:54.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-27T09:34:54.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-27T09:34:54.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.advise_random_on_open: 1 2024-05-27T09:34:54.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.db_write_buffer_size: 0 2024-05-27T09:34:54.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.write_buffer_manager: 0x55ec8e0308c0 2024-05-27T09:34:54.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-27T09:34:54.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-27T09:34:54.392 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-27T09:34:54.392 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.rate_limiter: (nil) 2024-05-27T09:34:54.392 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-27T09:34:54.392 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.wal_recovery_mode: 2 2024-05-27T09:34:54.392 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.enable_thread_tracking: 0 2024-05-27T09:34:54.392 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.enable_pipelined_write: 0 2024-05-27T09:34:54.392 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.unordered_write: 0 2024-05-27T09:34:54.392 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-27T09:34:54.392 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-27T09:34:54.392 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-27T09:34:54.392 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-27T09:34:54.392 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.row_cache: None 2024-05-27T09:34:54.393 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.wal_filter: None 2024-05-27T09:34:54.393 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-27T09:34:54.393 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.allow_ingest_behind: 0 2024-05-27T09:34:54.393 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.two_write_queues: 0 2024-05-27T09:34:54.393 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.manual_wal_flush: 0 2024-05-27T09:34:54.393 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.wal_compression: 0 2024-05-27T09:34:54.393 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.atomic_flush: 0 2024-05-27T09:34:54.393 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-27T09:34:54.393 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-27T09:34:54.393 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-27T09:34:54.393 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.log_readahead_size: 0 2024-05-27T09:34:54.393 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-27T09:34:54.393 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.best_efforts_recovery: 0 2024-05-27T09:34:54.394 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-27T09:34:54.394 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-27T09:34:54.394 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.allow_data_in_errors: 0 2024-05-27T09:34:54.394 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.db_host_id: __hostname__ 2024-05-27T09:34:54.394 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-27T09:34:54.394 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_background_jobs: 2 2024-05-27T09:34:54.394 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_background_compactions: -1 2024-05-27T09:34:54.394 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_subcompactions: 1 2024-05-27T09:34:54.394 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-27T09:34:54.394 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-27T09:34:54.394 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-27T09:34:54.394 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_total_wal_size: 0 2024-05-27T09:34:54.394 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-27T09:34:54.395 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-27T09:34:54.395 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-27T09:34:54.395 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-27T09:34:54.395 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_open_files: -1 2024-05-27T09:34:54.395 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.bytes_per_sync: 0 2024-05-27T09:34:54.395 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-27T09:34:54.395 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-27T09:34:54.395 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compaction_readahead_size: 0 2024-05-27T09:34:54.395 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_background_flushes: -1 2024-05-27T09:34:54.395 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Compression algorithms supported: 2024-05-27T09:34:54.395 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-27T09:34:54.395 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: kZSTD supported: 0 2024-05-27T09:34:54.395 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: kXpressCompression supported: 0 2024-05-27T09:34:54.395 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: kLZ4HCCompression supported: 1 2024-05-27T09:34:54.396 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: kZlibCompression supported: 1 2024-05-27T09:34:54.396 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: kSnappyCompression supported: 1 2024-05-27T09:34:54.396 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: kLZ4Compression supported: 1 2024-05-27T09:34:54.396 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: kBZip2Compression supported: 0 2024-05-27T09:34:54.396 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-27T09:34:54.396 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-27T09:34:54.396 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi138/store.db/MANIFEST-000010 2024-05-27T09:34:54.396 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-27T09:34:54.396 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-27T09:34:54.396 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.merge_operator: 2024-05-27T09:34:54.396 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compaction_filter: None 2024-05-27T09:34:54.396 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compaction_filter_factory: None 2024-05-27T09:34:54.396 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.sst_partitioner_factory: None 2024-05-27T09:34:54.397 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-27T09:34:54.397 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-27T09:34:54.397 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ec8e1660c0) 2024-05-27T09:34:54.397 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: cache_index_and_filter_blocks: 1 2024-05-27T09:34:54.397 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-27T09:34:54.397 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-27T09:34:54.397 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: pin_top_level_index_and_filter: 1 2024-05-27T09:34:54.397 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: index_type: 0 2024-05-27T09:34:54.397 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: data_block_index_type: 0 2024-05-27T09:34:54.397 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: index_shortening: 1 2024-05-27T09:34:54.397 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-27T09:34:54.397 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: checksum: 4 2024-05-27T09:34:54.397 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: no_block_cache: 0 2024-05-27T09:34:54.397 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_cache: 0x55ec8e0631f0 2024-05-27T09:34:54.397 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_cache_name: BinnedLRUCache 2024-05-27T09:34:54.398 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_cache_options: 2024-05-27T09:34:54.398 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: capacity : 536870912 2024-05-27T09:34:54.398 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: num_shard_bits : 4 2024-05-27T09:34:54.398 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: strict_capacity_limit : 0 2024-05-27T09:34:54.398 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: high_pri_pool_ratio: 0.000 2024-05-27T09:34:54.398 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_cache_compressed: (nil) 2024-05-27T09:34:54.398 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: persistent_cache: (nil) 2024-05-27T09:34:54.398 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_size: 4096 2024-05-27T09:34:54.398 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_size_deviation: 10 2024-05-27T09:34:54.398 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_restart_interval: 16 2024-05-27T09:34:54.398 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: index_block_restart_interval: 1 2024-05-27T09:34:54.398 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: metadata_block_size: 4096 2024-05-27T09:34:54.398 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: partition_filters: 0 2024-05-27T09:34:54.399 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: use_delta_encoding: 1 2024-05-27T09:34:54.399 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: filter_policy: bloomfilter 2024-05-27T09:34:54.399 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: whole_key_filtering: 1 2024-05-27T09:34:54.399 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: verify_compression: 0 2024-05-27T09:34:54.399 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: read_amp_bytes_per_bit: 0 2024-05-27T09:34:54.399 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: format_version: 5 2024-05-27T09:34:54.399 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: enable_index_compression: 1 2024-05-27T09:34:54.399 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_align: 0 2024-05-27T09:34:54.399 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: max_auto_readahead_size: 262144 2024-05-27T09:34:54.399 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: prepopulate_block_cache: 0 2024-05-27T09:34:54.399 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: initial_auto_readahead_size: 8192 2024-05-27T09:34:54.399 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: num_file_reads_for_auto_readahead: 2 2024-05-27T09:34:54.399 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.write_buffer_size: 33554432 2024-05-27T09:34:54.399 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_write_buffer_number: 2 2024-05-27T09:34:54.400 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compression: NoCompression 2024-05-27T09:34:54.400 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.bottommost_compression: Disabled 2024-05-27T09:34:54.400 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.prefix_extractor: nullptr 2024-05-27T09:34:54.400 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-27T09:34:54.400 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.num_levels: 7 2024-05-27T09:34:54.401 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-27T09:34:54.401 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-27T09:34:54.401 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-27T09:34:54.401 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-27T09:34:54.401 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-27T09:34:54.401 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-27T09:34:54.401 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-27T09:34:54.401 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-27T09:34:54.401 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-27T09:34:54.402 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-27T09:34:54.402 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-27T09:34:54.402 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-27T09:34:54.402 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-27T09:34:54.402 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compression_opts.level: 32767 2024-05-27T09:34:54.402 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compression_opts.strategy: 0 2024-05-27T09:34:54.402 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-27T09:34:54.402 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-27T09:34:54.402 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-27T09:34:54.402 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-27T09:34:54.402 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compression_opts.enabled: false 2024-05-27T09:34:54.402 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-27T09:34:54.402 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-27T09:34:54.403 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-27T09:34:54.403 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-27T09:34:54.403 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.target_file_size_base: 67108864 2024-05-27T09:34:54.403 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-27T09:34:54.403 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-27T09:34:54.403 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-27T09:34:54.403 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-27T09:34:54.403 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-27T09:34:54.403 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-27T09:34:54.403 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-27T09:34:54.403 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-27T09:34:54.403 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-27T09:34:54.403 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-27T09:34:54.404 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-27T09:34:54.404 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-27T09:34:54.404 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-27T09:34:54.404 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-27T09:34:54.404 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.arena_block_size: 1048576 2024-05-27T09:34:54.404 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-27T09:34:54.404 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-27T09:34:54.404 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.disable_auto_compactions: 0 2024-05-27T09:34:54.404 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-27T09:34:54.404 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-27T09:34:54.404 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-27T09:34:54.404 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-27T09:34:54.404 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-27T09:34:54.404 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-27T09:34:54.405 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-27T09:34:54.405 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-27T09:34:54.405 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-27T09:34:54.405 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-27T09:34:54.405 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-27T09:34:54.405 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.inplace_update_support: 0 2024-05-27T09:34:54.405 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-27T09:34:54.405 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-27T09:34:54.405 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-27T09:34:54.405 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-27T09:34:54.405 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.bloom_locality: 0 2024-05-27T09:34:54.405 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.max_successive_merges: 0 2024-05-27T09:34:54.405 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-27T09:34:54.405 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.paranoid_file_checks: 0 2024-05-27T09:34:54.405 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.force_consistency_checks: 1 2024-05-27T09:34:54.406 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.report_bg_io_stats: 0 2024-05-27T09:34:54.406 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.ttl: 2592000 2024-05-27T09:34:54.406 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-27T09:34:54.406 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-27T09:34:54.406 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-27T09:34:54.406 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.enable_blob_files: false 2024-05-27T09:34:54.406 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.min_blob_size: 0 2024-05-27T09:34:54.406 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.blob_file_size: 268435456 2024-05-27T09:34:54.406 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-27T09:34:54.406 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-27T09:34:54.406 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-27T09:34:54.406 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-27T09:34:54.406 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-27T09:34:54.407 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.blob_file_starting_level: 0 2024-05-27T09:34:54.407 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-27T09:34:54.407 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-27T09:34:54.407 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi138/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 32, last_sequence is 16884, log_number is 28,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 28 2024-05-27T09:34:54.407 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 28 2024-05-27T09:34:54.407 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 565d2627-d6ba-44a0-823b-2c4b78d6dd5d 2024-05-27T09:34:54.407 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716802494245481, "job": 1, "event": "recovery_started", "wal_files": [28]} 2024-05-27T09:34:54.407 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #28 mode 2 2024-05-27T09:34:54.407 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 systemd[1]: Started Ceph mon.smithi138 for b966b06e-1c09-11ef-bc9b-c7b262605968. 2024-05-27T09:34:54.407 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716802494261806, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 33, "file_size": 2304024, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 16889, "largest_seqno": 18382, "table_properties": {"data_size": 2297784, "index_size": 3595, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1605, "raw_key_size": 15167, "raw_average_key_size": 24, "raw_value_size": 2284664, "raw_average_value_size": 3667, "num_data_blocks": 162, "num_entries": 623, "num_filter_entries": 623, "num_deletions": 6, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716802494, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "565d2627-d6ba-44a0-823b-2c4b78d6dd5d", "db_session_id": "T5LMHE54OMMRNUZQIN09", "orig_file_number": 33, "seqno_to_time_mapping": "N/A"}} 2024-05-27T09:34:54.407 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716802494262122, "job": 1, "event": "recovery_finished"} 2024-05-27T09:34:54.407 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: [db/version_set.cc:5047] Creating manifest 35 2024-05-27T09:34:54.407 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-27T09:34:54.407 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi138/store.db/000028.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-27T09:34:54.408 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ec8e047c00 2024-05-27T09:34:54.408 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: DB pointer 0x55ec92e3a000 2024-05-27T09:34:54.408 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-27T09:34:54.408 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-27T09:34:54.408 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: ** DB Stats ** 2024-05-27T09:34:54.408 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T09:34:54.408 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-27T09:34:54.408 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T09:34:54.408 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T09:34:54.408 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-27T09:34:54.408 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T09:34:54.408 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T09:34:54.408 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-27T09:34:54.409 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: ** Compaction Stats [default] ** 2024-05-27T09:34:54.409 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T09:34:54.409 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-27T09:34:54.409 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: L0 1/0 2.20 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 457.2 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-05-27T09:34:54.409 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: L6 1/0 8.27 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-05-27T09:34:54.409 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Sum 2/0 10.47 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 457.2 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-05-27T09:34:54.409 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 457.2 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-05-27T09:34:54.409 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-27T09:34:54.409 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: ** Compaction Stats [default] ** 2024-05-27T09:34:54.409 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T09:34:54.409 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-27T09:34:54.409 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 457.2 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-05-27T09:34:54.409 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-27T09:34:54.409 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-27T09:34:54.410 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-27T09:34:54.410 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T09:34:54.410 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-05-27T09:34:54.410 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-27T09:34:54.410 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-27T09:34:54.410 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-27T09:34:54.410 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-27T09:34:54.410 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Cumulative compaction: 0.00 GB write, 99.87 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T09:34:54.410 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Interval compaction: 0.00 GB write, 99.87 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T09:34:54.410 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-27T09:34:54.410 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Block cache BinnedLRUCache@0x55ec8e0631f0#2 capacity: 512.00 MB usage: 5.87 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-05-27T09:34:54.410 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.79 KB,0.000341237%) IndexBlock(1,4.08 KB,0.000777841%) Misc(1,0.00 KB,0%) 2024-05-27T09:34:54.410 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-27T09:34:54.410 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-27T09:34:54.411 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: starting mon.smithi138 rank 1 at public addrs [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] at bind addrs [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi138 fsid b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:34:54.411 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: mon.smithi138@-1(???) e2 preinit fsid b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:34:54.411 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: mon.smithi138@-1(???).mds e22 new map 2024-05-27T09:34:54.411 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: mon.smithi138@-1(???).mds e22 print_map 2024-05-27T09:34:54.411 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: e22 2024-05-27T09:34:54.411 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T09:34:54.411 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:34:54.411 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: legacy client fscid: 1 2024-05-27T09:34:54.411 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-27T09:34:54.411 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Filesystem 'cephfs' (1) 2024-05-27T09:34:54.411 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: fs_name cephfs 2024-05-27T09:34:54.411 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: epoch 22 2024-05-27T09:34:54.411 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-27T09:34:54.412 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: created 2024-05-27T09:25:53.387388+0000 2024-05-27T09:34:54.412 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: modified 2024-05-27T09:34:01.250304+0000 2024-05-27T09:34:54.412 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: tableserver 0 2024-05-27T09:34:54.412 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: root 0 2024-05-27T09:34:54.412 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: session_timeout 60 2024-05-27T09:34:54.412 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: session_autoclose 300 2024-05-27T09:34:54.412 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: max_file_size 1099511627776 2024-05-27T09:34:54.412 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: max_xattr_size 65536 2024-05-27T09:34:54.412 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: required_client_features {} 2024-05-27T09:34:54.412 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: last_failure 0 2024-05-27T09:34:54.412 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: last_failure_osd_epoch 46 2024-05-27T09:34:54.412 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:34:54.412 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: max_mds 2 2024-05-27T09:34:54.412 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: in 0,1 2024-05-27T09:34:54.412 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: up {0=14472,1=24241} 2024-05-27T09:34:54.413 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: failed 2024-05-27T09:34:54.413 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: damaged 2024-05-27T09:34:54.413 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: stopped 2024-05-27T09:34:54.413 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: data_pools [3] 2024-05-27T09:34:54.413 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: metadata_pool 2 2024-05-27T09:34:54.413 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: inline_data enabled 2024-05-27T09:34:54.413 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: balancer 2024-05-27T09:34:54.413 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: bal_rank_mask -1 2024-05-27T09:34:54.413 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: standby_count_wanted 1 2024-05-27T09:34:54.413 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: qdb_cluster leader: 0 members: 2024-05-27T09:34:54.413 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: [mds.cephfs.smithi119.ppmzzk{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:34:54.413 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: [mds.cephfs.smithi138.ndfkhd{1:24241} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/897058594,v1:172.21.15.138:6825/897058594] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:34:54.413 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-27T09:34:54.413 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-27T09:34:54.414 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Standby daemons: 2024-05-27T09:34:54.414 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-05-27T09:34:54.414 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: [mds.cephfs.smithi119.nghcey{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.119:6826/413736191,v1:172.21.15.119:6827/413736191] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:34:54.414 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: [mds.cephfs.smithi138.alszpk{-1:24253} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/2345607053,v1:172.21.15.138:6827/2345607053] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:34:54.414 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: mon.smithi138@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-05-27T09:34:54.414 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: mon.smithi138@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-27T09:34:54.414 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: mon.smithi138@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-27T09:34:54.414 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: mon.smithi138@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-27T09:34:54.414 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: mon.smithi138@-1(???).paxosservice(auth 1..25) refresh upgraded, format 0 -> 3 2024-05-27T09:34:54.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:54 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi119"}]: dispatch 2024-05-27T09:34:54.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:54 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:34:55.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi119"}]: dispatch 2024-05-27T09:34:55.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:54 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:34:56.109 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-27T09:34:56.109 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:34:56.109 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:34:56.109 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: mon.smithi138 calling monitor election 2024-05-27T09:34:56.109 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: mon.smithi119 calling monitor election 2024-05-27T09:34:56.110 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi119"}]: dispatch 2024-05-27T09:34:56.110 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:34:56.110 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: mon.smithi119 is new leader, mons smithi119,smithi138 in quorum (ranks 0,1) 2024-05-27T09:34:56.110 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: monmap epoch 3 2024-05-27T09:34:56.110 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: fsid b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:34:56.110 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: last_changed 2024-05-27T09:34:54.754651+0000 2024-05-27T09:34:56.110 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: created 2024-05-27T09:16:53.014813+0000 2024-05-27T09:34:56.110 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: min_mon_release 19 (squid) 2024-05-27T09:34:56.110 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: election_strategy: 1 2024-05-27T09:34:56.110 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: 0: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.smithi119 2024-05-27T09:34:56.110 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.smithi138 2024-05-27T09:34:56.110 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: fsmap cephfs:2 {0=cephfs.smithi119.ppmzzk=up:active,1=cephfs.smithi138.ndfkhd=up:active} 2 up:standby 2024-05-27T09:34:56.110 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: osdmap e49: 6 total, 6 up, 6 in 2024-05-27T09:34:56.111 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: mgrmap e38: smithi119.ncwmog(active, since 27s), standbys: smithi138.dxsuqf 2024-05-27T09:34:56.111 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-27T09:34:56.111 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:34:56.111 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:34:56.111 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:56.111 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:56.111 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:55 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:34:56.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-27T09:34:56.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:34:56.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:34:56.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: mon.smithi138 calling monitor election 2024-05-27T09:34:56.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: mon.smithi119 calling monitor election 2024-05-27T09:34:56.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi119"}]: dispatch 2024-05-27T09:34:56.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-05-27T09:34:56.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: mon.smithi119 is new leader, mons smithi119,smithi138 in quorum (ranks 0,1) 2024-05-27T09:34:56.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: monmap epoch 3 2024-05-27T09:34:56.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: fsid b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:34:56.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: last_changed 2024-05-27T09:34:54.754651+0000 2024-05-27T09:34:56.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: created 2024-05-27T09:16:53.014813+0000 2024-05-27T09:34:56.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: min_mon_release 19 (squid) 2024-05-27T09:34:56.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: election_strategy: 1 2024-05-27T09:34:56.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: 0: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.smithi119 2024-05-27T09:34:56.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.smithi138 2024-05-27T09:34:56.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: fsmap cephfs:2 {0=cephfs.smithi119.ppmzzk=up:active,1=cephfs.smithi138.ndfkhd=up:active} 2 up:standby 2024-05-27T09:34:56.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: osdmap e49: 6 total, 6 up, 6 in 2024-05-27T09:34:56.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: mgrmap e38: smithi119.ncwmog(active, since 27s), standbys: smithi138.dxsuqf 2024-05-27T09:34:56.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-27T09:34:56.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:34:56.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:34:56.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:56.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:34:56.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:55 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:34:56.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:56 smithi138 ceph-mon[70004]: pgmap v17: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:57.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:56 smithi119 ceph-mon[88269]: pgmap v17: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:58.079 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:34:58.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:34:59.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:34:58 smithi138 ceph-mon[70004]: pgmap v18: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:34:59.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:34:58 smithi119 ceph-mon[88269]: pgmap v18: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:00.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:00 smithi119 ceph-mon[88269]: pgmap v19: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:00.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:00 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:00.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:00 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:00.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:00 smithi138 ceph-mon[70004]: pgmap v19: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:00.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:00 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:00.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:00 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:02.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:02 smithi119 ceph-mon[88269]: pgmap v20: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:02.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:02 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:02.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:02 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:02.814 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:02 smithi138 ceph-mon[70004]: pgmap v20: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:02.814 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:02 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:02.814 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:02 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:04.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:04 smithi119 ceph-mon[88269]: pgmap v21: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:04.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:04 smithi138 ceph-mon[70004]: pgmap v21: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:06.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:06 smithi119 ceph-mon[88269]: pgmap v22: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:06.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:06 smithi138 ceph-mon[70004]: pgmap v22: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:08.367 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:08 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:08.368 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:08 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:08.368 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:08 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:08.368 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:08 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:35:08.368 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:08 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:08.368 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:08 smithi119 ceph-mon[88269]: pgmap v23: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:08.368 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:08 smithi119 ceph-mon[88269]: Reconfiguring mon.smithi119 (monmap changed)... 2024-05-27T09:35:08.368 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:08 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T09:35:08.368 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:08 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T09:35:08.368 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:08 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:08.368 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:08 smithi119 ceph-mon[88269]: Reconfiguring daemon mon.smithi119 on smithi119 2024-05-27T09:35:08.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:08 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:08.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:08 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:08.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:08 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:08.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:08 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:35:08.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:08 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:08.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:08 smithi138 ceph-mon[70004]: pgmap v23: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:08.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:08 smithi138 ceph-mon[70004]: Reconfiguring mon.smithi119 (monmap changed)... 2024-05-27T09:35:08.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:08 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T09:35:08.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:08 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T09:35:08.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:08 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:08.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:08 smithi138 ceph-mon[70004]: Reconfiguring daemon mon.smithi119 on smithi119 2024-05-27T09:35:10.939 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:10 smithi119 ceph-mon[88269]: pgmap v24: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:10.939 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:10 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:10.939 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:10 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:10.939 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:10 smithi119 ceph-mon[88269]: Reconfiguring mgr.smithi119.ncwmog (monmap changed)... 2024-05-27T09:35:10.939 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:10 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi119.ncwmog", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T09:35:10.939 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:10 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T09:35:10.939 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:10 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:10.939 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:10 smithi119 ceph-mon[88269]: Reconfiguring daemon mgr.smithi119.ncwmog on smithi119 2024-05-27T09:35:11.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:10 smithi138 ceph-mon[70004]: pgmap v24: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:11.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:10 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:11.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:10 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:11.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:10 smithi138 ceph-mon[70004]: Reconfiguring mgr.smithi119.ncwmog (monmap changed)... 2024-05-27T09:35:11.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:10 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi119.ncwmog", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T09:35:11.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:10 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T09:35:11.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:10 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:11.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:10 smithi138 ceph-mon[70004]: Reconfiguring daemon mgr.smithi119.ncwmog on smithi119 2024-05-27T09:35:12.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:12 smithi119 ceph-mon[88269]: pgmap v25: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:12.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:12.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:12.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi119", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-27T09:35:12.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi119", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-27T09:35:12.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi119", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-27T09:35:12.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi119"}]: dispatch 2024-05-27T09:35:12.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:12.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:35:13.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:12 smithi138 ceph-mon[70004]: pgmap v25: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:13.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:13.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:13.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi119", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-27T09:35:13.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi119", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-27T09:35:13.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi119", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-27T09:35:13.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi119"}]: dispatch 2024-05-27T09:35:13.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:13.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:35:14.128 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:13 smithi119 ceph-mon[88269]: Reconfiguring ceph-exporter.smithi119 (monmap changed)... 2024-05-27T09:35:14.128 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:13 smithi119 ceph-mon[88269]: Unable to update caps for client.ceph-exporter.smithi119 2024-05-27T09:35:14.128 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:13 smithi119 ceph-mon[88269]: Reconfiguring daemon ceph-exporter.smithi119 on smithi119 2024-05-27T09:35:14.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:13 smithi138 ceph-mon[70004]: Reconfiguring ceph-exporter.smithi119 (monmap changed)... 2024-05-27T09:35:14.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:13 smithi138 ceph-mon[70004]: Unable to update caps for client.ceph-exporter.smithi119 2024-05-27T09:35:14.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:13 smithi138 ceph-mon[70004]: Reconfiguring daemon ceph-exporter.smithi119 on smithi119 2024-05-27T09:35:14.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.740+0000 7ff6a8600700 1 -- 172.21.15.119:0/1503361656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a0100d50 msgr2=0x7ff6a0104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:14.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.740+0000 7ff6a8600700 1 --2- 172.21.15.119:0/1503361656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a0100d50 0x7ff6a0104d20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009910 tx=0x7ff68c009c20 comp rx=0 tx=0).stop 2024-05-27T09:35:14.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.741+0000 7ff6a8600700 1 -- 172.21.15.119:0/1503361656 shutdown_connections 2024-05-27T09:35:14.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.741+0000 7ff6a8600700 1 --2- 172.21.15.119:0/1503361656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a0100d50 0x7ff6a0104d20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:14.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.741+0000 7ff6a8600700 1 --2- 172.21.15.119:0/1503361656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6a01003b0 0x7ff6a0100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:14.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.741+0000 7ff6a8600700 1 -- 172.21.15.119:0/1503361656 >> 172.21.15.119:0/1503361656 conn(0x7ff6a00fbc30 msgr2=0x7ff6a00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:14.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.741+0000 7ff6a8600700 1 -- 172.21.15.119:0/1503361656 shutdown_connections 2024-05-27T09:35:14.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.741+0000 7ff6a8600700 1 -- 172.21.15.119:0/1503361656 wait complete. 2024-05-27T09:35:14.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.743+0000 7ff6a8600700 1 Processor -- start 2024-05-27T09:35:14.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.743+0000 7ff6a8600700 1 -- start start 2024-05-27T09:35:14.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.743+0000 7ff6a8600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6a01003b0 0x7ff6a0195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:14.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.743+0000 7ff6a8600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a0100d50 0x7ff6a0196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:14.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.743+0000 7ff6a8600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a01969e0 con 0x7ff6a0100d50 2024-05-27T09:35:14.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.743+0000 7ff6a8600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a019a700 con 0x7ff6a01003b0 2024-05-27T09:35:14.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.744+0000 7ff6a5200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a0100d50 0x7ff6a0196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:14.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.744+0000 7ff6a5200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a0100d50 0x7ff6a0196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38174/0 (socket says 172.21.15.119:38174) 2024-05-27T09:35:14.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.744+0000 7ff6a5200700 1 -- 172.21.15.119:0/265125924 learned_addr learned my addr 172.21.15.119:0/265125924 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:35:14.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.744+0000 7ff6a5c00700 1 --2- 172.21.15.119:0/265125924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6a01003b0 0x7ff6a0195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:14.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.744+0000 7ff6a5200700 1 -- 172.21.15.119:0/265125924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6a01003b0 msgr2=0x7ff6a0195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:14.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.744+0000 7ff6a5200700 1 --2- 172.21.15.119:0/265125924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6a01003b0 0x7ff6a0195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:14.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.744+0000 7ff6a5200700 1 -- 172.21.15.119:0/265125924 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68c0095f0 con 0x7ff6a0100d50 2024-05-27T09:35:14.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.745+0000 7ff6a5200700 1 --2- 172.21.15.119:0/265125924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a0100d50 0x7ff6a0196300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff68c005f50 tx=0x7ff68c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:14.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.745+0000 7ff69e000700 1 -- 172.21.15.119:0/265125924 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff68c01d070 con 0x7ff6a0100d50 2024-05-27T09:35:14.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.745+0000 7ff69e000700 1 -- 172.21.15.119:0/265125924 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff68c00f460 con 0x7ff6a0100d50 2024-05-27T09:35:14.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.745+0000 7ff69e000700 1 -- 172.21.15.119:0/265125924 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff68c017630 con 0x7ff6a0100d50 2024-05-27T09:35:14.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.745+0000 7ff6a8600700 1 -- 172.21.15.119:0/265125924 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a019a980 con 0x7ff6a0100d50 2024-05-27T09:35:14.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.745+0000 7ff6a8600700 1 -- 172.21.15.119:0/265125924 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a019ae20 con 0x7ff6a0100d50 2024-05-27T09:35:14.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.746+0000 7ff6a8600700 1 -- 172.21.15.119:0/265125924 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a0108720 con 0x7ff6a0100d50 2024-05-27T09:35:14.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.748+0000 7ff69e000700 1 -- 172.21.15.119:0/265125924 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 38) v1 ==== 100184+0+0 (secure 0 0 0) 0x7ff68c00f5d0 con 0x7ff6a0100d50 2024-05-27T09:35:14.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.750+0000 7ff69e000700 1 --2- 172.21.15.119:0/265125924 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff688078b60 0x7ff68807b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:14.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.750+0000 7ff69e000700 1 -- 172.21.15.119:0/265125924 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff68c09c5e0 con 0x7ff6a0100d50 2024-05-27T09:35:14.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.750+0000 7ff6a5c00700 1 --2- 172.21.15.119:0/265125924 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff688078b60 0x7ff68807b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:14.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.751+0000 7ff6a5c00700 1 --2- 172.21.15.119:0/265125924 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff688078b60 0x7ff68807b010 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff694005fd0 tx=0x7ff69400a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:14.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:14.756+0000 7ff69e000700 1 -- 172.21.15.119:0/265125924 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7ff68c063fa0 con 0x7ff6a0100d50 2024-05-27T09:35:14.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:14 smithi119 ceph-mon[88269]: pgmap v26: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:14.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:14.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:14.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi119", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-27T09:35:14.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:15.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.011+0000 7ff6a8600700 1 -- 172.21.15.119:0/265125924 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6a002cd10 con 0x7ff688078b60 2024-05-27T09:35:15.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.013+0000 7ff69e000700 1 -- 172.21.15.119:0/265125924 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7ff6a002cd10 con 0x7ff688078b60 2024-05-27T09:35:15.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.015+0000 7ff6a8600700 1 -- 172.21.15.119:0/265125924 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff688078b60 msgr2=0x7ff68807b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:15.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.016+0000 7ff6a8600700 1 --2- 172.21.15.119:0/265125924 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff688078b60 0x7ff68807b010 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff694005fd0 tx=0x7ff69400a560 comp rx=0 tx=0).stop 2024-05-27T09:35:15.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.016+0000 7ff6a8600700 1 -- 172.21.15.119:0/265125924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a0100d50 msgr2=0x7ff6a0196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:15.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.016+0000 7ff6a8600700 1 --2- 172.21.15.119:0/265125924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a0100d50 0x7ff6a0196300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff68c005f50 tx=0x7ff68c00be20 comp rx=0 tx=0).stop 2024-05-27T09:35:15.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.016+0000 7ff6a8600700 1 -- 172.21.15.119:0/265125924 shutdown_connections 2024-05-27T09:35:15.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.016+0000 7ff6a8600700 1 --2- 172.21.15.119:0/265125924 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff688078b60 0x7ff68807b010 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:15.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.016+0000 7ff6a8600700 1 --2- 172.21.15.119:0/265125924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6a01003b0 0x7ff6a0195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:15.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.016+0000 7ff6a8600700 1 --2- 172.21.15.119:0/265125924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a0100d50 0x7ff6a0196300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:15.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.016+0000 7ff6a8600700 1 -- 172.21.15.119:0/265125924 >> 172.21.15.119:0/265125924 conn(0x7ff6a00fbc30 msgr2=0x7ff6a00fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:15.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.016+0000 7ff6a8600700 1 -- 172.21.15.119:0/265125924 shutdown_connections 2024-05-27T09:35:15.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.016+0000 7ff6a8600700 1 -- 172.21.15.119:0/265125924 wait complete. 2024-05-27T09:35:15.033 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:35:15.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:14 smithi138 ceph-mon[70004]: pgmap v26: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:15.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:15.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:15.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi119", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-27T09:35:15.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:15.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.176+0000 7ff357200700 1 -- 172.21.15.119:0/3057591544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3500730f0 msgr2=0x7ff3500734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:15.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.176+0000 7ff357200700 1 --2- 172.21.15.119:0/3057591544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3500730f0 0x7ff3500734c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff340009910 tx=0x7ff340009c20 comp rx=0 tx=0).stop 2024-05-27T09:35:15.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.176+0000 7ff357200700 1 -- 172.21.15.119:0/3057591544 shutdown_connections 2024-05-27T09:35:15.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.176+0000 7ff357200700 1 --2- 172.21.15.119:0/3057591544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff350073a00 0x7ff350110c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:15.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.176+0000 7ff357200700 1 --2- 172.21.15.119:0/3057591544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3500730f0 0x7ff3500734c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:15.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.176+0000 7ff357200700 1 -- 172.21.15.119:0/3057591544 >> 172.21.15.119:0/3057591544 conn(0x7ff3500fbc30 msgr2=0x7ff3500fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:15.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.176+0000 7ff357200700 1 -- 172.21.15.119:0/3057591544 shutdown_connections 2024-05-27T09:35:15.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.176+0000 7ff357200700 1 -- 172.21.15.119:0/3057591544 wait complete. 2024-05-27T09:35:15.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.177+0000 7ff357200700 1 Processor -- start 2024-05-27T09:35:15.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.177+0000 7ff357200700 1 -- start start 2024-05-27T09:35:15.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.177+0000 7ff357200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3500730f0 0x7ff35019c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:15.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.177+0000 7ff357200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff350073a00 0x7ff35019c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:15.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.177+0000 7ff357200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35019cfa0 con 0x7ff3500730f0 2024-05-27T09:35:15.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.177+0000 7ff357200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3501a0d30 con 0x7ff350073a00 2024-05-27T09:35:15.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.178+0000 7ff34fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3500730f0 0x7ff35019c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:15.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.178+0000 7ff34fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3500730f0 0x7ff35019c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38200/0 (socket says 172.21.15.119:38200) 2024-05-27T09:35:15.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.178+0000 7ff34fe00700 1 -- 172.21.15.119:0/3408065626 learned_addr learned my addr 172.21.15.119:0/3408065626 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:35:15.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.178+0000 7ff34fe00700 1 -- 172.21.15.119:0/3408065626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff350073a00 msgr2=0x7ff35019c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:15.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.178+0000 7ff34f400700 1 --2- 172.21.15.119:0/3408065626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff350073a00 0x7ff35019c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:15.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.178+0000 7ff34fe00700 1 --2- 172.21.15.119:0/3408065626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff350073a00 0x7ff35019c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:15.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.178+0000 7ff34fe00700 1 -- 172.21.15.119:0/3408065626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3400095f0 con 0x7ff3500730f0 2024-05-27T09:35:15.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.179+0000 7ff34fe00700 1 --2- 172.21.15.119:0/3408065626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3500730f0 0x7ff35019c380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff340006010 tx=0x7ff34000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:15.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.179+0000 7ff34cc00700 1 -- 172.21.15.119:0/3408065626 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff34001d070 con 0x7ff3500730f0 2024-05-27T09:35:15.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.179+0000 7ff34cc00700 1 -- 172.21.15.119:0/3408065626 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff340004fb0 con 0x7ff3500730f0 2024-05-27T09:35:15.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.179+0000 7ff357200700 1 -- 172.21.15.119:0/3408065626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3501a1010 con 0x7ff3500730f0 2024-05-27T09:35:15.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.179+0000 7ff34cc00700 1 -- 172.21.15.119:0/3408065626 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff340022ea0 con 0x7ff3500730f0 2024-05-27T09:35:15.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.179+0000 7ff357200700 1 -- 172.21.15.119:0/3408065626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3501a1560 con 0x7ff3500730f0 2024-05-27T09:35:15.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.180+0000 7ff357200700 1 -- 172.21.15.119:0/3408065626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff35010e3a0 con 0x7ff3500730f0 2024-05-27T09:35:15.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.181+0000 7ff34cc00700 1 -- 172.21.15.119:0/3408065626 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 38) v1 ==== 100184+0+0 (secure 0 0 0) 0x7ff340022470 con 0x7ff3500730f0 2024-05-27T09:35:15.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.182+0000 7ff34cc00700 1 --2- 172.21.15.119:0/3408065626 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff334078b70 0x7ff33407b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:15.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.183+0000 7ff34cc00700 1 -- 172.21.15.119:0/3408065626 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff34009c510 con 0x7ff3500730f0 2024-05-27T09:35:15.183 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.183+0000 7ff34f400700 1 --2- 172.21.15.119:0/3408065626 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff334078b70 0x7ff33407b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:15.184 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.184+0000 7ff34f400700 1 --2- 172.21.15.119:0/3408065626 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff334078b70 0x7ff33407b020 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff33c006fd0 tx=0x7ff33c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:15.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.188+0000 7ff34cc00700 1 -- 172.21.15.119:0/3408065626 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7ff340063f80 con 0x7ff3500730f0 2024-05-27T09:35:15.447 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.446+0000 7ff357200700 1 -- 172.21.15.119:0/3408065626 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3501a1890 con 0x7ff334078b70 2024-05-27T09:35:15.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.449+0000 7ff34cc00700 1 -- 172.21.15.119:0/3408065626 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7ff3501a1890 con 0x7ff334078b70 2024-05-27T09:35:15.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.451+0000 7ff357200700 1 -- 172.21.15.119:0/3408065626 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff334078b70 msgr2=0x7ff33407b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:15.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.451+0000 7ff357200700 1 --2- 172.21.15.119:0/3408065626 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff334078b70 0x7ff33407b020 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff33c006fd0 tx=0x7ff33c008040 comp rx=0 tx=0).stop 2024-05-27T09:35:15.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.451+0000 7ff357200700 1 -- 172.21.15.119:0/3408065626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3500730f0 msgr2=0x7ff35019c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:15.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.451+0000 7ff357200700 1 --2- 172.21.15.119:0/3408065626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3500730f0 0x7ff35019c380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff340006010 tx=0x7ff34000be20 comp rx=0 tx=0).stop 2024-05-27T09:35:15.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.452+0000 7ff357200700 1 -- 172.21.15.119:0/3408065626 shutdown_connections 2024-05-27T09:35:15.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.452+0000 7ff357200700 1 --2- 172.21.15.119:0/3408065626 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff334078b70 0x7ff33407b020 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:15.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.452+0000 7ff357200700 1 --2- 172.21.15.119:0/3408065626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3500730f0 0x7ff35019c380 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:15.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.452+0000 7ff357200700 1 --2- 172.21.15.119:0/3408065626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff350073a00 0x7ff35019c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:15.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.452+0000 7ff357200700 1 -- 172.21.15.119:0/3408065626 >> 172.21.15.119:0/3408065626 conn(0x7ff3500fbc30 msgr2=0x7ff350102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:15.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.452+0000 7ff357200700 1 -- 172.21.15.119:0/3408065626 shutdown_connections 2024-05-27T09:35:15.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.452+0000 7ff357200700 1 -- 172.21.15.119:0/3408065626 wait complete. 2024-05-27T09:35:15.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.610+0000 7ff9fa600700 1 -- 172.21.15.119:0/490809019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9f4107780 msgr2=0x7ff9f410b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:15.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.610+0000 7ff9fa600700 1 --2- 172.21.15.119:0/490809019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9f4107780 0x7ff9f410b1f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009910 tx=0x7ff9e0009c20 comp rx=0 tx=0).stop 2024-05-27T09:35:15.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.611+0000 7ff9fa600700 1 -- 172.21.15.119:0/490809019 shutdown_connections 2024-05-27T09:35:15.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.611+0000 7ff9fa600700 1 --2- 172.21.15.119:0/490809019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9f4107780 0x7ff9f410b1f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:15.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.611+0000 7ff9fa600700 1 --2- 172.21.15.119:0/490809019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f4106de0 0x7ff9f41071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:15.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.611+0000 7ff9fa600700 1 -- 172.21.15.119:0/490809019 >> 172.21.15.119:0/490809019 conn(0x7ff9f4075de0 msgr2=0x7ff9f40781f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:15.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.611+0000 7ff9fa600700 1 -- 172.21.15.119:0/490809019 shutdown_connections 2024-05-27T09:35:15.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.611+0000 7ff9fa600700 1 -- 172.21.15.119:0/490809019 wait complete. 2024-05-27T09:35:15.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.612+0000 7ff9fa600700 1 Processor -- start 2024-05-27T09:35:15.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.612+0000 7ff9fa600700 1 -- start start 2024-05-27T09:35:15.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.613+0000 7ff9fa600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f4106de0 0x7ff9f419c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:15.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.613+0000 7ff9fa600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9f4107780 0x7ff9f419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:15.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.613+0000 7ff9fa600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f419cdc0 con 0x7ff9f4107780 2024-05-27T09:35:15.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.613+0000 7ff9fa600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f41a0b50 con 0x7ff9f4106de0 2024-05-27T09:35:15.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.613+0000 7ff9f3e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9f4107780 0x7ff9f419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:15.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.613+0000 7ff9f3e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9f4107780 0x7ff9f419c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38226/0 (socket says 172.21.15.119:38226) 2024-05-27T09:35:15.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.613+0000 7ff9f3e00700 1 -- 172.21.15.119:0/342945787 learned_addr learned my addr 172.21.15.119:0/342945787 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:35:15.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.613+0000 7ff9f9200700 1 --2- 172.21.15.119:0/342945787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f4106de0 0x7ff9f419c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:15.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.614+0000 7ff9f3e00700 1 -- 172.21.15.119:0/342945787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f4106de0 msgr2=0x7ff9f419c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:15.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.614+0000 7ff9f3e00700 1 --2- 172.21.15.119:0/342945787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f4106de0 0x7ff9f419c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:15.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.614+0000 7ff9f3e00700 1 -- 172.21.15.119:0/342945787 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e00095f0 con 0x7ff9f4107780 2024-05-27T09:35:15.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.614+0000 7ff9f3e00700 1 --2- 172.21.15.119:0/342945787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9f4107780 0x7ff9f419c6e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0005f50 tx=0x7ff9e000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:15.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.615+0000 7ff9f1600700 1 -- 172.21.15.119:0/342945787 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9e001d070 con 0x7ff9f4107780 2024-05-27T09:35:15.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.615+0000 7ff9f1600700 1 -- 172.21.15.119:0/342945787 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff9e000f460 con 0x7ff9f4107780 2024-05-27T09:35:15.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.615+0000 7ff9f1600700 1 -- 172.21.15.119:0/342945787 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9e0017630 con 0x7ff9f4107780 2024-05-27T09:35:15.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.615+0000 7ff9fa600700 1 -- 172.21.15.119:0/342945787 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f41a0dd0 con 0x7ff9f4107780 2024-05-27T09:35:15.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.615+0000 7ff9fa600700 1 -- 172.21.15.119:0/342945787 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f41a12c0 con 0x7ff9f4107780 2024-05-27T09:35:15.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.616+0000 7ff9fa600700 1 -- 172.21.15.119:0/342945787 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f410ead0 con 0x7ff9f4107780 2024-05-27T09:35:15.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.619+0000 7ff9f1600700 1 -- 172.21.15.119:0/342945787 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 38) v1 ==== 100184+0+0 (secure 0 0 0) 0x7ff9e000f5d0 con 0x7ff9f4107780 2024-05-27T09:35:15.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.620+0000 7ff9f1600700 1 --2- 172.21.15.119:0/342945787 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff9dc078b60 0x7ff9dc07b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:15.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.620+0000 7ff9f1600700 1 -- 172.21.15.119:0/342945787 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff9e009c4f0 con 0x7ff9f4107780 2024-05-27T09:35:15.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.621+0000 7ff9f9200700 1 --2- 172.21.15.119:0/342945787 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff9dc078b60 0x7ff9dc07b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:15.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.621+0000 7ff9f9200700 1 --2- 172.21.15.119:0/342945787 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff9dc078b60 0x7ff9dc07b010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8005fd0 tx=0x7ff9e8005dd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:15.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.625+0000 7ff9f1600700 1 -- 172.21.15.119:0/342945787 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7ff9e0063f30 con 0x7ff9f4107780 2024-05-27T09:35:15.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.868+0000 7ff9fa600700 1 -- 172.21.15.119:0/342945787 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9f41a16b0 con 0x7ff9dc078b60 2024-05-27T09:35:15.870 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:15 smithi119 ceph-mon[88269]: Reconfiguring crash.smithi119 (monmap changed)... 2024-05-27T09:35:15.870 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:15 smithi119 ceph-mon[88269]: Reconfiguring daemon crash.smithi119 on smithi119 2024-05-27T09:35:15.870 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:15 smithi119 ceph-mon[88269]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:35:15.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.881+0000 7ff9f1600700 1 -- 172.21.15.119:0/342945787 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff9f41a16b0 con 0x7ff9dc078b60 2024-05-27T09:35:15.882 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:35:15.882 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (2m) 40s ago 16m 13.9M - 0.25.0 c8568f914cd2 6b90564dcef7 2024-05-27T09:35:15.882 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (16m) 40s ago 16m 8190k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:35:15.882 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (15m) 16s ago 15m 8086k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:35:15.882 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (16m) 40s ago 16m 6665k - 18.2.0 dc2bc1663786 e5fafde44183 2024-05-27T09:35:15.882 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (15m) 16s ago 15m 6660k - 18.2.0 dc2bc1663786 46956ab99bab 2024-05-27T09:35:15.882 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (2m) 40s ago 16m 74.7M - 9.4.12 2bacad6d85d8 b3abdab08db4 2024-05-27T09:35:15.882 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (9m) 40s ago 9m 16.6M - 18.2.0 dc2bc1663786 93b21713fb1e 2024-05-27T09:35:15.882 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (9m) 40s ago 9m 130M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:35:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (9m) 16s ago 9m 16.5M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:35:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (9m) 16s ago 9m 128M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:35:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (5m) 40s ago 18m 558M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:35:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (4m) 16s ago 15m 531M - 19.0.0-3868-ga6507388 850a06a84d28 2fc3a80696fa 2024-05-27T09:35:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (59s) 40s ago 18m 96.6M 2048M 19.0.0-3868-ga6507388 850a06a84d28 3e5c7a9ad372 2024-05-27T09:35:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (22s) 16s ago 14m 86.9M 2048M 19.0.0-3868-ga6507388 850a06a84d28 e4d471274cf1 2024-05-27T09:35:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (3m) 40s ago 16m 7482k - 1.5.0 0da6a335fe13 709c37815cff 2024-05-27T09:35:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (3m) 16s ago 15m 7814k - 1.5.0 0da6a335fe13 999047a1a5bf 2024-05-27T09:35:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (14m) 40s ago 14m 225M 1216M 18.2.0 dc2bc1663786 825bd1392508 2024-05-27T09:35:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (13m) 40s ago 13m 253M 1216M 18.2.0 dc2bc1663786 fce9819903d9 2024-05-27T09:35:15.883 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (12m) 40s ago 12m 190M 1216M 18.2.0 dc2bc1663786 3955a6112c1c 2024-05-27T09:35:15.884 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (12m) 16s ago 12m 271M 2240M 18.2.0 dc2bc1663786 693c92220622 2024-05-27T09:35:15.884 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (11m) 16s ago 11m 247M 2240M 18.2.0 dc2bc1663786 7559e658e917 2024-05-27T09:35:15.884 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (11m) 16s ago 11m 232M 2240M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:35:15.884 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (3m) 40s ago 15m 62.3M - 2.43.0 a07b618ecd1d cba50454ed21 2024-05-27T09:35:15.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.884+0000 7ff9fa600700 1 -- 172.21.15.119:0/342945787 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff9dc078b60 msgr2=0x7ff9dc07b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:15.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.884+0000 7ff9fa600700 1 --2- 172.21.15.119:0/342945787 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff9dc078b60 0x7ff9dc07b010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8005fd0 tx=0x7ff9e8005dd0 comp rx=0 tx=0).stop 2024-05-27T09:35:15.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.884+0000 7ff9fa600700 1 -- 172.21.15.119:0/342945787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9f4107780 msgr2=0x7ff9f419c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:15.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.884+0000 7ff9fa600700 1 --2- 172.21.15.119:0/342945787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9f4107780 0x7ff9f419c6e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0005f50 tx=0x7ff9e000be20 comp rx=0 tx=0).stop 2024-05-27T09:35:15.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.884+0000 7ff9fa600700 1 -- 172.21.15.119:0/342945787 shutdown_connections 2024-05-27T09:35:15.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.884+0000 7ff9fa600700 1 --2- 172.21.15.119:0/342945787 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff9dc078b60 0x7ff9dc07b010 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:15.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.884+0000 7ff9fa600700 1 --2- 172.21.15.119:0/342945787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f4106de0 0x7ff9f419c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:15.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.884+0000 7ff9fa600700 1 --2- 172.21.15.119:0/342945787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9f4107780 0x7ff9f419c6e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:15.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.884+0000 7ff9fa600700 1 -- 172.21.15.119:0/342945787 >> 172.21.15.119:0/342945787 conn(0x7ff9f4075de0 msgr2=0x7ff9f410a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:15.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.884+0000 7ff9fa600700 1 -- 172.21.15.119:0/342945787 shutdown_connections 2024-05-27T09:35:15.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:15.884+0000 7ff9fa600700 1 -- 172.21.15.119:0/342945787 wait complete. 2024-05-27T09:35:16.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.039+0000 7fd471c00700 1 -- 172.21.15.119:0/3324190482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd46c10c010 msgr2=0x7fd46c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:16.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.039+0000 7fd471c00700 1 --2- 172.21.15.119:0/3324190482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd46c10c010 0x7fd46c106b80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd454009910 tx=0x7fd454009c20 comp rx=0 tx=0).stop 2024-05-27T09:35:16.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.039+0000 7fd471c00700 1 -- 172.21.15.119:0/3324190482 shutdown_connections 2024-05-27T09:35:16.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.039+0000 7fd471c00700 1 --2- 172.21.15.119:0/3324190482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd46c10c010 0x7fd46c106b80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:16.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.039+0000 7fd471c00700 1 --2- 172.21.15.119:0/3324190482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd46c10b700 0x7fd46c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:16.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.039+0000 7fd471c00700 1 -- 172.21.15.119:0/3324190482 >> 172.21.15.119:0/3324190482 conn(0x7fd46c076000 msgr2=0x7fd46c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:16.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.039+0000 7fd471c00700 1 -- 172.21.15.119:0/3324190482 shutdown_connections 2024-05-27T09:35:16.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.039+0000 7fd471c00700 1 -- 172.21.15.119:0/3324190482 wait complete. 2024-05-27T09:35:16.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.040+0000 7fd471c00700 1 Processor -- start 2024-05-27T09:35:16.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.040+0000 7fd471c00700 1 -- start start 2024-05-27T09:35:16.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.040+0000 7fd471c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd46c10b700 0x7fd46c19c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:16.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.040+0000 7fd471c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd46c10c010 0x7fd46c19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:16.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.040+0000 7fd471c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c19cf00 con 0x7fd46c10b700 2024-05-27T09:35:16.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.040+0000 7fd471c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c1a0c90 con 0x7fd46c10c010 2024-05-27T09:35:16.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.041+0000 7fd46aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd46c10b700 0x7fd46c19c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:16.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.041+0000 7fd46aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd46c10b700 0x7fd46c19c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38242/0 (socket says 172.21.15.119:38242) 2024-05-27T09:35:16.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.041+0000 7fd46aa00700 1 -- 172.21.15.119:0/1932811620 learned_addr learned my addr 172.21.15.119:0/1932811620 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:35:16.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.041+0000 7fd463e00700 1 --2- 172.21.15.119:0/1932811620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd46c10c010 0x7fd46c19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:16.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.041+0000 7fd46aa00700 1 -- 172.21.15.119:0/1932811620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd46c10c010 msgr2=0x7fd46c19c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:16.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.041+0000 7fd46aa00700 1 --2- 172.21.15.119:0/1932811620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd46c10c010 0x7fd46c19c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:16.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.041+0000 7fd46aa00700 1 -- 172.21.15.119:0/1932811620 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4540095f0 con 0x7fd46c10b700 2024-05-27T09:35:16.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.042+0000 7fd46aa00700 1 --2- 172.21.15.119:0/1932811620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd46c10b700 0x7fd46c19c2e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00c8a0 tx=0x7fd45c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:16.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.042+0000 7fd463400700 1 -- 172.21.15.119:0/1932811620 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd45c00cea0 con 0x7fd46c10b700 2024-05-27T09:35:16.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.042+0000 7fd463400700 1 -- 172.21.15.119:0/1932811620 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd45c004830 con 0x7fd46c10b700 2024-05-27T09:35:16.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.042+0000 7fd471c00700 1 -- 172.21.15.119:0/1932811620 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd46c1a0f70 con 0x7fd46c10b700 2024-05-27T09:35:16.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.042+0000 7fd463400700 1 -- 172.21.15.119:0/1932811620 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd45c0055b0 con 0x7fd46c10b700 2024-05-27T09:35:16.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.042+0000 7fd471c00700 1 -- 172.21.15.119:0/1932811620 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46c1a1540 con 0x7fd46c10b700 2024-05-27T09:35:16.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.043+0000 7fd471c00700 1 -- 172.21.15.119:0/1932811620 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd46c04ea50 con 0x7fd46c10b700 2024-05-27T09:35:16.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.045+0000 7fd463400700 1 -- 172.21.15.119:0/1932811620 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 38) v1 ==== 100184+0+0 (secure 0 0 0) 0x7fd45c0049a0 con 0x7fd46c10b700 2024-05-27T09:35:16.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.046+0000 7fd463400700 1 --2- 172.21.15.119:0/1932811620 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fd450078b60 0x7fd45007b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:16.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.046+0000 7fd463400700 1 -- 172.21.15.119:0/1932811620 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd45c09abc0 con 0x7fd46c10b700 2024-05-27T09:35:16.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.046+0000 7fd463e00700 1 --2- 172.21.15.119:0/1932811620 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fd450078b60 0x7fd45007b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:16.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.047+0000 7fd463e00700 1 --2- 172.21.15.119:0/1932811620 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fd450078b60 0x7fd45007b010 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd46c19d900 tx=0x7fd45401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:16.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.051+0000 7fd463400700 1 -- 172.21.15.119:0/1932811620 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fd45c062600 con 0x7fd46c10b700 2024-05-27T09:35:16.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:15 smithi138 ceph-mon[70004]: Reconfiguring crash.smithi119 (monmap changed)... 2024-05-27T09:35:16.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:15 smithi138 ceph-mon[70004]: Reconfiguring daemon crash.smithi119 on smithi119 2024-05-27T09:35:16.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:15 smithi138 ceph-mon[70004]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:35:16.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.401+0000 7fd471c00700 1 -- 172.21.15.119:0/1932811620 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd46c1a1a60 con 0x7fd46c10b700 2024-05-27T09:35:16.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.402+0000 7fd463400700 1 -- 172.21.15.119:0/1932811620 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7fd45c004c50 con 0x7fd46c10b700 2024-05-27T09:35:16.403 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:35:16.403 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:35:16.403 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:35:16.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:35:16.403 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:35:16.403 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:35:16.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:35:16.403 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:35:16.403 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-05-27T09:35:16.403 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:35:16.404 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:35:16.404 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-05-27T09:35:16.404 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:35:16.404 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:35:16.404 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 10, 2024-05-27T09:35:16.404 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 4 2024-05-27T09:35:16.404 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:35:16.404 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:35:16.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.405+0000 7fd471c00700 1 -- 172.21.15.119:0/1932811620 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fd450078b60 msgr2=0x7fd45007b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:16.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.405+0000 7fd471c00700 1 --2- 172.21.15.119:0/1932811620 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fd450078b60 0x7fd45007b010 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd46c19d900 tx=0x7fd45401a040 comp rx=0 tx=0).stop 2024-05-27T09:35:16.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.405+0000 7fd471c00700 1 -- 172.21.15.119:0/1932811620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd46c10b700 msgr2=0x7fd46c19c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:16.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.405+0000 7fd471c00700 1 --2- 172.21.15.119:0/1932811620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd46c10b700 0x7fd46c19c2e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00c8a0 tx=0x7fd45c00cc60 comp rx=0 tx=0).stop 2024-05-27T09:35:16.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.405+0000 7fd471c00700 1 -- 172.21.15.119:0/1932811620 shutdown_connections 2024-05-27T09:35:16.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.405+0000 7fd471c00700 1 --2- 172.21.15.119:0/1932811620 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fd450078b60 0x7fd45007b010 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:16.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.405+0000 7fd471c00700 1 --2- 172.21.15.119:0/1932811620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd46c10b700 0x7fd46c19c2e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:16.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.405+0000 7fd471c00700 1 --2- 172.21.15.119:0/1932811620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd46c10c010 0x7fd46c19c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:16.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.405+0000 7fd471c00700 1 -- 172.21.15.119:0/1932811620 >> 172.21.15.119:0/1932811620 conn(0x7fd46c076000 msgr2=0x7fd46c076940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:16.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.405+0000 7fd471c00700 1 -- 172.21.15.119:0/1932811620 shutdown_connections 2024-05-27T09:35:16.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.405+0000 7fd471c00700 1 -- 172.21.15.119:0/1932811620 wait complete. 2024-05-27T09:35:16.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.573+0000 7ff45e200700 1 -- 172.21.15.119:0/3934549899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff45810b700 msgr2=0x7ff45810bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:16.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.573+0000 7ff45e200700 1 --2- 172.21.15.119:0/3934549899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff45810b700 0x7ff45810bad0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff448009910 tx=0x7ff448009c20 comp rx=0 tx=0).stop 2024-05-27T09:35:16.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.573+0000 7ff45e200700 1 -- 172.21.15.119:0/3934549899 shutdown_connections 2024-05-27T09:35:16.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.573+0000 7ff45e200700 1 --2- 172.21.15.119:0/3934549899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff45810c010 0x7ff458106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:16.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.573+0000 7ff45e200700 1 --2- 172.21.15.119:0/3934549899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff45810b700 0x7ff45810bad0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:16.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.573+0000 7ff45e200700 1 -- 172.21.15.119:0/3934549899 >> 172.21.15.119:0/3934549899 conn(0x7ff458076000 msgr2=0x7ff458078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:16.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.573+0000 7ff45e200700 1 -- 172.21.15.119:0/3934549899 shutdown_connections 2024-05-27T09:35:16.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.573+0000 7ff45e200700 1 -- 172.21.15.119:0/3934549899 wait complete. 2024-05-27T09:35:16.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.574+0000 7ff45e200700 1 Processor -- start 2024-05-27T09:35:16.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.574+0000 7ff45e200700 1 -- start start 2024-05-27T09:35:16.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.574+0000 7ff45e200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff45810b700 0x7ff45819c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:16.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.574+0000 7ff45e200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff45810c010 0x7ff45819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:16.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.574+0000 7ff45e200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45819ced0 con 0x7ff45810c010 2024-05-27T09:35:16.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.574+0000 7ff45e200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4581a0c60 con 0x7ff45810b700 2024-05-27T09:35:16.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.575+0000 7ff456a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff45810c010 0x7ff45819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:16.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.575+0000 7ff456a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff45810c010 0x7ff45819c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38264/0 (socket says 172.21.15.119:38264) 2024-05-27T09:35:16.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.575+0000 7ff456a00700 1 -- 172.21.15.119:0/1845793296 learned_addr learned my addr 172.21.15.119:0/1845793296 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:35:16.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.575+0000 7ff457400700 1 --2- 172.21.15.119:0/1845793296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff45810b700 0x7ff45819c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:16.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.575+0000 7ff456a00700 1 -- 172.21.15.119:0/1845793296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff45810b700 msgr2=0x7ff45819c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:16.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.575+0000 7ff456a00700 1 --2- 172.21.15.119:0/1845793296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff45810b700 0x7ff45819c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:16.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.575+0000 7ff456a00700 1 -- 172.21.15.119:0/1845793296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4480095f0 con 0x7ff45810c010 2024-05-27T09:35:16.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.575+0000 7ff457400700 1 --2- 172.21.15.119:0/1845793296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff45810b700 0x7ff45819c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:35:16.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.576+0000 7ff456a00700 1 --2- 172.21.15.119:0/1845793296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff45810c010 0x7ff45819c7f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff44400c8f0 tx=0x7ff44400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:16.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.576+0000 7ff44fe00700 1 -- 172.21.15.119:0/1845793296 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff444007a50 con 0x7ff45810c010 2024-05-27T09:35:16.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.576+0000 7ff45e200700 1 -- 172.21.15.119:0/1845793296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4581a0f40 con 0x7ff45810c010 2024-05-27T09:35:16.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.576+0000 7ff44fe00700 1 -- 172.21.15.119:0/1845793296 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff444007bb0 con 0x7ff45810c010 2024-05-27T09:35:16.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.576+0000 7ff44fe00700 1 -- 172.21.15.119:0/1845793296 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff444018630 con 0x7ff45810c010 2024-05-27T09:35:16.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.576+0000 7ff45e200700 1 -- 172.21.15.119:0/1845793296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4581a1510 con 0x7ff45810c010 2024-05-27T09:35:16.578 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.577+0000 7ff45e200700 1 -- 172.21.15.119:0/1845793296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff45804ea50 con 0x7ff45810c010 2024-05-27T09:35:16.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.578+0000 7ff44fe00700 1 -- 172.21.15.119:0/1845793296 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 38) v1 ==== 100184+0+0 (secure 0 0 0) 0x7ff444007d20 con 0x7ff45810c010 2024-05-27T09:35:16.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.579+0000 7ff44fe00700 1 --2- 172.21.15.119:0/1845793296 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff43c078b60 0x7ff43c07b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:16.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.579+0000 7ff44fe00700 1 -- 172.21.15.119:0/1845793296 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff44409afe0 con 0x7ff45810c010 2024-05-27T09:35:16.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.580+0000 7ff457400700 1 --2- 172.21.15.119:0/1845793296 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff43c078b60 0x7ff43c07b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:16.581 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.580+0000 7ff457400700 1 --2- 172.21.15.119:0/1845793296 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff43c078b60 0x7ff43c07b010 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff448006010 tx=0x7ff44800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:16.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.584+0000 7ff44fe00700 1 -- 172.21.15.119:0/1845793296 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7ff4440d0df0 con 0x7ff45810c010 2024-05-27T09:35:16.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.870+0000 7ff45e200700 1 -- 172.21.15.119:0/1845793296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff45819d610 con 0x7ff45810c010 2024-05-27T09:35:16.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.872+0000 7ff44fe00700 1 -- 172.21.15.119:0/1845793296 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1889 (secure 0 0 0) 0x7ff4440629a0 con 0x7ff45810c010 2024-05-27T09:35:16.873 INFO:teuthology.orchestra.run.smithi119.stdout:e22 2024-05-27T09:35:16.873 INFO:teuthology.orchestra.run.smithi119.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T09:35:16.873 INFO:teuthology.orchestra.run.smithi119.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:35:16.873 INFO:teuthology.orchestra.run.smithi119.stdout:legacy client fscid: 1 2024-05-27T09:35:16.873 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:35:16.873 INFO:teuthology.orchestra.run.smithi119.stdout:Filesystem 'cephfs' (1) 2024-05-27T09:35:16.873 INFO:teuthology.orchestra.run.smithi119.stdout:fs_name cephfs 2024-05-27T09:35:16.873 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 22 2024-05-27T09:35:16.873 INFO:teuthology.orchestra.run.smithi119.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-27T09:35:16.873 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-05-27T09:25:53.387388+0000 2024-05-27T09:35:16.873 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2024-05-27T09:34:01.250304+0000 2024-05-27T09:35:16.874 INFO:teuthology.orchestra.run.smithi119.stdout:tableserver 0 2024-05-27T09:35:16.874 INFO:teuthology.orchestra.run.smithi119.stdout:root 0 2024-05-27T09:35:16.874 INFO:teuthology.orchestra.run.smithi119.stdout:session_timeout 60 2024-05-27T09:35:16.874 INFO:teuthology.orchestra.run.smithi119.stdout:session_autoclose 300 2024-05-27T09:35:16.874 INFO:teuthology.orchestra.run.smithi119.stdout:max_file_size 1099511627776 2024-05-27T09:35:16.874 INFO:teuthology.orchestra.run.smithi119.stdout:max_xattr_size 65536 2024-05-27T09:35:16.874 INFO:teuthology.orchestra.run.smithi119.stdout:required_client_features {} 2024-05-27T09:35:16.874 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure 0 2024-05-27T09:35:16.874 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure_osd_epoch 46 2024-05-27T09:35:16.874 INFO:teuthology.orchestra.run.smithi119.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:35:16.874 INFO:teuthology.orchestra.run.smithi119.stdout:max_mds 2 2024-05-27T09:35:16.874 INFO:teuthology.orchestra.run.smithi119.stdout:in 0,1 2024-05-27T09:35:16.874 INFO:teuthology.orchestra.run.smithi119.stdout:up {0=14472,1=24241} 2024-05-27T09:35:16.875 INFO:teuthology.orchestra.run.smithi119.stdout:failed 2024-05-27T09:35:16.875 INFO:teuthology.orchestra.run.smithi119.stdout:damaged 2024-05-27T09:35:16.875 INFO:teuthology.orchestra.run.smithi119.stdout:stopped 2024-05-27T09:35:16.875 INFO:teuthology.orchestra.run.smithi119.stdout:data_pools [3] 2024-05-27T09:35:16.875 INFO:teuthology.orchestra.run.smithi119.stdout:metadata_pool 2 2024-05-27T09:35:16.875 INFO:teuthology.orchestra.run.smithi119.stdout:inline_data enabled 2024-05-27T09:35:16.875 INFO:teuthology.orchestra.run.smithi119.stdout:balancer 2024-05-27T09:35:16.875 INFO:teuthology.orchestra.run.smithi119.stdout:bal_rank_mask -1 2024-05-27T09:35:16.875 INFO:teuthology.orchestra.run.smithi119.stdout:standby_count_wanted 1 2024-05-27T09:35:16.875 INFO:teuthology.orchestra.run.smithi119.stdout:qdb_cluster leader: 0 members: 2024-05-27T09:35:16.875 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.ppmzzk{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:35:16.875 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.ndfkhd{1:24241} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/897058594,v1:172.21.15.138:6825/897058594] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:35:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:35:16.875 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:35:16.875 INFO:teuthology.orchestra.run.smithi119.stdout:Standby daemons: 2024-05-27T09:35:16.876 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:35:16.876 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.nghcey{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.119:6826/413736191,v1:172.21.15.119:6827/413736191] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:35:16.876 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.alszpk{-1:24253} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/2345607053,v1:172.21.15.138:6827/2345607053] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:35:16.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.876+0000 7ff45e200700 1 -- 172.21.15.119:0/1845793296 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff43c078b60 msgr2=0x7ff43c07b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:16.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.876+0000 7ff45e200700 1 --2- 172.21.15.119:0/1845793296 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff43c078b60 0x7ff43c07b010 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff448006010 tx=0x7ff44800b540 comp rx=0 tx=0).stop 2024-05-27T09:35:16.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.876+0000 7ff45e200700 1 -- 172.21.15.119:0/1845793296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff45810c010 msgr2=0x7ff45819c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:16.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.876+0000 7ff45e200700 1 --2- 172.21.15.119:0/1845793296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff45810c010 0x7ff45819c7f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff44400c8f0 tx=0x7ff44400ccb0 comp rx=0 tx=0).stop 2024-05-27T09:35:16.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.877+0000 7ff45e200700 1 -- 172.21.15.119:0/1845793296 shutdown_connections 2024-05-27T09:35:16.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.877+0000 7ff45e200700 1 --2- 172.21.15.119:0/1845793296 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff43c078b60 0x7ff43c07b010 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:16.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.877+0000 7ff45e200700 1 --2- 172.21.15.119:0/1845793296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff45810b700 0x7ff45819c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:16.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.877+0000 7ff45e200700 1 --2- 172.21.15.119:0/1845793296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff45810c010 0x7ff45819c7f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:16.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.877+0000 7ff45e200700 1 -- 172.21.15.119:0/1845793296 >> 172.21.15.119:0/1845793296 conn(0x7ff458076000 msgr2=0x7ff458076880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:16.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.877+0000 7ff45e200700 1 -- 172.21.15.119:0/1845793296 shutdown_connections 2024-05-27T09:35:16.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:16.877+0000 7ff45e200700 1 -- 172.21.15.119:0/1845793296 wait complete. 2024-05-27T09:35:16.879 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 22 2024-05-27T09:35:17.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.041+0000 7fdba5e00700 1 -- 172.21.15.119:0/2678505225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba0073a00 msgr2=0x7fdba010c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:17.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.041+0000 7fdba5e00700 1 --2- 172.21.15.119:0/2678505225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba0073a00 0x7fdba010c450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009910 tx=0x7fdb8c009c20 comp rx=0 tx=0).stop 2024-05-27T09:35:17.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.041+0000 7fdba5e00700 1 -- 172.21.15.119:0/2678505225 shutdown_connections 2024-05-27T09:35:17.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.041+0000 7fdba5e00700 1 --2- 172.21.15.119:0/2678505225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba0073a00 0x7fdba010c450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:17.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.041+0000 7fdba5e00700 1 --2- 172.21.15.119:0/2678505225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdba00730f0 0x7fdba00734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:17.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.041+0000 7fdba5e00700 1 -- 172.21.15.119:0/2678505225 >> 172.21.15.119:0/2678505225 conn(0x7fdba00fbc30 msgr2=0x7fdba00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:17.043 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:16 smithi119 ceph-mon[88269]: pgmap v27: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:17.043 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:16 smithi119 ceph-mon[88269]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:35:17.043 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:16 smithi119 ceph-mon[88269]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:35:17.043 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:16 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/1932811620' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:35:17.043 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:16 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:17.043 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:16 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:17.043 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:16 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-27T09:35:17.043 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:16 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:17.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.041+0000 7fdba5e00700 1 -- 172.21.15.119:0/2678505225 shutdown_connections 2024-05-27T09:35:17.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.041+0000 7fdba5e00700 1 -- 172.21.15.119:0/2678505225 wait complete. 2024-05-27T09:35:17.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.042+0000 7fdba5e00700 1 Processor -- start 2024-05-27T09:35:17.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.043+0000 7fdba5e00700 1 -- start start 2024-05-27T09:35:17.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.043+0000 7fdba5e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdba00730f0 0x7fdba0197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:17.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.043+0000 7fdba5e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba0073a00 0x7fdba01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:17.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.043+0000 7fdba5e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba019c8f0 con 0x7fdba0073a00 2024-05-27T09:35:17.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.043+0000 7fdba5e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba019ca60 con 0x7fdba00730f0 2024-05-27T09:35:17.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.043+0000 7fdb9e000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba0073a00 0x7fdba01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:17.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.043+0000 7fdb9e000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba0073a00 0x7fdba01985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38282/0 (socket says 172.21.15.119:38282) 2024-05-27T09:35:17.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.043+0000 7fdb9e000700 1 -- 172.21.15.119:0/2631840907 learned_addr learned my addr 172.21.15.119:0/2631840907 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:35:17.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.044+0000 7fdb9ea00700 1 --2- 172.21.15.119:0/2631840907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdba00730f0 0x7fdba0197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:17.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.044+0000 7fdb9e000700 1 -- 172.21.15.119:0/2631840907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdba00730f0 msgr2=0x7fdba0197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:17.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.044+0000 7fdb9e000700 1 --2- 172.21.15.119:0/2631840907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdba00730f0 0x7fdba0197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:17.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.044+0000 7fdb9e000700 1 -- 172.21.15.119:0/2631840907 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8c0095f0 con 0x7fdba0073a00 2024-05-27T09:35:17.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.044+0000 7fdb9e000700 1 --2- 172.21.15.119:0/2631840907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba0073a00 0x7fdba01985b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00b5c0 tx=0x7fdb8c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:17.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.045+0000 7fdba4a00700 1 -- 172.21.15.119:0/2631840907 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb8c01d070 con 0x7fdba0073a00 2024-05-27T09:35:17.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.045+0000 7fdba4a00700 1 -- 172.21.15.119:0/2631840907 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb8c00f460 con 0x7fdba0073a00 2024-05-27T09:35:17.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.045+0000 7fdba5e00700 1 -- 172.21.15.119:0/2631840907 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba019cd40 con 0x7fdba0073a00 2024-05-27T09:35:17.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.045+0000 7fdba4a00700 1 -- 172.21.15.119:0/2631840907 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb8c017630 con 0x7fdba0073a00 2024-05-27T09:35:17.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.045+0000 7fdba5e00700 1 -- 172.21.15.119:0/2631840907 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba019d290 con 0x7fdba0073a00 2024-05-27T09:35:17.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.046+0000 7fdba5e00700 1 -- 172.21.15.119:0/2631840907 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba0109b50 con 0x7fdba0073a00 2024-05-27T09:35:17.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.047+0000 7fdba4a00700 1 -- 172.21.15.119:0/2631840907 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 38) v1 ==== 100184+0+0 (secure 0 0 0) 0x7fdb8c003a60 con 0x7fdba0073a00 2024-05-27T09:35:17.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.048+0000 7fdba4a00700 1 --2- 172.21.15.119:0/2631840907 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdb84078b60 0x7fdb8407b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:17.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.049+0000 7fdba4a00700 1 -- 172.21.15.119:0/2631840907 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fdb8c09c4b0 con 0x7fdba0073a00 2024-05-27T09:35:17.049 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.049+0000 7fdb9ea00700 1 --2- 172.21.15.119:0/2631840907 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdb84078b60 0x7fdb8407b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:17.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.050+0000 7fdb9ea00700 1 --2- 172.21.15.119:0/2631840907 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdb84078b60 0x7fdb8407b010 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdb90006fd0 tx=0x7fdb90008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:17.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.054+0000 7fdba4a00700 1 -- 172.21.15.119:0/2631840907 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fdb8c063ef0 con 0x7fdba0073a00 2024-05-27T09:35:17.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:16 smithi138 ceph-mon[70004]: pgmap v27: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:17.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:16 smithi138 ceph-mon[70004]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:35:17.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:16 smithi138 ceph-mon[70004]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:35:17.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:16 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/1932811620' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:35:17.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:16 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:17.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:16 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:17.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:16 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-27T09:35:17.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:16 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:17.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.314+0000 7fdba5e00700 1 -- 172.21.15.119:0/2631840907 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdba019d500 con 0x7fdb84078b60 2024-05-27T09:35:17.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.316+0000 7fdba4a00700 1 -- 172.21.15.119:0/2631840907 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fdba019d500 con 0x7fdb84078b60 2024-05-27T09:35:17.317 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:35:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:35:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:35:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-27T09:35:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [ 2024-05-27T09:35:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: "mon", 2024-05-27T09:35:17.317 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr" 2024-05-27T09:35:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-05-27T09:35:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "4/23 daemons upgraded", 2024-05-27T09:35:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "Currently upgrading mon daemons", 2024-05-27T09:35:17.318 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:35:17.318 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:35:17.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.319+0000 7fdba5e00700 1 -- 172.21.15.119:0/2631840907 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdb84078b60 msgr2=0x7fdb8407b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:17.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.319+0000 7fdba5e00700 1 --2- 172.21.15.119:0/2631840907 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdb84078b60 0x7fdb8407b010 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdb90006fd0 tx=0x7fdb90008040 comp rx=0 tx=0).stop 2024-05-27T09:35:17.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.319+0000 7fdba5e00700 1 -- 172.21.15.119:0/2631840907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba0073a00 msgr2=0x7fdba01985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:17.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.319+0000 7fdba5e00700 1 --2- 172.21.15.119:0/2631840907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba0073a00 0x7fdba01985b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00b5c0 tx=0x7fdb8c00be00 comp rx=0 tx=0).stop 2024-05-27T09:35:17.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.319+0000 7fdba5e00700 1 -- 172.21.15.119:0/2631840907 shutdown_connections 2024-05-27T09:35:17.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.319+0000 7fdba5e00700 1 --2- 172.21.15.119:0/2631840907 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdb84078b60 0x7fdb8407b010 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:17.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.319+0000 7fdba5e00700 1 --2- 172.21.15.119:0/2631840907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdba00730f0 0x7fdba0197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:17.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.319+0000 7fdba5e00700 1 --2- 172.21.15.119:0/2631840907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdba0073a00 0x7fdba01985b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:17.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.319+0000 7fdba5e00700 1 -- 172.21.15.119:0/2631840907 >> 172.21.15.119:0/2631840907 conn(0x7fdba00fbc30 msgr2=0x7fdba0106c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:17.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.320+0000 7fdba5e00700 1 -- 172.21.15.119:0/2631840907 shutdown_connections 2024-05-27T09:35:17.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.320+0000 7fdba5e00700 1 -- 172.21.15.119:0/2631840907 wait complete. 2024-05-27T09:35:17.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.469+0000 7f324be00700 1 -- 172.21.15.119:0/2028147012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f324410b700 msgr2=0x7f324410bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:17.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.469+0000 7f324be00700 1 --2- 172.21.15.119:0/2028147012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f324410b700 0x7f324410bad0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3238009910 tx=0x7f3238009c20 comp rx=0 tx=0).stop 2024-05-27T09:35:17.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.469+0000 7f324be00700 1 -- 172.21.15.119:0/2028147012 shutdown_connections 2024-05-27T09:35:17.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.469+0000 7f324be00700 1 --2- 172.21.15.119:0/2028147012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f324410c010 0x7f3244106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:17.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.469+0000 7f324be00700 1 --2- 172.21.15.119:0/2028147012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f324410b700 0x7f324410bad0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:17.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.469+0000 7f324be00700 1 -- 172.21.15.119:0/2028147012 >> 172.21.15.119:0/2028147012 conn(0x7f3244076000 msgr2=0x7f3244078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:17.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.469+0000 7f324be00700 1 -- 172.21.15.119:0/2028147012 shutdown_connections 2024-05-27T09:35:17.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.469+0000 7f324be00700 1 -- 172.21.15.119:0/2028147012 wait complete. 2024-05-27T09:35:17.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.470+0000 7f324be00700 1 Processor -- start 2024-05-27T09:35:17.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.470+0000 7f324be00700 1 -- start start 2024-05-27T09:35:17.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.471+0000 7f324be00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f324410b700 0x7f324419c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:17.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.471+0000 7f324be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f324410c010 0x7f324419c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:17.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.471+0000 7f324be00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324419d030 con 0x7f324410c010 2024-05-27T09:35:17.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.471+0000 7f324be00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32441a0dc0 con 0x7f324410b700 2024-05-27T09:35:17.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.471+0000 7f3248a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f324410c010 0x7f324419c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:17.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.471+0000 7f3248a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f324410c010 0x7f324419c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38302/0 (socket says 172.21.15.119:38302) 2024-05-27T09:35:17.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.471+0000 7f3248a00700 1 -- 172.21.15.119:0/2420373180 learned_addr learned my addr 172.21.15.119:0/2420373180 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:35:17.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.471+0000 7f3249400700 1 --2- 172.21.15.119:0/2420373180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f324410b700 0x7f324419c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:17.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.471+0000 7f3248a00700 1 -- 172.21.15.119:0/2420373180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f324410b700 msgr2=0x7f324419c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:17.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.472+0000 7f3248a00700 1 --2- 172.21.15.119:0/2420373180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f324410b700 0x7f324419c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:17.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.472+0000 7f3248a00700 1 -- 172.21.15.119:0/2420373180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32380095f0 con 0x7f324410c010 2024-05-27T09:35:17.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.472+0000 7f3248a00700 1 --2- 172.21.15.119:0/2420373180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f324410c010 0x7f324419c950 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f323000b480 tx=0x7f323000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:17.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.472+0000 7f3242000700 1 -- 172.21.15.119:0/2420373180 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32300118c0 con 0x7f324410c010 2024-05-27T09:35:17.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.472+0000 7f3242000700 1 -- 172.21.15.119:0/2420373180 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3230010380 con 0x7f324410c010 2024-05-27T09:35:17.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.472+0000 7f324be00700 1 -- 172.21.15.119:0/2420373180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32441a10a0 con 0x7f324410c010 2024-05-27T09:35:17.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.472+0000 7f3242000700 1 -- 172.21.15.119:0/2420373180 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f323000f4f0 con 0x7f324410c010 2024-05-27T09:35:17.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.472+0000 7f324be00700 1 -- 172.21.15.119:0/2420373180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32441a1670 con 0x7f324410c010 2024-05-27T09:35:17.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.473+0000 7f324be00700 1 -- 172.21.15.119:0/2420373180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f324410ecd0 con 0x7f324410c010 2024-05-27T09:35:17.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.475+0000 7f3242000700 1 -- 172.21.15.119:0/2420373180 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 38) v1 ==== 100184+0+0 (secure 0 0 0) 0x7f32300104f0 con 0x7f324410c010 2024-05-27T09:35:17.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.475+0000 7f3242000700 1 --2- 172.21.15.119:0/2420373180 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f322c078b60 0x7f322c07b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:17.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.476+0000 7f3242000700 1 -- 172.21.15.119:0/2420373180 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f323009aaf0 con 0x7f324410c010 2024-05-27T09:35:17.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.476+0000 7f3249400700 1 --2- 172.21.15.119:0/2420373180 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f322c078b60 0x7f322c07b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:17.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.477+0000 7f3249400700 1 --2- 172.21.15.119:0/2420373180 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f322c078b60 0x7f322c07b010 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3238006010 tx=0x7f323801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:17.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.481+0000 7f3242000700 1 -- 172.21.15.119:0/2420373180 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f32300625c0 con 0x7f324410c010 2024-05-27T09:35:17.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.818+0000 7f324be00700 1 -- 172.21.15.119:0/2420373180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f324404ea50 con 0x7f324410c010 2024-05-27T09:35:17.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.818+0000 7f3242000700 1 -- 172.21.15.119:0/2420373180 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f3230061f60 con 0x7f324410c010 2024-05-27T09:35:17.819 INFO:teuthology.orchestra.run.smithi119.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-27T09:35:17.819 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:35:17.819 INFO:teuthology.orchestra.run.smithi119.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:35:17.820 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:17 smithi119 ceph-mon[88269]: Reconfiguring osd.0 (monmap changed)... 2024-05-27T09:35:17.820 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:17 smithi119 ceph-mon[88269]: Reconfiguring daemon osd.0 on smithi119 2024-05-27T09:35:17.820 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:17 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/1845793296' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:35:17.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.821+0000 7f324be00700 1 -- 172.21.15.119:0/2420373180 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f322c078b60 msgr2=0x7f322c07b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:17.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.821+0000 7f324be00700 1 --2- 172.21.15.119:0/2420373180 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f322c078b60 0x7f322c07b010 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3238006010 tx=0x7f323801a040 comp rx=0 tx=0).stop 2024-05-27T09:35:17.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.821+0000 7f324be00700 1 -- 172.21.15.119:0/2420373180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f324410c010 msgr2=0x7f324419c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:17.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.821+0000 7f324be00700 1 --2- 172.21.15.119:0/2420373180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f324410c010 0x7f324419c950 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f323000b480 tx=0x7f323000b840 comp rx=0 tx=0).stop 2024-05-27T09:35:17.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.822+0000 7f324be00700 1 -- 172.21.15.119:0/2420373180 shutdown_connections 2024-05-27T09:35:17.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.822+0000 7f324be00700 1 --2- 172.21.15.119:0/2420373180 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f322c078b60 0x7f322c07b010 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:17.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.822+0000 7f324be00700 1 --2- 172.21.15.119:0/2420373180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f324410b700 0x7f324419c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:17.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.822+0000 7f324be00700 1 --2- 172.21.15.119:0/2420373180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f324410c010 0x7f324419c950 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:17.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.822+0000 7f324be00700 1 -- 172.21.15.119:0/2420373180 >> 172.21.15.119:0/2420373180 conn(0x7f3244076000 msgr2=0x7f3244076ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:17.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.822+0000 7f324be00700 1 -- 172.21.15.119:0/2420373180 shutdown_connections 2024-05-27T09:35:17.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:17.822+0000 7f324be00700 1 -- 172.21.15.119:0/2420373180 wait complete. 2024-05-27T09:35:18.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:17 smithi138 ceph-mon[70004]: Reconfiguring osd.0 (monmap changed)... 2024-05-27T09:35:18.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:17 smithi138 ceph-mon[70004]: Reconfiguring daemon osd.0 on smithi119 2024-05-27T09:35:18.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:17 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/1845793296' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:35:18.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:18 smithi119 ceph-mon[88269]: pgmap v28: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:18.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:18 smithi119 ceph-mon[88269]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:35:18.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:18 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/2420373180' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:35:19.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:18 smithi138 ceph-mon[70004]: pgmap v28: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:19.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:18 smithi138 ceph-mon[70004]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:35:19.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:18 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/2420373180' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:35:20.161 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:19 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:20.161 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:19 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:20.161 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:19 smithi119 ceph-mon[88269]: Reconfiguring osd.1 (monmap changed)... 2024-05-27T09:35:20.161 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:19 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-27T09:35:20.161 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:19 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:20.161 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:19 smithi119 ceph-mon[88269]: Reconfiguring daemon osd.1 on smithi119 2024-05-27T09:35:20.384 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:19 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:20.384 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:19 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:20.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:19 smithi138 ceph-mon[70004]: Reconfiguring osd.1 (monmap changed)... 2024-05-27T09:35:20.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:19 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-27T09:35:20.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:19 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:20.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:19 smithi138 ceph-mon[70004]: Reconfiguring daemon osd.1 on smithi119 2024-05-27T09:35:21.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:20 smithi119 ceph-mon[88269]: pgmap v29: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:21.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:20 smithi138 ceph-mon[70004]: pgmap v29: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:22.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:22 smithi119 ceph-mon[88269]: pgmap v30: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:22.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:22 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:22.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:22 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:22.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:22 smithi119 ceph-mon[88269]: Reconfiguring osd.2 (monmap changed)... 2024-05-27T09:35:22.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:22 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-27T09:35:22.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:22 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:22.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:22 smithi119 ceph-mon[88269]: Reconfiguring daemon osd.2 on smithi119 2024-05-27T09:35:22.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:22 smithi138 ceph-mon[70004]: pgmap v30: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:22.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:22 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:22.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:22 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:22.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:22 smithi138 ceph-mon[70004]: Reconfiguring osd.2 (monmap changed)... 2024-05-27T09:35:22.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:22 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-27T09:35:22.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:22 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:22.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:22 smithi138 ceph-mon[70004]: Reconfiguring daemon osd.2 on smithi119 2024-05-27T09:35:24.867 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:24 smithi119 ceph-mon[88269]: pgmap v31: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:24.867 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:24 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:24.867 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:24 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:24.867 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:24 smithi119 ceph-mon[88269]: Reconfiguring mds.cephfs.smithi119.nghcey (monmap changed)... 2024-05-27T09:35:24.867 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:24 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi119.nghcey", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:35:24.867 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:24 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:24.867 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:24 smithi119 ceph-mon[88269]: Reconfiguring daemon mds.cephfs.smithi119.nghcey on smithi119 2024-05-27T09:35:24.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:24 smithi138 ceph-mon[70004]: pgmap v31: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:24.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:24 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:24.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:24 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:24.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:24 smithi138 ceph-mon[70004]: Reconfiguring mds.cephfs.smithi119.nghcey (monmap changed)... 2024-05-27T09:35:24.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:24 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi119.nghcey", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:35:24.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:24 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:24.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:24 smithi138 ceph-mon[70004]: Reconfiguring daemon mds.cephfs.smithi119.nghcey on smithi119 2024-05-27T09:35:27.207 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:26 smithi119 ceph-mon[88269]: pgmap v32: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:27.208 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:26 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:27.208 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:26 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:27.208 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:26 smithi119 ceph-mon[88269]: Reconfiguring mds.cephfs.smithi119.ppmzzk (monmap changed)... 2024-05-27T09:35:27.208 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:26 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi119.ppmzzk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:35:27.208 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:26 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:27.208 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:26 smithi119 ceph-mon[88269]: Reconfiguring daemon mds.cephfs.smithi119.ppmzzk on smithi119 2024-05-27T09:35:27.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:26 smithi138 ceph-mon[70004]: pgmap v32: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:27.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:26 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:27.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:26 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:27.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:26 smithi138 ceph-mon[70004]: Reconfiguring mds.cephfs.smithi119.ppmzzk (monmap changed)... 2024-05-27T09:35:27.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:26 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi119.ppmzzk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:35:27.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:26 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:27.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:26 smithi138 ceph-mon[70004]: Reconfiguring daemon mds.cephfs.smithi119.ppmzzk on smithi119 2024-05-27T09:35:28.195 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:27 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:35:28.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:27 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:35:29.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:28 smithi119 ceph-mon[88269]: pgmap v33: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:29.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:29.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:29.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi138", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-27T09:35:29.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi138", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-27T09:35:29.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi138", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-27T09:35:29.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi138"}]: dispatch 2024-05-27T09:35:29.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:29.356 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:28 smithi138 ceph-mon[70004]: pgmap v33: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:29.356 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:28 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:29.356 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:28 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:29.356 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:28 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi138", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-27T09:35:29.356 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:28 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi138", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-27T09:35:29.357 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:28 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi138", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-05-27T09:35:29.357 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:28 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi138"}]: dispatch 2024-05-27T09:35:29.357 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:28 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:30.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:30 smithi138 ceph-mon[70004]: Reconfiguring ceph-exporter.smithi138 (monmap changed)... 2024-05-27T09:35:30.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:30 smithi138 ceph-mon[70004]: Unable to update caps for client.ceph-exporter.smithi138 2024-05-27T09:35:30.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:30 smithi138 ceph-mon[70004]: Reconfiguring daemon ceph-exporter.smithi138 on smithi138 2024-05-27T09:35:30.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:30 smithi119 ceph-mon[88269]: Reconfiguring ceph-exporter.smithi138 (monmap changed)... 2024-05-27T09:35:30.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:30 smithi119 ceph-mon[88269]: Unable to update caps for client.ceph-exporter.smithi138 2024-05-27T09:35:30.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:30 smithi119 ceph-mon[88269]: Reconfiguring daemon ceph-exporter.smithi138 on smithi138 2024-05-27T09:35:31.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:31 smithi119 ceph-mon[88269]: pgmap v34: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:31.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:31 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:31.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:31 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:31.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:31 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-27T09:35:31.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:31 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:31.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:31 smithi138 ceph-mon[70004]: pgmap v34: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:31.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:31 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:31.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:31 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:31.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:31 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-27T09:35:31.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:31 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:32.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:32 smithi138 ceph-mon[70004]: Reconfiguring crash.smithi138 (monmap changed)... 2024-05-27T09:35:32.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:32 smithi138 ceph-mon[70004]: Reconfiguring daemon crash.smithi138 on smithi138 2024-05-27T09:35:32.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:32 smithi119 ceph-mon[88269]: Reconfiguring crash.smithi138 (monmap changed)... 2024-05-27T09:35:32.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:32 smithi119 ceph-mon[88269]: Reconfiguring daemon crash.smithi138 on smithi138 2024-05-27T09:35:33.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:33 smithi119 ceph-mon[88269]: pgmap v35: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:33.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:33 smithi138 ceph-mon[70004]: pgmap v35: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:34.444 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:34 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:34.444 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:34 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:34.444 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:34 smithi138 ceph-mon[70004]: Reconfiguring mgr.smithi138.dxsuqf (monmap changed)... 2024-05-27T09:35:34.444 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:34 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.dxsuqf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T09:35:34.444 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:34 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T09:35:34.444 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:34 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:34.444 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:34 smithi138 ceph-mon[70004]: Reconfiguring daemon mgr.smithi138.dxsuqf on smithi138 2024-05-27T09:35:34.444 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:34 smithi138 ceph-mon[70004]: pgmap v36: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:34.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:34 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:34.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:34 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:34.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:34 smithi119 ceph-mon[88269]: Reconfiguring mgr.smithi138.dxsuqf (monmap changed)... 2024-05-27T09:35:34.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:34 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.dxsuqf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T09:35:34.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:34 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T09:35:34.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:34 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:34.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:34 smithi119 ceph-mon[88269]: Reconfiguring daemon mgr.smithi138.dxsuqf on smithi138 2024-05-27T09:35:34.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:34 smithi119 ceph-mon[88269]: pgmap v36: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:36.850 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:36 smithi138 ceph-mon[70004]: pgmap v37: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:36.850 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:36 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:36.850 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:36 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:36.850 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:36 smithi138 ceph-mon[70004]: Reconfiguring mon.smithi138 (monmap changed)... 2024-05-27T09:35:36.850 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:36 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T09:35:36.851 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:36 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T09:35:36.851 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:36 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:36.851 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:36 smithi138 ceph-mon[70004]: Reconfiguring daemon mon.smithi138 on smithi138 2024-05-27T09:35:36.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:36 smithi119 ceph-mon[88269]: pgmap v37: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:36.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:36 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:36.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:36 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:36.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:36 smithi119 ceph-mon[88269]: Reconfiguring mon.smithi138 (monmap changed)... 2024-05-27T09:35:36.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:36 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T09:35:36.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:36 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T09:35:36.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:36 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:36.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:36 smithi119 ceph-mon[88269]: Reconfiguring daemon mon.smithi138 on smithi138 2024-05-27T09:35:39.072 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:38 smithi138 ceph-mon[70004]: pgmap v38: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:39.072 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:38 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:39.072 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:38 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:39.072 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:38 smithi138 ceph-mon[70004]: Reconfiguring osd.3 (monmap changed)... 2024-05-27T09:35:39.072 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:38 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-27T09:35:39.072 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:38 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:39.072 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:38 smithi138 ceph-mon[70004]: Reconfiguring daemon osd.3 on smithi138 2024-05-27T09:35:39.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:38 smithi119 ceph-mon[88269]: pgmap v38: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:39.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:38 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:39.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:38 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:39.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:38 smithi119 ceph-mon[88269]: Reconfiguring osd.3 (monmap changed)... 2024-05-27T09:35:39.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:38 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-27T09:35:39.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:38 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:39.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:38 smithi119 ceph-mon[88269]: Reconfiguring daemon osd.3 on smithi138 2024-05-27T09:35:41.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:40 smithi138 ceph-mon[70004]: pgmap v39: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:41.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:40 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:41.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:40 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:41.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:40 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-27T09:35:41.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:40 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:41.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:40 smithi119 ceph-mon[88269]: pgmap v39: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:41.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:40 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:41.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:40 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:41.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:40 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-27T09:35:41.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:40 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:42.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:41 smithi138 ceph-mon[70004]: Reconfiguring osd.4 (monmap changed)... 2024-05-27T09:35:42.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:41 smithi138 ceph-mon[70004]: Reconfiguring daemon osd.4 on smithi138 2024-05-27T09:35:42.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:41 smithi119 ceph-mon[88269]: Reconfiguring osd.4 (monmap changed)... 2024-05-27T09:35:42.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:41 smithi119 ceph-mon[88269]: Reconfiguring daemon osd.4 on smithi138 2024-05-27T09:35:43.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:42 smithi119 ceph-mon[88269]: pgmap v40: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:43.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:42 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:35:43.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:42 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:43.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:42 smithi138 ceph-mon[70004]: pgmap v40: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:43.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:42 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:35:43.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:42 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:44.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:43 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:44.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:43 smithi138 ceph-mon[70004]: Reconfiguring osd.5 (monmap changed)... 2024-05-27T09:35:44.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:43 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-27T09:35:44.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:43 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:44.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:43 smithi138 ceph-mon[70004]: Reconfiguring daemon osd.5 on smithi138 2024-05-27T09:35:44.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:43 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:44.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:43 smithi119 ceph-mon[88269]: Reconfiguring osd.5 (monmap changed)... 2024-05-27T09:35:44.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:43 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-27T09:35:44.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:43 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:44.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:43 smithi119 ceph-mon[88269]: Reconfiguring daemon osd.5 on smithi138 2024-05-27T09:35:45.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:44 smithi138 ceph-mon[70004]: pgmap v41: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:45.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:44 smithi119 ceph-mon[88269]: pgmap v41: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:46.686 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:46 smithi138 ceph-mon[70004]: pgmap v42: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:46.686 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:46 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:46.686 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:46 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:46.686 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:46 smithi138 ceph-mon[70004]: Reconfiguring mds.cephfs.smithi138.ndfkhd (monmap changed)... 2024-05-27T09:35:46.686 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:46 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.ndfkhd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:35:46.686 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:46 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:46.686 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:46 smithi138 ceph-mon[70004]: Reconfiguring daemon mds.cephfs.smithi138.ndfkhd on smithi138 2024-05-27T09:35:46.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:46 smithi119 ceph-mon[88269]: pgmap v42: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:46.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:46 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:46.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:46 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:46.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:46 smithi119 ceph-mon[88269]: Reconfiguring mds.cephfs.smithi138.ndfkhd (monmap changed)... 2024-05-27T09:35:46.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:46 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.ndfkhd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:35:46.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:46 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:46.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:46 smithi119 ceph-mon[88269]: Reconfiguring daemon mds.cephfs.smithi138.ndfkhd on smithi138 2024-05-27T09:35:47.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:47.996+0000 7fb70c200700 1 -- 172.21.15.119:0/1285899280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb704075280 msgr2=0x7fb704075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:47.996+0000 7fb70c200700 1 --2- 172.21.15.119:0/1285899280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb704075280 0x7fb704075650 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009910 tx=0x7fb6f8009c20 comp rx=0 tx=0).stop 2024-05-27T09:35:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:47.996+0000 7fb70c200700 1 -- 172.21.15.119:0/1285899280 shutdown_connections 2024-05-27T09:35:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:47.996+0000 7fb70c200700 1 --2- 172.21.15.119:0/1285899280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb704075b90 0x7fb704110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:47.996+0000 7fb70c200700 1 --2- 172.21.15.119:0/1285899280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb704075280 0x7fb704075650 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:47.996+0000 7fb70c200700 1 -- 172.21.15.119:0/1285899280 >> 172.21.15.119:0/1285899280 conn(0x7fb7040fde90 msgr2=0x7fb7041002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:47.997+0000 7fb70c200700 1 -- 172.21.15.119:0/1285899280 shutdown_connections 2024-05-27T09:35:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:47.997+0000 7fb70c200700 1 -- 172.21.15.119:0/1285899280 wait complete. 2024-05-27T09:35:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:47.998+0000 7fb70c200700 1 Processor -- start 2024-05-27T09:35:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:47.998+0000 7fb70c200700 1 -- start start 2024-05-27T09:35:48.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:47.998+0000 7fb70c200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb704075280 0x7fb70419c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:48.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:47.998+0000 7fb70c200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb704075b90 0x7fb70419c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:48.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:47.998+0000 7fb70c200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70419cf10 con 0x7fb704075280 2024-05-27T09:35:48.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:47.998+0000 7fb70c200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7041a0ca0 con 0x7fb704075b90 2024-05-27T09:35:48.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:47.999+0000 7fb709800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb704075280 0x7fb70419c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:48.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:47.999+0000 7fb709800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb704075280 0x7fb70419c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46740/0 (socket says 172.21.15.119:46740) 2024-05-27T09:35:48.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:47.999+0000 7fb709800700 1 -- 172.21.15.119:0/2092321993 learned_addr learned my addr 172.21.15.119:0/2092321993 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:35:48.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:47.999+0000 7fb708e00700 1 --2- 172.21.15.119:0/2092321993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb704075b90 0x7fb70419c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:48.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:47.999+0000 7fb709800700 1 -- 172.21.15.119:0/2092321993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb704075b90 msgr2=0x7fb70419c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:48.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:47.999+0000 7fb709800700 1 --2- 172.21.15.119:0/2092321993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb704075b90 0x7fb70419c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:48.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:47.999+0000 7fb709800700 1 -- 172.21.15.119:0/2092321993 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f80095f0 con 0x7fb704075280 2024-05-27T09:35:48.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.000+0000 7fb709800700 1 --2- 172.21.15.119:0/2092321993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb704075280 0x7fb70419c2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb6f80092e0 tx=0x7fb6f800b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:48.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.000+0000 7fb702000700 1 -- 172.21.15.119:0/2092321993 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6f801d070 con 0x7fb704075280 2024-05-27T09:35:48.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.000+0000 7fb702000700 1 -- 172.21.15.119:0/2092321993 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6f80052a0 con 0x7fb704075280 2024-05-27T09:35:48.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.000+0000 7fb70c200700 1 -- 172.21.15.119:0/2092321993 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7041a0f20 con 0x7fb704075280 2024-05-27T09:35:48.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.000+0000 7fb702000700 1 -- 172.21.15.119:0/2092321993 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6f8017400 con 0x7fb704075280 2024-05-27T09:35:48.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.000+0000 7fb70c200700 1 -- 172.21.15.119:0/2092321993 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7041a1410 con 0x7fb704075280 2024-05-27T09:35:48.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.001+0000 7fb70c200700 1 -- 172.21.15.119:0/2092321993 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb70410e260 con 0x7fb704075280 2024-05-27T09:35:48.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.003+0000 7fb702000700 1 -- 172.21.15.119:0/2092321993 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 38) v1 ==== 100184+0+0 (secure 0 0 0) 0x7fb6f800f820 con 0x7fb704075280 2024-05-27T09:35:48.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.004+0000 7fb702000700 1 --2- 172.21.15.119:0/2092321993 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb6ec078b60 0x7fb6ec07b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:48.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.004+0000 7fb702000700 1 -- 172.21.15.119:0/2092321993 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb6f809c3a0 con 0x7fb704075280 2024-05-27T09:35:48.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.004+0000 7fb708e00700 1 --2- 172.21.15.119:0/2092321993 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb6ec078b60 0x7fb6ec07b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:48.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.005+0000 7fb708e00700 1 --2- 172.21.15.119:0/2092321993 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb6ec078b60 0x7fb6ec07b010 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb70419d910 tx=0x7fb6f4009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:48.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.009+0000 7fb702000700 1 -- 172.21.15.119:0/2092321993 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fb6f8063d60 con 0x7fb704075280 2024-05-27T09:35:48.268 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.267+0000 7fb70c200700 1 -- 172.21.15.119:0/2092321993 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb70402cd40 con 0x7fb6ec078b60 2024-05-27T09:35:48.269 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.269+0000 7fb702000700 1 -- 172.21.15.119:0/2092321993 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fb70402cd40 con 0x7fb6ec078b60 2024-05-27T09:35:48.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.271+0000 7fb70c200700 1 -- 172.21.15.119:0/2092321993 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb6ec078b60 msgr2=0x7fb6ec07b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:48.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.271+0000 7fb70c200700 1 --2- 172.21.15.119:0/2092321993 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb6ec078b60 0x7fb6ec07b010 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb70419d910 tx=0x7fb6f4009260 comp rx=0 tx=0).stop 2024-05-27T09:35:48.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.271+0000 7fb70c200700 1 -- 172.21.15.119:0/2092321993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb704075280 msgr2=0x7fb70419c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:48.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.271+0000 7fb70c200700 1 --2- 172.21.15.119:0/2092321993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb704075280 0x7fb70419c2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb6f80092e0 tx=0x7fb6f800b680 comp rx=0 tx=0).stop 2024-05-27T09:35:48.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.272+0000 7fb70c200700 1 -- 172.21.15.119:0/2092321993 shutdown_connections 2024-05-27T09:35:48.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.272+0000 7fb70c200700 1 --2- 172.21.15.119:0/2092321993 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb6ec078b60 0x7fb6ec07b010 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:48.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.272+0000 7fb70c200700 1 --2- 172.21.15.119:0/2092321993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb704075280 0x7fb70419c2f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:48.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.272+0000 7fb70c200700 1 --2- 172.21.15.119:0/2092321993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb704075b90 0x7fb70419c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:48.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.272+0000 7fb70c200700 1 -- 172.21.15.119:0/2092321993 >> 172.21.15.119:0/2092321993 conn(0x7fb7040fde90 msgr2=0x7fb7040feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:48.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.272+0000 7fb70c200700 1 -- 172.21.15.119:0/2092321993 shutdown_connections 2024-05-27T09:35:48.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.272+0000 7fb70c200700 1 -- 172.21.15.119:0/2092321993 wait complete. 2024-05-27T09:35:48.288 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:35:48.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.430+0000 7f8bcd000700 1 -- 172.21.15.119:0/1842433710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bc810b610 msgr2=0x7f8bc810b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:48.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.430+0000 7f8bcd000700 1 --2- 172.21.15.119:0/1842433710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bc810b610 0x7f8bc810b9e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009910 tx=0x7f8bb8009c20 comp rx=0 tx=0).stop 2024-05-27T09:35:48.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.430+0000 7f8bcd000700 1 -- 172.21.15.119:0/1842433710 shutdown_connections 2024-05-27T09:35:48.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.430+0000 7f8bcd000700 1 --2- 172.21.15.119:0/1842433710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bc8106b80 0x7f8bc8106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:48.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.430+0000 7f8bcd000700 1 --2- 172.21.15.119:0/1842433710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bc810b610 0x7f8bc810b9e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:48.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.430+0000 7f8bcd000700 1 -- 172.21.15.119:0/1842433710 >> 172.21.15.119:0/1842433710 conn(0x7f8bc8075df0 msgr2=0x7f8bc8078200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:48.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.431+0000 7f8bcd000700 1 -- 172.21.15.119:0/1842433710 shutdown_connections 2024-05-27T09:35:48.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.431+0000 7f8bcd000700 1 -- 172.21.15.119:0/1842433710 wait complete. 2024-05-27T09:35:48.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.432+0000 7f8bcd000700 1 Processor -- start 2024-05-27T09:35:48.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.432+0000 7f8bcd000700 1 -- start start 2024-05-27T09:35:48.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.432+0000 7f8bcd000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bc8106b80 0x7f8bc819c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:48.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.432+0000 7f8bcd000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bc810b610 0x7f8bc819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:48.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.432+0000 7f8bcd000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc819cd40 con 0x7f8bc8106b80 2024-05-27T09:35:48.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.432+0000 7f8bcd000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81a0ad0 con 0x7f8bc810b610 2024-05-27T09:35:48.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.432+0000 7f8bc7400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bc8106b80 0x7f8bc819c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:48.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.432+0000 7f8bc7400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bc8106b80 0x7f8bc819c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46764/0 (socket says 172.21.15.119:46764) 2024-05-27T09:35:48.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.432+0000 7f8bc7400700 1 -- 172.21.15.119:0/1559543995 learned_addr learned my addr 172.21.15.119:0/1559543995 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:35:48.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.433+0000 7f8bbfe00700 1 --2- 172.21.15.119:0/1559543995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bc810b610 0x7f8bc819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:48.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.433+0000 7f8bc7400700 1 -- 172.21.15.119:0/1559543995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bc810b610 msgr2=0x7f8bc819c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:48.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.433+0000 7f8bc7400700 1 --2- 172.21.15.119:0/1559543995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bc810b610 0x7f8bc819c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:48.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.433+0000 7f8bc7400700 1 -- 172.21.15.119:0/1559543995 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb80095f0 con 0x7f8bc8106b80 2024-05-27T09:35:48.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.433+0000 7f8bc7400700 1 --2- 172.21.15.119:0/1559543995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bc8106b80 0x7f8bc819c120 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8bb800b5c0 tx=0x7f8bb800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:48.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.434+0000 7f8bc4c00700 1 -- 172.21.15.119:0/1559543995 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bb801d070 con 0x7f8bc8106b80 2024-05-27T09:35:48.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.434+0000 7f8bc4c00700 1 -- 172.21.15.119:0/1559543995 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8bb8005e00 con 0x7f8bc8106b80 2024-05-27T09:35:48.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.434+0000 7f8bcd000700 1 -- 172.21.15.119:0/1559543995 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc81a0d50 con 0x7f8bc8106b80 2024-05-27T09:35:48.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.434+0000 7f8bc4c00700 1 -- 172.21.15.119:0/1559543995 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bb8017400 con 0x7f8bc8106b80 2024-05-27T09:35:48.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.434+0000 7f8bcd000700 1 -- 172.21.15.119:0/1559543995 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc81a12c0 con 0x7f8bc8106b80 2024-05-27T09:35:48.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.435+0000 7f8bcd000700 1 -- 172.21.15.119:0/1559543995 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc804f2a0 con 0x7f8bc8106b80 2024-05-27T09:35:48.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.436+0000 7f8bc4c00700 1 -- 172.21.15.119:0/1559543995 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 38) v1 ==== 100184+0+0 (secure 0 0 0) 0x7f8bb800f6f0 con 0x7f8bc8106b80 2024-05-27T09:35:48.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.437+0000 7f8bc4c00700 1 --2- 172.21.15.119:0/1559543995 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8bac078b60 0x7f8bac07b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:48.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.437+0000 7f8bc4c00700 1 -- 172.21.15.119:0/1559543995 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8bb809c480 con 0x7f8bc8106b80 2024-05-27T09:35:48.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.437+0000 7f8bbfe00700 1 --2- 172.21.15.119:0/1559543995 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8bac078b60 0x7f8bac07b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:48.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.438+0000 7f8bbfe00700 1 --2- 172.21.15.119:0/1559543995 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8bac078b60 0x7f8bac07b010 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8bc819d740 tx=0x7f8bb000b3c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:48.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.442+0000 7f8bc4c00700 1 -- 172.21.15.119:0/1559543995 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f8bb8063e40 con 0x7f8bc8106b80 2024-05-27T09:35:48.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.681+0000 7f8bcd000700 1 -- 172.21.15.119:0/1559543995 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8bc81a1800 con 0x7f8bac078b60 2024-05-27T09:35:48.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.684+0000 7f8bc4c00700 1 -- 172.21.15.119:0/1559543995 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f8bc81a1800 con 0x7f8bac078b60 2024-05-27T09:35:48.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.686+0000 7f8bcd000700 1 -- 172.21.15.119:0/1559543995 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8bac078b60 msgr2=0x7f8bac07b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:48.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.686+0000 7f8bcd000700 1 --2- 172.21.15.119:0/1559543995 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8bac078b60 0x7f8bac07b010 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8bc819d740 tx=0x7f8bb000b3c0 comp rx=0 tx=0).stop 2024-05-27T09:35:48.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.686+0000 7f8bcd000700 1 -- 172.21.15.119:0/1559543995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bc8106b80 msgr2=0x7f8bc819c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:48.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.686+0000 7f8bcd000700 1 --2- 172.21.15.119:0/1559543995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bc8106b80 0x7f8bc819c120 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8bb800b5c0 tx=0x7f8bb800bd70 comp rx=0 tx=0).stop 2024-05-27T09:35:48.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.687+0000 7f8bcd000700 1 -- 172.21.15.119:0/1559543995 shutdown_connections 2024-05-27T09:35:48.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.687+0000 7f8bcd000700 1 --2- 172.21.15.119:0/1559543995 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8bac078b60 0x7f8bac07b010 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:48.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.687+0000 7f8bcd000700 1 --2- 172.21.15.119:0/1559543995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bc8106b80 0x7f8bc819c120 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:48.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.687+0000 7f8bcd000700 1 --2- 172.21.15.119:0/1559543995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bc810b610 0x7f8bc819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:48.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.687+0000 7f8bcd000700 1 -- 172.21.15.119:0/1559543995 >> 172.21.15.119:0/1559543995 conn(0x7f8bc8075df0 msgr2=0x7f8bc8077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:48.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.687+0000 7f8bcd000700 1 -- 172.21.15.119:0/1559543995 shutdown_connections 2024-05-27T09:35:48.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.687+0000 7f8bcd000700 1 -- 172.21.15.119:0/1559543995 wait complete. 2024-05-27T09:35:48.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.841+0000 7f5c45000700 1 -- 172.21.15.119:0/2209544445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c400ffbb0 msgr2=0x7f5c400fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:48.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.841+0000 7f5c45000700 1 --2- 172.21.15.119:0/2209544445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c400ffbb0 0x7f5c400fff80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009940 tx=0x7f5c30009c50 comp rx=0 tx=0).stop 2024-05-27T09:35:48.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.841+0000 7f5c45000700 1 -- 172.21.15.119:0/2209544445 shutdown_connections 2024-05-27T09:35:48.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.841+0000 7f5c45000700 1 --2- 172.21.15.119:0/2209544445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c401004c0 0x7f5c4010c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:48.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.841+0000 7f5c45000700 1 --2- 172.21.15.119:0/2209544445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c400ffbb0 0x7f5c400fff80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:48.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.841+0000 7f5c45000700 1 -- 172.21.15.119:0/2209544445 >> 172.21.15.119:0/2209544445 conn(0x7f5c40074f50 msgr2=0x7f5c40075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:48.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.841+0000 7f5c45000700 1 -- 172.21.15.119:0/2209544445 shutdown_connections 2024-05-27T09:35:48.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.841+0000 7f5c45000700 1 -- 172.21.15.119:0/2209544445 wait complete. 2024-05-27T09:35:48.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.842+0000 7f5c45000700 1 Processor -- start 2024-05-27T09:35:48.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.842+0000 7f5c45000700 1 -- start start 2024-05-27T09:35:48.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.843+0000 7f5c45000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c400ffbb0 0x7f5c40197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:48.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.843+0000 7f5c45000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c401004c0 0x7f5c40198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:48.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.843+0000 7f5c45000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c40198a40 con 0x7f5c400ffbb0 2024-05-27T09:35:48.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.843+0000 7f5c45000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4019c7d0 con 0x7f5c401004c0 2024-05-27T09:35:48.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.843+0000 7f5c3de00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c400ffbb0 0x7f5c40197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:48.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.843+0000 7f5c3de00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c400ffbb0 0x7f5c40197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52692/0 (socket says 172.21.15.119:52692) 2024-05-27T09:35:48.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.843+0000 7f5c3de00700 1 -- 172.21.15.119:0/2102888197 learned_addr learned my addr 172.21.15.119:0/2102888197 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:35:48.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.844+0000 7f5c3d400700 1 --2- 172.21.15.119:0/2102888197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c401004c0 0x7f5c40198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:48.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.844+0000 7f5c3de00700 1 -- 172.21.15.119:0/2102888197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c401004c0 msgr2=0x7f5c40198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:48.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.844+0000 7f5c3de00700 1 --2- 172.21.15.119:0/2102888197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c401004c0 0x7f5c40198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:48.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.844+0000 7f5c3de00700 1 -- 172.21.15.119:0/2102888197 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c300095f0 con 0x7f5c400ffbb0 2024-05-27T09:35:48.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.844+0000 7f5c3d400700 1 --2- 172.21.15.119:0/2102888197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c401004c0 0x7f5c40198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:35:48.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.844+0000 7f5c3de00700 1 --2- 172.21.15.119:0/2102888197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c400ffbb0 0x7f5c40197e20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5c3000b580 tx=0x7f5c300056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:48.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.844+0000 7f5c36a00700 1 -- 172.21.15.119:0/2102888197 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c3001d070 con 0x7f5c400ffbb0 2024-05-27T09:35:48.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.844+0000 7f5c36a00700 1 -- 172.21.15.119:0/2102888197 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c30005dc0 con 0x7f5c400ffbb0 2024-05-27T09:35:48.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.844+0000 7f5c36a00700 1 -- 172.21.15.119:0/2102888197 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c30021410 con 0x7f5c400ffbb0 2024-05-27T09:35:48.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.844+0000 7f5c45000700 1 -- 172.21.15.119:0/2102888197 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4019cab0 con 0x7f5c400ffbb0 2024-05-27T09:35:48.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.845+0000 7f5c45000700 1 -- 172.21.15.119:0/2102888197 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4019d000 con 0x7f5c400ffbb0 2024-05-27T09:35:48.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.846+0000 7f5c45000700 1 -- 172.21.15.119:0/2102888197 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c40109f10 con 0x7f5c400ffbb0 2024-05-27T09:35:48.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.847+0000 7f5c36a00700 1 -- 172.21.15.119:0/2102888197 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 38) v1 ==== 100184+0+0 (secure 0 0 0) 0x7f5c3000f7b0 con 0x7f5c400ffbb0 2024-05-27T09:35:48.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.848+0000 7f5c36a00700 1 --2- 172.21.15.119:0/2102888197 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5c24078b70 0x7f5c2407b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:48.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.848+0000 7f5c36a00700 1 -- 172.21.15.119:0/2102888197 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f5c3009c290 con 0x7f5c400ffbb0 2024-05-27T09:35:48.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.848+0000 7f5c3d400700 1 --2- 172.21.15.119:0/2102888197 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5c24078b70 0x7f5c2407b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:48.849 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.849+0000 7f5c3d400700 1 --2- 172.21.15.119:0/2102888197 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5c24078b70 0x7f5c2407b020 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c005fd0 tx=0x7f5c2c005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:48.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:48.853+0000 7f5c36a00700 1 -- 172.21.15.119:0/2102888197 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f5c30063c50 con 0x7f5c400ffbb0 2024-05-27T09:35:48.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:48 smithi119 ceph-mon[88269]: pgmap v43: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:48.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:48 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:48.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:48 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:48.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:48 smithi119 ceph-mon[88269]: Reconfiguring mds.cephfs.smithi138.alszpk (monmap changed)... 2024-05-27T09:35:48.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:48 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.alszpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:35:48.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:48 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:48.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:48 smithi119 ceph-mon[88269]: Reconfiguring daemon mds.cephfs.smithi138.alszpk on smithi138 2024-05-27T09:35:49.050 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:48 smithi138 ceph-mon[70004]: pgmap v43: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:49.050 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:48 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:49.050 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:48 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:49.050 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:48 smithi138 ceph-mon[70004]: Reconfiguring mds.cephfs.smithi138.alszpk (monmap changed)... 2024-05-27T09:35:49.050 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:48 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.alszpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:35:49.050 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:48 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:49.050 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:48 smithi138 ceph-mon[70004]: Reconfiguring daemon mds.cephfs.smithi138.alszpk on smithi138 2024-05-27T09:35:49.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.100+0000 7f5c45000700 1 -- 172.21.15.119:0/2102888197 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c4019d360 con 0x7f5c24078b70 2024-05-27T09:35:49.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.131+0000 7f5c36a00700 1 -- 172.21.15.119:0/2102888197 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5c4019d360 con 0x7f5c24078b70 2024-05-27T09:35:49.133 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:35:49.133 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (3m) 74s ago 16m 13.9M - 0.25.0 c8568f914cd2 6b90564dcef7 2024-05-27T09:35:49.133 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (17m) 74s ago 17m 8190k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:35:49.133 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (15m) 49s ago 15m 8086k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:35:49.133 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (17m) 74s ago 17m 6665k - 18.2.0 dc2bc1663786 e5fafde44183 2024-05-27T09:35:49.133 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (15m) 49s ago 15m 6660k - 18.2.0 dc2bc1663786 46956ab99bab 2024-05-27T09:35:49.133 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (2m) 74s ago 16m 74.7M - 9.4.12 2bacad6d85d8 b3abdab08db4 2024-05-27T09:35:49.133 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (9m) 74s ago 9m 16.6M - 18.2.0 dc2bc1663786 93b21713fb1e 2024-05-27T09:35:49.133 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (9m) 74s ago 9m 130M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:35:49.133 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (9m) 49s ago 9m 16.5M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:35:49.133 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (9m) 49s ago 9m 128M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:35:49.134 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (5m) 74s ago 18m 558M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:35:49.134 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (4m) 49s ago 15m 531M - 19.0.0-3868-ga6507388 850a06a84d28 2fc3a80696fa 2024-05-27T09:35:49.134 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (92s) 74s ago 18m 96.6M 2048M 19.0.0-3868-ga6507388 850a06a84d28 3e5c7a9ad372 2024-05-27T09:35:49.134 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (55s) 49s ago 15m 86.9M 2048M 19.0.0-3868-ga6507388 850a06a84d28 e4d471274cf1 2024-05-27T09:35:49.134 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (4m) 74s ago 17m 7482k - 1.5.0 0da6a335fe13 709c37815cff 2024-05-27T09:35:49.134 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (4m) 49s ago 15m 7814k - 1.5.0 0da6a335fe13 999047a1a5bf 2024-05-27T09:35:49.134 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (14m) 74s ago 14m 225M 1216M 18.2.0 dc2bc1663786 825bd1392508 2024-05-27T09:35:49.134 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (13m) 74s ago 13m 253M 1216M 18.2.0 dc2bc1663786 fce9819903d9 2024-05-27T09:35:49.134 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (13m) 74s ago 13m 190M 1216M 18.2.0 dc2bc1663786 3955a6112c1c 2024-05-27T09:35:49.134 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (12m) 49s ago 12m 271M 2240M 18.2.0 dc2bc1663786 693c92220622 2024-05-27T09:35:49.134 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (12m) 49s ago 12m 247M 2240M 18.2.0 dc2bc1663786 7559e658e917 2024-05-27T09:35:49.134 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (11m) 49s ago 11m 232M 2240M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:35:49.134 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (3m) 74s ago 16m 62.3M - 2.43.0 a07b618ecd1d cba50454ed21 2024-05-27T09:35:49.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.135+0000 7f5c45000700 1 -- 172.21.15.119:0/2102888197 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5c24078b70 msgr2=0x7f5c2407b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:49.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.135+0000 7f5c45000700 1 --2- 172.21.15.119:0/2102888197 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5c24078b70 0x7f5c2407b020 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c005fd0 tx=0x7f5c2c005ee0 comp rx=0 tx=0).stop 2024-05-27T09:35:49.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.135+0000 7f5c45000700 1 -- 172.21.15.119:0/2102888197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c400ffbb0 msgr2=0x7f5c40197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:49.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.135+0000 7f5c45000700 1 --2- 172.21.15.119:0/2102888197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c400ffbb0 0x7f5c40197e20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5c3000b580 tx=0x7f5c300056a0 comp rx=0 tx=0).stop 2024-05-27T09:35:49.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.136+0000 7f5c45000700 1 -- 172.21.15.119:0/2102888197 shutdown_connections 2024-05-27T09:35:49.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.136+0000 7f5c45000700 1 --2- 172.21.15.119:0/2102888197 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5c24078b70 0x7f5c2407b020 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:49.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.136+0000 7f5c45000700 1 --2- 172.21.15.119:0/2102888197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c400ffbb0 0x7f5c40197e20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:49.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.136+0000 7f5c45000700 1 --2- 172.21.15.119:0/2102888197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c401004c0 0x7f5c40198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:49.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.136+0000 7f5c45000700 1 -- 172.21.15.119:0/2102888197 >> 172.21.15.119:0/2102888197 conn(0x7f5c40074f50 msgr2=0x7f5c400fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:49.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.136+0000 7f5c45000700 1 -- 172.21.15.119:0/2102888197 shutdown_connections 2024-05-27T09:35:49.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.136+0000 7f5c45000700 1 -- 172.21.15.119:0/2102888197 wait complete. 2024-05-27T09:35:49.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.290+0000 7f0923a00700 1 -- 172.21.15.119:0/1202041170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091c1003b0 msgr2=0x7f091c100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:49.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.290+0000 7f0923a00700 1 --2- 172.21.15.119:0/1202041170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091c1003b0 0x7f091c100780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0904009910 tx=0x7f0904009c20 comp rx=0 tx=0).stop 2024-05-27T09:35:49.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.290+0000 7f0923a00700 1 -- 172.21.15.119:0/1202041170 shutdown_connections 2024-05-27T09:35:49.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.290+0000 7f0923a00700 1 --2- 172.21.15.119:0/1202041170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f091c100d50 0x7f091c104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:49.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.290+0000 7f0923a00700 1 --2- 172.21.15.119:0/1202041170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091c1003b0 0x7f091c100780 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:49.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.290+0000 7f0923a00700 1 -- 172.21.15.119:0/1202041170 >> 172.21.15.119:0/1202041170 conn(0x7f091c0fbc30 msgr2=0x7f091c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:49.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.291+0000 7f0923a00700 1 -- 172.21.15.119:0/1202041170 shutdown_connections 2024-05-27T09:35:49.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.291+0000 7f0923a00700 1 -- 172.21.15.119:0/1202041170 wait complete. 2024-05-27T09:35:49.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.292+0000 7f0923a00700 1 Processor -- start 2024-05-27T09:35:49.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.292+0000 7f0923a00700 1 -- start start 2024-05-27T09:35:49.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.292+0000 7f0923a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091c1003b0 0x7f091c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:49.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.292+0000 7f0923a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f091c100d50 0x7f091c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:49.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.292+0000 7f0923a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c073ad0 con 0x7f091c1003b0 2024-05-27T09:35:49.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.292+0000 7f0923a00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c073c10 con 0x7f091c100d50 2024-05-27T09:35:49.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.292+0000 7f0921000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091c1003b0 0x7f091c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:49.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.293+0000 7f0921000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091c1003b0 0x7f091c073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52712/0 (socket says 172.21.15.119:52712) 2024-05-27T09:35:49.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.293+0000 7f0921000700 1 -- 172.21.15.119:0/82321778 learned_addr learned my addr 172.21.15.119:0/82321778 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:35:49.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.293+0000 7f0913e00700 1 --2- 172.21.15.119:0/82321778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f091c100d50 0x7f091c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:49.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.293+0000 7f0921000700 1 -- 172.21.15.119:0/82321778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f091c100d50 msgr2=0x7f091c073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:49.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.293+0000 7f0921000700 1 --2- 172.21.15.119:0/82321778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f091c100d50 0x7f091c073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:49.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.293+0000 7f0921000700 1 -- 172.21.15.119:0/82321778 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09040095f0 con 0x7f091c1003b0 2024-05-27T09:35:49.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.293+0000 7f0921000700 1 --2- 172.21.15.119:0/82321778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091c1003b0 0x7f091c073050 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f090400f690 tx=0x7f090400f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:49.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.294+0000 7f091a000700 1 -- 172.21.15.119:0/82321778 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f090401d070 con 0x7f091c1003b0 2024-05-27T09:35:49.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.294+0000 7f091a000700 1 -- 172.21.15.119:0/82321778 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f09040050d0 con 0x7f091c1003b0 2024-05-27T09:35:49.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.294+0000 7f0923a00700 1 -- 172.21.15.119:0/82321778 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091c073e90 con 0x7f091c1003b0 2024-05-27T09:35:49.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.294+0000 7f091a000700 1 -- 172.21.15.119:0/82321778 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0904022470 con 0x7f091c1003b0 2024-05-27T09:35:49.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.294+0000 7f0923a00700 1 -- 172.21.15.119:0/82321778 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091c1a2210 con 0x7f091c1003b0 2024-05-27T09:35:49.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.295+0000 7f0923a00700 1 -- 172.21.15.119:0/82321778 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091c1086c0 con 0x7f091c1003b0 2024-05-27T09:35:49.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.296+0000 7f091a000700 1 -- 172.21.15.119:0/82321778 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 38) v1 ==== 100184+0+0 (secure 0 0 0) 0x7f0904003680 con 0x7f091c1003b0 2024-05-27T09:35:49.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.297+0000 7f091a000700 1 --2- 172.21.15.119:0/82321778 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f08fc078b60 0x7f08fc07b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:49.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.297+0000 7f091a000700 1 -- 172.21.15.119:0/82321778 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f090409c3b0 con 0x7f091c1003b0 2024-05-27T09:35:49.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.298+0000 7f0913e00700 1 --2- 172.21.15.119:0/82321778 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f08fc078b60 0x7f08fc07b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:49.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.298+0000 7f0913e00700 1 --2- 172.21.15.119:0/82321778 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f08fc078b60 0x7f08fc07b010 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f091c074840 tx=0x7f090c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:49.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.302+0000 7f091a000700 1 -- 172.21.15.119:0/82321778 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f0904063ec0 con 0x7f091c1003b0 2024-05-27T09:35:49.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.624+0000 7f0923a00700 1 -- 172.21.15.119:0/82321778 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f091c04ea50 con 0x7f091c1003b0 2024-05-27T09:35:49.626 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.626+0000 7f091a000700 1 -- 172.21.15.119:0/82321778 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f0904063860 con 0x7f091c1003b0 2024-05-27T09:35:49.627 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:35:49.627 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:35:49.627 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:35:49.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:35:49.627 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:35:49.627 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:35:49.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:35:49.627 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:35:49.627 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-05-27T09:35:49.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:35:49.627 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:35:49.627 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-05-27T09:35:49.627 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:35:49.627 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:35:49.628 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 10, 2024-05-27T09:35:49.628 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 4 2024-05-27T09:35:49.628 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:35:49.628 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:35:49.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.628+0000 7f0923a00700 1 -- 172.21.15.119:0/82321778 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f08fc078b60 msgr2=0x7f08fc07b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:49.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.628+0000 7f0923a00700 1 --2- 172.21.15.119:0/82321778 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f08fc078b60 0x7f08fc07b010 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f091c074840 tx=0x7f090c009260 comp rx=0 tx=0).stop 2024-05-27T09:35:49.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.629+0000 7f0923a00700 1 -- 172.21.15.119:0/82321778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091c1003b0 msgr2=0x7f091c073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:49.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.629+0000 7f0923a00700 1 --2- 172.21.15.119:0/82321778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091c1003b0 0x7f091c073050 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f090400f690 tx=0x7f090400f770 comp rx=0 tx=0).stop 2024-05-27T09:35:49.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.629+0000 7f0923a00700 1 -- 172.21.15.119:0/82321778 shutdown_connections 2024-05-27T09:35:49.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.629+0000 7f0923a00700 1 --2- 172.21.15.119:0/82321778 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f08fc078b60 0x7f08fc07b010 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:49.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.629+0000 7f0923a00700 1 --2- 172.21.15.119:0/82321778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091c1003b0 0x7f091c073050 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:49.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.629+0000 7f0923a00700 1 --2- 172.21.15.119:0/82321778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f091c100d50 0x7f091c073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:49.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.629+0000 7f0923a00700 1 -- 172.21.15.119:0/82321778 >> 172.21.15.119:0/82321778 conn(0x7f091c0fbc30 msgr2=0x7f091c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:49.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.629+0000 7f0923a00700 1 -- 172.21.15.119:0/82321778 shutdown_connections 2024-05-27T09:35:49.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.629+0000 7f0923a00700 1 -- 172.21.15.119:0/82321778 wait complete. 2024-05-27T09:35:49.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.784+0000 7fefa3200700 1 -- 172.21.15.119:0/274240458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef9c1003d0 msgr2=0x7fef9c1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:49.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.784+0000 7fefa3200700 1 --2- 172.21.15.119:0/274240458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef9c1003d0 0x7fef9c1007a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009910 tx=0x7fef8c009c20 comp rx=0 tx=0).stop 2024-05-27T09:35:49.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.784+0000 7fefa3200700 1 -- 172.21.15.119:0/274240458 shutdown_connections 2024-05-27T09:35:49.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.784+0000 7fefa3200700 1 --2- 172.21.15.119:0/274240458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef9c100d70 0x7fef9c104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:49.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.784+0000 7fefa3200700 1 --2- 172.21.15.119:0/274240458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef9c1003d0 0x7fef9c1007a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:49.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.784+0000 7fefa3200700 1 -- 172.21.15.119:0/274240458 >> 172.21.15.119:0/274240458 conn(0x7fef9c0fbc90 msgr2=0x7fef9c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:49.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.784+0000 7fefa3200700 1 -- 172.21.15.119:0/274240458 shutdown_connections 2024-05-27T09:35:49.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.784+0000 7fefa3200700 1 -- 172.21.15.119:0/274240458 wait complete. 2024-05-27T09:35:49.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.785+0000 7fefa3200700 1 Processor -- start 2024-05-27T09:35:49.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.785+0000 7fefa3200700 1 -- start start 2024-05-27T09:35:49.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.785+0000 7fefa3200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef9c1003d0 0x7fef9c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:49.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.785+0000 7fefa3200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef9c100d70 0x7fef9c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:49.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.785+0000 7fefa3200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c198b40 con 0x7fef9c1003d0 2024-05-27T09:35:49.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.785+0000 7fefa3200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c19c8d0 con 0x7fef9c100d70 2024-05-27T09:35:49.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.786+0000 7fef9be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef9c1003d0 0x7fef9c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:49.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.786+0000 7fef9be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef9c1003d0 0x7fef9c197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52728/0 (socket says 172.21.15.119:52728) 2024-05-27T09:35:49.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.786+0000 7fef9be00700 1 -- 172.21.15.119:0/278686584 learned_addr learned my addr 172.21.15.119:0/278686584 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:35:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.786+0000 7fef9b400700 1 --2- 172.21.15.119:0/278686584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef9c100d70 0x7fef9c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.786+0000 7fef9be00700 1 -- 172.21.15.119:0/278686584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef9c100d70 msgr2=0x7fef9c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.786+0000 7fef9be00700 1 --2- 172.21.15.119:0/278686584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef9c100d70 0x7fef9c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.786+0000 7fef9be00700 1 -- 172.21.15.119:0/278686584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c0095f0 con 0x7fef9c1003d0 2024-05-27T09:35:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.786+0000 7fef9b400700 1 --2- 172.21.15.119:0/278686584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef9c100d70 0x7fef9c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:35:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.787+0000 7fef9be00700 1 --2- 172.21.15.119:0/278686584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef9c1003d0 0x7fef9c197f20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0092e0 tx=0x7fef8c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.787+0000 7fef98c00700 1 -- 172.21.15.119:0/278686584 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef8c01d070 con 0x7fef9c1003d0 2024-05-27T09:35:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.787+0000 7fef98c00700 1 -- 172.21.15.119:0/278686584 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef8c005e00 con 0x7fef9c1003d0 2024-05-27T09:35:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.787+0000 7fefa3200700 1 -- 172.21.15.119:0/278686584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9c19cb50 con 0x7fef9c1003d0 2024-05-27T09:35:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.787+0000 7fef98c00700 1 -- 172.21.15.119:0/278686584 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef8c017400 con 0x7fef9c1003d0 2024-05-27T09:35:49.788 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.787+0000 7fefa3200700 1 -- 172.21.15.119:0/278686584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9c19d040 con 0x7fef9c1003d0 2024-05-27T09:35:49.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.788+0000 7fefa3200700 1 -- 172.21.15.119:0/278686584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9c1044c0 con 0x7fef9c1003d0 2024-05-27T09:35:49.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.790+0000 7fef98c00700 1 -- 172.21.15.119:0/278686584 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 38) v1 ==== 100184+0+0 (secure 0 0 0) 0x7fef8c00f460 con 0x7fef9c1003d0 2024-05-27T09:35:49.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.790+0000 7fef98c00700 1 --2- 172.21.15.119:0/278686584 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fef80078b60 0x7fef8007b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:49.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.791+0000 7fef98c00700 1 -- 172.21.15.119:0/278686584 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fef8c09c4a0 con 0x7fef9c1003d0 2024-05-27T09:35:49.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.791+0000 7fef9b400700 1 --2- 172.21.15.119:0/278686584 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fef80078b60 0x7fef8007b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:49.792 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.791+0000 7fef9b400700 1 --2- 172.21.15.119:0/278686584 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fef80078b60 0x7fef8007b010 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fef88005fd0 tx=0x7fef88005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:49.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:49.796+0000 7fef98c00700 1 -- 172.21.15.119:0/278686584 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fef8c063f00 con 0x7fef9c1003d0 2024-05-27T09:35:50.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.081+0000 7fefa3200700 1 -- 172.21.15.119:0/278686584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fef9c19d320 con 0x7fef9c1003d0 2024-05-27T09:35:50.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.082+0000 7fef98c00700 1 -- 172.21.15.119:0/278686584 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1889 (secure 0 0 0) 0x7fef8c0638a0 con 0x7fef9c1003d0 2024-05-27T09:35:50.083 INFO:teuthology.orchestra.run.smithi119.stdout:e22 2024-05-27T09:35:50.083 INFO:teuthology.orchestra.run.smithi119.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T09:35:50.083 INFO:teuthology.orchestra.run.smithi119.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:35:50.083 INFO:teuthology.orchestra.run.smithi119.stdout:legacy client fscid: 1 2024-05-27T09:35:50.083 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:35:50.083 INFO:teuthology.orchestra.run.smithi119.stdout:Filesystem 'cephfs' (1) 2024-05-27T09:35:50.083 INFO:teuthology.orchestra.run.smithi119.stdout:fs_name cephfs 2024-05-27T09:35:50.084 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 22 2024-05-27T09:35:50.084 INFO:teuthology.orchestra.run.smithi119.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-27T09:35:50.084 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-05-27T09:25:53.387388+0000 2024-05-27T09:35:50.084 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2024-05-27T09:34:01.250304+0000 2024-05-27T09:35:50.084 INFO:teuthology.orchestra.run.smithi119.stdout:tableserver 0 2024-05-27T09:35:50.084 INFO:teuthology.orchestra.run.smithi119.stdout:root 0 2024-05-27T09:35:50.084 INFO:teuthology.orchestra.run.smithi119.stdout:session_timeout 60 2024-05-27T09:35:50.084 INFO:teuthology.orchestra.run.smithi119.stdout:session_autoclose 300 2024-05-27T09:35:50.084 INFO:teuthology.orchestra.run.smithi119.stdout:max_file_size 1099511627776 2024-05-27T09:35:50.084 INFO:teuthology.orchestra.run.smithi119.stdout:max_xattr_size 65536 2024-05-27T09:35:50.084 INFO:teuthology.orchestra.run.smithi119.stdout:required_client_features {} 2024-05-27T09:35:50.084 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure 0 2024-05-27T09:35:50.084 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure_osd_epoch 46 2024-05-27T09:35:50.085 INFO:teuthology.orchestra.run.smithi119.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:35:50.085 INFO:teuthology.orchestra.run.smithi119.stdout:max_mds 2 2024-05-27T09:35:50.085 INFO:teuthology.orchestra.run.smithi119.stdout:in 0,1 2024-05-27T09:35:50.085 INFO:teuthology.orchestra.run.smithi119.stdout:up {0=14472,1=24241} 2024-05-27T09:35:50.085 INFO:teuthology.orchestra.run.smithi119.stdout:failed 2024-05-27T09:35:50.085 INFO:teuthology.orchestra.run.smithi119.stdout:damaged 2024-05-27T09:35:50.085 INFO:teuthology.orchestra.run.smithi119.stdout:stopped 2024-05-27T09:35:50.085 INFO:teuthology.orchestra.run.smithi119.stdout:data_pools [3] 2024-05-27T09:35:50.085 INFO:teuthology.orchestra.run.smithi119.stdout:metadata_pool 2 2024-05-27T09:35:50.085 INFO:teuthology.orchestra.run.smithi119.stdout:inline_data enabled 2024-05-27T09:35:50.085 INFO:teuthology.orchestra.run.smithi119.stdout:balancer 2024-05-27T09:35:50.085 INFO:teuthology.orchestra.run.smithi119.stdout:bal_rank_mask -1 2024-05-27T09:35:50.085 INFO:teuthology.orchestra.run.smithi119.stdout:standby_count_wanted 1 2024-05-27T09:35:50.085 INFO:teuthology.orchestra.run.smithi119.stdout:qdb_cluster leader: 0 members: 2024-05-27T09:35:50.086 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.ppmzzk{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:35:50.086 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.ndfkhd{1:24241} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/897058594,v1:172.21.15.138:6825/897058594] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:35:50.086 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:35:50.086 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:35:50.086 INFO:teuthology.orchestra.run.smithi119.stdout:Standby daemons: 2024-05-27T09:35:50.086 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:35:50.086 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.nghcey{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.119:6826/413736191,v1:172.21.15.119:6827/413736191] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:35:50.086 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.alszpk{-1:24253} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/2345607053,v1:172.21.15.138:6827/2345607053] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:35:50.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.085+0000 7fefa3200700 1 -- 172.21.15.119:0/278686584 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fef80078b60 msgr2=0x7fef8007b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:50.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.085+0000 7fefa3200700 1 --2- 172.21.15.119:0/278686584 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fef80078b60 0x7fef8007b010 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fef88005fd0 tx=0x7fef88005ee0 comp rx=0 tx=0).stop 2024-05-27T09:35:50.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.085+0000 7fefa3200700 1 -- 172.21.15.119:0/278686584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef9c1003d0 msgr2=0x7fef9c197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:50.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.085+0000 7fefa3200700 1 --2- 172.21.15.119:0/278686584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef9c1003d0 0x7fef9c197f20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0092e0 tx=0x7fef8c00be20 comp rx=0 tx=0).stop 2024-05-27T09:35:50.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.085+0000 7fefa3200700 1 -- 172.21.15.119:0/278686584 shutdown_connections 2024-05-27T09:35:50.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.085+0000 7fefa3200700 1 --2- 172.21.15.119:0/278686584 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fef80078b60 0x7fef8007b010 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:50.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.085+0000 7fefa3200700 1 --2- 172.21.15.119:0/278686584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef9c1003d0 0x7fef9c197f20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:50.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.085+0000 7fefa3200700 1 --2- 172.21.15.119:0/278686584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef9c100d70 0x7fef9c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:50.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.085+0000 7fefa3200700 1 -- 172.21.15.119:0/278686584 >> 172.21.15.119:0/278686584 conn(0x7fef9c0fbc90 msgr2=0x7fef9c0fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:50.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.085+0000 7fefa3200700 1 -- 172.21.15.119:0/278686584 shutdown_connections 2024-05-27T09:35:50.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.086+0000 7fefa3200700 1 -- 172.21.15.119:0/278686584 wait complete. 2024-05-27T09:35:50.088 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 22 2024-05-27T09:35:50.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:49 smithi138 ceph-mon[70004]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:35:50.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:49 smithi138 ceph-mon[70004]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:35:50.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:49 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/82321778' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:35:50.245 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.245+0000 7fdd2ea00700 1 -- 172.21.15.119:0/3114355076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd280730f0 msgr2=0x7fdd280734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:50.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.245+0000 7fdd2ea00700 1 --2- 172.21.15.119:0/3114355076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd280730f0 0x7fdd280734c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009910 tx=0x7fdd18009c20 comp rx=0 tx=0).stop 2024-05-27T09:35:50.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.245+0000 7fdd2ea00700 1 -- 172.21.15.119:0/3114355076 shutdown_connections 2024-05-27T09:35:50.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.245+0000 7fdd2ea00700 1 --2- 172.21.15.119:0/3114355076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd28073a00 0x7fdd28110c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:50.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.245+0000 7fdd2ea00700 1 --2- 172.21.15.119:0/3114355076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd280730f0 0x7fdd280734c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:50.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.245+0000 7fdd2ea00700 1 -- 172.21.15.119:0/3114355076 >> 172.21.15.119:0/3114355076 conn(0x7fdd280fbc90 msgr2=0x7fdd280fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:50.246 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:49 smithi119 ceph-mon[88269]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:35:50.246 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:49 smithi119 ceph-mon[88269]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:35:50.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:49 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/82321778' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:35:50.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.245+0000 7fdd2ea00700 1 -- 172.21.15.119:0/3114355076 shutdown_connections 2024-05-27T09:35:50.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.245+0000 7fdd2ea00700 1 -- 172.21.15.119:0/3114355076 wait complete. 2024-05-27T09:35:50.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.246+0000 7fdd2ea00700 1 Processor -- start 2024-05-27T09:35:50.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.246+0000 7fdd2ea00700 1 -- start start 2024-05-27T09:35:50.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.246+0000 7fdd2ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd280730f0 0x7fdd2819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:50.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.247+0000 7fdd2ea00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd28073a00 0x7fdd2819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:50.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.247+0000 7fdd2ea00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2819cee0 con 0x7fdd280730f0 2024-05-27T09:35:50.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.247+0000 7fdd2ea00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd281a0c70 con 0x7fdd28073a00 2024-05-27T09:35:50.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.247+0000 7fdd27e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd280730f0 0x7fdd2819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:50.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.247+0000 7fdd27e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd280730f0 0x7fdd2819c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52744/0 (socket says 172.21.15.119:52744) 2024-05-27T09:35:50.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.247+0000 7fdd27e00700 1 -- 172.21.15.119:0/2404122008 learned_addr learned my addr 172.21.15.119:0/2404122008 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:35:50.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.247+0000 7fdd27e00700 1 -- 172.21.15.119:0/2404122008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd28073a00 msgr2=0x7fdd2819c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:50.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.247+0000 7fdd1f400700 1 --2- 172.21.15.119:0/2404122008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd28073a00 0x7fdd2819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:50.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.247+0000 7fdd27e00700 1 --2- 172.21.15.119:0/2404122008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd28073a00 0x7fdd2819c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:50.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.247+0000 7fdd27e00700 1 -- 172.21.15.119:0/2404122008 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd180095f0 con 0x7fdd280730f0 2024-05-27T09:35:50.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.248+0000 7fdd1f400700 1 --2- 172.21.15.119:0/2404122008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd28073a00 0x7fdd2819c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:35:50.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.248+0000 7fdd27e00700 1 --2- 172.21.15.119:0/2404122008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd280730f0 0x7fdd2819c2c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800b5c0 tx=0x7fdd1800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:50.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.248+0000 7fdd25600700 1 -- 172.21.15.119:0/2404122008 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd1801d070 con 0x7fdd280730f0 2024-05-27T09:35:50.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.248+0000 7fdd25600700 1 -- 172.21.15.119:0/2404122008 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd1800f460 con 0x7fdd280730f0 2024-05-27T09:35:50.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.248+0000 7fdd25600700 1 -- 172.21.15.119:0/2404122008 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd18017630 con 0x7fdd280730f0 2024-05-27T09:35:50.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.248+0000 7fdd2ea00700 1 -- 172.21.15.119:0/2404122008 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd281a0f50 con 0x7fdd280730f0 2024-05-27T09:35:50.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.248+0000 7fdd2ea00700 1 -- 172.21.15.119:0/2404122008 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd281a1450 con 0x7fdd280730f0 2024-05-27T09:35:50.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.250+0000 7fdd2ea00700 1 -- 172.21.15.119:0/2404122008 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd2810e3c0 con 0x7fdd280730f0 2024-05-27T09:35:50.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.251+0000 7fdd25600700 1 -- 172.21.15.119:0/2404122008 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 38) v1 ==== 100184+0+0 (secure 0 0 0) 0x7fdd1800f5d0 con 0x7fdd280730f0 2024-05-27T09:35:50.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.252+0000 7fdd25600700 1 --2- 172.21.15.119:0/2404122008 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdd0c078b60 0x7fdd0c07b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:50.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.252+0000 7fdd25600700 1 -- 172.21.15.119:0/2404122008 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fdd1809c4e0 con 0x7fdd280730f0 2024-05-27T09:35:50.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.252+0000 7fdd1f400700 1 --2- 172.21.15.119:0/2404122008 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdd0c078b60 0x7fdd0c07b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:50.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.253+0000 7fdd1f400700 1 --2- 172.21.15.119:0/2404122008 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdd0c078b60 0x7fdd0c07b010 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdd10005fd0 tx=0x7fdd10005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:50.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.257+0000 7fdd25600700 1 -- 172.21.15.119:0/2404122008 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fdd18063f40 con 0x7fdd280730f0 2024-05-27T09:35:50.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.513+0000 7fdd2ea00700 1 -- 172.21.15.119:0/2404122008 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd281a17f0 con 0x7fdd0c078b60 2024-05-27T09:35:50.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.515+0000 7fdd25600700 1 -- 172.21.15.119:0/2404122008 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7fdd281a17f0 con 0x7fdd0c078b60 2024-05-27T09:35:50.515 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:35:50.515 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:35:50.516 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:35:50.516 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-27T09:35:50.516 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [ 2024-05-27T09:35:50.516 INFO:teuthology.orchestra.run.smithi119.stdout: "mon", 2024-05-27T09:35:50.516 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr" 2024-05-27T09:35:50.516 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-05-27T09:35:50.516 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "4/23 daemons upgraded", 2024-05-27T09:35:50.516 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "Currently upgrading crash daemons", 2024-05-27T09:35:50.516 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:35:50.516 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:35:50.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.517+0000 7fdd2ea00700 1 -- 172.21.15.119:0/2404122008 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdd0c078b60 msgr2=0x7fdd0c07b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:50.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.517+0000 7fdd2ea00700 1 --2- 172.21.15.119:0/2404122008 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdd0c078b60 0x7fdd0c07b010 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdd10005fd0 tx=0x7fdd10005dc0 comp rx=0 tx=0).stop 2024-05-27T09:35:50.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.517+0000 7fdd2ea00700 1 -- 172.21.15.119:0/2404122008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd280730f0 msgr2=0x7fdd2819c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:50.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.517+0000 7fdd2ea00700 1 --2- 172.21.15.119:0/2404122008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd280730f0 0x7fdd2819c2c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800b5c0 tx=0x7fdd1800be00 comp rx=0 tx=0).stop 2024-05-27T09:35:50.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.518+0000 7fdd2ea00700 1 -- 172.21.15.119:0/2404122008 shutdown_connections 2024-05-27T09:35:50.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.518+0000 7fdd2ea00700 1 --2- 172.21.15.119:0/2404122008 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdd0c078b60 0x7fdd0c07b010 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:50.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.518+0000 7fdd2ea00700 1 --2- 172.21.15.119:0/2404122008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd280730f0 0x7fdd2819c2c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:50.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.518+0000 7fdd2ea00700 1 --2- 172.21.15.119:0/2404122008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd28073a00 0x7fdd2819c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:50.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.518+0000 7fdd2ea00700 1 -- 172.21.15.119:0/2404122008 >> 172.21.15.119:0/2404122008 conn(0x7fdd280fbc90 msgr2=0x7fdd28102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:50.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.518+0000 7fdd2ea00700 1 -- 172.21.15.119:0/2404122008 shutdown_connections 2024-05-27T09:35:50.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.518+0000 7fdd2ea00700 1 -- 172.21.15.119:0/2404122008 wait complete. 2024-05-27T09:35:50.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.670+0000 7f9cafc00700 1 -- 172.21.15.119:0/2597780608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca8102fa0 msgr2=0x7f9ca8106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:50.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.670+0000 7f9cafc00700 1 --2- 172.21.15.119:0/2597780608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca8102fa0 0x7f9ca8106f70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009910 tx=0x7f9c94009c20 comp rx=0 tx=0).stop 2024-05-27T09:35:50.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.671+0000 7f9cafc00700 1 -- 172.21.15.119:0/2597780608 shutdown_connections 2024-05-27T09:35:50.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.671+0000 7f9cafc00700 1 --2- 172.21.15.119:0/2597780608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca8102fa0 0x7f9ca8106f70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:50.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.671+0000 7f9cafc00700 1 --2- 172.21.15.119:0/2597780608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ca8102600 0x7f9ca81029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:50.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.671+0000 7f9cafc00700 1 -- 172.21.15.119:0/2597780608 >> 172.21.15.119:0/2597780608 conn(0x7f9ca80fdec0 msgr2=0x7f9ca81002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:50.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.671+0000 7f9cafc00700 1 -- 172.21.15.119:0/2597780608 shutdown_connections 2024-05-27T09:35:50.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.671+0000 7f9cafc00700 1 -- 172.21.15.119:0/2597780608 wait complete. 2024-05-27T09:35:50.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.672+0000 7f9cafc00700 1 Processor -- start 2024-05-27T09:35:50.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.672+0000 7f9cafc00700 1 -- start start 2024-05-27T09:35:50.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.672+0000 7f9cafc00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ca8102600 0x7f9ca8197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:50.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.672+0000 7f9cafc00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca8102fa0 0x7f9ca8198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:50.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.672+0000 7f9cafc00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca8198c10 con 0x7f9ca8102fa0 2024-05-27T09:35:50.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.672+0000 7f9cafc00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca819c9a0 con 0x7f9ca8102600 2024-05-27T09:35:50.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.673+0000 7f9ca7e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca8102fa0 0x7f9ca8198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:50.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.673+0000 7f9ca7e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca8102fa0 0x7f9ca8198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52762/0 (socket says 172.21.15.119:52762) 2024-05-27T09:35:50.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.673+0000 7f9ca7e00700 1 -- 172.21.15.119:0/2598807743 learned_addr learned my addr 172.21.15.119:0/2598807743 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:35:50.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.673+0000 7f9cad200700 1 --2- 172.21.15.119:0/2598807743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ca8102600 0x7f9ca8197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:50.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.673+0000 7f9ca7e00700 1 -- 172.21.15.119:0/2598807743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ca8102600 msgr2=0x7f9ca8197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:50.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.673+0000 7f9ca7e00700 1 --2- 172.21.15.119:0/2598807743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ca8102600 0x7f9ca8197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:50.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.673+0000 7f9ca7e00700 1 -- 172.21.15.119:0/2598807743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c940095f0 con 0x7f9ca8102fa0 2024-05-27T09:35:50.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.673+0000 7f9cad200700 1 --2- 172.21.15.119:0/2598807743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ca8102600 0x7f9ca8197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:35:50.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.674+0000 7f9ca7e00700 1 --2- 172.21.15.119:0/2598807743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca8102fa0 0x7f9ca8198530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400bda0 tx=0x7f9c9400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:50.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.674+0000 7f9ca5600700 1 -- 172.21.15.119:0/2598807743 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c9401d070 con 0x7f9ca8102fa0 2024-05-27T09:35:50.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.674+0000 7f9ca5600700 1 -- 172.21.15.119:0/2598807743 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c94004fb0 con 0x7f9ca8102fa0 2024-05-27T09:35:50.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.674+0000 7f9ca5600700 1 -- 172.21.15.119:0/2598807743 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c9400f460 con 0x7f9ca8102fa0 2024-05-27T09:35:50.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.674+0000 7f9cafc00700 1 -- 172.21.15.119:0/2598807743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca819cc20 con 0x7f9ca8102fa0 2024-05-27T09:35:50.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.674+0000 7f9cafc00700 1 -- 172.21.15.119:0/2598807743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca819d0c0 con 0x7f9ca8102fa0 2024-05-27T09:35:50.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.675+0000 7f9cafc00700 1 -- 172.21.15.119:0/2598807743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca804ea50 con 0x7f9ca8102fa0 2024-05-27T09:35:50.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.677+0000 7f9ca5600700 1 -- 172.21.15.119:0/2598807743 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 38) v1 ==== 100184+0+0 (secure 0 0 0) 0x7f9c94022800 con 0x7f9ca8102fa0 2024-05-27T09:35:50.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.677+0000 7f9ca5600700 1 --2- 172.21.15.119:0/2598807743 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9c90078b60 0x7f9c9007b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:35:50.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.678+0000 7f9ca5600700 1 -- 172.21.15.119:0/2598807743 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f9c9409c490 con 0x7f9ca8102fa0 2024-05-27T09:35:50.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.678+0000 7f9cad200700 1 --2- 172.21.15.119:0/2598807743 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9c90078b60 0x7f9c9007b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:35:50.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.679+0000 7f9cad200700 1 --2- 172.21.15.119:0/2598807743 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9c90078b60 0x7f9c9007b010 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0060b0 tx=0x7f9c9c0076a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:35:50.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:50.683+0000 7f9ca5600700 1 -- 172.21.15.119:0/2598807743 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f9c94063fa0 con 0x7f9ca8102fa0 2024-05-27T09:35:51.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:51.001+0000 7f9cafc00700 1 -- 172.21.15.119:0/2598807743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9ca8199350 con 0x7f9ca8102fa0 2024-05-27T09:35:51.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:51.002+0000 7f9ca5600700 1 -- 172.21.15.119:0/2598807743 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f9c94063940 con 0x7f9ca8102fa0 2024-05-27T09:35:51.003 INFO:teuthology.orchestra.run.smithi119.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-27T09:35:51.003 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:35:51.003 INFO:teuthology.orchestra.run.smithi119.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:35:51.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:51.005+0000 7f9cafc00700 1 -- 172.21.15.119:0/2598807743 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9c90078b60 msgr2=0x7f9c9007b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:51.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:51.005+0000 7f9cafc00700 1 --2- 172.21.15.119:0/2598807743 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9c90078b60 0x7f9c9007b010 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0060b0 tx=0x7f9c9c0076a0 comp rx=0 tx=0).stop 2024-05-27T09:35:51.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:51.005+0000 7f9cafc00700 1 -- 172.21.15.119:0/2598807743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca8102fa0 msgr2=0x7f9ca8198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:35:51.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:51.005+0000 7f9cafc00700 1 --2- 172.21.15.119:0/2598807743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca8102fa0 0x7f9ca8198530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400bda0 tx=0x7f9c9400be80 comp rx=0 tx=0).stop 2024-05-27T09:35:51.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:51.005+0000 7f9cafc00700 1 -- 172.21.15.119:0/2598807743 shutdown_connections 2024-05-27T09:35:51.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:51.005+0000 7f9cafc00700 1 --2- 172.21.15.119:0/2598807743 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9c90078b60 0x7f9c9007b010 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:51.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:51.005+0000 7f9cafc00700 1 --2- 172.21.15.119:0/2598807743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ca8102600 0x7f9ca8197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:51.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:51.005+0000 7f9cafc00700 1 --2- 172.21.15.119:0/2598807743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca8102fa0 0x7f9ca8198530 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:35:51.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:51.005+0000 7f9cafc00700 1 -- 172.21.15.119:0/2598807743 >> 172.21.15.119:0/2598807743 conn(0x7f9ca80fdec0 msgr2=0x7f9ca80ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:35:51.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:51.005+0000 7f9cafc00700 1 -- 172.21.15.119:0/2598807743 shutdown_connections 2024-05-27T09:35:51.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:35:51.005+0000 7f9cafc00700 1 -- 172.21.15.119:0/2598807743 wait complete. 2024-05-27T09:35:51.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:50 smithi138 ceph-mon[70004]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:35:51.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:50 smithi138 ceph-mon[70004]: pgmap v44: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:51.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:50 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/278686584' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:35:51.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:50 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:51.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:50 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:51.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:50 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:35:51.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:50 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:35:51.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:50 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:35:51.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:50 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:51.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:50 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi119"}]: dispatch 2024-05-27T09:35:51.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:50 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi119"}]': finished 2024-05-27T09:35:51.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:50 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi138"}]: dispatch 2024-05-27T09:35:51.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:50 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi138"}]': finished 2024-05-27T09:35:51.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:50 smithi119 ceph-mon[88269]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:35:51.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:50 smithi119 ceph-mon[88269]: pgmap v44: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:51.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:50 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/278686584' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:35:51.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:50 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:51.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:50 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:51.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:50 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:35:51.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:50 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:35:51.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:50 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:35:51.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:50 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:51.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:50 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi119"}]: dispatch 2024-05-27T09:35:51.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:50 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi119"}]': finished 2024-05-27T09:35:51.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:50 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi138"}]: dispatch 2024-05-27T09:35:51.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:50 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi138"}]': finished 2024-05-27T09:35:51.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:51 smithi119 ceph-mon[88269]: Upgrade: Setting container_image for all mon 2024-05-27T09:35:51.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:51 smithi119 ceph-mon[88269]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:35:51.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:51 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/2598807743' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:35:52.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:51 smithi138 ceph-mon[70004]: Upgrade: Setting container_image for all mon 2024-05-27T09:35:52.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:51 smithi138 ceph-mon[70004]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:35:52.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:51 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/2598807743' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:35:52.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:52 smithi119 ceph-mon[88269]: pgmap v45: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:52.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:52 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:52.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:52 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi119", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-27T09:35:52.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:52 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:53.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:52 smithi138 ceph-mon[70004]: pgmap v45: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:53.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:52 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:35:53.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:52 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi119", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-27T09:35:53.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:52 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:35:53.944 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:53 smithi119 ceph-mon[88269]: Upgrade: Updating crash.smithi119 (1/2) 2024-05-27T09:35:53.944 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:53 smithi119 ceph-mon[88269]: Deploying daemon crash.smithi119 on smithi119 2024-05-27T09:35:54.287 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:53 smithi138 ceph-mon[70004]: Upgrade: Updating crash.smithi119 (1/2) 2024-05-27T09:35:54.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:53 smithi138 ceph-mon[70004]: Deploying daemon crash.smithi119 on smithi119 2024-05-27T09:35:54.997 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:54 smithi138 ceph-mon[70004]: pgmap v46: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:55.198 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:54 smithi119 ceph-mon[88269]: pgmap v46: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:57.233 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:56 smithi119 ceph-mon[88269]: pgmap v47: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:57.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:56 smithi138 ceph-mon[70004]: pgmap v47: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:58.311 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:58 smithi119 ceph-mon[88269]: pgmap v48: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:58.312 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:58 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:35:58.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:58 smithi138 ceph-mon[70004]: pgmap v48: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:35:58.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:58 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:35:59.717 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:59 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:00.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:35:59 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:00.038 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:59 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:00.038 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:35:59 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:00.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:00 smithi138 ceph-mon[70004]: pgmap v49: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:36:00.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:00 smithi138 ceph-mon[70004]: mgrmap e39: smithi119.ncwmog(active, since 92s), standbys: smithi138.dxsuqf 2024-05-27T09:36:00.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:00 smithi119 ceph-mon[88269]: pgmap v49: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:36:00.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:00 smithi119 ceph-mon[88269]: mgrmap e39: smithi119.ncwmog(active, since 92s), standbys: smithi138.dxsuqf 2024-05-27T09:36:02.271 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:02 smithi138 ceph-mon[70004]: Upgrade: Updating crash.smithi138 (2/2) 2024-05-27T09:36:02.272 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:02 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:02.272 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:02 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-27T09:36:02.272 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:02 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:36:02.272 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:02 smithi138 ceph-mon[70004]: Deploying daemon crash.smithi138 on smithi138 2024-05-27T09:36:02.272 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:02 smithi138 ceph-mon[70004]: pgmap v50: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:36:02.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:02 smithi119 ceph-mon[88269]: Upgrade: Updating crash.smithi138 (2/2) 2024-05-27T09:36:02.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:02 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:02.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:02 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-27T09:36:02.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:02 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:36:02.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:02 smithi119 ceph-mon[88269]: Deploying daemon crash.smithi138 on smithi138 2024-05-27T09:36:02.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:02 smithi119 ceph-mon[88269]: pgmap v50: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:36:04.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:04 smithi138 ceph-mon[70004]: pgmap v51: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:36:04.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:04 smithi119 ceph-mon[88269]: pgmap v51: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:36:06.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:06 smithi138 ceph-mon[70004]: pgmap v52: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:36:06.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:06 smithi119 ceph-mon[88269]: pgmap v52: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:36:08.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:08 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:08.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:08 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:08.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:08 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:36:08.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:08 smithi119 ceph-mon[88269]: pgmap v53: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:36:08.550 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:08 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:08.551 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:08 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:08.551 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:08 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:36:08.551 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:08 smithi138 ceph-mon[70004]: pgmap v53: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:36:10.626 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:10 smithi119 ceph-mon[88269]: pgmap v54: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:36:10.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:10 smithi138 ceph-mon[70004]: pgmap v54: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-05-27T09:36:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:12 smithi119 ceph-mon[88269]: pgmap v55: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:36:13.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:12 smithi138 ceph-mon[70004]: pgmap v55: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:13.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:13.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:13.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:36:15.220 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:14 smithi119 ceph-mon[88269]: pgmap v56: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:15.220 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:15.220 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:15.221 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:15.221 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:15.221 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:14 smithi138 ceph-mon[70004]: pgmap v56: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:15.221 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:15.221 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:15.221 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:15.221 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:17.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:16 smithi119 ceph-mon[88269]: pgmap v57: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:17.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:16 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:17.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:16 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:17.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:16 smithi138 ceph-mon[70004]: pgmap v57: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:17.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:16 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:17.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:16 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:19.224 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:18 smithi119 ceph-mon[88269]: pgmap v58: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:19.307 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:18 smithi138 ceph-mon[70004]: pgmap v58: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:20.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:20 smithi138 ceph-mon[70004]: pgmap v59: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:20.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:20 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:20.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:20 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:20.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:20 smithi119 ceph-mon[88269]: pgmap v59: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:20.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:20 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:20.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:20 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:21.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.194+0000 7f4413a00700 1 -- 172.21.15.119:0/1807160438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f440c075b90 msgr2=0x7f440c110b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:21.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.194+0000 7f4413a00700 1 --2- 172.21.15.119:0/1807160438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f440c075b90 0x7f440c110b10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f43f8009910 tx=0x7f43f8009c20 comp rx=0 tx=0).stop 2024-05-27T09:36:21.202 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.195+0000 7f4413a00700 1 -- 172.21.15.119:0/1807160438 shutdown_connections 2024-05-27T09:36:21.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.195+0000 7f4413a00700 1 --2- 172.21.15.119:0/1807160438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f440c075b90 0x7f440c110b10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:21.203 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.195+0000 7f4413a00700 1 --2- 172.21.15.119:0/1807160438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f440c075280 0x7f440c075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:21.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.195+0000 7f4413a00700 1 -- 172.21.15.119:0/1807160438 >> 172.21.15.119:0/1807160438 conn(0x7f440c0fded0 msgr2=0x7f440c1002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:21.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.195+0000 7f4413a00700 1 -- 172.21.15.119:0/1807160438 shutdown_connections 2024-05-27T09:36:21.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.195+0000 7f4413a00700 1 -- 172.21.15.119:0/1807160438 wait complete. 2024-05-27T09:36:21.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.196+0000 7f4413a00700 1 Processor -- start 2024-05-27T09:36:21.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.196+0000 7f4413a00700 1 -- start start 2024-05-27T09:36:21.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.196+0000 7f4413a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f440c075280 0x7f440c19c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:21.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.196+0000 7f4413a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f440c075b90 0x7f440c19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:21.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.196+0000 7f4413a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c19cf40 con 0x7f440c075b90 2024-05-27T09:36:21.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.196+0000 7f4413a00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c1a0cd0 con 0x7f440c075280 2024-05-27T09:36:21.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.197+0000 7f440be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f440c075b90 0x7f440c19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:21.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.197+0000 7f440be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f440c075b90 0x7f440c19c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55336/0 (socket says 172.21.15.119:55336) 2024-05-27T09:36:21.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.197+0000 7f440be00700 1 -- 172.21.15.119:0/4117203521 learned_addr learned my addr 172.21.15.119:0/4117203521 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:36:21.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.197+0000 7f4411000700 1 --2- 172.21.15.119:0/4117203521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f440c075280 0x7f440c19c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:21.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.197+0000 7f440be00700 1 -- 172.21.15.119:0/4117203521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f440c075280 msgr2=0x7f440c19c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:21.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.197+0000 7f440be00700 1 --2- 172.21.15.119:0/4117203521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f440c075280 0x7f440c19c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:21.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.197+0000 7f440be00700 1 -- 172.21.15.119:0/4117203521 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f80095f0 con 0x7f440c075b90 2024-05-27T09:36:21.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.198+0000 7f440be00700 1 --2- 172.21.15.119:0/4117203521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f440c075b90 0x7f440c19c860 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f43f80092e0 tx=0x7f43f80056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:21.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.198+0000 7f4409600700 1 -- 172.21.15.119:0/4117203521 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43f801d070 con 0x7f440c075b90 2024-05-27T09:36:21.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.198+0000 7f4409600700 1 -- 172.21.15.119:0/4117203521 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43f800f460 con 0x7f440c075b90 2024-05-27T09:36:21.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.198+0000 7f4409600700 1 -- 172.21.15.119:0/4117203521 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43f8021640 con 0x7f440c075b90 2024-05-27T09:36:21.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.198+0000 7f4413a00700 1 -- 172.21.15.119:0/4117203521 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f440c1a0f50 con 0x7f440c075b90 2024-05-27T09:36:21.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.198+0000 7f4413a00700 1 -- 172.21.15.119:0/4117203521 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440c1a14a0 con 0x7f440c075b90 2024-05-27T09:36:21.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.200+0000 7f4413a00700 1 -- 172.21.15.119:0/4117203521 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f440c10e280 con 0x7f440c075b90 2024-05-27T09:36:21.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.201+0000 7f4409600700 1 -- 172.21.15.119:0/4117203521 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f43f800f5d0 con 0x7f440c075b90 2024-05-27T09:36:21.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.202+0000 7f4409600700 1 --2- 172.21.15.119:0/4117203521 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f43f4078a40 0x7f43f407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:21.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.202+0000 7f4409600700 1 -- 172.21.15.119:0/4117203521 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f43f809bfe0 con 0x7f440c075b90 2024-05-27T09:36:21.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.202+0000 7f4411000700 1 --2- 172.21.15.119:0/4117203521 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f43f4078a40 0x7f43f407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:21.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.203+0000 7f4411000700 1 --2- 172.21.15.119:0/4117203521 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f43f4078a40 0x7f43f407aef0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4400009d10 tx=0x7f4400009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:21.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.208+0000 7f4409600700 1 -- 172.21.15.119:0/4117203521 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f43f8063c00 con 0x7f440c075b90 2024-05-27T09:36:21.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.472+0000 7f4413a00700 1 -- 172.21.15.119:0/4117203521 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f440c1a1780 con 0x7f43f4078a40 2024-05-27T09:36:21.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.477+0000 7f4409600700 1 -- 172.21.15.119:0/4117203521 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7f440c1a1780 con 0x7f43f4078a40 2024-05-27T09:36:21.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.480+0000 7f4413a00700 1 -- 172.21.15.119:0/4117203521 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f43f4078a40 msgr2=0x7f43f407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:21.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.480+0000 7f4413a00700 1 --2- 172.21.15.119:0/4117203521 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f43f4078a40 0x7f43f407aef0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4400009d10 tx=0x7f4400009040 comp rx=0 tx=0).stop 2024-05-27T09:36:21.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.480+0000 7f4413a00700 1 -- 172.21.15.119:0/4117203521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f440c075b90 msgr2=0x7f440c19c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:21.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.480+0000 7f4413a00700 1 --2- 172.21.15.119:0/4117203521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f440c075b90 0x7f440c19c860 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f43f80092e0 tx=0x7f43f80056a0 comp rx=0 tx=0).stop 2024-05-27T09:36:21.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.480+0000 7f4413a00700 1 -- 172.21.15.119:0/4117203521 shutdown_connections 2024-05-27T09:36:21.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.480+0000 7f4413a00700 1 --2- 172.21.15.119:0/4117203521 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f43f4078a40 0x7f43f407aef0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:21.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.480+0000 7f4413a00700 1 --2- 172.21.15.119:0/4117203521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f440c075280 0x7f440c19c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:21.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.480+0000 7f4413a00700 1 --2- 172.21.15.119:0/4117203521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f440c075b90 0x7f440c19c860 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:21.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.480+0000 7f4413a00700 1 -- 172.21.15.119:0/4117203521 >> 172.21.15.119:0/4117203521 conn(0x7f440c0fded0 msgr2=0x7f440c0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:21.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.480+0000 7f4413a00700 1 -- 172.21.15.119:0/4117203521 shutdown_connections 2024-05-27T09:36:21.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.481+0000 7f4413a00700 1 -- 172.21.15.119:0/4117203521 wait complete. 2024-05-27T09:36:21.499 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:36:21.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.653+0000 7f71ebe00700 1 -- 172.21.15.119:0/1620435974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71ec102de0 msgr2=0x7f71ec106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:21.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.653+0000 7f71ebe00700 1 --2- 172.21.15.119:0/1620435974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71ec102de0 0x7f71ec106db0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009910 tx=0x7f71d8009c20 comp rx=0 tx=0).stop 2024-05-27T09:36:21.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.654+0000 7f71ebe00700 1 -- 172.21.15.119:0/1620435974 shutdown_connections 2024-05-27T09:36:21.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.654+0000 7f71ebe00700 1 --2- 172.21.15.119:0/1620435974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71ec102de0 0x7f71ec106db0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:21.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.654+0000 7f71ebe00700 1 --2- 172.21.15.119:0/1620435974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71ec102440 0x7f71ec102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:21.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.654+0000 7f71ebe00700 1 -- 172.21.15.119:0/1620435974 >> 172.21.15.119:0/1620435974 conn(0x7f71ec0fdce0 msgr2=0x7f71ec1000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:21.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.654+0000 7f71ebe00700 1 -- 172.21.15.119:0/1620435974 shutdown_connections 2024-05-27T09:36:21.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.654+0000 7f71ebe00700 1 -- 172.21.15.119:0/1620435974 wait complete. 2024-05-27T09:36:21.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.655+0000 7f71ebe00700 1 Processor -- start 2024-05-27T09:36:21.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.655+0000 7f71ebe00700 1 -- start start 2024-05-27T09:36:21.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.655+0000 7f71ebe00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71ec102440 0x7f71ec197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:21.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.655+0000 7f71ebe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71ec102de0 0x7f71ec198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:21.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.655+0000 7f71ebe00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec198af0 con 0x7f71ec102de0 2024-05-27T09:36:21.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.655+0000 7f71ebe00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec19c880 con 0x7f71ec102440 2024-05-27T09:36:21.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.656+0000 7f71eaa00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71ec102440 0x7f71ec197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:21.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.656+0000 7f71eaa00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71ec102440 0x7f71ec197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.119:59916/0 (socket says 172.21.15.119:59916) 2024-05-27T09:36:21.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.656+0000 7f71eaa00700 1 -- 172.21.15.119:0/3596260440 learned_addr learned my addr 172.21.15.119:0/3596260440 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:36:21.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.656+0000 7f71ea000700 1 --2- 172.21.15.119:0/3596260440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71ec102de0 0x7f71ec198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:21.658 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.657+0000 7f71ea000700 1 -- 172.21.15.119:0/3596260440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71ec102440 msgr2=0x7f71ec197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:21.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.657+0000 7f71ea000700 1 --2- 172.21.15.119:0/3596260440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71ec102440 0x7f71ec197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:21.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.657+0000 7f71ea000700 1 -- 172.21.15.119:0/3596260440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d80095f0 con 0x7f71ec102de0 2024-05-27T09:36:21.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.657+0000 7f71ea000700 1 --2- 172.21.15.119:0/3596260440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71ec102de0 0x7f71ec198410 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f71ec074a00 tx=0x7f71d800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:21.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.657+0000 7f71e3400700 1 -- 172.21.15.119:0/3596260440 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71d801d070 con 0x7f71ec102de0 2024-05-27T09:36:21.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.657+0000 7f71e3400700 1 -- 172.21.15.119:0/3596260440 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71d800f460 con 0x7f71ec102de0 2024-05-27T09:36:21.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.657+0000 7f71ebe00700 1 -- 172.21.15.119:0/3596260440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ec19cb00 con 0x7f71ec102de0 2024-05-27T09:36:21.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.657+0000 7f71e3400700 1 -- 172.21.15.119:0/3596260440 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71d8017630 con 0x7f71ec102de0 2024-05-27T09:36:21.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.658+0000 7f71ebe00700 1 -- 172.21.15.119:0/3596260440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ec19cff0 con 0x7f71ec102de0 2024-05-27T09:36:21.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.659+0000 7f71ebe00700 1 -- 172.21.15.119:0/3596260440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71ec10a700 con 0x7f71ec102de0 2024-05-27T09:36:21.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.660+0000 7f71e3400700 1 -- 172.21.15.119:0/3596260440 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f71d800fab0 con 0x7f71ec102de0 2024-05-27T09:36:21.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.661+0000 7f71e3400700 1 --2- 172.21.15.119:0/3596260440 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f71d0078a40 0x7f71d007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:21.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.661+0000 7f71e3400700 1 -- 172.21.15.119:0/3596260440 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f71d809c0d0 con 0x7f71ec102de0 2024-05-27T09:36:21.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.661+0000 7f71eaa00700 1 --2- 172.21.15.119:0/3596260440 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f71d0078a40 0x7f71d007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:21.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.662+0000 7f71eaa00700 1 --2- 172.21.15.119:0/3596260440 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f71d0078a40 0x7f71d007aef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009ad0 tx=0x7f71dc009290 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:21.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.667+0000 7f71e3400700 1 -- 172.21.15.119:0/3596260440 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f71d8063d30 con 0x7f71ec102de0 2024-05-27T09:36:21.930 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.928+0000 7f71ebe00700 1 -- 172.21.15.119:0/3596260440 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f71ec02cd10 con 0x7f71d0078a40 2024-05-27T09:36:21.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.930+0000 7f71e3400700 1 -- 172.21.15.119:0/3596260440 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7f71ec02cd10 con 0x7f71d0078a40 2024-05-27T09:36:21.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.934+0000 7f71ebe00700 1 -- 172.21.15.119:0/3596260440 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f71d0078a40 msgr2=0x7f71d007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:21.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.934+0000 7f71ebe00700 1 --2- 172.21.15.119:0/3596260440 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f71d0078a40 0x7f71d007aef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009ad0 tx=0x7f71dc009290 comp rx=0 tx=0).stop 2024-05-27T09:36:21.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.934+0000 7f71ebe00700 1 -- 172.21.15.119:0/3596260440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71ec102de0 msgr2=0x7f71ec198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:21.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.934+0000 7f71ebe00700 1 --2- 172.21.15.119:0/3596260440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71ec102de0 0x7f71ec198410 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f71ec074a00 tx=0x7f71d800bd70 comp rx=0 tx=0).stop 2024-05-27T09:36:21.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.934+0000 7f71ebe00700 1 -- 172.21.15.119:0/3596260440 shutdown_connections 2024-05-27T09:36:21.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.934+0000 7f71ebe00700 1 --2- 172.21.15.119:0/3596260440 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f71d0078a40 0x7f71d007aef0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:21.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.934+0000 7f71ebe00700 1 --2- 172.21.15.119:0/3596260440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71ec102440 0x7f71ec197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:21.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.934+0000 7f71ebe00700 1 --2- 172.21.15.119:0/3596260440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71ec102de0 0x7f71ec198410 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:21.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.934+0000 7f71ebe00700 1 -- 172.21.15.119:0/3596260440 >> 172.21.15.119:0/3596260440 conn(0x7f71ec0fdce0 msgr2=0x7f71ec0ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:21.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.934+0000 7f71ebe00700 1 -- 172.21.15.119:0/3596260440 shutdown_connections 2024-05-27T09:36:21.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:21.934+0000 7f71ebe00700 1 -- 172.21.15.119:0/3596260440 wait complete. 2024-05-27T09:36:22.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.093+0000 7f9891800700 1 -- 172.21.15.119:0/1672235591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f988c073a00 msgr2=0x7f988c110c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:22.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.093+0000 7f9891800700 1 --2- 172.21.15.119:0/1672235591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f988c073a00 0x7f988c110c20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9878009910 tx=0x7f9878009c20 comp rx=0 tx=0).stop 2024-05-27T09:36:22.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.094+0000 7f9891800700 1 -- 172.21.15.119:0/1672235591 shutdown_connections 2024-05-27T09:36:22.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.094+0000 7f9891800700 1 --2- 172.21.15.119:0/1672235591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f988c073a00 0x7f988c110c20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:22.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.094+0000 7f9891800700 1 --2- 172.21.15.119:0/1672235591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f988c0730f0 0x7f988c0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:22.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.094+0000 7f9891800700 1 -- 172.21.15.119:0/1672235591 >> 172.21.15.119:0/1672235591 conn(0x7f988c0fbc30 msgr2=0x7f988c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:22.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.094+0000 7f9891800700 1 -- 172.21.15.119:0/1672235591 shutdown_connections 2024-05-27T09:36:22.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.094+0000 7f9891800700 1 -- 172.21.15.119:0/1672235591 wait complete. 2024-05-27T09:36:22.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.095+0000 7f9891800700 1 Processor -- start 2024-05-27T09:36:22.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.095+0000 7f9891800700 1 -- start start 2024-05-27T09:36:22.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.096+0000 7f9891800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f988c0730f0 0x7f988c19c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:22.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.096+0000 7f9891800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f988c073a00 0x7f988c19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:22.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.096+0000 7f9891800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c19d040 con 0x7f988c0730f0 2024-05-27T09:36:22.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.096+0000 7f9891800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c1a0dd0 con 0x7f988c073a00 2024-05-27T09:36:22.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.096+0000 7f988aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f988c0730f0 0x7f988c19c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:22.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.096+0000 7f988aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f988c0730f0 0x7f988c19c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55372/0 (socket says 172.21.15.119:55372) 2024-05-27T09:36:22.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.096+0000 7f988aa00700 1 -- 172.21.15.119:0/2602531715 learned_addr learned my addr 172.21.15.119:0/2602531715 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:36:22.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.097+0000 7f988a000700 1 --2- 172.21.15.119:0/2602531715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f988c073a00 0x7f988c19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:22.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.097+0000 7f988aa00700 1 -- 172.21.15.119:0/2602531715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f988c073a00 msgr2=0x7f988c19c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:22.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.097+0000 7f988aa00700 1 --2- 172.21.15.119:0/2602531715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f988c073a00 0x7f988c19c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:22.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.097+0000 7f988aa00700 1 -- 172.21.15.119:0/2602531715 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98780095f0 con 0x7f988c0730f0 2024-05-27T09:36:22.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.097+0000 7f988a000700 1 --2- 172.21.15.119:0/2602531715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f988c073a00 0x7f988c19c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:36:22.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.097+0000 7f988aa00700 1 --2- 172.21.15.119:0/2602531715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f988c0730f0 0x7f988c19c420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f987c00b4b0 tx=0x7f987c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:22.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.098+0000 7f9883400700 1 -- 172.21.15.119:0/2602531715 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f987c011800 con 0x7f988c0730f0 2024-05-27T09:36:22.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.098+0000 7f9883400700 1 -- 172.21.15.119:0/2602531715 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f987c011e20 con 0x7f988c0730f0 2024-05-27T09:36:22.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.098+0000 7f9891800700 1 -- 172.21.15.119:0/2602531715 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f988c1a10b0 con 0x7f988c0730f0 2024-05-27T09:36:22.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.098+0000 7f9883400700 1 -- 172.21.15.119:0/2602531715 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f987c00f2e0 con 0x7f988c0730f0 2024-05-27T09:36:22.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.098+0000 7f9891800700 1 -- 172.21.15.119:0/2602531715 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988c1a1600 con 0x7f988c0730f0 2024-05-27T09:36:22.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.099+0000 7f9891800700 1 -- 172.21.15.119:0/2602531715 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f988c10e3a0 con 0x7f988c0730f0 2024-05-27T09:36:22.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.101+0000 7f9883400700 1 -- 172.21.15.119:0/2602531715 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f987c010380 con 0x7f988c0730f0 2024-05-27T09:36:22.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.103+0000 7f9883400700 1 --2- 172.21.15.119:0/2602531715 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9870078a40 0x7f987007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:22.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.103+0000 7f9883400700 1 -- 172.21.15.119:0/2602531715 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f987c09abb0 con 0x7f988c0730f0 2024-05-27T09:36:22.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.103+0000 7f988a000700 1 --2- 172.21.15.119:0/2602531715 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9870078a40 0x7f987007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:22.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.104+0000 7f988a000700 1 --2- 172.21.15.119:0/2602531715 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9870078a40 0x7f987007aef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f987800b5c0 tx=0x7f9878005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:22.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.108+0000 7f9883400700 1 -- 172.21.15.119:0/2602531715 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f987c062950 con 0x7f988c0730f0 2024-05-27T09:36:22.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.379+0000 7f9891800700 1 -- 172.21.15.119:0/2602531715 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f988c1a1a10 con 0x7f9870078a40 2024-05-27T09:36:22.415 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.413+0000 7f9883400700 1 -- 172.21.15.119:0/2602531715 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f988c1a1a10 con 0x7f9870078a40 2024-05-27T09:36:22.416 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:36:22.416 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (3m) 8s ago 17m 14.1M - 0.25.0 c8568f914cd2 6b90564dcef7 2024-05-27T09:36:22.416 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (17m) 8s ago 17m 8321k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:36:22.416 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (16m) 10s ago 16m 8182k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:36:22.416 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (24s) 8s ago 17m 6669k - 19.0.0-3868-ga6507388 850a06a84d28 011d811ce7fc 2024-05-27T09:36:22.416 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (15s) 10s ago 16m 6665k - 19.0.0-3868-ga6507388 850a06a84d28 b546b3ed1afc 2024-05-27T09:36:22.416 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (3m) 8s ago 17m 74.3M - 9.4.12 2bacad6d85d8 b3abdab08db4 2024-05-27T09:36:22.416 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (10m) 8s ago 10m 17.3M - 18.2.0 dc2bc1663786 93b21713fb1e 2024-05-27T09:36:22.416 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (10m) 8s ago 10m 130M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:36:22.416 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (10m) 10s ago 10m 17.0M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:36:22.416 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (10m) 10s ago 10m 128M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:36:22.417 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (6m) 8s ago 19m 581M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:36:22.417 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (5m) 10s ago 16m 532M - 19.0.0-3868-ga6507388 850a06a84d28 2fc3a80696fa 2024-05-27T09:36:22.417 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (2m) 8s ago 19m 113M 2048M 19.0.0-3868-ga6507388 850a06a84d28 3e5c7a9ad372 2024-05-27T09:36:22.417 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (88s) 10s ago 16m 103M 2048M 19.0.0-3868-ga6507388 850a06a84d28 e4d471274cf1 2024-05-27T09:36:22.417 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (4m) 8s ago 17m 7604k - 1.5.0 0da6a335fe13 709c37815cff 2024-05-27T09:36:22.417 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (4m) 10s ago 16m 7776k - 1.5.0 0da6a335fe13 999047a1a5bf 2024-05-27T09:36:22.417 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (15m) 8s ago 15m 226M 1216M 18.2.0 dc2bc1663786 825bd1392508 2024-05-27T09:36:22.417 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (14m) 8s ago 14m 254M 1216M 18.2.0 dc2bc1663786 fce9819903d9 2024-05-27T09:36:22.417 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (13m) 8s ago 13m 191M 1216M 18.2.0 dc2bc1663786 3955a6112c1c 2024-05-27T09:36:22.417 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (13m) 10s ago 13m 270M 2240M 18.2.0 dc2bc1663786 693c92220622 2024-05-27T09:36:22.417 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (12m) 10s ago 12m 248M 2240M 18.2.0 dc2bc1663786 7559e658e917 2024-05-27T09:36:22.417 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (12m) 10s ago 12m 232M 2240M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:36:22.417 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (4m) 8s ago 16m 67.4M - 2.43.0 a07b618ecd1d cba50454ed21 2024-05-27T09:36:22.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.417+0000 7f9891800700 1 -- 172.21.15.119:0/2602531715 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9870078a40 msgr2=0x7f987007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:22.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.417+0000 7f9891800700 1 --2- 172.21.15.119:0/2602531715 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9870078a40 0x7f987007aef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f987800b5c0 tx=0x7f9878005960 comp rx=0 tx=0).stop 2024-05-27T09:36:22.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.417+0000 7f9891800700 1 -- 172.21.15.119:0/2602531715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f988c0730f0 msgr2=0x7f988c19c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:22.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.417+0000 7f9891800700 1 --2- 172.21.15.119:0/2602531715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f988c0730f0 0x7f988c19c420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f987c00b4b0 tx=0x7f987c00b870 comp rx=0 tx=0).stop 2024-05-27T09:36:22.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.417+0000 7f9891800700 1 -- 172.21.15.119:0/2602531715 shutdown_connections 2024-05-27T09:36:22.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.417+0000 7f9891800700 1 --2- 172.21.15.119:0/2602531715 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9870078a40 0x7f987007aef0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:22.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.417+0000 7f9891800700 1 --2- 172.21.15.119:0/2602531715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f988c0730f0 0x7f988c19c420 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:22.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.417+0000 7f9891800700 1 --2- 172.21.15.119:0/2602531715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f988c073a00 0x7f988c19c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:22.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.417+0000 7f9891800700 1 -- 172.21.15.119:0/2602531715 >> 172.21.15.119:0/2602531715 conn(0x7f988c0fbc30 msgr2=0x7f988c102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:22.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.418+0000 7f9891800700 1 -- 172.21.15.119:0/2602531715 shutdown_connections 2024-05-27T09:36:22.420 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.418+0000 7f9891800700 1 -- 172.21.15.119:0/2602531715 wait complete. 2024-05-27T09:36:22.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.584+0000 7f10fce00700 1 -- 172.21.15.119:0/628025993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10f80ffbb0 msgr2=0x7f10f80fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:22.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.584+0000 7f10fce00700 1 --2- 172.21.15.119:0/628025993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10f80ffbb0 0x7f10f80fff80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009940 tx=0x7f10e8009c50 comp rx=0 tx=0).stop 2024-05-27T09:36:22.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.585+0000 7f10fce00700 1 -- 172.21.15.119:0/628025993 shutdown_connections 2024-05-27T09:36:22.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.585+0000 7f10fce00700 1 --2- 172.21.15.119:0/628025993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10f81004c0 0x7f10f810c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:22.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.585+0000 7f10fce00700 1 --2- 172.21.15.119:0/628025993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10f80ffbb0 0x7f10f80fff80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:22.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.585+0000 7f10fce00700 1 -- 172.21.15.119:0/628025993 >> 172.21.15.119:0/628025993 conn(0x7f10f8074f50 msgr2=0x7f10f8075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:22.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.585+0000 7f10fce00700 1 -- 172.21.15.119:0/628025993 shutdown_connections 2024-05-27T09:36:22.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.585+0000 7f10fce00700 1 -- 172.21.15.119:0/628025993 wait complete. 2024-05-27T09:36:22.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.586+0000 7f10fce00700 1 Processor -- start 2024-05-27T09:36:22.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.586+0000 7f10fce00700 1 -- start start 2024-05-27T09:36:22.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.587+0000 7f10fce00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10f80ffbb0 0x7f10f8197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:22.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.587+0000 7f10fce00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10f81004c0 0x7f10f8198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:22.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.587+0000 7f10fce00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f81989b0 con 0x7f10f80ffbb0 2024-05-27T09:36:22.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.587+0000 7f10fce00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f8198af0 con 0x7f10f81004c0 2024-05-27T09:36:22.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.587+0000 7f10f5e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10f80ffbb0 0x7f10f8197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:22.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.587+0000 7f10f5e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10f80ffbb0 0x7f10f8197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55388/0 (socket says 172.21.15.119:55388) 2024-05-27T09:36:22.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.587+0000 7f10f5e00700 1 -- 172.21.15.119:0/1835890217 learned_addr learned my addr 172.21.15.119:0/1835890217 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:36:22.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.588+0000 7f10f5400700 1 --2- 172.21.15.119:0/1835890217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10f81004c0 0x7f10f8198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:22.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.588+0000 7f10f5e00700 1 -- 172.21.15.119:0/1835890217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10f81004c0 msgr2=0x7f10f8198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:22.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.588+0000 7f10f5e00700 1 --2- 172.21.15.119:0/1835890217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10f81004c0 0x7f10f8198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:22.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.588+0000 7f10f5e00700 1 -- 172.21.15.119:0/1835890217 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e80095f0 con 0x7f10f80ffbb0 2024-05-27T09:36:22.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.588+0000 7f10f5e00700 1 --2- 172.21.15.119:0/1835890217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10f80ffbb0 0x7f10f8197e20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f10e800b870 tx=0x7f10e80056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:22.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.588+0000 7f10eea00700 1 -- 172.21.15.119:0/1835890217 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10e801d070 con 0x7f10f80ffbb0 2024-05-27T09:36:22.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.589+0000 7f10eea00700 1 -- 172.21.15.119:0/1835890217 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f10e800f460 con 0x7f10f80ffbb0 2024-05-27T09:36:22.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.589+0000 7f10eea00700 1 -- 172.21.15.119:0/1835890217 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10e8021640 con 0x7f10f80ffbb0 2024-05-27T09:36:22.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.589+0000 7f10fce00700 1 -- 172.21.15.119:0/1835890217 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f819c940 con 0x7f10f80ffbb0 2024-05-27T09:36:22.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.589+0000 7f10fce00700 1 -- 172.21.15.119:0/1835890217 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f819ce90 con 0x7f10f80ffbb0 2024-05-27T09:36:22.592 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.590+0000 7f10fce00700 1 -- 172.21.15.119:0/1835890217 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f8109f10 con 0x7f10f80ffbb0 2024-05-27T09:36:22.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.591+0000 7f10eea00700 1 -- 172.21.15.119:0/1835890217 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f10e800fa40 con 0x7f10f80ffbb0 2024-05-27T09:36:22.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.592+0000 7f10eea00700 1 --2- 172.21.15.119:0/1835890217 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f10dc078a40 0x7f10dc07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:22.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.592+0000 7f10eea00700 1 -- 172.21.15.119:0/1835890217 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f10e809c0a0 con 0x7f10f80ffbb0 2024-05-27T09:36:22.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.593+0000 7f10f5400700 1 --2- 172.21.15.119:0/1835890217 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f10dc078a40 0x7f10dc07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:22.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.593+0000 7f10f5400700 1 --2- 172.21.15.119:0/1835890217 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f10dc078a40 0x7f10dc07aef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f10e4005ea0 tx=0x7f10e4005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:22.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.598+0000 7f10eea00700 1 -- 172.21.15.119:0/1835890217 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f10e8063c80 con 0x7f10f80ffbb0 2024-05-27T09:36:22.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:22 smithi138 ceph-mon[70004]: pgmap v60: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:22.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:22 smithi138 ceph-mon[70004]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:36:22.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:22 smithi138 ceph-mon[70004]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:36:22.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:22 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:22.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:22 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:22.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:22 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:36:22.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:22 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:36:22.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:22 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:22.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:22 smithi119 ceph-mon[88269]: pgmap v60: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:22.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:22 smithi119 ceph-mon[88269]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:36:22.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:22 smithi119 ceph-mon[88269]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:36:22.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:22 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:22.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:22 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:22.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:22 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:36:22.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:22 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:36:22.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:22 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:22.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.935+0000 7f10fce00700 1 -- 172.21.15.119:0/1835890217 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f10f819cad0 con 0x7f10f80ffbb0 2024-05-27T09:36:22.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.936+0000 7f10eea00700 1 -- 172.21.15.119:0/1835890217 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f10f819cad0 con 0x7f10f80ffbb0 2024-05-27T09:36:22.938 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:36:22.939 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:36:22.939 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:36:22.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:36:22.939 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:36:22.939 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:36:22.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:36:22.939 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:36:22.939 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-05-27T09:36:22.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:36:22.939 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:36:22.939 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-05-27T09:36:22.939 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:36:22.939 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:36:22.939 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 10, 2024-05-27T09:36:22.940 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 4 2024-05-27T09:36:22.940 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:36:22.940 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:36:22.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.940+0000 7f10fce00700 1 -- 172.21.15.119:0/1835890217 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f10dc078a40 msgr2=0x7f10dc07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:22.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.940+0000 7f10fce00700 1 --2- 172.21.15.119:0/1835890217 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f10dc078a40 0x7f10dc07aef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f10e4005ea0 tx=0x7f10e4005e30 comp rx=0 tx=0).stop 2024-05-27T09:36:22.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.940+0000 7f10fce00700 1 -- 172.21.15.119:0/1835890217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10f80ffbb0 msgr2=0x7f10f8197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:22.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.940+0000 7f10fce00700 1 --2- 172.21.15.119:0/1835890217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10f80ffbb0 0x7f10f8197e20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f10e800b870 tx=0x7f10e80056a0 comp rx=0 tx=0).stop 2024-05-27T09:36:22.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.940+0000 7f10fce00700 1 -- 172.21.15.119:0/1835890217 shutdown_connections 2024-05-27T09:36:22.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.940+0000 7f10fce00700 1 --2- 172.21.15.119:0/1835890217 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f10dc078a40 0x7f10dc07aef0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:22.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.940+0000 7f10fce00700 1 --2- 172.21.15.119:0/1835890217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10f80ffbb0 0x7f10f8197e20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:22.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.940+0000 7f10fce00700 1 --2- 172.21.15.119:0/1835890217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10f81004c0 0x7f10f8198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:22.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.940+0000 7f10fce00700 1 -- 172.21.15.119:0/1835890217 >> 172.21.15.119:0/1835890217 conn(0x7f10f8074f50 msgr2=0x7f10f80fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:22.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.940+0000 7f10fce00700 1 -- 172.21.15.119:0/1835890217 shutdown_connections 2024-05-27T09:36:22.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:22.940+0000 7f10fce00700 1 -- 172.21.15.119:0/1835890217 wait complete. 2024-05-27T09:36:23.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.092+0000 7efecb200700 1 -- 172.21.15.119:0/271889328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec41003b0 msgr2=0x7efec4100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:23.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.092+0000 7efecb200700 1 --2- 172.21.15.119:0/271889328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec41003b0 0x7efec4100780 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efeb4009910 tx=0x7efeb4009c20 comp rx=0 tx=0).stop 2024-05-27T09:36:23.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.092+0000 7efecb200700 1 -- 172.21.15.119:0/271889328 shutdown_connections 2024-05-27T09:36:23.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.092+0000 7efecb200700 1 --2- 172.21.15.119:0/271889328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efec4100d50 0x7efec4104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:23.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.092+0000 7efecb200700 1 --2- 172.21.15.119:0/271889328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec41003b0 0x7efec4100780 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:23.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.092+0000 7efecb200700 1 -- 172.21.15.119:0/271889328 >> 172.21.15.119:0/271889328 conn(0x7efec40fbc70 msgr2=0x7efec40fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:23.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.092+0000 7efecb200700 1 -- 172.21.15.119:0/271889328 shutdown_connections 2024-05-27T09:36:23.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.092+0000 7efecb200700 1 -- 172.21.15.119:0/271889328 wait complete. 2024-05-27T09:36:23.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.094+0000 7efecb200700 1 Processor -- start 2024-05-27T09:36:23.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.094+0000 7efecb200700 1 -- start start 2024-05-27T09:36:23.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.094+0000 7efecb200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efec41003b0 0x7efec419bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:23.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.094+0000 7efecb200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec4100d50 0x7efec419bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:23.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.094+0000 7efecb200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec419c680 con 0x7efec4100d50 2024-05-27T09:36:23.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.094+0000 7efecb200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec4195b30 con 0x7efec41003b0 2024-05-27T09:36:23.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.095+0000 7efec3400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec4100d50 0x7efec419bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:23.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.095+0000 7efec3400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec4100d50 0x7efec419bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55400/0 (socket says 172.21.15.119:55400) 2024-05-27T09:36:23.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.095+0000 7efec3400700 1 -- 172.21.15.119:0/3139539246 learned_addr learned my addr 172.21.15.119:0/3139539246 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:36:23.096 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.095+0000 7efec3e00700 1 --2- 172.21.15.119:0/3139539246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efec41003b0 0x7efec419bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:23.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.095+0000 7efec3400700 1 -- 172.21.15.119:0/3139539246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efec41003b0 msgr2=0x7efec419bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:23.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.095+0000 7efec3400700 1 --2- 172.21.15.119:0/3139539246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efec41003b0 0x7efec419bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:23.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.095+0000 7efec3400700 1 -- 172.21.15.119:0/3139539246 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb40095f0 con 0x7efec4100d50 2024-05-27T09:36:23.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.096+0000 7efec3400700 1 --2- 172.21.15.119:0/3139539246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec4100d50 0x7efec419bff0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efeb000d900 tx=0x7efeb000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:23.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.096+0000 7efec0c00700 1 -- 172.21.15.119:0/3139539246 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efeb0009900 con 0x7efec4100d50 2024-05-27T09:36:23.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.096+0000 7efec0c00700 1 -- 172.21.15.119:0/3139539246 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efeb0010460 con 0x7efec4100d50 2024-05-27T09:36:23.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.096+0000 7efecb200700 1 -- 172.21.15.119:0/3139539246 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec4195e10 con 0x7efec4100d50 2024-05-27T09:36:23.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.096+0000 7efec0c00700 1 -- 172.21.15.119:0/3139539246 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efeb000f5f0 con 0x7efec4100d50 2024-05-27T09:36:23.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.096+0000 7efecb200700 1 -- 172.21.15.119:0/3139539246 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec4196360 con 0x7efec4100d50 2024-05-27T09:36:23.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.097+0000 7efecb200700 1 -- 172.21.15.119:0/3139539246 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec41086c0 con 0x7efec4100d50 2024-05-27T09:36:23.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.099+0000 7efec0c00700 1 -- 172.21.15.119:0/3139539246 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7efeb00105d0 con 0x7efec4100d50 2024-05-27T09:36:23.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.100+0000 7efec0c00700 1 --2- 172.21.15.119:0/3139539246 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efea8078a40 0x7efea807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:23.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.100+0000 7efec0c00700 1 -- 172.21.15.119:0/3139539246 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7efeb009ab10 con 0x7efec4100d50 2024-05-27T09:36:23.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.100+0000 7efec3e00700 1 --2- 172.21.15.119:0/3139539246 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efea8078a40 0x7efea807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:23.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.101+0000 7efec3e00700 1 --2- 172.21.15.119:0/3139539246 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efea8078a40 0x7efea807aef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efeb4006010 tx=0x7efeb401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:23.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.105+0000 7efec0c00700 1 -- 172.21.15.119:0/3139539246 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7efeb0062770 con 0x7efec4100d50 2024-05-27T09:36:23.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.394+0000 7efecb200700 1 -- 172.21.15.119:0/3139539246 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efec404ea50 con 0x7efec4100d50 2024-05-27T09:36:23.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.395+0000 7efec0c00700 1 -- 172.21.15.119:0/3139539246 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1889 (secure 0 0 0) 0x7efeb0062110 con 0x7efec4100d50 2024-05-27T09:36:23.397 INFO:teuthology.orchestra.run.smithi119.stdout:e22 2024-05-27T09:36:23.397 INFO:teuthology.orchestra.run.smithi119.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T09:36:23.397 INFO:teuthology.orchestra.run.smithi119.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:36:23.397 INFO:teuthology.orchestra.run.smithi119.stdout:legacy client fscid: 1 2024-05-27T09:36:23.397 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:36:23.397 INFO:teuthology.orchestra.run.smithi119.stdout:Filesystem 'cephfs' (1) 2024-05-27T09:36:23.397 INFO:teuthology.orchestra.run.smithi119.stdout:fs_name cephfs 2024-05-27T09:36:23.398 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 22 2024-05-27T09:36:23.398 INFO:teuthology.orchestra.run.smithi119.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-27T09:36:23.398 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-05-27T09:25:53.387388+0000 2024-05-27T09:36:23.398 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2024-05-27T09:34:01.250304+0000 2024-05-27T09:36:23.398 INFO:teuthology.orchestra.run.smithi119.stdout:tableserver 0 2024-05-27T09:36:23.398 INFO:teuthology.orchestra.run.smithi119.stdout:root 0 2024-05-27T09:36:23.398 INFO:teuthology.orchestra.run.smithi119.stdout:session_timeout 60 2024-05-27T09:36:23.398 INFO:teuthology.orchestra.run.smithi119.stdout:session_autoclose 300 2024-05-27T09:36:23.398 INFO:teuthology.orchestra.run.smithi119.stdout:max_file_size 1099511627776 2024-05-27T09:36:23.398 INFO:teuthology.orchestra.run.smithi119.stdout:max_xattr_size 65536 2024-05-27T09:36:23.398 INFO:teuthology.orchestra.run.smithi119.stdout:required_client_features {} 2024-05-27T09:36:23.398 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure 0 2024-05-27T09:36:23.398 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure_osd_epoch 46 2024-05-27T09:36:23.399 INFO:teuthology.orchestra.run.smithi119.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:36:23.399 INFO:teuthology.orchestra.run.smithi119.stdout:max_mds 2 2024-05-27T09:36:23.399 INFO:teuthology.orchestra.run.smithi119.stdout:in 0,1 2024-05-27T09:36:23.399 INFO:teuthology.orchestra.run.smithi119.stdout:up {0=14472,1=24241} 2024-05-27T09:36:23.399 INFO:teuthology.orchestra.run.smithi119.stdout:failed 2024-05-27T09:36:23.399 INFO:teuthology.orchestra.run.smithi119.stdout:damaged 2024-05-27T09:36:23.399 INFO:teuthology.orchestra.run.smithi119.stdout:stopped 2024-05-27T09:36:23.399 INFO:teuthology.orchestra.run.smithi119.stdout:data_pools [3] 2024-05-27T09:36:23.399 INFO:teuthology.orchestra.run.smithi119.stdout:metadata_pool 2 2024-05-27T09:36:23.399 INFO:teuthology.orchestra.run.smithi119.stdout:inline_data enabled 2024-05-27T09:36:23.399 INFO:teuthology.orchestra.run.smithi119.stdout:balancer 2024-05-27T09:36:23.399 INFO:teuthology.orchestra.run.smithi119.stdout:bal_rank_mask -1 2024-05-27T09:36:23.399 INFO:teuthology.orchestra.run.smithi119.stdout:standby_count_wanted 1 2024-05-27T09:36:23.399 INFO:teuthology.orchestra.run.smithi119.stdout:qdb_cluster leader: 0 members: 2024-05-27T09:36:23.400 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.ppmzzk{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:36:23.400 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.ndfkhd{1:24241} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/897058594,v1:172.21.15.138:6825/897058594] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:36:23.400 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:36:23.400 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:36:23.400 INFO:teuthology.orchestra.run.smithi119.stdout:Standby daemons: 2024-05-27T09:36:23.400 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:36:23.400 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.nghcey{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.119:6826/413736191,v1:172.21.15.119:6827/413736191] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:36:23.400 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.alszpk{-1:24253} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/2345607053,v1:172.21.15.138:6827/2345607053] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:36:23.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.399+0000 7efecb200700 1 -- 172.21.15.119:0/3139539246 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efea8078a40 msgr2=0x7efea807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:23.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.399+0000 7efecb200700 1 --2- 172.21.15.119:0/3139539246 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efea8078a40 0x7efea807aef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efeb4006010 tx=0x7efeb401a040 comp rx=0 tx=0).stop 2024-05-27T09:36:23.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.399+0000 7efecb200700 1 -- 172.21.15.119:0/3139539246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec4100d50 msgr2=0x7efec419bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:23.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.399+0000 7efecb200700 1 --2- 172.21.15.119:0/3139539246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec4100d50 0x7efec419bff0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efeb000d900 tx=0x7efeb000dcc0 comp rx=0 tx=0).stop 2024-05-27T09:36:23.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.399+0000 7efecb200700 1 -- 172.21.15.119:0/3139539246 shutdown_connections 2024-05-27T09:36:23.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.399+0000 7efecb200700 1 --2- 172.21.15.119:0/3139539246 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efea8078a40 0x7efea807aef0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:23.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.399+0000 7efecb200700 1 --2- 172.21.15.119:0/3139539246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efec41003b0 0x7efec419bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:23.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.399+0000 7efecb200700 1 --2- 172.21.15.119:0/3139539246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec4100d50 0x7efec419bff0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:23.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.399+0000 7efecb200700 1 -- 172.21.15.119:0/3139539246 >> 172.21.15.119:0/3139539246 conn(0x7efec40fbc70 msgr2=0x7efec40fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:23.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.399+0000 7efecb200700 1 -- 172.21.15.119:0/3139539246 shutdown_connections 2024-05-27T09:36:23.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.399+0000 7efecb200700 1 -- 172.21.15.119:0/3139539246 wait complete. 2024-05-27T09:36:23.402 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 22 2024-05-27T09:36:23.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.552+0000 7fea0d800700 1 -- 172.21.15.119:0/551734426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea08073a90 msgr2=0x7fea0810c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:23.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.552+0000 7fea0d800700 1 --2- 172.21.15.119:0/551734426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea08073a90 0x7fea0810c2b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009910 tx=0x7fe9f4009c20 comp rx=0 tx=0).stop 2024-05-27T09:36:23.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.552+0000 7fea0d800700 1 -- 172.21.15.119:0/551734426 shutdown_connections 2024-05-27T09:36:23.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.552+0000 7fea0d800700 1 --2- 172.21.15.119:0/551734426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea08073a90 0x7fea0810c2b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:23.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.552+0000 7fea0d800700 1 --2- 172.21.15.119:0/551734426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea080730f0 0x7fea080734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:23.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.552+0000 7fea0d800700 1 -- 172.21.15.119:0/551734426 >> 172.21.15.119:0/551734426 conn(0x7fea080fbaf0 msgr2=0x7fea080fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:23.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.552+0000 7fea0d800700 1 -- 172.21.15.119:0/551734426 shutdown_connections 2024-05-27T09:36:23.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.552+0000 7fea0d800700 1 -- 172.21.15.119:0/551734426 wait complete. 2024-05-27T09:36:23.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.554+0000 7fea0d800700 1 Processor -- start 2024-05-27T09:36:23.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.554+0000 7fea0d800700 1 -- start start 2024-05-27T09:36:23.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.554+0000 7fea0d800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea080730f0 0x7fea0810be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:23.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.554+0000 7fea0d800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea08073a90 0x7fea08102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:23.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.554+0000 7fea0d800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0810c650 con 0x7fea08073a90 2024-05-27T09:36:23.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.554+0000 7fea0d800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea081032d0 con 0x7fea080730f0 2024-05-27T09:36:23.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.554+0000 7fea07400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea08073a90 0x7fea08102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:23.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.554+0000 7fea07400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea08073a90 0x7fea08102d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55410/0 (socket says 172.21.15.119:55410) 2024-05-27T09:36:23.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.555+0000 7fea07400700 1 -- 172.21.15.119:0/408429466 learned_addr learned my addr 172.21.15.119:0/408429466 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:36:23.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.555+0000 7fea07e00700 1 --2- 172.21.15.119:0/408429466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea080730f0 0x7fea0810be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:23.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.555+0000 7fea07400700 1 -- 172.21.15.119:0/408429466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea080730f0 msgr2=0x7fea0810be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:23.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.555+0000 7fea07400700 1 --2- 172.21.15.119:0/408429466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea080730f0 0x7fea0810be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:23.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.555+0000 7fea07400700 1 -- 172.21.15.119:0/408429466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f40095f0 con 0x7fea08073a90 2024-05-27T09:36:23.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.555+0000 7fea07400700 1 --2- 172.21.15.119:0/408429466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea08073a90 0x7fea08102d90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4005b40 tx=0x7fe9f40056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:23.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.556+0000 7fea04c00700 1 -- 172.21.15.119:0/408429466 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9f401d070 con 0x7fea08073a90 2024-05-27T09:36:23.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.556+0000 7fea04c00700 1 -- 172.21.15.119:0/408429466 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe9f400f460 con 0x7fea08073a90 2024-05-27T09:36:23.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.556+0000 7fea0d800700 1 -- 172.21.15.119:0/408429466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea081034f0 con 0x7fea08073a90 2024-05-27T09:36:23.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.556+0000 7fea04c00700 1 -- 172.21.15.119:0/408429466 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9f4021640 con 0x7fea08073a90 2024-05-27T09:36:23.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.556+0000 7fea0d800700 1 -- 172.21.15.119:0/408429466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea08103a40 con 0x7fea08073a90 2024-05-27T09:36:23.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.558+0000 7fea0d800700 1 -- 172.21.15.119:0/408429466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea0804f2a0 con 0x7fea08073a90 2024-05-27T09:36:23.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.558+0000 7fea04c00700 1 -- 172.21.15.119:0/408429466 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fe9f400f5d0 con 0x7fea08073a90 2024-05-27T09:36:23.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.559+0000 7fea04c00700 1 --2- 172.21.15.119:0/408429466 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe9ec078a40 0x7fe9ec07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:23.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.560+0000 7fea04c00700 1 -- 172.21.15.119:0/408429466 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe9f409c080 con 0x7fea08073a90 2024-05-27T09:36:23.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.560+0000 7fea07e00700 1 --2- 172.21.15.119:0/408429466 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe9ec078a40 0x7fe9ec07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:23.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.561+0000 7fea07e00700 1 --2- 172.21.15.119:0/408429466 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe9ec078a40 0x7fe9ec07aef0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8006fd0 tx=0x7fe9f8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:23.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.565+0000 7fea04c00700 1 -- 172.21.15.119:0/408429466 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fe9f4063d00 con 0x7fea08073a90 2024-05-27T09:36:23.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:23 smithi138 ceph-mon[70004]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:36:23.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:23 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:36:23.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:23 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:36:23.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:23 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:36:23.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:23 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:36:23.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:23 smithi138 ceph-mon[70004]: Upgrade: Setting container_image for all crash 2024-05-27T09:36:23.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:23 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:23.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:23 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi119"}]: dispatch 2024-05-27T09:36:23.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:23 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi119"}]': finished 2024-05-27T09:36:23.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:23 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi138"}]: dispatch 2024-05-27T09:36:23.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:23 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi138"}]': finished 2024-05-27T09:36:23.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:23 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-27T09:36:23.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:23 smithi138 ceph-mon[70004]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-27T09:36:23.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:23 smithi138 ceph-mon[70004]: Upgrade: osd.0 is safe to restart 2024-05-27T09:36:23.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:23 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/1835890217' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:36:23.684 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:23 smithi119 ceph-mon[88269]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:36:23.684 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:23 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:36:23.685 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:23 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:36:23.685 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:23 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:36:23.685 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:23 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:36:23.685 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:23 smithi119 ceph-mon[88269]: Upgrade: Setting container_image for all crash 2024-05-27T09:36:23.685 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:23 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:23.685 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:23 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi119"}]: dispatch 2024-05-27T09:36:23.685 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:23 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi119"}]': finished 2024-05-27T09:36:23.685 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:23 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi138"}]: dispatch 2024-05-27T09:36:23.685 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:23 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi138"}]': finished 2024-05-27T09:36:23.685 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:23 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-27T09:36:23.685 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:23 smithi119 ceph-mon[88269]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-27T09:36:23.685 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:23 smithi119 ceph-mon[88269]: Upgrade: osd.0 is safe to restart 2024-05-27T09:36:23.685 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:23 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/1835890217' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:36:23.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.822+0000 7fea0d800700 1 -- 172.21.15.119:0/408429466 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fea08107900 con 0x7fe9ec078a40 2024-05-27T09:36:23.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.825+0000 7fea04c00700 1 -- 172.21.15.119:0/408429466 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fea08107900 con 0x7fe9ec078a40 2024-05-27T09:36:23.826 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:36:23.826 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:36:23.826 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:36:23.826 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-27T09:36:23.827 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [ 2024-05-27T09:36:23.827 INFO:teuthology.orchestra.run.smithi119.stdout: "crash", 2024-05-27T09:36:23.827 INFO:teuthology.orchestra.run.smithi119.stdout: "mon", 2024-05-27T09:36:23.827 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr" 2024-05-27T09:36:23.827 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-05-27T09:36:23.827 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "6/23 daemons upgraded", 2024-05-27T09:36:23.827 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "Currently upgrading osd daemons", 2024-05-27T09:36:23.827 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:36:23.827 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:36:23.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.829+0000 7fea0d800700 1 -- 172.21.15.119:0/408429466 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe9ec078a40 msgr2=0x7fe9ec07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:23.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.829+0000 7fea0d800700 1 --2- 172.21.15.119:0/408429466 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe9ec078a40 0x7fe9ec07aef0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8006fd0 tx=0x7fe9f8008040 comp rx=0 tx=0).stop 2024-05-27T09:36:23.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.829+0000 7fea0d800700 1 -- 172.21.15.119:0/408429466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea08073a90 msgr2=0x7fea08102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:23.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.829+0000 7fea0d800700 1 --2- 172.21.15.119:0/408429466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea08073a90 0x7fea08102d90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4005b40 tx=0x7fe9f40056a0 comp rx=0 tx=0).stop 2024-05-27T09:36:23.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.829+0000 7fea0d800700 1 -- 172.21.15.119:0/408429466 shutdown_connections 2024-05-27T09:36:23.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.829+0000 7fea0d800700 1 --2- 172.21.15.119:0/408429466 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe9ec078a40 0x7fe9ec07aef0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:23.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.829+0000 7fea0d800700 1 --2- 172.21.15.119:0/408429466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea080730f0 0x7fea0810be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:23.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.829+0000 7fea0d800700 1 --2- 172.21.15.119:0/408429466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea08073a90 0x7fea08102d90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:23.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.829+0000 7fea0d800700 1 -- 172.21.15.119:0/408429466 >> 172.21.15.119:0/408429466 conn(0x7fea080fbaf0 msgr2=0x7fea08196a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:23.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.829+0000 7fea0d800700 1 -- 172.21.15.119:0/408429466 shutdown_connections 2024-05-27T09:36:23.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.829+0000 7fea0d800700 1 -- 172.21.15.119:0/408429466 wait complete. 2024-05-27T09:36:23.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.984+0000 7fe4fcc00700 1 -- 172.21.15.119:0/3636788011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f810b700 msgr2=0x7fe4f810bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:23.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.984+0000 7fe4fcc00700 1 --2- 172.21.15.119:0/3636788011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f810b700 0x7fe4f810bad0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009910 tx=0x7fe4e8009c20 comp rx=0 tx=0).stop 2024-05-27T09:36:23.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.985+0000 7fe4fcc00700 1 -- 172.21.15.119:0/3636788011 shutdown_connections 2024-05-27T09:36:23.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.985+0000 7fe4fcc00700 1 --2- 172.21.15.119:0/3636788011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4f810c010 0x7fe4f8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:23.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.985+0000 7fe4fcc00700 1 --2- 172.21.15.119:0/3636788011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f810b700 0x7fe4f810bad0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:23.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.985+0000 7fe4fcc00700 1 -- 172.21.15.119:0/3636788011 >> 172.21.15.119:0/3636788011 conn(0x7fe4f8076000 msgr2=0x7fe4f8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:23.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.985+0000 7fe4fcc00700 1 -- 172.21.15.119:0/3636788011 shutdown_connections 2024-05-27T09:36:23.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.985+0000 7fe4fcc00700 1 -- 172.21.15.119:0/3636788011 wait complete. 2024-05-27T09:36:23.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.986+0000 7fe4fcc00700 1 Processor -- start 2024-05-27T09:36:23.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.986+0000 7fe4fcc00700 1 -- start start 2024-05-27T09:36:23.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.986+0000 7fe4fcc00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f810b700 0x7fe4f819c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:23.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.986+0000 7fe4fcc00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4f810c010 0x7fe4f819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:23.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.986+0000 7fe4fcc00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f819ced0 con 0x7fe4f810b700 2024-05-27T09:36:23.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.986+0000 7fe4fcc00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f81a0c60 con 0x7fe4f810c010 2024-05-27T09:36:23.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.987+0000 7fe4f5e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f810b700 0x7fe4f819c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:23.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.987+0000 7fe4f5e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f810b700 0x7fe4f819c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55426/0 (socket says 172.21.15.119:55426) 2024-05-27T09:36:23.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.987+0000 7fe4f5e00700 1 -- 172.21.15.119:0/3258702914 learned_addr learned my addr 172.21.15.119:0/3258702914 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:36:23.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.987+0000 7fe4efe00700 1 --2- 172.21.15.119:0/3258702914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4f810c010 0x7fe4f819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:23.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.987+0000 7fe4f5e00700 1 -- 172.21.15.119:0/3258702914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4f810c010 msgr2=0x7fe4f819c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:23.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.987+0000 7fe4f5e00700 1 --2- 172.21.15.119:0/3258702914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4f810c010 0x7fe4f819c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:23.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.987+0000 7fe4f5e00700 1 -- 172.21.15.119:0/3258702914 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e80095f0 con 0x7fe4f810b700 2024-05-27T09:36:23.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.988+0000 7fe4f5e00700 1 --2- 172.21.15.119:0/3258702914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f810b700 0x7fe4f819c2b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe4e80092e0 tx=0x7fe4e800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:23.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.988+0000 7fe4eea00700 1 -- 172.21.15.119:0/3258702914 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4e801d070 con 0x7fe4f810b700 2024-05-27T09:36:23.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.988+0000 7fe4eea00700 1 -- 172.21.15.119:0/3258702914 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4e8005e00 con 0x7fe4f810b700 2024-05-27T09:36:23.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.988+0000 7fe4fcc00700 1 -- 172.21.15.119:0/3258702914 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f81a0ee0 con 0x7fe4f810b700 2024-05-27T09:36:23.989 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.988+0000 7fe4eea00700 1 -- 172.21.15.119:0/3258702914 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4e8017400 con 0x7fe4f810b700 2024-05-27T09:36:23.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.989+0000 7fe4fcc00700 1 -- 172.21.15.119:0/3258702914 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f81a1450 con 0x7fe4f810b700 2024-05-27T09:36:23.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.990+0000 7fe4fcc00700 1 -- 172.21.15.119:0/3258702914 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f804ea50 con 0x7fe4f810b700 2024-05-27T09:36:23.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.991+0000 7fe4eea00700 1 -- 172.21.15.119:0/3258702914 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fe4e800f460 con 0x7fe4f810b700 2024-05-27T09:36:23.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.992+0000 7fe4eea00700 1 --2- 172.21.15.119:0/3258702914 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe4dc078a40 0x7fe4dc07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:23.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.992+0000 7fe4eea00700 1 -- 172.21.15.119:0/3258702914 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe4e809c2a0 con 0x7fe4f810b700 2024-05-27T09:36:23.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.993+0000 7fe4efe00700 1 --2- 172.21.15.119:0/3258702914 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe4dc078a40 0x7fe4dc07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:23.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.993+0000 7fe4efe00700 1 --2- 172.21.15.119:0/3258702914 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe4dc078a40 0x7fe4dc07aef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0005950 tx=0x7fe4e00058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:23.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:23.998+0000 7fe4eea00700 1 -- 172.21.15.119:0/3258702914 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fe4e8063e80 con 0x7fe4f810b700 2024-05-27T09:36:24.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:24.348+0000 7fe4fcc00700 1 -- 172.21.15.119:0/3258702914 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe4f819d610 con 0x7fe4f810b700 2024-05-27T09:36:24.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:24.348+0000 7fe4eea00700 1 -- 172.21.15.119:0/3258702914 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fe4e8063820 con 0x7fe4f810b700 2024-05-27T09:36:24.349 INFO:teuthology.orchestra.run.smithi119.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-27T09:36:24.349 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:36:24.349 INFO:teuthology.orchestra.run.smithi119.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:36:24.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:24.351+0000 7fe4fcc00700 1 -- 172.21.15.119:0/3258702914 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe4dc078a40 msgr2=0x7fe4dc07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:24.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:24.351+0000 7fe4fcc00700 1 --2- 172.21.15.119:0/3258702914 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe4dc078a40 0x7fe4dc07aef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0005950 tx=0x7fe4e00058e0 comp rx=0 tx=0).stop 2024-05-27T09:36:24.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:24.351+0000 7fe4fcc00700 1 -- 172.21.15.119:0/3258702914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f810b700 msgr2=0x7fe4f819c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:24.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:24.351+0000 7fe4fcc00700 1 --2- 172.21.15.119:0/3258702914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f810b700 0x7fe4f819c2b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe4e80092e0 tx=0x7fe4e800be20 comp rx=0 tx=0).stop 2024-05-27T09:36:24.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:24.351+0000 7fe4fcc00700 1 -- 172.21.15.119:0/3258702914 shutdown_connections 2024-05-27T09:36:24.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:24.351+0000 7fe4fcc00700 1 --2- 172.21.15.119:0/3258702914 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe4dc078a40 0x7fe4dc07aef0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:24.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:24.351+0000 7fe4fcc00700 1 --2- 172.21.15.119:0/3258702914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4f810b700 0x7fe4f819c2b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:24.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:24.351+0000 7fe4fcc00700 1 --2- 172.21.15.119:0/3258702914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4f810c010 0x7fe4f819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:24.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:24.351+0000 7fe4fcc00700 1 -- 172.21.15.119:0/3258702914 >> 172.21.15.119:0/3258702914 conn(0x7fe4f8076000 msgr2=0x7fe4f8076880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:24.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:24.351+0000 7fe4fcc00700 1 -- 172.21.15.119:0/3258702914 shutdown_connections 2024-05-27T09:36:24.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:24.351+0000 7fe4fcc00700 1 -- 172.21.15.119:0/3258702914 wait complete. 2024-05-27T09:36:24.645 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:24 smithi119 ceph-mon[88269]: pgmap v61: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:24.645 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:24 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/3139539246' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:36:24.645 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:24 smithi119 ceph-mon[88269]: from='client.34236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:36:24.645 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:24 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/3258702914' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:36:24.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:24 smithi138 ceph-mon[70004]: pgmap v61: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:24.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:24 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/3139539246' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:36:24.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:24 smithi138 ceph-mon[70004]: from='client.34236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:36:24.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:24 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/3258702914' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:36:25.875 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:25 smithi119 ceph-mon[88269]: Upgrade: Updating osd.0 2024-05-27T09:36:25.875 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:25 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:25.875 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:25 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-27T09:36:25.875 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:25 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:36:25.876 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:25 smithi119 ceph-mon[88269]: Deploying daemon osd.0 on smithi119 2024-05-27T09:36:26.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:25 smithi138 ceph-mon[70004]: Upgrade: Updating osd.0 2024-05-27T09:36:26.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:25 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:26.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:25 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-27T09:36:26.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:25 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:36:26.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:25 smithi138 ceph-mon[70004]: Deploying daemon osd.0 on smithi119 2024-05-27T09:36:26.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:26 smithi119 ceph-mon[88269]: pgmap v62: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:27.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:26 smithi138 ceph-mon[70004]: pgmap v62: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:28.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:28 smithi119 ceph-mon[88269]: pgmap v63: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:28.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:36:28.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:28 smithi138 ceph-mon[70004]: pgmap v63: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:28.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:28 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:36:29.477 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:29 smithi119 systemd[1]: Stopping Ceph osd.0 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:36:30.248 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:29 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0[43827]: 2024-05-27T09:36:29.920+0000 7fadf9000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T09:36:30.248 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:29 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0[43827]: 2024-05-27T09:36:29.920+0000 7fadf9000700 -1 osd.0 49 *** Got signal Terminated *** 2024-05-27T09:36:30.248 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:29 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0[43827]: 2024-05-27T09:36:29.920+0000 7fadf9000700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T09:36:30.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:30 smithi138 ceph-mon[70004]: pgmap v64: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:30.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:30 smithi138 ceph-mon[70004]: osd.0 marked itself down and dead 2024-05-27T09:36:30.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:30 smithi119 ceph-mon[88269]: pgmap v64: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:30.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:30 smithi119 ceph-mon[88269]: osd.0 marked itself down and dead 2024-05-27T09:36:31.365 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:31 smithi119 podman[99570]: 2024-05-27 09:36:31.083787625 +0000 UTC m=+1.298846437 container died 825bd139250811ed56f217e88a93a88c85051c23342986a69957424b03c33c73 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20231212, RELEASE=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, ceph=True) 2024-05-27T09:36:31.622 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:31 smithi119 ceph-mon[88269]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-27T09:36:31.623 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:31 smithi119 ceph-mon[88269]: osdmap e50: 6 total, 5 up, 6 in 2024-05-27T09:36:31.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:31 smithi138 ceph-mon[70004]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-27T09:36:31.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:31 smithi138 ceph-mon[70004]: osdmap e50: 6 total, 5 up, 6 in 2024-05-27T09:36:31.998 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:31 smithi119 podman[99570]: 2024-05-27 09:36:31.622292223 +0000 UTC m=+1.837351035 container cleanup 825bd139250811ed56f217e88a93a88c85051c23342986a69957424b03c33c73 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-05-27T09:36:31.998 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:31 smithi119 bash[99570]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0 2024-05-27T09:36:31.998 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:31 smithi119 podman[99587]: 2024-05-27 09:36:31.824004853 +0000 UTC m=+0.720168766 container remove 825bd139250811ed56f217e88a93a88c85051c23342986a69957424b03c33c73 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-27T09:36:32.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:32 smithi138 ceph-mon[70004]: pgmap v66: 65 pgs: 9 stale+active+clean, 56 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:32.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:32 smithi138 ceph-mon[70004]: osdmap e51: 6 total, 5 up, 6 in 2024-05-27T09:36:32.639 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:32 smithi119 ceph-mon[88269]: pgmap v66: 65 pgs: 9 stale+active+clean, 56 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:32.639 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:32 smithi119 ceph-mon[88269]: osdmap e51: 6 total, 5 up, 6 in 2024-05-27T09:36:32.947 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:32 smithi119 podman[99674]: 2024-05-27 09:36:32.543748667 +0000 UTC m=+0.031519189 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:36:32.947 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:32 smithi119 podman[99674]: 2024-05-27 09:36:32.651462719 +0000 UTC m=+0.139233265 container create 5e65ad216787af18a5bfbc57b5dcbd58d966e5d096fae485b49cfe86efabf825 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-deactivate, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521) 2024-05-27T09:36:32.947 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:32 smithi119 podman[99674]: 2024-05-27 09:36:32.945603099 +0000 UTC m=+0.433373715 container init 5e65ad216787af18a5bfbc57b5dcbd58d966e5d096fae485b49cfe86efabf825 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-27T09:36:33.248 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:32 smithi119 podman[99674]: 2024-05-27 09:36:32.95798338 +0000 UTC m=+0.445753934 container start 5e65ad216787af18a5bfbc57b5dcbd58d966e5d096fae485b49cfe86efabf825 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T09:36:33.248 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:33 smithi119 podman[99674]: 2024-05-27 09:36:33.027602001 +0000 UTC m=+0.515372586 container attach 5e65ad216787af18a5bfbc57b5dcbd58d966e5d096fae485b49cfe86efabf825 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_CLEAN=True) 2024-05-27T09:36:33.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:33 smithi119 ceph-mon[88269]: Health check failed: Degraded data redundancy: 13/381 objects degraded (3.412%), 3 pgs degraded (PG_DEGRADED) 2024-05-27T09:36:33.748 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:33 smithi119 podman[99674]: 2024-05-27 09:36:33.42155984 +0000 UTC m=+0.909330406 container died 5e65ad216787af18a5bfbc57b5dcbd58d966e5d096fae485b49cfe86efabf825 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-05-27T09:36:33.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:33 smithi138 ceph-mon[70004]: Health check failed: Degraded data redundancy: 13/381 objects degraded (3.412%), 3 pgs degraded (PG_DEGRADED) 2024-05-27T09:36:34.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:34 smithi119 ceph-mon[88269]: pgmap v68: 65 pgs: 3 active+undersized+degraded, 3 active+undersized, 54 active+clean, 5 stale+active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 13/381 objects degraded (3.412%) 2024-05-27T09:36:34.498 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:34 smithi119 podman[99699]: 2024-05-27 09:36:34.232052304 +0000 UTC m=+0.787204403 container remove 5e65ad216787af18a5bfbc57b5dcbd58d966e5d096fae485b49cfe86efabf825 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-27T09:36:34.498 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:34 smithi119 systemd[1]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.0.service: Deactivated successfully. 2024-05-27T09:36:34.498 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:34 smithi119 systemd[1]: Stopped Ceph osd.0 for b966b06e-1c09-11ef-bc9b-c7b262605968. 2024-05-27T09:36:34.498 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:34 smithi119 systemd[1]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.0.service: Consumed 1min 40.315s CPU time. 2024-05-27T09:36:34.715 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:34 smithi138 ceph-mon[70004]: pgmap v68: 65 pgs: 3 active+undersized+degraded, 3 active+undersized, 54 active+clean, 5 stale+active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 13/381 objects degraded (3.412%) 2024-05-27T09:36:34.998 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:34 smithi119 systemd[1]: Starting Ceph osd.0 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:36:35.313 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:35 smithi119 podman[99799]: 2024-05-27 09:36:34.939542207 +0000 UTC m=+0.034404064 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:36:35.313 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:35 smithi119 podman[99799]: 2024-05-27 09:36:35.049545697 +0000 UTC m=+0.144407587 container create 8232dee419d53e700a2730dbf4499064243c6ef95215ed0a032dda1ec3d5d502 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-activate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-27T09:36:35.313 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:35 smithi119 podman[99799]: 2024-05-27 09:36:35.311978785 +0000 UTC m=+0.406840719 container init 8232dee419d53e700a2730dbf4499064243c6ef95215ed0a032dda1ec3d5d502 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-activate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240521) 2024-05-27T09:36:35.748 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:35 smithi119 podman[99799]: 2024-05-27 09:36:35.323371447 +0000 UTC m=+0.418233360 container start 8232dee419d53e700a2730dbf4499064243c6ef95215ed0a032dda1ec3d5d502 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-activate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-05-27T09:36:35.748 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:35 smithi119 podman[99799]: 2024-05-27 09:36:35.418764823 +0000 UTC m=+0.513626730 container attach 8232dee419d53e700a2730dbf4499064243c6ef95215ed0a032dda1ec3d5d502 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-activate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-27T09:36:36.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:36 smithi119 ceph-mon[88269]: pgmap v69: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 76/381 objects degraded (19.948%) 2024-05-27T09:36:36.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:36 smithi138 ceph-mon[70004]: pgmap v69: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 76/381 objects degraded (19.948%) 2024-05-27T09:36:37.427 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:37 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-activate[99813]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:36:37.427 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:37 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-activate[99813]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:36:37.427 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:37 smithi119 bash[99799]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:36:37.427 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:37 smithi119 bash[99799]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:36:37.427 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:37 smithi119 bash[99799]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-27T09:36:37.427 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:37 smithi119 bash[99799]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-27T09:36:37.427 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:37 smithi119 bash[99799]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-27T09:36:37.427 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:37 smithi119 bash[99799]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-27T09:36:37.427 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:37 smithi119 bash[99799]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-05-27T09:36:37.427 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:37 smithi119 bash[99799]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-27T09:36:37.428 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:37 smithi119 bash[99799]: --> ceph-volume raw activate successful for osd ID: 0 2024-05-27T09:36:37.428 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:37 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-activate[99813]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-27T09:36:37.428 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:37 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-activate[99813]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-27T09:36:37.428 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:37 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-activate[99813]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-27T09:36:37.428 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:37 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-activate[99813]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-27T09:36:37.428 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:37 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-activate[99813]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-05-27T09:36:37.428 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:37 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-activate[99813]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-27T09:36:37.428 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:37 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-activate[99813]: --> ceph-volume raw activate successful for osd ID: 0 2024-05-27T09:36:37.428 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:37 smithi119 podman[99799]: 2024-05-27 09:36:37.220043495 +0000 UTC m=+2.314905402 container died 8232dee419d53e700a2730dbf4499064243c6ef95215ed0a032dda1ec3d5d502 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-activate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240521) 2024-05-27T09:36:38.368 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:38 smithi119 podman[99958]: 2024-05-27 09:36:38.023075028 +0000 UTC m=+0.781712620 container remove 8232dee419d53e700a2730dbf4499064243c6ef95215ed0a032dda1ec3d5d502 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-activate, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.license=GPLv2) 2024-05-27T09:36:38.695 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:38 smithi119 ceph-mon[88269]: pgmap v70: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 76/381 objects degraded (19.948%) 2024-05-27T09:36:38.695 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:38 smithi119 podman[100018]: 2024-05-27 09:36:38.274235282 +0000 UTC m=+0.035755714 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:36:38.695 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:38 smithi119 podman[100018]: 2024-05-27 09:36:38.399140987 +0000 UTC m=+0.160661446 container create 5a0e162d870ecb78aa3fd00f593a8789de937ea70e8a6c2e3939448da312b6dd (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T09:36:38.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:38 smithi138 ceph-mon[70004]: pgmap v70: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 76/381 objects degraded (19.948%) 2024-05-27T09:36:38.997 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:38 smithi119 podman[100018]: 2024-05-27 09:36:38.694821592 +0000 UTC m=+0.456342131 container init 5a0e162d870ecb78aa3fd00f593a8789de937ea70e8a6c2e3939448da312b6dd (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0) 2024-05-27T09:36:38.998 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:38 smithi119 podman[100018]: 2024-05-27 09:36:38.705990703 +0000 UTC m=+0.467511132 container start 5a0e162d870ecb78aa3fd00f593a8789de937ea70e8a6c2e3939448da312b6dd (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.license=GPLv2) 2024-05-27T09:36:38.998 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:38 smithi119 bash[100018]: 5a0e162d870ecb78aa3fd00f593a8789de937ea70e8a6c2e3939448da312b6dd 2024-05-27T09:36:38.998 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:38 smithi119 ceph-osd[100036]: -- 172.21.15.119:0/501399532 <== mon.0 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5615b1ac1680 con 0x5615b1b68880 2024-05-27T09:36:38.998 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:38 smithi119 systemd[1]: Started Ceph osd.0 for b966b06e-1c09-11ef-bc9b-c7b262605968. 2024-05-27T09:36:39.747 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:39 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0[100032]: 2024-05-27T09:36:39.350+0000 7f35cdd2f7c0 -1 Falling back to public interface 2024-05-27T09:36:40.167 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:39 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:40.168 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:39 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:40.168 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:39 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:36:40.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:39 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:40.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:39 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:40.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:39 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:36:41.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:40 smithi119 ceph-mon[88269]: pgmap v71: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 76/381 objects degraded (19.948%) 2024-05-27T09:36:41.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:40 smithi138 ceph-mon[70004]: pgmap v71: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 76/381 objects degraded (19.948%) 2024-05-27T09:36:43.184 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:42 smithi119 ceph-mon[88269]: pgmap v72: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 76/381 objects degraded (19.948%) 2024-05-27T09:36:43.185 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:42 smithi119 ceph-mon[88269]: Health check update: Degraded data redundancy: 76/381 objects degraded (19.948%), 20 pgs degraded (PG_DEGRADED) 2024-05-27T09:36:43.185 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:42 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:43.185 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:42 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:36:43.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:42 smithi138 ceph-mon[70004]: pgmap v72: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 76/381 objects degraded (19.948%) 2024-05-27T09:36:43.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:42 smithi138 ceph-mon[70004]: Health check update: Degraded data redundancy: 76/381 objects degraded (19.948%), 20 pgs degraded (PG_DEGRADED) 2024-05-27T09:36:43.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:42 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:43.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:42 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:36:45.219 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:44 smithi119 ceph-mon[88269]: pgmap v73: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 76/381 objects degraded (19.948%) 2024-05-27T09:36:45.220 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:44 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0[100032]: 2024-05-27T09:36:44.891+0000 7f35cdd2f7c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-05-27T09:36:45.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:44 smithi138 ceph-mon[70004]: pgmap v73: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 76/381 objects degraded (19.948%) 2024-05-27T09:36:45.941 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:45 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0[100032]: 2024-05-27T09:36:45.580+0000 7f35cdd2f7c0 -1 osd.0 49 log_to_monitors true 2024-05-27T09:36:46.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:45 smithi119 ceph-mon[88269]: from='osd.0 [v2:172.21.15.119:6802/2940432646,v1:172.21.15.119:6803/2940432646]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-27T09:36:46.248 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:36:46 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0[100032]: 2024-05-27T09:36:46.000+0000 7f35bd000700 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T09:36:46.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:45 smithi138 ceph-mon[70004]: from='osd.0 [v2:172.21.15.119:6802/2940432646,v1:172.21.15.119:6803/2940432646]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-27T09:36:47.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:46 smithi119 ceph-mon[88269]: pgmap v74: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 76/381 objects degraded (19.948%) 2024-05-27T09:36:47.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:46 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:47.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:46 smithi119 ceph-mon[88269]: from='osd.0 [v2:172.21.15.119:6802/2940432646,v1:172.21.15.119:6803/2940432646]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-27T09:36:47.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:46 smithi119 ceph-mon[88269]: osdmap e52: 6 total, 5 up, 6 in 2024-05-27T09:36:47.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:46 smithi119 ceph-mon[88269]: from='osd.0 [v2:172.21.15.119:6802/2940432646,v1:172.21.15.119:6803/2940432646]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-05-27T09:36:47.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:46 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:47.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:46 smithi138 ceph-mon[70004]: pgmap v74: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 76/381 objects degraded (19.948%) 2024-05-27T09:36:47.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:46 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:47.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:46 smithi138 ceph-mon[70004]: from='osd.0 [v2:172.21.15.119:6802/2940432646,v1:172.21.15.119:6803/2940432646]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-27T09:36:47.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:46 smithi138 ceph-mon[70004]: osdmap e52: 6 total, 5 up, 6 in 2024-05-27T09:36:47.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:46 smithi138 ceph-mon[70004]: from='osd.0 [v2:172.21.15.119:6802/2940432646,v1:172.21.15.119:6803/2940432646]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-05-27T09:36:47.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:46 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:48.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:47 smithi119 ceph-mon[88269]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-27T09:36:48.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:47 smithi119 ceph-mon[88269]: osd.0 [v2:172.21.15.119:6802/2940432646,v1:172.21.15.119:6803/2940432646] boot 2024-05-27T09:36:48.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:47 smithi119 ceph-mon[88269]: osdmap e53: 6 total, 6 up, 6 in 2024-05-27T09:36:48.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:47 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T09:36:48.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:47 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:48.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:47 smithi138 ceph-mon[70004]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-27T09:36:48.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:47 smithi138 ceph-mon[70004]: osd.0 [v2:172.21.15.119:6802/2940432646,v1:172.21.15.119:6803/2940432646] boot 2024-05-27T09:36:48.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:47 smithi138 ceph-mon[70004]: osdmap e53: 6 total, 6 up, 6 in 2024-05-27T09:36:48.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:47 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T09:36:48.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:47 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:49.243 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:48 smithi119 ceph-mon[88269]: pgmap v77: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 76/381 objects degraded (19.948%) 2024-05-27T09:36:49.243 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:48 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:49.243 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:48 smithi119 ceph-mon[88269]: osdmap e54: 6 total, 6 up, 6 in 2024-05-27T09:36:49.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:48 smithi138 ceph-mon[70004]: pgmap v77: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 76/381 objects degraded (19.948%) 2024-05-27T09:36:49.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:48 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:49.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:48 smithi138 ceph-mon[70004]: osdmap e54: 6 total, 6 up, 6 in 2024-05-27T09:36:50.334 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:50 smithi119 ceph-mon[88269]: Health check update: Degraded data redundancy: 15/381 objects degraded (3.937%), 3 pgs degraded (PG_DEGRADED) 2024-05-27T09:36:50.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:50 smithi138 ceph-mon[70004]: Health check update: Degraded data redundancy: 15/381 objects degraded (3.937%), 3 pgs degraded (PG_DEGRADED) 2024-05-27T09:36:51.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:51 smithi138 ceph-mon[70004]: pgmap v79: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 15/381 objects degraded (3.937%) 2024-05-27T09:36:51.503 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:51 smithi119 ceph-mon[88269]: pgmap v79: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 15/381 objects degraded (3.937%) 2024-05-27T09:36:52.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:52 smithi138 ceph-mon[70004]: pgmap v80: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:52.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:52 smithi138 ceph-mon[70004]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/381 objects degraded (3.937%), 3 pgs degraded) 2024-05-27T09:36:52.392 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:52 smithi119 ceph-mon[88269]: pgmap v80: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:52.392 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:52 smithi119 ceph-mon[88269]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/381 objects degraded (3.937%), 3 pgs degraded) 2024-05-27T09:36:54.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.518+0000 7f44fc400700 1 -- 172.21.15.119:0/4002944385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f41003b0 msgr2=0x7f44f4100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:54.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.518+0000 7f44fc400700 1 --2- 172.21.15.119:0/4002944385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f41003b0 0x7f44f4100780 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009910 tx=0x7f44e8009c20 comp rx=0 tx=0).stop 2024-05-27T09:36:54.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.519+0000 7f44fc400700 1 -- 172.21.15.119:0/4002944385 shutdown_connections 2024-05-27T09:36:54.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.519+0000 7f44fc400700 1 --2- 172.21.15.119:0/4002944385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44f4100d50 0x7f44f4104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:54.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.519+0000 7f44fc400700 1 --2- 172.21.15.119:0/4002944385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f41003b0 0x7f44f4100780 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:54.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.519+0000 7f44fc400700 1 -- 172.21.15.119:0/4002944385 >> 172.21.15.119:0/4002944385 conn(0x7f44f40fbc30 msgr2=0x7f44f40fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:54.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.519+0000 7f44fc400700 1 -- 172.21.15.119:0/4002944385 shutdown_connections 2024-05-27T09:36:54.520 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.519+0000 7f44fc400700 1 -- 172.21.15.119:0/4002944385 wait complete. 2024-05-27T09:36:54.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.520+0000 7f44fc400700 1 Processor -- start 2024-05-27T09:36:54.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.520+0000 7f44fc400700 1 -- start start 2024-05-27T09:36:54.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.520+0000 7f44fc400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f41003b0 0x7f44f4195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:54.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.520+0000 7f44fc400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44f4100d50 0x7f44f4196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:54.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.520+0000 7f44fc400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f4196930 con 0x7f44f41003b0 2024-05-27T09:36:54.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.520+0000 7f44fc400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f419a6c0 con 0x7f44f4100d50 2024-05-27T09:36:54.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.521+0000 7f44f9a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f41003b0 0x7f44f4195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:54.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.521+0000 7f44f9a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f41003b0 0x7f44f4195d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46940/0 (socket says 172.21.15.119:46940) 2024-05-27T09:36:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.521+0000 7f44f9a00700 1 -- 172.21.15.119:0/1003920432 learned_addr learned my addr 172.21.15.119:0/1003920432 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:36:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.521+0000 7f44f9000700 1 --2- 172.21.15.119:0/1003920432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44f4100d50 0x7f44f4196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.521+0000 7f44f9a00700 1 -- 172.21.15.119:0/1003920432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44f4100d50 msgr2=0x7f44f4196250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.521+0000 7f44f9a00700 1 --2- 172.21.15.119:0/1003920432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44f4100d50 0x7f44f4196250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.521+0000 7f44f9a00700 1 -- 172.21.15.119:0/1003920432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e80095f0 con 0x7f44f41003b0 2024-05-27T09:36:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.522+0000 7f44f9a00700 1 --2- 172.21.15.119:0/1003920432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f41003b0 0x7f44f4195d10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f44e8005f50 tx=0x7f44e800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.522+0000 7f44f2000700 1 -- 172.21.15.119:0/1003920432 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44e801d070 con 0x7f44f41003b0 2024-05-27T09:36:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.522+0000 7f44fc400700 1 -- 172.21.15.119:0/1003920432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f419a940 con 0x7f44f41003b0 2024-05-27T09:36:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.522+0000 7f44f2000700 1 -- 172.21.15.119:0/1003920432 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44e800b610 con 0x7f44f41003b0 2024-05-27T09:36:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.522+0000 7f44f2000700 1 -- 172.21.15.119:0/1003920432 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44e8022470 con 0x7f44f41003b0 2024-05-27T09:36:54.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.522+0000 7f44fc400700 1 -- 172.21.15.119:0/1003920432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f419ae30 con 0x7f44f41003b0 2024-05-27T09:36:54.524 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.524+0000 7f44fc400700 1 -- 172.21.15.119:0/1003920432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f41086c0 con 0x7f44f41003b0 2024-05-27T09:36:54.525 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.524+0000 7f44f2000700 1 -- 172.21.15.119:0/1003920432 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f44e800fca0 con 0x7f44f41003b0 2024-05-27T09:36:54.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.525+0000 7f44f2000700 1 --2- 172.21.15.119:0/1003920432 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f44dc078a40 0x7f44dc07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:54.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.526+0000 7f44f2000700 1 -- 172.21.15.119:0/1003920432 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f44e809c110 con 0x7f44f41003b0 2024-05-27T09:36:54.526 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.526+0000 7f44f9000700 1 --2- 172.21.15.119:0/1003920432 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f44dc078a40 0x7f44dc07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:54.527 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.527+0000 7f44f9000700 1 --2- 172.21.15.119:0/1003920432 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f44dc078a40 0x7f44dc07aef0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f44f4197330 tx=0x7f44e0009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:54.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.531+0000 7f44f2000700 1 -- 172.21.15.119:0/1003920432 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f44e8063d70 con 0x7f44f41003b0 2024-05-27T09:36:54.745 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:54 smithi119 ceph-mon[88269]: pgmap v81: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:54.745 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:54 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:54.745 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:54 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:54.745 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:54 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:36:54.745 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:54 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:36:54.745 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:54 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:54.745 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:54 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:36:54.745 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:54 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:36:54.745 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:54 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:36:54.745 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:54 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:36:54.746 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:54 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-27T09:36:54.746 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:54 smithi119 ceph-mon[88269]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-27T09:36:54.746 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:54 smithi119 ceph-mon[88269]: Upgrade: osd.1 is safe to restart 2024-05-27T09:36:54.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.802+0000 7f44fc400700 1 -- 172.21.15.119:0/1003920432 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f44f419b160 con 0x7f44dc078a40 2024-05-27T09:36:54.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.805+0000 7f44f2000700 1 -- 172.21.15.119:0/1003920432 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f44f419b160 con 0x7f44dc078a40 2024-05-27T09:36:54.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.808+0000 7f44fc400700 1 -- 172.21.15.119:0/1003920432 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f44dc078a40 msgr2=0x7f44dc07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:54.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.808+0000 7f44fc400700 1 --2- 172.21.15.119:0/1003920432 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f44dc078a40 0x7f44dc07aef0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f44f4197330 tx=0x7f44e0009260 comp rx=0 tx=0).stop 2024-05-27T09:36:54.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.808+0000 7f44fc400700 1 -- 172.21.15.119:0/1003920432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f41003b0 msgr2=0x7f44f4195d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:54.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.808+0000 7f44fc400700 1 --2- 172.21.15.119:0/1003920432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f41003b0 0x7f44f4195d10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f44e8005f50 tx=0x7f44e800f740 comp rx=0 tx=0).stop 2024-05-27T09:36:54.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.808+0000 7f44fc400700 1 -- 172.21.15.119:0/1003920432 shutdown_connections 2024-05-27T09:36:54.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.808+0000 7f44fc400700 1 --2- 172.21.15.119:0/1003920432 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f44dc078a40 0x7f44dc07aef0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:54.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.808+0000 7f44fc400700 1 --2- 172.21.15.119:0/1003920432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44f41003b0 0x7f44f4195d10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:54.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.808+0000 7f44fc400700 1 --2- 172.21.15.119:0/1003920432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44f4100d50 0x7f44f4196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:54.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.808+0000 7f44fc400700 1 -- 172.21.15.119:0/1003920432 >> 172.21.15.119:0/1003920432 conn(0x7f44f40fbc30 msgr2=0x7f44f40fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:54.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.808+0000 7f44fc400700 1 -- 172.21.15.119:0/1003920432 shutdown_connections 2024-05-27T09:36:54.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.808+0000 7f44fc400700 1 -- 172.21.15.119:0/1003920432 wait complete. 2024-05-27T09:36:54.824 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:36:54.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:54 smithi138 ceph-mon[70004]: pgmap v81: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:54.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:54 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:54.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:54 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:54.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:54 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:36:54.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:54 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:36:54.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:54 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:54.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:54 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:36:54.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:54 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:36:54.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:54 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:36:54.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:54 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:36:54.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:54 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-27T09:36:54.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:54 smithi138 ceph-mon[70004]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-27T09:36:54.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:54 smithi138 ceph-mon[70004]: Upgrade: osd.1 is safe to restart 2024-05-27T09:36:54.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.961+0000 7f3c71a00700 1 -- 172.21.15.119:0/308109555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c6c0683d0 msgr2=0x7f3c6c068840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:54.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.961+0000 7f3c71a00700 1 --2- 172.21.15.119:0/308109555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c6c0683d0 0x7f3c6c068840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3c58009910 tx=0x7f3c58009c20 comp rx=0 tx=0).stop 2024-05-27T09:36:54.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.961+0000 7f3c71a00700 1 -- 172.21.15.119:0/308109555 shutdown_connections 2024-05-27T09:36:54.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.961+0000 7f3c71a00700 1 --2- 172.21.15.119:0/308109555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c6c0683d0 0x7f3c6c068840 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:54.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.961+0000 7f3c71a00700 1 --2- 172.21.15.119:0/308109555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c6c105f90 0x7f3c6c106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:54.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.961+0000 7f3c71a00700 1 -- 172.21.15.119:0/308109555 >> 172.21.15.119:0/308109555 conn(0x7f3c6c074d00 msgr2=0x7f3c6c075100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:54.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.961+0000 7f3c71a00700 1 -- 172.21.15.119:0/308109555 shutdown_connections 2024-05-27T09:36:54.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.961+0000 7f3c71a00700 1 -- 172.21.15.119:0/308109555 wait complete. 2024-05-27T09:36:54.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.962+0000 7f3c71a00700 1 Processor -- start 2024-05-27T09:36:54.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.962+0000 7f3c71a00700 1 -- start start 2024-05-27T09:36:54.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.963+0000 7f3c71a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c6c0683d0 0x7f3c6c197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:54.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.963+0000 7f3c71a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c6c105f90 0x7f3c6c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:54.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.963+0000 7f3c71a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c1989f0 con 0x7f3c6c105f90 2024-05-27T09:36:54.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.963+0000 7f3c71a00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c19c730 con 0x7f3c6c0683d0 2024-05-27T09:36:54.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.963+0000 7f3c6b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c6c105f90 0x7f3c6c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:54.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.963+0000 7f3c6b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c6c105f90 0x7f3c6c198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46960/0 (socket says 172.21.15.119:46960) 2024-05-27T09:36:54.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.963+0000 7f3c6b400700 1 -- 172.21.15.119:0/2192774279 learned_addr learned my addr 172.21.15.119:0/2192774279 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:36:54.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.963+0000 7f3c6be00700 1 --2- 172.21.15.119:0/2192774279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c6c0683d0 0x7f3c6c197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:54.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.963+0000 7f3c6b400700 1 -- 172.21.15.119:0/2192774279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c6c0683d0 msgr2=0x7f3c6c197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:54.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.963+0000 7f3c6b400700 1 --2- 172.21.15.119:0/2192774279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c6c0683d0 0x7f3c6c197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:54.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.964+0000 7f3c6b400700 1 -- 172.21.15.119:0/2192774279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c580095f0 con 0x7f3c6c105f90 2024-05-27T09:36:54.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.964+0000 7f3c6b400700 1 --2- 172.21.15.119:0/2192774279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c6c105f90 0x7f3c6c198310 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3c58006010 tx=0x7f3c5800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:54.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.964+0000 7f3c68c00700 1 -- 172.21.15.119:0/2192774279 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c5801d070 con 0x7f3c6c105f90 2024-05-27T09:36:54.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.964+0000 7f3c68c00700 1 -- 172.21.15.119:0/2192774279 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c5800f460 con 0x7f3c6c105f90 2024-05-27T09:36:54.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.964+0000 7f3c71a00700 1 -- 172.21.15.119:0/2192774279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c6c19c9b0 con 0x7f3c6c105f90 2024-05-27T09:36:54.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.964+0000 7f3c68c00700 1 -- 172.21.15.119:0/2192774279 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c58017630 con 0x7f3c6c105f90 2024-05-27T09:36:54.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.964+0000 7f3c71a00700 1 -- 172.21.15.119:0/2192774279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c6c19cea0 con 0x7f3c6c105f90 2024-05-27T09:36:54.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.965+0000 7f3c71a00700 1 -- 172.21.15.119:0/2192774279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c6c04f2a0 con 0x7f3c6c105f90 2024-05-27T09:36:54.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.966+0000 7f3c68c00700 1 -- 172.21.15.119:0/2192774279 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f3c5800f5d0 con 0x7f3c6c105f90 2024-05-27T09:36:54.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.967+0000 7f3c68c00700 1 --2- 172.21.15.119:0/2192774279 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3c50078a40 0x7f3c5007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:54.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.968+0000 7f3c68c00700 1 -- 172.21.15.119:0/2192774279 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f3c5809c1f0 con 0x7f3c6c105f90 2024-05-27T09:36:54.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.968+0000 7f3c6be00700 1 --2- 172.21.15.119:0/2192774279 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3c50078a40 0x7f3c5007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:54.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.968+0000 7f3c6be00700 1 --2- 172.21.15.119:0/2192774279 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3c50078a40 0x7f3c5007aef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c005950 tx=0x7f3c5c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:54.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:54.972+0000 7f3c68c00700 1 -- 172.21.15.119:0/2192774279 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f3c58063dd0 con 0x7f3c6c105f90 2024-05-27T09:36:55.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.229+0000 7f3c71a00700 1 -- 172.21.15.119:0/2192774279 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c6c19d380 con 0x7f3c50078a40 2024-05-27T09:36:55.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.231+0000 7f3c68c00700 1 -- 172.21.15.119:0/2192774279 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f3c6c19d380 con 0x7f3c50078a40 2024-05-27T09:36:55.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.234+0000 7f3c71a00700 1 -- 172.21.15.119:0/2192774279 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3c50078a40 msgr2=0x7f3c5007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:55.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.234+0000 7f3c71a00700 1 --2- 172.21.15.119:0/2192774279 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3c50078a40 0x7f3c5007aef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c005950 tx=0x7f3c5c0058e0 comp rx=0 tx=0).stop 2024-05-27T09:36:55.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.234+0000 7f3c71a00700 1 -- 172.21.15.119:0/2192774279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c6c105f90 msgr2=0x7f3c6c198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:55.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.234+0000 7f3c71a00700 1 --2- 172.21.15.119:0/2192774279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c6c105f90 0x7f3c6c198310 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3c58006010 tx=0x7f3c5800be20 comp rx=0 tx=0).stop 2024-05-27T09:36:55.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.234+0000 7f3c71a00700 1 -- 172.21.15.119:0/2192774279 shutdown_connections 2024-05-27T09:36:55.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.234+0000 7f3c71a00700 1 --2- 172.21.15.119:0/2192774279 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3c50078a40 0x7f3c5007aef0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:55.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.234+0000 7f3c71a00700 1 --2- 172.21.15.119:0/2192774279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c6c0683d0 0x7f3c6c197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:55.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.234+0000 7f3c71a00700 1 --2- 172.21.15.119:0/2192774279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c6c105f90 0x7f3c6c198310 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:55.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.234+0000 7f3c71a00700 1 -- 172.21.15.119:0/2192774279 >> 172.21.15.119:0/2192774279 conn(0x7f3c6c074d00 msgr2=0x7f3c6c0fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:55.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.235+0000 7f3c71a00700 1 -- 172.21.15.119:0/2192774279 shutdown_connections 2024-05-27T09:36:55.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.235+0000 7f3c71a00700 1 -- 172.21.15.119:0/2192774279 wait complete. 2024-05-27T09:36:55.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.387+0000 7f2f18000700 1 -- 172.21.15.119:0/3033312571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f101003b0 msgr2=0x7f2f10100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:55.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.387+0000 7f2f18000700 1 --2- 172.21.15.119:0/3033312571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f101003b0 0x7f2f10100780 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009910 tx=0x7f2f04009c20 comp rx=0 tx=0).stop 2024-05-27T09:36:55.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.387+0000 7f2f18000700 1 -- 172.21.15.119:0/3033312571 shutdown_connections 2024-05-27T09:36:55.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.387+0000 7f2f18000700 1 --2- 172.21.15.119:0/3033312571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f10100d50 0x7f2f10104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:55.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.387+0000 7f2f18000700 1 --2- 172.21.15.119:0/3033312571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f101003b0 0x7f2f10100780 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:55.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.387+0000 7f2f18000700 1 -- 172.21.15.119:0/3033312571 >> 172.21.15.119:0/3033312571 conn(0x7f2f100fbc30 msgr2=0x7f2f100fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:55.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.387+0000 7f2f18000700 1 -- 172.21.15.119:0/3033312571 shutdown_connections 2024-05-27T09:36:55.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.387+0000 7f2f18000700 1 -- 172.21.15.119:0/3033312571 wait complete. 2024-05-27T09:36:55.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.389+0000 7f2f18000700 1 Processor -- start 2024-05-27T09:36:55.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.389+0000 7f2f18000700 1 -- start start 2024-05-27T09:36:55.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.389+0000 7f2f18000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f101003b0 0x7f2f10197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:55.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.389+0000 7f2f18000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f10100d50 0x7f2f10198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:55.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.389+0000 7f2f18000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f10198b00 con 0x7f2f10100d50 2024-05-27T09:36:55.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.389+0000 7f2f18000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1019c890 con 0x7f2f101003b0 2024-05-27T09:36:55.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.389+0000 7f2f14c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f10100d50 0x7f2f10198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:55.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.390+0000 7f2f14c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f10100d50 0x7f2f10198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46978/0 (socket says 172.21.15.119:46978) 2024-05-27T09:36:55.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.390+0000 7f2f14c00700 1 -- 172.21.15.119:0/3186558705 learned_addr learned my addr 172.21.15.119:0/3186558705 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:36:55.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.390+0000 7f2f15600700 1 --2- 172.21.15.119:0/3186558705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f101003b0 0x7f2f10197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:55.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.390+0000 7f2f14c00700 1 -- 172.21.15.119:0/3186558705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f101003b0 msgr2=0x7f2f10197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:55.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.390+0000 7f2f14c00700 1 --2- 172.21.15.119:0/3186558705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f101003b0 0x7f2f10197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:55.391 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.390+0000 7f2f14c00700 1 -- 172.21.15.119:0/3186558705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f040095f0 con 0x7f2f10100d50 2024-05-27T09:36:55.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.390+0000 7f2f14c00700 1 --2- 172.21.15.119:0/3186558705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f10100d50 0x7f2f10198420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2efc00d900 tx=0x7f2efc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:55.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.391+0000 7f2f0e000700 1 -- 172.21.15.119:0/3186558705 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2efc009900 con 0x7f2f10100d50 2024-05-27T09:36:55.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.391+0000 7f2f18000700 1 -- 172.21.15.119:0/3186558705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1019cb70 con 0x7f2f10100d50 2024-05-27T09:36:55.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.391+0000 7f2f0e000700 1 -- 172.21.15.119:0/3186558705 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2efc010460 con 0x7f2f10100d50 2024-05-27T09:36:55.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.391+0000 7f2f0e000700 1 -- 172.21.15.119:0/3186558705 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2efc00f5f0 con 0x7f2f10100d50 2024-05-27T09:36:55.392 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.391+0000 7f2f18000700 1 -- 172.21.15.119:0/3186558705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1019d0c0 con 0x7f2f10100d50 2024-05-27T09:36:55.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.393+0000 7f2f18000700 1 -- 172.21.15.119:0/3186558705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f101044a0 con 0x7f2f10100d50 2024-05-27T09:36:55.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.394+0000 7f2f0e000700 1 -- 172.21.15.119:0/3186558705 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f2efc0105d0 con 0x7f2f10100d50 2024-05-27T09:36:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.395+0000 7f2f0e000700 1 --2- 172.21.15.119:0/3186558705 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2ef8078a40 0x7f2ef807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.395+0000 7f2f0e000700 1 -- 172.21.15.119:0/3186558705 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2efc09abc0 con 0x7f2f10100d50 2024-05-27T09:36:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.395+0000 7f2f15600700 1 --2- 172.21.15.119:0/3186558705 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2ef8078a40 0x7f2ef807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:55.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.396+0000 7f2f15600700 1 --2- 172.21.15.119:0/3186558705 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2ef8078a40 0x7f2ef807aef0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400b5c0 tx=0x7f2f0401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:55.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.400+0000 7f2f0e000700 1 -- 172.21.15.119:0/3186558705 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f2efc0627a0 con 0x7f2f10100d50 2024-05-27T09:36:55.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.656+0000 7f2f18000700 1 -- 172.21.15.119:0/3186558705 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f1019d3a0 con 0x7f2ef8078a40 2024-05-27T09:36:55.658 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:55 smithi119 ceph-mon[88269]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:36:55.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.669+0000 7f2f0e000700 1 -- 172.21.15.119:0/3186558705 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f2f1019d3a0 con 0x7f2ef8078a40 2024-05-27T09:36:55.670 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:36:55.670 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (4m) 9s ago 18m 14.1M - 0.25.0 c8568f914cd2 6b90564dcef7 2024-05-27T09:36:55.670 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (18m) 9s ago 18m 8368k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:36:55.670 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (17m) 43s ago 17m 8182k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:36:55.670 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (57s) 9s ago 18m 6669k - 19.0.0-3868-ga6507388 850a06a84d28 011d811ce7fc 2024-05-27T09:36:55.671 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (49s) 43s ago 16m 6665k - 19.0.0-3868-ga6507388 850a06a84d28 b546b3ed1afc 2024-05-27T09:36:55.671 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (3m) 9s ago 17m 74.3M - 9.4.12 2bacad6d85d8 b3abdab08db4 2024-05-27T09:36:55.671 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (10m) 9s ago 10m 17.5M - 18.2.0 dc2bc1663786 93b21713fb1e 2024-05-27T09:36:55.671 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (10m) 9s ago 10m 130M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:36:55.671 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (10m) 43s ago 10m 17.0M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:36:55.671 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (10m) 43s ago 10m 128M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:36:55.671 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (6m) 9s ago 19m 584M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:36:55.671 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (5m) 43s ago 16m 532M - 19.0.0-3868-ga6507388 850a06a84d28 2fc3a80696fa 2024-05-27T09:36:55.671 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (2m) 9s ago 19m 113M 2048M 19.0.0-3868-ga6507388 850a06a84d28 3e5c7a9ad372 2024-05-27T09:36:55.671 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (2m) 43s ago 16m 103M 2048M 19.0.0-3868-ga6507388 850a06a84d28 e4d471274cf1 2024-05-27T09:36:55.671 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (5m) 9s ago 18m 7822k - 1.5.0 0da6a335fe13 709c37815cff 2024-05-27T09:36:55.671 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (5m) 43s ago 16m 7776k - 1.5.0 0da6a335fe13 999047a1a5bf 2024-05-27T09:36:55.671 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (17s) 9s ago 15m 78.5M 1216M 19.0.0-3868-ga6507388 850a06a84d28 5a0e162d870e 2024-05-27T09:36:55.671 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (15m) 9s ago 15m 254M 1216M 18.2.0 dc2bc1663786 fce9819903d9 2024-05-27T09:36:55.672 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (14m) 9s ago 14m 190M 1216M 18.2.0 dc2bc1663786 3955a6112c1c 2024-05-27T09:36:55.672 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (13m) 43s ago 13m 270M 2240M 18.2.0 dc2bc1663786 693c92220622 2024-05-27T09:36:55.672 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (13m) 43s ago 13m 248M 2240M 18.2.0 dc2bc1663786 7559e658e917 2024-05-27T09:36:55.672 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (12m) 43s ago 12m 232M 2240M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:36:55.672 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (4m) 9s ago 17m 68.3M - 2.43.0 a07b618ecd1d cba50454ed21 2024-05-27T09:36:55.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.672+0000 7f2f18000700 1 -- 172.21.15.119:0/3186558705 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2ef8078a40 msgr2=0x7f2ef807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:55.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.672+0000 7f2f18000700 1 --2- 172.21.15.119:0/3186558705 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2ef8078a40 0x7f2ef807aef0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400b5c0 tx=0x7f2f0401a040 comp rx=0 tx=0).stop 2024-05-27T09:36:55.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.672+0000 7f2f18000700 1 -- 172.21.15.119:0/3186558705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f10100d50 msgr2=0x7f2f10198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:55.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.672+0000 7f2f18000700 1 --2- 172.21.15.119:0/3186558705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f10100d50 0x7f2f10198420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2efc00d900 tx=0x7f2efc00dcc0 comp rx=0 tx=0).stop 2024-05-27T09:36:55.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.672+0000 7f2f18000700 1 -- 172.21.15.119:0/3186558705 shutdown_connections 2024-05-27T09:36:55.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.672+0000 7f2f18000700 1 --2- 172.21.15.119:0/3186558705 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2ef8078a40 0x7f2ef807aef0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:55.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.672+0000 7f2f18000700 1 --2- 172.21.15.119:0/3186558705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f101003b0 0x7f2f10197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:55.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.672+0000 7f2f18000700 1 --2- 172.21.15.119:0/3186558705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f10100d50 0x7f2f10198420 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:55.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.672+0000 7f2f18000700 1 -- 172.21.15.119:0/3186558705 >> 172.21.15.119:0/3186558705 conn(0x7f2f100fbc30 msgr2=0x7f2f100fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:55.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.672+0000 7f2f18000700 1 -- 172.21.15.119:0/3186558705 shutdown_connections 2024-05-27T09:36:55.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.672+0000 7f2f18000700 1 -- 172.21.15.119:0/3186558705 wait complete. 2024-05-27T09:36:55.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.825+0000 7f7cd3c00700 1 -- 172.21.15.119:0/2880104743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc0ffbb0 msgr2=0x7f7ccc0fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:55.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.825+0000 7f7cd3c00700 1 --2- 172.21.15.119:0/2880104743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc0ffbb0 0x7f7ccc0fff80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00b0d0 tx=0x7f7cbc00b3e0 comp rx=0 tx=0).stop 2024-05-27T09:36:55.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.825+0000 7f7cd3c00700 1 -- 172.21.15.119:0/2880104743 shutdown_connections 2024-05-27T09:36:55.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.825+0000 7f7cd3c00700 1 --2- 172.21.15.119:0/2880104743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ccc1004c0 0x7f7ccc10c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:55.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.825+0000 7f7cd3c00700 1 --2- 172.21.15.119:0/2880104743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc0ffbb0 0x7f7ccc0fff80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:55.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.825+0000 7f7cd3c00700 1 -- 172.21.15.119:0/2880104743 >> 172.21.15.119:0/2880104743 conn(0x7f7ccc074f50 msgr2=0x7f7ccc075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:55.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.825+0000 7f7cd3c00700 1 -- 172.21.15.119:0/2880104743 shutdown_connections 2024-05-27T09:36:55.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.825+0000 7f7cd3c00700 1 -- 172.21.15.119:0/2880104743 wait complete. 2024-05-27T09:36:55.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.826+0000 7f7cd3c00700 1 Processor -- start 2024-05-27T09:36:55.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.826+0000 7f7cd3c00700 1 -- start start 2024-05-27T09:36:55.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.826+0000 7f7cd3c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc0ffbb0 0x7f7ccc197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:55.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.826+0000 7f7cd3c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ccc1004c0 0x7f7ccc1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:55.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.826+0000 7f7cd3c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc198b80 con 0x7f7ccc0ffbb0 2024-05-27T09:36:55.827 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.826+0000 7f7cd3c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc19c910 con 0x7f7ccc1004c0 2024-05-27T09:36:55.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.827+0000 7f7cd1200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc0ffbb0 0x7f7ccc197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:55.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.827+0000 7f7cd1200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc0ffbb0 0x7f7ccc197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47006/0 (socket says 172.21.15.119:47006) 2024-05-27T09:36:55.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.827+0000 7f7cd1200700 1 -- 172.21.15.119:0/1548584 learned_addr learned my addr 172.21.15.119:0/1548584 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:36:55.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.827+0000 7f7cc3e00700 1 --2- 172.21.15.119:0/1548584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ccc1004c0 0x7f7ccc1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:55.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.827+0000 7f7cd1200700 1 -- 172.21.15.119:0/1548584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ccc1004c0 msgr2=0x7f7ccc1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:55.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.827+0000 7f7cd1200700 1 --2- 172.21.15.119:0/1548584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ccc1004c0 0x7f7ccc1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:55.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.827+0000 7f7cd1200700 1 -- 172.21.15.119:0/1548584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cbc009d00 con 0x7f7ccc0ffbb0 2024-05-27T09:36:55.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.827+0000 7f7cc3e00700 1 --2- 172.21.15.119:0/1548584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ccc1004c0 0x7f7ccc1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:36:55.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.827+0000 7f7cd1200700 1 --2- 172.21.15.119:0/1548584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc0ffbb0 0x7f7ccc197f60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc006130 tx=0x7f7cbc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:55.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.828+0000 7f7cca000700 1 -- 172.21.15.119:0/1548584 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cbc00e040 con 0x7f7ccc0ffbb0 2024-05-27T09:36:55.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.828+0000 7f7cd3c00700 1 -- 172.21.15.119:0/1548584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ccc19cb90 con 0x7f7ccc0ffbb0 2024-05-27T09:36:55.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.828+0000 7f7cca000700 1 -- 172.21.15.119:0/1548584 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7cbc003e50 con 0x7f7ccc0ffbb0 2024-05-27T09:36:55.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.828+0000 7f7cca000700 1 -- 172.21.15.119:0/1548584 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cbc01da80 con 0x7f7ccc0ffbb0 2024-05-27T09:36:55.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.828+0000 7f7cd3c00700 1 -- 172.21.15.119:0/1548584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ccc19d0e0 con 0x7f7ccc0ffbb0 2024-05-27T09:36:55.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.829+0000 7f7cd3c00700 1 -- 172.21.15.119:0/1548584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ccc109f10 con 0x7f7ccc0ffbb0 2024-05-27T09:36:55.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.830+0000 7f7cca000700 1 -- 172.21.15.119:0/1548584 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f7cbc01dcf0 con 0x7f7ccc0ffbb0 2024-05-27T09:36:55.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.831+0000 7f7cca000700 1 --2- 172.21.15.119:0/1548584 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7cb0078a40 0x7f7cb007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:55.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.832+0000 7f7cca000700 1 -- 172.21.15.119:0/1548584 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7cbc09c7d0 con 0x7f7ccc0ffbb0 2024-05-27T09:36:55.832 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.832+0000 7f7cc3e00700 1 --2- 172.21.15.119:0/1548584 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7cb0078a40 0x7f7cb007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:55.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.832+0000 7f7cc3e00700 1 --2- 172.21.15.119:0/1548584 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7cb0078a40 0x7f7cb007aef0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc199580 tx=0x7f7cb8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:55.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:55.836+0000 7f7cca000700 1 -- 172.21.15.119:0/1548584 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f7cbc064450 con 0x7f7ccc0ffbb0 2024-05-27T09:36:55.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:55 smithi138 ceph-mon[70004]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:36:56.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.186+0000 7f7cd3c00700 1 -- 172.21.15.119:0/1548584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7ccc04ea50 con 0x7f7ccc0ffbb0 2024-05-27T09:36:56.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.187+0000 7f7cca000700 1 -- 172.21.15.119:0/1548584 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f7cbc063df0 con 0x7f7ccc0ffbb0 2024-05-27T09:36:56.188 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:36:56.188 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:36:56.188 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:36:56.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:36:56.188 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:36:56.188 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:36:56.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:36:56.188 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:36:56.188 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 5, 2024-05-27T09:36:56.188 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 1 2024-05-27T09:36:56.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:36:56.188 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:36:56.189 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-05-27T09:36:56.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:36:56.189 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:36:56.189 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 9, 2024-05-27T09:36:56.189 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 5 2024-05-27T09:36:56.189 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:36:56.189 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:36:56.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.190+0000 7f7cd3c00700 1 -- 172.21.15.119:0/1548584 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7cb0078a40 msgr2=0x7f7cb007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:56.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.190+0000 7f7cd3c00700 1 --2- 172.21.15.119:0/1548584 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7cb0078a40 0x7f7cb007aef0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc199580 tx=0x7f7cb8008040 comp rx=0 tx=0).stop 2024-05-27T09:36:56.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.190+0000 7f7cd3c00700 1 -- 172.21.15.119:0/1548584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc0ffbb0 msgr2=0x7f7ccc197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:56.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.190+0000 7f7cd3c00700 1 --2- 172.21.15.119:0/1548584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc0ffbb0 0x7f7ccc197f60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc006130 tx=0x7f7cbc00ce30 comp rx=0 tx=0).stop 2024-05-27T09:36:56.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.190+0000 7f7cd3c00700 1 -- 172.21.15.119:0/1548584 shutdown_connections 2024-05-27T09:36:56.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.190+0000 7f7cd3c00700 1 --2- 172.21.15.119:0/1548584 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7cb0078a40 0x7f7cb007aef0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:56.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.190+0000 7f7cd3c00700 1 --2- 172.21.15.119:0/1548584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc0ffbb0 0x7f7ccc197f60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:56.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.190+0000 7f7cd3c00700 1 --2- 172.21.15.119:0/1548584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ccc1004c0 0x7f7ccc1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:56.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.190+0000 7f7cd3c00700 1 -- 172.21.15.119:0/1548584 >> 172.21.15.119:0/1548584 conn(0x7f7ccc074f50 msgr2=0x7f7ccc0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:56.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.191+0000 7f7cd3c00700 1 -- 172.21.15.119:0/1548584 shutdown_connections 2024-05-27T09:36:56.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.191+0000 7f7cd3c00700 1 -- 172.21.15.119:0/1548584 wait complete. 2024-05-27T09:36:56.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.347+0000 7f5dbfe00700 1 -- 172.21.15.119:0/2349303607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc0102df0 msgr2=0x7f5dc0106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:56.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.347+0000 7f5dbfe00700 1 --2- 172.21.15.119:0/2349303607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc0102df0 0x7f5dc0106dc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009910 tx=0x7f5dac009c20 comp rx=0 tx=0).stop 2024-05-27T09:36:56.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.347+0000 7f5dbfe00700 1 -- 172.21.15.119:0/2349303607 shutdown_connections 2024-05-27T09:36:56.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.347+0000 7f5dbfe00700 1 --2- 172.21.15.119:0/2349303607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc0102df0 0x7f5dc0106dc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:56.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.347+0000 7f5dbfe00700 1 --2- 172.21.15.119:0/2349303607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dc0102450 0x7f5dc0102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:56.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.347+0000 7f5dbfe00700 1 -- 172.21.15.119:0/2349303607 >> 172.21.15.119:0/2349303607 conn(0x7f5dc00fdd30 msgr2=0x7f5dc0100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:56.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.347+0000 7f5dbfe00700 1 -- 172.21.15.119:0/2349303607 shutdown_connections 2024-05-27T09:36:56.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.347+0000 7f5dbfe00700 1 -- 172.21.15.119:0/2349303607 wait complete. 2024-05-27T09:36:56.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.349+0000 7f5dbfe00700 1 Processor -- start 2024-05-27T09:36:56.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.349+0000 7f5dbfe00700 1 -- start start 2024-05-27T09:36:56.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.349+0000 7f5dbfe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc0102450 0x7f5dc0075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:56.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.349+0000 7f5dbfe00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dc0102df0 0x7f5dc00757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:56.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.349+0000 7f5dbfe00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc00793f0 con 0x7f5dc0102450 2024-05-27T09:36:56.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.349+0000 7f5dbfe00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc0075ce0 con 0x7f5dc0102df0 2024-05-27T09:36:56.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.350+0000 7f5dbea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc0102450 0x7f5dc0075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:56.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.350+0000 7f5dbea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc0102450 0x7f5dc0075260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47016/0 (socket says 172.21.15.119:47016) 2024-05-27T09:36:56.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.350+0000 7f5dbea00700 1 -- 172.21.15.119:0/3461639413 learned_addr learned my addr 172.21.15.119:0/3461639413 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:36:56.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.350+0000 7f5dbe000700 1 --2- 172.21.15.119:0/3461639413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dc0102df0 0x7f5dc00757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:56.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.350+0000 7f5dbea00700 1 -- 172.21.15.119:0/3461639413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dc0102df0 msgr2=0x7f5dc00757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:56.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.350+0000 7f5dbea00700 1 --2- 172.21.15.119:0/3461639413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dc0102df0 0x7f5dc00757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:56.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.350+0000 7f5dbea00700 1 -- 172.21.15.119:0/3461639413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dac0095f0 con 0x7f5dc0102450 2024-05-27T09:36:56.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.351+0000 7f5dbea00700 1 --2- 172.21.15.119:0/3461639413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc0102450 0x7f5dc0075260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5db000d8d0 tx=0x7f5db000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:56.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.351+0000 7f5dc4a00700 1 -- 172.21.15.119:0/3461639413 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5db0009940 con 0x7f5dc0102450 2024-05-27T09:36:56.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.351+0000 7f5dc4a00700 1 -- 172.21.15.119:0/3461639413 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5db0010460 con 0x7f5dc0102450 2024-05-27T09:36:56.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.351+0000 7f5dc4a00700 1 -- 172.21.15.119:0/3461639413 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5db000f5f0 con 0x7f5dc0102450 2024-05-27T09:36:56.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.351+0000 7f5dbfe00700 1 -- 172.21.15.119:0/3461639413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc0075fc0 con 0x7f5dc0102450 2024-05-27T09:36:56.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.351+0000 7f5dbfe00700 1 -- 172.21.15.119:0/3461639413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc01a6550 con 0x7f5dc0102450 2024-05-27T09:36:56.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.353+0000 7f5dbfe00700 1 -- 172.21.15.119:0/3461639413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc004f2a0 con 0x7f5dc0102450 2024-05-27T09:36:56.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.354+0000 7f5dc4a00700 1 -- 172.21.15.119:0/3461639413 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f5db00105d0 con 0x7f5dc0102450 2024-05-27T09:36:56.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.355+0000 7f5dc4a00700 1 --2- 172.21.15.119:0/3461639413 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5da4078a40 0x7f5da407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:56.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.355+0000 7f5dc4a00700 1 -- 172.21.15.119:0/3461639413 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f5db009ac00 con 0x7f5dc0102450 2024-05-27T09:36:56.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.356+0000 7f5dbe000700 1 --2- 172.21.15.119:0/3461639413 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5da4078a40 0x7f5da407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:56.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.356+0000 7f5dbe000700 1 --2- 172.21.15.119:0/3461639413 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5da4078a40 0x7f5da407aef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00fe4d0 tx=0x7f5dac01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:56.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.360+0000 7f5dc4a00700 1 -- 172.21.15.119:0/3461639413 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f5db00a0020 con 0x7f5dc0102450 2024-05-27T09:36:56.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.649+0000 7f5dbfe00700 1 -- 172.21.15.119:0/3461639413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5dc01a6950 con 0x7f5dc0102450 2024-05-27T09:36:56.650 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:56 smithi119 ceph-mon[88269]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:36:56.650 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:56 smithi119 ceph-mon[88269]: pgmap v82: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:56.650 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:56 smithi119 ceph-mon[88269]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:36:56.650 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:56 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/1548584' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:36:56.651 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:56 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:56.651 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:56 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-27T09:36:56.651 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:56 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:36:56.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.650+0000 7f5dc4a00700 1 -- 172.21.15.119:0/3461639413 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1889 (secure 0 0 0) 0x7f5db0016780 con 0x7f5dc0102450 2024-05-27T09:36:56.651 INFO:teuthology.orchestra.run.smithi119.stdout:e22 2024-05-27T09:36:56.651 INFO:teuthology.orchestra.run.smithi119.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T09:36:56.652 INFO:teuthology.orchestra.run.smithi119.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:36:56.652 INFO:teuthology.orchestra.run.smithi119.stdout:legacy client fscid: 1 2024-05-27T09:36:56.652 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:36:56.652 INFO:teuthology.orchestra.run.smithi119.stdout:Filesystem 'cephfs' (1) 2024-05-27T09:36:56.652 INFO:teuthology.orchestra.run.smithi119.stdout:fs_name cephfs 2024-05-27T09:36:56.652 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 22 2024-05-27T09:36:56.652 INFO:teuthology.orchestra.run.smithi119.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-27T09:36:56.652 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-05-27T09:25:53.387388+0000 2024-05-27T09:36:56.652 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2024-05-27T09:34:01.250304+0000 2024-05-27T09:36:56.652 INFO:teuthology.orchestra.run.smithi119.stdout:tableserver 0 2024-05-27T09:36:56.652 INFO:teuthology.orchestra.run.smithi119.stdout:root 0 2024-05-27T09:36:56.652 INFO:teuthology.orchestra.run.smithi119.stdout:session_timeout 60 2024-05-27T09:36:56.652 INFO:teuthology.orchestra.run.smithi119.stdout:session_autoclose 300 2024-05-27T09:36:56.653 INFO:teuthology.orchestra.run.smithi119.stdout:max_file_size 1099511627776 2024-05-27T09:36:56.653 INFO:teuthology.orchestra.run.smithi119.stdout:max_xattr_size 65536 2024-05-27T09:36:56.653 INFO:teuthology.orchestra.run.smithi119.stdout:required_client_features {} 2024-05-27T09:36:56.653 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure 0 2024-05-27T09:36:56.653 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure_osd_epoch 46 2024-05-27T09:36:56.653 INFO:teuthology.orchestra.run.smithi119.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:36:56.653 INFO:teuthology.orchestra.run.smithi119.stdout:max_mds 2 2024-05-27T09:36:56.653 INFO:teuthology.orchestra.run.smithi119.stdout:in 0,1 2024-05-27T09:36:56.653 INFO:teuthology.orchestra.run.smithi119.stdout:up {0=14472,1=24241} 2024-05-27T09:36:56.653 INFO:teuthology.orchestra.run.smithi119.stdout:failed 2024-05-27T09:36:56.653 INFO:teuthology.orchestra.run.smithi119.stdout:damaged 2024-05-27T09:36:56.653 INFO:teuthology.orchestra.run.smithi119.stdout:stopped 2024-05-27T09:36:56.654 INFO:teuthology.orchestra.run.smithi119.stdout:data_pools [3] 2024-05-27T09:36:56.654 INFO:teuthology.orchestra.run.smithi119.stdout:metadata_pool 2 2024-05-27T09:36:56.654 INFO:teuthology.orchestra.run.smithi119.stdout:inline_data enabled 2024-05-27T09:36:56.654 INFO:teuthology.orchestra.run.smithi119.stdout:balancer 2024-05-27T09:36:56.654 INFO:teuthology.orchestra.run.smithi119.stdout:bal_rank_mask -1 2024-05-27T09:36:56.654 INFO:teuthology.orchestra.run.smithi119.stdout:standby_count_wanted 1 2024-05-27T09:36:56.654 INFO:teuthology.orchestra.run.smithi119.stdout:qdb_cluster leader: 0 members: 2024-05-27T09:36:56.654 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.ppmzzk{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:36:56.654 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.ndfkhd{1:24241} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/897058594,v1:172.21.15.138:6825/897058594] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:36:56.654 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:36:56.654 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:36:56.654 INFO:teuthology.orchestra.run.smithi119.stdout:Standby daemons: 2024-05-27T09:36:56.655 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:36:56.655 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.nghcey{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.119:6826/413736191,v1:172.21.15.119:6827/413736191] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:36:56.655 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.alszpk{-1:24253} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/2345607053,v1:172.21.15.138:6827/2345607053] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:36:56.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.653+0000 7f5dbfe00700 1 -- 172.21.15.119:0/3461639413 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5da4078a40 msgr2=0x7f5da407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:56.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.653+0000 7f5dbfe00700 1 --2- 172.21.15.119:0/3461639413 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5da4078a40 0x7f5da407aef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00fe4d0 tx=0x7f5dac01a040 comp rx=0 tx=0).stop 2024-05-27T09:36:56.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.653+0000 7f5dbfe00700 1 -- 172.21.15.119:0/3461639413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc0102450 msgr2=0x7f5dc0075260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:56.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.653+0000 7f5dbfe00700 1 --2- 172.21.15.119:0/3461639413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc0102450 0x7f5dc0075260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5db000d8d0 tx=0x7f5db000dc90 comp rx=0 tx=0).stop 2024-05-27T09:36:56.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.653+0000 7f5dbfe00700 1 -- 172.21.15.119:0/3461639413 shutdown_connections 2024-05-27T09:36:56.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.653+0000 7f5dbfe00700 1 --2- 172.21.15.119:0/3461639413 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5da4078a40 0x7f5da407aef0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:56.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.653+0000 7f5dbfe00700 1 --2- 172.21.15.119:0/3461639413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc0102450 0x7f5dc0075260 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:56.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.653+0000 7f5dbfe00700 1 --2- 172.21.15.119:0/3461639413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dc0102df0 0x7f5dc00757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:56.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.653+0000 7f5dbfe00700 1 -- 172.21.15.119:0/3461639413 >> 172.21.15.119:0/3461639413 conn(0x7f5dc00fdd30 msgr2=0x7f5dc00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:56.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.653+0000 7f5dbfe00700 1 -- 172.21.15.119:0/3461639413 shutdown_connections 2024-05-27T09:36:56.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.653+0000 7f5dbfe00700 1 -- 172.21.15.119:0/3461639413 wait complete. 2024-05-27T09:36:56.656 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 22 2024-05-27T09:36:56.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.807+0000 7fb4d7800700 1 -- 172.21.15.119:0/2736458644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4d0073a00 msgr2=0x7fb4d010c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:56.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.807+0000 7fb4d7800700 1 --2- 172.21.15.119:0/2736458644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4d0073a00 0x7fb4d010c450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009910 tx=0x7fb4bc009c20 comp rx=0 tx=0).stop 2024-05-27T09:36:56.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.807+0000 7fb4d7800700 1 -- 172.21.15.119:0/2736458644 shutdown_connections 2024-05-27T09:36:56.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.807+0000 7fb4d7800700 1 --2- 172.21.15.119:0/2736458644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4d0073a00 0x7fb4d010c450 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:56.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.807+0000 7fb4d7800700 1 --2- 172.21.15.119:0/2736458644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4d00730f0 0x7fb4d00734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:56.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.807+0000 7fb4d7800700 1 -- 172.21.15.119:0/2736458644 >> 172.21.15.119:0/2736458644 conn(0x7fb4d00fbc30 msgr2=0x7fb4d00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:56.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.807+0000 7fb4d7800700 1 -- 172.21.15.119:0/2736458644 shutdown_connections 2024-05-27T09:36:56.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.807+0000 7fb4d7800700 1 -- 172.21.15.119:0/2736458644 wait complete. 2024-05-27T09:36:56.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.808+0000 7fb4d7800700 1 Processor -- start 2024-05-27T09:36:56.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.809+0000 7fb4d7800700 1 -- start start 2024-05-27T09:36:56.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.809+0000 7fb4d7800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4d00730f0 0x7fb4d0197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:56.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.809+0000 7fb4d7800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4d0073a00 0x7fb4d0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:56.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.809+0000 7fb4d7800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d019c7c0 con 0x7fb4d0073a00 2024-05-27T09:36:56.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.809+0000 7fb4d7800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d019c930 con 0x7fb4d00730f0 2024-05-27T09:36:56.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.809+0000 7fb4cfe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4d0073a00 0x7fb4d0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:56.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.809+0000 7fb4cfe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4d0073a00 0x7fb4d0198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47048/0 (socket says 172.21.15.119:47048) 2024-05-27T09:36:56.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.809+0000 7fb4cfe00700 1 -- 172.21.15.119:0/2327370788 learned_addr learned my addr 172.21.15.119:0/2327370788 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:36:56.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.810+0000 7fb4d4e00700 1 --2- 172.21.15.119:0/2327370788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4d00730f0 0x7fb4d0197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:56.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.810+0000 7fb4cfe00700 1 -- 172.21.15.119:0/2327370788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4d00730f0 msgr2=0x7fb4d0197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:56.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.810+0000 7fb4cfe00700 1 --2- 172.21.15.119:0/2327370788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4d00730f0 0x7fb4d0197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:56.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.810+0000 7fb4cfe00700 1 -- 172.21.15.119:0/2327370788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4bc0095f0 con 0x7fb4d0073a00 2024-05-27T09:36:56.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.810+0000 7fb4d4e00700 1 --2- 172.21.15.119:0/2327370788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4d00730f0 0x7fb4d0197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:36:56.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.810+0000 7fb4cfe00700 1 --2- 172.21.15.119:0/2327370788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4d0073a00 0x7fb4d0198480 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc006010 tx=0x7fb4bc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:56.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.811+0000 7fb4cd600700 1 -- 172.21.15.119:0/2327370788 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4bc01d070 con 0x7fb4d0073a00 2024-05-27T09:36:56.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.811+0000 7fb4d7800700 1 -- 172.21.15.119:0/2327370788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d019cc10 con 0x7fb4d0073a00 2024-05-27T09:36:56.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.811+0000 7fb4cd600700 1 -- 172.21.15.119:0/2327370788 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb4bc004fb0 con 0x7fb4d0073a00 2024-05-27T09:36:56.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.811+0000 7fb4d7800700 1 -- 172.21.15.119:0/2327370788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d019d160 con 0x7fb4d0073a00 2024-05-27T09:36:56.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.811+0000 7fb4cd600700 1 -- 172.21.15.119:0/2327370788 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4bc022ea0 con 0x7fb4d0073a00 2024-05-27T09:36:56.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.812+0000 7fb4d7800700 1 -- 172.21.15.119:0/2327370788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d0109b50 con 0x7fb4d0073a00 2024-05-27T09:36:56.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.813+0000 7fb4cd600700 1 -- 172.21.15.119:0/2327370788 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fb4bc022470 con 0x7fb4d0073a00 2024-05-27T09:36:56.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.814+0000 7fb4cd600700 1 --2- 172.21.15.119:0/2327370788 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb4b8078a40 0x7fb4b807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:56.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.814+0000 7fb4cd600700 1 -- 172.21.15.119:0/2327370788 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb4bc09c1d0 con 0x7fb4d0073a00 2024-05-27T09:36:56.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.815+0000 7fb4d4e00700 1 --2- 172.21.15.119:0/2327370788 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb4b8078a40 0x7fb4b807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:56.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.815+0000 7fb4d4e00700 1 --2- 172.21.15.119:0/2327370788 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb4b8078a40 0x7fb4b807aef0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4005fd0 tx=0x7fb4c4005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:56.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:56.824+0000 7fb4cd600700 1 -- 172.21.15.119:0/2327370788 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fb4bc063e60 con 0x7fb4d0073a00 2024-05-27T09:36:56.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:56 smithi138 ceph-mon[70004]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:36:56.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:56 smithi138 ceph-mon[70004]: pgmap v82: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:56.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:56 smithi138 ceph-mon[70004]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:36:56.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:56 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/1548584' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:36:56.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:56 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:56.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:56 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-27T09:36:56.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:56 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:36:57.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.083+0000 7fb4d7800700 1 -- 172.21.15.119:0/2327370788 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4d019d3d0 con 0x7fb4b8078a40 2024-05-27T09:36:57.086 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.085+0000 7fb4cd600700 1 -- 172.21.15.119:0/2327370788 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb4d019d3d0 con 0x7fb4b8078a40 2024-05-27T09:36:57.087 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:36:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:36:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:36:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-27T09:36:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [ 2024-05-27T09:36:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: "crash", 2024-05-27T09:36:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: "mon", 2024-05-27T09:36:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr" 2024-05-27T09:36:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-05-27T09:36:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "7/23 daemons upgraded", 2024-05-27T09:36:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "Currently upgrading osd daemons", 2024-05-27T09:36:57.087 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:36:57.088 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:36:57.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.088+0000 7fb4d7800700 1 -- 172.21.15.119:0/2327370788 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb4b8078a40 msgr2=0x7fb4b807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:57.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.088+0000 7fb4d7800700 1 --2- 172.21.15.119:0/2327370788 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb4b8078a40 0x7fb4b807aef0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4005fd0 tx=0x7fb4c4005dc0 comp rx=0 tx=0).stop 2024-05-27T09:36:57.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.088+0000 7fb4d7800700 1 -- 172.21.15.119:0/2327370788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4d0073a00 msgr2=0x7fb4d0198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:57.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.088+0000 7fb4d7800700 1 --2- 172.21.15.119:0/2327370788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4d0073a00 0x7fb4d0198480 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc006010 tx=0x7fb4bc00be20 comp rx=0 tx=0).stop 2024-05-27T09:36:57.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.088+0000 7fb4d7800700 1 -- 172.21.15.119:0/2327370788 shutdown_connections 2024-05-27T09:36:57.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.088+0000 7fb4d7800700 1 --2- 172.21.15.119:0/2327370788 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb4b8078a40 0x7fb4b807aef0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:57.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.088+0000 7fb4d7800700 1 --2- 172.21.15.119:0/2327370788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4d00730f0 0x7fb4d0197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:57.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.088+0000 7fb4d7800700 1 --2- 172.21.15.119:0/2327370788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4d0073a00 0x7fb4d0198480 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:57.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.088+0000 7fb4d7800700 1 -- 172.21.15.119:0/2327370788 >> 172.21.15.119:0/2327370788 conn(0x7fb4d00fbc30 msgr2=0x7fb4d0106c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:57.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.089+0000 7fb4d7800700 1 -- 172.21.15.119:0/2327370788 shutdown_connections 2024-05-27T09:36:57.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.089+0000 7fb4d7800700 1 -- 172.21.15.119:0/2327370788 wait complete. 2024-05-27T09:36:57.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.233+0000 7fda9be00700 1 -- 172.21.15.119:0/483720703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda9c100210 msgr2=0x7fda9c1005e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:57.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.233+0000 7fda9be00700 1 --2- 172.21.15.119:0/483720703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda9c100210 0x7fda9c1005e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fda8c009910 tx=0x7fda8c009c20 comp rx=0 tx=0).stop 2024-05-27T09:36:57.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.233+0000 7fda9be00700 1 -- 172.21.15.119:0/483720703 shutdown_connections 2024-05-27T09:36:57.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.233+0000 7fda9be00700 1 --2- 172.21.15.119:0/483720703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9c100bb0 0x7fda9c104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:57.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.233+0000 7fda9be00700 1 --2- 172.21.15.119:0/483720703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda9c100210 0x7fda9c1005e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:57.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.233+0000 7fda9be00700 1 -- 172.21.15.119:0/483720703 >> 172.21.15.119:0/483720703 conn(0x7fda9c0fbab0 msgr2=0x7fda9c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:57.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.233+0000 7fda9be00700 1 -- 172.21.15.119:0/483720703 shutdown_connections 2024-05-27T09:36:57.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.234+0000 7fda9be00700 1 -- 172.21.15.119:0/483720703 wait complete. 2024-05-27T09:36:57.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.234+0000 7fda9be00700 1 Processor -- start 2024-05-27T09:36:57.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.235+0000 7fda9be00700 1 -- start start 2024-05-27T09:36:57.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.235+0000 7fda9be00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9c100210 0x7fda9c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:57.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.235+0000 7fda9be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda9c100bb0 0x7fda9c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:57.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.235+0000 7fda9be00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c198af0 con 0x7fda9c100bb0 2024-05-27T09:36:57.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.235+0000 7fda9be00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c19c880 con 0x7fda9c100210 2024-05-27T09:36:57.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.235+0000 7fda9a000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda9c100bb0 0x7fda9c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:57.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.235+0000 7fda9a000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda9c100bb0 0x7fda9c198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47066/0 (socket says 172.21.15.119:47066) 2024-05-27T09:36:57.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.235+0000 7fda9a000700 1 -- 172.21.15.119:0/4052326844 learned_addr learned my addr 172.21.15.119:0/4052326844 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:36:57.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.236+0000 7fda9aa00700 1 --2- 172.21.15.119:0/4052326844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9c100210 0x7fda9c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:57.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.236+0000 7fda9a000700 1 -- 172.21.15.119:0/4052326844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9c100210 msgr2=0x7fda9c197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:57.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.236+0000 7fda9a000700 1 --2- 172.21.15.119:0/4052326844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9c100210 0x7fda9c197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:57.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.236+0000 7fda9a000700 1 -- 172.21.15.119:0/4052326844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8c0095f0 con 0x7fda9c100bb0 2024-05-27T09:36:57.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.236+0000 7fda9a000700 1 --2- 172.21.15.119:0/4052326844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda9c100bb0 0x7fda9c198410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fda8800b480 tx=0x7fda8800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:57.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.236+0000 7fda93400700 1 -- 172.21.15.119:0/4052326844 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda880118c0 con 0x7fda9c100bb0 2024-05-27T09:36:57.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.236+0000 7fda93400700 1 -- 172.21.15.119:0/4052326844 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda88010380 con 0x7fda9c100bb0 2024-05-27T09:36:57.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.236+0000 7fda9be00700 1 -- 172.21.15.119:0/4052326844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9c19cb60 con 0x7fda9c100bb0 2024-05-27T09:36:57.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.236+0000 7fda93400700 1 -- 172.21.15.119:0/4052326844 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda8800f4f0 con 0x7fda9c100bb0 2024-05-27T09:36:57.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.237+0000 7fda9be00700 1 -- 172.21.15.119:0/4052326844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda9c19d0b0 con 0x7fda9c100bb0 2024-05-27T09:36:57.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.238+0000 7fda9be00700 1 -- 172.21.15.119:0/4052326844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda9c104300 con 0x7fda9c100bb0 2024-05-27T09:36:57.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.239+0000 7fda93400700 1 -- 172.21.15.119:0/4052326844 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fda880104f0 con 0x7fda9c100bb0 2024-05-27T09:36:57.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.239+0000 7fda93400700 1 --2- 172.21.15.119:0/4052326844 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fda80078a40 0x7fda8007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:36:57.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.240+0000 7fda93400700 1 -- 172.21.15.119:0/4052326844 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fda8809a8e0 con 0x7fda9c100bb0 2024-05-27T09:36:57.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.240+0000 7fda9aa00700 1 --2- 172.21.15.119:0/4052326844 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fda80078a40 0x7fda8007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:36:57.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.241+0000 7fda9aa00700 1 --2- 172.21.15.119:0/4052326844 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fda80078a40 0x7fda8007aef0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fda8c006010 tx=0x7fda8c00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:36:57.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.245+0000 7fda93400700 1 -- 172.21.15.119:0/4052326844 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fda88062560 con 0x7fda9c100bb0 2024-05-27T09:36:57.589 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.587+0000 7fda9be00700 1 -- 172.21.15.119:0/4052326844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fda9c04f2a0 con 0x7fda9c100bb0 2024-05-27T09:36:57.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.588+0000 7fda93400700 1 -- 172.21.15.119:0/4052326844 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fda88061f00 con 0x7fda9c100bb0 2024-05-27T09:36:57.590 INFO:teuthology.orchestra.run.smithi119.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-27T09:36:57.590 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:36:57.590 INFO:teuthology.orchestra.run.smithi119.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:36:57.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.591+0000 7fda9be00700 1 -- 172.21.15.119:0/4052326844 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fda80078a40 msgr2=0x7fda8007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:57.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.591+0000 7fda9be00700 1 --2- 172.21.15.119:0/4052326844 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fda80078a40 0x7fda8007aef0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fda8c006010 tx=0x7fda8c00b540 comp rx=0 tx=0).stop 2024-05-27T09:36:57.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.591+0000 7fda9be00700 1 -- 172.21.15.119:0/4052326844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda9c100bb0 msgr2=0x7fda9c198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:36:57.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.591+0000 7fda9be00700 1 --2- 172.21.15.119:0/4052326844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda9c100bb0 0x7fda9c198410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fda8800b480 tx=0x7fda8800b840 comp rx=0 tx=0).stop 2024-05-27T09:36:57.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.591+0000 7fda9be00700 1 -- 172.21.15.119:0/4052326844 shutdown_connections 2024-05-27T09:36:57.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.591+0000 7fda9be00700 1 --2- 172.21.15.119:0/4052326844 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fda80078a40 0x7fda8007aef0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:57.593 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.591+0000 7fda9be00700 1 --2- 172.21.15.119:0/4052326844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9c100210 0x7fda9c197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:57.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.591+0000 7fda9be00700 1 --2- 172.21.15.119:0/4052326844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda9c100bb0 0x7fda9c198410 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:36:57.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.591+0000 7fda9be00700 1 -- 172.21.15.119:0/4052326844 >> 172.21.15.119:0/4052326844 conn(0x7fda9c0fbab0 msgr2=0x7fda9c0fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:36:57.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.591+0000 7fda9be00700 1 -- 172.21.15.119:0/4052326844 shutdown_connections 2024-05-27T09:36:57.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:36:57.591+0000 7fda9be00700 1 -- 172.21.15.119:0/4052326844 wait complete. 2024-05-27T09:36:57.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:57 smithi119 ceph-mon[88269]: Upgrade: Updating osd.1 2024-05-27T09:36:57.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:57 smithi119 ceph-mon[88269]: Deploying daemon osd.1 on smithi119 2024-05-27T09:36:57.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:57 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/3461639413' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:36:57.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:57.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:36:57.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:57.876 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:57 smithi138 ceph-mon[70004]: Upgrade: Updating osd.1 2024-05-27T09:36:57.876 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:57 smithi138 ceph-mon[70004]: Deploying daemon osd.1 on smithi119 2024-05-27T09:36:57.877 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:57 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/3461639413' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:36:57.877 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:57.877 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:36:57.877 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:36:58.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:58 smithi119 ceph-mon[88269]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:36:58.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:58 smithi119 ceph-mon[88269]: pgmap v83: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:58.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:36:58 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/4052326844' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:36:58.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:58 smithi138 ceph-mon[70004]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:36:58.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:58 smithi138 ceph-mon[70004]: pgmap v83: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:36:58.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:36:58 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/4052326844' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:37:00.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:00 smithi119 ceph-mon[88269]: pgmap v84: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:00.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:00 smithi138 ceph-mon[70004]: pgmap v84: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:01.697 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:01 smithi119 systemd[1]: Stopping Ceph osd.1 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:37:01.998 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:01 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1[49408]: 2024-05-27T09:37:01.691+0000 7f49b2e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T09:37:01.998 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:01 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1[49408]: 2024-05-27T09:37:01.691+0000 7f49b2e00700 -1 osd.1 54 *** Got signal Terminated *** 2024-05-27T09:37:01.998 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:01 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1[49408]: 2024-05-27T09:37:01.691+0000 7f49b2e00700 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T09:37:02.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:02 smithi119 ceph-mon[88269]: pgmap v85: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:02.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:02 smithi119 ceph-mon[88269]: osd.1 marked itself down and dead 2024-05-27T09:37:02.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:02 smithi138 ceph-mon[70004]: pgmap v85: 65 pgs: 65 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:02.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:02 smithi138 ceph-mon[70004]: osd.1 marked itself down and dead 2024-05-27T09:37:03.095 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:02 smithi119 podman[104930]: 2024-05-27 09:37:02.996398849 +0000 UTC m=+1.424959330 container died fce9819903d9a6da6a015acd0f90b6ba627cc0db9b942792a3bb667b64717e68 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, RELEASE=HEAD, io.buildah.version=1.29.1, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T09:37:03.748 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:03 smithi119 podman[104930]: 2024-05-27 09:37:03.572332517 +0000 UTC m=+2.000892953 container cleanup fce9819903d9a6da6a015acd0f90b6ba627cc0db9b942792a3bb667b64717e68 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1, RELEASE=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, org.label-schema.build-date=20231212, ceph=True) 2024-05-27T09:37:03.748 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:03 smithi119 bash[104930]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1 2024-05-27T09:37:03.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:03 smithi119 ceph-mon[88269]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-27T09:37:03.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:03 smithi119 ceph-mon[88269]: osdmap e55: 6 total, 5 up, 6 in 2024-05-27T09:37:03.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:03 smithi138 ceph-mon[70004]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-27T09:37:03.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:03 smithi138 ceph-mon[70004]: osdmap e55: 6 total, 5 up, 6 in 2024-05-27T09:37:04.247 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:03 smithi119 podman[104948]: 2024-05-27 09:37:03.787626669 +0000 UTC m=+0.772117598 container remove fce9819903d9a6da6a015acd0f90b6ba627cc0db9b942792a3bb667b64717e68 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=-18.2.0, io.buildah.version=1.29.1, GIT_CLEAN=True, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, RELEASE=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-05-27T09:37:04.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:04 smithi119 ceph-mon[88269]: pgmap v87: 65 pgs: 22 peering, 6 stale+active+clean, 37 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:04.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:04 smithi119 ceph-mon[88269]: Health check failed: Reduced data availability: 6 pgs peering (PG_AVAILABILITY) 2024-05-27T09:37:04.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:04 smithi119 ceph-mon[88269]: osdmap e56: 6 total, 5 up, 6 in 2024-05-27T09:37:04.748 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:04 smithi119 podman[105037]: 2024-05-27 09:37:04.488949507 +0000 UTC m=+0.031326084 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:37:04.748 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:04 smithi119 podman[105037]: 2024-05-27 09:37:04.622015965 +0000 UTC m=+0.164392577 container create fe54273b7bc9245093586288961bb89dc8810ad8a893b9305cc68cf3958206e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=) 2024-05-27T09:37:04.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:04 smithi138 ceph-mon[70004]: pgmap v87: 65 pgs: 22 peering, 6 stale+active+clean, 37 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:04.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:04 smithi138 ceph-mon[70004]: Health check failed: Reduced data availability: 6 pgs peering (PG_AVAILABILITY) 2024-05-27T09:37:04.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:04 smithi138 ceph-mon[70004]: osdmap e56: 6 total, 5 up, 6 in 2024-05-27T09:37:05.220 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:04 smithi119 podman[105037]: 2024-05-27 09:37:04.88577006 +0000 UTC m=+0.428146699 container init fe54273b7bc9245093586288961bb89dc8810ad8a893b9305cc68cf3958206e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-deactivate, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-27T09:37:05.220 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:04 smithi119 podman[105037]: 2024-05-27 09:37:04.8970887 +0000 UTC m=+0.439465349 container start fe54273b7bc9245093586288961bb89dc8810ad8a893b9305cc68cf3958206e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-05-27T09:37:05.220 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:04 smithi119 podman[105037]: 2024-05-27 09:37:04.974445096 +0000 UTC m=+0.516821715 container attach fe54273b7bc9245093586288961bb89dc8810ad8a893b9305cc68cf3958206e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-27T09:37:05.498 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:05 smithi119 podman[105037]: 2024-05-27 09:37:05.351580178 +0000 UTC m=+0.893956826 container died fe54273b7bc9245093586288961bb89dc8810ad8a893b9305cc68cf3958206e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-27T09:37:06.442 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:06 smithi119 podman[105063]: 2024-05-27 09:37:06.037382007 +0000 UTC m=+0.669501836 container remove fe54273b7bc9245093586288961bb89dc8810ad8a893b9305cc68cf3958206e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, ceph=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T09:37:06.442 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:06 smithi119 systemd[1]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.1.service: Deactivated successfully. 2024-05-27T09:37:06.442 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:06 smithi119 systemd[1]: Stopped Ceph osd.1 for b966b06e-1c09-11ef-bc9b-c7b262605968. 2024-05-27T09:37:06.442 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:06 smithi119 systemd[1]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.1.service: Consumed 1min 51.417s CPU time. 2024-05-27T09:37:06.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:06 smithi119 ceph-mon[88269]: pgmap v89: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:06.748 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:06 smithi119 systemd[1]: Starting Ceph osd.1 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:37:06.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:06 smithi138 ceph-mon[70004]: pgmap v89: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:07.175 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:06 smithi119 podman[105161]: 2024-05-27 09:37:06.763090274 +0000 UTC m=+0.033550803 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:37:07.175 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:06 smithi119 podman[105161]: 2024-05-27 09:37:06.879191613 +0000 UTC m=+0.149652161 container create 4736b795f328a05b405e81c3c3c2b26ac32f69fe0762d5af713ac8f2b1fec3d4 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-activate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T09:37:07.176 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:07 smithi119 podman[105161]: 2024-05-27 09:37:07.174460973 +0000 UTC m=+0.444921558 container init 4736b795f328a05b405e81c3c3c2b26ac32f69fe0762d5af713ac8f2b1fec3d4 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-activate, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T09:37:07.496 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:07 smithi119 podman[105161]: 2024-05-27 09:37:07.185888311 +0000 UTC m=+0.456348865 container start 4736b795f328a05b405e81c3c3c2b26ac32f69fe0762d5af713ac8f2b1fec3d4 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-activate, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-27T09:37:07.496 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:07 smithi119 podman[105161]: 2024-05-27 09:37:07.264828158 +0000 UTC m=+0.535288749 container attach 4736b795f328a05b405e81c3c3c2b26ac32f69fe0762d5af713ac8f2b1fec3d4 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-activate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-27T09:37:07.751 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:07 smithi119 ceph-mon[88269]: Health check failed: Degraded data redundancy: 9/381 objects degraded (2.362%), 2 pgs degraded (PG_DEGRADED) 2024-05-27T09:37:07.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:07 smithi138 ceph-mon[70004]: Health check failed: Degraded data redundancy: 9/381 objects degraded (2.362%), 2 pgs degraded (PG_DEGRADED) 2024-05-27T09:37:08.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:08 smithi138 ceph-mon[70004]: pgmap v90: 65 pgs: 2 active+undersized+degraded, 29 peering, 3 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 9/381 objects degraded (2.362%) 2024-05-27T09:37:08.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:08 smithi119 ceph-mon[88269]: pgmap v90: 65 pgs: 2 active+undersized+degraded, 29 peering, 3 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 9/381 objects degraded (2.362%) 2024-05-27T09:37:09.417 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:09 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-activate[105176]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:37:09.417 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:09 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-activate[105176]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:37:09.417 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:09 smithi119 bash[105161]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:37:09.417 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:09 smithi119 bash[105161]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:37:09.417 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:09 smithi119 bash[105161]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-27T09:37:09.418 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:09 smithi119 bash[105161]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-27T09:37:09.418 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:09 smithi119 bash[105161]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-27T09:37:09.418 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:09 smithi119 bash[105161]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-27T09:37:09.418 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:09 smithi119 bash[105161]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-05-27T09:37:09.418 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:09 smithi119 bash[105161]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-27T09:37:09.418 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:09 smithi119 bash[105161]: --> ceph-volume raw activate successful for osd ID: 1 2024-05-27T09:37:09.418 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:09 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-activate[105176]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-27T09:37:09.418 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:09 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-activate[105176]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-27T09:37:09.418 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:09 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-activate[105176]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-27T09:37:09.418 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:09 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-activate[105176]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-27T09:37:09.418 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:09 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-activate[105176]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-05-27T09:37:09.418 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:09 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-activate[105176]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-27T09:37:09.419 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:09 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-activate[105176]: --> ceph-volume raw activate successful for osd ID: 1 2024-05-27T09:37:09.419 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:09 smithi119 podman[105161]: 2024-05-27 09:37:09.099169453 +0000 UTC m=+2.369630013 container died 4736b795f328a05b405e81c3c3c2b26ac32f69fe0762d5af713ac8f2b1fec3d4 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-activate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T09:37:09.736 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:09 smithi119 ceph-mon[88269]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-05-27T09:37:09.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:09 smithi138 ceph-mon[70004]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-05-27T09:37:09.998 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:09 smithi119 podman[105306]: 2024-05-27 09:37:09.734660996 +0000 UTC m=+0.612026949 container remove 4736b795f328a05b405e81c3c3c2b26ac32f69fe0762d5af713ac8f2b1fec3d4 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-activate, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T09:37:10.357 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:10 smithi119 podman[105365]: 2024-05-27 09:37:09.991753339 +0000 UTC m=+0.034492462 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:37:10.357 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:10 smithi119 podman[105365]: 2024-05-27 09:37:10.102362829 +0000 UTC m=+0.145101975 container create effcc7c53dd3bd55f6e6fa6153bdb88e85e4ccfc7202275c4b63a1bfe29bb26e (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T09:37:10.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:10 smithi119 ceph-mon[88269]: pgmap v91: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 58/381 objects degraded (15.223%) 2024-05-27T09:37:10.748 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:10 smithi119 podman[105365]: 2024-05-27 09:37:10.356315217 +0000 UTC m=+0.399054396 container init effcc7c53dd3bd55f6e6fa6153bdb88e85e4ccfc7202275c4b63a1bfe29bb26e (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T09:37:10.748 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:10 smithi119 podman[105365]: 2024-05-27 09:37:10.368653357 +0000 UTC m=+0.411392470 container start effcc7c53dd3bd55f6e6fa6153bdb88e85e4ccfc7202275c4b63a1bfe29bb26e (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T09:37:10.748 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:10 smithi119 bash[105365]: effcc7c53dd3bd55f6e6fa6153bdb88e85e4ccfc7202275c4b63a1bfe29bb26e 2024-05-27T09:37:10.748 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:10 smithi119 systemd[1]: Started Ceph osd.1 for b966b06e-1c09-11ef-bc9b-c7b262605968. 2024-05-27T09:37:10.748 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:10 smithi119 ceph-osd[105384]: -- 172.21.15.119:0/557553670 <== mon.0 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ac960f1680 con 0x55ac96198880 2024-05-27T09:37:10.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:10 smithi138 ceph-mon[70004]: pgmap v91: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 58/381 objects degraded (15.223%) 2024-05-27T09:37:11.497 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:11 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1[105380]: 2024-05-27T09:37:11.015+0000 7f71501857c0 -1 Falling back to public interface 2024-05-27T09:37:11.853 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:11 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:11.854 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:11 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:11.854 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:11 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:37:11.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:11 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:11.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:11 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:11.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:11 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:37:12.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:12 smithi138 ceph-mon[70004]: pgmap v92: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 58/381 objects degraded (15.223%) 2024-05-27T09:37:12.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:12.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:37:12.978 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:12 smithi119 ceph-mon[88269]: pgmap v92: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 58/381 objects degraded (15.223%) 2024-05-27T09:37:12.978 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:12.978 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:37:14.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:14 smithi138 ceph-mon[70004]: pgmap v93: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 58/381 objects degraded (15.223%) 2024-05-27T09:37:14.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:14 smithi119 ceph-mon[88269]: pgmap v93: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 58/381 objects degraded (15.223%) 2024-05-27T09:37:16.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:16 smithi119 ceph-mon[88269]: pgmap v94: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 58/381 objects degraded (15.223%) 2024-05-27T09:37:16.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:16 smithi138 ceph-mon[70004]: pgmap v94: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 58/381 objects degraded (15.223%) 2024-05-27T09:37:17.497 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:17 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1[105380]: 2024-05-27T09:37:17.198+0000 7f71501857c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-05-27T09:37:17.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:17 smithi138 ceph-mon[70004]: Health check update: Degraded data redundancy: 58/381 objects degraded (15.223%), 16 pgs degraded (PG_DEGRADED) 2024-05-27T09:37:17.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:17 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:17.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:17 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:17.898 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:17 smithi119 ceph-mon[88269]: Health check update: Degraded data redundancy: 58/381 objects degraded (15.223%), 16 pgs degraded (PG_DEGRADED) 2024-05-27T09:37:17.898 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:17 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:17.898 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:17 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:18.247 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:17 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1[105380]: 2024-05-27T09:37:17.897+0000 7f71501857c0 -1 osd.1 54 log_to_monitors true 2024-05-27T09:37:18.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:18 smithi119 ceph-mon[88269]: pgmap v95: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 58/381 objects degraded (15.223%) 2024-05-27T09:37:18.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:18 smithi119 ceph-mon[88269]: from='osd.1 [v2:172.21.15.119:6810/34735403,v1:172.21.15.119:6811/34735403]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T09:37:18.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:18 smithi138 ceph-mon[70004]: pgmap v95: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 58/381 objects degraded (15.223%) 2024-05-27T09:37:18.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:18 smithi138 ceph-mon[70004]: from='osd.1 [v2:172.21.15.119:6810/34735403,v1:172.21.15.119:6811/34735403]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T09:37:19.998 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:37:19 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1[105380]: 2024-05-27T09:37:19.719+0000 7f713a600700 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T09:37:19.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:19 smithi119 ceph-mon[88269]: from='osd.1 [v2:172.21.15.119:6810/34735403,v1:172.21.15.119:6811/34735403]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-27T09:37:19.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:19 smithi119 ceph-mon[88269]: osdmap e57: 6 total, 5 up, 6 in 2024-05-27T09:37:19.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:19 smithi119 ceph-mon[88269]: from='osd.1 [v2:172.21.15.119:6810/34735403,v1:172.21.15.119:6811/34735403]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-05-27T09:37:19.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:19 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:19.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:19 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:20.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:19 smithi138 ceph-mon[70004]: from='osd.1 [v2:172.21.15.119:6810/34735403,v1:172.21.15.119:6811/34735403]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-27T09:37:20.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:19 smithi138 ceph-mon[70004]: osdmap e57: 6 total, 5 up, 6 in 2024-05-27T09:37:20.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:19 smithi138 ceph-mon[70004]: from='osd.1 [v2:172.21.15.119:6810/34735403,v1:172.21.15.119:6811/34735403]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-05-27T09:37:20.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:19 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:20.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:19 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:20.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:20 smithi119 ceph-mon[88269]: pgmap v97: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/381 objects degraded (15.223%) 2024-05-27T09:37:20.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:20 smithi119 ceph-mon[88269]: from='osd.1 [v2:172.21.15.119:6810/34735403,v1:172.21.15.119:6811/34735403]' entity='osd.1' 2024-05-27T09:37:21.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:20 smithi138 ceph-mon[70004]: pgmap v97: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/381 objects degraded (15.223%) 2024-05-27T09:37:21.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:20 smithi138 ceph-mon[70004]: from='osd.1 [v2:172.21.15.119:6810/34735403,v1:172.21.15.119:6811/34735403]' entity='osd.1' 2024-05-27T09:37:22.091 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:21 smithi119 ceph-mon[88269]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-27T09:37:22.091 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:21 smithi119 ceph-mon[88269]: osd.1 [v2:172.21.15.119:6810/34735403,v1:172.21.15.119:6811/34735403] boot 2024-05-27T09:37:22.091 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:21 smithi119 ceph-mon[88269]: osdmap e58: 6 total, 6 up, 6 in 2024-05-27T09:37:22.091 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:21 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T09:37:22.091 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:21 smithi119 ceph-mon[88269]: osdmap e59: 6 total, 6 up, 6 in 2024-05-27T09:37:22.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:21 smithi138 ceph-mon[70004]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-27T09:37:22.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:21 smithi138 ceph-mon[70004]: osd.1 [v2:172.21.15.119:6810/34735403,v1:172.21.15.119:6811/34735403] boot 2024-05-27T09:37:22.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:21 smithi138 ceph-mon[70004]: osdmap e58: 6 total, 6 up, 6 in 2024-05-27T09:37:22.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:21 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T09:37:22.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:21 smithi138 ceph-mon[70004]: osdmap e59: 6 total, 6 up, 6 in 2024-05-27T09:37:23.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:22 smithi138 ceph-mon[70004]: pgmap v99: 65 pgs: 2 peering, 16 active+undersized+degraded, 16 active+undersized, 31 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/381 objects degraded (15.223%) 2024-05-27T09:37:23.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:22 smithi119 ceph-mon[88269]: pgmap v99: 65 pgs: 2 peering, 16 active+undersized+degraded, 16 active+undersized, 31 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/381 objects degraded (15.223%) 2024-05-27T09:37:23.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:23 smithi138 ceph-mon[70004]: Health check update: Degraded data redundancy: 25/381 objects degraded (6.562%), 7 pgs degraded (PG_DEGRADED) 2024-05-27T09:37:24.100 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:23 smithi119 ceph-mon[88269]: Health check update: Degraded data redundancy: 25/381 objects degraded (6.562%), 7 pgs degraded (PG_DEGRADED) 2024-05-27T09:37:25.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:24 smithi138 ceph-mon[70004]: pgmap v101: 65 pgs: 2 peering, 7 active+undersized+degraded, 5 active+undersized, 51 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 25/381 objects degraded (6.562%) 2024-05-27T09:37:25.219 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:24 smithi119 ceph-mon[88269]: pgmap v101: 65 pgs: 2 peering, 7 active+undersized+degraded, 5 active+undersized, 51 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 25/381 objects degraded (6.562%) 2024-05-27T09:37:26.372 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:26 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:26.372 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:26 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:26.372 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:26 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:37:26.372 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:26 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:37:26.372 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:26 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:26.373 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:26 smithi119 ceph-mon[88269]: pgmap v102: 65 pgs: 2 peering, 63 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:26.373 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:26 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:37:26.373 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:26 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:37:26.373 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:26 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:37:26.373 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:26 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:37:26.373 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:26 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-27T09:37:26.373 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:26 smithi119 ceph-mon[88269]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-27T09:37:26.373 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:26 smithi119 ceph-mon[88269]: Upgrade: osd.2 is safe to restart 2024-05-27T09:37:26.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:26 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:26.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:26 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:26.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:26 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:37:26.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:26 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:37:26.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:26 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:26.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:26 smithi138 ceph-mon[70004]: pgmap v102: 65 pgs: 2 peering, 63 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:26.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:26 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:37:26.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:26 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:37:26.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:26 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:37:26.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:26 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:37:26.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:26 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-27T09:37:26.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:26 smithi138 ceph-mon[70004]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-27T09:37:26.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:26 smithi138 ceph-mon[70004]: Upgrade: osd.2 is safe to restart 2024-05-27T09:37:27.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:27 smithi138 ceph-mon[70004]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 25/381 objects degraded (6.562%), 7 pgs degraded) 2024-05-27T09:37:27.417 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:27 smithi119 ceph-mon[88269]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 25/381 objects degraded (6.562%), 7 pgs degraded) 2024-05-27T09:37:27.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.764+0000 7ffbcb200700 1 -- 172.21.15.119:0/1043624232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc41003d0 msgr2=0x7ffbc41007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:27.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.764+0000 7ffbcb200700 1 --2- 172.21.15.119:0/1043624232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc41003d0 0x7ffbc41007a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4009910 tx=0x7ffbb4009c20 comp rx=0 tx=0).stop 2024-05-27T09:37:27.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.764+0000 7ffbcb200700 1 -- 172.21.15.119:0/1043624232 shutdown_connections 2024-05-27T09:37:27.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.764+0000 7ffbcb200700 1 --2- 172.21.15.119:0/1043624232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc4100d70 0x7ffbc4104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:27.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.764+0000 7ffbcb200700 1 --2- 172.21.15.119:0/1043624232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc41003d0 0x7ffbc41007a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:27.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.764+0000 7ffbcb200700 1 -- 172.21.15.119:0/1043624232 >> 172.21.15.119:0/1043624232 conn(0x7ffbc40fbc90 msgr2=0x7ffbc40fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:27.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.764+0000 7ffbcb200700 1 -- 172.21.15.119:0/1043624232 shutdown_connections 2024-05-27T09:37:27.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.765+0000 7ffbcb200700 1 -- 172.21.15.119:0/1043624232 wait complete. 2024-05-27T09:37:27.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.766+0000 7ffbcb200700 1 Processor -- start 2024-05-27T09:37:27.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.766+0000 7ffbcb200700 1 -- start start 2024-05-27T09:37:27.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.766+0000 7ffbcb200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc41003d0 0x7ffbc4198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:27.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.766+0000 7ffbcb200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc4100d70 0x7ffbc4198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:27.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.766+0000 7ffbcb200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc4198c50 con 0x7ffbc4100d70 2024-05-27T09:37:27.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.766+0000 7ffbcb200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc419c9e0 con 0x7ffbc41003d0 2024-05-27T09:37:27.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.766+0000 7ffbc3400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc4100d70 0x7ffbc4198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:27.768 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.767+0000 7ffbc3400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc4100d70 0x7ffbc4198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55060/0 (socket says 172.21.15.119:55060) 2024-05-27T09:37:27.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.767+0000 7ffbc3400700 1 -- 172.21.15.119:0/532650934 learned_addr learned my addr 172.21.15.119:0/532650934 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:37:27.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.767+0000 7ffbc3e00700 1 --2- 172.21.15.119:0/532650934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc41003d0 0x7ffbc4198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:27.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.767+0000 7ffbc3400700 1 -- 172.21.15.119:0/532650934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc41003d0 msgr2=0x7ffbc4198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:27.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.767+0000 7ffbc3400700 1 --2- 172.21.15.119:0/532650934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc41003d0 0x7ffbc4198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:27.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.767+0000 7ffbc3400700 1 -- 172.21.15.119:0/532650934 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb40095f0 con 0x7ffbc4100d70 2024-05-27T09:37:27.769 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.767+0000 7ffbc3400700 1 --2- 172.21.15.119:0/532650934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc4100d70 0x7ffbc4198570 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000b480 tx=0x7ffbb000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:27.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.768+0000 7ffbc0c00700 1 -- 172.21.15.119:0/532650934 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbb00118c0 con 0x7ffbc4100d70 2024-05-27T09:37:27.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.768+0000 7ffbc0c00700 1 -- 172.21.15.119:0/532650934 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffbb0010380 con 0x7ffbc4100d70 2024-05-27T09:37:27.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.768+0000 7ffbcb200700 1 -- 172.21.15.119:0/532650934 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc419ccc0 con 0x7ffbc4100d70 2024-05-27T09:37:27.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.768+0000 7ffbc0c00700 1 -- 172.21.15.119:0/532650934 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbb000f4f0 con 0x7ffbc4100d70 2024-05-27T09:37:27.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.768+0000 7ffbcb200700 1 -- 172.21.15.119:0/532650934 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc419d210 con 0x7ffbc4100d70 2024-05-27T09:37:27.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.769+0000 7ffbcb200700 1 -- 172.21.15.119:0/532650934 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc41044c0 con 0x7ffbc4100d70 2024-05-27T09:37:27.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.770+0000 7ffbc0c00700 1 -- 172.21.15.119:0/532650934 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7ffbb00104f0 con 0x7ffbc4100d70 2024-05-27T09:37:27.771 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.771+0000 7ffbc0c00700 1 --2- 172.21.15.119:0/532650934 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ffba8078a40 0x7ffba807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:27.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.771+0000 7ffbc0c00700 1 -- 172.21.15.119:0/532650934 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ffbb009a8e0 con 0x7ffbc4100d70 2024-05-27T09:37:27.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.771+0000 7ffbc3e00700 1 --2- 172.21.15.119:0/532650934 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ffba8078a40 0x7ffba807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:27.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.772+0000 7ffbc3e00700 1 --2- 172.21.15.119:0/532650934 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ffba8078a40 0x7ffba807aef0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4006010 tx=0x7ffbb401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:27.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:27.777+0000 7ffbc0c00700 1 -- 172.21.15.119:0/532650934 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7ffbb0062560 con 0x7ffbc4100d70 2024-05-27T09:37:28.046 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.044+0000 7ffbcb200700 1 -- 172.21.15.119:0/532650934 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffbc419d5d0 con 0x7ffba8078a40 2024-05-27T09:37:28.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.046+0000 7ffbc0c00700 1 -- 172.21.15.119:0/532650934 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ffbc419d5d0 con 0x7ffba8078a40 2024-05-27T09:37:28.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.049+0000 7ffbcb200700 1 -- 172.21.15.119:0/532650934 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ffba8078a40 msgr2=0x7ffba807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:28.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.049+0000 7ffbcb200700 1 --2- 172.21.15.119:0/532650934 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ffba8078a40 0x7ffba807aef0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4006010 tx=0x7ffbb401a040 comp rx=0 tx=0).stop 2024-05-27T09:37:28.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.049+0000 7ffbcb200700 1 -- 172.21.15.119:0/532650934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc4100d70 msgr2=0x7ffbc4198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:28.050 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.049+0000 7ffbcb200700 1 --2- 172.21.15.119:0/532650934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc4100d70 0x7ffbc4198570 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000b480 tx=0x7ffbb000b840 comp rx=0 tx=0).stop 2024-05-27T09:37:28.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.049+0000 7ffbcb200700 1 -- 172.21.15.119:0/532650934 shutdown_connections 2024-05-27T09:37:28.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.049+0000 7ffbcb200700 1 --2- 172.21.15.119:0/532650934 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ffba8078a40 0x7ffba807aef0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.049+0000 7ffbcb200700 1 --2- 172.21.15.119:0/532650934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc41003d0 0x7ffbc4198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.049+0000 7ffbcb200700 1 --2- 172.21.15.119:0/532650934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc4100d70 0x7ffbc4198570 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.049+0000 7ffbcb200700 1 -- 172.21.15.119:0/532650934 >> 172.21.15.119:0/532650934 conn(0x7ffbc40fbc90 msgr2=0x7ffbc40fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:28.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.050+0000 7ffbcb200700 1 -- 172.21.15.119:0/532650934 shutdown_connections 2024-05-27T09:37:28.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.050+0000 7ffbcb200700 1 -- 172.21.15.119:0/532650934 wait complete. 2024-05-27T09:37:28.068 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:37:28.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.207+0000 7f9f8a200700 1 -- 172.21.15.119:0/3931944203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f840ff7c0 msgr2=0x7f9f8410c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:28.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.207+0000 7f9f8a200700 1 --2- 172.21.15.119:0/3931944203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f840ff7c0 0x7f9f8410c520 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009910 tx=0x7f9f6c009c20 comp rx=0 tx=0).stop 2024-05-27T09:37:28.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.208+0000 7f9f8a200700 1 -- 172.21.15.119:0/3931944203 shutdown_connections 2024-05-27T09:37:28.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.208+0000 7f9f8a200700 1 --2- 172.21.15.119:0/3931944203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f840ff7c0 0x7f9f8410c520 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.208+0000 7f9f8a200700 1 --2- 172.21.15.119:0/3931944203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f840feeb0 0x7f9f840ff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.208+0000 7f9f8a200700 1 -- 172.21.15.119:0/3931944203 >> 172.21.15.119:0/3931944203 conn(0x7f9f840fab80 msgr2=0x7f9f840fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:28.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.208+0000 7f9f8a200700 1 -- 172.21.15.119:0/3931944203 shutdown_connections 2024-05-27T09:37:28.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.208+0000 7f9f8a200700 1 -- 172.21.15.119:0/3931944203 wait complete. 2024-05-27T09:37:28.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.209+0000 7f9f8a200700 1 Processor -- start 2024-05-27T09:37:28.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.209+0000 7f9f8a200700 1 -- start start 2024-05-27T09:37:28.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.209+0000 7f9f8a200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f840feeb0 0x7f9f84197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:28.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.209+0000 7f9f8a200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f840ff7c0 0x7f9f841983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:28.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.209+0000 7f9f8a200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f84198a40 con 0x7f9f840ff7c0 2024-05-27T09:37:28.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.209+0000 7f9f8a200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f84198b80 con 0x7f9f840feeb0 2024-05-27T09:37:28.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.210+0000 7f9f7be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f840ff7c0 0x7f9f841983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:28.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.210+0000 7f9f7be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f840ff7c0 0x7f9f841983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55072/0 (socket says 172.21.15.119:55072) 2024-05-27T09:37:28.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.210+0000 7f9f7be00700 1 -- 172.21.15.119:0/2349115553 learned_addr learned my addr 172.21.15.119:0/2349115553 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:37:28.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.210+0000 7f9f83400700 1 --2- 172.21.15.119:0/2349115553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f840feeb0 0x7f9f84197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:28.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.210+0000 7f9f7be00700 1 -- 172.21.15.119:0/2349115553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f840feeb0 msgr2=0x7f9f84197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:28.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.210+0000 7f9f7be00700 1 --2- 172.21.15.119:0/2349115553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f840feeb0 0x7f9f84197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.210+0000 7f9f7be00700 1 -- 172.21.15.119:0/2349115553 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f6c0095f0 con 0x7f9f840ff7c0 2024-05-27T09:37:28.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.211+0000 7f9f7be00700 1 --2- 172.21.15.119:0/2349115553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f840ff7c0 0x7f9f841983f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c006010 tx=0x7f9f6c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:28.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.211+0000 7f9f80c00700 1 -- 172.21.15.119:0/2349115553 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f6c01d070 con 0x7f9f840ff7c0 2024-05-27T09:37:28.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.211+0000 7f9f80c00700 1 -- 172.21.15.119:0/2349115553 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f6c004fb0 con 0x7f9f840ff7c0 2024-05-27T09:37:28.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.211+0000 7f9f8a200700 1 -- 172.21.15.119:0/2349115553 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f8419c9d0 con 0x7f9f840ff7c0 2024-05-27T09:37:28.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.211+0000 7f9f8a200700 1 -- 172.21.15.119:0/2349115553 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8419cf20 con 0x7f9f840ff7c0 2024-05-27T09:37:28.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.211+0000 7f9f80c00700 1 -- 172.21.15.119:0/2349115553 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f6c022ea0 con 0x7f9f840ff7c0 2024-05-27T09:37:28.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.212+0000 7f9f8a200700 1 -- 172.21.15.119:0/2349115553 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f60005320 con 0x7f9f840ff7c0 2024-05-27T09:37:28.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.214+0000 7f9f80c00700 1 -- 172.21.15.119:0/2349115553 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f9f6c004af0 con 0x7f9f840ff7c0 2024-05-27T09:37:28.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.215+0000 7f9f80c00700 1 --2- 172.21.15.119:0/2349115553 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9f64078a40 0x7f9f6407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:28.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.215+0000 7f9f80c00700 1 -- 172.21.15.119:0/2349115553 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f9f6c09c1b0 con 0x7f9f840ff7c0 2024-05-27T09:37:28.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.215+0000 7f9f83400700 1 --2- 172.21.15.119:0/2349115553 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9f64078a40 0x7f9f6407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:28.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.216+0000 7f9f83400700 1 --2- 172.21.15.119:0/2349115553 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9f64078a40 0x7f9f6407aef0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9f74005ea0 tx=0x7f9f74005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:28.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.220+0000 7f9f80c00700 1 -- 172.21.15.119:0/2349115553 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f9f6c063ee0 con 0x7f9f840ff7c0 2024-05-27T09:37:28.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.487+0000 7f9f8a200700 1 -- 172.21.15.119:0/2349115553 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f60000bf0 con 0x7f9f64078a40 2024-05-27T09:37:28.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.489+0000 7f9f80c00700 1 -- 172.21.15.119:0/2349115553 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f9f60000bf0 con 0x7f9f64078a40 2024-05-27T09:37:28.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.492+0000 7f9f8a200700 1 -- 172.21.15.119:0/2349115553 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9f64078a40 msgr2=0x7f9f6407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:28.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.492+0000 7f9f8a200700 1 --2- 172.21.15.119:0/2349115553 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9f64078a40 0x7f9f6407aef0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9f74005ea0 tx=0x7f9f74005e30 comp rx=0 tx=0).stop 2024-05-27T09:37:28.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.492+0000 7f9f8a200700 1 -- 172.21.15.119:0/2349115553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f840ff7c0 msgr2=0x7f9f841983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:28.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.492+0000 7f9f8a200700 1 --2- 172.21.15.119:0/2349115553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f840ff7c0 0x7f9f841983f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c006010 tx=0x7f9f6c00be20 comp rx=0 tx=0).stop 2024-05-27T09:37:28.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.493+0000 7f9f8a200700 1 -- 172.21.15.119:0/2349115553 shutdown_connections 2024-05-27T09:37:28.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.493+0000 7f9f8a200700 1 --2- 172.21.15.119:0/2349115553 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9f64078a40 0x7f9f6407aef0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.493+0000 7f9f8a200700 1 --2- 172.21.15.119:0/2349115553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f840feeb0 0x7f9f84197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.493+0000 7f9f8a200700 1 --2- 172.21.15.119:0/2349115553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f840ff7c0 0x7f9f841983f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.493+0000 7f9f8a200700 1 -- 172.21.15.119:0/2349115553 >> 172.21.15.119:0/2349115553 conn(0x7f9f840fab80 msgr2=0x7f9f840fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:28.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.493+0000 7f9f8a200700 1 -- 172.21.15.119:0/2349115553 shutdown_connections 2024-05-27T09:37:28.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.493+0000 7f9f8a200700 1 -- 172.21.15.119:0/2349115553 wait complete. 2024-05-27T09:37:28.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:28 smithi138 ceph-mon[70004]: pgmap v103: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:28.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:28 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:28.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:28 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:37:28.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:28 smithi138 ceph-mon[70004]: Upgrade: Updating osd.2 2024-05-27T09:37:28.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:28 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:28.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:28 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-27T09:37:28.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:28 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:37:28.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:28 smithi138 ceph-mon[70004]: Deploying daemon osd.2 on smithi119 2024-05-27T09:37:28.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:28 smithi138 ceph-mon[70004]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:37:28.640 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:28 smithi119 ceph-mon[88269]: pgmap v103: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:28.640 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:28.640 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:37:28.640 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:28 smithi119 ceph-mon[88269]: Upgrade: Updating osd.2 2024-05-27T09:37:28.640 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:28.640 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-27T09:37:28.640 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:37:28.640 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:28 smithi119 ceph-mon[88269]: Deploying daemon osd.2 on smithi119 2024-05-27T09:37:28.640 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:28 smithi119 ceph-mon[88269]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:37:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.645+0000 7eff18400700 1 -- 172.21.15.119:0/1125240171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff1010c010 msgr2=0x7eff10106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.645+0000 7eff18400700 1 --2- 172.21.15.119:0/1125240171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff1010c010 0x7eff10106b80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efefc009910 tx=0x7efefc009c20 comp rx=0 tx=0).stop 2024-05-27T09:37:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.646+0000 7eff18400700 1 -- 172.21.15.119:0/1125240171 shutdown_connections 2024-05-27T09:37:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.646+0000 7eff18400700 1 --2- 172.21.15.119:0/1125240171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff1010c010 0x7eff10106b80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.646+0000 7eff18400700 1 --2- 172.21.15.119:0/1125240171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff1010b700 0x7eff1010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.646+0000 7eff18400700 1 -- 172.21.15.119:0/1125240171 >> 172.21.15.119:0/1125240171 conn(0x7eff10076000 msgr2=0x7eff10078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.646+0000 7eff18400700 1 -- 172.21.15.119:0/1125240171 shutdown_connections 2024-05-27T09:37:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.646+0000 7eff18400700 1 -- 172.21.15.119:0/1125240171 wait complete. 2024-05-27T09:37:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.647+0000 7eff18400700 1 Processor -- start 2024-05-27T09:37:28.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.647+0000 7eff18400700 1 -- start start 2024-05-27T09:37:28.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.648+0000 7eff18400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff1010b700 0x7eff1019c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:28.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.648+0000 7eff18400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff1010c010 0x7eff1019c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:28.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.648+0000 7eff18400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1019cf00 con 0x7eff1010b700 2024-05-27T09:37:28.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.648+0000 7eff18400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff101a0c90 con 0x7eff1010c010 2024-05-27T09:37:28.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.648+0000 7eff15a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff1010b700 0x7eff1019c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:28.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.648+0000 7eff15a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff1010b700 0x7eff1019c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55094/0 (socket says 172.21.15.119:55094) 2024-05-27T09:37:28.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.648+0000 7eff15a00700 1 -- 172.21.15.119:0/3271415566 learned_addr learned my addr 172.21.15.119:0/3271415566 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:37:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.648+0000 7eff15000700 1 --2- 172.21.15.119:0/3271415566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff1010c010 0x7eff1019c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.649+0000 7eff15a00700 1 -- 172.21.15.119:0/3271415566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff1010c010 msgr2=0x7eff1019c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.649+0000 7eff15a00700 1 --2- 172.21.15.119:0/3271415566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff1010c010 0x7eff1019c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.649+0000 7eff15a00700 1 -- 172.21.15.119:0/3271415566 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efefc0095f0 con 0x7eff1010b700 2024-05-27T09:37:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.649+0000 7eff15000700 1 --2- 172.21.15.119:0/3271415566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff1010c010 0x7eff1019c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:37:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.649+0000 7eff15a00700 1 --2- 172.21.15.119:0/3271415566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff1010b700 0x7eff1019c2e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7eff0400d8d0 tx=0x7eff0400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.649+0000 7eff0e000700 1 -- 172.21.15.119:0/3271415566 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff04009940 con 0x7eff1010b700 2024-05-27T09:37:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.649+0000 7eff0e000700 1 -- 172.21.15.119:0/3271415566 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff04010460 con 0x7eff1010b700 2024-05-27T09:37:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.649+0000 7eff18400700 1 -- 172.21.15.119:0/3271415566 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff101a0f70 con 0x7eff1010b700 2024-05-27T09:37:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.649+0000 7eff0e000700 1 -- 172.21.15.119:0/3271415566 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff0400f5f0 con 0x7eff1010b700 2024-05-27T09:37:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.649+0000 7eff18400700 1 -- 172.21.15.119:0/3271415566 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff101a1540 con 0x7eff1010b700 2024-05-27T09:37:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.651+0000 7eff18400700 1 -- 172.21.15.119:0/3271415566 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef0005320 con 0x7eff1010b700 2024-05-27T09:37:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.652+0000 7eff0e000700 1 -- 172.21.15.119:0/3271415566 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7eff0400f870 con 0x7eff1010b700 2024-05-27T09:37:28.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.653+0000 7eff0e000700 1 --2- 172.21.15.119:0/3271415566 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efef8078a40 0x7efef807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:28.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.653+0000 7eff0e000700 1 -- 172.21.15.119:0/3271415566 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6414+0+0 (secure 0 0 0) 0x7eff0409ad20 con 0x7eff1010b700 2024-05-27T09:37:28.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.654+0000 7eff15000700 1 --2- 172.21.15.119:0/3271415566 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efef8078a40 0x7efef807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:28.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.654+0000 7eff15000700 1 --2- 172.21.15.119:0/3271415566 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efef8078a40 0x7efef807aef0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efefc006010 tx=0x7efefc00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:28.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.658+0000 7eff0e000700 1 -- 172.21.15.119:0/3271415566 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7eff04062900 con 0x7eff1010b700 2024-05-27T09:37:28.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.910+0000 7eff18400700 1 -- 172.21.15.119:0/3271415566 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efef0000bf0 con 0x7efef8078a40 2024-05-27T09:37:28.942 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.942+0000 7eff0e000700 1 -- 172.21.15.119:0/3271415566 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7efef0000bf0 con 0x7efef8078a40 2024-05-27T09:37:28.943 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:37:28.943 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (4m) 11s ago 18m 14.2M - 0.25.0 c8568f914cd2 6b90564dcef7 2024-05-27T09:37:28.943 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (18m) 11s ago 18m 8606k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:37:28.943 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (17m) 77s ago 17m 8182k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:37:28.943 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (91s) 11s ago 18m 6669k - 19.0.0-3868-ga6507388 850a06a84d28 011d811ce7fc 2024-05-27T09:37:28.943 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (82s) 77s ago 17m 6665k - 19.0.0-3868-ga6507388 850a06a84d28 b546b3ed1afc 2024-05-27T09:37:28.943 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (4m) 11s ago 18m 74.3M - 9.4.12 2bacad6d85d8 b3abdab08db4 2024-05-27T09:37:28.943 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (11m) 11s ago 11m 17.8M - 18.2.0 dc2bc1663786 93b21713fb1e 2024-05-27T09:37:28.943 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (11m) 11s ago 11m 130M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:37:28.944 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (11m) 76s ago 11m 17.0M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:37:28.944 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (11m) 76s ago 11m 128M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:37:28.944 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (7m) 11s ago 20m 586M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:37:28.944 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (6m) 77s ago 17m 532M - 19.0.0-3868-ga6507388 850a06a84d28 2fc3a80696fa 2024-05-27T09:37:28.944 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (3m) 11s ago 20m 114M 2048M 19.0.0-3868-ga6507388 850a06a84d28 3e5c7a9ad372 2024-05-27T09:37:28.944 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (2m) 77s ago 17m 103M 2048M 19.0.0-3868-ga6507388 850a06a84d28 e4d471274cf1 2024-05-27T09:37:28.944 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (5m) 11s ago 18m 7734k - 1.5.0 0da6a335fe13 709c37815cff 2024-05-27T09:37:28.944 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (5m) 77s ago 17m 7776k - 1.5.0 0da6a335fe13 999047a1a5bf 2024-05-27T09:37:28.944 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (50s) 11s ago 16m 152M 1216M 19.0.0-3868-ga6507388 850a06a84d28 5a0e162d870e 2024-05-27T09:37:28.944 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (18s) 11s ago 15m 78.5M 1216M 19.0.0-3868-ga6507388 850a06a84d28 effcc7c53dd3 2024-05-27T09:37:28.944 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (15m) 11s ago 15m 189M 1216M 18.2.0 dc2bc1663786 3955a6112c1c 2024-05-27T09:37:28.944 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (14m) 77s ago 14m 270M 2240M 18.2.0 dc2bc1663786 693c92220622 2024-05-27T09:37:28.944 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (13m) 77s ago 13m 248M 2240M 18.2.0 dc2bc1663786 7559e658e917 2024-05-27T09:37:28.944 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (13m) 76s ago 13m 232M 2240M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:37:28.944 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (5m) 11s ago 18m 71.0M - 2.43.0 a07b618ecd1d cba50454ed21 2024-05-27T09:37:28.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.945+0000 7eff18400700 1 -- 172.21.15.119:0/3271415566 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efef8078a40 msgr2=0x7efef807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:28.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.945+0000 7eff18400700 1 --2- 172.21.15.119:0/3271415566 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efef8078a40 0x7efef807aef0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efefc006010 tx=0x7efefc00b540 comp rx=0 tx=0).stop 2024-05-27T09:37:28.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.946+0000 7eff18400700 1 -- 172.21.15.119:0/3271415566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff1010b700 msgr2=0x7eff1019c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:28.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.946+0000 7eff18400700 1 --2- 172.21.15.119:0/3271415566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff1010b700 0x7eff1019c2e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7eff0400d8d0 tx=0x7eff0400dc90 comp rx=0 tx=0).stop 2024-05-27T09:37:28.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.946+0000 7eff18400700 1 -- 172.21.15.119:0/3271415566 shutdown_connections 2024-05-27T09:37:28.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.946+0000 7eff18400700 1 --2- 172.21.15.119:0/3271415566 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efef8078a40 0x7efef807aef0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.946+0000 7eff18400700 1 --2- 172.21.15.119:0/3271415566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff1010b700 0x7eff1019c2e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.946+0000 7eff18400700 1 --2- 172.21.15.119:0/3271415566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff1010c010 0x7eff1019c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:28.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.946+0000 7eff18400700 1 -- 172.21.15.119:0/3271415566 >> 172.21.15.119:0/3271415566 conn(0x7eff10076000 msgr2=0x7eff10076940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:28.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.946+0000 7eff18400700 1 -- 172.21.15.119:0/3271415566 shutdown_connections 2024-05-27T09:37:28.947 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:28.946+0000 7eff18400700 1 -- 172.21.15.119:0/3271415566 wait complete. 2024-05-27T09:37:29.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.105+0000 7fdee5400700 1 -- 172.21.15.119:0/2769381207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdee0102fa0 msgr2=0x7fdee0106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:29.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.105+0000 7fdee5400700 1 --2- 172.21.15.119:0/2769381207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdee0102fa0 0x7fdee0106f70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdec8009910 tx=0x7fdec8009c20 comp rx=0 tx=0).stop 2024-05-27T09:37:29.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.105+0000 7fdee5400700 1 -- 172.21.15.119:0/2769381207 shutdown_connections 2024-05-27T09:37:29.106 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.105+0000 7fdee5400700 1 --2- 172.21.15.119:0/2769381207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdee0102fa0 0x7fdee0106f70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:29.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.105+0000 7fdee5400700 1 --2- 172.21.15.119:0/2769381207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdee0102600 0x7fdee01029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:29.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.105+0000 7fdee5400700 1 -- 172.21.15.119:0/2769381207 >> 172.21.15.119:0/2769381207 conn(0x7fdee00fde60 msgr2=0x7fdee0100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:29.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.105+0000 7fdee5400700 1 -- 172.21.15.119:0/2769381207 shutdown_connections 2024-05-27T09:37:29.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.105+0000 7fdee5400700 1 -- 172.21.15.119:0/2769381207 wait complete. 2024-05-27T09:37:29.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.106+0000 7fdee5400700 1 Processor -- start 2024-05-27T09:37:29.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.106+0000 7fdee5400700 1 -- start start 2024-05-27T09:37:29.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.107+0000 7fdee5400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdee0102600 0x7fdee01980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:29.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.107+0000 7fdee5400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdee0102fa0 0x7fdee01985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:29.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.107+0000 7fdee5400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee0198cd0 con 0x7fdee0102600 2024-05-27T09:37:29.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.107+0000 7fdee5400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee019ca60 con 0x7fdee0102fa0 2024-05-27T09:37:29.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.107+0000 7fdede800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdee0102600 0x7fdee01980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:29.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.107+0000 7fdede800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdee0102600 0x7fdee01980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42214/0 (socket says 172.21.15.119:42214) 2024-05-27T09:37:29.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.107+0000 7fdede800700 1 -- 172.21.15.119:0/1851184978 learned_addr learned my addr 172.21.15.119:0/1851184978 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:37:29.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.107+0000 7fded7e00700 1 --2- 172.21.15.119:0/1851184978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdee0102fa0 0x7fdee01985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:29.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.108+0000 7fdede800700 1 -- 172.21.15.119:0/1851184978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdee0102fa0 msgr2=0x7fdee01985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:29.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.108+0000 7fdede800700 1 --2- 172.21.15.119:0/1851184978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdee0102fa0 0x7fdee01985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:29.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.108+0000 7fdede800700 1 -- 172.21.15.119:0/1851184978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec80095f0 con 0x7fdee0102600 2024-05-27T09:37:29.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.108+0000 7fdede800700 1 --2- 172.21.15.119:0/1851184978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdee0102600 0x7fdee01980b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fded000b4b0 tx=0x7fded000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:29.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.108+0000 7fded7400700 1 -- 172.21.15.119:0/1851184978 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fded0011800 con 0x7fdee0102600 2024-05-27T09:37:29.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.108+0000 7fded7400700 1 -- 172.21.15.119:0/1851184978 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fded0011e20 con 0x7fdee0102600 2024-05-27T09:37:29.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.108+0000 7fded7400700 1 -- 172.21.15.119:0/1851184978 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fded000f2e0 con 0x7fdee0102600 2024-05-27T09:37:29.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.109+0000 7fdee5400700 1 -- 172.21.15.119:0/1851184978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee019cd40 con 0x7fdee0102600 2024-05-27T09:37:29.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.109+0000 7fdee5400700 1 -- 172.21.15.119:0/1851184978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee019d240 con 0x7fdee0102600 2024-05-27T09:37:29.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.110+0000 7fdee5400700 1 -- 172.21.15.119:0/1851184978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee010a8a0 con 0x7fdee0102600 2024-05-27T09:37:29.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.111+0000 7fded7400700 1 -- 172.21.15.119:0/1851184978 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fded0010380 con 0x7fdee0102600 2024-05-27T09:37:29.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.112+0000 7fded7400700 1 --2- 172.21.15.119:0/1851184978 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdec4078a40 0x7fdec407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:29.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.112+0000 7fded7400700 1 -- 172.21.15.119:0/1851184978 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fded009a8d0 con 0x7fdee0102600 2024-05-27T09:37:29.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.112+0000 7fded7e00700 1 --2- 172.21.15.119:0/1851184978 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdec4078a40 0x7fdec407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:29.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.113+0000 7fded7e00700 1 --2- 172.21.15.119:0/1851184978 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdec4078a40 0x7fdec407aef0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdee01996d0 tx=0x7fdec801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:29.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.117+0000 7fded7400700 1 -- 172.21.15.119:0/1851184978 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fded00624b0 con 0x7fdee0102600 2024-05-27T09:37:29.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.450+0000 7fdee5400700 1 -- 172.21.15.119:0/1851184978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdee019d570 con 0x7fdee0102600 2024-05-27T09:37:29.452 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:29 smithi119 ceph-mon[88269]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:37:29.452 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:29 smithi119 ceph-mon[88269]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:37:29.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.452+0000 7fded7400700 1 -- 172.21.15.119:0/1851184978 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fded0061e50 con 0x7fdee0102600 2024-05-27T09:37:29.453 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:37:29.453 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:37:29.453 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:37:29.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:37:29.453 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:37:29.453 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:37:29.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:37:29.453 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:37:29.453 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4, 2024-05-27T09:37:29.453 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:37:29.453 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:37:29.454 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:37:29.454 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-05-27T09:37:29.454 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:37:29.454 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:37:29.454 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 8, 2024-05-27T09:37:29.454 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 6 2024-05-27T09:37:29.454 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:37:29.454 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:37:29.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.455+0000 7fdee5400700 1 -- 172.21.15.119:0/1851184978 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdec4078a40 msgr2=0x7fdec407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:29.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.455+0000 7fdee5400700 1 --2- 172.21.15.119:0/1851184978 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdec4078a40 0x7fdec407aef0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdee01996d0 tx=0x7fdec801a040 comp rx=0 tx=0).stop 2024-05-27T09:37:29.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.455+0000 7fdee5400700 1 -- 172.21.15.119:0/1851184978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdee0102600 msgr2=0x7fdee01980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:29.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.455+0000 7fdee5400700 1 --2- 172.21.15.119:0/1851184978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdee0102600 0x7fdee01980b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fded000b4b0 tx=0x7fded000b870 comp rx=0 tx=0).stop 2024-05-27T09:37:29.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.455+0000 7fdee5400700 1 -- 172.21.15.119:0/1851184978 shutdown_connections 2024-05-27T09:37:29.456 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.455+0000 7fdee5400700 1 --2- 172.21.15.119:0/1851184978 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdec4078a40 0x7fdec407aef0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:29.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.455+0000 7fdee5400700 1 --2- 172.21.15.119:0/1851184978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdee0102600 0x7fdee01980b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:29.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.455+0000 7fdee5400700 1 --2- 172.21.15.119:0/1851184978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdee0102fa0 0x7fdee01985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:29.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.455+0000 7fdee5400700 1 -- 172.21.15.119:0/1851184978 >> 172.21.15.119:0/1851184978 conn(0x7fdee00fde60 msgr2=0x7fdee00ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:29.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.456+0000 7fdee5400700 1 -- 172.21.15.119:0/1851184978 shutdown_connections 2024-05-27T09:37:29.457 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.456+0000 7fdee5400700 1 -- 172.21.15.119:0/1851184978 wait complete. 2024-05-27T09:37:29.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.610+0000 7fee02a00700 1 -- 172.21.15.119:0/3401193776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedfc1003b0 msgr2=0x7fedfc100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:29.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.610+0000 7fee02a00700 1 --2- 172.21.15.119:0/3401193776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedfc1003b0 0x7fedfc100780 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fedec009910 tx=0x7fedec009c20 comp rx=0 tx=0).stop 2024-05-27T09:37:29.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.610+0000 7fee02a00700 1 -- 172.21.15.119:0/3401193776 shutdown_connections 2024-05-27T09:37:29.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.610+0000 7fee02a00700 1 --2- 172.21.15.119:0/3401193776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedfc100d50 0x7fedfc104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:29.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.610+0000 7fee02a00700 1 --2- 172.21.15.119:0/3401193776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedfc1003b0 0x7fedfc100780 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:29.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.610+0000 7fee02a00700 1 -- 172.21.15.119:0/3401193776 >> 172.21.15.119:0/3401193776 conn(0x7fedfc0fbc30 msgr2=0x7fedfc0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:29.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.610+0000 7fee02a00700 1 -- 172.21.15.119:0/3401193776 shutdown_connections 2024-05-27T09:37:29.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.610+0000 7fee02a00700 1 -- 172.21.15.119:0/3401193776 wait complete. 2024-05-27T09:37:29.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.611+0000 7fee02a00700 1 Processor -- start 2024-05-27T09:37:29.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.611+0000 7fee02a00700 1 -- start start 2024-05-27T09:37:29.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.612+0000 7fee02a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedfc1003b0 0x7fedfc197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:29.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.612+0000 7fee02a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedfc100d50 0x7fedfc198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:29.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.612+0000 7fee02a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc198c10 con 0x7fedfc100d50 2024-05-27T09:37:29.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.612+0000 7fee02a00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc19c9a0 con 0x7fedfc1003b0 2024-05-27T09:37:29.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.612+0000 7fedf3e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedfc100d50 0x7fedfc198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:29.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.612+0000 7fedf3e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedfc100d50 0x7fedfc198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42230/0 (socket says 172.21.15.119:42230) 2024-05-27T09:37:29.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.612+0000 7fedf3e00700 1 -- 172.21.15.119:0/1970056610 learned_addr learned my addr 172.21.15.119:0/1970056610 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:37:29.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.612+0000 7fedfbe00700 1 --2- 172.21.15.119:0/1970056610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedfc1003b0 0x7fedfc197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:29.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.612+0000 7fedf3e00700 1 -- 172.21.15.119:0/1970056610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedfc1003b0 msgr2=0x7fedfc197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:29.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.612+0000 7fedf3e00700 1 --2- 172.21.15.119:0/1970056610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedfc1003b0 0x7fedfc197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:29.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.612+0000 7fedf3e00700 1 -- 172.21.15.119:0/1970056610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedec0095f0 con 0x7fedfc100d50 2024-05-27T09:37:29.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.613+0000 7fedf3e00700 1 --2- 172.21.15.119:0/1970056610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedfc100d50 0x7fedfc198530 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fede400b480 tx=0x7fede400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:29.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.613+0000 7fedf9600700 1 -- 172.21.15.119:0/1970056610 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fede40118c0 con 0x7fedfc100d50 2024-05-27T09:37:29.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.613+0000 7fedf9600700 1 -- 172.21.15.119:0/1970056610 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fede4010380 con 0x7fedfc100d50 2024-05-27T09:37:29.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.613+0000 7fee02a00700 1 -- 172.21.15.119:0/1970056610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedfc19cc80 con 0x7fedfc100d50 2024-05-27T09:37:29.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.613+0000 7fedf9600700 1 -- 172.21.15.119:0/1970056610 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fede400f4f0 con 0x7fedfc100d50 2024-05-27T09:37:29.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.613+0000 7fee02a00700 1 -- 172.21.15.119:0/1970056610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedfc19d1d0 con 0x7fedfc100d50 2024-05-27T09:37:29.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.615+0000 7fee02a00700 1 -- 172.21.15.119:0/1970056610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedfc1044a0 con 0x7fedfc100d50 2024-05-27T09:37:29.616 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.616+0000 7fedf9600700 1 -- 172.21.15.119:0/1970056610 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fede40104f0 con 0x7fedfc100d50 2024-05-27T09:37:29.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.616+0000 7fedf9600700 1 --2- 172.21.15.119:0/1970056610 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fede0078a40 0x7fede007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:29.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.617+0000 7fedf9600700 1 -- 172.21.15.119:0/1970056610 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fede409a8e0 con 0x7fedfc100d50 2024-05-27T09:37:29.617 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.617+0000 7fedfbe00700 1 --2- 172.21.15.119:0/1970056610 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fede0078a40 0x7fede007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:29.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.618+0000 7fedfbe00700 1 --2- 172.21.15.119:0/1970056610 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fede0078a40 0x7fede007aef0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fedec006010 tx=0x7fedec01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:29.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.622+0000 7fedf9600700 1 -- 172.21.15.119:0/1970056610 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fede4062540 con 0x7fedfc100d50 2024-05-27T09:37:29.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:29 smithi138 ceph-mon[70004]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:37:29.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:29 smithi138 ceph-mon[70004]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:37:29.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.920+0000 7fee02a00700 1 -- 172.21.15.119:0/1970056610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fedfc04ea50 con 0x7fedfc100d50 2024-05-27T09:37:29.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.921+0000 7fedf9600700 1 -- 172.21.15.119:0/1970056610 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1889 (secure 0 0 0) 0x7fede4061ee0 con 0x7fedfc100d50 2024-05-27T09:37:29.922 INFO:teuthology.orchestra.run.smithi119.stdout:e22 2024-05-27T09:37:29.922 INFO:teuthology.orchestra.run.smithi119.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T09:37:29.922 INFO:teuthology.orchestra.run.smithi119.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:37:29.922 INFO:teuthology.orchestra.run.smithi119.stdout:legacy client fscid: 1 2024-05-27T09:37:29.922 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:37:29.922 INFO:teuthology.orchestra.run.smithi119.stdout:Filesystem 'cephfs' (1) 2024-05-27T09:37:29.922 INFO:teuthology.orchestra.run.smithi119.stdout:fs_name cephfs 2024-05-27T09:37:29.922 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 22 2024-05-27T09:37:29.922 INFO:teuthology.orchestra.run.smithi119.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-27T09:37:29.923 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-05-27T09:25:53.387388+0000 2024-05-27T09:37:29.923 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2024-05-27T09:34:01.250304+0000 2024-05-27T09:37:29.923 INFO:teuthology.orchestra.run.smithi119.stdout:tableserver 0 2024-05-27T09:37:29.923 INFO:teuthology.orchestra.run.smithi119.stdout:root 0 2024-05-27T09:37:29.923 INFO:teuthology.orchestra.run.smithi119.stdout:session_timeout 60 2024-05-27T09:37:29.923 INFO:teuthology.orchestra.run.smithi119.stdout:session_autoclose 300 2024-05-27T09:37:29.923 INFO:teuthology.orchestra.run.smithi119.stdout:max_file_size 1099511627776 2024-05-27T09:37:29.923 INFO:teuthology.orchestra.run.smithi119.stdout:max_xattr_size 65536 2024-05-27T09:37:29.923 INFO:teuthology.orchestra.run.smithi119.stdout:required_client_features {} 2024-05-27T09:37:29.923 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure 0 2024-05-27T09:37:29.923 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure_osd_epoch 46 2024-05-27T09:37:29.923 INFO:teuthology.orchestra.run.smithi119.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:37:29.923 INFO:teuthology.orchestra.run.smithi119.stdout:max_mds 2 2024-05-27T09:37:29.923 INFO:teuthology.orchestra.run.smithi119.stdout:in 0,1 2024-05-27T09:37:29.924 INFO:teuthology.orchestra.run.smithi119.stdout:up {0=14472,1=24241} 2024-05-27T09:37:29.924 INFO:teuthology.orchestra.run.smithi119.stdout:failed 2024-05-27T09:37:29.924 INFO:teuthology.orchestra.run.smithi119.stdout:damaged 2024-05-27T09:37:29.924 INFO:teuthology.orchestra.run.smithi119.stdout:stopped 2024-05-27T09:37:29.924 INFO:teuthology.orchestra.run.smithi119.stdout:data_pools [3] 2024-05-27T09:37:29.924 INFO:teuthology.orchestra.run.smithi119.stdout:metadata_pool 2 2024-05-27T09:37:29.924 INFO:teuthology.orchestra.run.smithi119.stdout:inline_data enabled 2024-05-27T09:37:29.924 INFO:teuthology.orchestra.run.smithi119.stdout:balancer 2024-05-27T09:37:29.924 INFO:teuthology.orchestra.run.smithi119.stdout:bal_rank_mask -1 2024-05-27T09:37:29.924 INFO:teuthology.orchestra.run.smithi119.stdout:standby_count_wanted 1 2024-05-27T09:37:29.924 INFO:teuthology.orchestra.run.smithi119.stdout:qdb_cluster leader: 0 members: 2024-05-27T09:37:29.924 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.ppmzzk{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:37:29.924 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.ndfkhd{1:24241} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/897058594,v1:172.21.15.138:6825/897058594] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:37:29.925 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:37:29.925 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:37:29.925 INFO:teuthology.orchestra.run.smithi119.stdout:Standby daemons: 2024-05-27T09:37:29.925 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:37:29.925 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.nghcey{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.119:6826/413736191,v1:172.21.15.119:6827/413736191] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:37:29.925 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.alszpk{-1:24253} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/2345607053,v1:172.21.15.138:6827/2345607053] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:37:29.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.925+0000 7fee02a00700 1 -- 172.21.15.119:0/1970056610 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fede0078a40 msgr2=0x7fede007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:29.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.925+0000 7fee02a00700 1 --2- 172.21.15.119:0/1970056610 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fede0078a40 0x7fede007aef0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fedec006010 tx=0x7fedec01a040 comp rx=0 tx=0).stop 2024-05-27T09:37:29.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.925+0000 7fee02a00700 1 -- 172.21.15.119:0/1970056610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedfc100d50 msgr2=0x7fedfc198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:29.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.925+0000 7fee02a00700 1 --2- 172.21.15.119:0/1970056610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedfc100d50 0x7fedfc198530 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fede400b480 tx=0x7fede400b840 comp rx=0 tx=0).stop 2024-05-27T09:37:29.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.925+0000 7fee02a00700 1 -- 172.21.15.119:0/1970056610 shutdown_connections 2024-05-27T09:37:29.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.925+0000 7fee02a00700 1 --2- 172.21.15.119:0/1970056610 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fede0078a40 0x7fede007aef0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:29.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.925+0000 7fee02a00700 1 --2- 172.21.15.119:0/1970056610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedfc1003b0 0x7fedfc197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:29.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.925+0000 7fee02a00700 1 --2- 172.21.15.119:0/1970056610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedfc100d50 0x7fedfc198530 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:29.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.925+0000 7fee02a00700 1 -- 172.21.15.119:0/1970056610 >> 172.21.15.119:0/1970056610 conn(0x7fedfc0fbc30 msgr2=0x7fedfc0fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:29.926 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.925+0000 7fee02a00700 1 -- 172.21.15.119:0/1970056610 shutdown_connections 2024-05-27T09:37:29.927 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:29.925+0000 7fee02a00700 1 -- 172.21.15.119:0/1970056610 wait complete. 2024-05-27T09:37:29.927 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 22 2024-05-27T09:37:30.097 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.096+0000 7faacc000700 1 -- 172.21.15.119:0/3005407166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faac4101210 msgr2=0x7faac41015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:30.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.096+0000 7faacc000700 1 --2- 172.21.15.119:0/3005407166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faac4101210 0x7faac41015e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faab8009910 tx=0x7faab8009c20 comp rx=0 tx=0).stop 2024-05-27T09:37:30.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.096+0000 7faacc000700 1 -- 172.21.15.119:0/3005407166 shutdown_connections 2024-05-27T09:37:30.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.096+0000 7faacc000700 1 --2- 172.21.15.119:0/3005407166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faac4101b20 0x7faac410a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:30.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.096+0000 7faacc000700 1 --2- 172.21.15.119:0/3005407166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faac4101210 0x7faac41015e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:30.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.096+0000 7faacc000700 1 -- 172.21.15.119:0/3005407166 >> 172.21.15.119:0/3005407166 conn(0x7faac40fab80 msgr2=0x7faac40fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:30.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.097+0000 7faacc000700 1 -- 172.21.15.119:0/3005407166 shutdown_connections 2024-05-27T09:37:30.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.097+0000 7faacc000700 1 -- 172.21.15.119:0/3005407166 wait complete. 2024-05-27T09:37:30.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.098+0000 7faacc000700 1 Processor -- start 2024-05-27T09:37:30.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.098+0000 7faacc000700 1 -- start start 2024-05-27T09:37:30.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.098+0000 7faacc000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faac4101210 0x7faac4197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:30.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.098+0000 7faacc000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faac4101b20 0x7faac4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:30.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.098+0000 7faacc000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac4198b50 con 0x7faac4101b20 2024-05-27T09:37:30.099 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.098+0000 7faacc000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac419c8e0 con 0x7faac4101210 2024-05-27T09:37:30.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.099+0000 7faac8c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faac4101b20 0x7faac4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:30.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.099+0000 7faac8c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faac4101b20 0x7faac4198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42238/0 (socket says 172.21.15.119:42238) 2024-05-27T09:37:30.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.099+0000 7faac8c00700 1 -- 172.21.15.119:0/1414358239 learned_addr learned my addr 172.21.15.119:0/1414358239 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:37:30.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.099+0000 7faac9600700 1 --2- 172.21.15.119:0/1414358239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faac4101210 0x7faac4197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:30.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.100+0000 7faac8c00700 1 -- 172.21.15.119:0/1414358239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faac4101210 msgr2=0x7faac4197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:30.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.100+0000 7faac8c00700 1 --2- 172.21.15.119:0/1414358239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faac4101210 0x7faac4197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:30.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.100+0000 7faac8c00700 1 -- 172.21.15.119:0/1414358239 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab80095f0 con 0x7faac4101b20 2024-05-27T09:37:30.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.100+0000 7faac8c00700 1 --2- 172.21.15.119:0/1414358239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faac4101b20 0x7faac4198470 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7faab000d8d0 tx=0x7faab000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:30.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.100+0000 7faac2000700 1 -- 172.21.15.119:0/1414358239 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faab00098a0 con 0x7faac4101b20 2024-05-27T09:37:30.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.100+0000 7faacc000700 1 -- 172.21.15.119:0/1414358239 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac419cbc0 con 0x7faac4101b20 2024-05-27T09:37:30.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.101+0000 7faac2000700 1 -- 172.21.15.119:0/1414358239 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faab0010460 con 0x7faac4101b20 2024-05-27T09:37:30.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.101+0000 7faac2000700 1 -- 172.21.15.119:0/1414358239 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faab000f5f0 con 0x7faac4101b20 2024-05-27T09:37:30.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.101+0000 7faacc000700 1 -- 172.21.15.119:0/1414358239 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac419d110 con 0x7faac4101b20 2024-05-27T09:37:30.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.103+0000 7faacc000700 1 -- 172.21.15.119:0/1414358239 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac4109d50 con 0x7faac4101b20 2024-05-27T09:37:30.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.104+0000 7faac2000700 1 -- 172.21.15.119:0/1414358239 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7faab0009a00 con 0x7faac4101b20 2024-05-27T09:37:30.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.106+0000 7faac2000700 1 --2- 172.21.15.119:0/1414358239 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7faaac078a40 0x7faaac07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:30.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.106+0000 7faac2000700 1 -- 172.21.15.119:0/1414358239 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6414+0+0 (secure 0 0 0) 0x7faab009ac40 con 0x7faac4101b20 2024-05-27T09:37:30.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.106+0000 7faac9600700 1 --2- 172.21.15.119:0/1414358239 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7faaac078a40 0x7faaac07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:30.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.107+0000 7faac9600700 1 --2- 172.21.15.119:0/1414358239 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7faaac078a40 0x7faaac07aef0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faab8006010 tx=0x7faab800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:30.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.112+0000 7faac2000700 1 -- 172.21.15.119:0/1414358239 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7faab0062820 con 0x7faac4101b20 2024-05-27T09:37:30.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.387+0000 7faacc000700 1 -- 172.21.15.119:0/1414358239 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faac419d420 con 0x7faaac078a40 2024-05-27T09:37:30.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.388+0000 7faac2000700 1 -- 172.21.15.119:0/1414358239 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7faac419d420 con 0x7faaac078a40 2024-05-27T09:37:30.389 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:37:30.389 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:37:30.389 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:37:30.389 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-27T09:37:30.389 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [ 2024-05-27T09:37:30.389 INFO:teuthology.orchestra.run.smithi119.stdout: "crash", 2024-05-27T09:37:30.390 INFO:teuthology.orchestra.run.smithi119.stdout: "mon", 2024-05-27T09:37:30.390 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr" 2024-05-27T09:37:30.390 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-05-27T09:37:30.390 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "8/23 daemons upgraded", 2024-05-27T09:37:30.390 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "Currently upgrading osd daemons", 2024-05-27T09:37:30.390 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:37:30.390 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:37:30.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.392+0000 7faacc000700 1 -- 172.21.15.119:0/1414358239 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7faaac078a40 msgr2=0x7faaac07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:30.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.392+0000 7faacc000700 1 --2- 172.21.15.119:0/1414358239 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7faaac078a40 0x7faaac07aef0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faab8006010 tx=0x7faab800b540 comp rx=0 tx=0).stop 2024-05-27T09:37:30.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.392+0000 7faacc000700 1 -- 172.21.15.119:0/1414358239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faac4101b20 msgr2=0x7faac4198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:30.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.392+0000 7faacc000700 1 --2- 172.21.15.119:0/1414358239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faac4101b20 0x7faac4198470 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7faab000d8d0 tx=0x7faab000dbe0 comp rx=0 tx=0).stop 2024-05-27T09:37:30.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.392+0000 7faacc000700 1 -- 172.21.15.119:0/1414358239 shutdown_connections 2024-05-27T09:37:30.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.392+0000 7faacc000700 1 --2- 172.21.15.119:0/1414358239 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7faaac078a40 0x7faaac07aef0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:30.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.392+0000 7faacc000700 1 --2- 172.21.15.119:0/1414358239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faac4101210 0x7faac4197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:30.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.392+0000 7faacc000700 1 --2- 172.21.15.119:0/1414358239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faac4101b20 0x7faac4198470 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:30.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.392+0000 7faacc000700 1 -- 172.21.15.119:0/1414358239 >> 172.21.15.119:0/1414358239 conn(0x7faac40fab80 msgr2=0x7faac40ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:30.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.392+0000 7faacc000700 1 -- 172.21.15.119:0/1414358239 shutdown_connections 2024-05-27T09:37:30.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.393+0000 7faacc000700 1 -- 172.21.15.119:0/1414358239 wait complete. 2024-05-27T09:37:30.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.551+0000 7f5d31e00700 1 -- 172.21.15.119:0/2120826079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d2c101060 msgr2=0x7f5d2c101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:30.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.551+0000 7f5d31e00700 1 --2- 172.21.15.119:0/2120826079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d2c101060 0x7f5d2c101430 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009910 tx=0x7f5d20009c20 comp rx=0 tx=0).stop 2024-05-27T09:37:30.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.551+0000 7f5d31e00700 1 -- 172.21.15.119:0/2120826079 shutdown_connections 2024-05-27T09:37:30.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.551+0000 7f5d31e00700 1 --2- 172.21.15.119:0/2120826079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d2c101970 0x7f5d2c10a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:30.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.551+0000 7f5d31e00700 1 --2- 172.21.15.119:0/2120826079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d2c101060 0x7f5d2c101430 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:30.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.551+0000 7f5d31e00700 1 -- 172.21.15.119:0/2120826079 >> 172.21.15.119:0/2120826079 conn(0x7f5d2c0fa9b0 msgr2=0x7f5d2c0fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:30.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.551+0000 7f5d31e00700 1 -- 172.21.15.119:0/2120826079 shutdown_connections 2024-05-27T09:37:30.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.551+0000 7f5d31e00700 1 -- 172.21.15.119:0/2120826079 wait complete. 2024-05-27T09:37:30.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.552+0000 7f5d31e00700 1 Processor -- start 2024-05-27T09:37:30.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.552+0000 7f5d31e00700 1 -- start start 2024-05-27T09:37:30.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.553+0000 7f5d31e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d2c101060 0x7f5d2c0fee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:30.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.553+0000 7f5d31e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d2c101970 0x7f5d2c0ff3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:30.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.553+0000 7f5d31e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0ff910 con 0x7f5d2c101970 2024-05-27T09:37:30.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.553+0000 7f5d31e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0ffa50 con 0x7f5d2c101060 2024-05-27T09:37:30.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.553+0000 7f5d2be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d2c101970 0x7f5d2c0ff3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:30.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.553+0000 7f5d2be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d2c101970 0x7f5d2c0ff3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42252/0 (socket says 172.21.15.119:42252) 2024-05-27T09:37:30.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.553+0000 7f5d2be00700 1 -- 172.21.15.119:0/2499562482 learned_addr learned my addr 172.21.15.119:0/2499562482 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:37:30.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.553+0000 7f5d30a00700 1 --2- 172.21.15.119:0/2499562482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d2c101060 0x7f5d2c0fee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:30.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.554+0000 7f5d2be00700 1 -- 172.21.15.119:0/2499562482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d2c101060 msgr2=0x7f5d2c0fee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:30.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.554+0000 7f5d2be00700 1 --2- 172.21.15.119:0/2499562482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d2c101060 0x7f5d2c0fee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:30.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.554+0000 7f5d2be00700 1 -- 172.21.15.119:0/2499562482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d200095f0 con 0x7f5d2c101970 2024-05-27T09:37:30.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.554+0000 7f5d2be00700 1 --2- 172.21.15.119:0/2499562482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d2c101970 0x7f5d2c0ff3d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5d1800d8d0 tx=0x7f5d1800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:30.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.554+0000 7f5d29600700 1 -- 172.21.15.119:0/2499562482 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d18009940 con 0x7f5d2c101970 2024-05-27T09:37:30.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.554+0000 7f5d29600700 1 -- 172.21.15.119:0/2499562482 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d18010460 con 0x7f5d2c101970 2024-05-27T09:37:30.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.554+0000 7f5d29600700 1 -- 172.21.15.119:0/2499562482 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d1800f5f0 con 0x7f5d2c101970 2024-05-27T09:37:30.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.554+0000 7f5d31e00700 1 -- 172.21.15.119:0/2499562482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2c1a1d40 con 0x7f5d2c101970 2024-05-27T09:37:30.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.554+0000 7f5d31e00700 1 -- 172.21.15.119:0/2499562482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2c1a2150 con 0x7f5d2c101970 2024-05-27T09:37:30.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.555+0000 7f5d31e00700 1 -- 172.21.15.119:0/2499562482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d2c107740 con 0x7f5d2c101970 2024-05-27T09:37:30.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.557+0000 7f5d29600700 1 -- 172.21.15.119:0/2499562482 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f5d180105d0 con 0x7f5d2c101970 2024-05-27T09:37:30.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.558+0000 7f5d29600700 1 --2- 172.21.15.119:0/2499562482 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5d14078a40 0x7f5d1407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:37:30.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.558+0000 7f5d29600700 1 -- 172.21.15.119:0/2499562482 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f5d1809abf0 con 0x7f5d2c101970 2024-05-27T09:37:30.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.558+0000 7f5d30a00700 1 --2- 172.21.15.119:0/2499562482 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5d14078a40 0x7f5d1407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:37:30.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.559+0000 7f5d30a00700 1 --2- 172.21.15.119:0/2499562482 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5d14078a40 0x7f5d1407aef0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000b5c0 tx=0x7f5d20005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:37:30.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.563+0000 7f5d29600700 1 -- 172.21.15.119:0/2499562482 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f5d180628f0 con 0x7f5d2c101970 2024-05-27T09:37:30.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:30 smithi138 ceph-mon[70004]: pgmap v104: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:30.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:30 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/1851184978' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:37:30.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:30 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/1970056610' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:37:30.684 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:30 smithi119 ceph-mon[88269]: pgmap v104: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:30.684 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:30 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/1851184978' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:37:30.684 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:30 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/1970056610' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:37:30.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.886+0000 7f5d31e00700 1 -- 172.21.15.119:0/2499562482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5d2c04f2a0 con 0x7f5d2c101970 2024-05-27T09:37:30.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.886+0000 7f5d29600700 1 -- 172.21.15.119:0/2499562482 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f5d180160e0 con 0x7f5d2c101970 2024-05-27T09:37:30.887 INFO:teuthology.orchestra.run.smithi119.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-27T09:37:30.887 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:37:30.887 INFO:teuthology.orchestra.run.smithi119.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:37:30.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.889+0000 7f5d31e00700 1 -- 172.21.15.119:0/2499562482 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5d14078a40 msgr2=0x7f5d1407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:30.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.889+0000 7f5d31e00700 1 --2- 172.21.15.119:0/2499562482 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5d14078a40 0x7f5d1407aef0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000b5c0 tx=0x7f5d20005a90 comp rx=0 tx=0).stop 2024-05-27T09:37:30.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.889+0000 7f5d31e00700 1 -- 172.21.15.119:0/2499562482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d2c101970 msgr2=0x7f5d2c0ff3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:37:30.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.889+0000 7f5d31e00700 1 --2- 172.21.15.119:0/2499562482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d2c101970 0x7f5d2c0ff3d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5d1800d8d0 tx=0x7f5d1800dc90 comp rx=0 tx=0).stop 2024-05-27T09:37:30.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.889+0000 7f5d31e00700 1 -- 172.21.15.119:0/2499562482 shutdown_connections 2024-05-27T09:37:30.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.889+0000 7f5d31e00700 1 --2- 172.21.15.119:0/2499562482 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5d14078a40 0x7f5d1407aef0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:30.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.889+0000 7f5d31e00700 1 --2- 172.21.15.119:0/2499562482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d2c101060 0x7f5d2c0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:30.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.889+0000 7f5d31e00700 1 --2- 172.21.15.119:0/2499562482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d2c101970 0x7f5d2c0ff3d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:37:30.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.889+0000 7f5d31e00700 1 -- 172.21.15.119:0/2499562482 >> 172.21.15.119:0/2499562482 conn(0x7f5d2c0fa9b0 msgr2=0x7f5d2c104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:37:30.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.890+0000 7f5d31e00700 1 -- 172.21.15.119:0/2499562482 shutdown_connections 2024-05-27T09:37:30.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:37:30.890+0000 7f5d31e00700 1 -- 172.21.15.119:0/2499562482 wait complete. 2024-05-27T09:37:31.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:31 smithi138 ceph-mon[70004]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:37:31.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:31 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/2499562482' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:37:31.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:31 smithi119 ceph-mon[88269]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:37:31.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:31 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/2499562482' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:37:32.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:32 smithi119 ceph-mon[88269]: pgmap v105: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:32.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:32 smithi138 ceph-mon[70004]: pgmap v105: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:32.985 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:32 smithi119 systemd[1]: Stopping Ceph osd.2 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:37:33.249 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:32 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2[54880]: 2024-05-27T09:37:32.982+0000 7fd30d000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T09:37:33.249 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:32 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2[54880]: 2024-05-27T09:37:32.982+0000 7fd30d000700 -1 osd.2 59 *** Got signal Terminated *** 2024-05-27T09:37:33.249 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:32 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2[54880]: 2024-05-27T09:37:32.982+0000 7fd30d000700 -1 osd.2 59 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T09:37:33.712 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:33 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:33.712 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:33 smithi119 ceph-mon[88269]: osd.2 marked itself down and dead 2024-05-27T09:37:33.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:33 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:33.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:33 smithi138 ceph-mon[70004]: osd.2 marked itself down and dead 2024-05-27T09:37:33.997 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:33 smithi119 podman[110739]: 2024-05-27 09:37:33.94224194 +0000 UTC m=+1.074092420 container died 3955a6112c1cc999c759abb896adfecabf4422fd9759c16b6ccd167a9cf4654d (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.0) 2024-05-27T09:37:34.653 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:34 smithi119 ceph-mon[88269]: pgmap v106: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:34.653 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:34 smithi119 ceph-mon[88269]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-27T09:37:34.653 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:34 smithi119 ceph-mon[88269]: osdmap e60: 6 total, 5 up, 6 in 2024-05-27T09:37:34.653 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:34 smithi119 podman[110739]: 2024-05-27 09:37:34.366846914 +0000 UTC m=+1.498697461 container cleanup 3955a6112c1cc999c759abb896adfecabf4422fd9759c16b6ccd167a9cf4654d (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2, CEPH_POINT_RELEASE=-18.2.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, RELEASE=HEAD, GIT_BRANCH=HEAD, org.label-schema.build-date=20231212) 2024-05-27T09:37:34.653 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:34 smithi119 bash[110739]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2 2024-05-27T09:37:34.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:34 smithi138 ceph-mon[70004]: pgmap v106: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:34.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:34 smithi138 ceph-mon[70004]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-27T09:37:34.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:34 smithi138 ceph-mon[70004]: osdmap e60: 6 total, 5 up, 6 in 2024-05-27T09:37:34.998 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:34 smithi119 podman[110758]: 2024-05-27 09:37:34.651497281 +0000 UTC m=+0.693250034 container remove 3955a6112c1cc999c759abb896adfecabf4422fd9759c16b6ccd167a9cf4654d (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux ) 2024-05-27T09:37:35.481 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:35 smithi119 ceph-mon[88269]: osdmap e61: 6 total, 5 up, 6 in 2024-05-27T09:37:35.740 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:35 smithi119 podman[110845]: 2024-05-27 09:37:35.385444246 +0000 UTC m=+0.033712706 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:37:35.741 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:35 smithi119 podman[110845]: 2024-05-27 09:37:35.494170979 +0000 UTC m=+0.142439479 container create 54495a45659f74446216a8f434e496aa01fbcb6e86905da23aaa4e41c2d39874 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T09:37:35.741 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:35 smithi119 podman[110845]: 2024-05-27 09:37:35.739626302 +0000 UTC m=+0.387894806 container init 54495a45659f74446216a8f434e496aa01fbcb6e86905da23aaa4e41c2d39874 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5) 2024-05-27T09:37:35.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:35 smithi138 ceph-mon[70004]: osdmap e61: 6 total, 5 up, 6 in 2024-05-27T09:37:35.997 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:35 smithi119 podman[110845]: 2024-05-27 09:37:35.751028696 +0000 UTC m=+0.399297168 container start 54495a45659f74446216a8f434e496aa01fbcb6e86905da23aaa4e41c2d39874 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T09:37:35.998 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:35 smithi119 podman[110845]: 2024-05-27 09:37:35.880164975 +0000 UTC m=+0.528433495 container attach 54495a45659f74446216a8f434e496aa01fbcb6e86905da23aaa4e41c2d39874 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-05-27T09:37:36.470 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:36 smithi119 podman[110845]: 2024-05-27 09:37:36.194826601 +0000 UTC m=+0.843095138 container died 54495a45659f74446216a8f434e496aa01fbcb6e86905da23aaa4e41c2d39874 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-27T09:37:36.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:36 smithi119 ceph-mon[88269]: pgmap v109: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:36.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:36 smithi119 ceph-mon[88269]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-05-27T09:37:36.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:36 smithi138 ceph-mon[70004]: pgmap v109: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:36.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:36 smithi138 ceph-mon[70004]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-05-27T09:37:37.248 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:36 smithi119 podman[110871]: 2024-05-27 09:37:36.92573624 +0000 UTC m=+0.713015080 container remove 54495a45659f74446216a8f434e496aa01fbcb6e86905da23aaa4e41c2d39874 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-27T09:37:37.248 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:36 smithi119 systemd[1]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.2.service: Deactivated successfully. 2024-05-27T09:37:37.249 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:36 smithi119 systemd[1]: Stopped Ceph osd.2 for b966b06e-1c09-11ef-bc9b-c7b262605968. 2024-05-27T09:37:37.249 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:36 smithi119 systemd[1]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.2.service: Consumed 1min 26.585s CPU time. 2024-05-27T09:37:37.695 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:37 smithi119 ceph-mon[88269]: Health check failed: Degraded data redundancy: 15/381 objects degraded (3.937%), 4 pgs degraded (PG_DEGRADED) 2024-05-27T09:37:37.695 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:37 smithi119 systemd[1]: Starting Ceph osd.2 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:37:37.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:37 smithi138 ceph-mon[70004]: Health check failed: Degraded data redundancy: 15/381 objects degraded (3.937%), 4 pgs degraded (PG_DEGRADED) 2024-05-27T09:37:37.992 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:37 smithi119 podman[110969]: 2024-05-27 09:37:37.599550396 +0000 UTC m=+0.031396660 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:37:37.992 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:37 smithi119 podman[110969]: 2024-05-27 09:37:37.701418725 +0000 UTC m=+0.133265026 container create 99d89d4e0b1baf031964cf8d228545ef5fa01991900299bf470fc660973191a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-activate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T09:37:38.251 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:37 smithi119 podman[110969]: 2024-05-27 09:37:37.990082379 +0000 UTC m=+0.421928680 container init 99d89d4e0b1baf031964cf8d228545ef5fa01991900299bf470fc660973191a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-activate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD) 2024-05-27T09:37:38.251 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:38 smithi119 podman[110969]: 2024-05-27 09:37:38.002391455 +0000 UTC m=+0.434237726 container start 99d89d4e0b1baf031964cf8d228545ef5fa01991900299bf470fc660973191a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-activate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD) 2024-05-27T09:37:38.252 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:38 smithi119 podman[110969]: 2024-05-27 09:37:38.095731978 +0000 UTC m=+0.527578312 container attach 99d89d4e0b1baf031964cf8d228545ef5fa01991900299bf470fc660973191a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-activate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-27T09:37:38.866 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:38 smithi119 ceph-mon[88269]: pgmap v110: 65 pgs: 4 active+undersized+degraded, 7 peering, 2 stale+active+clean, 4 active+undersized, 48 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 15/381 objects degraded (3.937%) 2024-05-27T09:37:38.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:38 smithi138 ceph-mon[70004]: pgmap v110: 65 pgs: 4 active+undersized+degraded, 7 peering, 2 stale+active+clean, 4 active+undersized, 48 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 15/381 objects degraded (3.937%) 2024-05-27T09:37:39.840 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:39 smithi119 ceph-mon[88269]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-05-27T09:37:39.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:39 smithi138 ceph-mon[70004]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-05-27T09:37:40.248 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:39 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-activate[110983]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:37:40.248 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:39 smithi119 bash[110969]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:37:40.248 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:39 smithi119 bash[110969]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:37:40.248 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:39 smithi119 bash[110969]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-27T09:37:40.248 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:39 smithi119 bash[110969]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-27T09:37:40.248 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:39 smithi119 bash[110969]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-27T09:37:40.248 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:39 smithi119 bash[110969]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-27T09:37:40.248 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:39 smithi119 bash[110969]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-05-27T09:37:40.249 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:39 smithi119 bash[110969]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-27T09:37:40.249 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:39 smithi119 bash[110969]: --> ceph-volume raw activate successful for osd ID: 2 2024-05-27T09:37:40.249 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:39 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-activate[110983]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:37:40.249 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:39 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-activate[110983]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-27T09:37:40.249 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:39 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-activate[110983]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-27T09:37:40.249 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:39 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-activate[110983]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-27T09:37:40.249 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:39 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-activate[110983]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-27T09:37:40.249 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:39 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-activate[110983]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-05-27T09:37:40.249 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:39 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-activate[110983]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-27T09:37:40.249 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:39 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-activate[110983]: --> ceph-volume raw activate successful for osd ID: 2 2024-05-27T09:37:40.249 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:39 smithi119 podman[110969]: 2024-05-27 09:37:39.903638252 +0000 UTC m=+2.335484616 container died 99d89d4e0b1baf031964cf8d228545ef5fa01991900299bf470fc660973191a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-activate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-27T09:37:40.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:40 smithi138 ceph-mon[70004]: pgmap v111: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/381 objects degraded (12.073%) 2024-05-27T09:37:40.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:40 smithi119 ceph-mon[88269]: pgmap v111: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/381 objects degraded (12.073%) 2024-05-27T09:37:40.998 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:40 smithi119 podman[111124]: 2024-05-27 09:37:40.665947323 +0000 UTC m=+0.742474579 container remove 99d89d4e0b1baf031964cf8d228545ef5fa01991900299bf470fc660973191a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-27T09:37:41.307 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:41 smithi119 podman[111183]: 2024-05-27 09:37:41.025388151 +0000 UTC m=+0.128121901 container create f0bd34ca416f96bbd725da3952c260f184d177f64497284dd5559c4c869af748 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-27T09:37:41.307 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:41 smithi119 podman[111183]: 2024-05-27 09:37:40.931840709 +0000 UTC m=+0.034574420 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:37:41.307 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:41 smithi119 podman[111183]: 2024-05-27 09:37:41.306303116 +0000 UTC m=+0.409036869 container init f0bd34ca416f96bbd725da3952c260f184d177f64497284dd5559c4c869af748 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-27T09:37:41.747 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:41 smithi119 podman[111183]: 2024-05-27 09:37:41.317157978 +0000 UTC m=+0.419891695 container start f0bd34ca416f96bbd725da3952c260f184d177f64497284dd5559c4c869af748 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-27T09:37:41.748 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:41 smithi119 bash[111183]: f0bd34ca416f96bbd725da3952c260f184d177f64497284dd5559c4c869af748 2024-05-27T09:37:41.748 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:41 smithi119 systemd[1]: Started Ceph osd.2 for b966b06e-1c09-11ef-bc9b-c7b262605968. 2024-05-27T09:37:41.748 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:41 smithi119 ceph-osd[111202]: -- 172.21.15.119:0/2141209066 <== mon.0 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ea38525680 con 0x55ea385ccd00 2024-05-27T09:37:42.247 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:41 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2[111198]: 2024-05-27T09:37:41.960+0000 7f98a38db7c0 -1 Falling back to public interface 2024-05-27T09:37:42.779 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:42 smithi119 ceph-mon[88269]: pgmap v112: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/381 objects degraded (12.073%) 2024-05-27T09:37:42.779 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:42 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:42.780 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:42 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:42.780 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:42 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:37:42.780 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:42 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:42.780 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:42 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:37:42.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:42 smithi138 ceph-mon[70004]: pgmap v112: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/381 objects degraded (12.073%) 2024-05-27T09:37:42.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:42 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:42.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:42 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:42.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:42 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:37:42.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:42 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:42.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:42 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:37:44.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:44 smithi138 ceph-mon[70004]: pgmap v113: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/381 objects degraded (12.073%) 2024-05-27T09:37:44.934 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:44 smithi119 ceph-mon[88269]: pgmap v113: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/381 objects degraded (12.073%) 2024-05-27T09:37:46.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:46 smithi138 ceph-mon[70004]: pgmap v114: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/381 objects degraded (12.073%) 2024-05-27T09:37:46.900 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:46 smithi119 ceph-mon[88269]: pgmap v114: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/381 objects degraded (12.073%) 2024-05-27T09:37:47.247 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:46 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2[111198]: 2024-05-27T09:37:46.922+0000 7f98a38db7c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-05-27T09:37:47.874 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:47 smithi119 ceph-mon[88269]: Health check update: Degraded data redundancy: 46/381 objects degraded (12.073%), 13 pgs degraded (PG_DEGRADED) 2024-05-27T09:37:47.875 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:47 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2[111198]: 2024-05-27T09:37:47.525+0000 7f98a38db7c0 -1 osd.2 59 log_to_monitors true 2024-05-27T09:37:47.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:47 smithi138 ceph-mon[70004]: Health check update: Degraded data redundancy: 46/381 objects degraded (12.073%), 13 pgs degraded (PG_DEGRADED) 2024-05-27T09:37:48.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:48 smithi138 ceph-mon[70004]: pgmap v115: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/381 objects degraded (12.073%) 2024-05-27T09:37:48.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:48 smithi138 ceph-mon[70004]: from='osd.2 [v2:172.21.15.119:6818/1386851269,v1:172.21.15.119:6819/1386851269]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T09:37:48.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:48 smithi138 ceph-mon[70004]: from='osd.2 [v2:172.21.15.119:6818/1386851269,v1:172.21.15.119:6819/1386851269]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-27T09:37:48.997 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:37:48 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2[111198]: 2024-05-27T09:37:48.558+0000 7f9892c00700 -1 osd.2 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T09:37:48.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:48 smithi119 ceph-mon[88269]: pgmap v115: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/381 objects degraded (12.073%) 2024-05-27T09:37:48.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:48 smithi119 ceph-mon[88269]: from='osd.2 [v2:172.21.15.119:6818/1386851269,v1:172.21.15.119:6819/1386851269]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T09:37:48.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:48 smithi119 ceph-mon[88269]: from='osd.2 [v2:172.21.15.119:6818/1386851269,v1:172.21.15.119:6819/1386851269]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-27T09:37:49.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:49 smithi138 ceph-mon[70004]: osdmap e62: 6 total, 5 up, 6 in 2024-05-27T09:37:49.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:49 smithi138 ceph-mon[70004]: from='osd.2 [v2:172.21.15.119:6818/1386851269,v1:172.21.15.119:6819/1386851269]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-05-27T09:37:49.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:49 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:49.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:49 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:49.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:49 smithi119 ceph-mon[88269]: osdmap e62: 6 total, 5 up, 6 in 2024-05-27T09:37:49.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:49 smithi119 ceph-mon[88269]: from='osd.2 [v2:172.21.15.119:6818/1386851269,v1:172.21.15.119:6819/1386851269]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-05-27T09:37:49.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:49 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:49.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:49 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:50.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:50 smithi138 ceph-mon[70004]: pgmap v117: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/381 objects degraded (12.073%) 2024-05-27T09:37:50.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:50 smithi138 ceph-mon[70004]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-27T09:37:50.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:50 smithi138 ceph-mon[70004]: osd.2 [v2:172.21.15.119:6818/1386851269,v1:172.21.15.119:6819/1386851269] boot 2024-05-27T09:37:50.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:50 smithi138 ceph-mon[70004]: osdmap e63: 6 total, 6 up, 6 in 2024-05-27T09:37:50.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:50 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T09:37:50.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:50 smithi138 ceph-mon[70004]: osdmap e64: 6 total, 6 up, 6 in 2024-05-27T09:37:50.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:50 smithi119 ceph-mon[88269]: pgmap v117: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/381 objects degraded (12.073%) 2024-05-27T09:37:50.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:50 smithi119 ceph-mon[88269]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-27T09:37:50.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:50 smithi119 ceph-mon[88269]: osd.2 [v2:172.21.15.119:6818/1386851269,v1:172.21.15.119:6819/1386851269] boot 2024-05-27T09:37:50.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:50 smithi119 ceph-mon[88269]: osdmap e63: 6 total, 6 up, 6 in 2024-05-27T09:37:50.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:50 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T09:37:50.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:50 smithi119 ceph-mon[88269]: osdmap e64: 6 total, 6 up, 6 in 2024-05-27T09:37:52.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:51 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:52.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:51 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:52.145 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:51 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:52.145 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:51 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:53.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:52 smithi119 ceph-mon[88269]: pgmap v120: 65 pgs: 11 active+undersized+degraded, 11 active+undersized, 43 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 36/381 objects degraded (9.449%) 2024-05-27T09:37:53.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:52 smithi119 ceph-mon[88269]: Health check update: Degraded data redundancy: 36/381 objects degraded (9.449%), 11 pgs degraded (PG_DEGRADED) 2024-05-27T09:37:53.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:52 smithi138 ceph-mon[70004]: pgmap v120: 65 pgs: 11 active+undersized+degraded, 11 active+undersized, 43 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 36/381 objects degraded (9.449%) 2024-05-27T09:37:53.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:52 smithi138 ceph-mon[70004]: Health check update: Degraded data redundancy: 36/381 objects degraded (9.449%), 11 pgs degraded (PG_DEGRADED) 2024-05-27T09:37:55.219 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:54 smithi119 ceph-mon[88269]: pgmap v121: 65 pgs: 6 active+undersized+degraded, 6 active+undersized, 53 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 19/381 objects degraded (4.987%) 2024-05-27T09:37:55.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:54 smithi138 ceph-mon[70004]: pgmap v121: 65 pgs: 6 active+undersized+degraded, 6 active+undersized, 53 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 19/381 objects degraded (4.987%) 2024-05-27T09:37:56.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:55 smithi119 ceph-mon[88269]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 19/381 objects degraded (4.987%), 6 pgs degraded) 2024-05-27T09:37:56.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:55 smithi138 ceph-mon[70004]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 19/381 objects degraded (4.987%), 6 pgs degraded) 2024-05-27T09:37:57.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:56 smithi119 ceph-mon[88269]: pgmap v122: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:57.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:56 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:57.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:56 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:57.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:56 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:37:57.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:56 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:37:57.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:56 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:57.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:56 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:37:57.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:56 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:37:57.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:56 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:37:57.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:56 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:37:57.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:56 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-27T09:37:57.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:56 smithi138 ceph-mon[70004]: pgmap v122: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:57.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:56 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:57.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:56 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:57.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:56 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:37:57.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:56 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:37:57.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:56 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:57.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:56 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:37:57.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:56 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:37:57.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:56 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:37:57.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:56 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:37:57.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:56 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-27T09:37:58.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:57 smithi119 ceph-mon[88269]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-27T09:37:58.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:57 smithi119 ceph-mon[88269]: Upgrade: osd.3 is safe to restart 2024-05-27T09:37:58.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:58.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:37:58.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:58.252 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:57 smithi138 ceph-mon[70004]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-27T09:37:58.252 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:57 smithi138 ceph-mon[70004]: Upgrade: osd.3 is safe to restart 2024-05-27T09:37:58.252 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:58.252 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:37:58.252 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:37:59.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:37:58 smithi119 ceph-mon[88269]: pgmap v123: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:37:59.346 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:37:58 smithi138 ceph-mon[70004]: pgmap v123: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:00.604 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:00 smithi138 ceph-mon[70004]: pgmap v124: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:00.604 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:00 smithi138 ceph-mon[70004]: Upgrade: Updating osd.3 2024-05-27T09:38:00.604 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:00 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:00.604 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:00 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-27T09:38:00.604 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:00 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:38:00.604 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:00 smithi138 ceph-mon[70004]: Deploying daemon osd.3 on smithi138 2024-05-27T09:38:00.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:00 smithi119 ceph-mon[88269]: pgmap v124: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:00.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:00 smithi119 ceph-mon[88269]: Upgrade: Updating osd.3 2024-05-27T09:38:00.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:00 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:00.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:00 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-27T09:38:00.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:00 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:38:00.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:00 smithi119 ceph-mon[88269]: Deploying daemon osd.3 on smithi138 2024-05-27T09:38:01.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.061+0000 7fb9b8c00700 1 -- 172.21.15.119:0/578557769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9b4102610 msgr2=0x7fb9b41029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:01.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.061+0000 7fb9b8c00700 1 --2- 172.21.15.119:0/578557769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9b4102610 0x7fb9b41029e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009910 tx=0x7fb9a4009c20 comp rx=0 tx=0).stop 2024-05-27T09:38:01.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.061+0000 7fb9b8c00700 1 -- 172.21.15.119:0/578557769 shutdown_connections 2024-05-27T09:38:01.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.061+0000 7fb9b8c00700 1 --2- 172.21.15.119:0/578557769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b4102fb0 0x7fb9b4106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:01.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.061+0000 7fb9b8c00700 1 --2- 172.21.15.119:0/578557769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9b4102610 0x7fb9b41029e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:01.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.061+0000 7fb9b8c00700 1 -- 172.21.15.119:0/578557769 >> 172.21.15.119:0/578557769 conn(0x7fb9b40fdef0 msgr2=0x7fb9b4100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:01.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.061+0000 7fb9b8c00700 1 -- 172.21.15.119:0/578557769 shutdown_connections 2024-05-27T09:38:01.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.062+0000 7fb9b8c00700 1 -- 172.21.15.119:0/578557769 wait complete. 2024-05-27T09:38:01.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.063+0000 7fb9b8c00700 1 Processor -- start 2024-05-27T09:38:01.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.063+0000 7fb9b8c00700 1 -- start start 2024-05-27T09:38:01.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.063+0000 7fb9b8c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9b4102610 0x7fb9b4197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:01.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.063+0000 7fb9b8c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b4102fb0 0x7fb9b41984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:01.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.063+0000 7fb9b8c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b4198b80 con 0x7fb9b4102610 2024-05-27T09:38:01.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.063+0000 7fb9b8c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b419c910 con 0x7fb9b4102fb0 2024-05-27T09:38:01.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.063+0000 7fb9b1e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9b4102610 0x7fb9b4197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:01.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.064+0000 7fb9b1e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9b4102610 0x7fb9b4197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59366/0 (socket says 172.21.15.119:59366) 2024-05-27T09:38:01.065 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.064+0000 7fb9b1e00700 1 -- 172.21.15.119:0/4171431084 learned_addr learned my addr 172.21.15.119:0/4171431084 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:38:01.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.064+0000 7fb9b1400700 1 --2- 172.21.15.119:0/4171431084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b4102fb0 0x7fb9b41984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:01.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.064+0000 7fb9b1e00700 1 -- 172.21.15.119:0/4171431084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b4102fb0 msgr2=0x7fb9b41984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:01.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.064+0000 7fb9b1e00700 1 --2- 172.21.15.119:0/4171431084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b4102fb0 0x7fb9b41984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:01.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.064+0000 7fb9b1e00700 1 -- 172.21.15.119:0/4171431084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a40095f0 con 0x7fb9b4102610 2024-05-27T09:38:01.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.064+0000 7fb9b1e00700 1 --2- 172.21.15.119:0/4171431084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9b4102610 0x7fb9b4197f60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009fd0 tx=0x7fb9a400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:01.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.065+0000 7fb9aaa00700 1 -- 172.21.15.119:0/4171431084 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9a401d070 con 0x7fb9b4102610 2024-05-27T09:38:01.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.065+0000 7fb9aaa00700 1 -- 172.21.15.119:0/4171431084 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9a40050d0 con 0x7fb9b4102610 2024-05-27T09:38:01.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.065+0000 7fb9aaa00700 1 -- 172.21.15.119:0/4171431084 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9a4017400 con 0x7fb9b4102610 2024-05-27T09:38:01.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.065+0000 7fb9b8c00700 1 -- 172.21.15.119:0/4171431084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b419cb90 con 0x7fb9b4102610 2024-05-27T09:38:01.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.065+0000 7fb9b8c00700 1 -- 172.21.15.119:0/4171431084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b419d080 con 0x7fb9b4102610 2024-05-27T09:38:01.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.066+0000 7fb9b8c00700 1 -- 172.21.15.119:0/4171431084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b410a920 con 0x7fb9b4102610 2024-05-27T09:38:01.069 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.068+0000 7fb9aaa00700 1 -- 172.21.15.119:0/4171431084 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fb9a4003710 con 0x7fb9b4102610 2024-05-27T09:38:01.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.069+0000 7fb9aaa00700 1 --2- 172.21.15.119:0/4171431084 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb998078a40 0x7fb99807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:01.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.069+0000 7fb9aaa00700 1 -- 172.21.15.119:0/4171431084 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb9a409bfc0 con 0x7fb9b4102610 2024-05-27T09:38:01.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.070+0000 7fb9b1400700 1 --2- 172.21.15.119:0/4171431084 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb998078a40 0x7fb99807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:01.071 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.070+0000 7fb9b1400700 1 --2- 172.21.15.119:0/4171431084 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb998078a40 0x7fb99807aef0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4199580 tx=0x7fb9a0009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:01.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.075+0000 7fb9aaa00700 1 -- 172.21.15.119:0/4171431084 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fb9a40d1df0 con 0x7fb9b4102610 2024-05-27T09:38:01.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.322+0000 7fb9b8c00700 1 -- 172.21.15.119:0/4171431084 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9b402cd40 con 0x7fb998078a40 2024-05-27T09:38:01.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.326+0000 7fb9aaa00700 1 -- 172.21.15.119:0/4171431084 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb9b402cd40 con 0x7fb998078a40 2024-05-27T09:38:01.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.329+0000 7fb9b8c00700 1 -- 172.21.15.119:0/4171431084 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb998078a40 msgr2=0x7fb99807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:01.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.329+0000 7fb9b8c00700 1 --2- 172.21.15.119:0/4171431084 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb998078a40 0x7fb99807aef0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4199580 tx=0x7fb9a0009260 comp rx=0 tx=0).stop 2024-05-27T09:38:01.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.329+0000 7fb9b8c00700 1 -- 172.21.15.119:0/4171431084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9b4102610 msgr2=0x7fb9b4197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:01.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.329+0000 7fb9b8c00700 1 --2- 172.21.15.119:0/4171431084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9b4102610 0x7fb9b4197f60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009fd0 tx=0x7fb9a400bfd0 comp rx=0 tx=0).stop 2024-05-27T09:38:01.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.329+0000 7fb9b8c00700 1 -- 172.21.15.119:0/4171431084 shutdown_connections 2024-05-27T09:38:01.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.329+0000 7fb9b8c00700 1 --2- 172.21.15.119:0/4171431084 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb998078a40 0x7fb99807aef0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:01.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.329+0000 7fb9b8c00700 1 --2- 172.21.15.119:0/4171431084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9b4102610 0x7fb9b4197f60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:01.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.329+0000 7fb9b8c00700 1 --2- 172.21.15.119:0/4171431084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b4102fb0 0x7fb9b41984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:01.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.329+0000 7fb9b8c00700 1 -- 172.21.15.119:0/4171431084 >> 172.21.15.119:0/4171431084 conn(0x7fb9b40fdef0 msgr2=0x7fb9b41002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:01.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.329+0000 7fb9b8c00700 1 -- 172.21.15.119:0/4171431084 shutdown_connections 2024-05-27T09:38:01.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.329+0000 7fb9b8c00700 1 -- 172.21.15.119:0/4171431084 wait complete. 2024-05-27T09:38:01.346 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:38:01.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.486+0000 7f8e36e00700 1 -- 172.21.15.119:0/2277738394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e301050c0 msgr2=0x7f8e300683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:01.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.486+0000 7f8e36e00700 1 --2- 172.21.15.119:0/2277738394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e301050c0 0x7f8e300683d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000b0a0 tx=0x7f8e2000b3b0 comp rx=0 tx=0).stop 2024-05-27T09:38:01.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.486+0000 7f8e36e00700 1 -- 172.21.15.119:0/2277738394 shutdown_connections 2024-05-27T09:38:01.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.486+0000 7f8e36e00700 1 --2- 172.21.15.119:0/2277738394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e30068910 0x7f8e30068d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:01.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.486+0000 7f8e36e00700 1 --2- 172.21.15.119:0/2277738394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e301050c0 0x7f8e300683d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:01.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.486+0000 7f8e36e00700 1 -- 172.21.15.119:0/2277738394 >> 172.21.15.119:0/2277738394 conn(0x7f8e30074f50 msgr2=0x7f8e30075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:01.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.486+0000 7f8e36e00700 1 -- 172.21.15.119:0/2277738394 shutdown_connections 2024-05-27T09:38:01.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.486+0000 7f8e36e00700 1 -- 172.21.15.119:0/2277738394 wait complete. 2024-05-27T09:38:01.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.487+0000 7f8e36e00700 1 Processor -- start 2024-05-27T09:38:01.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.488+0000 7f8e36e00700 1 -- start start 2024-05-27T09:38:01.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.488+0000 7f8e36e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e30068910 0x7f8e3019c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:01.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.488+0000 7f8e36e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e301050c0 0x7f8e3019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:01.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.488+0000 7f8e36e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3019cf70 con 0x7f8e30068910 2024-05-27T09:38:01.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.488+0000 7f8e36e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e301a0d00 con 0x7f8e301050c0 2024-05-27T09:38:01.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.488+0000 7f8e2fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e30068910 0x7f8e3019c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:01.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.488+0000 7f8e2fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e30068910 0x7f8e3019c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59376/0 (socket says 172.21.15.119:59376) 2024-05-27T09:38:01.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.488+0000 7f8e2fe00700 1 -- 172.21.15.119:0/4272720183 learned_addr learned my addr 172.21.15.119:0/4272720183 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:38:01.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.489+0000 7f8e2f400700 1 --2- 172.21.15.119:0/4272720183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e301050c0 0x7f8e3019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:01.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.489+0000 7f8e2fe00700 1 -- 172.21.15.119:0/4272720183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e301050c0 msgr2=0x7f8e3019c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:01.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.489+0000 7f8e2fe00700 1 --2- 172.21.15.119:0/4272720183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e301050c0 0x7f8e3019c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:01.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.489+0000 7f8e2fe00700 1 -- 172.21.15.119:0/4272720183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e20009d00 con 0x7f8e30068910 2024-05-27T09:38:01.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.489+0000 7f8e2f400700 1 --2- 172.21.15.119:0/4272720183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e301050c0 0x7f8e3019c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:38:01.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.489+0000 7f8e2fe00700 1 --2- 172.21.15.119:0/4272720183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e30068910 0x7f8e3019c350 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000b070 tx=0x7f8e2000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:01.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.489+0000 7f8e2cc00700 1 -- 172.21.15.119:0/4272720183 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e2000e040 con 0x7f8e30068910 2024-05-27T09:38:01.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.489+0000 7f8e2cc00700 1 -- 172.21.15.119:0/4272720183 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e20003de0 con 0x7f8e30068910 2024-05-27T09:38:01.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.489+0000 7f8e36e00700 1 -- 172.21.15.119:0/4272720183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e301a0f80 con 0x7f8e30068910 2024-05-27T09:38:01.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.489+0000 7f8e2cc00700 1 -- 172.21.15.119:0/4272720183 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e2001d8e0 con 0x7f8e30068910 2024-05-27T09:38:01.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.489+0000 7f8e36e00700 1 -- 172.21.15.119:0/4272720183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e301a1470 con 0x7f8e30068910 2024-05-27T09:38:01.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.491+0000 7f8e36e00700 1 -- 172.21.15.119:0/4272720183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3010f700 con 0x7f8e30068910 2024-05-27T09:38:01.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.492+0000 7f8e2cc00700 1 -- 172.21.15.119:0/4272720183 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f8e20004040 con 0x7f8e30068910 2024-05-27T09:38:01.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.493+0000 7f8e2cc00700 1 --2- 172.21.15.119:0/4272720183 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8e14078a40 0x7f8e1407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:01.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.493+0000 7f8e2cc00700 1 -- 172.21.15.119:0/4272720183 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8e2009c470 con 0x7f8e30068910 2024-05-27T09:38:01.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.493+0000 7f8e2f400700 1 --2- 172.21.15.119:0/4272720183 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8e14078a40 0x7f8e1407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:01.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.494+0000 7f8e2f400700 1 --2- 172.21.15.119:0/4272720183 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8e14078a40 0x7f8e1407aef0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c005fd0 tx=0x7f8e1c005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:01.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.498+0000 7f8e2cc00700 1 -- 172.21.15.119:0/4272720183 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f8e20064170 con 0x7f8e30068910 2024-05-27T09:38:01.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.739+0000 7f8e36e00700 1 -- 172.21.15.119:0/4272720183 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e301a1810 con 0x7f8e14078a40 2024-05-27T09:38:01.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.741+0000 7f8e2cc00700 1 -- 172.21.15.119:0/4272720183 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8e301a1810 con 0x7f8e14078a40 2024-05-27T09:38:01.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.744+0000 7f8e36e00700 1 -- 172.21.15.119:0/4272720183 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8e14078a40 msgr2=0x7f8e1407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:01.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.744+0000 7f8e36e00700 1 --2- 172.21.15.119:0/4272720183 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8e14078a40 0x7f8e1407aef0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c005fd0 tx=0x7f8e1c005ee0 comp rx=0 tx=0).stop 2024-05-27T09:38:01.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.744+0000 7f8e36e00700 1 -- 172.21.15.119:0/4272720183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e30068910 msgr2=0x7f8e3019c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:01.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.744+0000 7f8e36e00700 1 --2- 172.21.15.119:0/4272720183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e30068910 0x7f8e3019c350 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000b070 tx=0x7f8e2000cdd0 comp rx=0 tx=0).stop 2024-05-27T09:38:01.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.744+0000 7f8e36e00700 1 -- 172.21.15.119:0/4272720183 shutdown_connections 2024-05-27T09:38:01.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.744+0000 7f8e36e00700 1 --2- 172.21.15.119:0/4272720183 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8e14078a40 0x7f8e1407aef0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:01.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.744+0000 7f8e36e00700 1 --2- 172.21.15.119:0/4272720183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e30068910 0x7f8e3019c350 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:01.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.744+0000 7f8e36e00700 1 --2- 172.21.15.119:0/4272720183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e301050c0 0x7f8e3019c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:01.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.744+0000 7f8e36e00700 1 -- 172.21.15.119:0/4272720183 >> 172.21.15.119:0/4272720183 conn(0x7f8e30074f50 msgr2=0x7f8e300fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:01.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.745+0000 7f8e36e00700 1 -- 172.21.15.119:0/4272720183 shutdown_connections 2024-05-27T09:38:01.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.745+0000 7f8e36e00700 1 -- 172.21.15.119:0/4272720183 wait complete. 2024-05-27T09:38:01.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.902+0000 7fac20a00700 1 -- 172.21.15.119:0/4283534253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac1c102de0 msgr2=0x7fac1c106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:01.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.902+0000 7fac20a00700 1 --2- 172.21.15.119:0/4283534253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac1c102de0 0x7fac1c106db0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fac08009910 tx=0x7fac08009c20 comp rx=0 tx=0).stop 2024-05-27T09:38:01.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.902+0000 7fac20a00700 1 -- 172.21.15.119:0/4283534253 shutdown_connections 2024-05-27T09:38:01.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.902+0000 7fac20a00700 1 --2- 172.21.15.119:0/4283534253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac1c102de0 0x7fac1c106db0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:01.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.902+0000 7fac20a00700 1 --2- 172.21.15.119:0/4283534253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac1c102440 0x7fac1c102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:01.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.902+0000 7fac20a00700 1 -- 172.21.15.119:0/4283534253 >> 172.21.15.119:0/4283534253 conn(0x7fac1c0fdce0 msgr2=0x7fac1c1000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:01.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.902+0000 7fac20a00700 1 -- 172.21.15.119:0/4283534253 shutdown_connections 2024-05-27T09:38:01.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.902+0000 7fac20a00700 1 -- 172.21.15.119:0/4283534253 wait complete. 2024-05-27T09:38:01.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.903+0000 7fac20a00700 1 Processor -- start 2024-05-27T09:38:01.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.903+0000 7fac20a00700 1 -- start start 2024-05-27T09:38:01.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.904+0000 7fac20a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac1c102440 0x7fac1c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:01.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.904+0000 7fac20a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac1c102de0 0x7fac1c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:01.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.904+0000 7fac20a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c198af0 con 0x7fac1c102440 2024-05-27T09:38:01.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.904+0000 7fac20a00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c19c880 con 0x7fac1c102de0 2024-05-27T09:38:01.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.904+0000 7fac1a800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac1c102440 0x7fac1c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:01.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.904+0000 7fac1a800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac1c102440 0x7fac1c197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59396/0 (socket says 172.21.15.119:59396) 2024-05-27T09:38:01.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.904+0000 7fac1a800700 1 -- 172.21.15.119:0/4161829943 learned_addr learned my addr 172.21.15.119:0/4161829943 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:38:01.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.904+0000 7fac19e00700 1 --2- 172.21.15.119:0/4161829943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac1c102de0 0x7fac1c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:01.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.904+0000 7fac1a800700 1 -- 172.21.15.119:0/4161829943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac1c102de0 msgr2=0x7fac1c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:01.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.904+0000 7fac1a800700 1 --2- 172.21.15.119:0/4161829943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac1c102de0 0x7fac1c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:01.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.905+0000 7fac1a800700 1 -- 172.21.15.119:0/4161829943 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac080095f0 con 0x7fac1c102440 2024-05-27T09:38:01.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.905+0000 7fac1a800700 1 --2- 172.21.15.119:0/4161829943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac1c102440 0x7fac1c197ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00b4b0 tx=0x7fac0c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:01.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.905+0000 7fac13400700 1 -- 172.21.15.119:0/4161829943 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac0c011800 con 0x7fac1c102440 2024-05-27T09:38:01.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.905+0000 7fac13400700 1 -- 172.21.15.119:0/4161829943 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac0c011e20 con 0x7fac1c102440 2024-05-27T09:38:01.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.905+0000 7fac20a00700 1 -- 172.21.15.119:0/4161829943 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac1c19cb60 con 0x7fac1c102440 2024-05-27T09:38:01.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.905+0000 7fac13400700 1 -- 172.21.15.119:0/4161829943 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac0c00f2e0 con 0x7fac1c102440 2024-05-27T09:38:01.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.905+0000 7fac20a00700 1 -- 172.21.15.119:0/4161829943 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac1c19d0b0 con 0x7fac1c102440 2024-05-27T09:38:01.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.906+0000 7fac20a00700 1 -- 172.21.15.119:0/4161829943 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac1c10a6e0 con 0x7fac1c102440 2024-05-27T09:38:01.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.908+0000 7fac13400700 1 -- 172.21.15.119:0/4161829943 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fac0c010380 con 0x7fac1c102440 2024-05-27T09:38:01.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.909+0000 7fac13400700 1 --2- 172.21.15.119:0/4161829943 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fac00078a40 0x7fac0007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:01.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.909+0000 7fac13400700 1 -- 172.21.15.119:0/4161829943 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fac0c09a8d0 con 0x7fac1c102440 2024-05-27T09:38:01.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.909+0000 7fac19e00700 1 --2- 172.21.15.119:0/4161829943 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fac00078a40 0x7fac0007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:01.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.910+0000 7fac19e00700 1 --2- 172.21.15.119:0/4161829943 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fac00078a40 0x7fac0007aef0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fac1c1994f0 tx=0x7fac0801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:01.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:01.914+0000 7fac13400700 1 -- 172.21.15.119:0/4161829943 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fac0c0624b0 con 0x7fac1c102440 2024-05-27T09:38:02.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.158+0000 7fac20a00700 1 -- 172.21.15.119:0/4161829943 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fac1c19d410 con 0x7fac00078a40 2024-05-27T09:38:02.171 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.170+0000 7fac13400700 1 -- 172.21.15.119:0/4161829943 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fac1c19d410 con 0x7fac00078a40 2024-05-27T09:38:02.171 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:38:02.171 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (5m) 13s ago 19m 14.3M - 0.25.0 c8568f914cd2 6b90564dcef7 2024-05-27T09:38:02.171 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (19m) 13s ago 19m 8850k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:38:02.171 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (18m) 110s ago 18m 8182k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:38:02.171 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (2m) 13s ago 19m 6669k - 19.0.0-3868-ga6507388 850a06a84d28 011d811ce7fc 2024-05-27T09:38:02.171 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (115s) 110s ago 18m 6665k - 19.0.0-3868-ga6507388 850a06a84d28 b546b3ed1afc 2024-05-27T09:38:02.172 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (4m) 13s ago 18m 74.4M - 9.4.12 2bacad6d85d8 b3abdab08db4 2024-05-27T09:38:02.172 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (12m) 13s ago 12m 18.0M - 18.2.0 dc2bc1663786 93b21713fb1e 2024-05-27T09:38:02.172 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (11m) 13s ago 11m 130M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:38:02.172 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (11m) 110s ago 11m 17.0M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:38:02.172 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (12m) 110s ago 12m 128M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:38:02.172 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (7m) 13s ago 20m 587M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:38:02.172 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (6m) 110s ago 17m 532M - 19.0.0-3868-ga6507388 850a06a84d28 2fc3a80696fa 2024-05-27T09:38:02.172 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (3m) 13s ago 21m 114M 2048M 19.0.0-3868-ga6507388 850a06a84d28 3e5c7a9ad372 2024-05-27T09:38:02.172 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (3m) 110s ago 17m 103M 2048M 19.0.0-3868-ga6507388 850a06a84d28 e4d471274cf1 2024-05-27T09:38:02.172 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (6m) 13s ago 19m 8002k - 1.5.0 0da6a335fe13 709c37815cff 2024-05-27T09:38:02.172 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (6m) 110s ago 17m 7776k - 1.5.0 0da6a335fe13 999047a1a5bf 2024-05-27T09:38:02.172 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (83s) 13s ago 16m 152M 1216M 19.0.0-3868-ga6507388 850a06a84d28 5a0e162d870e 2024-05-27T09:38:02.173 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (52s) 13s ago 16m 167M 1216M 19.0.0-3868-ga6507388 850a06a84d28 effcc7c53dd3 2024-05-27T09:38:02.173 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (21s) 13s ago 15m 78.5M 1216M 19.0.0-3868-ga6507388 850a06a84d28 f0bd34ca416f 2024-05-27T09:38:02.173 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (15m) 110s ago 15m 270M 2240M 18.2.0 dc2bc1663786 693c92220622 2024-05-27T09:38:02.173 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (14m) 110s ago 14m 248M 2240M 18.2.0 dc2bc1663786 7559e658e917 2024-05-27T09:38:02.173 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (13m) 110s ago 13m 232M 2240M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:38:02.173 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (5m) 13s ago 18m 71.1M - 2.43.0 a07b618ecd1d cba50454ed21 2024-05-27T09:38:02.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.173+0000 7fac20a00700 1 -- 172.21.15.119:0/4161829943 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fac00078a40 msgr2=0x7fac0007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:02.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.173+0000 7fac20a00700 1 --2- 172.21.15.119:0/4161829943 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fac00078a40 0x7fac0007aef0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fac1c1994f0 tx=0x7fac0801a040 comp rx=0 tx=0).stop 2024-05-27T09:38:02.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.173+0000 7fac20a00700 1 -- 172.21.15.119:0/4161829943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac1c102440 msgr2=0x7fac1c197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:02.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.173+0000 7fac20a00700 1 --2- 172.21.15.119:0/4161829943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac1c102440 0x7fac1c197ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00b4b0 tx=0x7fac0c00b870 comp rx=0 tx=0).stop 2024-05-27T09:38:02.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.173+0000 7fac20a00700 1 -- 172.21.15.119:0/4161829943 shutdown_connections 2024-05-27T09:38:02.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.173+0000 7fac20a00700 1 --2- 172.21.15.119:0/4161829943 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fac00078a40 0x7fac0007aef0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:02.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.173+0000 7fac20a00700 1 --2- 172.21.15.119:0/4161829943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac1c102440 0x7fac1c197ed0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:02.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.173+0000 7fac20a00700 1 --2- 172.21.15.119:0/4161829943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac1c102de0 0x7fac1c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:02.174 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.173+0000 7fac20a00700 1 -- 172.21.15.119:0/4161829943 >> 172.21.15.119:0/4161829943 conn(0x7fac1c0fdce0 msgr2=0x7fac1c0ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:02.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.173+0000 7fac20a00700 1 -- 172.21.15.119:0/4161829943 shutdown_connections 2024-05-27T09:38:02.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.173+0000 7fac20a00700 1 -- 172.21.15.119:0/4161829943 wait complete. 2024-05-27T09:38:02.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.329+0000 7f126b000700 1 -- 172.21.15.119:0/1579707870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1264068fd0 msgr2=0x7f1264104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:02.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.329+0000 7f126b000700 1 --2- 172.21.15.119:0/1579707870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1264068fd0 0x7f1264104df0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1250009910 tx=0x7f1250009c20 comp rx=0 tx=0).stop 2024-05-27T09:38:02.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.330+0000 7f126b000700 1 -- 172.21.15.119:0/1579707870 shutdown_connections 2024-05-27T09:38:02.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.330+0000 7f126b000700 1 --2- 172.21.15.119:0/1579707870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1264068fd0 0x7f1264104df0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:02.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.330+0000 7f126b000700 1 --2- 172.21.15.119:0/1579707870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1264068630 0x7f1264068a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:02.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.330+0000 7f126b000700 1 -- 172.21.15.119:0/1579707870 >> 172.21.15.119:0/1579707870 conn(0x7f1264074f50 msgr2=0x7f1264075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:02.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.330+0000 7f126b000700 1 -- 172.21.15.119:0/1579707870 shutdown_connections 2024-05-27T09:38:02.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.330+0000 7f126b000700 1 -- 172.21.15.119:0/1579707870 wait complete. 2024-05-27T09:38:02.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.331+0000 7f126b000700 1 Processor -- start 2024-05-27T09:38:02.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.331+0000 7f126b000700 1 -- start start 2024-05-27T09:38:02.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.332+0000 7f126b000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1264068630 0x7f1264197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:02.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.332+0000 7f126b000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1264068fd0 0x7f1264198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:02.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.332+0000 7f126b000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1264198b40 con 0x7f1264068630 2024-05-27T09:38:02.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.332+0000 7f126b000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126419c8d0 con 0x7f1264068fd0 2024-05-27T09:38:02.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.332+0000 7f1263e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1264068630 0x7f1264197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:02.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.332+0000 7f1263e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1264068630 0x7f1264197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59412/0 (socket says 172.21.15.119:59412) 2024-05-27T09:38:02.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.332+0000 7f1263e00700 1 -- 172.21.15.119:0/3380502230 learned_addr learned my addr 172.21.15.119:0/3380502230 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:38:02.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.332+0000 7f1263400700 1 --2- 172.21.15.119:0/3380502230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1264068fd0 0x7f1264198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:02.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.333+0000 7f1263e00700 1 -- 172.21.15.119:0/3380502230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1264068fd0 msgr2=0x7f1264198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:02.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.333+0000 7f1263e00700 1 --2- 172.21.15.119:0/3380502230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1264068fd0 0x7f1264198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:02.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.333+0000 7f1263e00700 1 -- 172.21.15.119:0/3380502230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12500095f0 con 0x7f1264068630 2024-05-27T09:38:02.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.333+0000 7f1263400700 1 --2- 172.21.15.119:0/3380502230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1264068fd0 0x7f1264198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:38:02.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.333+0000 7f1263e00700 1 --2- 172.21.15.119:0/3380502230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1264068630 0x7f1264197f20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f125400c8a0 tx=0x7f125400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:02.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.333+0000 7f1260c00700 1 -- 172.21.15.119:0/3380502230 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f125400cea0 con 0x7f1264068630 2024-05-27T09:38:02.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.334+0000 7f1260c00700 1 -- 172.21.15.119:0/3380502230 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1254004830 con 0x7f1264068630 2024-05-27T09:38:02.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.334+0000 7f126b000700 1 -- 172.21.15.119:0/3380502230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126419cbb0 con 0x7f1264068630 2024-05-27T09:38:02.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.334+0000 7f1260c00700 1 -- 172.21.15.119:0/3380502230 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12540055b0 con 0x7f1264068630 2024-05-27T09:38:02.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.334+0000 7f126b000700 1 -- 172.21.15.119:0/3380502230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126419d100 con 0x7f1264068630 2024-05-27T09:38:02.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.335+0000 7f126b000700 1 -- 172.21.15.119:0/3380502230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1264104570 con 0x7f1264068630 2024-05-27T09:38:02.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.337+0000 7f1260c00700 1 -- 172.21.15.119:0/3380502230 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f12540049a0 con 0x7f1264068630 2024-05-27T09:38:02.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.338+0000 7f1260c00700 1 --2- 172.21.15.119:0/3380502230 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1248078a40 0x7f124807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:02.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.338+0000 7f1260c00700 1 -- 172.21.15.119:0/3380502230 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f125409aa10 con 0x7f1264068630 2024-05-27T09:38:02.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.338+0000 7f1263400700 1 --2- 172.21.15.119:0/3380502230 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1248078a40 0x7f124807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:02.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.339+0000 7f1263400700 1 --2- 172.21.15.119:0/3380502230 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1248078a40 0x7f124807aef0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1264199540 tx=0x7f125001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:02.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.344+0000 7f1260c00700 1 -- 172.21.15.119:0/3380502230 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f12540625f0 con 0x7f1264068630 2024-05-27T09:38:02.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:02 smithi138 ceph-mon[70004]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:38:02.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:02 smithi138 ceph-mon[70004]: pgmap v125: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:02.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:02 smithi138 ceph-mon[70004]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:38:02.661 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:02 smithi119 ceph-mon[88269]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:38:02.661 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:02 smithi119 ceph-mon[88269]: pgmap v125: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:02.661 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:02 smithi119 ceph-mon[88269]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:38:02.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.678+0000 7f126b000700 1 -- 172.21.15.119:0/3380502230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f126419d420 con 0x7f1264068630 2024-05-27T09:38:02.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.679+0000 7f1260c00700 1 -- 172.21.15.119:0/3380502230 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f1254004c50 con 0x7f1264068630 2024-05-27T09:38:02.680 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:38:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:38:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:38:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:38:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:38:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:38:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:38:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:38:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 3, 2024-05-27T09:38:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 3 2024-05-27T09:38:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:38:02.680 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:38:02.681 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-05-27T09:38:02.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:38:02.681 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:38:02.681 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 7, 2024-05-27T09:38:02.681 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 7 2024-05-27T09:38:02.681 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:38:02.681 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:38:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.682+0000 7f126b000700 1 -- 172.21.15.119:0/3380502230 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1248078a40 msgr2=0x7f124807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.682+0000 7f126b000700 1 --2- 172.21.15.119:0/3380502230 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1248078a40 0x7f124807aef0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1264199540 tx=0x7f125001a040 comp rx=0 tx=0).stop 2024-05-27T09:38:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.682+0000 7f126b000700 1 -- 172.21.15.119:0/3380502230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1264068630 msgr2=0x7f1264197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.682+0000 7f126b000700 1 --2- 172.21.15.119:0/3380502230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1264068630 0x7f1264197f20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f125400c8a0 tx=0x7f125400cc60 comp rx=0 tx=0).stop 2024-05-27T09:38:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.682+0000 7f126b000700 1 -- 172.21.15.119:0/3380502230 shutdown_connections 2024-05-27T09:38:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.682+0000 7f126b000700 1 --2- 172.21.15.119:0/3380502230 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1248078a40 0x7f124807aef0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.682+0000 7f126b000700 1 --2- 172.21.15.119:0/3380502230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1264068630 0x7f1264197f20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.683+0000 7f126b000700 1 --2- 172.21.15.119:0/3380502230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1264068fd0 0x7f1264198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.683+0000 7f126b000700 1 -- 172.21.15.119:0/3380502230 >> 172.21.15.119:0/3380502230 conn(0x7f1264074f50 msgr2=0x7f12640fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.683+0000 7f126b000700 1 -- 172.21.15.119:0/3380502230 shutdown_connections 2024-05-27T09:38:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.683+0000 7f126b000700 1 -- 172.21.15.119:0/3380502230 wait complete. 2024-05-27T09:38:02.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.838+0000 7f1eb0800700 1 -- 172.21.15.119:0/3520508959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea8068630 msgr2=0x7f1ea8068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:02.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.838+0000 7f1eb0800700 1 --2- 172.21.15.119:0/3520508959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea8068630 0x7f1ea8068a00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c009910 tx=0x7f1e9c009c20 comp rx=0 tx=0).stop 2024-05-27T09:38:02.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.838+0000 7f1eb0800700 1 -- 172.21.15.119:0/3520508959 shutdown_connections 2024-05-27T09:38:02.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.838+0000 7f1eb0800700 1 --2- 172.21.15.119:0/3520508959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ea8068fd0 0x7f1ea8104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:02.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.838+0000 7f1eb0800700 1 --2- 172.21.15.119:0/3520508959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea8068630 0x7f1ea8068a00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:02.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.838+0000 7f1eb0800700 1 -- 172.21.15.119:0/3520508959 >> 172.21.15.119:0/3520508959 conn(0x7f1ea8074f50 msgr2=0x7f1ea8075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:02.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.838+0000 7f1eb0800700 1 -- 172.21.15.119:0/3520508959 shutdown_connections 2024-05-27T09:38:02.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.838+0000 7f1eb0800700 1 -- 172.21.15.119:0/3520508959 wait complete. 2024-05-27T09:38:02.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.839+0000 7f1eb0800700 1 Processor -- start 2024-05-27T09:38:02.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.840+0000 7f1eb0800700 1 -- start start 2024-05-27T09:38:02.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.840+0000 7f1eb0800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ea8068630 0x7f1ea8195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:02.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.840+0000 7f1eb0800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea8068fd0 0x7f1ea8196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:02.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.840+0000 7f1eb0800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea8196950 con 0x7f1ea8068fd0 2024-05-27T09:38:02.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.840+0000 7f1eb0800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea819a6e0 con 0x7f1ea8068630 2024-05-27T09:38:02.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.841+0000 7f1ead400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea8068fd0 0x7f1ea8196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:02.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.841+0000 7f1ead400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea8068fd0 0x7f1ea8196270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59428/0 (socket says 172.21.15.119:59428) 2024-05-27T09:38:02.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.841+0000 7f1ead400700 1 -- 172.21.15.119:0/4174788700 learned_addr learned my addr 172.21.15.119:0/4174788700 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:38:02.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.841+0000 7f1eade00700 1 --2- 172.21.15.119:0/4174788700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ea8068630 0x7f1ea8195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:02.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.841+0000 7f1ead400700 1 -- 172.21.15.119:0/4174788700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ea8068630 msgr2=0x7f1ea8195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:02.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.841+0000 7f1ead400700 1 --2- 172.21.15.119:0/4174788700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ea8068630 0x7f1ea8195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:02.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.841+0000 7f1ead400700 1 -- 172.21.15.119:0/4174788700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9c0095f0 con 0x7f1ea8068fd0 2024-05-27T09:38:02.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.842+0000 7f1ead400700 1 --2- 172.21.15.119:0/4174788700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea8068fd0 0x7f1ea8196270 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800d900 tx=0x7f1e9800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:02.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.842+0000 7f1ea2a00700 1 -- 172.21.15.119:0/4174788700 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e98009900 con 0x7f1ea8068fd0 2024-05-27T09:38:02.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.842+0000 7f1eb0800700 1 -- 172.21.15.119:0/4174788700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea819a9c0 con 0x7f1ea8068fd0 2024-05-27T09:38:02.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.842+0000 7f1ea2a00700 1 -- 172.21.15.119:0/4174788700 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e98010460 con 0x7f1ea8068fd0 2024-05-27T09:38:02.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.842+0000 7f1eb0800700 1 -- 172.21.15.119:0/4174788700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea819af10 con 0x7f1ea8068fd0 2024-05-27T09:38:02.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.842+0000 7f1ea2a00700 1 -- 172.21.15.119:0/4174788700 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e9801b5d0 con 0x7f1ea8068fd0 2024-05-27T09:38:02.844 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.843+0000 7f1eb0800700 1 -- 172.21.15.119:0/4174788700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea81086f0 con 0x7f1ea8068fd0 2024-05-27T09:38:02.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.845+0000 7f1ea2a00700 1 -- 172.21.15.119:0/4174788700 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f1e980105d0 con 0x7f1ea8068fd0 2024-05-27T09:38:02.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.846+0000 7f1ea2a00700 1 --2- 172.21.15.119:0/4174788700 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1e90078a40 0x7f1e9007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:02.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.846+0000 7f1ea2a00700 1 -- 172.21.15.119:0/4174788700 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f1e9809ab30 con 0x7f1ea8068fd0 2024-05-27T09:38:02.846 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.846+0000 7f1eade00700 1 --2- 172.21.15.119:0/4174788700 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1e90078a40 0x7f1e9007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:02.847 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.847+0000 7f1eade00700 1 --2- 172.21.15.119:0/4174788700 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1e90078a40 0x7f1e9007aef0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c006010 tx=0x7f1e9c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:02.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:02.850+0000 7f1ea2a00700 1 -- 172.21.15.119:0/4174788700 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f1e980628d0 con 0x7f1ea8068fd0 2024-05-27T09:38:03.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.129+0000 7f1eb0800700 1 -- 172.21.15.119:0/4174788700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1ea8197090 con 0x7f1ea8068fd0 2024-05-27T09:38:03.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.137+0000 7f1ea2a00700 1 -- 172.21.15.119:0/4174788700 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1889 (secure 0 0 0) 0x7f1e98062270 con 0x7f1ea8068fd0 2024-05-27T09:38:03.138 INFO:teuthology.orchestra.run.smithi119.stdout:e22 2024-05-27T09:38:03.139 INFO:teuthology.orchestra.run.smithi119.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T09:38:03.139 INFO:teuthology.orchestra.run.smithi119.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:38:03.139 INFO:teuthology.orchestra.run.smithi119.stdout:legacy client fscid: 1 2024-05-27T09:38:03.139 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:38:03.139 INFO:teuthology.orchestra.run.smithi119.stdout:Filesystem 'cephfs' (1) 2024-05-27T09:38:03.139 INFO:teuthology.orchestra.run.smithi119.stdout:fs_name cephfs 2024-05-27T09:38:03.139 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 22 2024-05-27T09:38:03.139 INFO:teuthology.orchestra.run.smithi119.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-27T09:38:03.139 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-05-27T09:25:53.387388+0000 2024-05-27T09:38:03.139 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2024-05-27T09:34:01.250304+0000 2024-05-27T09:38:03.139 INFO:teuthology.orchestra.run.smithi119.stdout:tableserver 0 2024-05-27T09:38:03.139 INFO:teuthology.orchestra.run.smithi119.stdout:root 0 2024-05-27T09:38:03.139 INFO:teuthology.orchestra.run.smithi119.stdout:session_timeout 60 2024-05-27T09:38:03.140 INFO:teuthology.orchestra.run.smithi119.stdout:session_autoclose 300 2024-05-27T09:38:03.140 INFO:teuthology.orchestra.run.smithi119.stdout:max_file_size 1099511627776 2024-05-27T09:38:03.140 INFO:teuthology.orchestra.run.smithi119.stdout:max_xattr_size 65536 2024-05-27T09:38:03.140 INFO:teuthology.orchestra.run.smithi119.stdout:required_client_features {} 2024-05-27T09:38:03.140 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure 0 2024-05-27T09:38:03.140 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure_osd_epoch 46 2024-05-27T09:38:03.140 INFO:teuthology.orchestra.run.smithi119.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:38:03.140 INFO:teuthology.orchestra.run.smithi119.stdout:max_mds 2 2024-05-27T09:38:03.140 INFO:teuthology.orchestra.run.smithi119.stdout:in 0,1 2024-05-27T09:38:03.140 INFO:teuthology.orchestra.run.smithi119.stdout:up {0=14472,1=24241} 2024-05-27T09:38:03.140 INFO:teuthology.orchestra.run.smithi119.stdout:failed 2024-05-27T09:38:03.140 INFO:teuthology.orchestra.run.smithi119.stdout:damaged 2024-05-27T09:38:03.140 INFO:teuthology.orchestra.run.smithi119.stdout:stopped 2024-05-27T09:38:03.140 INFO:teuthology.orchestra.run.smithi119.stdout:data_pools [3] 2024-05-27T09:38:03.140 INFO:teuthology.orchestra.run.smithi119.stdout:metadata_pool 2 2024-05-27T09:38:03.141 INFO:teuthology.orchestra.run.smithi119.stdout:inline_data enabled 2024-05-27T09:38:03.141 INFO:teuthology.orchestra.run.smithi119.stdout:balancer 2024-05-27T09:38:03.141 INFO:teuthology.orchestra.run.smithi119.stdout:bal_rank_mask -1 2024-05-27T09:38:03.141 INFO:teuthology.orchestra.run.smithi119.stdout:standby_count_wanted 1 2024-05-27T09:38:03.141 INFO:teuthology.orchestra.run.smithi119.stdout:qdb_cluster leader: 0 members: 2024-05-27T09:38:03.141 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.ppmzzk{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:38:03.141 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.ndfkhd{1:24241} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/897058594,v1:172.21.15.138:6825/897058594] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:38:03.141 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:38:03.141 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:38:03.141 INFO:teuthology.orchestra.run.smithi119.stdout:Standby daemons: 2024-05-27T09:38:03.142 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:38:03.142 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.nghcey{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.119:6826/413736191,v1:172.21.15.119:6827/413736191] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:38:03.142 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.alszpk{-1:24253} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/2345607053,v1:172.21.15.138:6827/2345607053] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:38:03.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.140+0000 7f1eb0800700 1 -- 172.21.15.119:0/4174788700 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1e90078a40 msgr2=0x7f1e9007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:03.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.140+0000 7f1eb0800700 1 --2- 172.21.15.119:0/4174788700 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1e90078a40 0x7f1e9007aef0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c006010 tx=0x7f1e9c01a040 comp rx=0 tx=0).stop 2024-05-27T09:38:03.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.140+0000 7f1eb0800700 1 -- 172.21.15.119:0/4174788700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea8068fd0 msgr2=0x7f1ea8196270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:03.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.140+0000 7f1eb0800700 1 --2- 172.21.15.119:0/4174788700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea8068fd0 0x7f1ea8196270 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800d900 tx=0x7f1e9800dc10 comp rx=0 tx=0).stop 2024-05-27T09:38:03.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.141+0000 7f1eb0800700 1 -- 172.21.15.119:0/4174788700 shutdown_connections 2024-05-27T09:38:03.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.141+0000 7f1eb0800700 1 --2- 172.21.15.119:0/4174788700 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1e90078a40 0x7f1e9007aef0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:03.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.141+0000 7f1eb0800700 1 --2- 172.21.15.119:0/4174788700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ea8068630 0x7f1ea8195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:03.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.141+0000 7f1eb0800700 1 --2- 172.21.15.119:0/4174788700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea8068fd0 0x7f1ea8196270 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:03.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.141+0000 7f1eb0800700 1 -- 172.21.15.119:0/4174788700 >> 172.21.15.119:0/4174788700 conn(0x7f1ea8074f50 msgr2=0x7f1ea80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:03.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.141+0000 7f1eb0800700 1 -- 172.21.15.119:0/4174788700 shutdown_connections 2024-05-27T09:38:03.143 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.141+0000 7f1eb0800700 1 -- 172.21.15.119:0/4174788700 wait complete. 2024-05-27T09:38:03.144 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 22 2024-05-27T09:38:03.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.292+0000 7fe658e00700 1 -- 172.21.15.119:0/1760478666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe654068fd0 msgr2=0x7fe654104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:03.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.292+0000 7fe658e00700 1 --2- 172.21.15.119:0/1760478666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe654068fd0 0x7fe654104df0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe640009910 tx=0x7fe640009c20 comp rx=0 tx=0).stop 2024-05-27T09:38:03.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.293+0000 7fe658e00700 1 -- 172.21.15.119:0/1760478666 shutdown_connections 2024-05-27T09:38:03.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.293+0000 7fe658e00700 1 --2- 172.21.15.119:0/1760478666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe654068fd0 0x7fe654104df0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:03.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.293+0000 7fe658e00700 1 --2- 172.21.15.119:0/1760478666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe654068630 0x7fe654068a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:03.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.293+0000 7fe658e00700 1 -- 172.21.15.119:0/1760478666 >> 172.21.15.119:0/1760478666 conn(0x7fe654074f50 msgr2=0x7fe654075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:03.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.293+0000 7fe658e00700 1 -- 172.21.15.119:0/1760478666 shutdown_connections 2024-05-27T09:38:03.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.293+0000 7fe658e00700 1 -- 172.21.15.119:0/1760478666 wait complete. 2024-05-27T09:38:03.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.294+0000 7fe658e00700 1 Processor -- start 2024-05-27T09:38:03.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.294+0000 7fe658e00700 1 -- start start 2024-05-27T09:38:03.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.294+0000 7fe658e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe654068630 0x7fe6540ff200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:03.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.294+0000 7fe658e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe654068fd0 0x7fe6540ff740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:03.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.294+0000 7fe658e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6540ffc80 con 0x7fe654068630 2024-05-27T09:38:03.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.294+0000 7fe658e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6540ffdc0 con 0x7fe654068fd0 2024-05-27T09:38:03.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.295+0000 7fe651e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe654068630 0x7fe6540ff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:03.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.295+0000 7fe651e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe654068630 0x7fe6540ff200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59436/0 (socket says 172.21.15.119:59436) 2024-05-27T09:38:03.296 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.295+0000 7fe651e00700 1 -- 172.21.15.119:0/2805327591 learned_addr learned my addr 172.21.15.119:0/2805327591 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:38:03.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.295+0000 7fe651400700 1 --2- 172.21.15.119:0/2805327591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe654068fd0 0x7fe6540ff740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:03.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.295+0000 7fe651e00700 1 -- 172.21.15.119:0/2805327591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe654068fd0 msgr2=0x7fe6540ff740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:03.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.295+0000 7fe651e00700 1 --2- 172.21.15.119:0/2805327591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe654068fd0 0x7fe6540ff740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:03.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.295+0000 7fe651e00700 1 -- 172.21.15.119:0/2805327591 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6400095f0 con 0x7fe654068630 2024-05-27T09:38:03.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.296+0000 7fe651e00700 1 --2- 172.21.15.119:0/2805327591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe654068630 0x7fe6540ff200 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe64400d8d0 tx=0x7fe64400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:03.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.296+0000 7fe64aa00700 1 -- 172.21.15.119:0/2805327591 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe644009940 con 0x7fe654068630 2024-05-27T09:38:03.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.296+0000 7fe64aa00700 1 -- 172.21.15.119:0/2805327591 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe644010460 con 0x7fe654068630 2024-05-27T09:38:03.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.296+0000 7fe64aa00700 1 -- 172.21.15.119:0/2805327591 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe64400f5f0 con 0x7fe654068630 2024-05-27T09:38:03.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.296+0000 7fe658e00700 1 -- 172.21.15.119:0/2805327591 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6541a1ff0 con 0x7fe654068630 2024-05-27T09:38:03.297 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.296+0000 7fe658e00700 1 -- 172.21.15.119:0/2805327591 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6541a2400 con 0x7fe654068630 2024-05-27T09:38:03.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.297+0000 7fe658e00700 1 -- 172.21.15.119:0/2805327591 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6541086d0 con 0x7fe654068630 2024-05-27T09:38:03.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.299+0000 7fe64aa00700 1 -- 172.21.15.119:0/2805327591 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fe6440105d0 con 0x7fe654068630 2024-05-27T09:38:03.301 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.300+0000 7fe64aa00700 1 --2- 172.21.15.119:0/2805327591 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe638078a40 0x7fe63807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:03.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.300+0000 7fe64aa00700 1 -- 172.21.15.119:0/2805327591 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe64409abf0 con 0x7fe654068630 2024-05-27T09:38:03.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.300+0000 7fe651400700 1 --2- 172.21.15.119:0/2805327591 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe638078a40 0x7fe63807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:03.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.301+0000 7fe651400700 1 --2- 172.21.15.119:0/2805327591 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe638078a40 0x7fe63807aef0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe6541009f0 tx=0x7fe64001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:03.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.305+0000 7fe64aa00700 1 -- 172.21.15.119:0/2805327591 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fe6440628f0 con 0x7fe654068630 2024-05-27T09:38:03.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.556+0000 7fe658e00700 1 -- 172.21.15.119:0/2805327591 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe65402ccf0 con 0x7fe638078a40 2024-05-27T09:38:03.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.557+0000 7fe64aa00700 1 -- 172.21.15.119:0/2805327591 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fe65402ccf0 con 0x7fe638078a40 2024-05-27T09:38:03.558 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:38:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:38:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:38:03.558 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-27T09:38:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [ 2024-05-27T09:38:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "crash", 2024-05-27T09:38:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "mon", 2024-05-27T09:38:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr" 2024-05-27T09:38:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-05-27T09:38:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "9/23 daemons upgraded", 2024-05-27T09:38:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "Currently upgrading osd daemons", 2024-05-27T09:38:03.559 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:38:03.559 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:38:03.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.560+0000 7fe658e00700 1 -- 172.21.15.119:0/2805327591 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe638078a40 msgr2=0x7fe63807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:03.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.560+0000 7fe658e00700 1 --2- 172.21.15.119:0/2805327591 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe638078a40 0x7fe63807aef0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe6541009f0 tx=0x7fe64001a040 comp rx=0 tx=0).stop 2024-05-27T09:38:03.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.560+0000 7fe658e00700 1 -- 172.21.15.119:0/2805327591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe654068630 msgr2=0x7fe6540ff200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:03.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.560+0000 7fe658e00700 1 --2- 172.21.15.119:0/2805327591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe654068630 0x7fe6540ff200 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe64400d8d0 tx=0x7fe64400dc90 comp rx=0 tx=0).stop 2024-05-27T09:38:03.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.561+0000 7fe658e00700 1 -- 172.21.15.119:0/2805327591 shutdown_connections 2024-05-27T09:38:03.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.561+0000 7fe658e00700 1 --2- 172.21.15.119:0/2805327591 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe638078a40 0x7fe63807aef0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:03.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.561+0000 7fe658e00700 1 --2- 172.21.15.119:0/2805327591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe654068630 0x7fe6540ff200 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:03.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.561+0000 7fe658e00700 1 --2- 172.21.15.119:0/2805327591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe654068fd0 0x7fe6540ff740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:03.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.561+0000 7fe658e00700 1 -- 172.21.15.119:0/2805327591 >> 172.21.15.119:0/2805327591 conn(0x7fe654074f50 msgr2=0x7fe6540fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:03.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.561+0000 7fe658e00700 1 -- 172.21.15.119:0/2805327591 shutdown_connections 2024-05-27T09:38:03.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.561+0000 7fe658e00700 1 -- 172.21.15.119:0/2805327591 wait complete. 2024-05-27T09:38:03.628 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:03 smithi138 ceph-mon[70004]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:38:03.628 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:03 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/3380502230' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:38:03.628 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:03 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/4174788700' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:38:03.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.704+0000 7fb94c200700 1 -- 172.21.15.119:0/3370283636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb944102fd0 msgr2=0x7fb944106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:03.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.704+0000 7fb94c200700 1 --2- 172.21.15.119:0/3370283636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb944102fd0 0x7fb944106fa0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb930009910 tx=0x7fb930009c20 comp rx=0 tx=0).stop 2024-05-27T09:38:03.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.705+0000 7fb94c200700 1 -- 172.21.15.119:0/3370283636 shutdown_connections 2024-05-27T09:38:03.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.705+0000 7fb94c200700 1 --2- 172.21.15.119:0/3370283636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb944102fd0 0x7fb944106fa0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:03.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.705+0000 7fb94c200700 1 --2- 172.21.15.119:0/3370283636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb944102630 0x7fb944102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:03.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.705+0000 7fb94c200700 1 -- 172.21.15.119:0/3370283636 >> 172.21.15.119:0/3370283636 conn(0x7fb9440fdef0 msgr2=0x7fb944100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:03.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.705+0000 7fb94c200700 1 -- 172.21.15.119:0/3370283636 shutdown_connections 2024-05-27T09:38:03.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.705+0000 7fb94c200700 1 -- 172.21.15.119:0/3370283636 wait complete. 2024-05-27T09:38:03.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.706+0000 7fb94c200700 1 Processor -- start 2024-05-27T09:38:03.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.706+0000 7fb94c200700 1 -- start start 2024-05-27T09:38:03.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.707+0000 7fb94c200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb944102630 0x7fb944197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:03.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.707+0000 7fb94c200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb944102fd0 0x7fb944198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:03.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.707+0000 7fb94c200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb944198b70 con 0x7fb944102fd0 2024-05-27T09:38:03.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.707+0000 7fb94c200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb944198cb0 con 0x7fb944102630 2024-05-27T09:38:03.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.707+0000 7fb948e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb944102fd0 0x7fb944198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:03.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.707+0000 7fb948e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb944102fd0 0x7fb944198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59464/0 (socket says 172.21.15.119:59464) 2024-05-27T09:38:03.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.707+0000 7fb948e00700 1 -- 172.21.15.119:0/2829933021 learned_addr learned my addr 172.21.15.119:0/2829933021 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:38:03.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.708+0000 7fb949800700 1 --2- 172.21.15.119:0/2829933021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb944102630 0x7fb944197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:03.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.708+0000 7fb948e00700 1 -- 172.21.15.119:0/2829933021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb944102630 msgr2=0x7fb944197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:03.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.708+0000 7fb948e00700 1 --2- 172.21.15.119:0/2829933021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb944102630 0x7fb944197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:03.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.708+0000 7fb948e00700 1 -- 172.21.15.119:0/2829933021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9300095f0 con 0x7fb944102fd0 2024-05-27T09:38:03.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.708+0000 7fb948e00700 1 --2- 172.21.15.119:0/2829933021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb944102fd0 0x7fb944198520 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb93000bda0 tx=0x7fb93000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:03.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.708+0000 7fb942000700 1 -- 172.21.15.119:0/2829933021 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb93001d070 con 0x7fb944102fd0 2024-05-27T09:38:03.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.709+0000 7fb942000700 1 -- 172.21.15.119:0/2829933021 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb930004fb0 con 0x7fb944102fd0 2024-05-27T09:38:03.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.709+0000 7fb94c200700 1 -- 172.21.15.119:0/2829933021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94419caa0 con 0x7fb944102fd0 2024-05-27T09:38:03.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.709+0000 7fb942000700 1 -- 172.21.15.119:0/2829933021 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb93000f460 con 0x7fb944102fd0 2024-05-27T09:38:03.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.709+0000 7fb94c200700 1 -- 172.21.15.119:0/2829933021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94419cf90 con 0x7fb944102fd0 2024-05-27T09:38:03.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.710+0000 7fb94c200700 1 -- 172.21.15.119:0/2829933021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb94404ea50 con 0x7fb944102fd0 2024-05-27T09:38:03.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.711+0000 7fb942000700 1 -- 172.21.15.119:0/2829933021 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fb930022800 con 0x7fb944102fd0 2024-05-27T09:38:03.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.713+0000 7fb942000700 1 --2- 172.21.15.119:0/2829933021 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb92c078a40 0x7fb92c07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:03.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.713+0000 7fb942000700 1 -- 172.21.15.119:0/2829933021 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb93009c240 con 0x7fb944102fd0 2024-05-27T09:38:03.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.713+0000 7fb949800700 1 --2- 172.21.15.119:0/2829933021 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb92c078a40 0x7fb92c07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:03.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.714+0000 7fb949800700 1 --2- 172.21.15.119:0/2829933021 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb92c078a40 0x7fb92c07aef0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb938005ea0 tx=0x7fb938005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:03.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:03.718+0000 7fb942000700 1 -- 172.21.15.119:0/2829933021 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fb930063f50 con 0x7fb944102fd0 2024-05-27T09:38:03.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:03 smithi119 ceph-mon[88269]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:38:03.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:03 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/3380502230' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:38:03.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:03 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/4174788700' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:38:04.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:04.030+0000 7fb94c200700 1 -- 172.21.15.119:0/2829933021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb944199340 con 0x7fb944102fd0 2024-05-27T09:38:04.032 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:04.031+0000 7fb942000700 1 -- 172.21.15.119:0/2829933021 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fb9300638f0 con 0x7fb944102fd0 2024-05-27T09:38:04.032 INFO:teuthology.orchestra.run.smithi119.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-27T09:38:04.032 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:38:04.032 INFO:teuthology.orchestra.run.smithi119.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:38:04.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:04.034+0000 7fb94c200700 1 -- 172.21.15.119:0/2829933021 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb92c078a40 msgr2=0x7fb92c07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:04.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:04.034+0000 7fb94c200700 1 --2- 172.21.15.119:0/2829933021 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb92c078a40 0x7fb92c07aef0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb938005ea0 tx=0x7fb938005e30 comp rx=0 tx=0).stop 2024-05-27T09:38:04.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:04.034+0000 7fb94c200700 1 -- 172.21.15.119:0/2829933021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb944102fd0 msgr2=0x7fb944198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:04.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:04.034+0000 7fb94c200700 1 --2- 172.21.15.119:0/2829933021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb944102fd0 0x7fb944198520 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb93000bda0 tx=0x7fb93000be80 comp rx=0 tx=0).stop 2024-05-27T09:38:04.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:04.034+0000 7fb94c200700 1 -- 172.21.15.119:0/2829933021 shutdown_connections 2024-05-27T09:38:04.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:04.034+0000 7fb94c200700 1 --2- 172.21.15.119:0/2829933021 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb92c078a40 0x7fb92c07aef0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:04.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:04.034+0000 7fb94c200700 1 --2- 172.21.15.119:0/2829933021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb944102630 0x7fb944197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:04.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:04.034+0000 7fb94c200700 1 --2- 172.21.15.119:0/2829933021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb944102fd0 0x7fb944198520 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:04.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:04.034+0000 7fb94c200700 1 -- 172.21.15.119:0/2829933021 >> 172.21.15.119:0/2829933021 conn(0x7fb9440fdef0 msgr2=0x7fb9440ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:04.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:04.034+0000 7fb94c200700 1 -- 172.21.15.119:0/2829933021 shutdown_connections 2024-05-27T09:38:04.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:04.035+0000 7fb94c200700 1 -- 172.21.15.119:0/2829933021 wait complete. 2024-05-27T09:38:04.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:04 smithi138 ceph-mon[70004]: pgmap v126: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:04.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:04 smithi138 ceph-mon[70004]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:38:04.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:04 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/2829933021' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:38:04.636 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:04 smithi138 systemd[1]: Stopping Ceph osd.3 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:38:04.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:04 smithi119 ceph-mon[88269]: pgmap v126: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:04.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:04 smithi119 ceph-mon[88269]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:38:04.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:04 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/2829933021' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:38:05.136 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:04 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3[40910]: 2024-05-27T09:38:04.820+0000 7fc06a600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T09:38:05.136 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:04 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3[40910]: 2024-05-27T09:38:04.820+0000 7fc06a600700 -1 osd.3 64 *** Got signal Terminated *** 2024-05-27T09:38:05.136 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:04 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3[40910]: 2024-05-27T09:38:04.820+0000 7fc06a600700 -1 osd.3 64 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T09:38:05.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:05 smithi119 ceph-mon[88269]: osd.3 marked itself down and dead 2024-05-27T09:38:05.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:05 smithi138 ceph-mon[70004]: osd.3 marked itself down and dead 2024-05-27T09:38:06.386 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:06 smithi138 podman[78746]: 2024-05-27 09:38:06.086928151 +0000 UTC m=+1.408772238 container died 693c922206225aca220086a81dbe8cbbd8a0b6fb97e5927c99dc733bdf300bc5 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-27T09:38:06.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:06 smithi119 ceph-mon[88269]: pgmap v127: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:06.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:06 smithi119 ceph-mon[88269]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-27T09:38:06.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:06 smithi119 ceph-mon[88269]: osdmap e65: 6 total, 5 up, 6 in 2024-05-27T09:38:06.817 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:06 smithi138 podman[78746]: 2024-05-27 09:38:06.598528569 +0000 UTC m=+1.920372640 container cleanup 693c922206225aca220086a81dbe8cbbd8a0b6fb97e5927c99dc733bdf300bc5 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_CLEAN=True, RELEASE=HEAD) 2024-05-27T09:38:06.817 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:06 smithi138 bash[78746]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3 2024-05-27T09:38:06.817 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:06 smithi138 ceph-mon[70004]: pgmap v127: 65 pgs: 65 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:06.817 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:06 smithi138 ceph-mon[70004]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-27T09:38:06.817 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:06 smithi138 ceph-mon[70004]: osdmap e65: 6 total, 5 up, 6 in 2024-05-27T09:38:07.135 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:06 smithi138 podman[78761]: 2024-05-27 09:38:06.81651799 +0000 UTC m=+0.717591378 container remove 693c922206225aca220086a81dbe8cbbd8a0b6fb97e5927c99dc733bdf300bc5 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, ceph=True, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-05-27T09:38:07.768 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:07 smithi138 podman[78853]: 2024-05-27 09:38:07.557423601 +0000 UTC m=+0.034046760 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:38:07.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:07 smithi138 ceph-mon[70004]: osdmap e66: 6 total, 5 up, 6 in 2024-05-27T09:38:07.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:07 smithi119 ceph-mon[88269]: osdmap e66: 6 total, 5 up, 6 in 2024-05-27T09:38:08.114 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:07 smithi138 podman[78853]: 2024-05-27 09:38:07.767900144 +0000 UTC m=+0.244523340 container create d1f6303c9d13a0f1c8d9de65bf90763c87f7f7d37c71c5e2c8a293a4f0130811 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-27T09:38:08.114 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:08 smithi138 podman[78853]: 2024-05-27 09:38:08.113949012 +0000 UTC m=+0.590572214 container init d1f6303c9d13a0f1c8d9de65bf90763c87f7f7d37c71c5e2c8a293a4f0130811 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T09:38:08.386 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:08 smithi138 podman[78853]: 2024-05-27 09:38:08.125567934 +0000 UTC m=+0.602191145 container start d1f6303c9d13a0f1c8d9de65bf90763c87f7f7d37c71c5e2c8a293a4f0130811 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T09:38:08.386 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:08 smithi138 podman[78853]: 2024-05-27 09:38:08.220665173 +0000 UTC m=+0.697288379 container attach d1f6303c9d13a0f1c8d9de65bf90763c87f7f7d37c71c5e2c8a293a4f0130811 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-27T09:38:08.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:08 smithi138 ceph-mon[70004]: pgmap v130: 65 pgs: 7 peering, 13 stale+active+clean, 45 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:08.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:08 smithi138 ceph-mon[70004]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-05-27T09:38:08.886 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:08 smithi138 podman[78853]: 2024-05-27 09:38:08.593811284 +0000 UTC m=+1.070434487 container died d1f6303c9d13a0f1c8d9de65bf90763c87f7f7d37c71c5e2c8a293a4f0130811 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-27T09:38:08.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:08 smithi119 ceph-mon[88269]: pgmap v130: 65 pgs: 7 peering, 13 stale+active+clean, 45 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:08.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:08 smithi119 ceph-mon[88269]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-05-27T09:38:09.738 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:09 smithi138 ceph-mon[70004]: Health check failed: Degraded data redundancy: 67/381 objects degraded (17.585%), 17 pgs degraded (PG_DEGRADED) 2024-05-27T09:38:09.738 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:09 smithi138 podman[78879]: 2024-05-27 09:38:09.391599128 +0000 UTC m=+0.776877972 container remove d1f6303c9d13a0f1c8d9de65bf90763c87f7f7d37c71c5e2c8a293a4f0130811 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-27T09:38:09.739 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:09 smithi138 systemd[1]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.3.service: Deactivated successfully. 2024-05-27T09:38:09.739 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:09 smithi138 systemd[1]: Stopped Ceph osd.3 for b966b06e-1c09-11ef-bc9b-c7b262605968. 2024-05-27T09:38:09.739 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:09 smithi138 systemd[1]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.3.service: Consumed 1min 55.830s CPU time. 2024-05-27T09:38:09.855 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:09 smithi119 ceph-mon[88269]: Health check failed: Degraded data redundancy: 67/381 objects degraded (17.585%), 17 pgs degraded (PG_DEGRADED) 2024-05-27T09:38:10.135 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:09 smithi138 systemd[1]: Starting Ceph osd.3 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:38:10.539 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:10 smithi138 podman[78979]: 2024-05-27 09:38:10.116155828 +0000 UTC m=+0.035317656 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:38:10.539 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:10 smithi138 podman[78979]: 2024-05-27 09:38:10.225922297 +0000 UTC m=+0.145084152 container create 80f2240a002face9cd7f1986f38902c7c291b0dd838509e90a2ad15b301f113c (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-activate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-27T09:38:10.539 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:10 smithi138 podman[78979]: 2024-05-27 09:38:10.538080927 +0000 UTC m=+0.457242852 container init 80f2240a002face9cd7f1986f38902c7c291b0dd838509e90a2ad15b301f113c (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-activate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521) 2024-05-27T09:38:10.885 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:10 smithi138 podman[78979]: 2024-05-27 09:38:10.549515933 +0000 UTC m=+0.468677808 container start 80f2240a002face9cd7f1986f38902c7c291b0dd838509e90a2ad15b301f113c (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T09:38:10.885 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:10 smithi138 podman[78979]: 2024-05-27 09:38:10.653645785 +0000 UTC m=+0.572807716 container attach 80f2240a002face9cd7f1986f38902c7c291b0dd838509e90a2ad15b301f113c (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-activate, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-27T09:38:10.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:10 smithi138 ceph-mon[70004]: pgmap v131: 65 pgs: 17 active+undersized+degraded, 7 peering, 14 active+undersized, 27 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 67/381 objects degraded (17.585%) 2024-05-27T09:38:10.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:10 smithi119 ceph-mon[88269]: pgmap v131: 65 pgs: 17 active+undersized+degraded, 7 peering, 14 active+undersized, 27 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 67/381 objects degraded (17.585%) 2024-05-27T09:38:11.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:11 smithi138 ceph-mon[70004]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-05-27T09:38:11.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:11 smithi119 ceph-mon[88269]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-05-27T09:38:12.637 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:12 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-activate[78994]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:38:12.638 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:12 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-activate[78994]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:38:12.638 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:12 smithi138 bash[78979]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:38:12.638 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:12 smithi138 bash[78979]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:38:12.638 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:12 smithi138 bash[78979]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-27T09:38:12.638 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:12 smithi138 bash[78979]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-27T09:38:12.638 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:12 smithi138 bash[78979]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-27T09:38:12.638 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:12 smithi138 bash[78979]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-27T09:38:12.638 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:12 smithi138 bash[78979]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-05-27T09:38:12.638 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:12 smithi138 bash[78979]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-27T09:38:12.638 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:12 smithi138 bash[78979]: --> ceph-volume raw activate successful for osd ID: 3 2024-05-27T09:38:12.638 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:12 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-activate[78994]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-27T09:38:12.638 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:12 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-activate[78994]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-27T09:38:12.638 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:12 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-activate[78994]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-27T09:38:12.639 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:12 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-activate[78994]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-27T09:38:12.639 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:12 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-activate[78994]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-05-27T09:38:12.639 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:12 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-activate[78994]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-27T09:38:12.639 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:12 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-activate[78994]: --> ceph-volume raw activate successful for osd ID: 3 2024-05-27T09:38:12.639 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:12 smithi138 podman[78979]: 2024-05-27 09:38:12.445891272 +0000 UTC m=+2.365053112 container died 80f2240a002face9cd7f1986f38902c7c291b0dd838509e90a2ad15b301f113c (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-activate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T09:38:12.890 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:12 smithi138 ceph-mon[70004]: pgmap v132: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 77/381 objects degraded (20.210%) 2024-05-27T09:38:12.890 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:12.890 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:38:12.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:12 smithi119 ceph-mon[88269]: pgmap v132: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 77/381 objects degraded (20.210%) 2024-05-27T09:38:12.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:12.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:38:13.622 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:13 smithi138 podman[79125]: 2024-05-27 09:38:13.257687267 +0000 UTC m=+0.790835479 container remove 80f2240a002face9cd7f1986f38902c7c291b0dd838509e90a2ad15b301f113c (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T09:38:13.885 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:13 smithi138 podman[79184]: 2024-05-27 09:38:13.529626226 +0000 UTC m=+0.039890088 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:38:13.886 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:13 smithi138 podman[79184]: 2024-05-27 09:38:13.650386239 +0000 UTC m=+0.160650105 container create 5e8d3e398c210896afc240ac7734da3ee6cdd4a86aaea5301790d35ac3fc5f5b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-27T09:38:14.385 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:13 smithi138 podman[79184]: 2024-05-27 09:38:13.947791079 +0000 UTC m=+0.458054929 container init 5e8d3e398c210896afc240ac7734da3ee6cdd4a86aaea5301790d35ac3fc5f5b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-27T09:38:14.385 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:13 smithi138 podman[79184]: 2024-05-27 09:38:13.95905517 +0000 UTC m=+0.469319007 container start 5e8d3e398c210896afc240ac7734da3ee6cdd4a86aaea5301790d35ac3fc5f5b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-27T09:38:14.386 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:14 smithi138 bash[79184]: 5e8d3e398c210896afc240ac7734da3ee6cdd4a86aaea5301790d35ac3fc5f5b 2024-05-27T09:38:14.386 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:14 smithi138 systemd[1]: Started Ceph osd.3 for b966b06e-1c09-11ef-bc9b-c7b262605968. 2024-05-27T09:38:14.386 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:14 smithi138 ceph-osd[79203]: -- 172.21.15.138:0/1679768877 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d26f6b1680 con 0x55d26f758880 2024-05-27T09:38:14.886 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:14 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3[79199]: 2024-05-27T09:38:14.610+0000 7fd3edb307c0 -1 Falling back to public interface 2024-05-27T09:38:14.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:14 smithi138 ceph-mon[70004]: pgmap v133: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 77/381 objects degraded (20.210%) 2024-05-27T09:38:14.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:14.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:14.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:38:14.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:14 smithi119 ceph-mon[88269]: pgmap v133: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 77/381 objects degraded (20.210%) 2024-05-27T09:38:14.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:14.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:14.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:38:16.349 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:16 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:16.349 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:16 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:16.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:16 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:16.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:16 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:17.342 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:17 smithi138 ceph-mon[70004]: pgmap v134: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 77/381 objects degraded (20.210%) 2024-05-27T09:38:17.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:17 smithi119 ceph-mon[88269]: pgmap v134: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 77/381 objects degraded (20.210%) 2024-05-27T09:38:18.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:18 smithi138 ceph-mon[70004]: Health check update: Degraded data redundancy: 77/381 objects degraded (20.210%), 20 pgs degraded (PG_DEGRADED) 2024-05-27T09:38:18.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:18 smithi119 ceph-mon[88269]: Health check update: Degraded data redundancy: 77/381 objects degraded (20.210%), 20 pgs degraded (PG_DEGRADED) 2024-05-27T09:38:19.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:19 smithi138 ceph-mon[70004]: pgmap v135: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 77/381 objects degraded (20.210%) 2024-05-27T09:38:19.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:19 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:19.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:19 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:19.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:19 smithi119 ceph-mon[88269]: pgmap v135: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 77/381 objects degraded (20.210%) 2024-05-27T09:38:19.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:19 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:19.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:19 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:21.385 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:20 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3[79199]: 2024-05-27T09:38:20.884+0000 7fd3edb307c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-05-27T09:38:21.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:21 smithi138 ceph-mon[70004]: pgmap v136: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 77/381 objects degraded (20.210%) 2024-05-27T09:38:21.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:21 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:21.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:21 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:21.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:21 smithi119 ceph-mon[88269]: pgmap v136: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 77/381 objects degraded (20.210%) 2024-05-27T09:38:21.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:21 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:21.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:21 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:21.886 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:21 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3[79199]: 2024-05-27T09:38:21.609+0000 7fd3edb307c0 -1 osd.3 64 log_to_monitors true 2024-05-27T09:38:22.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:22 smithi138 ceph-mon[70004]: from='osd.3 [v2:172.21.15.138:6800/1356115643,v1:172.21.15.138:6801/1356115643]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T09:38:22.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:22 smithi138 ceph-mon[70004]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T09:38:22.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:22 smithi119 ceph-mon[88269]: from='osd.3 [v2:172.21.15.138:6800/1356115643,v1:172.21.15.138:6801/1356115643]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T09:38:22.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:22 smithi119 ceph-mon[88269]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T09:38:23.231 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:38:23 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3[79199]: 2024-05-27T09:38:23.152+0000 7fd3d7e00700 -1 osd.3 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T09:38:23.231 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:23 smithi138 ceph-mon[70004]: pgmap v137: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 77/381 objects degraded (20.210%) 2024-05-27T09:38:23.231 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:23 smithi138 ceph-mon[70004]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-27T09:38:23.231 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:23 smithi138 ceph-mon[70004]: osdmap e67: 6 total, 5 up, 6 in 2024-05-27T09:38:23.231 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:23 smithi138 ceph-mon[70004]: from='osd.3 [v2:172.21.15.138:6800/1356115643,v1:172.21.15.138:6801/1356115643]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:38:23.231 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:23 smithi138 ceph-mon[70004]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:38:23.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:23 smithi119 ceph-mon[88269]: pgmap v137: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 332 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 77/381 objects degraded (20.210%) 2024-05-27T09:38:23.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:23 smithi119 ceph-mon[88269]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-27T09:38:23.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:23 smithi119 ceph-mon[88269]: osdmap e67: 6 total, 5 up, 6 in 2024-05-27T09:38:23.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:23 smithi119 ceph-mon[88269]: from='osd.3 [v2:172.21.15.138:6800/1356115643,v1:172.21.15.138:6801/1356115643]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:38:23.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:23 smithi119 ceph-mon[88269]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:38:24.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:24 smithi119 ceph-mon[88269]: from='osd.3 ' entity='osd.3' 2024-05-27T09:38:24.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:24 smithi119 ceph-mon[88269]: pgmap v139: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 77/381 objects degraded (20.210%) 2024-05-27T09:38:24.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:24 smithi138 ceph-mon[70004]: from='osd.3 ' entity='osd.3' 2024-05-27T09:38:24.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:24 smithi138 ceph-mon[70004]: pgmap v139: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 77/381 objects degraded (20.210%) 2024-05-27T09:38:25.316 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:25 smithi138 ceph-mon[70004]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-27T09:38:25.316 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:25 smithi138 ceph-mon[70004]: osd.3 [v2:172.21.15.138:6800/1356115643,v1:172.21.15.138:6801/1356115643] boot 2024-05-27T09:38:25.316 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:25 smithi138 ceph-mon[70004]: osdmap e68: 6 total, 6 up, 6 in 2024-05-27T09:38:25.316 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:25 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T09:38:25.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:25 smithi119 ceph-mon[88269]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-27T09:38:25.501 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:25 smithi119 ceph-mon[88269]: osd.3 [v2:172.21.15.138:6800/1356115643,v1:172.21.15.138:6801/1356115643] boot 2024-05-27T09:38:25.501 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:25 smithi119 ceph-mon[88269]: osdmap e68: 6 total, 6 up, 6 in 2024-05-27T09:38:25.501 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:25 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T09:38:26.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:26 smithi138 ceph-mon[70004]: osdmap e69: 6 total, 6 up, 6 in 2024-05-27T09:38:26.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:26 smithi138 ceph-mon[70004]: pgmap v142: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 77/381 objects degraded (20.210%) 2024-05-27T09:38:26.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:26 smithi119 ceph-mon[88269]: osdmap e69: 6 total, 6 up, 6 in 2024-05-27T09:38:26.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:26 smithi119 ceph-mon[88269]: pgmap v142: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 77/381 objects degraded (20.210%) 2024-05-27T09:38:27.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:27 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:27.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:27 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:27.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:27 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:38:27.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:27 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:38:27.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:27 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:27.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:27 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:38:27.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:27 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:38:27.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:27 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:38:27.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:27 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:38:27.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:27 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-27T09:38:27.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:27 smithi138 ceph-mon[70004]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-27T09:38:27.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:27 smithi138 ceph-mon[70004]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-05-27T09:38:27.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:27 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:27.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:27 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:27.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:27 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:38:27.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:27 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:38:27.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:27 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:27.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:27 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:38:27.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:27 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:38:27.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:27 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:38:27.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:27 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:38:27.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:27 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-27T09:38:27.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:27 smithi119 ceph-mon[88269]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-27T09:38:27.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:27 smithi119 ceph-mon[88269]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-05-27T09:38:28.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:28 smithi138 ceph-mon[70004]: pgmap v143: 65 pgs: 9 active+undersized+degraded, 9 active+undersized, 47 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27/381 objects degraded (7.087%) 2024-05-27T09:38:28.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:28 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:28.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:28 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:38:28.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:28 smithi119 ceph-mon[88269]: pgmap v143: 65 pgs: 9 active+undersized+degraded, 9 active+undersized, 47 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27/381 objects degraded (7.087%) 2024-05-27T09:38:28.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:28.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:38:29.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:29 smithi138 ceph-mon[70004]: Health check update: Degraded data redundancy: 27/381 objects degraded (7.087%), 9 pgs degraded (PG_DEGRADED) 2024-05-27T09:38:29.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:29 smithi119 ceph-mon[88269]: Health check update: Degraded data redundancy: 27/381 objects degraded (7.087%), 9 pgs degraded (PG_DEGRADED) 2024-05-27T09:38:30.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:30 smithi138 ceph-mon[70004]: pgmap v144: 65 pgs: 65 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:30.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:30 smithi138 ceph-mon[70004]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 27/381 objects degraded (7.087%), 9 pgs degraded) 2024-05-27T09:38:30.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:30 smithi119 ceph-mon[88269]: pgmap v144: 65 pgs: 65 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:30.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:30 smithi119 ceph-mon[88269]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 27/381 objects degraded (7.087%), 9 pgs degraded) 2024-05-27T09:38:32.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:32 smithi138 ceph-mon[70004]: pgmap v145: 65 pgs: 65 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:32.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:32 smithi119 ceph-mon[88269]: pgmap v145: 65 pgs: 65 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:34.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.205+0000 7fbc7fc00700 1 -- 172.21.15.119:0/299513738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc78073a00 msgr2=0x7fbc78110c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:34.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.205+0000 7fbc7fc00700 1 --2- 172.21.15.119:0/299513738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc78073a00 0x7fbc78110c20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbc68009910 tx=0x7fbc68009c20 comp rx=0 tx=0).stop 2024-05-27T09:38:34.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.205+0000 7fbc7fc00700 1 -- 172.21.15.119:0/299513738 shutdown_connections 2024-05-27T09:38:34.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.205+0000 7fbc7fc00700 1 --2- 172.21.15.119:0/299513738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc78073a00 0x7fbc78110c20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:34.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.205+0000 7fbc7fc00700 1 --2- 172.21.15.119:0/299513738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc780730f0 0x7fbc780734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:34.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.205+0000 7fbc7fc00700 1 -- 172.21.15.119:0/299513738 >> 172.21.15.119:0/299513738 conn(0x7fbc780fbc30 msgr2=0x7fbc780fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:34.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.205+0000 7fbc7fc00700 1 -- 172.21.15.119:0/299513738 shutdown_connections 2024-05-27T09:38:34.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.205+0000 7fbc7fc00700 1 -- 172.21.15.119:0/299513738 wait complete. 2024-05-27T09:38:34.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.206+0000 7fbc7fc00700 1 Processor -- start 2024-05-27T09:38:34.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.206+0000 7fbc7fc00700 1 -- start start 2024-05-27T09:38:34.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.207+0000 7fbc7fc00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc780730f0 0x7fbc7819c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:34.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.207+0000 7fbc7fc00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc78073a00 0x7fbc7819c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:34.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.207+0000 7fbc7fc00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7819cf20 con 0x7fbc78073a00 2024-05-27T09:38:34.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.207+0000 7fbc7fc00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc781a0cb0 con 0x7fbc780730f0 2024-05-27T09:38:34.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.207+0000 7fbc6fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc78073a00 0x7fbc7819c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:34.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.207+0000 7fbc6fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc78073a00 0x7fbc7819c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41496/0 (socket says 172.21.15.119:41496) 2024-05-27T09:38:34.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.207+0000 7fbc6fe00700 1 -- 172.21.15.119:0/2034258756 learned_addr learned my addr 172.21.15.119:0/2034258756 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:38:34.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.208+0000 7fbc7d200700 1 --2- 172.21.15.119:0/2034258756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc780730f0 0x7fbc7819c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:34.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.208+0000 7fbc6fe00700 1 -- 172.21.15.119:0/2034258756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc780730f0 msgr2=0x7fbc7819c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:34.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.208+0000 7fbc6fe00700 1 --2- 172.21.15.119:0/2034258756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc780730f0 0x7fbc7819c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:34.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.208+0000 7fbc6fe00700 1 -- 172.21.15.119:0/2034258756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc680095f0 con 0x7fbc78073a00 2024-05-27T09:38:34.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.208+0000 7fbc6fe00700 1 --2- 172.21.15.119:0/2034258756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc78073a00 0x7fbc7819c840 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800bda0 tx=0x7fbc6800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:34.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.208+0000 7fbc76000700 1 -- 172.21.15.119:0/2034258756 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc6801d070 con 0x7fbc78073a00 2024-05-27T09:38:34.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.208+0000 7fbc76000700 1 -- 172.21.15.119:0/2034258756 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc68004fb0 con 0x7fbc78073a00 2024-05-27T09:38:34.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.208+0000 7fbc76000700 1 -- 172.21.15.119:0/2034258756 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc6800f690 con 0x7fbc78073a00 2024-05-27T09:38:34.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.208+0000 7fbc7fc00700 1 -- 172.21.15.119:0/2034258756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc781a0f90 con 0x7fbc78073a00 2024-05-27T09:38:34.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.209+0000 7fbc7fc00700 1 -- 172.21.15.119:0/2034258756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc781a14e0 con 0x7fbc78073a00 2024-05-27T09:38:34.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.210+0000 7fbc6ea00700 1 -- 172.21.15.119:0/2034258756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc7804ea50 con 0x7fbc78073a00 2024-05-27T09:38:34.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.211+0000 7fbc76000700 1 -- 172.21.15.119:0/2034258756 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fbc68004af0 con 0x7fbc78073a00 2024-05-27T09:38:34.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.212+0000 7fbc76000700 1 --2- 172.21.15.119:0/2034258756 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fbc58078a40 0x7fbc5807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:34.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.212+0000 7fbc76000700 1 -- 172.21.15.119:0/2034258756 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbc6809c2b0 con 0x7fbc78073a00 2024-05-27T09:38:34.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.213+0000 7fbc7d200700 1 --2- 172.21.15.119:0/2034258756 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fbc58078a40 0x7fbc5807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:34.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.213+0000 7fbc7d200700 1 --2- 172.21.15.119:0/2034258756 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fbc58078a40 0x7fbc5807aef0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbc60005fd0 tx=0x7fbc6000a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:34.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.218+0000 7fbc76000700 1 -- 172.21.15.119:0/2034258756 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fbc68063fb0 con 0x7fbc78073a00 2024-05-27T09:38:34.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.475+0000 7fbc6ea00700 1 -- 172.21.15.119:0/2034258756 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc7819d660 con 0x7fbc58078a40 2024-05-27T09:38:34.477 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:34 smithi119 ceph-mon[88269]: pgmap v146: 65 pgs: 65 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:34.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.478+0000 7fbc76000700 1 -- 172.21.15.119:0/2034258756 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fbc7819d660 con 0x7fbc58078a40 2024-05-27T09:38:34.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.481+0000 7fbc6ea00700 1 -- 172.21.15.119:0/2034258756 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fbc58078a40 msgr2=0x7fbc5807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:34.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.481+0000 7fbc6ea00700 1 --2- 172.21.15.119:0/2034258756 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fbc58078a40 0x7fbc5807aef0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbc60005fd0 tx=0x7fbc6000a560 comp rx=0 tx=0).stop 2024-05-27T09:38:34.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.481+0000 7fbc6ea00700 1 -- 172.21.15.119:0/2034258756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc78073a00 msgr2=0x7fbc7819c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:34.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.481+0000 7fbc6ea00700 1 --2- 172.21.15.119:0/2034258756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc78073a00 0x7fbc7819c840 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800bda0 tx=0x7fbc6800be80 comp rx=0 tx=0).stop 2024-05-27T09:38:34.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.481+0000 7fbc6ea00700 1 -- 172.21.15.119:0/2034258756 shutdown_connections 2024-05-27T09:38:34.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.481+0000 7fbc6ea00700 1 --2- 172.21.15.119:0/2034258756 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fbc58078a40 0x7fbc5807aef0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:34.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.481+0000 7fbc6ea00700 1 --2- 172.21.15.119:0/2034258756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc780730f0 0x7fbc7819c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:34.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.481+0000 7fbc6ea00700 1 --2- 172.21.15.119:0/2034258756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc78073a00 0x7fbc7819c840 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:34.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.481+0000 7fbc6ea00700 1 -- 172.21.15.119:0/2034258756 >> 172.21.15.119:0/2034258756 conn(0x7fbc780fbc30 msgr2=0x7fbc78102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:34.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.482+0000 7fbc6ea00700 1 -- 172.21.15.119:0/2034258756 shutdown_connections 2024-05-27T09:38:34.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.482+0000 7fbc6ea00700 1 -- 172.21.15.119:0/2034258756 wait complete. 2024-05-27T09:38:34.498 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:38:34.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:34 smithi138 ceph-mon[70004]: pgmap v146: 65 pgs: 65 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:34.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.636+0000 7f90f7800700 1 -- 172.21.15.119:0/1980323205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90f010b700 msgr2=0x7f90f010bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:34.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.636+0000 7f90f7800700 1 --2- 172.21.15.119:0/1980323205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90f010b700 0x7f90f010bad0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009910 tx=0x7f90e4009c20 comp rx=0 tx=0).stop 2024-05-27T09:38:34.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.636+0000 7f90f7800700 1 -- 172.21.15.119:0/1980323205 shutdown_connections 2024-05-27T09:38:34.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.636+0000 7f90f7800700 1 --2- 172.21.15.119:0/1980323205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f010c010 0x7f90f0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:34.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.636+0000 7f90f7800700 1 --2- 172.21.15.119:0/1980323205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90f010b700 0x7f90f010bad0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:34.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.636+0000 7f90f7800700 1 -- 172.21.15.119:0/1980323205 >> 172.21.15.119:0/1980323205 conn(0x7f90f0076000 msgr2=0x7f90f0078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:34.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.636+0000 7f90f7800700 1 -- 172.21.15.119:0/1980323205 shutdown_connections 2024-05-27T09:38:34.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.636+0000 7f90f7800700 1 -- 172.21.15.119:0/1980323205 wait complete. 2024-05-27T09:38:34.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.637+0000 7f90f7800700 1 Processor -- start 2024-05-27T09:38:34.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.638+0000 7f90f7800700 1 -- start start 2024-05-27T09:38:34.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.638+0000 7f90f7800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90f010b700 0x7f90f019c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:34.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.638+0000 7f90f7800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f010c010 0x7f90f019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:34.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.638+0000 7f90f7800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f019d070 con 0x7f90f010b700 2024-05-27T09:38:34.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.638+0000 7f90f7800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f01a0d60 con 0x7f90f010c010 2024-05-27T09:38:34.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.638+0000 7f90f4e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90f010b700 0x7f90f019c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:34.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.638+0000 7f90f4e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90f010b700 0x7f90f019c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41522/0 (socket says 172.21.15.119:41522) 2024-05-27T09:38:34.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.638+0000 7f90f4e00700 1 -- 172.21.15.119:0/256740900 learned_addr learned my addr 172.21.15.119:0/256740900 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:38:34.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.639+0000 7f90f4e00700 1 -- 172.21.15.119:0/256740900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f010c010 msgr2=0x7f90f019c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:34.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.639+0000 7f90f4e00700 1 --2- 172.21.15.119:0/256740900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f010c010 0x7f90f019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:34.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.639+0000 7f90f4e00700 1 -- 172.21.15.119:0/256740900 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e40095f0 con 0x7f90f010b700 2024-05-27T09:38:34.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.639+0000 7f90f4e00700 1 --2- 172.21.15.119:0/256740900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90f010b700 0x7f90f019c450 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f90e400b5c0 tx=0x7f90e400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:34.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.639+0000 7f90ed600700 1 -- 172.21.15.119:0/256740900 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90e401d070 con 0x7f90f010b700 2024-05-27T09:38:34.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.639+0000 7f90ed600700 1 -- 172.21.15.119:0/256740900 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90e400f460 con 0x7f90f010b700 2024-05-27T09:38:34.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.639+0000 7f90ed600700 1 -- 172.21.15.119:0/256740900 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90e4017610 con 0x7f90f010b700 2024-05-27T09:38:34.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.639+0000 7f90f7800700 1 -- 172.21.15.119:0/256740900 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f01a0fe0 con 0x7f90f010b700 2024-05-27T09:38:34.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.640+0000 7f90f7800700 1 -- 172.21.15.119:0/256740900 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f01a1500 con 0x7f90f010b700 2024-05-27T09:38:34.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.641+0000 7f90f7800700 1 -- 172.21.15.119:0/256740900 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f004ea50 con 0x7f90f010b700 2024-05-27T09:38:34.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.642+0000 7f90ed600700 1 -- 172.21.15.119:0/256740900 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f90e400f5d0 con 0x7f90f010b700 2024-05-27T09:38:34.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.643+0000 7f90ed600700 1 --2- 172.21.15.119:0/256740900 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f90d8078a40 0x7f90d807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:34.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.643+0000 7f90ed600700 1 -- 172.21.15.119:0/256740900 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f90e409c3a0 con 0x7f90f010b700 2024-05-27T09:38:34.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.643+0000 7f90efe00700 1 --2- 172.21.15.119:0/256740900 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f90d8078a40 0x7f90d807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:34.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.644+0000 7f90efe00700 1 --2- 172.21.15.119:0/256740900 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f90d8078a40 0x7f90d807aef0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f90f019d9d0 tx=0x7f90dc006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:34.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.648+0000 7f90ed600700 1 -- 172.21.15.119:0/256740900 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f90e4063f80 con 0x7f90f010b700 2024-05-27T09:38:34.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.905+0000 7f90f7800700 1 -- 172.21.15.119:0/256740900 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f90f01a1a50 con 0x7f90d8078a40 2024-05-27T09:38:34.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.908+0000 7f90ed600700 1 -- 172.21.15.119:0/256740900 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f90f01a1a50 con 0x7f90d8078a40 2024-05-27T09:38:34.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.911+0000 7f90f7800700 1 -- 172.21.15.119:0/256740900 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f90d8078a40 msgr2=0x7f90d807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:34.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.911+0000 7f90f7800700 1 --2- 172.21.15.119:0/256740900 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f90d8078a40 0x7f90d807aef0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f90f019d9d0 tx=0x7f90dc006d20 comp rx=0 tx=0).stop 2024-05-27T09:38:34.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.911+0000 7f90f7800700 1 -- 172.21.15.119:0/256740900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90f010b700 msgr2=0x7f90f019c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:34.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.911+0000 7f90f7800700 1 --2- 172.21.15.119:0/256740900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90f010b700 0x7f90f019c450 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f90e400b5c0 tx=0x7f90e400be00 comp rx=0 tx=0).stop 2024-05-27T09:38:34.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.912+0000 7f90f7800700 1 -- 172.21.15.119:0/256740900 shutdown_connections 2024-05-27T09:38:34.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.912+0000 7f90f7800700 1 --2- 172.21.15.119:0/256740900 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f90d8078a40 0x7f90d807aef0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:34.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.912+0000 7f90f7800700 1 --2- 172.21.15.119:0/256740900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90f010b700 0x7f90f019c450 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:34.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.912+0000 7f90f7800700 1 --2- 172.21.15.119:0/256740900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f010c010 0x7f90f019c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:34.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.912+0000 7f90f7800700 1 -- 172.21.15.119:0/256740900 >> 172.21.15.119:0/256740900 conn(0x7f90f0076000 msgr2=0x7f90f0076a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:34.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.912+0000 7f90f7800700 1 -- 172.21.15.119:0/256740900 shutdown_connections 2024-05-27T09:38:34.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:34.912+0000 7f90f7800700 1 -- 172.21.15.119:0/256740900 wait complete. 2024-05-27T09:38:35.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.075+0000 7f2e73000700 1 -- 172.21.15.119:0/3812637987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e6c10b700 msgr2=0x7f2e6c10bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:35.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.075+0000 7f2e73000700 1 --2- 172.21.15.119:0/3812637987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e6c10b700 0x7f2e6c10bad0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009910 tx=0x7f2e5c009c20 comp rx=0 tx=0).stop 2024-05-27T09:38:35.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.075+0000 7f2e73000700 1 -- 172.21.15.119:0/3812637987 shutdown_connections 2024-05-27T09:38:35.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.075+0000 7f2e73000700 1 --2- 172.21.15.119:0/3812637987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e6c10c010 0x7f2e6c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:35.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.075+0000 7f2e73000700 1 --2- 172.21.15.119:0/3812637987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e6c10b700 0x7f2e6c10bad0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:35.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.075+0000 7f2e73000700 1 -- 172.21.15.119:0/3812637987 >> 172.21.15.119:0/3812637987 conn(0x7f2e6c076000 msgr2=0x7f2e6c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:35.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.075+0000 7f2e73000700 1 -- 172.21.15.119:0/3812637987 shutdown_connections 2024-05-27T09:38:35.076 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.075+0000 7f2e73000700 1 -- 172.21.15.119:0/3812637987 wait complete. 2024-05-27T09:38:35.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.076+0000 7f2e73000700 1 Processor -- start 2024-05-27T09:38:35.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.076+0000 7f2e73000700 1 -- start start 2024-05-27T09:38:35.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.076+0000 7f2e73000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e6c10b700 0x7f2e6c19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:35.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.076+0000 7f2e73000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e6c10c010 0x7f2e6c19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:35.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.076+0000 7f2e73000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c19d070 con 0x7f2e6c10c010 2024-05-27T09:38:35.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.076+0000 7f2e73000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c1a0d60 con 0x7f2e6c10b700 2024-05-27T09:38:35.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.077+0000 7f2e6b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e6c10c010 0x7f2e6c19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:35.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.077+0000 7f2e6b400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e6c10c010 0x7f2e6c19c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41540/0 (socket says 172.21.15.119:41540) 2024-05-27T09:38:35.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.077+0000 7f2e6b400700 1 -- 172.21.15.119:0/2820380351 learned_addr learned my addr 172.21.15.119:0/2820380351 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:38:35.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.077+0000 7f2e6be00700 1 --2- 172.21.15.119:0/2820380351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e6c10b700 0x7f2e6c19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:35.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.077+0000 7f2e6b400700 1 -- 172.21.15.119:0/2820380351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e6c10b700 msgr2=0x7f2e6c19c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:35.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.077+0000 7f2e6b400700 1 --2- 172.21.15.119:0/2820380351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e6c10b700 0x7f2e6c19c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:35.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.077+0000 7f2e6b400700 1 -- 172.21.15.119:0/2820380351 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5c0095f0 con 0x7f2e6c10c010 2024-05-27T09:38:35.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.077+0000 7f2e6be00700 1 --2- 172.21.15.119:0/2820380351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e6c10b700 0x7f2e6c19c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:38:35.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.078+0000 7f2e6b400700 1 --2- 172.21.15.119:0/2820380351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e6c10c010 0x7f2e6c19c990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2e5800b480 tx=0x7f2e5800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:35.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.078+0000 7f2e68c00700 1 -- 172.21.15.119:0/2820380351 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e580118c0 con 0x7f2e6c10c010 2024-05-27T09:38:35.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.078+0000 7f2e68c00700 1 -- 172.21.15.119:0/2820380351 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e58010380 con 0x7f2e6c10c010 2024-05-27T09:38:35.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.078+0000 7f2e73000700 1 -- 172.21.15.119:0/2820380351 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e6c1a1040 con 0x7f2e6c10c010 2024-05-27T09:38:35.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.078+0000 7f2e68c00700 1 -- 172.21.15.119:0/2820380351 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e5800f4f0 con 0x7f2e6c10c010 2024-05-27T09:38:35.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.078+0000 7f2e73000700 1 -- 172.21.15.119:0/2820380351 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e6c1a1610 con 0x7f2e6c10c010 2024-05-27T09:38:35.080 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.079+0000 7f2e73000700 1 -- 172.21.15.119:0/2820380351 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e6c04ea50 con 0x7f2e6c10c010 2024-05-27T09:38:35.081 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.081+0000 7f2e68c00700 1 -- 172.21.15.119:0/2820380351 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f2e580104f0 con 0x7f2e6c10c010 2024-05-27T09:38:35.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.081+0000 7f2e68c00700 1 --2- 172.21.15.119:0/2820380351 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2e50078a40 0x7f2e5007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:35.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.082+0000 7f2e68c00700 1 -- 172.21.15.119:0/2820380351 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2e5809a980 con 0x7f2e6c10c010 2024-05-27T09:38:35.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.082+0000 7f2e6be00700 1 --2- 172.21.15.119:0/2820380351 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2e50078a40 0x7f2e5007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:35.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.083+0000 7f2e6be00700 1 --2- 172.21.15.119:0/2820380351 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2e50078a40 0x7f2e5007aef0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c00b5c0 tx=0x7f2e5c005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:35.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.087+0000 7f2e68c00700 1 -- 172.21.15.119:0/2820380351 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f2e58062560 con 0x7f2e6c10c010 2024-05-27T09:38:35.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.318+0000 7f2e73000700 1 -- 172.21.15.119:0/2820380351 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e6c1a1b80 con 0x7f2e50078a40 2024-05-27T09:38:35.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.331+0000 7f2e68c00700 1 -- 172.21.15.119:0/2820380351 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f2e6c1a1b80 con 0x7f2e50078a40 2024-05-27T09:38:35.332 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:38:35.332 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (6m) 46s ago 19m 14.3M - 0.25.0 c8568f914cd2 6b90564dcef7 2024-05-27T09:38:35.332 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (20m) 46s ago 20m 8850k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:38:35.332 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (18m) 16s ago 18m 8350k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:38:35.332 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (2m) 46s ago 19m 6669k - 19.0.0-3868-ga6507388 850a06a84d28 011d811ce7fc 2024-05-27T09:38:35.332 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (2m) 16s ago 18m 6665k - 19.0.0-3868-ga6507388 850a06a84d28 b546b3ed1afc 2024-05-27T09:38:35.333 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (5m) 46s ago 19m 74.4M - 9.4.12 2bacad6d85d8 b3abdab08db4 2024-05-27T09:38:35.333 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (12m) 46s ago 12m 18.0M - 18.2.0 dc2bc1663786 93b21713fb1e 2024-05-27T09:38:35.333 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (12m) 46s ago 12m 130M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:38:35.333 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (12m) 16s ago 12m 17.9M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:38:35.333 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (12m) 16s ago 12m 128M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:38:35.333 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (8m) 46s ago 21m 587M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:38:35.333 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (7m) 16s ago 18m 533M - 19.0.0-3868-ga6507388 850a06a84d28 2fc3a80696fa 2024-05-27T09:38:35.333 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (4m) 46s ago 21m 114M 2048M 19.0.0-3868-ga6507388 850a06a84d28 3e5c7a9ad372 2024-05-27T09:38:35.333 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (3m) 16s ago 18m 105M 2048M 19.0.0-3868-ga6507388 850a06a84d28 e4d471274cf1 2024-05-27T09:38:35.333 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (7m) 46s ago 19m 8002k - 1.5.0 0da6a335fe13 709c37815cff 2024-05-27T09:38:35.333 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (6m) 16s ago 18m 7520k - 1.5.0 0da6a335fe13 999047a1a5bf 2024-05-27T09:38:35.333 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (117s) 46s ago 17m 152M 1216M 19.0.0-3868-ga6507388 850a06a84d28 5a0e162d870e 2024-05-27T09:38:35.334 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (85s) 46s ago 16m 167M 1216M 19.0.0-3868-ga6507388 850a06a84d28 effcc7c53dd3 2024-05-27T09:38:35.334 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (54s) 46s ago 16m 78.5M 1216M 19.0.0-3868-ga6507388 850a06a84d28 f0bd34ca416f 2024-05-27T09:38:35.334 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (21s) 16s ago 15m 79.8M 2240M 19.0.0-3868-ga6507388 850a06a84d28 5e8d3e398c21 2024-05-27T09:38:35.334 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (14m) 16s ago 14m 247M 2240M 18.2.0 dc2bc1663786 7559e658e917 2024-05-27T09:38:35.334 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (14m) 16s ago 14m 227M 2240M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:38:35.334 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (6m) 46s ago 19m 71.1M - 2.43.0 a07b618ecd1d cba50454ed21 2024-05-27T09:38:35.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.334+0000 7f2e73000700 1 -- 172.21.15.119:0/2820380351 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2e50078a40 msgr2=0x7f2e5007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:35.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.334+0000 7f2e73000700 1 --2- 172.21.15.119:0/2820380351 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2e50078a40 0x7f2e5007aef0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c00b5c0 tx=0x7f2e5c005960 comp rx=0 tx=0).stop 2024-05-27T09:38:35.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.334+0000 7f2e73000700 1 -- 172.21.15.119:0/2820380351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e6c10c010 msgr2=0x7f2e6c19c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:35.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.334+0000 7f2e73000700 1 --2- 172.21.15.119:0/2820380351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e6c10c010 0x7f2e6c19c990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2e5800b480 tx=0x7f2e5800b840 comp rx=0 tx=0).stop 2024-05-27T09:38:35.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.334+0000 7f2e73000700 1 -- 172.21.15.119:0/2820380351 shutdown_connections 2024-05-27T09:38:35.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.334+0000 7f2e73000700 1 --2- 172.21.15.119:0/2820380351 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2e50078a40 0x7f2e5007aef0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:35.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.334+0000 7f2e73000700 1 --2- 172.21.15.119:0/2820380351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e6c10b700 0x7f2e6c19c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:35.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.334+0000 7f2e73000700 1 --2- 172.21.15.119:0/2820380351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e6c10c010 0x7f2e6c19c990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:35.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.334+0000 7f2e73000700 1 -- 172.21.15.119:0/2820380351 >> 172.21.15.119:0/2820380351 conn(0x7f2e6c076000 msgr2=0x7f2e6c076a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:35.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.334+0000 7f2e73000700 1 -- 172.21.15.119:0/2820380351 shutdown_connections 2024-05-27T09:38:35.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.334+0000 7f2e73000700 1 -- 172.21.15.119:0/2820380351 wait complete. 2024-05-27T09:38:35.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.491+0000 7f39d7000700 1 -- 172.21.15.119:0/472097826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39d0106b80 msgr2=0x7f39d0106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:35.492 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.491+0000 7f39d7000700 1 --2- 172.21.15.119:0/472097826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39d0106b80 0x7f39d0106ff0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f39bc009910 tx=0x7f39bc009c20 comp rx=0 tx=0).stop 2024-05-27T09:38:35.493 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:35 smithi119 ceph-mon[88269]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:38:35.493 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:35 smithi119 ceph-mon[88269]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:38:35.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.492+0000 7f39d7000700 1 -- 172.21.15.119:0/472097826 shutdown_connections 2024-05-27T09:38:35.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.492+0000 7f39d7000700 1 --2- 172.21.15.119:0/472097826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39d0106b80 0x7f39d0106ff0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:35.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.492+0000 7f39d7000700 1 --2- 172.21.15.119:0/472097826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39d010b610 0x7f39d010b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:35.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.492+0000 7f39d7000700 1 -- 172.21.15.119:0/472097826 >> 172.21.15.119:0/472097826 conn(0x7f39d0075df0 msgr2=0x7f39d0078200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:35.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.492+0000 7f39d7000700 1 -- 172.21.15.119:0/472097826 shutdown_connections 2024-05-27T09:38:35.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.492+0000 7f39d7000700 1 -- 172.21.15.119:0/472097826 wait complete. 2024-05-27T09:38:35.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.493+0000 7f39d7000700 1 Processor -- start 2024-05-27T09:38:35.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.493+0000 7f39d7000700 1 -- start start 2024-05-27T09:38:35.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.494+0000 7f39d7000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39d0106b80 0x7f39d019c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:35.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.494+0000 7f39d7000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39d010b610 0x7f39d019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:35.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.494+0000 7f39d7000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d019cd10 con 0x7f39d0106b80 2024-05-27T09:38:35.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.494+0000 7f39d7000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d01a0aa0 con 0x7f39d010b610 2024-05-27T09:38:35.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.494+0000 7f39d5c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39d0106b80 0x7f39d019c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:35.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.494+0000 7f39d5c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39d0106b80 0x7f39d019c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41558/0 (socket says 172.21.15.119:41558) 2024-05-27T09:38:35.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.494+0000 7f39d5c00700 1 -- 172.21.15.119:0/2409169213 learned_addr learned my addr 172.21.15.119:0/2409169213 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:38:35.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.494+0000 7f39d5200700 1 --2- 172.21.15.119:0/2409169213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39d010b610 0x7f39d019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:35.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.494+0000 7f39d5c00700 1 -- 172.21.15.119:0/2409169213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39d010b610 msgr2=0x7f39d019c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:35.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.494+0000 7f39d5c00700 1 --2- 172.21.15.119:0/2409169213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39d010b610 0x7f39d019c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:35.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.495+0000 7f39d5c00700 1 -- 172.21.15.119:0/2409169213 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc0095f0 con 0x7f39d0106b80 2024-05-27T09:38:35.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.495+0000 7f39d5c00700 1 --2- 172.21.15.119:0/2409169213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39d0106b80 0x7f39d019c0f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f39c400c8a0 tx=0x7f39c400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:35.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.495+0000 7f39ce000700 1 -- 172.21.15.119:0/2409169213 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39c400cea0 con 0x7f39d0106b80 2024-05-27T09:38:35.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.495+0000 7f39ce000700 1 -- 172.21.15.119:0/2409169213 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39c4004830 con 0x7f39d0106b80 2024-05-27T09:38:35.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.495+0000 7f39ce000700 1 -- 172.21.15.119:0/2409169213 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39c40055b0 con 0x7f39d0106b80 2024-05-27T09:38:35.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.495+0000 7f39d7000700 1 -- 172.21.15.119:0/2409169213 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d01a0d80 con 0x7f39d0106b80 2024-05-27T09:38:35.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.496+0000 7f39d7000700 1 -- 172.21.15.119:0/2409169213 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d01a1350 con 0x7f39d0106b80 2024-05-27T09:38:35.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.497+0000 7f39d7000700 1 -- 172.21.15.119:0/2409169213 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d004f2a0 con 0x7f39d0106b80 2024-05-27T09:38:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.498+0000 7f39ce000700 1 -- 172.21.15.119:0/2409169213 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f39c40049a0 con 0x7f39d0106b80 2024-05-27T09:38:35.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.499+0000 7f39ce000700 1 --2- 172.21.15.119:0/2409169213 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f39b8078a40 0x7f39b807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:35.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.499+0000 7f39ce000700 1 -- 172.21.15.119:0/2409169213 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f39c409a940 con 0x7f39d0106b80 2024-05-27T09:38:35.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.499+0000 7f39d5200700 1 --2- 172.21.15.119:0/2409169213 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f39b8078a40 0x7f39b807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:35.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.500+0000 7f39d5200700 1 --2- 172.21.15.119:0/2409169213 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f39b8078a40 0x7f39b807aef0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f39d019d710 tx=0x7f39bc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:35.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.504+0000 7f39ce000700 1 -- 172.21.15.119:0/2409169213 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f39c40625a0 con 0x7f39d0106b80 2024-05-27T09:38:35.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:35 smithi138 ceph-mon[70004]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:38:35.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:35 smithi138 ceph-mon[70004]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:38:35.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.829+0000 7f39d7000700 1 -- 172.21.15.119:0/2409169213 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f39d01a1890 con 0x7f39d0106b80 2024-05-27T09:38:35.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.830+0000 7f39ce000700 1 -- 172.21.15.119:0/2409169213 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f39c4004c50 con 0x7f39d0106b80 2024-05-27T09:38:35.831 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:38:35.831 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:38:35.831 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:38:35.831 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:38:35.832 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:38:35.832 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:38:35.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:38:35.832 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:38:35.832 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2, 2024-05-27T09:38:35.832 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 4 2024-05-27T09:38:35.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:38:35.832 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:38:35.832 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-05-27T09:38:35.832 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:38:35.832 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:38:35.832 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6, 2024-05-27T09:38:35.832 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 8 2024-05-27T09:38:35.832 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:38:35.833 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:38:35.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.834+0000 7f39d7000700 1 -- 172.21.15.119:0/2409169213 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f39b8078a40 msgr2=0x7f39b807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:35.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.834+0000 7f39d7000700 1 --2- 172.21.15.119:0/2409169213 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f39b8078a40 0x7f39b807aef0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f39d019d710 tx=0x7f39bc01a040 comp rx=0 tx=0).stop 2024-05-27T09:38:35.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.834+0000 7f39d7000700 1 -- 172.21.15.119:0/2409169213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39d0106b80 msgr2=0x7f39d019c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:35.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.834+0000 7f39d7000700 1 --2- 172.21.15.119:0/2409169213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39d0106b80 0x7f39d019c0f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f39c400c8a0 tx=0x7f39c400cc60 comp rx=0 tx=0).stop 2024-05-27T09:38:35.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.834+0000 7f39d7000700 1 -- 172.21.15.119:0/2409169213 shutdown_connections 2024-05-27T09:38:35.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.834+0000 7f39d7000700 1 --2- 172.21.15.119:0/2409169213 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f39b8078a40 0x7f39b807aef0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:35.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.834+0000 7f39d7000700 1 --2- 172.21.15.119:0/2409169213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39d0106b80 0x7f39d019c0f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:35.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.834+0000 7f39d7000700 1 --2- 172.21.15.119:0/2409169213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39d010b610 0x7f39d019c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:35.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.834+0000 7f39d7000700 1 -- 172.21.15.119:0/2409169213 >> 172.21.15.119:0/2409169213 conn(0x7f39d0075df0 msgr2=0x7f39d00776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:35.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.834+0000 7f39d7000700 1 -- 172.21.15.119:0/2409169213 shutdown_connections 2024-05-27T09:38:35.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.835+0000 7f39d7000700 1 -- 172.21.15.119:0/2409169213 wait complete. 2024-05-27T09:38:35.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.979+0000 7fb183c00700 1 -- 172.21.15.119:0/3733207892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb17c10b700 msgr2=0x7fb17c10bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:35.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.979+0000 7fb183c00700 1 --2- 172.21.15.119:0/3733207892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb17c10b700 0x7fb17c10bad0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00b0a0 tx=0x7fb16c00b3b0 comp rx=0 tx=0).stop 2024-05-27T09:38:35.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.979+0000 7fb183c00700 1 -- 172.21.15.119:0/3733207892 shutdown_connections 2024-05-27T09:38:35.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.979+0000 7fb183c00700 1 --2- 172.21.15.119:0/3733207892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb17c10c010 0x7fb17c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:35.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.979+0000 7fb183c00700 1 --2- 172.21.15.119:0/3733207892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb17c10b700 0x7fb17c10bad0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:35.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.979+0000 7fb183c00700 1 -- 172.21.15.119:0/3733207892 >> 172.21.15.119:0/3733207892 conn(0x7fb17c076000 msgr2=0x7fb17c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:35.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.979+0000 7fb183c00700 1 -- 172.21.15.119:0/3733207892 shutdown_connections 2024-05-27T09:38:35.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.979+0000 7fb183c00700 1 -- 172.21.15.119:0/3733207892 wait complete. 2024-05-27T09:38:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.980+0000 7fb183c00700 1 Processor -- start 2024-05-27T09:38:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.980+0000 7fb183c00700 1 -- start start 2024-05-27T09:38:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.980+0000 7fb183c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb17c10b700 0x7fb17c19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.980+0000 7fb183c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb17c10c010 0x7fb17c19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.981+0000 7fb183c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c19ced0 con 0x7fb17c10b700 2024-05-27T09:38:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.981+0000 7fb183c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c1a0c60 con 0x7fb17c10c010 2024-05-27T09:38:35.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.981+0000 7fb181200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb17c10b700 0x7fb17c19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:35.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.981+0000 7fb181200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb17c10b700 0x7fb17c19c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41576/0 (socket says 172.21.15.119:41576) 2024-05-27T09:38:35.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.981+0000 7fb181200700 1 -- 172.21.15.119:0/3268618028 learned_addr learned my addr 172.21.15.119:0/3268618028 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:38:35.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.981+0000 7fb181200700 1 -- 172.21.15.119:0/3268618028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb17c10c010 msgr2=0x7fb17c19c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:35.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.981+0000 7fb173e00700 1 --2- 172.21.15.119:0/3268618028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb17c10c010 0x7fb17c19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:35.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.981+0000 7fb181200700 1 --2- 172.21.15.119:0/3268618028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb17c10c010 0x7fb17c19c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:35.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.981+0000 7fb181200700 1 -- 172.21.15.119:0/3268618028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb16c009d00 con 0x7fb17c10b700 2024-05-27T09:38:35.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.981+0000 7fb181200700 1 --2- 172.21.15.119:0/3268618028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb17c10b700 0x7fb17c19c2b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb16c000f80 tx=0x7fb16c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:35.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.982+0000 7fb17a000700 1 -- 172.21.15.119:0/3268618028 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb16c00e040 con 0x7fb17c10b700 2024-05-27T09:38:35.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.982+0000 7fb17a000700 1 -- 172.21.15.119:0/3268618028 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb16c007c00 con 0x7fb17c10b700 2024-05-27T09:38:35.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.982+0000 7fb17a000700 1 -- 172.21.15.119:0/3268618028 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb16c01d740 con 0x7fb17c10b700 2024-05-27T09:38:35.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.982+0000 7fb183c00700 1 -- 172.21.15.119:0/3268618028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb17c1a0ee0 con 0x7fb17c10b700 2024-05-27T09:38:35.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.982+0000 7fb183c00700 1 -- 172.21.15.119:0/3268618028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17c1a1450 con 0x7fb17c10b700 2024-05-27T09:38:35.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.983+0000 7fb183c00700 1 -- 172.21.15.119:0/3268618028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb17c04ea50 con 0x7fb17c10b700 2024-05-27T09:38:35.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.984+0000 7fb17a000700 1 -- 172.21.15.119:0/3268618028 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fb16c004040 con 0x7fb17c10b700 2024-05-27T09:38:35.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.985+0000 7fb17a000700 1 --2- 172.21.15.119:0/3268618028 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb160078a40 0x7fb16007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:35.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.985+0000 7fb17a000700 1 -- 172.21.15.119:0/3268618028 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb16c09c5a0 con 0x7fb17c10b700 2024-05-27T09:38:35.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.986+0000 7fb173e00700 1 --2- 172.21.15.119:0/3268618028 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb160078a40 0x7fb16007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:35.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.986+0000 7fb173e00700 1 --2- 172.21.15.119:0/3268618028 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb160078a40 0x7fb16007aef0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb17c19d8d0 tx=0x7fb164005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:35.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:35.990+0000 7fb17a000700 1 -- 172.21.15.119:0/3268618028 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fb16c064180 con 0x7fb17c10b700 2024-05-27T09:38:36.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.264+0000 7fb183c00700 1 -- 172.21.15.119:0/3268618028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb17c1a18c0 con 0x7fb17c10b700 2024-05-27T09:38:36.266 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.265+0000 7fb17a000700 1 -- 172.21.15.119:0/3268618028 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1889 (secure 0 0 0) 0x7fb16c063b20 con 0x7fb17c10b700 2024-05-27T09:38:36.266 INFO:teuthology.orchestra.run.smithi119.stdout:e22 2024-05-27T09:38:36.267 INFO:teuthology.orchestra.run.smithi119.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T09:38:36.267 INFO:teuthology.orchestra.run.smithi119.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:38:36.267 INFO:teuthology.orchestra.run.smithi119.stdout:legacy client fscid: 1 2024-05-27T09:38:36.267 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:38:36.267 INFO:teuthology.orchestra.run.smithi119.stdout:Filesystem 'cephfs' (1) 2024-05-27T09:38:36.267 INFO:teuthology.orchestra.run.smithi119.stdout:fs_name cephfs 2024-05-27T09:38:36.267 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 22 2024-05-27T09:38:36.267 INFO:teuthology.orchestra.run.smithi119.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-27T09:38:36.268 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-05-27T09:25:53.387388+0000 2024-05-27T09:38:36.268 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2024-05-27T09:34:01.250304+0000 2024-05-27T09:38:36.268 INFO:teuthology.orchestra.run.smithi119.stdout:tableserver 0 2024-05-27T09:38:36.268 INFO:teuthology.orchestra.run.smithi119.stdout:root 0 2024-05-27T09:38:36.268 INFO:teuthology.orchestra.run.smithi119.stdout:session_timeout 60 2024-05-27T09:38:36.268 INFO:teuthology.orchestra.run.smithi119.stdout:session_autoclose 300 2024-05-27T09:38:36.268 INFO:teuthology.orchestra.run.smithi119.stdout:max_file_size 1099511627776 2024-05-27T09:38:36.268 INFO:teuthology.orchestra.run.smithi119.stdout:max_xattr_size 65536 2024-05-27T09:38:36.268 INFO:teuthology.orchestra.run.smithi119.stdout:required_client_features {} 2024-05-27T09:38:36.268 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure 0 2024-05-27T09:38:36.268 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure_osd_epoch 46 2024-05-27T09:38:36.269 INFO:teuthology.orchestra.run.smithi119.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:38:36.269 INFO:teuthology.orchestra.run.smithi119.stdout:max_mds 2 2024-05-27T09:38:36.269 INFO:teuthology.orchestra.run.smithi119.stdout:in 0,1 2024-05-27T09:38:36.269 INFO:teuthology.orchestra.run.smithi119.stdout:up {0=14472,1=24241} 2024-05-27T09:38:36.269 INFO:teuthology.orchestra.run.smithi119.stdout:failed 2024-05-27T09:38:36.269 INFO:teuthology.orchestra.run.smithi119.stdout:damaged 2024-05-27T09:38:36.269 INFO:teuthology.orchestra.run.smithi119.stdout:stopped 2024-05-27T09:38:36.269 INFO:teuthology.orchestra.run.smithi119.stdout:data_pools [3] 2024-05-27T09:38:36.269 INFO:teuthology.orchestra.run.smithi119.stdout:metadata_pool 2 2024-05-27T09:38:36.269 INFO:teuthology.orchestra.run.smithi119.stdout:inline_data enabled 2024-05-27T09:38:36.269 INFO:teuthology.orchestra.run.smithi119.stdout:balancer 2024-05-27T09:38:36.269 INFO:teuthology.orchestra.run.smithi119.stdout:bal_rank_mask -1 2024-05-27T09:38:36.270 INFO:teuthology.orchestra.run.smithi119.stdout:standby_count_wanted 1 2024-05-27T09:38:36.270 INFO:teuthology.orchestra.run.smithi119.stdout:qdb_cluster leader: 0 members: 2024-05-27T09:38:36.270 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.ppmzzk{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:38:36.270 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.ndfkhd{1:24241} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/897058594,v1:172.21.15.138:6825/897058594] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:38:36.270 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:38:36.270 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:38:36.270 INFO:teuthology.orchestra.run.smithi119.stdout:Standby daemons: 2024-05-27T09:38:36.270 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:38:36.270 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.nghcey{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.119:6826/413736191,v1:172.21.15.119:6827/413736191] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:38:36.270 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.alszpk{-1:24253} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/2345607053,v1:172.21.15.138:6827/2345607053] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:38:36.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.269+0000 7fb183c00700 1 -- 172.21.15.119:0/3268618028 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb160078a40 msgr2=0x7fb16007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:36.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.269+0000 7fb183c00700 1 --2- 172.21.15.119:0/3268618028 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb160078a40 0x7fb16007aef0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb17c19d8d0 tx=0x7fb164005cf0 comp rx=0 tx=0).stop 2024-05-27T09:38:36.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.269+0000 7fb183c00700 1 -- 172.21.15.119:0/3268618028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb17c10b700 msgr2=0x7fb17c19c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:36.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.269+0000 7fb183c00700 1 --2- 172.21.15.119:0/3268618028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb17c10b700 0x7fb17c19c2b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb16c000f80 tx=0x7fb16c003ce0 comp rx=0 tx=0).stop 2024-05-27T09:38:36.271 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.269+0000 7fb183c00700 1 -- 172.21.15.119:0/3268618028 shutdown_connections 2024-05-27T09:38:36.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.269+0000 7fb183c00700 1 --2- 172.21.15.119:0/3268618028 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb160078a40 0x7fb16007aef0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:36.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.269+0000 7fb183c00700 1 --2- 172.21.15.119:0/3268618028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb17c10b700 0x7fb17c19c2b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:36.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.269+0000 7fb183c00700 1 --2- 172.21.15.119:0/3268618028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb17c10c010 0x7fb17c19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:36.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.269+0000 7fb183c00700 1 -- 172.21.15.119:0/3268618028 >> 172.21.15.119:0/3268618028 conn(0x7fb17c076000 msgr2=0x7fb17c076880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:36.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.269+0000 7fb183c00700 1 -- 172.21.15.119:0/3268618028 shutdown_connections 2024-05-27T09:38:36.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.269+0000 7fb183c00700 1 -- 172.21.15.119:0/3268618028 wait complete. 2024-05-27T09:38:36.272 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 22 2024-05-27T09:38:36.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.422+0000 7fe964200700 1 -- 172.21.15.119:0/4208718218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe95c102fa0 msgr2=0x7fe95c106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:36.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.422+0000 7fe964200700 1 --2- 172.21.15.119:0/4208718218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe95c102fa0 0x7fe95c106f70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe948009910 tx=0x7fe948009c20 comp rx=0 tx=0).stop 2024-05-27T09:38:36.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.423+0000 7fe964200700 1 -- 172.21.15.119:0/4208718218 shutdown_connections 2024-05-27T09:38:36.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.423+0000 7fe964200700 1 --2- 172.21.15.119:0/4208718218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe95c102fa0 0x7fe95c106f70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:36.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.423+0000 7fe964200700 1 --2- 172.21.15.119:0/4208718218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe95c102600 0x7fe95c1029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:36.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.423+0000 7fe964200700 1 -- 172.21.15.119:0/4208718218 >> 172.21.15.119:0/4208718218 conn(0x7fe95c0fde60 msgr2=0x7fe95c100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:36.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.423+0000 7fe964200700 1 -- 172.21.15.119:0/4208718218 shutdown_connections 2024-05-27T09:38:36.424 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.423+0000 7fe964200700 1 -- 172.21.15.119:0/4208718218 wait complete. 2024-05-27T09:38:36.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.424+0000 7fe964200700 1 Processor -- start 2024-05-27T09:38:36.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.424+0000 7fe964200700 1 -- start start 2024-05-27T09:38:36.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.425+0000 7fe964200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe95c102600 0x7fe95c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:36.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.425+0000 7fe964200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe95c102fa0 0x7fe95c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:36.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.425+0000 7fe964200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c198cf0 con 0x7fe95c102fa0 2024-05-27T09:38:36.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.425+0000 7fe964200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c19ca80 con 0x7fe95c102600 2024-05-27T09:38:36.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.425+0000 7fe960e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe95c102fa0 0x7fe95c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:36.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.425+0000 7fe960e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe95c102fa0 0x7fe95c198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41594/0 (socket says 172.21.15.119:41594) 2024-05-27T09:38:36.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.425+0000 7fe960e00700 1 -- 172.21.15.119:0/2351824279 learned_addr learned my addr 172.21.15.119:0/2351824279 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:38:36.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.426+0000 7fe961800700 1 --2- 172.21.15.119:0/2351824279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe95c102600 0x7fe95c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:36.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.426+0000 7fe960e00700 1 -- 172.21.15.119:0/2351824279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe95c102600 msgr2=0x7fe95c1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:36.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.426+0000 7fe960e00700 1 --2- 172.21.15.119:0/2351824279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe95c102600 0x7fe95c1980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:36.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.426+0000 7fe960e00700 1 -- 172.21.15.119:0/2351824279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9480095f0 con 0x7fe95c102fa0 2024-05-27T09:38:36.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.426+0000 7fe960e00700 1 --2- 172.21.15.119:0/2351824279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe95c102fa0 0x7fe95c198610 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe94800bda0 tx=0x7fe94800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:36.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.426+0000 7fe95a000700 1 -- 172.21.15.119:0/2351824279 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe94801d070 con 0x7fe95c102fa0 2024-05-27T09:38:36.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.426+0000 7fe95a000700 1 -- 172.21.15.119:0/2351824279 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe948004fb0 con 0x7fe95c102fa0 2024-05-27T09:38:36.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.426+0000 7fe95a000700 1 -- 172.21.15.119:0/2351824279 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe94800f480 con 0x7fe95c102fa0 2024-05-27T09:38:36.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.426+0000 7fe964200700 1 -- 172.21.15.119:0/2351824279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95c19cd00 con 0x7fe95c102fa0 2024-05-27T09:38:36.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.427+0000 7fe964200700 1 -- 172.21.15.119:0/2351824279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95c19d1f0 con 0x7fe95c102fa0 2024-05-27T09:38:36.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.428+0000 7fe964200700 1 -- 172.21.15.119:0/2351824279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe95c10a8c0 con 0x7fe95c102fa0 2024-05-27T09:38:36.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.429+0000 7fe95a000700 1 -- 172.21.15.119:0/2351824279 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fe948022870 con 0x7fe95c102fa0 2024-05-27T09:38:36.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.430+0000 7fe95a000700 1 --2- 172.21.15.119:0/2351824279 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe940078a40 0x7fe94007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:36.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.430+0000 7fe95a000700 1 -- 172.21.15.119:0/2351824279 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe94809c130 con 0x7fe95c102fa0 2024-05-27T09:38:36.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.430+0000 7fe961800700 1 --2- 172.21.15.119:0/2351824279 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe940078a40 0x7fe94007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:36.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.431+0000 7fe961800700 1 --2- 172.21.15.119:0/2351824279 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe940078a40 0x7fe94007aef0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009f20 tx=0x7fe94c0094f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:36.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.435+0000 7fe95a000700 1 -- 172.21.15.119:0/2351824279 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fe948063ee0 con 0x7fe95c102fa0 2024-05-27T09:38:36.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:36 smithi138 ceph-mon[70004]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:38:36.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:36 smithi138 ceph-mon[70004]: pgmap v147: 65 pgs: 65 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:36.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:36 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/2409169213' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:38:36.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:36 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/3268618028' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:38:36.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.671+0000 7fe964200700 1 -- 172.21.15.119:0/2351824279 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe95c02cd10 con 0x7fe940078a40 2024-05-27T09:38:36.673 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:36 smithi119 ceph-mon[88269]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:38:36.673 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:36 smithi119 ceph-mon[88269]: pgmap v147: 65 pgs: 65 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:36.673 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:36 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/2409169213' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:38:36.673 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:36 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/3268618028' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:38:36.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.675+0000 7fe95a000700 1 -- 172.21.15.119:0/2351824279 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fe95c02cd10 con 0x7fe940078a40 2024-05-27T09:38:36.676 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:38:36.676 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:38:36.676 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:38:36.676 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-27T09:38:36.676 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [ 2024-05-27T09:38:36.676 INFO:teuthology.orchestra.run.smithi119.stdout: "crash", 2024-05-27T09:38:36.676 INFO:teuthology.orchestra.run.smithi119.stdout: "mon", 2024-05-27T09:38:36.676 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr" 2024-05-27T09:38:36.676 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-05-27T09:38:36.676 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "10/23 daemons upgraded", 2024-05-27T09:38:36.676 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "Currently upgrading osd daemons", 2024-05-27T09:38:36.676 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:38:36.676 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:38:36.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.678+0000 7fe964200700 1 -- 172.21.15.119:0/2351824279 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe940078a40 msgr2=0x7fe94007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:36.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.678+0000 7fe964200700 1 --2- 172.21.15.119:0/2351824279 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe940078a40 0x7fe94007aef0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009f20 tx=0x7fe94c0094f0 comp rx=0 tx=0).stop 2024-05-27T09:38:36.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.678+0000 7fe964200700 1 -- 172.21.15.119:0/2351824279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe95c102fa0 msgr2=0x7fe95c198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:36.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.678+0000 7fe964200700 1 --2- 172.21.15.119:0/2351824279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe95c102fa0 0x7fe95c198610 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe94800bda0 tx=0x7fe94800be80 comp rx=0 tx=0).stop 2024-05-27T09:38:36.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.678+0000 7fe964200700 1 -- 172.21.15.119:0/2351824279 shutdown_connections 2024-05-27T09:38:36.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.678+0000 7fe964200700 1 --2- 172.21.15.119:0/2351824279 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe940078a40 0x7fe94007aef0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:36.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.678+0000 7fe964200700 1 --2- 172.21.15.119:0/2351824279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe95c102600 0x7fe95c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:36.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.678+0000 7fe964200700 1 --2- 172.21.15.119:0/2351824279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe95c102fa0 0x7fe95c198610 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:36.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.678+0000 7fe964200700 1 -- 172.21.15.119:0/2351824279 >> 172.21.15.119:0/2351824279 conn(0x7fe95c0fde60 msgr2=0x7fe95c0ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:36.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.678+0000 7fe964200700 1 -- 172.21.15.119:0/2351824279 shutdown_connections 2024-05-27T09:38:36.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.678+0000 7fe964200700 1 -- 172.21.15.119:0/2351824279 wait complete. 2024-05-27T09:38:36.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.833+0000 7f9f2dc00700 1 -- 172.21.15.119:0/3051918657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f2810b700 msgr2=0x7f9f2810bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:36.834 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.833+0000 7f9f2dc00700 1 --2- 172.21.15.119:0/3051918657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f2810b700 0x7f9f2810bad0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800b0a0 tx=0x7f9f1800b3b0 comp rx=0 tx=0).stop 2024-05-27T09:38:36.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.833+0000 7f9f2dc00700 1 -- 172.21.15.119:0/3051918657 shutdown_connections 2024-05-27T09:38:36.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.833+0000 7f9f2dc00700 1 --2- 172.21.15.119:0/3051918657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f2810c010 0x7f9f28106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:36.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.833+0000 7f9f2dc00700 1 --2- 172.21.15.119:0/3051918657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f2810b700 0x7f9f2810bad0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:36.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.833+0000 7f9f2dc00700 1 -- 172.21.15.119:0/3051918657 >> 172.21.15.119:0/3051918657 conn(0x7f9f28076000 msgr2=0x7f9f28078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:36.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.834+0000 7f9f2dc00700 1 -- 172.21.15.119:0/3051918657 shutdown_connections 2024-05-27T09:38:36.835 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.834+0000 7f9f2dc00700 1 -- 172.21.15.119:0/3051918657 wait complete. 2024-05-27T09:38:36.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.834+0000 7f9f2dc00700 1 Processor -- start 2024-05-27T09:38:36.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.835+0000 7f9f2dc00700 1 -- start start 2024-05-27T09:38:36.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.835+0000 7f9f2dc00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f2810b700 0x7f9f2819c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:36.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.835+0000 7f9f2dc00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f2810c010 0x7f9f2819c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:36.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.835+0000 7f9f2dc00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2819cf50 con 0x7f9f2810c010 2024-05-27T09:38:36.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.835+0000 7f9f2dc00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f281a0ce0 con 0x7f9f2810b700 2024-05-27T09:38:36.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.835+0000 7f9f26000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f2810c010 0x7f9f2819c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:36.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.835+0000 7f9f26000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f2810c010 0x7f9f2819c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41616/0 (socket says 172.21.15.119:41616) 2024-05-27T09:38:36.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.835+0000 7f9f26000700 1 -- 172.21.15.119:0/3391875000 learned_addr learned my addr 172.21.15.119:0/3391875000 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:38:36.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.836+0000 7f9f26a00700 1 --2- 172.21.15.119:0/3391875000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f2810b700 0x7f9f2819c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:36.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.836+0000 7f9f26000700 1 -- 172.21.15.119:0/3391875000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f2810b700 msgr2=0x7f9f2819c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:36.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.836+0000 7f9f26000700 1 --2- 172.21.15.119:0/3391875000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f2810b700 0x7f9f2819c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:36.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.836+0000 7f9f26000700 1 -- 172.21.15.119:0/3391875000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f18009d00 con 0x7f9f2810c010 2024-05-27T09:38:36.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.836+0000 7f9f26000700 1 --2- 172.21.15.119:0/3391875000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f2810c010 0x7f9f2819c870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400c8f0 tx=0x7f9f1400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:36.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.836+0000 7f9f1f400700 1 -- 172.21.15.119:0/3391875000 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f14007a50 con 0x7f9f2810c010 2024-05-27T09:38:36.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.836+0000 7f9f1f400700 1 -- 172.21.15.119:0/3391875000 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f14007bb0 con 0x7f9f2810c010 2024-05-27T09:38:36.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.836+0000 7f9f1f400700 1 -- 172.21.15.119:0/3391875000 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f14018630 con 0x7f9f2810c010 2024-05-27T09:38:36.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.836+0000 7f9f2dc00700 1 -- 172.21.15.119:0/3391875000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f281a0fc0 con 0x7f9f2810c010 2024-05-27T09:38:36.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.836+0000 7f9f2dc00700 1 -- 172.21.15.119:0/3391875000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f281a1590 con 0x7f9f2810c010 2024-05-27T09:38:36.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.838+0000 7f9f2dc00700 1 -- 172.21.15.119:0/3391875000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2804ea50 con 0x7f9f2810c010 2024-05-27T09:38:36.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.839+0000 7f9f1f400700 1 -- 172.21.15.119:0/3391875000 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f9f14007d20 con 0x7f9f2810c010 2024-05-27T09:38:36.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.840+0000 7f9f1f400700 1 --2- 172.21.15.119:0/3391875000 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9f0c078a40 0x7f9f0c07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:38:36.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.840+0000 7f9f1f400700 1 -- 172.21.15.119:0/3391875000 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f9f1409ac90 con 0x7f9f2810c010 2024-05-27T09:38:36.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.840+0000 7f9f26a00700 1 --2- 172.21.15.119:0/3391875000 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9f0c078a40 0x7f9f0c07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:38:36.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.841+0000 7f9f26a00700 1 --2- 172.21.15.119:0/3391875000 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9f0c078a40 0x7f9f0c07aef0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009480 tx=0x7f9f18000f40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:38:36.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:36.845+0000 7f9f1f400700 1 -- 172.21.15.119:0/3391875000 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f9f140628f0 con 0x7f9f2810c010 2024-05-27T09:38:37.172 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:37.171+0000 7f9f2dc00700 1 -- 172.21.15.119:0/3391875000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9f2819d690 con 0x7f9f2810c010 2024-05-27T09:38:37.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:37.172+0000 7f9f1f400700 1 -- 172.21.15.119:0/3391875000 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f9f14062290 con 0x7f9f2810c010 2024-05-27T09:38:37.173 INFO:teuthology.orchestra.run.smithi119.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-27T09:38:37.173 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:38:37.174 INFO:teuthology.orchestra.run.smithi119.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:38:37.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:37.175+0000 7f9f2dc00700 1 -- 172.21.15.119:0/3391875000 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9f0c078a40 msgr2=0x7f9f0c07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:37.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:37.175+0000 7f9f2dc00700 1 --2- 172.21.15.119:0/3391875000 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9f0c078a40 0x7f9f0c07aef0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009480 tx=0x7f9f18000f40 comp rx=0 tx=0).stop 2024-05-27T09:38:37.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:37.175+0000 7f9f2dc00700 1 -- 172.21.15.119:0/3391875000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f2810c010 msgr2=0x7f9f2819c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:38:37.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:37.175+0000 7f9f2dc00700 1 --2- 172.21.15.119:0/3391875000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f2810c010 0x7f9f2819c870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400c8f0 tx=0x7f9f1400ccb0 comp rx=0 tx=0).stop 2024-05-27T09:38:37.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:37.175+0000 7f9f2dc00700 1 -- 172.21.15.119:0/3391875000 shutdown_connections 2024-05-27T09:38:37.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:37.175+0000 7f9f2dc00700 1 --2- 172.21.15.119:0/3391875000 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9f0c078a40 0x7f9f0c07aef0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:37.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:37.175+0000 7f9f2dc00700 1 --2- 172.21.15.119:0/3391875000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f2810b700 0x7f9f2819c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:37.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:37.175+0000 7f9f2dc00700 1 --2- 172.21.15.119:0/3391875000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f2810c010 0x7f9f2819c870 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:38:37.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:37.175+0000 7f9f2dc00700 1 -- 172.21.15.119:0/3391875000 >> 172.21.15.119:0/3391875000 conn(0x7f9f28076000 msgr2=0x7f9f28076920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:38:37.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:37.175+0000 7f9f2dc00700 1 -- 172.21.15.119:0/3391875000 shutdown_connections 2024-05-27T09:38:37.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:38:37.175+0000 7f9f2dc00700 1 -- 172.21.15.119:0/3391875000 wait complete. 2024-05-27T09:38:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:37 smithi119 ceph-mon[88269]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:38:37.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:37 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/3391875000' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:38:37.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:37 smithi138 ceph-mon[70004]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:38:37.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:37 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/3391875000' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:38:38.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:38 smithi119 ceph-mon[88269]: pgmap v148: 65 pgs: 65 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:38.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:38 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:38.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:38 smithi138 ceph-mon[70004]: pgmap v148: 65 pgs: 65 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:38.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:38 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:40.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:40 smithi138 ceph-mon[70004]: pgmap v149: 65 pgs: 65 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:40.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:40 smithi119 ceph-mon[88269]: pgmap v149: 65 pgs: 65 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:42.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:42 smithi119 ceph-mon[88269]: pgmap v150: 65 pgs: 65 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:42.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:42 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-27T09:38:42.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:42 smithi119 ceph-mon[88269]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-27T09:38:42.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:42 smithi119 ceph-mon[88269]: Upgrade: osd.4 is safe to restart 2024-05-27T09:38:42.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:42 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:42.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:42 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:38:42.774 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:42 smithi138 ceph-mon[70004]: pgmap v150: 65 pgs: 65 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:42.775 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:42 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-27T09:38:42.775 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:42 smithi138 ceph-mon[70004]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-27T09:38:42.775 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:42 smithi138 ceph-mon[70004]: Upgrade: osd.4 is safe to restart 2024-05-27T09:38:42.775 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:42 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:42.775 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:42 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:38:45.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:44 smithi138 ceph-mon[70004]: pgmap v151: 65 pgs: 65 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:45.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:44 smithi138 ceph-mon[70004]: Upgrade: Updating osd.4 2024-05-27T09:38:45.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:44 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:45.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:44 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-27T09:38:45.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:44 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:38:45.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:44 smithi138 ceph-mon[70004]: Deploying daemon osd.4 on smithi138 2024-05-27T09:38:45.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:44 smithi119 ceph-mon[88269]: pgmap v151: 65 pgs: 65 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:45.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:44 smithi119 ceph-mon[88269]: Upgrade: Updating osd.4 2024-05-27T09:38:45.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:44 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:45.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:44 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-27T09:38:45.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:44 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:38:45.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:44 smithi119 ceph-mon[88269]: Deploying daemon osd.4 on smithi138 2024-05-27T09:38:47.108 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:46 smithi138 ceph-mon[70004]: pgmap v152: 65 pgs: 65 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:47.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:46 smithi119 ceph-mon[88269]: pgmap v152: 65 pgs: 65 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:49.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:48 smithi119 ceph-mon[88269]: pgmap v153: 65 pgs: 65 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:49.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:48 smithi138 ceph-mon[70004]: pgmap v153: 65 pgs: 65 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:49.387 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:48 smithi138 systemd[1]: Stopping Ceph osd.4 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:38:49.886 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:49 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[45566]: 2024-05-27T09:38:49.464+0000 7fe3a4000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T09:38:49.886 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:49 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[45566]: 2024-05-27T09:38:49.464+0000 7fe3a4000700 -1 osd.4 69 *** Got signal Terminated *** 2024-05-27T09:38:49.886 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:49 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[45566]: 2024-05-27T09:38:49.464+0000 7fe3a4000700 -1 osd.4 69 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T09:38:50.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:50 smithi138 ceph-mon[70004]: osd.4 marked itself down and dead 2024-05-27T09:38:50.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:50 smithi119 ceph-mon[88269]: osd.4 marked itself down and dead 2024-05-27T09:38:50.885 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:50 smithi138 podman[84042]: 2024-05-27 09:38:50.617837777 +0000 UTC m=+1.321596517 container died 7559e658e917b68f14644f62d8b9e9e262428003dc18d78d1174b6f592e4c152 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4, io.buildah.version=1.29.1, RELEASE=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, org.label-schema.license=GPLv2) 2024-05-27T09:38:51.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:51 smithi138 ceph-mon[70004]: pgmap v154: 65 pgs: 65 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:51.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:51 smithi138 ceph-mon[70004]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-27T09:38:51.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:51 smithi138 ceph-mon[70004]: osdmap e70: 6 total, 5 up, 6 in 2024-05-27T09:38:51.387 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:51 smithi138 podman[84042]: 2024-05-27 09:38:51.191966554 +0000 UTC m=+1.895725311 container cleanup 7559e658e917b68f14644f62d8b9e9e262428003dc18d78d1174b6f592e4c152 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, GIT_CLEAN=True, RELEASE=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20231212, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0) 2024-05-27T09:38:51.387 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:51 smithi138 bash[84042]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4 2024-05-27T09:38:51.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:51 smithi119 ceph-mon[88269]: pgmap v154: 65 pgs: 65 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:51.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:51 smithi119 ceph-mon[88269]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-27T09:38:51.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:51 smithi119 ceph-mon[88269]: osdmap e70: 6 total, 5 up, 6 in 2024-05-27T09:38:51.886 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:51 smithi138 podman[84059]: 2024-05-27 09:38:51.584895648 +0000 UTC m=+0.941430051 container remove 7559e658e917b68f14644f62d8b9e9e262428003dc18d78d1174b6f592e4c152 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.0, maintainer=Guillaume Abrioux , GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, org.label-schema.build-date=20231212) 2024-05-27T09:38:52.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:52 smithi138 ceph-mon[70004]: osdmap e71: 6 total, 5 up, 6 in 2024-05-27T09:38:52.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:52 smithi119 ceph-mon[88269]: osdmap e71: 6 total, 5 up, 6 in 2024-05-27T09:38:52.757 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:52 smithi138 podman[84151]: 2024-05-27 09:38:52.369083994 +0000 UTC m=+0.036101390 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:38:52.757 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:52 smithi138 podman[84151]: 2024-05-27 09:38:52.477769317 +0000 UTC m=+0.144786714 container create dcf8b17da1afd61d2a0339cdeb1221db82dda41d8f4cc4da8ce032d258a563bf (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-27T09:38:52.757 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:52 smithi138 podman[84151]: 2024-05-27 09:38:52.756637568 +0000 UTC m=+0.423654996 container init dcf8b17da1afd61d2a0339cdeb1221db82dda41d8f4cc4da8ce032d258a563bf (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2) 2024-05-27T09:38:53.047 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:52 smithi138 podman[84151]: 2024-05-27 09:38:52.768788458 +0000 UTC m=+0.435805821 container start dcf8b17da1afd61d2a0339cdeb1221db82dda41d8f4cc4da8ce032d258a563bf (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-27T09:38:53.047 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:52 smithi138 podman[84151]: 2024-05-27 09:38:52.88036897 +0000 UTC m=+0.547386437 container attach dcf8b17da1afd61d2a0339cdeb1221db82dda41d8f4cc4da8ce032d258a563bf (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T09:38:53.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:53 smithi138 ceph-mon[70004]: pgmap v157: 65 pgs: 8 peering, 4 stale+active+clean, 53 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:53.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:53 smithi138 ceph-mon[70004]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-05-27T09:38:53.387 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:53 smithi138 podman[84151]: 2024-05-27 09:38:53.219807983 +0000 UTC m=+0.886825411 container died dcf8b17da1afd61d2a0339cdeb1221db82dda41d8f4cc4da8ce032d258a563bf (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-27T09:38:53.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:53 smithi119 ceph-mon[88269]: pgmap v157: 65 pgs: 8 peering, 4 stale+active+clean, 53 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:38:53.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:53 smithi119 ceph-mon[88269]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-05-27T09:38:54.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:54 smithi138 ceph-mon[70004]: pgmap v158: 65 pgs: 8 active+undersized+degraded, 8 peering, 2 stale+active+clean, 8 active+undersized, 39 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 35/381 objects degraded (9.186%) 2024-05-27T09:38:54.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:54 smithi138 ceph-mon[70004]: Health check failed: Degraded data redundancy: 35/381 objects degraded (9.186%), 8 pgs degraded (PG_DEGRADED) 2024-05-27T09:38:54.386 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:54 smithi138 podman[84176]: 2024-05-27 09:38:54.033986998 +0000 UTC m=+0.794823038 container remove dcf8b17da1afd61d2a0339cdeb1221db82dda41d8f4cc4da8ce032d258a563bf (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-27T09:38:54.386 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:54 smithi138 systemd[1]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.4.service: Deactivated successfully. 2024-05-27T09:38:54.387 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:54 smithi138 systemd[1]: Stopped Ceph osd.4 for b966b06e-1c09-11ef-bc9b-c7b262605968. 2024-05-27T09:38:54.387 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:54 smithi138 systemd[1]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.4.service: Consumed 1min 41.160s CPU time. 2024-05-27T09:38:54.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:54 smithi119 ceph-mon[88269]: pgmap v158: 65 pgs: 8 active+undersized+degraded, 8 peering, 2 stale+active+clean, 8 active+undersized, 39 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 35/381 objects degraded (9.186%) 2024-05-27T09:38:54.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:54 smithi119 ceph-mon[88269]: Health check failed: Degraded data redundancy: 35/381 objects degraded (9.186%), 8 pgs degraded (PG_DEGRADED) 2024-05-27T09:38:54.831 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:54 smithi138 systemd[1]: Starting Ceph osd.4 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:38:55.086 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:54 smithi138 podman[84275]: 2024-05-27 09:38:54.735626569 +0000 UTC m=+0.031989764 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:38:55.086 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:54 smithi138 podman[84275]: 2024-05-27 09:38:54.901478642 +0000 UTC m=+0.197841852 container create eff758967e3fdd3e9c5377371a7b18bfc93a5b8a7d0388f3d89ae0cea692736b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-activate, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521) 2024-05-27T09:38:55.363 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:55 smithi138 podman[84275]: 2024-05-27 09:38:55.206117254 +0000 UTC m=+0.502480520 container init eff758967e3fdd3e9c5377371a7b18bfc93a5b8a7d0388f3d89ae0cea692736b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-activate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-27T09:38:55.364 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:55 smithi138 podman[84275]: 2024-05-27 09:38:55.217847957 +0000 UTC m=+0.514211182 container start eff758967e3fdd3e9c5377371a7b18bfc93a5b8a7d0388f3d89ae0cea692736b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-activate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-27T09:38:55.635 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:55 smithi138 podman[84275]: 2024-05-27 09:38:55.363176953 +0000 UTC m=+0.659540239 container attach eff758967e3fdd3e9c5377371a7b18bfc93a5b8a7d0388f3d89ae0cea692736b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-activate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-27T09:38:56.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:56 smithi119 ceph-mon[88269]: pgmap v159: 65 pgs: 12 active+undersized+degraded, 8 peering, 14 active+undersized, 31 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 49/381 objects degraded (12.861%) 2024-05-27T09:38:56.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:56 smithi138 ceph-mon[70004]: pgmap v159: 65 pgs: 12 active+undersized+degraded, 8 peering, 14 active+undersized, 31 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 49/381 objects degraded (12.861%) 2024-05-27T09:38:57.386 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:57 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-activate[84289]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:38:57.386 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:57 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-activate[84289]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:38:57.386 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:57 smithi138 bash[84275]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:38:57.386 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:57 smithi138 bash[84275]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:38:57.387 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:57 smithi138 bash[84275]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-27T09:38:57.387 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:57 smithi138 bash[84275]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-27T09:38:57.387 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:57 smithi138 bash[84275]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-27T09:38:57.387 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:57 smithi138 bash[84275]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-27T09:38:57.387 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:57 smithi138 bash[84275]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-05-27T09:38:57.387 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:57 smithi138 bash[84275]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-27T09:38:57.387 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:57 smithi138 bash[84275]: --> ceph-volume raw activate successful for osd ID: 4 2024-05-27T09:38:57.387 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:57 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-activate[84289]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-27T09:38:57.387 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:57 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-activate[84289]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-27T09:38:57.387 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:57 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-activate[84289]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-27T09:38:57.387 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:57 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-activate[84289]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-27T09:38:57.387 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:57 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-activate[84289]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-05-27T09:38:57.387 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:57 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-activate[84289]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-27T09:38:57.388 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:57 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-activate[84289]: --> ceph-volume raw activate successful for osd ID: 4 2024-05-27T09:38:57.388 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:57 smithi138 podman[84275]: 2024-05-27 09:38:57.067612667 +0000 UTC m=+2.363975949 container died eff758967e3fdd3e9c5377371a7b18bfc93a5b8a7d0388f3d89ae0cea692736b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-activate, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-27T09:38:58.135 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:57 smithi138 podman[84418]: 2024-05-27 09:38:57.84233239 +0000 UTC m=+0.756576331 container remove eff758967e3fdd3e9c5377371a7b18bfc93a5b8a7d0388f3d89ae0cea692736b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-activate, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.build-date=20240521, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T09:38:58.556 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:58 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:58.556 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:58 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:38:58.556 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:58 smithi138 ceph-mon[70004]: pgmap v160: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 67/381 objects degraded (17.585%) 2024-05-27T09:38:58.557 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:58 smithi138 podman[84475]: 2024-05-27 09:38:58.112878456 +0000 UTC m=+0.032082409 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:38:58.557 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:58 smithi138 podman[84475]: 2024-05-27 09:38:58.251982545 +0000 UTC m=+0.171186533 container create 250e443364de2a3a7d490de113c3f9787dc35f40621cab2bea8c6c554fdf174b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, ceph=True) 2024-05-27T09:38:58.557 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:58 smithi138 podman[84475]: 2024-05-27 09:38:58.555969313 +0000 UTC m=+0.475173308 container init 250e443364de2a3a7d490de113c3f9787dc35f40621cab2bea8c6c554fdf174b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-05-27T09:38:58.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:58 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:58.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:58 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:38:58.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:58 smithi119 ceph-mon[88269]: pgmap v160: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 67/381 objects degraded (17.585%) 2024-05-27T09:38:58.886 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:58 smithi138 podman[84475]: 2024-05-27 09:38:58.567850156 +0000 UTC m=+0.487054184 container start 250e443364de2a3a7d490de113c3f9787dc35f40621cab2bea8c6c554fdf174b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-27T09:38:58.886 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:58 smithi138 ceph-osd[84493]: -- 172.21.15.138:0/1331079944 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c58e05b680 con 0x55c58e102d00 2024-05-27T09:38:58.886 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:58 smithi138 bash[84475]: 250e443364de2a3a7d490de113c3f9787dc35f40621cab2bea8c6c554fdf174b 2024-05-27T09:38:58.886 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:58 smithi138 systemd[1]: Started Ceph osd.4 for b966b06e-1c09-11ef-bc9b-c7b262605968. 2024-05-27T09:38:59.635 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:38:59 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:38:59.207+0000 7f4842f207c0 -1 Falling back to public interface 2024-05-27T09:38:59.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:59 smithi138 ceph-mon[70004]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-05-27T09:38:59.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:59 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:59.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:59 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:59.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:38:59 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:38:59.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:59 smithi119 ceph-mon[88269]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-05-27T09:38:59.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:59 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:59.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:59 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:38:59.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:38:59 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:39:00.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:00 smithi138 ceph-mon[70004]: pgmap v161: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 67/381 objects degraded (17.585%) 2024-05-27T09:39:00.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:00 smithi119 ceph-mon[88269]: pgmap v161: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 67/381 objects degraded (17.585%) 2024-05-27T09:39:02.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:02 smithi138 ceph-mon[70004]: pgmap v162: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 67/381 objects degraded (17.585%) 2024-05-27T09:39:02.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:02 smithi138 ceph-mon[70004]: Health check update: Degraded data redundancy: 67/381 objects degraded (17.585%), 16 pgs degraded (PG_DEGRADED) 2024-05-27T09:39:02.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:02 smithi119 ceph-mon[88269]: pgmap v162: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 67/381 objects degraded (17.585%) 2024-05-27T09:39:02.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:02 smithi119 ceph-mon[88269]: Health check update: Degraded data redundancy: 67/381 objects degraded (17.585%), 16 pgs degraded (PG_DEGRADED) 2024-05-27T09:39:04.445 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:04 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:04.445 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:04 smithi119 ceph-mon[88269]: pgmap v163: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 67/381 objects degraded (17.585%) 2024-05-27T09:39:04.445 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:04 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:04.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:04 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:04.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:04 smithi138 ceph-mon[70004]: pgmap v163: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 67/381 objects degraded (17.585%) 2024-05-27T09:39:04.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:04 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:05.885 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:39:05 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:39:05.468+0000 7f4842f207c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-05-27T09:39:06.479 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:06 smithi138 ceph-mon[70004]: pgmap v164: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 67/381 objects degraded (17.585%) 2024-05-27T09:39:06.479 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:06 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:06.479 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:06 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:06.479 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:06 smithi138 ceph-mon[70004]: from='osd.4 [v2:172.21.15.138:6808/2602648260,v1:172.21.15.138:6809/2602648260]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T09:39:06.479 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:06 smithi138 ceph-mon[70004]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T09:39:06.479 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:39:06 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:39:06.226+0000 7f4842f207c0 -1 osd.4 69 log_to_monitors true 2024-05-27T09:39:06.480 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:39:06 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:39:06.427+0000 7f4832200700 -1 osd.4 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T09:39:06.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:06 smithi119 ceph-mon[88269]: pgmap v164: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 67/381 objects degraded (17.585%) 2024-05-27T09:39:06.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:06 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:06.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:06 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:06.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:06 smithi119 ceph-mon[88269]: from='osd.4 [v2:172.21.15.138:6808/2602648260,v1:172.21.15.138:6809/2602648260]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T09:39:06.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:06 smithi119 ceph-mon[88269]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T09:39:07.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.349+0000 7fd495200700 1 -- 172.21.15.119:0/1914302782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4900730f0 msgr2=0x7fd4900734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:07.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.349+0000 7fd495200700 1 --2- 172.21.15.119:0/1914302782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4900730f0 0x7fd4900734c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd48000b0a0 tx=0x7fd48000b3b0 comp rx=0 tx=0).stop 2024-05-27T09:39:07.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.350+0000 7fd495200700 1 -- 172.21.15.119:0/1914302782 shutdown_connections 2024-05-27T09:39:07.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.350+0000 7fd495200700 1 --2- 172.21.15.119:0/1914302782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd490073a00 0x7fd490110a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:07.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.350+0000 7fd495200700 1 --2- 172.21.15.119:0/1914302782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4900730f0 0x7fd4900734c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:07.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.350+0000 7fd495200700 1 -- 172.21.15.119:0/1914302782 >> 172.21.15.119:0/1914302782 conn(0x7fd4900fba80 msgr2=0x7fd4900fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:07.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.350+0000 7fd495200700 1 -- 172.21.15.119:0/1914302782 shutdown_connections 2024-05-27T09:39:07.356 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.350+0000 7fd495200700 1 -- 172.21.15.119:0/1914302782 wait complete. 2024-05-27T09:39:07.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.351+0000 7fd495200700 1 Processor -- start 2024-05-27T09:39:07.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.351+0000 7fd495200700 1 -- start start 2024-05-27T09:39:07.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.351+0000 7fd495200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4900730f0 0x7fd49019c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:07.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.351+0000 7fd495200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd490073a00 0x7fd49019c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:07.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.351+0000 7fd495200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49019cd40 con 0x7fd490073a00 2024-05-27T09:39:07.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.351+0000 7fd495200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4901a0ad0 con 0x7fd4900730f0 2024-05-27T09:39:07.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.352+0000 7fd486a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd490073a00 0x7fd49019c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:07.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.352+0000 7fd486a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd490073a00 0x7fd49019c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48682/0 (socket says 172.21.15.119:48682) 2024-05-27T09:39:07.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.352+0000 7fd486a00700 1 -- 172.21.15.119:0/1848441854 learned_addr learned my addr 172.21.15.119:0/1848441854 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:39:07.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.352+0000 7fd48f400700 1 --2- 172.21.15.119:0/1848441854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4900730f0 0x7fd49019c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:07.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.352+0000 7fd486a00700 1 -- 172.21.15.119:0/1848441854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4900730f0 msgr2=0x7fd49019c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:07.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.352+0000 7fd486a00700 1 --2- 172.21.15.119:0/1848441854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4900730f0 0x7fd49019c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:07.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.352+0000 7fd486a00700 1 -- 172.21.15.119:0/1848441854 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd480009d00 con 0x7fd490073a00 2024-05-27T09:39:07.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.353+0000 7fd486a00700 1 --2- 172.21.15.119:0/1848441854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd490073a00 0x7fd49019c660 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd47800c8a0 tx=0x7fd47800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:07.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.353+0000 7fd48cc00700 1 -- 172.21.15.119:0/1848441854 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd47800cea0 con 0x7fd490073a00 2024-05-27T09:39:07.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.353+0000 7fd48cc00700 1 -- 172.21.15.119:0/1848441854 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd478004830 con 0x7fd490073a00 2024-05-27T09:39:07.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.353+0000 7fd495200700 1 -- 172.21.15.119:0/1848441854 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4901a0db0 con 0x7fd490073a00 2024-05-27T09:39:07.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.353+0000 7fd48cc00700 1 -- 172.21.15.119:0/1848441854 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4780055b0 con 0x7fd490073a00 2024-05-27T09:39:07.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.353+0000 7fd495200700 1 -- 172.21.15.119:0/1848441854 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4901a1300 con 0x7fd490073a00 2024-05-27T09:39:07.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.354+0000 7fd495200700 1 -- 172.21.15.119:0/1848441854 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd49010e1f0 con 0x7fd490073a00 2024-05-27T09:39:07.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.356+0000 7fd48cc00700 1 -- 172.21.15.119:0/1848441854 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fd4780049a0 con 0x7fd490073a00 2024-05-27T09:39:07.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.357+0000 7fd48cc00700 1 --2- 172.21.15.119:0/1848441854 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fd474078a40 0x7fd47407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:07.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.357+0000 7fd48cc00700 1 -- 172.21.15.119:0/1848441854 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd47809a940 con 0x7fd490073a00 2024-05-27T09:39:07.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.357+0000 7fd48f400700 1 --2- 172.21.15.119:0/1848441854 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fd474078a40 0x7fd47407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:07.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.358+0000 7fd48f400700 1 --2- 172.21.15.119:0/1848441854 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fd474078a40 0x7fd47407aef0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd480009850 tx=0x7fd4800097c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:07.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.363+0000 7fd48cc00700 1 -- 172.21.15.119:0/1848441854 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fd4780625c0 con 0x7fd490073a00 2024-05-27T09:39:07.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.635+0000 7fd495200700 1 -- 172.21.15.119:0/1848441854 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4901a1660 con 0x7fd474078a40 2024-05-27T09:39:07.638 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:07 smithi119 ceph-mon[88269]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-27T09:39:07.638 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:07 smithi119 ceph-mon[88269]: osdmap e72: 6 total, 5 up, 6 in 2024-05-27T09:39:07.638 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:07 smithi119 ceph-mon[88269]: from='osd.4 [v2:172.21.15.138:6808/2602648260,v1:172.21.15.138:6809/2602648260]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:39:07.638 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:07 smithi119 ceph-mon[88269]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:39:07.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.639+0000 7fd48cc00700 1 -- 172.21.15.119:0/1848441854 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fd4901a1660 con 0x7fd474078a40 2024-05-27T09:39:07.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.642+0000 7fd495200700 1 -- 172.21.15.119:0/1848441854 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fd474078a40 msgr2=0x7fd47407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:07.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.642+0000 7fd495200700 1 --2- 172.21.15.119:0/1848441854 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fd474078a40 0x7fd47407aef0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd480009850 tx=0x7fd4800097c0 comp rx=0 tx=0).stop 2024-05-27T09:39:07.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.642+0000 7fd495200700 1 -- 172.21.15.119:0/1848441854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd490073a00 msgr2=0x7fd49019c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:07.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.642+0000 7fd495200700 1 --2- 172.21.15.119:0/1848441854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd490073a00 0x7fd49019c660 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd47800c8a0 tx=0x7fd47800cc60 comp rx=0 tx=0).stop 2024-05-27T09:39:07.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.642+0000 7fd495200700 1 -- 172.21.15.119:0/1848441854 shutdown_connections 2024-05-27T09:39:07.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.642+0000 7fd495200700 1 --2- 172.21.15.119:0/1848441854 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fd474078a40 0x7fd47407aef0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:07.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.642+0000 7fd495200700 1 --2- 172.21.15.119:0/1848441854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4900730f0 0x7fd49019c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:07.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.642+0000 7fd495200700 1 --2- 172.21.15.119:0/1848441854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd490073a00 0x7fd49019c660 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:07.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.642+0000 7fd495200700 1 -- 172.21.15.119:0/1848441854 >> 172.21.15.119:0/1848441854 conn(0x7fd4900fba80 msgr2=0x7fd490102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:07.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.642+0000 7fd495200700 1 -- 172.21.15.119:0/1848441854 shutdown_connections 2024-05-27T09:39:07.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.642+0000 7fd495200700 1 -- 172.21.15.119:0/1848441854 wait complete. 2024-05-27T09:39:07.661 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:39:07.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.799+0000 7f14ab600700 1 -- 172.21.15.119:0/1728071059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14a410b700 msgr2=0x7f14a410bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:07.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.799+0000 7f14ab600700 1 --2- 172.21.15.119:0/1728071059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14a410b700 0x7f14a410bad0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1498009910 tx=0x7f1498009c20 comp rx=0 tx=0).stop 2024-05-27T09:39:07.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.799+0000 7f14ab600700 1 -- 172.21.15.119:0/1728071059 shutdown_connections 2024-05-27T09:39:07.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.799+0000 7f14ab600700 1 --2- 172.21.15.119:0/1728071059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14a410c010 0x7f14a4106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:07.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.799+0000 7f14ab600700 1 --2- 172.21.15.119:0/1728071059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14a410b700 0x7f14a410bad0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:07.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.799+0000 7f14ab600700 1 -- 172.21.15.119:0/1728071059 >> 172.21.15.119:0/1728071059 conn(0x7f14a4076000 msgr2=0x7f14a4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:07.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.799+0000 7f14ab600700 1 -- 172.21.15.119:0/1728071059 shutdown_connections 2024-05-27T09:39:07.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.799+0000 7f14ab600700 1 -- 172.21.15.119:0/1728071059 wait complete. 2024-05-27T09:39:07.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.800+0000 7f14ab600700 1 Processor -- start 2024-05-27T09:39:07.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.801+0000 7f14ab600700 1 -- start start 2024-05-27T09:39:07.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.801+0000 7f14ab600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14a410b700 0x7f14a419c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:07.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.801+0000 7f14ab600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14a410c010 0x7f14a419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:07.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.801+0000 7f14ab600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a419ced0 con 0x7f14a410c010 2024-05-27T09:39:07.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.801+0000 7f14ab600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a41a0c60 con 0x7f14a410b700 2024-05-27T09:39:07.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.801+0000 7f14a3e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14a410c010 0x7f14a419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:07.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.801+0000 7f14a8c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14a410b700 0x7f14a419c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:07.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.801+0000 7f14a8c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14a410b700 0x7f14a419c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.119:53936/0 (socket says 172.21.15.119:53936) 2024-05-27T09:39:07.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.801+0000 7f14a3e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14a410c010 0x7f14a419c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48692/0 (socket says 172.21.15.119:48692) 2024-05-27T09:39:07.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.801+0000 7f14a8c00700 1 -- 172.21.15.119:0/96188604 learned_addr learned my addr 172.21.15.119:0/96188604 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:39:07.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.802+0000 7f14a3e00700 1 -- 172.21.15.119:0/96188604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14a410b700 msgr2=0x7f14a419c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:07.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.802+0000 7f14a3e00700 1 --2- 172.21.15.119:0/96188604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14a410b700 0x7f14a419c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:07.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.802+0000 7f14a3e00700 1 -- 172.21.15.119:0/96188604 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14980095f0 con 0x7f14a410c010 2024-05-27T09:39:07.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.802+0000 7f14a8c00700 1 --2- 172.21.15.119:0/96188604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14a410b700 0x7f14a419c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T09:39:07.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.802+0000 7f14a3e00700 1 --2- 172.21.15.119:0/96188604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14a410c010 0x7f14a419c7f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f149000e940 tx=0x7f149000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:07.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.802+0000 7f14a1600700 1 -- 172.21.15.119:0/96188604 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f149000c990 con 0x7f14a410c010 2024-05-27T09:39:07.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.802+0000 7f14a1600700 1 -- 172.21.15.119:0/96188604 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1490010430 con 0x7f14a410c010 2024-05-27T09:39:07.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.802+0000 7f14a1600700 1 -- 172.21.15.119:0/96188604 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1490018650 con 0x7f14a410c010 2024-05-27T09:39:07.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.802+0000 7f14ab600700 1 -- 172.21.15.119:0/96188604 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a41a0f40 con 0x7f14a410c010 2024-05-27T09:39:07.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.803+0000 7f14ab600700 1 -- 172.21.15.119:0/96188604 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a41a1510 con 0x7f14a410c010 2024-05-27T09:39:07.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.804+0000 7f14ab600700 1 -- 172.21.15.119:0/96188604 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a404ea50 con 0x7f14a410c010 2024-05-27T09:39:07.806 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.805+0000 7f14a1600700 1 -- 172.21.15.119:0/96188604 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f14900105a0 con 0x7f14a410c010 2024-05-27T09:39:07.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.806+0000 7f14a1600700 1 --2- 172.21.15.119:0/96188604 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f148c078a40 0x7f148c07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:07.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.806+0000 7f14a1600700 1 -- 172.21.15.119:0/96188604 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f1490014070 con 0x7f14a410c010 2024-05-27T09:39:07.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.806+0000 7f14a8c00700 1 --2- 172.21.15.119:0/96188604 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f148c078a40 0x7f148c07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:07.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.807+0000 7f14a8c00700 1 --2- 172.21.15.119:0/96188604 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f148c078a40 0x7f148c07aef0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f149800b760 tx=0x7f1498005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:07.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:07.811+0000 7f14a1600700 1 -- 172.21.15.119:0/96188604 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f1490054df0 con 0x7f14a410c010 2024-05-27T09:39:07.883 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:07 smithi138 ceph-mon[70004]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-27T09:39:07.883 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:07 smithi138 ceph-mon[70004]: osdmap e72: 6 total, 5 up, 6 in 2024-05-27T09:39:07.883 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:07 smithi138 ceph-mon[70004]: from='osd.4 [v2:172.21.15.138:6808/2602648260,v1:172.21.15.138:6809/2602648260]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:39:07.883 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:07 smithi138 ceph-mon[70004]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:39:08.068 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.066+0000 7f14ab600700 1 -- 172.21.15.119:0/96188604 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f14a41a19c0 con 0x7f148c078a40 2024-05-27T09:39:08.070 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.069+0000 7f14a1600700 1 -- 172.21.15.119:0/96188604 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f14a41a19c0 con 0x7f148c078a40 2024-05-27T09:39:08.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.072+0000 7f14ab600700 1 -- 172.21.15.119:0/96188604 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f148c078a40 msgr2=0x7f148c07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:08.073 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.072+0000 7f14ab600700 1 --2- 172.21.15.119:0/96188604 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f148c078a40 0x7f148c07aef0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f149800b760 tx=0x7f1498005a90 comp rx=0 tx=0).stop 2024-05-27T09:39:08.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.072+0000 7f14ab600700 1 -- 172.21.15.119:0/96188604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14a410c010 msgr2=0x7f14a419c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:08.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.072+0000 7f14ab600700 1 --2- 172.21.15.119:0/96188604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14a410c010 0x7f14a419c7f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f149000e940 tx=0x7f149000ec50 comp rx=0 tx=0).stop 2024-05-27T09:39:08.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.072+0000 7f14ab600700 1 -- 172.21.15.119:0/96188604 shutdown_connections 2024-05-27T09:39:08.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.072+0000 7f14ab600700 1 --2- 172.21.15.119:0/96188604 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f148c078a40 0x7f148c07aef0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:08.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.072+0000 7f14ab600700 1 --2- 172.21.15.119:0/96188604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14a410b700 0x7f14a419c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:08.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.072+0000 7f14ab600700 1 --2- 172.21.15.119:0/96188604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14a410c010 0x7f14a419c7f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:08.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.072+0000 7f14ab600700 1 -- 172.21.15.119:0/96188604 >> 172.21.15.119:0/96188604 conn(0x7f14a4076000 msgr2=0x7f14a4076880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:08.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.072+0000 7f14ab600700 1 -- 172.21.15.119:0/96188604 shutdown_connections 2024-05-27T09:39:08.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.072+0000 7f14ab600700 1 -- 172.21.15.119:0/96188604 wait complete. 2024-05-27T09:39:08.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.229+0000 7f346f200700 1 -- 172.21.15.119:0/1372064899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3470102450 msgr2=0x7f3470102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:08.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.229+0000 7f346f200700 1 --2- 172.21.15.119:0/1372064899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3470102450 0x7f3470102820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3460009910 tx=0x7f3460009c20 comp rx=0 tx=0).stop 2024-05-27T09:39:08.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.229+0000 7f346f200700 1 -- 172.21.15.119:0/1372064899 shutdown_connections 2024-05-27T09:39:08.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.229+0000 7f346f200700 1 --2- 172.21.15.119:0/1372064899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3470102df0 0x7f3470106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:08.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.229+0000 7f346f200700 1 --2- 172.21.15.119:0/1372064899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3470102450 0x7f3470102820 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:08.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.229+0000 7f346f200700 1 -- 172.21.15.119:0/1372064899 >> 172.21.15.119:0/1372064899 conn(0x7f34700fdd10 msgr2=0x7f3470100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:08.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.230+0000 7f346f200700 1 -- 172.21.15.119:0/1372064899 shutdown_connections 2024-05-27T09:39:08.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.230+0000 7f346f200700 1 -- 172.21.15.119:0/1372064899 wait complete. 2024-05-27T09:39:08.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.231+0000 7f346f200700 1 Processor -- start 2024-05-27T09:39:08.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.231+0000 7f346f200700 1 -- start start 2024-05-27T09:39:08.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.231+0000 7f346f200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3470102450 0x7f3470197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:08.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.231+0000 7f346f200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3470102df0 0x7f34701982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:08.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.231+0000 7f346f200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34701989c0 con 0x7f3470102450 2024-05-27T09:39:08.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.231+0000 7f346f200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347019c750 con 0x7f3470102df0 2024-05-27T09:39:08.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.232+0000 7f346de00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3470102450 0x7f3470197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:08.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.232+0000 7f346de00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3470102450 0x7f3470197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48706/0 (socket says 172.21.15.119:48706) 2024-05-27T09:39:08.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.232+0000 7f346de00700 1 -- 172.21.15.119:0/1080004356 learned_addr learned my addr 172.21.15.119:0/1080004356 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:39:08.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.232+0000 7f346d400700 1 --2- 172.21.15.119:0/1080004356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3470102df0 0x7f34701982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:08.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.232+0000 7f346de00700 1 -- 172.21.15.119:0/1080004356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3470102df0 msgr2=0x7f34701982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:08.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.232+0000 7f346de00700 1 --2- 172.21.15.119:0/1080004356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3470102df0 0x7f34701982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:08.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.232+0000 7f346de00700 1 -- 172.21.15.119:0/1080004356 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34600095f0 con 0x7f3470102450 2024-05-27T09:39:08.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.232+0000 7f346de00700 1 --2- 172.21.15.119:0/1080004356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3470102450 0x7f3470197da0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3460006010 tx=0x7f346000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:08.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.233+0000 7f3466a00700 1 -- 172.21.15.119:0/1080004356 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f346001d070 con 0x7f3470102450 2024-05-27T09:39:08.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.233+0000 7f3466a00700 1 -- 172.21.15.119:0/1080004356 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f346000f460 con 0x7f3470102450 2024-05-27T09:39:08.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.233+0000 7f346f200700 1 -- 172.21.15.119:0/1080004356 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f347019c9d0 con 0x7f3470102450 2024-05-27T09:39:08.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.233+0000 7f3466a00700 1 -- 172.21.15.119:0/1080004356 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3460017610 con 0x7f3470102450 2024-05-27T09:39:08.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.233+0000 7f346f200700 1 -- 172.21.15.119:0/1080004356 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f347019cec0 con 0x7f3470102450 2024-05-27T09:39:08.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.234+0000 7f346f200700 1 -- 172.21.15.119:0/1080004356 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f347010a760 con 0x7f3470102450 2024-05-27T09:39:08.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.236+0000 7f3466a00700 1 -- 172.21.15.119:0/1080004356 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f346000f5d0 con 0x7f3470102450 2024-05-27T09:39:08.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.237+0000 7f3466a00700 1 --2- 172.21.15.119:0/1080004356 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3454078a40 0x7f345407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:08.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.237+0000 7f3466a00700 1 -- 172.21.15.119:0/1080004356 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f346009c1f0 con 0x7f3470102450 2024-05-27T09:39:08.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.237+0000 7f346d400700 1 --2- 172.21.15.119:0/1080004356 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3454078a40 0x7f345407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:08.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.238+0000 7f346d400700 1 --2- 172.21.15.119:0/1080004356 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3454078a40 0x7f345407aef0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f34701993c0 tx=0x7f345c00a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:08.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.242+0000 7f3466a00700 1 -- 172.21.15.119:0/1080004356 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f3460063e50 con 0x7f3470102450 2024-05-27T09:39:08.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.512+0000 7f346f200700 1 -- 172.21.15.119:0/1080004356 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f347002cd40 con 0x7f3454078a40 2024-05-27T09:39:08.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.527+0000 7f3466a00700 1 -- 172.21.15.119:0/1080004356 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f347002cd40 con 0x7f3454078a40 2024-05-27T09:39:08.529 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:39:08.529 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (6m) 79s ago 20m 14.3M - 0.25.0 c8568f914cd2 6b90564dcef7 2024-05-27T09:39:08.529 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (20m) 79s ago 20m 8850k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:39:08.529 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (19m) 5s ago 19m 8640k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:39:08.530 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (3m) 79s ago 20m 6669k - 19.0.0-3868-ga6507388 850a06a84d28 011d811ce7fc 2024-05-27T09:39:08.530 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (3m) 5s ago 19m 6665k - 19.0.0-3868-ga6507388 850a06a84d28 b546b3ed1afc 2024-05-27T09:39:08.530 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (6m) 79s ago 19m 74.4M - 9.4.12 2bacad6d85d8 b3abdab08db4 2024-05-27T09:39:08.530 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (13m) 79s ago 13m 18.0M - 18.2.0 dc2bc1663786 93b21713fb1e 2024-05-27T09:39:08.530 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (13m) 79s ago 13m 130M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:39:08.530 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (12m) 5s ago 12m 18.2M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:39:08.530 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (13m) 5s ago 13m 128M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:39:08.530 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (8m) 79s ago 22m 587M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:39:08.530 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (7m) 5s ago 18m 534M - 19.0.0-3868-ga6507388 850a06a84d28 2fc3a80696fa 2024-05-27T09:39:08.530 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (4m) 79s ago 22m 114M 2048M 19.0.0-3868-ga6507388 850a06a84d28 3e5c7a9ad372 2024-05-27T09:39:08.531 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (4m) 5s ago 18m 107M 2048M 19.0.0-3868-ga6507388 850a06a84d28 e4d471274cf1 2024-05-27T09:39:08.531 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (7m) 79s ago 20m 8002k - 1.5.0 0da6a335fe13 709c37815cff 2024-05-27T09:39:08.531 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (7m) 5s ago 19m 7990k - 1.5.0 0da6a335fe13 999047a1a5bf 2024-05-27T09:39:08.531 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (2m) 79s ago 17m 152M 1216M 19.0.0-3868-ga6507388 850a06a84d28 5a0e162d870e 2024-05-27T09:39:08.531 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (118s) 79s ago 17m 167M 1216M 19.0.0-3868-ga6507388 850a06a84d28 effcc7c53dd3 2024-05-27T09:39:08.531 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (87s) 79s ago 16m 78.5M 1216M 19.0.0-3868-ga6507388 850a06a84d28 f0bd34ca416f 2024-05-27T09:39:08.531 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (54s) 5s ago 16m 172M 2240M 19.0.0-3868-ga6507388 850a06a84d28 5e8d3e398c21 2024-05-27T09:39:08.531 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (10s) 5s ago 15m 78.5M 2240M 19.0.0-3868-ga6507388 850a06a84d28 250e443364de 2024-05-27T09:39:08.531 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (14m) 5s ago 14m 226M 2240M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:39:08.531 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (7m) 79s ago 19m 71.1M - 2.43.0 a07b618ecd1d cba50454ed21 2024-05-27T09:39:08.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.529+0000 7f346f200700 1 -- 172.21.15.119:0/1080004356 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3454078a40 msgr2=0x7f345407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:08.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.529+0000 7f346f200700 1 --2- 172.21.15.119:0/1080004356 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3454078a40 0x7f345407aef0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f34701993c0 tx=0x7f345c00a300 comp rx=0 tx=0).stop 2024-05-27T09:39:08.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.530+0000 7f346f200700 1 -- 172.21.15.119:0/1080004356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3470102450 msgr2=0x7f3470197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:08.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.530+0000 7f346f200700 1 --2- 172.21.15.119:0/1080004356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3470102450 0x7f3470197da0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3460006010 tx=0x7f346000bd70 comp rx=0 tx=0).stop 2024-05-27T09:39:08.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.530+0000 7f346f200700 1 -- 172.21.15.119:0/1080004356 shutdown_connections 2024-05-27T09:39:08.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.530+0000 7f346f200700 1 --2- 172.21.15.119:0/1080004356 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3454078a40 0x7f345407aef0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:08.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.530+0000 7f346f200700 1 --2- 172.21.15.119:0/1080004356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3470102450 0x7f3470197da0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:08.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.530+0000 7f346f200700 1 --2- 172.21.15.119:0/1080004356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3470102df0 0x7f34701982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:08.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.530+0000 7f346f200700 1 -- 172.21.15.119:0/1080004356 >> 172.21.15.119:0/1080004356 conn(0x7f34700fdd10 msgr2=0x7f34701000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:08.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.530+0000 7f346f200700 1 -- 172.21.15.119:0/1080004356 shutdown_connections 2024-05-27T09:39:08.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.530+0000 7f346f200700 1 -- 172.21.15.119:0/1080004356 wait complete. 2024-05-27T09:39:08.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:08 smithi138 ceph-mon[70004]: pgmap v166: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 67/381 objects degraded (17.585%) 2024-05-27T09:39:08.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:08 smithi138 ceph-mon[70004]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-27T09:39:08.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:08 smithi138 ceph-mon[70004]: osd.4 [v2:172.21.15.138:6808/2602648260,v1:172.21.15.138:6809/2602648260] boot 2024-05-27T09:39:08.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:08 smithi138 ceph-mon[70004]: osdmap e73: 6 total, 6 up, 6 in 2024-05-27T09:39:08.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:08 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T09:39:08.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:08 smithi138 ceph-mon[70004]: from='client.34372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:39:08.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.673+0000 7fe856400700 1 -- 172.21.15.119:0/2834346738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8500fed20 msgr2=0x7fe8500ff0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:08.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.673+0000 7fe856400700 1 --2- 172.21.15.119:0/2834346738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8500fed20 0x7fe8500ff0f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe844009910 tx=0x7fe844009c20 comp rx=0 tx=0).stop 2024-05-27T09:39:08.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.674+0000 7fe856400700 1 -- 172.21.15.119:0/2834346738 shutdown_connections 2024-05-27T09:39:08.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.674+0000 7fe856400700 1 --2- 172.21.15.119:0/2834346738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8500ff630 0x7fe85010c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:08.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.674+0000 7fe856400700 1 --2- 172.21.15.119:0/2834346738 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8500fed20 0x7fe8500ff0f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:08.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.674+0000 7fe856400700 1 -- 172.21.15.119:0/2834346738 >> 172.21.15.119:0/2834346738 conn(0x7fe8500fa9d0 msgr2=0x7fe8500fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:08.677 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:08 smithi119 ceph-mon[88269]: pgmap v166: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 67/381 objects degraded (17.585%) 2024-05-27T09:39:08.677 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:08 smithi119 ceph-mon[88269]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-27T09:39:08.677 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:08 smithi119 ceph-mon[88269]: osd.4 [v2:172.21.15.138:6808/2602648260,v1:172.21.15.138:6809/2602648260] boot 2024-05-27T09:39:08.677 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:08 smithi119 ceph-mon[88269]: osdmap e73: 6 total, 6 up, 6 in 2024-05-27T09:39:08.677 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:08 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T09:39:08.677 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:08 smithi119 ceph-mon[88269]: from='client.34372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:39:08.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.674+0000 7fe856400700 1 -- 172.21.15.119:0/2834346738 shutdown_connections 2024-05-27T09:39:08.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.674+0000 7fe856400700 1 -- 172.21.15.119:0/2834346738 wait complete. 2024-05-27T09:39:08.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.675+0000 7fe856400700 1 Processor -- start 2024-05-27T09:39:08.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.675+0000 7fe856400700 1 -- start start 2024-05-27T09:39:08.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.675+0000 7fe856400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8500fed20 0x7fe850197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:08.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.676+0000 7fe856400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8500ff630 0x7fe850198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:08.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.676+0000 7fe856400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe850198a10 con 0x7fe8500ff630 2024-05-27T09:39:08.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.676+0000 7fe856400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85019c7a0 con 0x7fe8500fed20 2024-05-27T09:39:08.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.676+0000 7fe84fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8500ff630 0x7fe850198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:08.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.676+0000 7fe84fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8500ff630 0x7fe850198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48732/0 (socket says 172.21.15.119:48732) 2024-05-27T09:39:08.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.676+0000 7fe84fe00700 1 -- 172.21.15.119:0/1289471420 learned_addr learned my addr 172.21.15.119:0/1289471420 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:39:08.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.676+0000 7fe855000700 1 --2- 172.21.15.119:0/1289471420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8500fed20 0x7fe850197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:08.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.676+0000 7fe84fe00700 1 -- 172.21.15.119:0/1289471420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8500fed20 msgr2=0x7fe850197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:08.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.676+0000 7fe84fe00700 1 --2- 172.21.15.119:0/1289471420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8500fed20 0x7fe850197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:08.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.676+0000 7fe84fe00700 1 -- 172.21.15.119:0/1289471420 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8440095f0 con 0x7fe8500ff630 2024-05-27T09:39:08.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.677+0000 7fe84fe00700 1 --2- 172.21.15.119:0/1289471420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8500ff630 0x7fe850198330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00b4b0 tx=0x7fe83c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:08.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.677+0000 7fe84d600700 1 -- 172.21.15.119:0/1289471420 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe83c011800 con 0x7fe8500ff630 2024-05-27T09:39:08.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.677+0000 7fe84d600700 1 -- 172.21.15.119:0/1289471420 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe83c011e20 con 0x7fe8500ff630 2024-05-27T09:39:08.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.677+0000 7fe856400700 1 -- 172.21.15.119:0/1289471420 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85019ca80 con 0x7fe8500ff630 2024-05-27T09:39:08.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.677+0000 7fe84d600700 1 -- 172.21.15.119:0/1289471420 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe83c00f2e0 con 0x7fe8500ff630 2024-05-27T09:39:08.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.677+0000 7fe856400700 1 -- 172.21.15.119:0/1289471420 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe85019cfd0 con 0x7fe8500ff630 2024-05-27T09:39:08.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.678+0000 7fe856400700 1 -- 172.21.15.119:0/1289471420 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe850109b00 con 0x7fe8500ff630 2024-05-27T09:39:08.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.680+0000 7fe84d600700 1 -- 172.21.15.119:0/1289471420 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fe83c010380 con 0x7fe8500ff630 2024-05-27T09:39:08.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.681+0000 7fe84d600700 1 --2- 172.21.15.119:0/1289471420 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe838078a40 0x7fe83807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:08.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.681+0000 7fe84d600700 1 -- 172.21.15.119:0/1289471420 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe83c09a830 con 0x7fe8500ff630 2024-05-27T09:39:08.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.681+0000 7fe855000700 1 --2- 172.21.15.119:0/1289471420 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe838078a40 0x7fe83807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.682+0000 7fe855000700 1 --2- 172.21.15.119:0/1289471420 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe838078a40 0x7fe83807aef0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe844006010 tx=0x7fe84401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:08.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.686+0000 7fe84d600700 1 -- 172.21.15.119:0/1289471420 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fe83c062410 con 0x7fe8500ff630 2024-05-27T09:39:08.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.994+0000 7fe856400700 1 -- 172.21.15.119:0/1289471420 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe85019d250 con 0x7fe8500ff630 2024-05-27T09:39:08.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.995+0000 7fe84d600700 1 -- 172.21.15.119:0/1289471420 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fe83c061db0 con 0x7fe8500ff630 2024-05-27T09:39:08.997 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:39:08.997 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:39:08.998 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:39:08.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:39:08.998 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:39:08.998 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:39:08.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:39:08.998 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:39:08.998 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-05-27T09:39:08.998 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 5 2024-05-27T09:39:08.998 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:39:08.998 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:39:08.998 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-05-27T09:39:08.999 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:39:08.999 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:39:08.999 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 5, 2024-05-27T09:39:08.999 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 9 2024-05-27T09:39:08.999 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:39:08.999 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:39:09.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.998+0000 7fe856400700 1 -- 172.21.15.119:0/1289471420 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe838078a40 msgr2=0x7fe83807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:09.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.998+0000 7fe856400700 1 --2- 172.21.15.119:0/1289471420 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe838078a40 0x7fe83807aef0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe844006010 tx=0x7fe84401a040 comp rx=0 tx=0).stop 2024-05-27T09:39:09.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.998+0000 7fe856400700 1 -- 172.21.15.119:0/1289471420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8500ff630 msgr2=0x7fe850198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:09.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.998+0000 7fe856400700 1 --2- 172.21.15.119:0/1289471420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8500ff630 0x7fe850198330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00b4b0 tx=0x7fe83c00b7c0 comp rx=0 tx=0).stop 2024-05-27T09:39:09.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.998+0000 7fe856400700 1 -- 172.21.15.119:0/1289471420 shutdown_connections 2024-05-27T09:39:09.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.998+0000 7fe856400700 1 --2- 172.21.15.119:0/1289471420 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe838078a40 0x7fe83807aef0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:09.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.998+0000 7fe856400700 1 --2- 172.21.15.119:0/1289471420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8500fed20 0x7fe850197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:09.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.998+0000 7fe856400700 1 --2- 172.21.15.119:0/1289471420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8500ff630 0x7fe850198330 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:09.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.998+0000 7fe856400700 1 -- 172.21.15.119:0/1289471420 >> 172.21.15.119:0/1289471420 conn(0x7fe8500fa9d0 msgr2=0x7fe8500fbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:09.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.998+0000 7fe856400700 1 -- 172.21.15.119:0/1289471420 shutdown_connections 2024-05-27T09:39:09.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:08.998+0000 7fe856400700 1 -- 172.21.15.119:0/1289471420 wait complete. 2024-05-27T09:39:09.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.153+0000 7f95b9200700 1 -- 172.21.15.119:0/1672694458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b4100460 msgr2=0x7f95b4100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:09.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.153+0000 7f95b9200700 1 --2- 172.21.15.119:0/1672694458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b4100460 0x7f95b4100830 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009910 tx=0x7f95a4009c20 comp rx=0 tx=0).stop 2024-05-27T09:39:09.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.153+0000 7f95b9200700 1 -- 172.21.15.119:0/1672694458 shutdown_connections 2024-05-27T09:39:09.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.153+0000 7f95b9200700 1 --2- 172.21.15.119:0/1672694458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95b4100e00 0x7f95b4104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:09.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.153+0000 7f95b9200700 1 --2- 172.21.15.119:0/1672694458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b4100460 0x7f95b4100830 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:09.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.153+0000 7f95b9200700 1 -- 172.21.15.119:0/1672694458 >> 172.21.15.119:0/1672694458 conn(0x7f95b40fbd20 msgr2=0x7f95b40fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:09.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.154+0000 7f95b9200700 1 -- 172.21.15.119:0/1672694458 shutdown_connections 2024-05-27T09:39:09.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.154+0000 7f95b9200700 1 -- 172.21.15.119:0/1672694458 wait complete. 2024-05-27T09:39:09.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.155+0000 7f95b9200700 1 Processor -- start 2024-05-27T09:39:09.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.155+0000 7f95b9200700 1 -- start start 2024-05-27T09:39:09.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.155+0000 7f95b9200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b4100460 0x7f95b4073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:09.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.155+0000 7f95b9200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95b4100e00 0x7f95b4073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:09.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.155+0000 7f95b9200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b4073ad0 con 0x7f95b4100460 2024-05-27T09:39:09.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.155+0000 7f95b9200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b4073c10 con 0x7f95b4100e00 2024-05-27T09:39:09.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.156+0000 7f95b1e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b4100460 0x7f95b4073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:09.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.156+0000 7f95b1e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b4100460 0x7f95b4073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60748/0 (socket says 172.21.15.119:60748) 2024-05-27T09:39:09.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.156+0000 7f95b1e00700 1 -- 172.21.15.119:0/676206638 learned_addr learned my addr 172.21.15.119:0/676206638 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:39:09.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.156+0000 7f95b1e00700 1 -- 172.21.15.119:0/676206638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95b4100e00 msgr2=0x7f95b4073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:09.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.156+0000 7f95b1400700 1 --2- 172.21.15.119:0/676206638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95b4100e00 0x7f95b4073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:09.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.156+0000 7f95b1e00700 1 --2- 172.21.15.119:0/676206638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95b4100e00 0x7f95b4073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:09.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.156+0000 7f95b1e00700 1 -- 172.21.15.119:0/676206638 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a40095f0 con 0x7f95b4100460 2024-05-27T09:39:09.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.156+0000 7f95b1e00700 1 --2- 172.21.15.119:0/676206638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b4100460 0x7f95b4073050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f95a4000c00 tx=0x7f95a400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:09.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.157+0000 7f95aaa00700 1 -- 172.21.15.119:0/676206638 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95a401d070 con 0x7f95b4100460 2024-05-27T09:39:09.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.157+0000 7f95aaa00700 1 -- 172.21.15.119:0/676206638 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95a4005dc0 con 0x7f95b4100460 2024-05-27T09:39:09.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.157+0000 7f95aaa00700 1 -- 172.21.15.119:0/676206638 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95a4017400 con 0x7f95b4100460 2024-05-27T09:39:09.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.157+0000 7f95b9200700 1 -- 172.21.15.119:0/676206638 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b4073e90 con 0x7f95b4100460 2024-05-27T09:39:09.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.157+0000 7f95b9200700 1 -- 172.21.15.119:0/676206638 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b41a22e0 con 0x7f95b4100460 2024-05-27T09:39:09.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.158+0000 7f95b9200700 1 -- 172.21.15.119:0/676206638 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b4108770 con 0x7f95b4100460 2024-05-27T09:39:09.160 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.159+0000 7f95aaa00700 1 -- 172.21.15.119:0/676206638 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f95a400f7f0 con 0x7f95b4100460 2024-05-27T09:39:09.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.160+0000 7f95aaa00700 1 --2- 172.21.15.119:0/676206638 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9598078a40 0x7f959807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:09.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.160+0000 7f95aaa00700 1 -- 172.21.15.119:0/676206638 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f95a409c1f0 con 0x7f95b4100460 2024-05-27T09:39:09.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.160+0000 7f95b1400700 1 --2- 172.21.15.119:0/676206638 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9598078a40 0x7f959807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:09.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.161+0000 7f95b1400700 1 --2- 172.21.15.119:0/676206638 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9598078a40 0x7f959807aef0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f95a0005950 tx=0x7f95a00058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:09.166 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.165+0000 7f95aaa00700 1 -- 172.21.15.119:0/676206638 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f95a4063e70 con 0x7f95b4100460 2024-05-27T09:39:09.449 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.448+0000 7f95b9200700 1 -- 172.21.15.119:0/676206638 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f95b404ea50 con 0x7f95b4100460 2024-05-27T09:39:09.450 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.449+0000 7f95aaa00700 1 -- 172.21.15.119:0/676206638 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1889 (secure 0 0 0) 0x7f95a4063810 con 0x7f95b4100460 2024-05-27T09:39:09.451 INFO:teuthology.orchestra.run.smithi119.stdout:e22 2024-05-27T09:39:09.451 INFO:teuthology.orchestra.run.smithi119.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T09:39:09.451 INFO:teuthology.orchestra.run.smithi119.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:39:09.451 INFO:teuthology.orchestra.run.smithi119.stdout:legacy client fscid: 1 2024-05-27T09:39:09.451 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:39:09.451 INFO:teuthology.orchestra.run.smithi119.stdout:Filesystem 'cephfs' (1) 2024-05-27T09:39:09.451 INFO:teuthology.orchestra.run.smithi119.stdout:fs_name cephfs 2024-05-27T09:39:09.451 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 22 2024-05-27T09:39:09.452 INFO:teuthology.orchestra.run.smithi119.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-27T09:39:09.452 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-05-27T09:25:53.387388+0000 2024-05-27T09:39:09.452 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2024-05-27T09:34:01.250304+0000 2024-05-27T09:39:09.452 INFO:teuthology.orchestra.run.smithi119.stdout:tableserver 0 2024-05-27T09:39:09.452 INFO:teuthology.orchestra.run.smithi119.stdout:root 0 2024-05-27T09:39:09.452 INFO:teuthology.orchestra.run.smithi119.stdout:session_timeout 60 2024-05-27T09:39:09.452 INFO:teuthology.orchestra.run.smithi119.stdout:session_autoclose 300 2024-05-27T09:39:09.452 INFO:teuthology.orchestra.run.smithi119.stdout:max_file_size 1099511627776 2024-05-27T09:39:09.452 INFO:teuthology.orchestra.run.smithi119.stdout:max_xattr_size 65536 2024-05-27T09:39:09.452 INFO:teuthology.orchestra.run.smithi119.stdout:required_client_features {} 2024-05-27T09:39:09.452 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure 0 2024-05-27T09:39:09.452 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure_osd_epoch 46 2024-05-27T09:39:09.452 INFO:teuthology.orchestra.run.smithi119.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:39:09.453 INFO:teuthology.orchestra.run.smithi119.stdout:max_mds 2 2024-05-27T09:39:09.453 INFO:teuthology.orchestra.run.smithi119.stdout:in 0,1 2024-05-27T09:39:09.453 INFO:teuthology.orchestra.run.smithi119.stdout:up {0=14472,1=24241} 2024-05-27T09:39:09.453 INFO:teuthology.orchestra.run.smithi119.stdout:failed 2024-05-27T09:39:09.453 INFO:teuthology.orchestra.run.smithi119.stdout:damaged 2024-05-27T09:39:09.453 INFO:teuthology.orchestra.run.smithi119.stdout:stopped 2024-05-27T09:39:09.453 INFO:teuthology.orchestra.run.smithi119.stdout:data_pools [3] 2024-05-27T09:39:09.453 INFO:teuthology.orchestra.run.smithi119.stdout:metadata_pool 2 2024-05-27T09:39:09.453 INFO:teuthology.orchestra.run.smithi119.stdout:inline_data enabled 2024-05-27T09:39:09.453 INFO:teuthology.orchestra.run.smithi119.stdout:balancer 2024-05-27T09:39:09.453 INFO:teuthology.orchestra.run.smithi119.stdout:bal_rank_mask -1 2024-05-27T09:39:09.453 INFO:teuthology.orchestra.run.smithi119.stdout:standby_count_wanted 1 2024-05-27T09:39:09.453 INFO:teuthology.orchestra.run.smithi119.stdout:qdb_cluster leader: 0 members: 2024-05-27T09:39:09.454 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.ppmzzk{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:39:09.454 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.ndfkhd{1:24241} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/897058594,v1:172.21.15.138:6825/897058594] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:39:09.454 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:39:09.454 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:39:09.454 INFO:teuthology.orchestra.run.smithi119.stdout:Standby daemons: 2024-05-27T09:39:09.454 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:39:09.454 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.nghcey{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.119:6826/413736191,v1:172.21.15.119:6827/413736191] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:39:09.454 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.alszpk{-1:24253} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/2345607053,v1:172.21.15.138:6827/2345607053] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:39:09.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.452+0000 7f95b9200700 1 -- 172.21.15.119:0/676206638 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9598078a40 msgr2=0x7f959807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:09.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.452+0000 7f95b9200700 1 --2- 172.21.15.119:0/676206638 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9598078a40 0x7f959807aef0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f95a0005950 tx=0x7f95a00058e0 comp rx=0 tx=0).stop 2024-05-27T09:39:09.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.452+0000 7f95b9200700 1 -- 172.21.15.119:0/676206638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b4100460 msgr2=0x7f95b4073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:09.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.452+0000 7f95b9200700 1 --2- 172.21.15.119:0/676206638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b4100460 0x7f95b4073050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f95a4000c00 tx=0x7f95a400be20 comp rx=0 tx=0).stop 2024-05-27T09:39:09.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.452+0000 7f95b9200700 1 -- 172.21.15.119:0/676206638 shutdown_connections 2024-05-27T09:39:09.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.452+0000 7f95b9200700 1 --2- 172.21.15.119:0/676206638 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9598078a40 0x7f959807aef0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:09.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.452+0000 7f95b9200700 1 --2- 172.21.15.119:0/676206638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b4100460 0x7f95b4073050 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:09.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.452+0000 7f95b9200700 1 --2- 172.21.15.119:0/676206638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95b4100e00 0x7f95b4073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:09.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.452+0000 7f95b9200700 1 -- 172.21.15.119:0/676206638 >> 172.21.15.119:0/676206638 conn(0x7f95b40fbd20 msgr2=0x7f95b40fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:09.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.453+0000 7f95b9200700 1 -- 172.21.15.119:0/676206638 shutdown_connections 2024-05-27T09:39:09.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.453+0000 7f95b9200700 1 -- 172.21.15.119:0/676206638 wait complete. 2024-05-27T09:39:09.456 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 22 2024-05-27T09:39:09.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.611+0000 7f310be00700 1 -- 172.21.15.119:0/3593506674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f310c10b610 msgr2=0x7f310c10b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:09.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.611+0000 7f310be00700 1 --2- 172.21.15.119:0/3593506674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f310c10b610 0x7f310c10b9e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f30fc009910 tx=0x7f30fc009c20 comp rx=0 tx=0).stop 2024-05-27T09:39:09.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.611+0000 7f310be00700 1 -- 172.21.15.119:0/3593506674 shutdown_connections 2024-05-27T09:39:09.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.611+0000 7f310be00700 1 --2- 172.21.15.119:0/3593506674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310c106b80 0x7f310c106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:09.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.611+0000 7f310be00700 1 --2- 172.21.15.119:0/3593506674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f310c10b610 0x7f310c10b9e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:09.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.611+0000 7f310be00700 1 -- 172.21.15.119:0/3593506674 >> 172.21.15.119:0/3593506674 conn(0x7f310c075df0 msgr2=0x7f310c078200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:09.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.611+0000 7f310be00700 1 -- 172.21.15.119:0/3593506674 shutdown_connections 2024-05-27T09:39:09.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.611+0000 7f310be00700 1 -- 172.21.15.119:0/3593506674 wait complete. 2024-05-27T09:39:09.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.612+0000 7f310be00700 1 Processor -- start 2024-05-27T09:39:09.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.613+0000 7f310be00700 1 -- start start 2024-05-27T09:39:09.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.613+0000 7f310be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f310c106b80 0x7f310c19c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:09.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.613+0000 7f310be00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310c10b610 0x7f310c19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:09.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.613+0000 7f310be00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c19cd10 con 0x7f310c106b80 2024-05-27T09:39:09.615 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.613+0000 7f310be00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c1a0aa0 con 0x7f310c10b610 2024-05-27T09:39:09.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.613+0000 7f310aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f310c106b80 0x7f310c19c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:09.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.613+0000 7f310aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f310c106b80 0x7f310c19c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60776/0 (socket says 172.21.15.119:60776) 2024-05-27T09:39:09.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.613+0000 7f310aa00700 1 -- 172.21.15.119:0/3818202559 learned_addr learned my addr 172.21.15.119:0/3818202559 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:39:09.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.614+0000 7f310a000700 1 --2- 172.21.15.119:0/3818202559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310c10b610 0x7f310c19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:09.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.614+0000 7f310aa00700 1 -- 172.21.15.119:0/3818202559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310c10b610 msgr2=0x7f310c19c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:09.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.614+0000 7f310aa00700 1 --2- 172.21.15.119:0/3818202559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310c10b610 0x7f310c19c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:09.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.614+0000 7f310aa00700 1 -- 172.21.15.119:0/3818202559 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30fc0095f0 con 0x7f310c106b80 2024-05-27T09:39:09.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.614+0000 7f310a000700 1 --2- 172.21.15.119:0/3818202559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310c10b610 0x7f310c19c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:39:09.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.614+0000 7f310aa00700 1 --2- 172.21.15.119:0/3818202559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f310c106b80 0x7f310c19c0f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f30fc005f50 tx=0x7f30fc00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:09.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.614+0000 7f3110a00700 1 -- 172.21.15.119:0/3818202559 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30fc01d070 con 0x7f310c106b80 2024-05-27T09:39:09.619 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.614+0000 7f3110a00700 1 -- 172.21.15.119:0/3818202559 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f30fc00b610 con 0x7f310c106b80 2024-05-27T09:39:09.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.614+0000 7f3110a00700 1 -- 172.21.15.119:0/3818202559 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30fc022470 con 0x7f310c106b80 2024-05-27T09:39:09.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.615+0000 7f310be00700 1 -- 172.21.15.119:0/3818202559 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310c1a0d20 con 0x7f310c106b80 2024-05-27T09:39:09.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.615+0000 7f310be00700 1 -- 172.21.15.119:0/3818202559 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310c1a1290 con 0x7f310c106b80 2024-05-27T09:39:09.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.616+0000 7f310be00700 1 -- 172.21.15.119:0/3818202559 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f310c04f2a0 con 0x7f310c106b80 2024-05-27T09:39:09.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.617+0000 7f3110a00700 1 -- 172.21.15.119:0/3818202559 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f30fc00fca0 con 0x7f310c106b80 2024-05-27T09:39:09.620 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.618+0000 7f3110a00700 1 --2- 172.21.15.119:0/3818202559 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f30f0078a40 0x7f30f007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:09.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.618+0000 7f3110a00700 1 -- 172.21.15.119:0/3818202559 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f30fc09c110 con 0x7f310c106b80 2024-05-27T09:39:09.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.618+0000 7f310a000700 1 --2- 172.21.15.119:0/3818202559 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f30f0078a40 0x7f30f007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:09.621 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.619+0000 7f310a000700 1 --2- 172.21.15.119:0/3818202559 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f30f0078a40 0x7f30f007aef0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f310c19d710 tx=0x7f30f8009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:09.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.623+0000 7f3110a00700 1 -- 172.21.15.119:0/3818202559 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f30fc063d70 con 0x7f310c106b80 2024-05-27T09:39:09.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:09 smithi119 ceph-mon[88269]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:39:09.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:09 smithi119 ceph-mon[88269]: osdmap e74: 6 total, 6 up, 6 in 2024-05-27T09:39:09.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:09 smithi119 ceph-mon[88269]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:39:09.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:09 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/1289471420' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:39:09.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.861+0000 7f310be00700 1 -- 172.21.15.119:0/3818202559 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f310c1a17b0 con 0x7f30f0078a40 2024-05-27T09:39:09.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.864+0000 7f3110a00700 1 -- 172.21.15.119:0/3818202559 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f310c1a17b0 con 0x7f30f0078a40 2024-05-27T09:39:09.866 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:39:09.866 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:39:09.866 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:39:09.866 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-27T09:39:09.866 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [ 2024-05-27T09:39:09.866 INFO:teuthology.orchestra.run.smithi119.stdout: "crash", 2024-05-27T09:39:09.866 INFO:teuthology.orchestra.run.smithi119.stdout: "mon", 2024-05-27T09:39:09.866 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr" 2024-05-27T09:39:09.866 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-05-27T09:39:09.866 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "11/23 daemons upgraded", 2024-05-27T09:39:09.866 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "Currently upgrading osd daemons", 2024-05-27T09:39:09.867 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:39:09.867 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:39:09.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.866+0000 7f310be00700 1 -- 172.21.15.119:0/3818202559 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f30f0078a40 msgr2=0x7f30f007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:09.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.866+0000 7f310be00700 1 --2- 172.21.15.119:0/3818202559 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f30f0078a40 0x7f30f007aef0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f310c19d710 tx=0x7f30f8009260 comp rx=0 tx=0).stop 2024-05-27T09:39:09.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.866+0000 7f310be00700 1 -- 172.21.15.119:0/3818202559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f310c106b80 msgr2=0x7f310c19c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:09.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.866+0000 7f310be00700 1 --2- 172.21.15.119:0/3818202559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f310c106b80 0x7f310c19c0f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f30fc005f50 tx=0x7f30fc00f740 comp rx=0 tx=0).stop 2024-05-27T09:39:09.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.866+0000 7f310be00700 1 -- 172.21.15.119:0/3818202559 shutdown_connections 2024-05-27T09:39:09.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.866+0000 7f310be00700 1 --2- 172.21.15.119:0/3818202559 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f30f0078a40 0x7f30f007aef0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:09.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.866+0000 7f310be00700 1 --2- 172.21.15.119:0/3818202559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f310c106b80 0x7f310c19c0f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:09.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.866+0000 7f310be00700 1 --2- 172.21.15.119:0/3818202559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310c10b610 0x7f310c19c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:09.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.866+0000 7f310be00700 1 -- 172.21.15.119:0/3818202559 >> 172.21.15.119:0/3818202559 conn(0x7f310c075df0 msgr2=0x7f310c077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:09.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.867+0000 7f310be00700 1 -- 172.21.15.119:0/3818202559 shutdown_connections 2024-05-27T09:39:09.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:09.867+0000 7f310be00700 1 -- 172.21.15.119:0/3818202559 wait complete. 2024-05-27T09:39:09.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:09 smithi138 ceph-mon[70004]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:39:09.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:09 smithi138 ceph-mon[70004]: osdmap e74: 6 total, 6 up, 6 in 2024-05-27T09:39:09.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:09 smithi138 ceph-mon[70004]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:39:09.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:09 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/1289471420' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:39:10.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.009+0000 7f0fb1c00700 1 -- 172.21.15.119:0/1138753612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fac100d50 msgr2=0x7f0fac104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:10.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.009+0000 7f0fb1c00700 1 --2- 172.21.15.119:0/1138753612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fac100d50 0x7f0fac104d20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009910 tx=0x7f0f98009c20 comp rx=0 tx=0).stop 2024-05-27T09:39:10.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.009+0000 7f0fb1c00700 1 -- 172.21.15.119:0/1138753612 shutdown_connections 2024-05-27T09:39:10.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.009+0000 7f0fb1c00700 1 --2- 172.21.15.119:0/1138753612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fac100d50 0x7f0fac104d20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:10.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.009+0000 7f0fb1c00700 1 --2- 172.21.15.119:0/1138753612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fac1003b0 0x7f0fac100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:10.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.009+0000 7f0fb1c00700 1 -- 172.21.15.119:0/1138753612 >> 172.21.15.119:0/1138753612 conn(0x7f0fac0fbc30 msgr2=0x7f0fac0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:10.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.009+0000 7f0fb1c00700 1 -- 172.21.15.119:0/1138753612 shutdown_connections 2024-05-27T09:39:10.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.009+0000 7f0fb1c00700 1 -- 172.21.15.119:0/1138753612 wait complete. 2024-05-27T09:39:10.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.011+0000 7f0fb1c00700 1 Processor -- start 2024-05-27T09:39:10.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.011+0000 7f0fb1c00700 1 -- start start 2024-05-27T09:39:10.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.011+0000 7f0fb1c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fac1003b0 0x7f0fac073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:10.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.011+0000 7f0fb1c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fac100d50 0x7f0fac073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:10.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.011+0000 7f0fb1c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac073ad0 con 0x7f0fac1003b0 2024-05-27T09:39:10.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.011+0000 7f0fb1c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac073c10 con 0x7f0fac100d50 2024-05-27T09:39:10.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.012+0000 7f0faaa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fac1003b0 0x7f0fac073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:10.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.012+0000 7f0faaa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fac1003b0 0x7f0fac073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60792/0 (socket says 172.21.15.119:60792) 2024-05-27T09:39:10.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.012+0000 7f0faaa00700 1 -- 172.21.15.119:0/3450535067 learned_addr learned my addr 172.21.15.119:0/3450535067 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:39:10.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.012+0000 7f0faa000700 1 --2- 172.21.15.119:0/3450535067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fac100d50 0x7f0fac073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:10.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.012+0000 7f0faaa00700 1 -- 172.21.15.119:0/3450535067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fac100d50 msgr2=0x7f0fac073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:10.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.012+0000 7f0faaa00700 1 --2- 172.21.15.119:0/3450535067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fac100d50 0x7f0fac073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:10.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.012+0000 7f0faaa00700 1 -- 172.21.15.119:0/3450535067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f980095f0 con 0x7f0fac1003b0 2024-05-27T09:39:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.013+0000 7f0faaa00700 1 --2- 172.21.15.119:0/3450535067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fac1003b0 0x7f0fac073050 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00c8a0 tx=0x7f0f9c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.013+0000 7f0fa3400700 1 -- 172.21.15.119:0/3450535067 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f9c0078e0 con 0x7f0fac1003b0 2024-05-27T09:39:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.013+0000 7f0fa3400700 1 -- 172.21.15.119:0/3450535067 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f9c00f450 con 0x7f0fac1003b0 2024-05-27T09:39:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.013+0000 7f0fb1c00700 1 -- 172.21.15.119:0/3450535067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fac1a1f90 con 0x7f0fac1003b0 2024-05-27T09:39:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.013+0000 7f0fa3400700 1 -- 172.21.15.119:0/3450535067 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f9c00e5e0 con 0x7f0fac1003b0 2024-05-27T09:39:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.013+0000 7f0fb1c00700 1 -- 172.21.15.119:0/3450535067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fac1a22f0 con 0x7f0fac1003b0 2024-05-27T09:39:10.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.014+0000 7f0fb1c00700 1 -- 172.21.15.119:0/3450535067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fac108720 con 0x7f0fac1003b0 2024-05-27T09:39:10.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.016+0000 7f0fa3400700 1 -- 172.21.15.119:0/3450535067 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f0f9c00f5c0 con 0x7f0fac1003b0 2024-05-27T09:39:10.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.017+0000 7f0fa3400700 1 --2- 172.21.15.119:0/3450535067 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0f90078a40 0x7f0f9007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:10.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.017+0000 7f0fa3400700 1 -- 172.21.15.119:0/3450535067 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0f9c09ad10 con 0x7f0fac1003b0 2024-05-27T09:39:10.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.017+0000 7f0faa000700 1 --2- 172.21.15.119:0/3450535067 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0f90078a40 0x7f0f9007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:10.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.018+0000 7f0faa000700 1 --2- 172.21.15.119:0/3450535067 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0f90078a40 0x7f0f9007aef0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0747d0 tx=0x7f0f9801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:10.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.022+0000 7f0fa3400700 1 -- 172.21.15.119:0/3450535067 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f0f9c0628f0 con 0x7f0fac1003b0 2024-05-27T09:39:10.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.351+0000 7f0fb1c00700 1 -- 172.21.15.119:0/3450535067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0fac02cc30 con 0x7f0fac1003b0 2024-05-27T09:39:10.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.352+0000 7f0fa3400700 1 -- 172.21.15.119:0/3450535067 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+987 (secure 0 0 0) 0x7f0f9c062290 con 0x7f0fac1003b0 2024-05-27T09:39:10.355 INFO:teuthology.orchestra.run.smithi119.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 49/381 objects degraded (12.861%), 11 pgs degraded 2024-05-27T09:39:10.355 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:39:10.355 INFO:teuthology.orchestra.run.smithi119.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:39:10.355 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 49/381 objects degraded (12.861%), 11 pgs degraded 2024-05-27T09:39:10.355 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.5 is active+undersized+degraded, acting [3,0] 2024-05-27T09:39:10.355 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.7 is active+undersized+degraded, acting [3,2] 2024-05-27T09:39:10.355 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.b is active+undersized+degraded, acting [3,5] 2024-05-27T09:39:10.355 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.f is active+undersized+degraded, acting [0,5] 2024-05-27T09:39:10.355 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.11 is active+undersized+degraded, acting [3,1] 2024-05-27T09:39:10.356 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.13 is active+undersized+degraded, acting [0,2] 2024-05-27T09:39:10.356 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.14 is active+undersized+degraded, acting [3,5] 2024-05-27T09:39:10.356 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.19 is active+undersized+degraded, acting [0,2] 2024-05-27T09:39:10.356 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.1a is active+undersized+degraded, acting [3,5] 2024-05-27T09:39:10.356 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.1c is active+undersized+degraded, acting [5,2] 2024-05-27T09:39:10.356 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.1f is active+undersized+degraded, acting [0,3] 2024-05-27T09:39:10.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.355+0000 7f0fb1c00700 1 -- 172.21.15.119:0/3450535067 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0f90078a40 msgr2=0x7f0f9007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:10.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.355+0000 7f0fb1c00700 1 --2- 172.21.15.119:0/3450535067 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0f90078a40 0x7f0f9007aef0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0747d0 tx=0x7f0f9801a040 comp rx=0 tx=0).stop 2024-05-27T09:39:10.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.355+0000 7f0fb1c00700 1 -- 172.21.15.119:0/3450535067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fac1003b0 msgr2=0x7f0fac073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:10.357 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.355+0000 7f0fb1c00700 1 --2- 172.21.15.119:0/3450535067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fac1003b0 0x7f0fac073050 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00c8a0 tx=0x7f0f9c00cbb0 comp rx=0 tx=0).stop 2024-05-27T09:39:10.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.356+0000 7f0fb1c00700 1 -- 172.21.15.119:0/3450535067 shutdown_connections 2024-05-27T09:39:10.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.356+0000 7f0fb1c00700 1 --2- 172.21.15.119:0/3450535067 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0f90078a40 0x7f0f9007aef0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:10.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.356+0000 7f0fb1c00700 1 --2- 172.21.15.119:0/3450535067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fac1003b0 0x7f0fac073050 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:10.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.356+0000 7f0fb1c00700 1 --2- 172.21.15.119:0/3450535067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fac100d50 0x7f0fac073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:10.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.356+0000 7f0fb1c00700 1 -- 172.21.15.119:0/3450535067 >> 172.21.15.119:0/3450535067 conn(0x7f0fac0fbc30 msgr2=0x7f0fac0fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:10.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.356+0000 7f0fb1c00700 1 -- 172.21.15.119:0/3450535067 shutdown_connections 2024-05-27T09:39:10.358 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:10.356+0000 7f0fb1c00700 1 -- 172.21.15.119:0/3450535067 wait complete. 2024-05-27T09:39:10.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:10 smithi119 ceph-mon[88269]: pgmap v169: 65 pgs: 4 peering, 11 active+undersized+degraded, 11 active+undersized, 39 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 49/381 objects degraded (12.861%) 2024-05-27T09:39:10.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:10 smithi119 ceph-mon[88269]: Health check update: Degraded data redundancy: 49/381 objects degraded (12.861%), 11 pgs degraded (PG_DEGRADED) 2024-05-27T09:39:10.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:10 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/676206638' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:39:10.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:10 smithi119 ceph-mon[88269]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:39:10.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:10 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/3450535067' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:39:10.752 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:10 smithi138 ceph-mon[70004]: pgmap v169: 65 pgs: 4 peering, 11 active+undersized+degraded, 11 active+undersized, 39 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 49/381 objects degraded (12.861%) 2024-05-27T09:39:10.752 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:10 smithi138 ceph-mon[70004]: Health check update: Degraded data redundancy: 49/381 objects degraded (12.861%), 11 pgs degraded (PG_DEGRADED) 2024-05-27T09:39:10.752 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:10 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/676206638' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:39:10.752 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:10 smithi138 ceph-mon[70004]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:39:10.752 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:10 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/3450535067' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:39:12.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:12.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:12.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:39:12.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:39:12.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:12.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:12 smithi119 ceph-mon[88269]: pgmap v170: 65 pgs: 4 peering, 8 active+undersized+degraded, 8 active+undersized, 45 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 34/381 objects degraded (8.924%) 2024-05-27T09:39:12.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:39:12.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:39:12.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:39:12.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:39:12.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-27T09:39:12.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:12 smithi119 ceph-mon[88269]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-27T09:39:12.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:12 smithi119 ceph-mon[88269]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-05-27T09:39:12.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:12.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:12.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:39:12.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:39:12.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:12.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:12 smithi138 ceph-mon[70004]: pgmap v170: 65 pgs: 4 peering, 8 active+undersized+degraded, 8 active+undersized, 45 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 34/381 objects degraded (8.924%) 2024-05-27T09:39:12.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:39:12.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:39:12.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:39:12.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:39:12.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-27T09:39:12.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:12 smithi138 ceph-mon[70004]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-27T09:39:12.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:12 smithi138 ceph-mon[70004]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-05-27T09:39:13.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:13 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:14.419 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:13 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:39:14.420 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:13 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:14.420 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:13 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:39:14.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:14 smithi138 ceph-mon[70004]: pgmap v171: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:14.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:14 smithi119 ceph-mon[88269]: pgmap v171: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:15.489 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:15 smithi119 ceph-mon[88269]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 34/381 objects degraded (8.924%), 8 pgs degraded) 2024-05-27T09:39:15.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:15 smithi138 ceph-mon[70004]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 34/381 objects degraded (8.924%), 8 pgs degraded) 2024-05-27T09:39:16.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:16 smithi138 ceph-mon[70004]: pgmap v172: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:16.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:16 smithi119 ceph-mon[88269]: pgmap v172: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:18.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:18 smithi119 ceph-mon[88269]: pgmap v173: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:18.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:18 smithi138 ceph-mon[70004]: pgmap v173: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:20.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:20 smithi119 ceph-mon[88269]: pgmap v174: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:20.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:20 smithi138 ceph-mon[70004]: pgmap v174: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:22.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:22 smithi119 ceph-mon[88269]: pgmap v175: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:22.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:22 smithi138 ceph-mon[70004]: pgmap v175: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:23.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:23 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:23.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:23 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:24.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:24 smithi138 ceph-mon[70004]: pgmap v176: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:24.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:24 smithi119 ceph-mon[88269]: pgmap v176: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:26.754 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:26 smithi138 ceph-mon[70004]: pgmap v177: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:26.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:26 smithi119 ceph-mon[88269]: pgmap v177: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:27.725 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:27 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-27T09:39:27.725 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:27 smithi138 ceph-mon[70004]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-27T09:39:27.725 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:27 smithi138 ceph-mon[70004]: Upgrade: osd.5 is safe to restart 2024-05-27T09:39:27.725 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:27 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:27.725 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:27 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:39:27.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:27 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-27T09:39:27.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:27 smithi119 ceph-mon[88269]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-27T09:39:27.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:27 smithi119 ceph-mon[88269]: Upgrade: osd.5 is safe to restart 2024-05-27T09:39:27.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:27 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:27.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:27 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:39:28.567 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:28 smithi138 ceph-mon[70004]: pgmap v178: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:28.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:28 smithi119 ceph-mon[88269]: pgmap v178: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:30.332 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:30 smithi138 ceph-mon[70004]: Upgrade: Updating osd.5 2024-05-27T09:39:30.333 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:30 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:30.333 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:30 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-27T09:39:30.333 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:30 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:39:30.333 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:30 smithi138 ceph-mon[70004]: Deploying daemon osd.5 on smithi138 2024-05-27T09:39:30.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:30 smithi119 ceph-mon[88269]: Upgrade: Updating osd.5 2024-05-27T09:39:30.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:30 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:30.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:30 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-27T09:39:30.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:30 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:39:30.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:30 smithi119 ceph-mon[88269]: Deploying daemon osd.5 on smithi138 2024-05-27T09:39:31.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:31 smithi138 ceph-mon[70004]: pgmap v179: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:31.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:31 smithi119 ceph-mon[88269]: pgmap v179: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:33.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:33 smithi138 ceph-mon[70004]: pgmap v180: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:33.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:33 smithi119 ceph-mon[88269]: pgmap v180: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:34.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:34 smithi138 systemd[1]: Stopping Ceph osd.5 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:39:34.886 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:34 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[50331]: 2024-05-27T09:39:34.525+0000 7fb6ec800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T09:39:34.886 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:34 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[50331]: 2024-05-27T09:39:34.526+0000 7fb6ec800700 -1 osd.5 74 *** Got signal Terminated *** 2024-05-27T09:39:34.886 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:34 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[50331]: 2024-05-27T09:39:34.526+0000 7fb6ec800700 -1 osd.5 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T09:39:35.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:35 smithi138 ceph-mon[70004]: pgmap v181: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:35.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:35 smithi138 ceph-mon[70004]: osd.5 marked itself down and dead 2024-05-27T09:39:35.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:35 smithi119 ceph-mon[88269]: pgmap v181: 65 pgs: 65 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:35.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:35 smithi119 ceph-mon[88269]: osd.5 marked itself down and dead 2024-05-27T09:39:36.081 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:35 smithi138 podman[89197]: 2024-05-27 09:39:35.773291035 +0000 UTC m=+1.397758782 container died 9f25646b37cd46c6a429d2bd32a0f720214656bd96ff3297c85aedc1bbf93837 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T09:39:36.338 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:36 smithi138 ceph-mon[70004]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-27T09:39:36.339 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:36 smithi138 ceph-mon[70004]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-05-27T09:39:36.339 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:36 smithi138 ceph-mon[70004]: osdmap e75: 6 total, 5 up, 6 in 2024-05-27T09:39:36.339 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:36 smithi138 ceph-mon[70004]: pgmap v183: 65 pgs: 13 stale+active+clean, 52 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:36.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:36 smithi119 ceph-mon[88269]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-27T09:39:36.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:36 smithi119 ceph-mon[88269]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-05-27T09:39:36.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:36 smithi119 ceph-mon[88269]: osdmap e75: 6 total, 5 up, 6 in 2024-05-27T09:39:36.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:36 smithi119 ceph-mon[88269]: pgmap v183: 65 pgs: 13 stale+active+clean, 52 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:36.636 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:36 smithi138 podman[89197]: 2024-05-27 09:39:36.336116373 +0000 UTC m=+1.960584087 container cleanup 9f25646b37cd46c6a429d2bd32a0f720214656bd96ff3297c85aedc1bbf93837 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.vendor=CentOS, RELEASE=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20231212, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308) 2024-05-27T09:39:36.636 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:36 smithi138 bash[89197]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5 2024-05-27T09:39:36.636 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:36 smithi138 podman[89214]: 2024-05-27 09:39:36.579241349 +0000 UTC m=+0.788767881 container remove 9f25646b37cd46c6a429d2bd32a0f720214656bd96ff3297c85aedc1bbf93837 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.vendor=CentOS, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20231212) 2024-05-27T09:39:37.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:37 smithi138 ceph-mon[70004]: osdmap e76: 6 total, 5 up, 6 in 2024-05-27T09:39:37.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:37 smithi119 ceph-mon[88269]: osdmap e76: 6 total, 5 up, 6 in 2024-05-27T09:39:37.726 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:37 smithi138 podman[89303]: 2024-05-27 09:39:37.330503292 +0000 UTC m=+0.034020248 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:39:37.727 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:37 smithi138 podman[89303]: 2024-05-27 09:39:37.438356305 +0000 UTC m=+0.141873332 container create f53b74b705b412f4fa5e63d1432d73f8b420be4bec9532cac5cc1479ec51abac (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521) 2024-05-27T09:39:37.727 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:37 smithi138 podman[89303]: 2024-05-27 09:39:37.725884084 +0000 UTC m=+0.429401084 container init f53b74b705b412f4fa5e63d1432d73f8b420be4bec9532cac5cc1479ec51abac (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-27T09:39:38.118 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:37 smithi138 podman[89303]: 2024-05-27 09:39:37.7392312 +0000 UTC m=+0.442748223 container start f53b74b705b412f4fa5e63d1432d73f8b420be4bec9532cac5cc1479ec51abac (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.build-date=20240521) 2024-05-27T09:39:38.118 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:37 smithi138 podman[89303]: 2024-05-27 09:39:37.858097957 +0000 UTC m=+0.561614994 container attach f53b74b705b412f4fa5e63d1432d73f8b420be4bec9532cac5cc1479ec51abac (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-27T09:39:38.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:38 smithi138 ceph-mon[70004]: pgmap v185: 65 pgs: 9 active+undersized+degraded, 3 peering, 4 stale+active+clean, 8 active+undersized, 41 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/381 objects degraded (10.761%) 2024-05-27T09:39:38.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:38 smithi138 podman[89303]: 2024-05-27 09:39:38.194465421 +0000 UTC m=+0.897982377 container died f53b74b705b412f4fa5e63d1432d73f8b420be4bec9532cac5cc1479ec51abac (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T09:39:38.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:38 smithi119 ceph-mon[88269]: pgmap v185: 65 pgs: 9 active+undersized+degraded, 3 peering, 4 stale+active+clean, 8 active+undersized, 41 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/381 objects degraded (10.761%) 2024-05-27T09:39:39.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:39 smithi138 ceph-mon[70004]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-05-27T09:39:39.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:39 smithi138 ceph-mon[70004]: Health check failed: Degraded data redundancy: 41/381 objects degraded (10.761%), 9 pgs degraded (PG_DEGRADED) 2024-05-27T09:39:39.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:39 smithi138 podman[89330]: 2024-05-27 09:39:39.061876549 +0000 UTC m=+0.851406882 container remove f53b74b705b412f4fa5e63d1432d73f8b420be4bec9532cac5cc1479ec51abac (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-deactivate, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-27T09:39:39.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:39 smithi138 systemd[1]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.5.service: Deactivated successfully. 2024-05-27T09:39:39.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:39 smithi138 systemd[1]: Stopped Ceph osd.5 for b966b06e-1c09-11ef-bc9b-c7b262605968. 2024-05-27T09:39:39.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:39 smithi138 systemd[1]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.5.service: Consumed 1min 43.079s CPU time. 2024-05-27T09:39:39.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:39 smithi119 ceph-mon[88269]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-05-27T09:39:39.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:39 smithi119 ceph-mon[88269]: Health check failed: Degraded data redundancy: 41/381 objects degraded (10.761%), 9 pgs degraded (PG_DEGRADED) 2024-05-27T09:39:39.867 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:39 smithi138 systemd[1]: Starting Ceph osd.5 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:39:40.120 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:39 smithi138 podman[89429]: 2024-05-27 09:39:39.770665909 +0000 UTC m=+0.033324430 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:39:40.120 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:39 smithi138 podman[89429]: 2024-05-27 09:39:39.962872639 +0000 UTC m=+0.225531194 container create 2451026e34ef3ef20290431767a5449d7cbe1cbbe936eece908dd5255e9df20b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-activate, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-27T09:39:40.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:40 smithi138 ceph-mon[70004]: pgmap v186: 65 pgs: 14 active+undersized+degraded, 3 peering, 11 active+undersized, 37 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 57/381 objects degraded (14.961%) 2024-05-27T09:39:40.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:40 smithi138 podman[89429]: 2024-05-27 09:39:40.258843196 +0000 UTC m=+0.521501801 container init 2451026e34ef3ef20290431767a5449d7cbe1cbbe936eece908dd5255e9df20b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-activate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-27T09:39:40.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:40 smithi138 podman[89429]: 2024-05-27 09:39:40.271673388 +0000 UTC m=+0.534331922 container start 2451026e34ef3ef20290431767a5449d7cbe1cbbe936eece908dd5255e9df20b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-activate, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T09:39:40.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:40 smithi138 podman[89429]: 2024-05-27 09:39:40.365178388 +0000 UTC m=+0.627836986 container attach 2451026e34ef3ef20290431767a5449d7cbe1cbbe936eece908dd5255e9df20b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-activate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521) 2024-05-27T09:39:40.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:40 smithi119 ceph-mon[88269]: pgmap v186: 65 pgs: 14 active+undersized+degraded, 3 peering, 11 active+undersized, 37 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 57/381 objects degraded (14.961%) 2024-05-27T09:39:40.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.529+0000 7fbe05600700 1 -- 172.21.15.119:0/351399264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe0010b700 msgr2=0x7fbe0010bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:40.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.529+0000 7fbe05600700 1 --2- 172.21.15.119:0/351399264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe0010b700 0x7fbe0010bad0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0009910 tx=0x7fbdf0009c20 comp rx=0 tx=0).stop 2024-05-27T09:39:40.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.529+0000 7fbe05600700 1 -- 172.21.15.119:0/351399264 shutdown_connections 2024-05-27T09:39:40.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.529+0000 7fbe05600700 1 --2- 172.21.15.119:0/351399264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe0010c010 0x7fbe00106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:40.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.529+0000 7fbe05600700 1 --2- 172.21.15.119:0/351399264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe0010b700 0x7fbe0010bad0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:40.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.529+0000 7fbe05600700 1 -- 172.21.15.119:0/351399264 >> 172.21.15.119:0/351399264 conn(0x7fbe00076000 msgr2=0x7fbe00078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:40.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.529+0000 7fbe05600700 1 -- 172.21.15.119:0/351399264 shutdown_connections 2024-05-27T09:39:40.531 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.529+0000 7fbe05600700 1 -- 172.21.15.119:0/351399264 wait complete. 2024-05-27T09:39:40.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.531+0000 7fbe05600700 1 Processor -- start 2024-05-27T09:39:40.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.531+0000 7fbe05600700 1 -- start start 2024-05-27T09:39:40.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.531+0000 7fbe05600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe0010b700 0x7fbe0019c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:40.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.531+0000 7fbe05600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe0010c010 0x7fbe0019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:40.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.531+0000 7fbe05600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0019ced0 con 0x7fbe0010b700 2024-05-27T09:39:40.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.531+0000 7fbe05600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe001a0c60 con 0x7fbe0010c010 2024-05-27T09:39:40.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.532+0000 7fbdfea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe0010b700 0x7fbe0019c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:40.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.532+0000 7fbdfea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe0010b700 0x7fbe0019c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44226/0 (socket says 172.21.15.119:44226) 2024-05-27T09:39:40.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.532+0000 7fbdfea00700 1 -- 172.21.15.119:0/2777490165 learned_addr learned my addr 172.21.15.119:0/2777490165 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:39:40.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.532+0000 7fbdfe000700 1 --2- 172.21.15.119:0/2777490165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe0010c010 0x7fbe0019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:40.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.532+0000 7fbdfea00700 1 -- 172.21.15.119:0/2777490165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe0010c010 msgr2=0x7fbe0019c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:40.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.532+0000 7fbdfea00700 1 --2- 172.21.15.119:0/2777490165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe0010c010 0x7fbe0019c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:40.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.532+0000 7fbdfea00700 1 -- 172.21.15.119:0/2777490165 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf00095f0 con 0x7fbe0010b700 2024-05-27T09:39:40.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.532+0000 7fbdfea00700 1 --2- 172.21.15.119:0/2777490165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe0010b700 0x7fbe0019c2b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000f690 tx=0x7fbdf000f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:40.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.533+0000 7fbdf7400700 1 -- 172.21.15.119:0/2777490165 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdf001d070 con 0x7fbe0010b700 2024-05-27T09:39:40.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.533+0000 7fbdf7400700 1 -- 172.21.15.119:0/2777490165 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbdf00050d0 con 0x7fbe0010b700 2024-05-27T09:39:40.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.533+0000 7fbe05600700 1 -- 172.21.15.119:0/2777490165 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe001a0ee0 con 0x7fbe0010b700 2024-05-27T09:39:40.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.533+0000 7fbdf7400700 1 -- 172.21.15.119:0/2777490165 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdf0022470 con 0x7fbe0010b700 2024-05-27T09:39:40.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.533+0000 7fbe05600700 1 -- 172.21.15.119:0/2777490165 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe001a1450 con 0x7fbe0010b700 2024-05-27T09:39:40.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.534+0000 7fbe05600700 1 -- 172.21.15.119:0/2777490165 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe0004ea50 con 0x7fbe0010b700 2024-05-27T09:39:40.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.536+0000 7fbdf7400700 1 -- 172.21.15.119:0/2777490165 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fbdf0003680 con 0x7fbe0010b700 2024-05-27T09:39:40.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.537+0000 7fbdf7400700 1 --2- 172.21.15.119:0/2777490165 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fbde4078a40 0x7fbde407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:40.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.537+0000 7fbdf7400700 1 -- 172.21.15.119:0/2777490165 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbdf009c1d0 con 0x7fbe0010b700 2024-05-27T09:39:40.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.538+0000 7fbdfe000700 1 --2- 172.21.15.119:0/2777490165 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fbde4078a40 0x7fbde407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:40.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.538+0000 7fbdfe000700 1 --2- 172.21.15.119:0/2777490165 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fbde4078a40 0x7fbde407aef0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbe0019d8d0 tx=0x7fbdec009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:40.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.543+0000 7fbdf7400700 1 -- 172.21.15.119:0/2777490165 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fbdf0063ee0 con 0x7fbe0010b700 2024-05-27T09:39:40.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.796+0000 7fbe05600700 1 -- 172.21.15.119:0/2777490165 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe001a1970 con 0x7fbde4078a40 2024-05-27T09:39:40.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.798+0000 7fbdf7400700 1 -- 172.21.15.119:0/2777490165 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fbe001a1970 con 0x7fbde4078a40 2024-05-27T09:39:40.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.801+0000 7fbe05600700 1 -- 172.21.15.119:0/2777490165 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fbde4078a40 msgr2=0x7fbde407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:40.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.801+0000 7fbe05600700 1 --2- 172.21.15.119:0/2777490165 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fbde4078a40 0x7fbde407aef0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbe0019d8d0 tx=0x7fbdec009260 comp rx=0 tx=0).stop 2024-05-27T09:39:40.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.801+0000 7fbe05600700 1 -- 172.21.15.119:0/2777490165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe0010b700 msgr2=0x7fbe0019c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:40.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.801+0000 7fbe05600700 1 --2- 172.21.15.119:0/2777490165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe0010b700 0x7fbe0019c2b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000f690 tx=0x7fbdf000f770 comp rx=0 tx=0).stop 2024-05-27T09:39:40.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.801+0000 7fbe05600700 1 -- 172.21.15.119:0/2777490165 shutdown_connections 2024-05-27T09:39:40.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.801+0000 7fbe05600700 1 --2- 172.21.15.119:0/2777490165 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fbde4078a40 0x7fbde407aef0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:40.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.801+0000 7fbe05600700 1 --2- 172.21.15.119:0/2777490165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe0010b700 0x7fbe0019c2b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:40.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.801+0000 7fbe05600700 1 --2- 172.21.15.119:0/2777490165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe0010c010 0x7fbe0019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:40.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.801+0000 7fbe05600700 1 -- 172.21.15.119:0/2777490165 >> 172.21.15.119:0/2777490165 conn(0x7fbe00076000 msgr2=0x7fbe00076880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:40.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.802+0000 7fbe05600700 1 -- 172.21.15.119:0/2777490165 shutdown_connections 2024-05-27T09:39:40.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.802+0000 7fbe05600700 1 -- 172.21.15.119:0/2777490165 wait complete. 2024-05-27T09:39:40.818 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:39:40.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.948+0000 7fc1f0000700 1 -- 172.21.15.119:0/2401345456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1e80730f0 msgr2=0x7fc1e80734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:40.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.948+0000 7fc1f0000700 1 --2- 172.21.15.119:0/2401345456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1e80730f0 0x7fc1e80734c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc009910 tx=0x7fc1dc009c20 comp rx=0 tx=0).stop 2024-05-27T09:39:40.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.948+0000 7fc1f0000700 1 -- 172.21.15.119:0/2401345456 shutdown_connections 2024-05-27T09:39:40.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.948+0000 7fc1f0000700 1 --2- 172.21.15.119:0/2401345456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1e8073a00 0x7fc1e810c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:40.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.948+0000 7fc1f0000700 1 --2- 172.21.15.119:0/2401345456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1e80730f0 0x7fc1e80734c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:40.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.948+0000 7fc1f0000700 1 -- 172.21.15.119:0/2401345456 >> 172.21.15.119:0/2401345456 conn(0x7fc1e80fbc50 msgr2=0x7fc1e80fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:40.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.948+0000 7fc1f0000700 1 -- 172.21.15.119:0/2401345456 shutdown_connections 2024-05-27T09:39:40.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.948+0000 7fc1f0000700 1 -- 172.21.15.119:0/2401345456 wait complete. 2024-05-27T09:39:40.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.949+0000 7fc1f0000700 1 Processor -- start 2024-05-27T09:39:40.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.949+0000 7fc1f0000700 1 -- start start 2024-05-27T09:39:40.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.949+0000 7fc1f0000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1e80730f0 0x7fc1e8197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:40.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.949+0000 7fc1f0000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1e8073a00 0x7fc1e8198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:40.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.950+0000 7fc1f0000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e819c7f0 con 0x7fc1e8073a00 2024-05-27T09:39:40.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.950+0000 7fc1f0000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e819c960 con 0x7fc1e80730f0 2024-05-27T09:39:40.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.950+0000 7fc1ecc00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1e8073a00 0x7fc1e8198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:40.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.950+0000 7fc1ecc00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1e8073a00 0x7fc1e8198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44246/0 (socket says 172.21.15.119:44246) 2024-05-27T09:39:40.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.950+0000 7fc1ecc00700 1 -- 172.21.15.119:0/3684315339 learned_addr learned my addr 172.21.15.119:0/3684315339 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:39:40.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.950+0000 7fc1ed600700 1 --2- 172.21.15.119:0/3684315339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1e80730f0 0x7fc1e8197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:40.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.950+0000 7fc1ecc00700 1 -- 172.21.15.119:0/3684315339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1e80730f0 msgr2=0x7fc1e8197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:40.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.950+0000 7fc1ecc00700 1 --2- 172.21.15.119:0/3684315339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1e80730f0 0x7fc1e8197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:40.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.950+0000 7fc1ecc00700 1 -- 172.21.15.119:0/3684315339 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1dc0095f0 con 0x7fc1e8073a00 2024-05-27T09:39:40.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.951+0000 7fc1ecc00700 1 --2- 172.21.15.119:0/3684315339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1e8073a00 0x7fc1e8198500 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc1d400d8d0 tx=0x7fc1d400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:40.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.951+0000 7fc1e6000700 1 -- 172.21.15.119:0/3684315339 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1d4009940 con 0x7fc1e8073a00 2024-05-27T09:39:40.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.951+0000 7fc1e6000700 1 -- 172.21.15.119:0/3684315339 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1d4010460 con 0x7fc1e8073a00 2024-05-27T09:39:40.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.951+0000 7fc1f0000700 1 -- 172.21.15.119:0/3684315339 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e819cc40 con 0x7fc1e8073a00 2024-05-27T09:39:40.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.951+0000 7fc1e6000700 1 -- 172.21.15.119:0/3684315339 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1d400f5f0 con 0x7fc1e8073a00 2024-05-27T09:39:40.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.951+0000 7fc1f0000700 1 -- 172.21.15.119:0/3684315339 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e819d190 con 0x7fc1e8073a00 2024-05-27T09:39:40.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.952+0000 7fc1f0000700 1 -- 172.21.15.119:0/3684315339 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e8109b70 con 0x7fc1e8073a00 2024-05-27T09:39:40.954 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.953+0000 7fc1e6000700 1 -- 172.21.15.119:0/3684315339 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fc1d40105d0 con 0x7fc1e8073a00 2024-05-27T09:39:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.955+0000 7fc1e6000700 1 --2- 172.21.15.119:0/3684315339 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc1d0078a40 0x7fc1d007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.955+0000 7fc1e6000700 1 -- 172.21.15.119:0/3684315339 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc1d409ada0 con 0x7fc1e8073a00 2024-05-27T09:39:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.955+0000 7fc1ed600700 1 --2- 172.21.15.119:0/3684315339 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc1d0078a40 0x7fc1d007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:40.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.956+0000 7fc1ed600700 1 --2- 172.21.15.119:0/3684315339 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc1d0078a40 0x7fc1d007aef0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc006010 tx=0x7fc1dc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:40.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:40.960+0000 7fc1e6000700 1 -- 172.21.15.119:0/3684315339 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fc1d4062980 con 0x7fc1e8073a00 2024-05-27T09:39:41.198 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.197+0000 7fc1f0000700 1 -- 172.21.15.119:0/3684315339 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1e810f470 con 0x7fc1d0078a40 2024-05-27T09:39:41.200 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.200+0000 7fc1e6000700 1 -- 172.21.15.119:0/3684315339 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fc1e810f470 con 0x7fc1d0078a40 2024-05-27T09:39:41.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.202+0000 7fc1f0000700 1 -- 172.21.15.119:0/3684315339 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc1d0078a40 msgr2=0x7fc1d007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:41.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.202+0000 7fc1f0000700 1 --2- 172.21.15.119:0/3684315339 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc1d0078a40 0x7fc1d007aef0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc006010 tx=0x7fc1dc01a040 comp rx=0 tx=0).stop 2024-05-27T09:39:41.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.202+0000 7fc1f0000700 1 -- 172.21.15.119:0/3684315339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1e8073a00 msgr2=0x7fc1e8198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:41.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.202+0000 7fc1f0000700 1 --2- 172.21.15.119:0/3684315339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1e8073a00 0x7fc1e8198500 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc1d400d8d0 tx=0x7fc1d400dc90 comp rx=0 tx=0).stop 2024-05-27T09:39:41.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.203+0000 7fc1f0000700 1 -- 172.21.15.119:0/3684315339 shutdown_connections 2024-05-27T09:39:41.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.203+0000 7fc1f0000700 1 --2- 172.21.15.119:0/3684315339 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc1d0078a40 0x7fc1d007aef0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:41.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.203+0000 7fc1f0000700 1 --2- 172.21.15.119:0/3684315339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1e80730f0 0x7fc1e8197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:41.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.203+0000 7fc1f0000700 1 --2- 172.21.15.119:0/3684315339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1e8073a00 0x7fc1e8198500 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:41.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.203+0000 7fc1f0000700 1 -- 172.21.15.119:0/3684315339 >> 172.21.15.119:0/3684315339 conn(0x7fc1e80fbc50 msgr2=0x7fc1e8106cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:41.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.203+0000 7fc1f0000700 1 -- 172.21.15.119:0/3684315339 shutdown_connections 2024-05-27T09:39:41.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.203+0000 7fc1f0000700 1 -- 172.21.15.119:0/3684315339 wait complete. 2024-05-27T09:39:41.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.359+0000 7f2cd0000700 1 -- 172.21.15.119:0/2238394364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc8101320 msgr2=0x7f2cc81016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:41.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.359+0000 7f2cd0000700 1 --2- 172.21.15.119:0/2238394364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc8101320 0x7f2cc81016f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009910 tx=0x7f2cbc009c20 comp rx=0 tx=0).stop 2024-05-27T09:39:41.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.359+0000 7f2cd0000700 1 -- 172.21.15.119:0/2238394364 shutdown_connections 2024-05-27T09:39:41.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.359+0000 7f2cd0000700 1 --2- 172.21.15.119:0/2238394364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cc8101c30 0x7f2cc810a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:41.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.359+0000 7f2cd0000700 1 --2- 172.21.15.119:0/2238394364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc8101320 0x7f2cc81016f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:41.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.359+0000 7f2cd0000700 1 -- 172.21.15.119:0/2238394364 >> 172.21.15.119:0/2238394364 conn(0x7f2cc80fab80 msgr2=0x7f2cc80fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:41.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.359+0000 7f2cd0000700 1 -- 172.21.15.119:0/2238394364 shutdown_connections 2024-05-27T09:39:41.361 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.359+0000 7f2cd0000700 1 -- 172.21.15.119:0/2238394364 wait complete. 2024-05-27T09:39:41.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.360+0000 7f2cd0000700 1 Processor -- start 2024-05-27T09:39:41.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.361+0000 7f2cd0000700 1 -- start start 2024-05-27T09:39:41.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.361+0000 7f2cd0000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc8101320 0x7f2cc80fef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:41.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.361+0000 7f2cd0000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cc8101c30 0x7f2cc80ff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:41.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.361+0000 7f2cd0000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc80ffa00 con 0x7f2cc8101320 2024-05-27T09:39:41.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.361+0000 7f2cd0000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc80ffb40 con 0x7f2cc8101c30 2024-05-27T09:39:41.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.361+0000 7f2ccd600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc8101320 0x7f2cc80fef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:41.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.361+0000 7f2ccd600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc8101320 0x7f2cc80fef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44256/0 (socket says 172.21.15.119:44256) 2024-05-27T09:39:41.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.361+0000 7f2ccd600700 1 -- 172.21.15.119:0/2359130712 learned_addr learned my addr 172.21.15.119:0/2359130712 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:39:41.362 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.361+0000 7f2cccc00700 1 --2- 172.21.15.119:0/2359130712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cc8101c30 0x7f2cc80ff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:41.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.362+0000 7f2ccd600700 1 -- 172.21.15.119:0/2359130712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cc8101c30 msgr2=0x7f2cc80ff4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:41.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.362+0000 7f2ccd600700 1 --2- 172.21.15.119:0/2359130712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cc8101c30 0x7f2cc80ff4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:41.363 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.362+0000 7f2ccd600700 1 -- 172.21.15.119:0/2359130712 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc0095f0 con 0x7f2cc8101320 2024-05-27T09:39:41.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.362+0000 7f2ccd600700 1 --2- 172.21.15.119:0/2359130712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc8101320 0x7f2cc80fef80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00f690 tx=0x7f2cbc00f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:41.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.362+0000 7f2cc6000700 1 -- 172.21.15.119:0/2359130712 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cbc01d070 con 0x7f2cc8101320 2024-05-27T09:39:41.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.362+0000 7f2cc6000700 1 -- 172.21.15.119:0/2359130712 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2cbc0050d0 con 0x7f2cc8101320 2024-05-27T09:39:41.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.362+0000 7f2cd0000700 1 -- 172.21.15.119:0/2359130712 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc80ffdc0 con 0x7f2cc8101320 2024-05-27T09:39:41.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.363+0000 7f2cc6000700 1 -- 172.21.15.119:0/2359130712 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cbc022470 con 0x7f2cc8101320 2024-05-27T09:39:41.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.363+0000 7f2cd0000700 1 -- 172.21.15.119:0/2359130712 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc81a2250 con 0x7f2cc8101320 2024-05-27T09:39:41.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.364+0000 7f2cd0000700 1 -- 172.21.15.119:0/2359130712 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc8107800 con 0x7f2cc8101320 2024-05-27T09:39:41.365 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.365+0000 7f2cc6000700 1 -- 172.21.15.119:0/2359130712 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f2cbc003780 con 0x7f2cc8101320 2024-05-27T09:39:41.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.367+0000 7f2cc6000700 1 --2- 172.21.15.119:0/2359130712 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2cb0078a40 0x7f2cb007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:41.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.367+0000 7f2cc6000700 1 -- 172.21.15.119:0/2359130712 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2cbc09c0d0 con 0x7f2cc8101320 2024-05-27T09:39:41.367 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.367+0000 7f2cccc00700 1 --2- 172.21.15.119:0/2359130712 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2cb0078a40 0x7f2cb007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:41.368 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.368+0000 7f2cccc00700 1 --2- 172.21.15.119:0/2359130712 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2cb0078a40 0x7f2cb007aef0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8100770 tx=0x7f2cb4009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:41.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.372+0000 7f2cc6000700 1 -- 172.21.15.119:0/2359130712 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f2cbc063d30 con 0x7f2cc8101320 2024-05-27T09:39:41.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:41 smithi119 ceph-mon[88269]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:39:41.613 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.612+0000 7f2cd0000700 1 -- 172.21.15.119:0/2359130712 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cc802cd40 con 0x7f2cb0078a40 2024-05-27T09:39:41.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:41 smithi138 ceph-mon[70004]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:39:41.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.642+0000 7f2cc6000700 1 -- 172.21.15.119:0/2359130712 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f2cc802cd40 con 0x7f2cb0078a40 2024-05-27T09:39:41.644 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:39:41.644 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (7m) 112s ago 20m 14.3M - 0.25.0 c8568f914cd2 6b90564dcef7 2024-05-27T09:39:41.644 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (21m) 112s ago 21m 8850k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:39:41.644 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (19m) 38s ago 19m 8640k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:39:41.644 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (3m) 112s ago 21m 6669k - 19.0.0-3868-ga6507388 850a06a84d28 011d811ce7fc 2024-05-27T09:39:41.644 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (3m) 38s ago 19m 6665k - 19.0.0-3868-ga6507388 850a06a84d28 b546b3ed1afc 2024-05-27T09:39:41.644 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (6m) 112s ago 20m 74.4M - 9.4.12 2bacad6d85d8 b3abdab08db4 2024-05-27T09:39:41.644 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (13m) 112s ago 13m 18.0M - 18.2.0 dc2bc1663786 93b21713fb1e 2024-05-27T09:39:41.645 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (13m) 112s ago 13m 130M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:39:41.645 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (13m) 38s ago 13m 18.2M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:39:41.645 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (13m) 38s ago 13m 128M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:39:41.645 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (9m) 112s ago 22m 587M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:39:41.645 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (8m) 38s ago 19m 534M - 19.0.0-3868-ga6507388 850a06a84d28 2fc3a80696fa 2024-05-27T09:39:41.645 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (5m) 112s ago 22m 114M 2048M 19.0.0-3868-ga6507388 850a06a84d28 3e5c7a9ad372 2024-05-27T09:39:41.645 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (4m) 38s ago 19m 107M 2048M 19.0.0-3868-ga6507388 850a06a84d28 e4d471274cf1 2024-05-27T09:39:41.645 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (8m) 112s ago 20m 8002k - 1.5.0 0da6a335fe13 709c37815cff 2024-05-27T09:39:41.645 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (8m) 38s ago 19m 7990k - 1.5.0 0da6a335fe13 999047a1a5bf 2024-05-27T09:39:41.645 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (3m) 112s ago 18m 152M 1216M 19.0.0-3868-ga6507388 850a06a84d28 5a0e162d870e 2024-05-27T09:39:41.645 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (2m) 112s ago 17m 167M 1216M 19.0.0-3868-ga6507388 850a06a84d28 effcc7c53dd3 2024-05-27T09:39:41.645 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (2m) 112s ago 17m 78.5M 1216M 19.0.0-3868-ga6507388 850a06a84d28 f0bd34ca416f 2024-05-27T09:39:41.645 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (88s) 38s ago 16m 172M 2240M 19.0.0-3868-ga6507388 850a06a84d28 5e8d3e398c21 2024-05-27T09:39:41.646 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (43s) 38s ago 16m 78.5M 2240M 19.0.0-3868-ga6507388 850a06a84d28 250e443364de 2024-05-27T09:39:41.646 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (15m) 38s ago 15m 226M 2240M 18.2.0 dc2bc1663786 9f25646b37cd 2024-05-27T09:39:41.646 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (7m) 112s ago 20m 71.1M - 2.43.0 a07b618ecd1d cba50454ed21 2024-05-27T09:39:41.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.646+0000 7f2cd0000700 1 -- 172.21.15.119:0/2359130712 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2cb0078a40 msgr2=0x7f2cb007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:41.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.646+0000 7f2cd0000700 1 --2- 172.21.15.119:0/2359130712 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2cb0078a40 0x7f2cb007aef0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8100770 tx=0x7f2cb4009260 comp rx=0 tx=0).stop 2024-05-27T09:39:41.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.646+0000 7f2cd0000700 1 -- 172.21.15.119:0/2359130712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc8101320 msgr2=0x7f2cc80fef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:41.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.646+0000 7f2cd0000700 1 --2- 172.21.15.119:0/2359130712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc8101320 0x7f2cc80fef80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00f690 tx=0x7f2cbc00f6c0 comp rx=0 tx=0).stop 2024-05-27T09:39:41.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.646+0000 7f2cd0000700 1 -- 172.21.15.119:0/2359130712 shutdown_connections 2024-05-27T09:39:41.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.646+0000 7f2cd0000700 1 --2- 172.21.15.119:0/2359130712 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2cb0078a40 0x7f2cb007aef0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:41.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.646+0000 7f2cd0000700 1 --2- 172.21.15.119:0/2359130712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cc8101320 0x7f2cc80fef80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:41.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.646+0000 7f2cd0000700 1 --2- 172.21.15.119:0/2359130712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cc8101c30 0x7f2cc80ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:41.647 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.646+0000 7f2cd0000700 1 -- 172.21.15.119:0/2359130712 >> 172.21.15.119:0/2359130712 conn(0x7f2cc80fab80 msgr2=0x7f2cc8104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:41.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.646+0000 7f2cd0000700 1 -- 172.21.15.119:0/2359130712 shutdown_connections 2024-05-27T09:39:41.648 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.646+0000 7f2cd0000700 1 -- 172.21.15.119:0/2359130712 wait complete. 2024-05-27T09:39:41.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.797+0000 7fc617200700 1 -- 172.21.15.119:0/897124196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6101003d0 msgr2=0x7fc6101007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:41.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.797+0000 7fc617200700 1 --2- 172.21.15.119:0/897124196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6101003d0 0x7fc6101007a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc600009910 tx=0x7fc600009c20 comp rx=0 tx=0).stop 2024-05-27T09:39:41.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.797+0000 7fc617200700 1 -- 172.21.15.119:0/897124196 shutdown_connections 2024-05-27T09:39:41.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.797+0000 7fc617200700 1 --2- 172.21.15.119:0/897124196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc610100d70 0x7fc610104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:41.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.797+0000 7fc617200700 1 --2- 172.21.15.119:0/897124196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6101003d0 0x7fc6101007a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:41.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.797+0000 7fc617200700 1 -- 172.21.15.119:0/897124196 >> 172.21.15.119:0/897124196 conn(0x7fc6100fbc90 msgr2=0x7fc6100fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:41.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.798+0000 7fc617200700 1 -- 172.21.15.119:0/897124196 shutdown_connections 2024-05-27T09:39:41.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.798+0000 7fc617200700 1 -- 172.21.15.119:0/897124196 wait complete. 2024-05-27T09:39:41.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.799+0000 7fc617200700 1 Processor -- start 2024-05-27T09:39:41.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.799+0000 7fc617200700 1 -- start start 2024-05-27T09:39:41.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.799+0000 7fc617200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6101003d0 0x7fc610195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:41.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.799+0000 7fc617200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc610100d70 0x7fc610196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:41.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.799+0000 7fc617200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc610196960 con 0x7fc6101003d0 2024-05-27T09:39:41.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.799+0000 7fc617200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61019a6f0 con 0x7fc610100d70 2024-05-27T09:39:41.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.799+0000 7fc60fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6101003d0 0x7fc610195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:41.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.800+0000 7fc60fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6101003d0 0x7fc610195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44272/0 (socket says 172.21.15.119:44272) 2024-05-27T09:39:41.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.800+0000 7fc60fe00700 1 -- 172.21.15.119:0/3412492458 learned_addr learned my addr 172.21.15.119:0/3412492458 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:39:41.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.800+0000 7fc60f400700 1 --2- 172.21.15.119:0/3412492458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc610100d70 0x7fc610196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:41.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.800+0000 7fc60fe00700 1 -- 172.21.15.119:0/3412492458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc610100d70 msgr2=0x7fc610196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:41.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.800+0000 7fc60fe00700 1 --2- 172.21.15.119:0/3412492458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc610100d70 0x7fc610196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:41.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.800+0000 7fc60fe00700 1 -- 172.21.15.119:0/3412492458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6000095f0 con 0x7fc6101003d0 2024-05-27T09:39:41.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.800+0000 7fc60f400700 1 --2- 172.21.15.119:0/3412492458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc610100d70 0x7fc610196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:39:41.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.800+0000 7fc60fe00700 1 --2- 172.21.15.119:0/3412492458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6101003d0 0x7fc610195d40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc60000b5c0 tx=0x7fc60000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:41.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.801+0000 7fc60cc00700 1 -- 172.21.15.119:0/3412492458 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc60001d070 con 0x7fc6101003d0 2024-05-27T09:39:41.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.801+0000 7fc60cc00700 1 -- 172.21.15.119:0/3412492458 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc600005e00 con 0x7fc6101003d0 2024-05-27T09:39:41.802 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.801+0000 7fc617200700 1 -- 172.21.15.119:0/3412492458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc61019a970 con 0x7fc6101003d0 2024-05-27T09:39:41.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.801+0000 7fc60cc00700 1 -- 172.21.15.119:0/3412492458 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc600017400 con 0x7fc6101003d0 2024-05-27T09:39:41.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.801+0000 7fc617200700 1 -- 172.21.15.119:0/3412492458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc61019ae60 con 0x7fc6101003d0 2024-05-27T09:39:41.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.802+0000 7fc617200700 1 -- 172.21.15.119:0/3412492458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6101086e0 con 0x7fc6101003d0 2024-05-27T09:39:41.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.803+0000 7fc60cc00700 1 -- 172.21.15.119:0/3412492458 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fc60000f460 con 0x7fc6101003d0 2024-05-27T09:39:41.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.804+0000 7fc60cc00700 1 --2- 172.21.15.119:0/3412492458 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc5f4078a40 0x7fc5f407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:41.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.804+0000 7fc60cc00700 1 -- 172.21.15.119:0/3412492458 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc60009c1b0 con 0x7fc6101003d0 2024-05-27T09:39:41.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.804+0000 7fc60f400700 1 --2- 172.21.15.119:0/3412492458 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc5f4078a40 0x7fc5f407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:41.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.805+0000 7fc60f400700 1 --2- 172.21.15.119:0/3412492458 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc5f4078a40 0x7fc5f407aef0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc005950 tx=0x7fc5fc0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:41.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:41.809+0000 7fc60cc00700 1 -- 172.21.15.119:0/3412492458 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fc600063e10 con 0x7fc6101003d0 2024-05-27T09:39:42.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.117+0000 7fc617200700 1 -- 172.21.15.119:0/3412492458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc61002cc30 con 0x7fc6101003d0 2024-05-27T09:39:42.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.118+0000 7fc60cc00700 1 -- 172.21.15.119:0/3412492458 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+700 (secure 0 0 0) 0x7fc6000637b0 con 0x7fc6101003d0 2024-05-27T09:39:42.119 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:39:42.119 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:39:42.119 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:39:42.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:39:42.119 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:39:42.119 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:39:42.119 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:39:42.119 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:39:42.120 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 5 2024-05-27T09:39:42.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:39:42.120 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:39:42.120 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-05-27T09:39:42.120 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:39:42.120 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:39:42.120 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4, 2024-05-27T09:39:42.120 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 9 2024-05-27T09:39:42.120 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:39:42.120 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:39:42.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.120+0000 7fc617200700 1 -- 172.21.15.119:0/3412492458 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc5f4078a40 msgr2=0x7fc5f407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:42.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.120+0000 7fc617200700 1 --2- 172.21.15.119:0/3412492458 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc5f4078a40 0x7fc5f407aef0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc005950 tx=0x7fc5fc0058e0 comp rx=0 tx=0).stop 2024-05-27T09:39:42.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.120+0000 7fc617200700 1 -- 172.21.15.119:0/3412492458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6101003d0 msgr2=0x7fc610195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:42.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.120+0000 7fc617200700 1 --2- 172.21.15.119:0/3412492458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6101003d0 0x7fc610195d40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc60000b5c0 tx=0x7fc60000be20 comp rx=0 tx=0).stop 2024-05-27T09:39:42.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.121+0000 7fc617200700 1 -- 172.21.15.119:0/3412492458 shutdown_connections 2024-05-27T09:39:42.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.121+0000 7fc617200700 1 --2- 172.21.15.119:0/3412492458 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc5f4078a40 0x7fc5f407aef0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:42.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.121+0000 7fc617200700 1 --2- 172.21.15.119:0/3412492458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6101003d0 0x7fc610195d40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:42.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.121+0000 7fc617200700 1 --2- 172.21.15.119:0/3412492458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc610100d70 0x7fc610196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:42.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.121+0000 7fc617200700 1 -- 172.21.15.119:0/3412492458 >> 172.21.15.119:0/3412492458 conn(0x7fc6100fbc90 msgr2=0x7fc6100fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:42.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.121+0000 7fc617200700 1 -- 172.21.15.119:0/3412492458 shutdown_connections 2024-05-27T09:39:42.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.121+0000 7fc617200700 1 -- 172.21.15.119:0/3412492458 wait complete. 2024-05-27T09:39:42.136 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:42 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-activate[89443]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:39:42.136 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:42 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-activate[89443]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:39:42.136 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:42 smithi138 bash[89429]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:39:42.136 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:42 smithi138 bash[89429]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-05-27T09:39:42.136 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:42 smithi138 bash[89429]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-27T09:39:42.136 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:42 smithi138 bash[89429]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-27T09:39:42.136 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:42 smithi138 bash[89429]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-27T09:39:42.137 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:42 smithi138 bash[89429]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-27T09:39:42.137 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:42 smithi138 bash[89429]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-05-27T09:39:42.137 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:42 smithi138 bash[89429]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-27T09:39:42.137 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:42 smithi138 bash[89429]: --> ceph-volume raw activate successful for osd ID: 5 2024-05-27T09:39:42.137 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:42 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-activate[89443]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-27T09:39:42.137 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:42 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-activate[89443]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-27T09:39:42.137 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:42 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-activate[89443]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-27T09:39:42.137 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:42 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-activate[89443]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-27T09:39:42.137 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:42 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-activate[89443]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-05-27T09:39:42.137 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:42 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-activate[89443]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-27T09:39:42.137 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:42 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-activate[89443]: --> ceph-volume raw activate successful for osd ID: 5 2024-05-27T09:39:42.137 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:42 smithi138 podman[89429]: 2024-05-27 09:39:42.110353531 +0000 UTC m=+2.373012096 container died 2451026e34ef3ef20290431767a5449d7cbe1cbbe936eece908dd5255e9df20b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-activate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-27T09:39:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.278+0000 7fca29c00700 1 -- 172.21.15.119:0/1344053111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca2410c010 msgr2=0x7fca24106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.278+0000 7fca29c00700 1 --2- 172.21.15.119:0/1344053111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca2410c010 0x7fca24106b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009910 tx=0x7fca0c009c20 comp rx=0 tx=0).stop 2024-05-27T09:39:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.278+0000 7fca29c00700 1 -- 172.21.15.119:0/1344053111 shutdown_connections 2024-05-27T09:39:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.278+0000 7fca29c00700 1 --2- 172.21.15.119:0/1344053111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca2410c010 0x7fca24106b80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.278+0000 7fca29c00700 1 --2- 172.21.15.119:0/1344053111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2410b700 0x7fca2410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.278+0000 7fca29c00700 1 -- 172.21.15.119:0/1344053111 >> 172.21.15.119:0/1344053111 conn(0x7fca24076000 msgr2=0x7fca24078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.278+0000 7fca29c00700 1 -- 172.21.15.119:0/1344053111 shutdown_connections 2024-05-27T09:39:42.279 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.278+0000 7fca29c00700 1 -- 172.21.15.119:0/1344053111 wait complete. 2024-05-27T09:39:42.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.279+0000 7fca29c00700 1 Processor -- start 2024-05-27T09:39:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.279+0000 7fca29c00700 1 -- start start 2024-05-27T09:39:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.279+0000 7fca29c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca2410b700 0x7fca2419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.279+0000 7fca29c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2410c010 0x7fca2419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.279+0000 7fca29c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2419ce70 con 0x7fca2410b700 2024-05-27T09:39:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.279+0000 7fca29c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2419cfb0 con 0x7fca2410c010 2024-05-27T09:39:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.280+0000 7fca22a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca2410b700 0x7fca2419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.280+0000 7fca22a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca2410b700 0x7fca2419c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44284/0 (socket says 172.21.15.119:44284) 2024-05-27T09:39:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.280+0000 7fca22a00700 1 -- 172.21.15.119:0/1492832941 learned_addr learned my addr 172.21.15.119:0/1492832941 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:39:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.280+0000 7fca1be00700 1 --2- 172.21.15.119:0/1492832941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2410c010 0x7fca2419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.280+0000 7fca22a00700 1 -- 172.21.15.119:0/1492832941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2410c010 msgr2=0x7fca2419c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.280+0000 7fca22a00700 1 --2- 172.21.15.119:0/1492832941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2410c010 0x7fca2419c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:42.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.280+0000 7fca22a00700 1 -- 172.21.15.119:0/1492832941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0c0095f0 con 0x7fca2410b700 2024-05-27T09:39:42.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.281+0000 7fca22a00700 1 --2- 172.21.15.119:0/1492832941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca2410b700 0x7fca2419c2e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fca1400c930 tx=0x7fca1400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:42.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.281+0000 7fca1b400700 1 -- 172.21.15.119:0/1492832941 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca14007ad0 con 0x7fca2410b700 2024-05-27T09:39:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.281+0000 7fca1b400700 1 -- 172.21.15.119:0/1492832941 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca14007c30 con 0x7fca2410b700 2024-05-27T09:39:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.281+0000 7fca29c00700 1 -- 172.21.15.119:0/1492832941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca241a0e00 con 0x7fca2410b700 2024-05-27T09:39:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.281+0000 7fca1b400700 1 -- 172.21.15.119:0/1492832941 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca14018610 con 0x7fca2410b700 2024-05-27T09:39:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.281+0000 7fca29c00700 1 -- 172.21.15.119:0/1492832941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca241a1380 con 0x7fca2410b700 2024-05-27T09:39:42.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.282+0000 7fca29c00700 1 -- 172.21.15.119:0/1492832941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca2404ea50 con 0x7fca2410b700 2024-05-27T09:39:42.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.283+0000 7fca1b400700 1 -- 172.21.15.119:0/1492832941 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fca1400f450 con 0x7fca2410b700 2024-05-27T09:39:42.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.284+0000 7fca1b400700 1 --2- 172.21.15.119:0/1492832941 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fca08078a40 0x7fca0807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:42.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.284+0000 7fca1b400700 1 -- 172.21.15.119:0/1492832941 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fca1409afa0 con 0x7fca2410b700 2024-05-27T09:39:42.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.284+0000 7fca1be00700 1 --2- 172.21.15.119:0/1492832941 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fca08078a40 0x7fca0807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:42.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.285+0000 7fca1be00700 1 --2- 172.21.15.119:0/1492832941 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fca08078a40 0x7fca0807aef0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fca2419d900 tx=0x7fca0c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:42.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.290+0000 7fca1b400700 1 -- 172.21.15.119:0/1492832941 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fca14062b80 con 0x7fca2410b700 2024-05-27T09:39:42.468 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:42 smithi138 ceph-mon[70004]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:39:42.468 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:42 smithi138 ceph-mon[70004]: pgmap v187: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 57/381 objects degraded (14.961%) 2024-05-27T09:39:42.468 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:42 smithi138 ceph-mon[70004]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:39:42.468 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:42 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/3412492458' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:39:42.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:42 smithi119 ceph-mon[88269]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:39:42.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:42 smithi119 ceph-mon[88269]: pgmap v187: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 57/381 objects degraded (14.961%) 2024-05-27T09:39:42.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:42 smithi119 ceph-mon[88269]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:39:42.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:42 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/3412492458' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:39:42.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.565+0000 7fca29c00700 1 -- 172.21.15.119:0/1492832941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fca241a17f0 con 0x7fca2410b700 2024-05-27T09:39:42.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.566+0000 7fca1b400700 1 -- 172.21.15.119:0/1492832941 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1889 (secure 0 0 0) 0x7fca14062520 con 0x7fca2410b700 2024-05-27T09:39:42.567 INFO:teuthology.orchestra.run.smithi119.stdout:e22 2024-05-27T09:39:42.567 INFO:teuthology.orchestra.run.smithi119.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T09:39:42.567 INFO:teuthology.orchestra.run.smithi119.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:39:42.567 INFO:teuthology.orchestra.run.smithi119.stdout:legacy client fscid: 1 2024-05-27T09:39:42.567 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:39:42.567 INFO:teuthology.orchestra.run.smithi119.stdout:Filesystem 'cephfs' (1) 2024-05-27T09:39:42.567 INFO:teuthology.orchestra.run.smithi119.stdout:fs_name cephfs 2024-05-27T09:39:42.567 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 22 2024-05-27T09:39:42.568 INFO:teuthology.orchestra.run.smithi119.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-27T09:39:42.568 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-05-27T09:25:53.387388+0000 2024-05-27T09:39:42.568 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2024-05-27T09:34:01.250304+0000 2024-05-27T09:39:42.568 INFO:teuthology.orchestra.run.smithi119.stdout:tableserver 0 2024-05-27T09:39:42.568 INFO:teuthology.orchestra.run.smithi119.stdout:root 0 2024-05-27T09:39:42.568 INFO:teuthology.orchestra.run.smithi119.stdout:session_timeout 60 2024-05-27T09:39:42.568 INFO:teuthology.orchestra.run.smithi119.stdout:session_autoclose 300 2024-05-27T09:39:42.568 INFO:teuthology.orchestra.run.smithi119.stdout:max_file_size 1099511627776 2024-05-27T09:39:42.568 INFO:teuthology.orchestra.run.smithi119.stdout:max_xattr_size 65536 2024-05-27T09:39:42.568 INFO:teuthology.orchestra.run.smithi119.stdout:required_client_features {} 2024-05-27T09:39:42.568 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure 0 2024-05-27T09:39:42.568 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure_osd_epoch 46 2024-05-27T09:39:42.568 INFO:teuthology.orchestra.run.smithi119.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:39:42.568 INFO:teuthology.orchestra.run.smithi119.stdout:max_mds 2 2024-05-27T09:39:42.569 INFO:teuthology.orchestra.run.smithi119.stdout:in 0,1 2024-05-27T09:39:42.569 INFO:teuthology.orchestra.run.smithi119.stdout:up {0=14472,1=24241} 2024-05-27T09:39:42.569 INFO:teuthology.orchestra.run.smithi119.stdout:failed 2024-05-27T09:39:42.569 INFO:teuthology.orchestra.run.smithi119.stdout:damaged 2024-05-27T09:39:42.569 INFO:teuthology.orchestra.run.smithi119.stdout:stopped 2024-05-27T09:39:42.569 INFO:teuthology.orchestra.run.smithi119.stdout:data_pools [3] 2024-05-27T09:39:42.569 INFO:teuthology.orchestra.run.smithi119.stdout:metadata_pool 2 2024-05-27T09:39:42.569 INFO:teuthology.orchestra.run.smithi119.stdout:inline_data enabled 2024-05-27T09:39:42.569 INFO:teuthology.orchestra.run.smithi119.stdout:balancer 2024-05-27T09:39:42.569 INFO:teuthology.orchestra.run.smithi119.stdout:bal_rank_mask -1 2024-05-27T09:39:42.569 INFO:teuthology.orchestra.run.smithi119.stdout:standby_count_wanted 1 2024-05-27T09:39:42.569 INFO:teuthology.orchestra.run.smithi119.stdout:qdb_cluster leader: 0 members: 2024-05-27T09:39:42.569 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.ppmzzk{0:14472} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.119:6828/2100782307,v1:172.21.15.119:6829/2100782307] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:39:42.569 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.ndfkhd{1:24241} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/897058594,v1:172.21.15.138:6825/897058594] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:39:42.569 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:39:42.570 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:39:42.570 INFO:teuthology.orchestra.run.smithi119.stdout:Standby daemons: 2024-05-27T09:39:42.570 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:39:42.570 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.nghcey{-1:14482} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.119:6826/413736191,v1:172.21.15.119:6827/413736191] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:39:42.570 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.alszpk{-1:24253} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/2345607053,v1:172.21.15.138:6827/2345607053] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:39:42.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.570+0000 7fca29c00700 1 -- 172.21.15.119:0/1492832941 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fca08078a40 msgr2=0x7fca0807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:42.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.570+0000 7fca29c00700 1 --2- 172.21.15.119:0/1492832941 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fca08078a40 0x7fca0807aef0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fca2419d900 tx=0x7fca0c01a040 comp rx=0 tx=0).stop 2024-05-27T09:39:42.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.570+0000 7fca29c00700 1 -- 172.21.15.119:0/1492832941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca2410b700 msgr2=0x7fca2419c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:42.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.570+0000 7fca29c00700 1 --2- 172.21.15.119:0/1492832941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca2410b700 0x7fca2419c2e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fca1400c930 tx=0x7fca1400ccf0 comp rx=0 tx=0).stop 2024-05-27T09:39:42.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.570+0000 7fca29c00700 1 -- 172.21.15.119:0/1492832941 shutdown_connections 2024-05-27T09:39:42.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.570+0000 7fca29c00700 1 --2- 172.21.15.119:0/1492832941 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fca08078a40 0x7fca0807aef0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:42.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.570+0000 7fca29c00700 1 --2- 172.21.15.119:0/1492832941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca2410b700 0x7fca2419c2e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:42.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.570+0000 7fca29c00700 1 --2- 172.21.15.119:0/1492832941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2410c010 0x7fca2419c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:42.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.570+0000 7fca29c00700 1 -- 172.21.15.119:0/1492832941 >> 172.21.15.119:0/1492832941 conn(0x7fca24076000 msgr2=0x7fca24076940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:42.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.570+0000 7fca29c00700 1 -- 172.21.15.119:0/1492832941 shutdown_connections 2024-05-27T09:39:42.572 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.570+0000 7fca29c00700 1 -- 172.21.15.119:0/1492832941 wait complete. 2024-05-27T09:39:42.573 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 22 2024-05-27T09:39:42.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.719+0000 7f5c69400700 1 -- 172.21.15.119:0/4069421182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c64101c30 msgr2=0x7f5c6410a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:42.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.720+0000 7f5c69400700 1 --2- 172.21.15.119:0/4069421182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c64101c30 0x7f5c6410a130 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009910 tx=0x7f5c50009c20 comp rx=0 tx=0).stop 2024-05-27T09:39:42.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.720+0000 7f5c69400700 1 -- 172.21.15.119:0/4069421182 shutdown_connections 2024-05-27T09:39:42.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.720+0000 7f5c69400700 1 --2- 172.21.15.119:0/4069421182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c64101c30 0x7f5c6410a130 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:42.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.720+0000 7f5c69400700 1 --2- 172.21.15.119:0/4069421182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c64101320 0x7f5c641016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:42.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.720+0000 7f5c69400700 1 -- 172.21.15.119:0/4069421182 >> 172.21.15.119:0/4069421182 conn(0x7f5c640fab80 msgr2=0x7f5c640fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:42.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.720+0000 7f5c69400700 1 -- 172.21.15.119:0/4069421182 shutdown_connections 2024-05-27T09:39:42.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.720+0000 7f5c69400700 1 -- 172.21.15.119:0/4069421182 wait complete. 2024-05-27T09:39:42.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.721+0000 7f5c69400700 1 Processor -- start 2024-05-27T09:39:42.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.721+0000 7f5c69400700 1 -- start start 2024-05-27T09:39:42.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.722+0000 7f5c69400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c64101320 0x7f5c64195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:42.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.722+0000 7f5c69400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c64101c30 0x7f5c64196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:42.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.722+0000 7f5c69400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c64196a70 con 0x7f5c64101320 2024-05-27T09:39:42.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.722+0000 7f5c69400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6419a800 con 0x7f5c64101c30 2024-05-27T09:39:42.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.722+0000 7f5c62800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c64101320 0x7f5c64195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:42.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.722+0000 7f5c62800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c64101320 0x7f5c64195e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44304/0 (socket says 172.21.15.119:44304) 2024-05-27T09:39:42.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.722+0000 7f5c62800700 1 -- 172.21.15.119:0/2850914192 learned_addr learned my addr 172.21.15.119:0/2850914192 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:39:42.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.722+0000 7f5c61e00700 1 --2- 172.21.15.119:0/2850914192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c64101c30 0x7f5c64196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:42.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.722+0000 7f5c62800700 1 -- 172.21.15.119:0/2850914192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c64101c30 msgr2=0x7f5c64196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:42.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.722+0000 7f5c62800700 1 --2- 172.21.15.119:0/2850914192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c64101c30 0x7f5c64196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:42.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.722+0000 7f5c62800700 1 -- 172.21.15.119:0/2850914192 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c500095f0 con 0x7f5c64101320 2024-05-27T09:39:42.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.723+0000 7f5c62800700 1 --2- 172.21.15.119:0/2850914192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c64101320 0x7f5c64195e50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400b4b0 tx=0x7f5c5400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:42.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.723+0000 7f5c5b400700 1 -- 172.21.15.119:0/2850914192 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c54011800 con 0x7f5c64101320 2024-05-27T09:39:42.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.723+0000 7f5c5b400700 1 -- 172.21.15.119:0/2850914192 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c54011e20 con 0x7f5c64101320 2024-05-27T09:39:42.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.723+0000 7f5c69400700 1 -- 172.21.15.119:0/2850914192 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6419aae0 con 0x7f5c64101320 2024-05-27T09:39:42.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.723+0000 7f5c5b400700 1 -- 172.21.15.119:0/2850914192 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c5400f2e0 con 0x7f5c64101320 2024-05-27T09:39:42.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.723+0000 7f5c69400700 1 -- 172.21.15.119:0/2850914192 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6419b030 con 0x7f5c64101320 2024-05-27T09:39:42.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.724+0000 7f5c69400700 1 -- 172.21.15.119:0/2850914192 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c641077a0 con 0x7f5c64101320 2024-05-27T09:39:42.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.725+0000 7f5c5b400700 1 -- 172.21.15.119:0/2850914192 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f5c54010380 con 0x7f5c64101320 2024-05-27T09:39:42.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.726+0000 7f5c5b400700 1 --2- 172.21.15.119:0/2850914192 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5c48078a40 0x7f5c4807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:42.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.726+0000 7f5c5b400700 1 -- 172.21.15.119:0/2850914192 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f5c5409a8d0 con 0x7f5c64101320 2024-05-27T09:39:42.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.727+0000 7f5c61e00700 1 --2- 172.21.15.119:0/2850914192 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5c48078a40 0x7f5c4807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:42.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.727+0000 7f5c61e00700 1 --2- 172.21.15.119:0/2850914192 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5c48078a40 0x7f5c4807aef0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5c50000c00 tx=0x7f5c50005fb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:42.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.731+0000 7f5c5b400700 1 -- 172.21.15.119:0/2850914192 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f5c540624b0 con 0x7f5c64101320 2024-05-27T09:39:42.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.968+0000 7f5c69400700 1 -- 172.21.15.119:0/2850914192 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c6419b3a0 con 0x7f5c48078a40 2024-05-27T09:39:42.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.971+0000 7f5c5b400700 1 -- 172.21.15.119:0/2850914192 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f5c6419b3a0 con 0x7f5c48078a40 2024-05-27T09:39:42.972 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:39:42.972 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:39:42.972 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:39:42.972 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-27T09:39:42.972 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [ 2024-05-27T09:39:42.973 INFO:teuthology.orchestra.run.smithi119.stdout: "crash", 2024-05-27T09:39:42.973 INFO:teuthology.orchestra.run.smithi119.stdout: "mon", 2024-05-27T09:39:42.973 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr" 2024-05-27T09:39:42.973 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-05-27T09:39:42.973 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "11/23 daemons upgraded", 2024-05-27T09:39:42.973 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "Currently upgrading osd daemons", 2024-05-27T09:39:42.973 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:39:42.973 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:39:42.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.975+0000 7f5c69400700 1 -- 172.21.15.119:0/2850914192 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5c48078a40 msgr2=0x7f5c4807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:42.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.975+0000 7f5c69400700 1 --2- 172.21.15.119:0/2850914192 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5c48078a40 0x7f5c4807aef0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5c50000c00 tx=0x7f5c50005fb0 comp rx=0 tx=0).stop 2024-05-27T09:39:42.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.975+0000 7f5c69400700 1 -- 172.21.15.119:0/2850914192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c64101320 msgr2=0x7f5c64195e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:42.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.975+0000 7f5c69400700 1 --2- 172.21.15.119:0/2850914192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c64101320 0x7f5c64195e50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400b4b0 tx=0x7f5c5400b870 comp rx=0 tx=0).stop 2024-05-27T09:39:42.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.975+0000 7f5c69400700 1 -- 172.21.15.119:0/2850914192 shutdown_connections 2024-05-27T09:39:42.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.975+0000 7f5c69400700 1 --2- 172.21.15.119:0/2850914192 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5c48078a40 0x7f5c4807aef0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:42.976 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.975+0000 7f5c69400700 1 --2- 172.21.15.119:0/2850914192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c64101320 0x7f5c64195e50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:42.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.975+0000 7f5c69400700 1 --2- 172.21.15.119:0/2850914192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c64101c30 0x7f5c64196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:42.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.975+0000 7f5c69400700 1 -- 172.21.15.119:0/2850914192 >> 172.21.15.119:0/2850914192 conn(0x7f5c640fab80 msgr2=0x7f5c641048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:42.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.976+0000 7f5c69400700 1 -- 172.21.15.119:0/2850914192 shutdown_connections 2024-05-27T09:39:42.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:42.976+0000 7f5c69400700 1 -- 172.21.15.119:0/2850914192 wait complete. 2024-05-27T09:39:43.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.131+0000 7f0bb3000700 1 -- 172.21.15.119:0/1640700234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bac100d70 msgr2=0x7f0bac104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:43.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.131+0000 7f0bb3000700 1 --2- 172.21.15.119:0/1640700234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bac100d70 0x7f0bac104d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009910 tx=0x7f0b98009c20 comp rx=0 tx=0).stop 2024-05-27T09:39:43.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.131+0000 7f0bb3000700 1 -- 172.21.15.119:0/1640700234 shutdown_connections 2024-05-27T09:39:43.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.131+0000 7f0bb3000700 1 --2- 172.21.15.119:0/1640700234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bac100d70 0x7f0bac104d40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:43.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.131+0000 7f0bb3000700 1 --2- 172.21.15.119:0/1640700234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bac1003d0 0x7f0bac1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:43.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.131+0000 7f0bb3000700 1 -- 172.21.15.119:0/1640700234 >> 172.21.15.119:0/1640700234 conn(0x7f0bac0fbc90 msgr2=0x7f0bac0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:43.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.131+0000 7f0bb3000700 1 -- 172.21.15.119:0/1640700234 shutdown_connections 2024-05-27T09:39:43.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.132+0000 7f0bb3000700 1 -- 172.21.15.119:0/1640700234 wait complete. 2024-05-27T09:39:43.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.132+0000 7f0bb3000700 1 Processor -- start 2024-05-27T09:39:43.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.133+0000 7f0bb3000700 1 -- start start 2024-05-27T09:39:43.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.133+0000 7f0bb3000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bac1003d0 0x7f0bac197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:43.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.133+0000 7f0bb3000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bac100d70 0x7f0bac198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:43.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.133+0000 7f0bb3000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac198b30 con 0x7f0bac100d70 2024-05-27T09:39:43.135 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.133+0000 7f0bb3000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac19c8c0 con 0x7f0bac1003d0 2024-05-27T09:39:43.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.133+0000 7f0bab400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bac100d70 0x7f0bac198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:43.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.133+0000 7f0bab400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bac100d70 0x7f0bac198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44330/0 (socket says 172.21.15.119:44330) 2024-05-27T09:39:43.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.133+0000 7f0bab400700 1 -- 172.21.15.119:0/770005820 learned_addr learned my addr 172.21.15.119:0/770005820 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:39:43.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.134+0000 7f0babe00700 1 --2- 172.21.15.119:0/770005820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bac1003d0 0x7f0bac197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:43.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.134+0000 7f0bab400700 1 -- 172.21.15.119:0/770005820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bac1003d0 msgr2=0x7f0bac197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:43.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.134+0000 7f0bab400700 1 --2- 172.21.15.119:0/770005820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bac1003d0 0x7f0bac197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:43.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.134+0000 7f0bab400700 1 -- 172.21.15.119:0/770005820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b980095f0 con 0x7f0bac100d70 2024-05-27T09:39:43.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.134+0000 7f0bab400700 1 --2- 172.21.15.119:0/770005820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bac100d70 0x7f0bac198450 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0b9800bd70 tx=0x7f0b9800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:43.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.134+0000 7f0ba8c00700 1 -- 172.21.15.119:0/770005820 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b9801d070 con 0x7f0bac100d70 2024-05-27T09:39:43.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.134+0000 7f0ba8c00700 1 -- 172.21.15.119:0/770005820 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b98005dc0 con 0x7f0bac100d70 2024-05-27T09:39:43.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.134+0000 7f0ba8c00700 1 -- 172.21.15.119:0/770005820 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b98017400 con 0x7f0bac100d70 2024-05-27T09:39:43.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.134+0000 7f0bb3000700 1 -- 172.21.15.119:0/770005820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bac19cb40 con 0x7f0bac100d70 2024-05-27T09:39:43.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.135+0000 7f0bb3000700 1 -- 172.21.15.119:0/770005820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bac19d030 con 0x7f0bac100d70 2024-05-27T09:39:43.137 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:42 smithi138 podman[89568]: 2024-05-27 09:39:42.802171696 +0000 UTC m=+0.670517880 container remove 2451026e34ef3ef20290431767a5449d7cbe1cbbe936eece908dd5255e9df20b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-activate, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-27T09:39:43.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.136+0000 7f0bb3000700 1 -- 172.21.15.119:0/770005820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bac1044c0 con 0x7f0bac100d70 2024-05-27T09:39:43.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.137+0000 7f0ba8c00700 1 -- 172.21.15.119:0/770005820 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f0b9800f7f0 con 0x7f0bac100d70 2024-05-27T09:39:43.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.138+0000 7f0ba8c00700 1 --2- 172.21.15.119:0/770005820 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0b90078a40 0x7f0b9007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:39:43.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.138+0000 7f0ba8c00700 1 -- 172.21.15.119:0/770005820 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0b9809c100 con 0x7f0bac100d70 2024-05-27T09:39:43.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.138+0000 7f0babe00700 1 --2- 172.21.15.119:0/770005820 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0b90078a40 0x7f0b9007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:39:43.139 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.139+0000 7f0babe00700 1 --2- 172.21.15.119:0/770005820 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0b90078a40 0x7f0b9007aef0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c005950 tx=0x7f0b9c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:39:43.144 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.143+0000 7f0ba8c00700 1 -- 172.21.15.119:0/770005820 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f0b98063e00 con 0x7f0bac100d70 2024-05-27T09:39:43.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:43 smithi119 ceph-mon[88269]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-05-27T09:39:43.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:43 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:43.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:43 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:39:43.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:43 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/1492832941' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:39:43.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:43 smithi119 ceph-mon[88269]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:39:43.404 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:43 smithi138 ceph-mon[70004]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-05-27T09:39:43.404 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:43 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:43.404 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:43 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:39:43.404 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:43 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/1492832941' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:39:43.404 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:43 smithi138 ceph-mon[70004]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:39:43.404 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:43 smithi138 podman[89627]: 2024-05-27 09:39:43.072101523 +0000 UTC m=+0.034156005 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:39:43.404 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:43 smithi138 podman[89627]: 2024-05-27 09:39:43.186299816 +0000 UTC m=+0.148354304 container create 90d98c5b2f163bd1ed8e56f7fadf9fa35495395416645ebb1516c3222cc2cfd7 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-27T09:39:43.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.479+0000 7f0bb3000700 1 -- 172.21.15.119:0/770005820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0bac19d400 con 0x7f0bac100d70 2024-05-27T09:39:43.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.480+0000 7f0ba8c00700 1 -- 172.21.15.119:0/770005820 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1480 (secure 0 0 0) 0x7f0b980637a0 con 0x7f0bac100d70 2024-05-27T09:39:43.481 INFO:teuthology.orchestra.run.smithi119.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; all OSDs are running squid or later but require_osd_release < squid; Degraded data redundancy: 57/381 objects degraded (14.961%), 14 pgs degraded 2024-05-27T09:39:43.481 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:39:43.481 INFO:teuthology.orchestra.run.smithi119.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:39:43.481 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] OSD_DOWN: 1 osds down 2024-05-27T09:39:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: osd.5 (root=default,host=smithi138) is down 2024-05-27T09:39:43.482 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-05-27T09:39:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-05-27T09:39:43.482 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 57/381 objects degraded (14.961%), 14 pgs degraded 2024-05-27T09:39:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-05-27T09:39:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.3 is active+undersized+degraded, acting [2,1] 2024-05-27T09:39:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.8 is active+undersized+degraded, acting [3,0] 2024-05-27T09:39:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.b is active+undersized+degraded, acting [3,4] 2024-05-27T09:39:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.f is active+undersized+degraded, acting [4,0] 2024-05-27T09:39:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.14 is active+undersized+degraded, acting [3,4] 2024-05-27T09:39:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.16 is active+undersized+degraded, acting [3,2] 2024-05-27T09:39:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.17 is active+undersized+degraded, acting [2,1] 2024-05-27T09:39:43.482 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-05-27T09:39:43.483 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.1a is active+undersized+degraded, acting [3,4] 2024-05-27T09:39:43.483 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.1b is active+undersized+degraded, acting [1,0] 2024-05-27T09:39:43.483 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.1c is active+undersized+degraded, acting [4,2] 2024-05-27T09:39:43.483 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.1d is active+undersized+degraded, acting [3,0] 2024-05-27T09:39:43.483 INFO:teuthology.orchestra.run.smithi119.stdout: pg 2.1e is active+undersized+degraded, acting [2,0] 2024-05-27T09:39:43.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.483+0000 7f0bb3000700 1 -- 172.21.15.119:0/770005820 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0b90078a40 msgr2=0x7f0b9007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:43.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.483+0000 7f0bb3000700 1 --2- 172.21.15.119:0/770005820 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0b90078a40 0x7f0b9007aef0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c005950 tx=0x7f0b9c0058e0 comp rx=0 tx=0).stop 2024-05-27T09:39:43.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.483+0000 7f0bb3000700 1 -- 172.21.15.119:0/770005820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bac100d70 msgr2=0x7f0bac198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:39:43.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.483+0000 7f0bb3000700 1 --2- 172.21.15.119:0/770005820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bac100d70 0x7f0bac198450 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0b9800bd70 tx=0x7f0b9800be50 comp rx=0 tx=0).stop 2024-05-27T09:39:43.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.483+0000 7f0bb3000700 1 -- 172.21.15.119:0/770005820 shutdown_connections 2024-05-27T09:39:43.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.483+0000 7f0bb3000700 1 --2- 172.21.15.119:0/770005820 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0b90078a40 0x7f0b9007aef0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:43.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.483+0000 7f0bb3000700 1 --2- 172.21.15.119:0/770005820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bac1003d0 0x7f0bac197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:43.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.483+0000 7f0bb3000700 1 --2- 172.21.15.119:0/770005820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bac100d70 0x7f0bac198450 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:39:43.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.483+0000 7f0bb3000700 1 -- 172.21.15.119:0/770005820 >> 172.21.15.119:0/770005820 conn(0x7f0bac0fbc90 msgr2=0x7f0bac0fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:39:43.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.483+0000 7f0bb3000700 1 -- 172.21.15.119:0/770005820 shutdown_connections 2024-05-27T09:39:43.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:39:43.484+0000 7f0bb3000700 1 -- 172.21.15.119:0/770005820 wait complete. 2024-05-27T09:39:43.886 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:43 smithi138 podman[89627]: 2024-05-27 09:39:43.523536864 +0000 UTC m=+0.485591429 container init 90d98c5b2f163bd1ed8e56f7fadf9fa35495395416645ebb1516c3222cc2cfd7 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T09:39:43.887 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:43 smithi138 podman[89627]: 2024-05-27 09:39:43.534658907 +0000 UTC m=+0.496713436 container start 90d98c5b2f163bd1ed8e56f7fadf9fa35495395416645ebb1516c3222cc2cfd7 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-27T09:39:43.887 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:43 smithi138 ceph-osd[89645]: -- 172.21.15.138:0/2339848716 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556e92df9680 con 0x556e92ea0d00 2024-05-27T09:39:43.887 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:43 smithi138 bash[89627]: 90d98c5b2f163bd1ed8e56f7fadf9fa35495395416645ebb1516c3222cc2cfd7 2024-05-27T09:39:43.887 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:43 smithi138 systemd[1]: Started Ceph osd.5 for b966b06e-1c09-11ef-bc9b-c7b262605968. 2024-05-27T09:39:44.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:44 smithi119 ceph-mon[88269]: pgmap v188: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 57/381 objects degraded (14.961%) 2024-05-27T09:39:44.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:44 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/770005820' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:39:44.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:44 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:44.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:44 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:44.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:44 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:39:44.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:44 smithi138 ceph-mon[70004]: pgmap v188: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 57/381 objects degraded (14.961%) 2024-05-27T09:39:44.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:44 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/770005820' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:39:44.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:44 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:44.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:44 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:44.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:44 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:39:44.636 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:44 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:39:44.189+0000 7f76487a97c0 -1 Falling back to public interface 2024-05-27T09:39:46.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:46 smithi119 ceph-mon[88269]: pgmap v189: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 57/381 objects degraded (14.961%) 2024-05-27T09:39:46.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:46 smithi138 ceph-mon[70004]: pgmap v189: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 57/381 objects degraded (14.961%) 2024-05-27T09:39:47.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:47 smithi138 ceph-mon[70004]: Health check update: Degraded data redundancy: 57/381 objects degraded (14.961%), 14 pgs degraded (PG_DEGRADED) 2024-05-27T09:39:47.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:47 smithi119 ceph-mon[88269]: Health check update: Degraded data redundancy: 57/381 objects degraded (14.961%), 14 pgs degraded (PG_DEGRADED) 2024-05-27T09:39:48.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:48 smithi119 ceph-mon[88269]: pgmap v190: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 57/381 objects degraded (14.961%) 2024-05-27T09:39:48.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:48 smithi138 ceph-mon[70004]: pgmap v190: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 57/381 objects degraded (14.961%) 2024-05-27T09:39:49.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:49 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:49.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:49 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:49.790 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:49 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:49.790 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:49 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:50.135 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:49 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:39:49.789+0000 7f76487a97c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-05-27T09:39:50.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:50 smithi138 ceph-mon[70004]: pgmap v191: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 57/381 objects degraded (14.961%) 2024-05-27T09:39:50.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:50 smithi138 ceph-mon[70004]: from='osd.5 [v2:172.21.15.138:6816/1302953422,v1:172.21.15.138:6817/1302953422]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T09:39:50.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:50 smithi138 ceph-mon[70004]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T09:39:50.636 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:50 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:39:50.383+0000 7f76487a97c0 -1 osd.5 74 log_to_monitors true 2024-05-27T09:39:50.636 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:39:50 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:39:50.553+0000 7f7637c00700 -1 osd.5 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T09:39:50.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:50 smithi119 ceph-mon[88269]: pgmap v191: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 332 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 57/381 objects degraded (14.961%) 2024-05-27T09:39:50.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:50 smithi119 ceph-mon[88269]: from='osd.5 [v2:172.21.15.138:6816/1302953422,v1:172.21.15.138:6817/1302953422]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T09:39:50.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:50 smithi119 ceph-mon[88269]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T09:39:51.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:51 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:51.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:51 smithi138 ceph-mon[70004]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-27T09:39:51.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:51 smithi138 ceph-mon[70004]: osdmap e77: 6 total, 5 up, 6 in 2024-05-27T09:39:51.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:51 smithi138 ceph-mon[70004]: from='osd.5 [v2:172.21.15.138:6816/1302953422,v1:172.21.15.138:6817/1302953422]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:39:51.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:51 smithi138 ceph-mon[70004]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:39:51.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:51 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:51.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:51 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:51.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:51 smithi119 ceph-mon[88269]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-27T09:39:51.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:51 smithi119 ceph-mon[88269]: osdmap e77: 6 total, 5 up, 6 in 2024-05-27T09:39:51.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:51 smithi119 ceph-mon[88269]: from='osd.5 [v2:172.21.15.138:6816/1302953422,v1:172.21.15.138:6817/1302953422]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:39:51.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:51 smithi119 ceph-mon[88269]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-05-27T09:39:51.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:51 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:52.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:52 smithi138 ceph-mon[70004]: pgmap v193: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 57/381 objects degraded (14.961%) 2024-05-27T09:39:52.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:52 smithi138 ceph-mon[70004]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-27T09:39:52.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:52 smithi138 ceph-mon[70004]: osd.5 [v2:172.21.15.138:6816/1302953422,v1:172.21.15.138:6817/1302953422] boot 2024-05-27T09:39:52.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:52 smithi138 ceph-mon[70004]: osdmap e78: 6 total, 6 up, 6 in 2024-05-27T09:39:52.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:52 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T09:39:52.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:52 smithi119 ceph-mon[88269]: pgmap v193: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 57/381 objects degraded (14.961%) 2024-05-27T09:39:52.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:52 smithi119 ceph-mon[88269]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-27T09:39:52.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:52 smithi119 ceph-mon[88269]: osd.5 [v2:172.21.15.138:6816/1302953422,v1:172.21.15.138:6817/1302953422] boot 2024-05-27T09:39:52.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:52 smithi119 ceph-mon[88269]: osdmap e78: 6 total, 6 up, 6 in 2024-05-27T09:39:52.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:52 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T09:39:53.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:53 smithi138 ceph-mon[70004]: osdmap e79: 6 total, 6 up, 6 in 2024-05-27T09:39:53.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:53 smithi119 ceph-mon[88269]: osdmap e79: 6 total, 6 up, 6 in 2024-05-27T09:39:54.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:54 smithi138 ceph-mon[70004]: pgmap v196: 65 pgs: 7 peering, 9 active+undersized+degraded, 12 active+undersized, 37 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 29/381 objects degraded (7.612%) 2024-05-27T09:39:54.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:54 smithi138 ceph-mon[70004]: Health check update: Degraded data redundancy: 29/381 objects degraded (7.612%), 9 pgs degraded (PG_DEGRADED) 2024-05-27T09:39:54.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:54 smithi119 ceph-mon[88269]: pgmap v196: 65 pgs: 7 peering, 9 active+undersized+degraded, 12 active+undersized, 37 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 29/381 objects degraded (7.612%) 2024-05-27T09:39:54.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:54 smithi119 ceph-mon[88269]: Health check update: Degraded data redundancy: 29/381 objects degraded (7.612%), 9 pgs degraded (PG_DEGRADED) 2024-05-27T09:39:56.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:56 smithi138 ceph-mon[70004]: pgmap v197: 65 pgs: 7 peering, 7 active+undersized+degraded, 10 active+undersized, 41 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 23/381 objects degraded (6.037%) 2024-05-27T09:39:56.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:56 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:56.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:56 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:56.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:56 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:39:56.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:56 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:39:56.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:56 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:56.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:56 smithi119 ceph-mon[88269]: pgmap v197: 65 pgs: 7 peering, 7 active+undersized+degraded, 10 active+undersized, 41 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 23/381 objects degraded (6.037%) 2024-05-27T09:39:56.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:56 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:56.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:56 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:56.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:56 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:39:56.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:56 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:39:56.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:56 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:57.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:39:57.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:39:57.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:39:57.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:39:57.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:39:57.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: Upgrade: Setting container_image for all osd 2024-05-27T09:39:57.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:57.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-05-27T09:39:57.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-05-27T09:39:57.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-05-27T09:39:57.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-05-27T09:39:57.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-05-27T09:39:57.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-05-27T09:39:57.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-05-27T09:39:57.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-05-27T09:39:57.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-05-27T09:39:57.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-05-27T09:39:57.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-05-27T09:39:57.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-05-27T09:39:57.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: Upgrade: Setting require_osd_release to 19 squid 2024-05-27T09:39:57.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-05-27T09:39:57.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:57.888 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:39:57.915 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:39:57.915 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:39:57.915 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:39:57.915 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:39:57.915 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:39:57.916 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: Upgrade: Setting container_image for all osd 2024-05-27T09:39:57.916 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:57.916 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-05-27T09:39:57.916 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-05-27T09:39:57.916 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-05-27T09:39:57.916 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-05-27T09:39:57.916 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-05-27T09:39:57.916 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-05-27T09:39:57.916 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-05-27T09:39:57.916 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-05-27T09:39:57.916 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-05-27T09:39:57.916 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-05-27T09:39:57.916 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-05-27T09:39:57.916 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-05-27T09:39:57.917 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: Upgrade: Setting require_osd_release to 19 squid 2024-05-27T09:39:57.917 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-05-27T09:39:57.917 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:39:57.917 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:39:58.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:57 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119[88265]: 2024-05-27T09:39:57.913+0000 7f9018600700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-27T09:39:58.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:58 smithi119 ceph-mon[88269]: pgmap v198: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:58.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:58 smithi119 ceph-mon[88269]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-05-27T09:39:58.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:58 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-05-27T09:39:58.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:58 smithi119 ceph-mon[88269]: osdmap e80: 6 total, 6 up, 6 in 2024-05-27T09:39:58.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:58 smithi119 ceph-mon[88269]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-05-27T09:39:58.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:58 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-27T09:39:58.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:58 smithi119 ceph-mon[88269]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-27T09:39:58.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:58 smithi119 ceph-mon[88269]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-27T09:39:58.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:58 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-27T09:39:58.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:58 smithi119 ceph-mon[88269]: osdmap e81: 6 total, 6 up, 6 in 2024-05-27T09:39:58.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:58 smithi119 ceph-mon[88269]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-05-27T09:39:58.749 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:58 smithi119 ceph-mon[88269]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/381 objects degraded (6.037%), 7 pgs degraded) 2024-05-27T09:39:58.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:58 smithi138 ceph-mon[70004]: pgmap v198: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:39:58.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:58 smithi138 ceph-mon[70004]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-05-27T09:39:58.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:58 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-05-27T09:39:58.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:58 smithi138 ceph-mon[70004]: osdmap e80: 6 total, 6 up, 6 in 2024-05-27T09:39:58.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:58 smithi138 ceph-mon[70004]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-05-27T09:39:58.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:58 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-27T09:39:58.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:58 smithi138 ceph-mon[70004]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-27T09:39:58.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:58 smithi138 ceph-mon[70004]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-27T09:39:58.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:58 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-27T09:39:58.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:58 smithi138 ceph-mon[70004]: osdmap e81: 6 total, 6 up, 6 in 2024-05-27T09:39:58.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:58 smithi138 ceph-mon[70004]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-05-27T09:39:58.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:58 smithi138 ceph-mon[70004]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/381 objects degraded (6.037%), 7 pgs degraded) 2024-05-27T09:40:00.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:59 smithi119 ceph-mon[88269]: mds.? [v2:172.21.15.119:6828/2854413001,v1:172.21.15.119:6829/2854413001] up:boot 2024-05-27T09:40:00.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:59 smithi119 ceph-mon[88269]: mds.? [v2:172.21.15.138:6824/2055700210,v1:172.21.15.138:6825/2055700210] up:boot 2024-05-27T09:40:00.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:59 smithi119 ceph-mon[88269]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-27T09:40:00.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:59 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.ppmzzk"}]: dispatch 2024-05-27T09:40:00.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:39:59 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.ndfkhd"}]: dispatch 2024-05-27T09:40:00.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:00 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119[88265]: 2024-05-27T09:39:59.999+0000 7f901b800700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-27T09:40:00.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:00 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119[88265]: 2024-05-27T09:39:59.999+0000 7f901b800700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-27T09:40:00.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:00 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119[88265]: 2024-05-27T09:39:59.999+0000 7f901b800700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-05-27T09:40:00.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:00 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119[88265]: 2024-05-27T09:39:59.999+0000 7f901b800700 -1 log_channel(cluster) log [ERR] : [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:40:00.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:00 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119[88265]: 2024-05-27T09:39:59.999+0000 7f901b800700 -1 log_channel(cluster) log [ERR] : fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:40:00.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:00 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119[88265]: 2024-05-27T09:39:59.999+0000 7f901b800700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-27T09:40:00.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:00 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119[88265]: 2024-05-27T09:39:59.999+0000 7f901b800700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-05-27T09:40:00.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:59 smithi138 ceph-mon[70004]: mds.? [v2:172.21.15.119:6828/2854413001,v1:172.21.15.119:6829/2854413001] up:boot 2024-05-27T09:40:00.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:59 smithi138 ceph-mon[70004]: mds.? [v2:172.21.15.138:6824/2055700210,v1:172.21.15.138:6825/2055700210] up:boot 2024-05-27T09:40:00.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:59 smithi138 ceph-mon[70004]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-27T09:40:00.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:59 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.ppmzzk"}]: dispatch 2024-05-27T09:40:00.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:39:59 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.ndfkhd"}]: dispatch 2024-05-27T09:40:01.222 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:00 smithi119 ceph-mon[88269]: pgmap v201: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:01.222 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:00 smithi119 ceph-mon[88269]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-27T09:40:01.222 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:00 smithi119 ceph-mon[88269]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-27T09:40:01.222 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:00 smithi119 ceph-mon[88269]: fs cephfs is degraded 2024-05-27T09:40:01.222 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:00 smithi119 ceph-mon[88269]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:40:01.222 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:00 smithi119 ceph-mon[88269]: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:40:01.222 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:00 smithi119 ceph-mon[88269]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-27T09:40:01.222 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:00 smithi119 ceph-mon[88269]: fs cephfs is offline because no MDS is active for it. 2024-05-27T09:40:01.223 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:00 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:01.223 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:00 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi119.nghcey", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:40:01.223 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:00 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:40:01.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:00 smithi138 ceph-mon[70004]: pgmap v201: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:01.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:00 smithi138 ceph-mon[70004]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-27T09:40:01.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:00 smithi138 ceph-mon[70004]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-27T09:40:01.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:00 smithi138 ceph-mon[70004]: fs cephfs is degraded 2024-05-27T09:40:01.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:00 smithi138 ceph-mon[70004]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:40:01.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:00 smithi138 ceph-mon[70004]: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:40:01.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:00 smithi138 ceph-mon[70004]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-27T09:40:01.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:00 smithi138 ceph-mon[70004]: fs cephfs is offline because no MDS is active for it. 2024-05-27T09:40:01.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:00 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:01.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:00 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi119.nghcey", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:40:01.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:00 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:40:02.115 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:01 smithi119 ceph-mon[88269]: Upgrade: Updating mds.cephfs.smithi119.nghcey 2024-05-27T09:40:02.115 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:01 smithi119 ceph-mon[88269]: Deploying daemon mds.cephfs.smithi119.nghcey on smithi119 2024-05-27T09:40:02.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:01 smithi138 ceph-mon[70004]: Upgrade: Updating mds.cephfs.smithi119.nghcey 2024-05-27T09:40:02.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:01 smithi138 ceph-mon[70004]: Deploying daemon mds.cephfs.smithi119.nghcey on smithi119 2024-05-27T09:40:03.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:02 smithi119 ceph-mon[88269]: pgmap v202: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:03.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:02 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:03.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:02 smithi138 ceph-mon[70004]: pgmap v202: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:03.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:02 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:05.220 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:04 smithi119 ceph-mon[88269]: pgmap v203: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:05.220 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:04 smithi119 ceph-mon[88269]: osdmap e82: 6 total, 6 up, 6 in 2024-05-27T09:40:05.220 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:04 smithi119 ceph-mon[88269]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-27T09:40:05.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:04 smithi138 ceph-mon[70004]: pgmap v203: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:05.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:04 smithi138 ceph-mon[70004]: osdmap e82: 6 total, 6 up, 6 in 2024-05-27T09:40:05.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:04 smithi138 ceph-mon[70004]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-27T09:40:07.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:06 smithi119 ceph-mon[88269]: pgmap v205: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:07.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:06 smithi138 ceph-mon[70004]: pgmap v205: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:08.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:08 smithi119 ceph-mon[88269]: pgmap v206: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:08.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:08 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:08.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:08 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:08.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:08 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:40:09.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:08 smithi138 ceph-mon[70004]: pgmap v206: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:09.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:08 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:09.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:08 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:09.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:08 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:40:10.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:09 smithi119 ceph-mon[88269]: mds.? [v2:172.21.15.119:6826/3357235708,v1:172.21.15.119:6827/3357235708] up:boot 2024-05-27T09:40:10.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:09 smithi119 ceph-mon[88269]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-27T09:40:10.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:09 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.nghcey"}]: dispatch 2024-05-27T09:40:10.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:09 smithi138 ceph-mon[70004]: mds.? [v2:172.21.15.119:6826/3357235708,v1:172.21.15.119:6827/3357235708] up:boot 2024-05-27T09:40:10.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:09 smithi138 ceph-mon[70004]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-27T09:40:10.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:09 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.nghcey"}]: dispatch 2024-05-27T09:40:11.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:11 smithi138 ceph-mon[70004]: pgmap v207: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:11.497 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:10 smithi119 ceph-mon[88269]: pgmap v207: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:13.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:13 smithi138 ceph-mon[70004]: pgmap v208: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:13.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:13 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:13.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:13 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:40:13.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:13 smithi119 ceph-mon[88269]: pgmap v208: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:13.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:13 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:13.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:13 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:40:13.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.663+0000 7f3240000700 1 -- 172.21.15.119:0/4061595628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32381004c0 msgr2=0x7f323810c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:13.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.663+0000 7f3240000700 1 --2- 172.21.15.119:0/4061595628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32381004c0 0x7f323810c7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3224009940 tx=0x7f3224009c50 comp rx=0 tx=0).stop 2024-05-27T09:40:13.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.663+0000 7f3240000700 1 -- 172.21.15.119:0/4061595628 shutdown_connections 2024-05-27T09:40:13.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.663+0000 7f3240000700 1 --2- 172.21.15.119:0/4061595628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32381004c0 0x7f323810c7a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:13.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.663+0000 7f3240000700 1 --2- 172.21.15.119:0/4061595628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32380ffbb0 0x7f32380fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:13.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.663+0000 7f3240000700 1 -- 172.21.15.119:0/4061595628 >> 172.21.15.119:0/4061595628 conn(0x7f3238074f50 msgr2=0x7f3238075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:13.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.663+0000 7f3240000700 1 -- 172.21.15.119:0/4061595628 shutdown_connections 2024-05-27T09:40:13.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.663+0000 7f3240000700 1 -- 172.21.15.119:0/4061595628 wait complete. 2024-05-27T09:40:13.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.664+0000 7f3240000700 1 Processor -- start 2024-05-27T09:40:13.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.664+0000 7f3240000700 1 -- start start 2024-05-27T09:40:13.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.665+0000 7f3240000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32380ffbb0 0x7f3238197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:13.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.665+0000 7f3240000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32381004c0 0x7f3238198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:13.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.665+0000 7f3240000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3238198a60 con 0x7f32381004c0 2024-05-27T09:40:13.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.665+0000 7f3240000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323819c7f0 con 0x7f32380ffbb0 2024-05-27T09:40:13.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.665+0000 7f323cc00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32381004c0 0x7f3238198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:13.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.665+0000 7f323cc00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32381004c0 0x7f3238198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57084/0 (socket says 172.21.15.119:57084) 2024-05-27T09:40:13.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.665+0000 7f323cc00700 1 -- 172.21.15.119:0/2358918520 learned_addr learned my addr 172.21.15.119:0/2358918520 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:40:13.666 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.665+0000 7f323d600700 1 --2- 172.21.15.119:0/2358918520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32380ffbb0 0x7f3238197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:13.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.666+0000 7f323cc00700 1 -- 172.21.15.119:0/2358918520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32380ffbb0 msgr2=0x7f3238197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:13.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.666+0000 7f323cc00700 1 --2- 172.21.15.119:0/2358918520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32380ffbb0 0x7f3238197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:13.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.666+0000 7f323cc00700 1 -- 172.21.15.119:0/2358918520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32240095f0 con 0x7f32381004c0 2024-05-27T09:40:13.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.666+0000 7f323d600700 1 --2- 172.21.15.119:0/2358918520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32380ffbb0 0x7f3238197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:40:13.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.666+0000 7f323cc00700 1 --2- 172.21.15.119:0/2358918520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32381004c0 0x7f3238198380 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f32240055f0 tx=0x7f32240056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:13.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.667+0000 7f3236000700 1 -- 172.21.15.119:0/2358918520 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f322401d070 con 0x7f32381004c0 2024-05-27T09:40:13.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.667+0000 7f3240000700 1 -- 172.21.15.119:0/2358918520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323819cad0 con 0x7f32381004c0 2024-05-27T09:40:13.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.667+0000 7f3236000700 1 -- 172.21.15.119:0/2358918520 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f322400f460 con 0x7f32381004c0 2024-05-27T09:40:13.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.667+0000 7f3236000700 1 -- 172.21.15.119:0/2358918520 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3224021640 con 0x7f32381004c0 2024-05-27T09:40:13.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.667+0000 7f3240000700 1 -- 172.21.15.119:0/2358918520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323819d020 con 0x7f32381004c0 2024-05-27T09:40:13.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.668+0000 7f3240000700 1 -- 172.21.15.119:0/2358918520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3238109f10 con 0x7f32381004c0 2024-05-27T09:40:13.670 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.669+0000 7f3236000700 1 -- 172.21.15.119:0/2358918520 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f322400fa20 con 0x7f32381004c0 2024-05-27T09:40:13.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.670+0000 7f3236000700 1 --2- 172.21.15.119:0/2358918520 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3220078a40 0x7f322007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:13.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.671+0000 7f3236000700 1 -- 172.21.15.119:0/2358918520 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f322409c010 con 0x7f32381004c0 2024-05-27T09:40:13.671 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.671+0000 7f323d600700 1 --2- 172.21.15.119:0/2358918520 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3220078a40 0x7f322007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:13.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.672+0000 7f323d600700 1 --2- 172.21.15.119:0/2358918520 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3220078a40 0x7f322007aef0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f322c005950 tx=0x7f322c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:13.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.676+0000 7f3236000700 1 -- 172.21.15.119:0/2358918520 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f3224063be0 con 0x7f32381004c0 2024-05-27T09:40:13.946 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.945+0000 7f3240000700 1 -- 172.21.15.119:0/2358918520 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f323819d3b0 con 0x7f3220078a40 2024-05-27T09:40:13.948 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.947+0000 7f3236000700 1 -- 172.21.15.119:0/2358918520 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f323819d3b0 con 0x7f3220078a40 2024-05-27T09:40:13.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.951+0000 7f3240000700 1 -- 172.21.15.119:0/2358918520 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3220078a40 msgr2=0x7f322007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:13.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.951+0000 7f3240000700 1 --2- 172.21.15.119:0/2358918520 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3220078a40 0x7f322007aef0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f322c005950 tx=0x7f322c0058e0 comp rx=0 tx=0).stop 2024-05-27T09:40:13.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.951+0000 7f3240000700 1 -- 172.21.15.119:0/2358918520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32381004c0 msgr2=0x7f3238198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:13.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.951+0000 7f3240000700 1 --2- 172.21.15.119:0/2358918520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32381004c0 0x7f3238198380 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f32240055f0 tx=0x7f32240056d0 comp rx=0 tx=0).stop 2024-05-27T09:40:13.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.951+0000 7f3240000700 1 -- 172.21.15.119:0/2358918520 shutdown_connections 2024-05-27T09:40:13.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.951+0000 7f3240000700 1 --2- 172.21.15.119:0/2358918520 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3220078a40 0x7f322007aef0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:13.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.951+0000 7f3240000700 1 --2- 172.21.15.119:0/2358918520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32380ffbb0 0x7f3238197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:13.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.951+0000 7f3240000700 1 --2- 172.21.15.119:0/2358918520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32381004c0 0x7f3238198380 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:13.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.951+0000 7f3240000700 1 -- 172.21.15.119:0/2358918520 >> 172.21.15.119:0/2358918520 conn(0x7f3238074f50 msgr2=0x7f32380fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:13.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.951+0000 7f3240000700 1 -- 172.21.15.119:0/2358918520 shutdown_connections 2024-05-27T09:40:13.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:13.951+0000 7f3240000700 1 -- 172.21.15.119:0/2358918520 wait complete. 2024-05-27T09:40:13.974 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:40:14.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.114+0000 7f7d6e400700 1 -- 172.21.15.119:0/1299610808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d68101aa0 msgr2=0x7f7d68109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:14.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.114+0000 7f7d6e400700 1 --2- 172.21.15.119:0/1299610808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d68101aa0 0x7f7d68109fa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009910 tx=0x7f7d54009c20 comp rx=0 tx=0).stop 2024-05-27T09:40:14.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.114+0000 7f7d6e400700 1 -- 172.21.15.119:0/1299610808 shutdown_connections 2024-05-27T09:40:14.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.114+0000 7f7d6e400700 1 --2- 172.21.15.119:0/1299610808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d68101aa0 0x7f7d68109fa0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:14.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.114+0000 7f7d6e400700 1 --2- 172.21.15.119:0/1299610808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d68101190 0x7f7d68101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:14.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.114+0000 7f7d6e400700 1 -- 172.21.15.119:0/1299610808 >> 172.21.15.119:0/1299610808 conn(0x7f7d680fa9d0 msgr2=0x7f7d680fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:14.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.114+0000 7f7d6e400700 1 -- 172.21.15.119:0/1299610808 shutdown_connections 2024-05-27T09:40:14.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.114+0000 7f7d6e400700 1 -- 172.21.15.119:0/1299610808 wait complete. 2024-05-27T09:40:14.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.115+0000 7f7d6e400700 1 Processor -- start 2024-05-27T09:40:14.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.116+0000 7f7d6e400700 1 -- start start 2024-05-27T09:40:14.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.116+0000 7f7d6e400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d68101190 0x7f7d6819b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:14.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.116+0000 7f7d6e400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d68101aa0 0x7f7d6819be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:14.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.116+0000 7f7d6e400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6819c4c0 con 0x7f7d68101190 2024-05-27T09:40:14.117 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.116+0000 7f7d6e400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d68195970 con 0x7f7d68101aa0 2024-05-27T09:40:14.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.116+0000 7f7d6d000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d68101190 0x7f7d6819b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:14.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.116+0000 7f7d6d000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d68101190 0x7f7d6819b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57100/0 (socket says 172.21.15.119:57100) 2024-05-27T09:40:14.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.116+0000 7f7d6d000700 1 -- 172.21.15.119:0/38118301 learned_addr learned my addr 172.21.15.119:0/38118301 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:40:14.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.117+0000 7f7d67e00700 1 --2- 172.21.15.119:0/38118301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d68101aa0 0x7f7d6819be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:14.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.117+0000 7f7d6d000700 1 -- 172.21.15.119:0/38118301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d68101aa0 msgr2=0x7f7d6819be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:14.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.117+0000 7f7d6d000700 1 --2- 172.21.15.119:0/38118301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d68101aa0 0x7f7d6819be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:14.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.117+0000 7f7d6d000700 1 -- 172.21.15.119:0/38118301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d540095f0 con 0x7f7d68101190 2024-05-27T09:40:14.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.117+0000 7f7d6d000700 1 --2- 172.21.15.119:0/38118301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d68101190 0x7f7d6819b8f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00c8a0 tx=0x7f7d5c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:14.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.117+0000 7f7d65600700 1 -- 172.21.15.119:0/38118301 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d5c00cea0 con 0x7f7d68101190 2024-05-27T09:40:14.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.118+0000 7f7d65600700 1 -- 172.21.15.119:0/38118301 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d5c004830 con 0x7f7d68101190 2024-05-27T09:40:14.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.118+0000 7f7d6e400700 1 -- 172.21.15.119:0/38118301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d68195c50 con 0x7f7d68101190 2024-05-27T09:40:14.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.118+0000 7f7d65600700 1 -- 172.21.15.119:0/38118301 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d5c0055b0 con 0x7f7d68101190 2024-05-27T09:40:14.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.118+0000 7f7d6e400700 1 -- 172.21.15.119:0/38118301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d681961a0 con 0x7f7d68101190 2024-05-27T09:40:14.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.119+0000 7f7d6e400700 1 -- 172.21.15.119:0/38118301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d68107610 con 0x7f7d68101190 2024-05-27T09:40:14.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.121+0000 7f7d65600700 1 -- 172.21.15.119:0/38118301 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f7d5c0049a0 con 0x7f7d68101190 2024-05-27T09:40:14.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.122+0000 7f7d65600700 1 --2- 172.21.15.119:0/38118301 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7d50078a40 0x7f7d5007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:14.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.122+0000 7f7d65600700 1 -- 172.21.15.119:0/38118301 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7d5c09a9f0 con 0x7f7d68101190 2024-05-27T09:40:14.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.122+0000 7f7d67e00700 1 --2- 172.21.15.119:0/38118301 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7d50078a40 0x7f7d5007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:14.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.123+0000 7f7d67e00700 1 --2- 172.21.15.119:0/38118301 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7d50078a40 0x7f7d5007aef0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7d680fc150 tx=0x7f7d5401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:14.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.130+0000 7f7d65600700 1 -- 172.21.15.119:0/38118301 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f7d5c0625a0 con 0x7f7d68101190 2024-05-27T09:40:14.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.389+0000 7f7d6e400700 1 -- 172.21.15.119:0/38118301 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d68196510 con 0x7f7d50078a40 2024-05-27T09:40:14.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.393+0000 7f7d65600700 1 -- 172.21.15.119:0/38118301 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7d68196510 con 0x7f7d50078a40 2024-05-27T09:40:14.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.395+0000 7f7d6e400700 1 -- 172.21.15.119:0/38118301 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7d50078a40 msgr2=0x7f7d5007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:14.396 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.395+0000 7f7d6e400700 1 --2- 172.21.15.119:0/38118301 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7d50078a40 0x7f7d5007aef0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7d680fc150 tx=0x7f7d5401a040 comp rx=0 tx=0).stop 2024-05-27T09:40:14.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.395+0000 7f7d6e400700 1 -- 172.21.15.119:0/38118301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d68101190 msgr2=0x7f7d6819b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:14.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.395+0000 7f7d6e400700 1 --2- 172.21.15.119:0/38118301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d68101190 0x7f7d6819b8f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00c8a0 tx=0x7f7d5c00cc60 comp rx=0 tx=0).stop 2024-05-27T09:40:14.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.395+0000 7f7d6e400700 1 -- 172.21.15.119:0/38118301 shutdown_connections 2024-05-27T09:40:14.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.395+0000 7f7d6e400700 1 --2- 172.21.15.119:0/38118301 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7d50078a40 0x7f7d5007aef0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:14.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.395+0000 7f7d6e400700 1 --2- 172.21.15.119:0/38118301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d68101190 0x7f7d6819b8f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:14.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.395+0000 7f7d6e400700 1 --2- 172.21.15.119:0/38118301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d68101aa0 0x7f7d6819be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:14.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.395+0000 7f7d6e400700 1 -- 172.21.15.119:0/38118301 >> 172.21.15.119:0/38118301 conn(0x7f7d680fa9d0 msgr2=0x7f7d68104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:14.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.396+0000 7f7d6e400700 1 -- 172.21.15.119:0/38118301 shutdown_connections 2024-05-27T09:40:14.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.396+0000 7f7d6e400700 1 -- 172.21.15.119:0/38118301 wait complete. 2024-05-27T09:40:14.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.551+0000 7fbd5c000700 1 -- 172.21.15.119:0/4292470509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd54100d50 msgr2=0x7fbd54104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:14.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.551+0000 7fbd5c000700 1 --2- 172.21.15.119:0/4292470509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd54100d50 0x7fbd54104d20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009910 tx=0x7fbd40009c20 comp rx=0 tx=0).stop 2024-05-27T09:40:14.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.551+0000 7fbd5c000700 1 -- 172.21.15.119:0/4292470509 shutdown_connections 2024-05-27T09:40:14.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.551+0000 7fbd5c000700 1 --2- 172.21.15.119:0/4292470509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd54100d50 0x7fbd54104d20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:14.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.551+0000 7fbd5c000700 1 --2- 172.21.15.119:0/4292470509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd541003b0 0x7fbd54100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:14.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.551+0000 7fbd5c000700 1 -- 172.21.15.119:0/4292470509 >> 172.21.15.119:0/4292470509 conn(0x7fbd540fbc30 msgr2=0x7fbd540fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:14.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.551+0000 7fbd5c000700 1 -- 172.21.15.119:0/4292470509 shutdown_connections 2024-05-27T09:40:14.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.551+0000 7fbd5c000700 1 -- 172.21.15.119:0/4292470509 wait complete. 2024-05-27T09:40:14.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.552+0000 7fbd5c000700 1 Processor -- start 2024-05-27T09:40:14.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.552+0000 7fbd5c000700 1 -- start start 2024-05-27T09:40:14.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.553+0000 7fbd5c000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd541003b0 0x7fbd54197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:14.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.553+0000 7fbd5c000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd54100d50 0x7fbd54198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:14.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.553+0000 7fbd5c000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd54198c00 con 0x7fbd54100d50 2024-05-27T09:40:14.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.553+0000 7fbd5c000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5419c990 con 0x7fbd541003b0 2024-05-27T09:40:14.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.553+0000 7fbd58c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd54100d50 0x7fbd54198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:14.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.553+0000 7fbd58c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd54100d50 0x7fbd54198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57110/0 (socket says 172.21.15.119:57110) 2024-05-27T09:40:14.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.553+0000 7fbd58c00700 1 -- 172.21.15.119:0/1404378577 learned_addr learned my addr 172.21.15.119:0/1404378577 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:40:14.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.553+0000 7fbd59600700 1 --2- 172.21.15.119:0/1404378577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd541003b0 0x7fbd54197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:14.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.554+0000 7fbd58c00700 1 -- 172.21.15.119:0/1404378577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd541003b0 msgr2=0x7fbd54197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:14.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.554+0000 7fbd58c00700 1 --2- 172.21.15.119:0/1404378577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd541003b0 0x7fbd54197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:14.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.554+0000 7fbd58c00700 1 -- 172.21.15.119:0/1404378577 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd400095f0 con 0x7fbd54100d50 2024-05-27T09:40:14.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.554+0000 7fbd58c00700 1 --2- 172.21.15.119:0/1404378577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd54100d50 0x7fbd54198520 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000bda0 tx=0x7fbd4000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:14.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.554+0000 7fbd52000700 1 -- 172.21.15.119:0/1404378577 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd4001d070 con 0x7fbd54100d50 2024-05-27T09:40:14.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.554+0000 7fbd52000700 1 -- 172.21.15.119:0/1404378577 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd40004fb0 con 0x7fbd54100d50 2024-05-27T09:40:14.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.554+0000 7fbd5c000700 1 -- 172.21.15.119:0/1404378577 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd5419cc10 con 0x7fbd54100d50 2024-05-27T09:40:14.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.554+0000 7fbd5c000700 1 -- 172.21.15.119:0/1404378577 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5419d100 con 0x7fbd54100d50 2024-05-27T09:40:14.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.554+0000 7fbd52000700 1 -- 172.21.15.119:0/1404378577 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd4000f460 con 0x7fbd54100d50 2024-05-27T09:40:14.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.556+0000 7fbd5c000700 1 -- 172.21.15.119:0/1404378577 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd34005320 con 0x7fbd54100d50 2024-05-27T09:40:14.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.558+0000 7fbd52000700 1 -- 172.21.15.119:0/1404378577 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fbd40004af0 con 0x7fbd54100d50 2024-05-27T09:40:14.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.559+0000 7fbd52000700 1 --2- 172.21.15.119:0/1404378577 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fbd3c078a40 0x7fbd3c07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:14.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.559+0000 7fbd52000700 1 -- 172.21.15.119:0/1404378577 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbd4009c610 con 0x7fbd54100d50 2024-05-27T09:40:14.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.559+0000 7fbd59600700 1 --2- 172.21.15.119:0/1404378577 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fbd3c078a40 0x7fbd3c07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:14.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.560+0000 7fbd59600700 1 --2- 172.21.15.119:0/1404378577 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fbd3c078a40 0x7fbd3c07aef0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbd480060b0 tx=0x7fbd48007b80 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:14.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.563+0000 7fbd52000700 1 -- 172.21.15.119:0/1404378577 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fbd40064410 con 0x7fbd54100d50 2024-05-27T09:40:14.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.807+0000 7fbd5c000700 1 -- 172.21.15.119:0/1404378577 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd34000bf0 con 0x7fbd3c078a40 2024-05-27T09:40:14.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.837+0000 7fbd52000700 1 -- 172.21.15.119:0/1404378577 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbd34000bf0 con 0x7fbd3c078a40 2024-05-27T09:40:14.838 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:40:14.838 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (7m) 2m ago 21m 14.3M - 0.25.0 c8568f914cd2 6b90564dcef7 2024-05-27T09:40:14.838 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (21m) 2m ago 21m 8850k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:40:14.838 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (20m) 26s ago 20m 8954k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:40:14.838 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (4m) 2m ago 21m 6669k - 19.0.0-3868-ga6507388 850a06a84d28 011d811ce7fc 2024-05-27T09:40:14.838 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (4m) 26s ago 20m 6665k - 19.0.0-3868-ga6507388 850a06a84d28 b546b3ed1afc 2024-05-27T09:40:14.838 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (7m) 2m ago 21m 74.4M - 9.4.12 2bacad6d85d8 b3abdab08db4 2024-05-27T09:40:14.838 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 starting - - - - 2024-05-27T09:40:14.838 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (14m) 2m ago 14m 130M - 18.2.0 dc2bc1663786 14d7d0ed5e64 2024-05-27T09:40:14.838 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (14m) 26s ago 14m 18.5M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:40:14.839 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (14m) 26s ago 14m 128M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:40:14.839 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (10m) 2m ago 23m 587M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:40:14.839 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (9m) 26s ago 20m 534M - 19.0.0-3868-ga6507388 850a06a84d28 2fc3a80696fa 2024-05-27T09:40:14.839 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (5m) 2m ago 23m 114M 2048M 19.0.0-3868-ga6507388 850a06a84d28 3e5c7a9ad372 2024-05-27T09:40:14.839 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (5m) 26s ago 19m 108M 2048M 19.0.0-3868-ga6507388 850a06a84d28 e4d471274cf1 2024-05-27T09:40:14.839 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (8m) 2m ago 21m 8002k - 1.5.0 0da6a335fe13 709c37815cff 2024-05-27T09:40:14.839 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (8m) 26s ago 20m 8292k - 1.5.0 0da6a335fe13 999047a1a5bf 2024-05-27T09:40:14.839 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (3m) 2m ago 18m 152M 1216M 19.0.0-3868-ga6507388 850a06a84d28 5a0e162d870e 2024-05-27T09:40:14.839 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (3m) 2m ago 18m 167M 1216M 19.0.0-3868-ga6507388 850a06a84d28 effcc7c53dd3 2024-05-27T09:40:14.839 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (2m) 2m ago 17m 78.5M 1216M 19.0.0-3868-ga6507388 850a06a84d28 f0bd34ca416f 2024-05-27T09:40:14.839 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (2m) 26s ago 17m 174M 2240M 19.0.0-3868-ga6507388 850a06a84d28 5e8d3e398c21 2024-05-27T09:40:14.839 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (76s) 26s ago 16m 162M 2240M 19.0.0-3868-ga6507388 850a06a84d28 250e443364de 2024-05-27T09:40:14.839 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (31s) 26s ago 16m 78.5M 2240M 19.0.0-3868-ga6507388 850a06a84d28 90d98c5b2f16 2024-05-27T09:40:14.840 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (8m) 2m ago 20m 71.1M - 2.43.0 a07b618ecd1d cba50454ed21 2024-05-27T09:40:14.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.840+0000 7fbd5c000700 1 -- 172.21.15.119:0/1404378577 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fbd3c078a40 msgr2=0x7fbd3c07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:14.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.840+0000 7fbd5c000700 1 --2- 172.21.15.119:0/1404378577 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fbd3c078a40 0x7fbd3c07aef0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbd480060b0 tx=0x7fbd48007b80 comp rx=0 tx=0).stop 2024-05-27T09:40:14.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.840+0000 7fbd5c000700 1 -- 172.21.15.119:0/1404378577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd54100d50 msgr2=0x7fbd54198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:14.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.840+0000 7fbd5c000700 1 --2- 172.21.15.119:0/1404378577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd54100d50 0x7fbd54198520 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000bda0 tx=0x7fbd4000be80 comp rx=0 tx=0).stop 2024-05-27T09:40:14.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.840+0000 7fbd5c000700 1 -- 172.21.15.119:0/1404378577 shutdown_connections 2024-05-27T09:40:14.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.840+0000 7fbd5c000700 1 --2- 172.21.15.119:0/1404378577 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fbd3c078a40 0x7fbd3c07aef0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:14.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.840+0000 7fbd5c000700 1 --2- 172.21.15.119:0/1404378577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd541003b0 0x7fbd54197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:14.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.840+0000 7fbd5c000700 1 --2- 172.21.15.119:0/1404378577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd54100d50 0x7fbd54198520 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:14.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.840+0000 7fbd5c000700 1 -- 172.21.15.119:0/1404378577 >> 172.21.15.119:0/1404378577 conn(0x7fbd540fbc30 msgr2=0x7fbd540fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:14.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.840+0000 7fbd5c000700 1 -- 172.21.15.119:0/1404378577 shutdown_connections 2024-05-27T09:40:14.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.840+0000 7fbd5c000700 1 -- 172.21.15.119:0/1404378577 wait complete. 2024-05-27T09:40:14.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.991+0000 7faec8600700 1 -- 172.21.15.119:0/2815823517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec00730f0 msgr2=0x7faec00734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:14.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.991+0000 7faec8600700 1 --2- 172.21.15.119:0/2815823517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec00730f0 0x7faec00734c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faeb0009910 tx=0x7faeb0009c20 comp rx=0 tx=0).stop 2024-05-27T09:40:14.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.991+0000 7faec8600700 1 -- 172.21.15.119:0/2815823517 shutdown_connections 2024-05-27T09:40:14.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.992+0000 7faec8600700 1 --2- 172.21.15.119:0/2815823517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faec0073a00 0x7faec010c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:14.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.992+0000 7faec8600700 1 --2- 172.21.15.119:0/2815823517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec00730f0 0x7faec00734c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:14.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.992+0000 7faec8600700 1 -- 172.21.15.119:0/2815823517 >> 172.21.15.119:0/2815823517 conn(0x7faec00fbc30 msgr2=0x7faec00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:14.993 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.992+0000 7faec8600700 1 -- 172.21.15.119:0/2815823517 shutdown_connections 2024-05-27T09:40:14.994 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.992+0000 7faec8600700 1 -- 172.21.15.119:0/2815823517 wait complete. 2024-05-27T09:40:14.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.993+0000 7faec8600700 1 Processor -- start 2024-05-27T09:40:14.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.993+0000 7faec8600700 1 -- start start 2024-05-27T09:40:14.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.993+0000 7faec8600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faec00730f0 0x7faec0197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:14.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.993+0000 7faec8600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec0073a00 0x7faec0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:14.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.993+0000 7faec8600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec019c7c0 con 0x7faec0073a00 2024-05-27T09:40:14.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.993+0000 7faec8600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec019c930 con 0x7faec00730f0 2024-05-27T09:40:14.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.994+0000 7faec5200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec0073a00 0x7faec0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:14.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.994+0000 7faec5200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec0073a00 0x7faec0198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57126/0 (socket says 172.21.15.119:57126) 2024-05-27T09:40:14.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.994+0000 7faec5200700 1 -- 172.21.15.119:0/491048146 learned_addr learned my addr 172.21.15.119:0/491048146 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:40:14.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.994+0000 7faec5c00700 1 --2- 172.21.15.119:0/491048146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faec00730f0 0x7faec0197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:14.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.995+0000 7faec5200700 1 -- 172.21.15.119:0/491048146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faec00730f0 msgr2=0x7faec0197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:14.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.995+0000 7faec5200700 1 --2- 172.21.15.119:0/491048146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faec00730f0 0x7faec0197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:14.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.995+0000 7faec5200700 1 -- 172.21.15.119:0/491048146 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb00095f0 con 0x7faec0073a00 2024-05-27T09:40:14.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.995+0000 7faec5200700 1 --2- 172.21.15.119:0/491048146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec0073a00 0x7faec0198480 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faeac00c8a0 tx=0x7faeac00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:14.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.996+0000 7faebe000700 1 -- 172.21.15.119:0/491048146 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faeac0078e0 con 0x7faec0073a00 2024-05-27T09:40:14.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.996+0000 7faec8600700 1 -- 172.21.15.119:0/491048146 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec019cc10 con 0x7faec0073a00 2024-05-27T09:40:14.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.996+0000 7faebe000700 1 -- 172.21.15.119:0/491048146 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faeac00f450 con 0x7faec0073a00 2024-05-27T09:40:14.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.996+0000 7faebe000700 1 -- 172.21.15.119:0/491048146 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faeac00e5e0 con 0x7faec0073a00 2024-05-27T09:40:14.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.996+0000 7faec8600700 1 -- 172.21.15.119:0/491048146 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec019d160 con 0x7faec0073a00 2024-05-27T09:40:14.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.997+0000 7faec8600700 1 -- 172.21.15.119:0/491048146 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec0109b50 con 0x7faec0073a00 2024-05-27T09:40:14.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.998+0000 7faebe000700 1 -- 172.21.15.119:0/491048146 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7faeac00f5c0 con 0x7faec0073a00 2024-05-27T09:40:15.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.999+0000 7faebe000700 1 --2- 172.21.15.119:0/491048146 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7faea4078a40 0x7faea407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:15.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:14.999+0000 7faebe000700 1 -- 172.21.15.119:0/491048146 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6586+0+0 (secure 0 0 0) 0x7faeac09acd0 con 0x7faec0073a00 2024-05-27T09:40:15.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.000+0000 7faec5c00700 1 --2- 172.21.15.119:0/491048146 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7faea4078a40 0x7faea407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:15.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.000+0000 7faec5c00700 1 --2- 172.21.15.119:0/491048146 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7faea4078a40 0x7faea407aef0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faeb0006010 tx=0x7faeb001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:15.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.005+0000 7faebe000700 1 -- 172.21.15.119:0/491048146 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7faeac062880 con 0x7faec0073a00 2024-05-27T09:40:15.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:15 smithi119 ceph-mon[88269]: pgmap v209: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:15.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:15 smithi119 ceph-mon[88269]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:40:15.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:15 smithi138 ceph-mon[70004]: pgmap v209: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:15.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:15 smithi138 ceph-mon[70004]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:40:15.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.392+0000 7faec8600700 1 -- 172.21.15.119:0/491048146 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7faec004ea50 con 0x7faec0073a00 2024-05-27T09:40:15.393 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.393+0000 7faebe000700 1 -- 172.21.15.119:0/491048146 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7faeac062220 con 0x7faec0073a00 2024-05-27T09:40:15.394 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:40:15.394 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:40:15.394 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:40:15.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:40:15.394 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:40:15.394 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:40:15.394 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:40:15.394 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:40:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 6 2024-05-27T09:40:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:40:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:40:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 3, 2024-05-27T09:40:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 1 2024-05-27T09:40:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:40:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:40:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 3, 2024-05-27T09:40:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 11 2024-05-27T09:40:15.395 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:40:15.395 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:40:15.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.397+0000 7faec8600700 1 -- 172.21.15.119:0/491048146 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7faea4078a40 msgr2=0x7faea407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:15.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.397+0000 7faec8600700 1 --2- 172.21.15.119:0/491048146 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7faea4078a40 0x7faea407aef0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faeb0006010 tx=0x7faeb001a040 comp rx=0 tx=0).stop 2024-05-27T09:40:15.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.397+0000 7faec8600700 1 -- 172.21.15.119:0/491048146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec0073a00 msgr2=0x7faec0198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:15.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.397+0000 7faec8600700 1 --2- 172.21.15.119:0/491048146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec0073a00 0x7faec0198480 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faeac00c8a0 tx=0x7faeac00cbb0 comp rx=0 tx=0).stop 2024-05-27T09:40:15.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.397+0000 7faec8600700 1 -- 172.21.15.119:0/491048146 shutdown_connections 2024-05-27T09:40:15.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.397+0000 7faec8600700 1 --2- 172.21.15.119:0/491048146 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7faea4078a40 0x7faea407aef0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:15.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.397+0000 7faec8600700 1 --2- 172.21.15.119:0/491048146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faec00730f0 0x7faec0197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:15.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.397+0000 7faec8600700 1 --2- 172.21.15.119:0/491048146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec0073a00 0x7faec0198480 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:15.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.397+0000 7faec8600700 1 -- 172.21.15.119:0/491048146 >> 172.21.15.119:0/491048146 conn(0x7faec00fbc30 msgr2=0x7faec0106c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:15.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.397+0000 7faec8600700 1 -- 172.21.15.119:0/491048146 shutdown_connections 2024-05-27T09:40:15.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.398+0000 7faec8600700 1 -- 172.21.15.119:0/491048146 wait complete. 2024-05-27T09:40:15.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.554+0000 7efd0cc00700 1 -- 172.21.15.119:0/2655195123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd081003b0 msgr2=0x7efd08100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:15.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.554+0000 7efd0cc00700 1 --2- 172.21.15.119:0/2655195123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd081003b0 0x7efd08100780 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009910 tx=0x7efcf8009c20 comp rx=0 tx=0).stop 2024-05-27T09:40:15.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.554+0000 7efd0cc00700 1 -- 172.21.15.119:0/2655195123 shutdown_connections 2024-05-27T09:40:15.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.554+0000 7efd0cc00700 1 --2- 172.21.15.119:0/2655195123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd08100d50 0x7efd08104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:15.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.554+0000 7efd0cc00700 1 --2- 172.21.15.119:0/2655195123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd081003b0 0x7efd08100780 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.554+0000 7efd0cc00700 1 -- 172.21.15.119:0/2655195123 >> 172.21.15.119:0/2655195123 conn(0x7efd080fbc30 msgr2=0x7efd080fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.554+0000 7efd0cc00700 1 -- 172.21.15.119:0/2655195123 shutdown_connections 2024-05-27T09:40:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.554+0000 7efd0cc00700 1 -- 172.21.15.119:0/2655195123 wait complete. 2024-05-27T09:40:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.555+0000 7efd0cc00700 1 Processor -- start 2024-05-27T09:40:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.555+0000 7efd0cc00700 1 -- start start 2024-05-27T09:40:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.556+0000 7efd0cc00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd081003b0 0x7efd08073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.556+0000 7efd0cc00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd08100d50 0x7efd08073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.556+0000 7efd0cc00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd08073ad0 con 0x7efd081003b0 2024-05-27T09:40:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.556+0000 7efd0cc00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd08073c10 con 0x7efd08100d50 2024-05-27T09:40:15.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.556+0000 7efd05e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd081003b0 0x7efd08073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:15.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.556+0000 7efd05e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd081003b0 0x7efd08073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57152/0 (socket says 172.21.15.119:57152) 2024-05-27T09:40:15.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.556+0000 7efd05e00700 1 -- 172.21.15.119:0/152505813 learned_addr learned my addr 172.21.15.119:0/152505813 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:40:15.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.556+0000 7efd05400700 1 --2- 172.21.15.119:0/152505813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd08100d50 0x7efd08073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:15.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.557+0000 7efd05e00700 1 -- 172.21.15.119:0/152505813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd08100d50 msgr2=0x7efd08073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:15.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.557+0000 7efd05e00700 1 --2- 172.21.15.119:0/152505813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd08100d50 0x7efd08073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:15.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.557+0000 7efd05e00700 1 -- 172.21.15.119:0/152505813 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf80095f0 con 0x7efd081003b0 2024-05-27T09:40:15.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.557+0000 7efd05400700 1 --2- 172.21.15.119:0/152505813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd08100d50 0x7efd08073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:40:15.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.557+0000 7efd05e00700 1 --2- 172.21.15.119:0/152505813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd081003b0 0x7efd08073050 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efcf8006010 tx=0x7efcf800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:15.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.558+0000 7efcfea00700 1 -- 172.21.15.119:0/152505813 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efcf801d070 con 0x7efd081003b0 2024-05-27T09:40:15.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.558+0000 7efd0cc00700 1 -- 172.21.15.119:0/152505813 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd08073e90 con 0x7efd081003b0 2024-05-27T09:40:15.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.558+0000 7efcfea00700 1 -- 172.21.15.119:0/152505813 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efcf8004fb0 con 0x7efd081003b0 2024-05-27T09:40:15.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.558+0000 7efcfea00700 1 -- 172.21.15.119:0/152505813 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efcf8022ea0 con 0x7efd081003b0 2024-05-27T09:40:15.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.558+0000 7efd0cc00700 1 -- 172.21.15.119:0/152505813 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd081a2320 con 0x7efd081003b0 2024-05-27T09:40:15.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.559+0000 7efd0cc00700 1 -- 172.21.15.119:0/152505813 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd081086c0 con 0x7efd081003b0 2024-05-27T09:40:15.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.564+0000 7efcfea00700 1 -- 172.21.15.119:0/152505813 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7efcf8022470 con 0x7efd081003b0 2024-05-27T09:40:15.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.565+0000 7efcfea00700 1 --2- 172.21.15.119:0/152505813 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efcec078a40 0x7efcec07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:15.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.565+0000 7efcfea00700 1 -- 172.21.15.119:0/152505813 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efcf809c760 con 0x7efd081003b0 2024-05-27T09:40:15.566 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.566+0000 7efd05400700 1 --2- 172.21.15.119:0/152505813 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efcec078a40 0x7efcec07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:15.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.566+0000 7efd05400700 1 --2- 172.21.15.119:0/152505813 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efcec078a40 0x7efcec07aef0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efd08074840 tx=0x7efcf4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:15.569 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.568+0000 7efcfea00700 1 -- 172.21.15.119:0/152505813 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7efcf8064560 con 0x7efd081003b0 2024-05-27T09:40:15.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.858+0000 7efd0cc00700 1 -- 172.21.15.119:0/152505813 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efd0804ea50 con 0x7efd081003b0 2024-05-27T09:40:15.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.858+0000 7efcfea00700 1 -- 172.21.15.119:0/152505813 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1877 (secure 0 0 0) 0x7efcf8022720 con 0x7efd081003b0 2024-05-27T09:40:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:e26 2024-05-27T09:40:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T09:40:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:40:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:legacy client fscid: 1 2024-05-27T09:40:15.861 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:40:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:Filesystem 'cephfs' (1) 2024-05-27T09:40:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:fs_name cephfs 2024-05-27T09:40:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 23 2024-05-27T09:40:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-27T09:40:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-05-27T09:25:53.387388+0000 2024-05-27T09:40:15.861 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2024-05-27T09:39:57.913154+0000 2024-05-27T09:40:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:tableserver 0 2024-05-27T09:40:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:root 0 2024-05-27T09:40:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:session_timeout 60 2024-05-27T09:40:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:session_autoclose 300 2024-05-27T09:40:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:max_file_size 1099511627776 2024-05-27T09:40:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:max_xattr_size 65536 2024-05-27T09:40:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:required_client_features {} 2024-05-27T09:40:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure 0 2024-05-27T09:40:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure_osd_epoch 81 2024-05-27T09:40:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:40:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:max_mds 2 2024-05-27T09:40:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:in 0,1 2024-05-27T09:40:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:up {} 2024-05-27T09:40:15.862 INFO:teuthology.orchestra.run.smithi119.stdout:failed 0,1 2024-05-27T09:40:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:damaged 2024-05-27T09:40:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:stopped 2024-05-27T09:40:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:data_pools [3] 2024-05-27T09:40:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:metadata_pool 2 2024-05-27T09:40:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:inline_data enabled 2024-05-27T09:40:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:balancer 2024-05-27T09:40:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:bal_rank_mask -1 2024-05-27T09:40:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:standby_count_wanted 1 2024-05-27T09:40:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:qdb_cluster leader: 0 members: 2024-05-27T09:40:15.863 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:40:15.863 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:40:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:Standby daemons: 2024-05-27T09:40:15.863 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:40:15.863 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.alszpk{-1:24253} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/2345607053,v1:172.21.15.138:6827/2345607053] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:40:15.864 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.ppmzzk{-1:34428} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.119:6828/2854413001,v1:172.21.15.119:6829/2854413001] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:40:15.864 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.nghcey{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.119:6826/3357235708,v1:172.21.15.119:6827/3357235708] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T09:40:15.864 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.ndfkhd{-1:44261} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/2055700210,v1:172.21.15.138:6825/2055700210] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:40:15.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.861+0000 7efd0cc00700 1 -- 172.21.15.119:0/152505813 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efcec078a40 msgr2=0x7efcec07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:15.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.861+0000 7efd0cc00700 1 --2- 172.21.15.119:0/152505813 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efcec078a40 0x7efcec07aef0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efd08074840 tx=0x7efcf4008040 comp rx=0 tx=0).stop 2024-05-27T09:40:15.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.861+0000 7efd0cc00700 1 -- 172.21.15.119:0/152505813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd081003b0 msgr2=0x7efd08073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:15.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.861+0000 7efd0cc00700 1 --2- 172.21.15.119:0/152505813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd081003b0 0x7efd08073050 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efcf8006010 tx=0x7efcf800be20 comp rx=0 tx=0).stop 2024-05-27T09:40:15.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.862+0000 7efd0cc00700 1 -- 172.21.15.119:0/152505813 shutdown_connections 2024-05-27T09:40:15.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.862+0000 7efd0cc00700 1 --2- 172.21.15.119:0/152505813 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efcec078a40 0x7efcec07aef0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:15.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.862+0000 7efd0cc00700 1 --2- 172.21.15.119:0/152505813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd081003b0 0x7efd08073050 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:15.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.862+0000 7efd0cc00700 1 --2- 172.21.15.119:0/152505813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd08100d50 0x7efd08073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:15.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.862+0000 7efd0cc00700 1 -- 172.21.15.119:0/152505813 >> 172.21.15.119:0/152505813 conn(0x7efd080fbc30 msgr2=0x7efd080fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:15.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.862+0000 7efd0cc00700 1 -- 172.21.15.119:0/152505813 shutdown_connections 2024-05-27T09:40:15.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:15.862+0000 7efd0cc00700 1 -- 172.21.15.119:0/152505813 wait complete. 2024-05-27T09:40:15.865 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 26 2024-05-27T09:40:16.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.011+0000 7ff06d000700 1 -- 172.21.15.119:0/2089024458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff068105f90 msgr2=0x7ff068106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:16.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.011+0000 7ff06d000700 1 --2- 172.21.15.119:0/2089024458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff068105f90 0x7ff068106360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff058009910 tx=0x7ff058009c20 comp rx=0 tx=0).stop 2024-05-27T09:40:16.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.011+0000 7ff06d000700 1 -- 172.21.15.119:0/2089024458 shutdown_connections 2024-05-27T09:40:16.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.011+0000 7ff06d000700 1 --2- 172.21.15.119:0/2089024458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0680683d0 0x7ff068068840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:16.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.011+0000 7ff06d000700 1 --2- 172.21.15.119:0/2089024458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff068105f90 0x7ff068106360 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:16.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.012+0000 7ff06d000700 1 -- 172.21.15.119:0/2089024458 >> 172.21.15.119:0/2089024458 conn(0x7ff068074d00 msgr2=0x7ff068075100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:16.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.012+0000 7ff06d000700 1 -- 172.21.15.119:0/2089024458 shutdown_connections 2024-05-27T09:40:16.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.012+0000 7ff06d000700 1 -- 172.21.15.119:0/2089024458 wait complete. 2024-05-27T09:40:16.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.013+0000 7ff06d000700 1 Processor -- start 2024-05-27T09:40:16.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.013+0000 7ff06d000700 1 -- start start 2024-05-27T09:40:16.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.013+0000 7ff06d000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0680683d0 0x7ff0680ff040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:16.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.013+0000 7ff06d000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff068105f90 0x7ff0680ff580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:16.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.013+0000 7ff06d000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0680ffac0 con 0x7ff0680683d0 2024-05-27T09:40:16.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.013+0000 7ff06d000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0680ffc00 con 0x7ff068105f90 2024-05-27T09:40:16.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.014+0000 7ff067400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0680683d0 0x7ff0680ff040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:16.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.014+0000 7ff067400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0680683d0 0x7ff0680ff040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57174/0 (socket says 172.21.15.119:57174) 2024-05-27T09:40:16.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.014+0000 7ff067400700 1 -- 172.21.15.119:0/320370235 learned_addr learned my addr 172.21.15.119:0/320370235 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:40:16.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.014+0000 7ff066a00700 1 --2- 172.21.15.119:0/320370235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff068105f90 0x7ff0680ff580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:16.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.014+0000 7ff067400700 1 -- 172.21.15.119:0/320370235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff068105f90 msgr2=0x7ff0680ff580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:16.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.014+0000 7ff067400700 1 --2- 172.21.15.119:0/320370235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff068105f90 0x7ff0680ff580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:16.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.014+0000 7ff067400700 1 -- 172.21.15.119:0/320370235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0580095f0 con 0x7ff0680683d0 2024-05-27T09:40:16.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.015+0000 7ff067400700 1 --2- 172.21.15.119:0/320370235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0680683d0 0x7ff0680ff040 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff058006010 tx=0x7ff05800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:16.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.015+0000 7ff05fe00700 1 -- 172.21.15.119:0/320370235 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff05801d070 con 0x7ff0680683d0 2024-05-27T09:40:16.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.015+0000 7ff05fe00700 1 -- 172.21.15.119:0/320370235 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff058004fb0 con 0x7ff0680683d0 2024-05-27T09:40:16.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.015+0000 7ff05fe00700 1 -- 172.21.15.119:0/320370235 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff05800f690 con 0x7ff0680683d0 2024-05-27T09:40:16.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.015+0000 7ff06d000700 1 -- 172.21.15.119:0/320370235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0680ffe80 con 0x7ff0680683d0 2024-05-27T09:40:16.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.015+0000 7ff06d000700 1 -- 172.21.15.119:0/320370235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0681a2080 con 0x7ff0680683d0 2024-05-27T09:40:16.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.016+0000 7ff06d000700 1 -- 172.21.15.119:0/320370235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff068108470 con 0x7ff0680683d0 2024-05-27T09:40:16.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.018+0000 7ff05fe00700 1 -- 172.21.15.119:0/320370235 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7ff0580228a0 con 0x7ff0680683d0 2024-05-27T09:40:16.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.019+0000 7ff05fe00700 1 --2- 172.21.15.119:0/320370235 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff04c078a40 0x7ff04c07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:16.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.019+0000 7ff05fe00700 1 -- 172.21.15.119:0/320370235 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff05809c5b0 con 0x7ff0680683d0 2024-05-27T09:40:16.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.019+0000 7ff066a00700 1 --2- 172.21.15.119:0/320370235 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff04c078a40 0x7ff04c07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:16.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.020+0000 7ff066a00700 1 --2- 172.21.15.119:0/320370235 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff04c078a40 0x7ff04c07aef0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff054005fd0 tx=0x7ff05400a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:16.024 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.024+0000 7ff05fe00700 1 -- 172.21.15.119:0/320370235 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7ff058064160 con 0x7ff0680683d0 2024-05-27T09:40:16.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:16 smithi119 ceph-mon[88269]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:40:16.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:16 smithi119 ceph-mon[88269]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:40:16.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:16 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/491048146' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:40:16.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:16 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:16.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:16 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:16.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:16 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/152505813' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:40:16.272 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.271+0000 7ff06d000700 1 -- 172.21.15.119:0/320370235 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff06802cdf0 con 0x7ff04c078a40 2024-05-27T09:40:16.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.274+0000 7ff05fe00700 1 -- 172.21.15.119:0/320370235 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff06802cdf0 con 0x7ff04c078a40 2024-05-27T09:40:16.275 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:40:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:40:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:40:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-27T09:40:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [ 2024-05-27T09:40:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: "crash", 2024-05-27T09:40:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: "mon", 2024-05-27T09:40:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: "osd", 2024-05-27T09:40:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr" 2024-05-27T09:40:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-05-27T09:40:16.275 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "13/23 daemons upgraded", 2024-05-27T09:40:16.276 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "Currently upgrading mds daemons", 2024-05-27T09:40:16.276 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:40:16.276 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:40:16.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.277+0000 7ff06d000700 1 -- 172.21.15.119:0/320370235 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff04c078a40 msgr2=0x7ff04c07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:16.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.277+0000 7ff06d000700 1 --2- 172.21.15.119:0/320370235 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff04c078a40 0x7ff04c07aef0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff054005fd0 tx=0x7ff05400a560 comp rx=0 tx=0).stop 2024-05-27T09:40:16.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.277+0000 7ff06d000700 1 -- 172.21.15.119:0/320370235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0680683d0 msgr2=0x7ff0680ff040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:16.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.277+0000 7ff06d000700 1 --2- 172.21.15.119:0/320370235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0680683d0 0x7ff0680ff040 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff058006010 tx=0x7ff05800be20 comp rx=0 tx=0).stop 2024-05-27T09:40:16.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.277+0000 7ff06d000700 1 -- 172.21.15.119:0/320370235 shutdown_connections 2024-05-27T09:40:16.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.277+0000 7ff06d000700 1 --2- 172.21.15.119:0/320370235 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff04c078a40 0x7ff04c07aef0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:16.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.277+0000 7ff06d000700 1 --2- 172.21.15.119:0/320370235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0680683d0 0x7ff0680ff040 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:16.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.277+0000 7ff06d000700 1 --2- 172.21.15.119:0/320370235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff068105f90 0x7ff0680ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:16.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.277+0000 7ff06d000700 1 -- 172.21.15.119:0/320370235 >> 172.21.15.119:0/320370235 conn(0x7ff068074d00 msgr2=0x7ff0680fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:16.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.277+0000 7ff06d000700 1 -- 172.21.15.119:0/320370235 shutdown_connections 2024-05-27T09:40:16.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.277+0000 7ff06d000700 1 -- 172.21.15.119:0/320370235 wait complete. 2024-05-27T09:40:16.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:16 smithi138 ceph-mon[70004]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:40:16.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:16 smithi138 ceph-mon[70004]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:40:16.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:16 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/491048146' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:40:16.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:16 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:16.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:16 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:16.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:16 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/152505813' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:40:16.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.429+0000 7efbf9a00700 1 -- 172.21.15.119:0/4221913179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf40ff390 msgr2=0x7efbf40ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:16.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.429+0000 7efbf9a00700 1 --2- 172.21.15.119:0/4221913179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf40ff390 0x7efbf40ff760 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efbe4009910 tx=0x7efbe4009c20 comp rx=0 tx=0).stop 2024-05-27T09:40:16.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.430+0000 7efbf9a00700 1 -- 172.21.15.119:0/4221913179 shutdown_connections 2024-05-27T09:40:16.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.430+0000 7efbf9a00700 1 --2- 172.21.15.119:0/4221913179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf40ffd30 0x7efbf4103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:16.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.430+0000 7efbf9a00700 1 --2- 172.21.15.119:0/4221913179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf40ff390 0x7efbf40ff760 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:16.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.430+0000 7efbf9a00700 1 -- 172.21.15.119:0/4221913179 >> 172.21.15.119:0/4221913179 conn(0x7efbf4074f50 msgr2=0x7efbf4075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:16.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.430+0000 7efbf9a00700 1 -- 172.21.15.119:0/4221913179 shutdown_connections 2024-05-27T09:40:16.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.430+0000 7efbf9a00700 1 -- 172.21.15.119:0/4221913179 wait complete. 2024-05-27T09:40:16.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.431+0000 7efbf9a00700 1 Processor -- start 2024-05-27T09:40:16.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.431+0000 7efbf9a00700 1 -- start start 2024-05-27T09:40:16.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.431+0000 7efbf9a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf40ff390 0x7efbf4197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:16.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.431+0000 7efbf9a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf40ffd30 0x7efbf4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:16.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.431+0000 7efbf9a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf4198b50 con 0x7efbf40ff390 2024-05-27T09:40:16.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.432+0000 7efbf9a00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf419c8e0 con 0x7efbf40ffd30 2024-05-27T09:40:16.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.432+0000 7efbf2a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf40ff390 0x7efbf4197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:16.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.432+0000 7efbf2a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf40ff390 0x7efbf4197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57194/0 (socket says 172.21.15.119:57194) 2024-05-27T09:40:16.433 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.432+0000 7efbf2a00700 1 -- 172.21.15.119:0/1258626100 learned_addr learned my addr 172.21.15.119:0/1258626100 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:40:16.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.432+0000 7efbf2000700 1 --2- 172.21.15.119:0/1258626100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf40ffd30 0x7efbf4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:16.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.432+0000 7efbf2a00700 1 -- 172.21.15.119:0/1258626100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf40ffd30 msgr2=0x7efbf4198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:16.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.432+0000 7efbf2a00700 1 --2- 172.21.15.119:0/1258626100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf40ffd30 0x7efbf4198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:16.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.432+0000 7efbf2a00700 1 -- 172.21.15.119:0/1258626100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbe40095f0 con 0x7efbf40ff390 2024-05-27T09:40:16.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.433+0000 7efbf2a00700 1 --2- 172.21.15.119:0/1258626100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf40ff390 0x7efbf4197f30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efbe400b5c0 tx=0x7efbe400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:16.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.433+0000 7efbeb400700 1 -- 172.21.15.119:0/1258626100 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbe401d070 con 0x7efbf40ff390 2024-05-27T09:40:16.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.433+0000 7efbeb400700 1 -- 172.21.15.119:0/1258626100 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efbe400f460 con 0x7efbf40ff390 2024-05-27T09:40:16.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.433+0000 7efbf9a00700 1 -- 172.21.15.119:0/1258626100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf419cb60 con 0x7efbf40ff390 2024-05-27T09:40:16.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.433+0000 7efbeb400700 1 -- 172.21.15.119:0/1258626100 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbe4017610 con 0x7efbf40ff390 2024-05-27T09:40:16.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.433+0000 7efbf9a00700 1 -- 172.21.15.119:0/1258626100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf419d050 con 0x7efbf40ff390 2024-05-27T09:40:16.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.434+0000 7efbf9a00700 1 -- 172.21.15.119:0/1258626100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf404ea50 con 0x7efbf40ff390 2024-05-27T09:40:16.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.436+0000 7efbeb400700 1 -- 172.21.15.119:0/1258626100 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7efbe400f5d0 con 0x7efbf40ff390 2024-05-27T09:40:16.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.436+0000 7efbeb400700 1 --2- 172.21.15.119:0/1258626100 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efbd8078a40 0x7efbd807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:16.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.437+0000 7efbeb400700 1 -- 172.21.15.119:0/1258626100 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efbe409c360 con 0x7efbf40ff390 2024-05-27T09:40:16.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.437+0000 7efbf2000700 1 --2- 172.21.15.119:0/1258626100 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efbd8078a40 0x7efbd807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:16.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.437+0000 7efbf2000700 1 --2- 172.21.15.119:0/1258626100 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efbd8078a40 0x7efbd807aef0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efbf4199550 tx=0x7efbe0005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:16.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.444+0000 7efbeb400700 1 -- 172.21.15.119:0/1258626100 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7efbe4063f10 con 0x7efbf40ff390 2024-05-27T09:40:16.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.784+0000 7efbf9a00700 1 -- 172.21.15.119:0/1258626100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7efbf4199290 con 0x7efbf40ff390 2024-05-27T09:40:16.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.785+0000 7efbeb400700 1 -- 172.21.15.119:0/1258626100 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7efbe40638b0 con 0x7efbf40ff390 2024-05-27T09:40:16.786 INFO:teuthology.orchestra.run.smithi119.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-27T09:40:16.786 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-27T09:40:16.786 INFO:teuthology.orchestra.run.smithi119.stdout: fs cephfs is degraded 2024-05-27T09:40:16.786 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:40:16.786 INFO:teuthology.orchestra.run.smithi119.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:40:16.786 INFO:teuthology.orchestra.run.smithi119.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-27T09:40:16.786 INFO:teuthology.orchestra.run.smithi119.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-27T09:40:16.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.788+0000 7efbf9a00700 1 -- 172.21.15.119:0/1258626100 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efbd8078a40 msgr2=0x7efbd807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:16.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.788+0000 7efbf9a00700 1 --2- 172.21.15.119:0/1258626100 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efbd8078a40 0x7efbd807aef0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efbf4199550 tx=0x7efbe0005cf0 comp rx=0 tx=0).stop 2024-05-27T09:40:16.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.788+0000 7efbf9a00700 1 -- 172.21.15.119:0/1258626100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf40ff390 msgr2=0x7efbf4197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:16.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.788+0000 7efbf9a00700 1 --2- 172.21.15.119:0/1258626100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf40ff390 0x7efbf4197f30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efbe400b5c0 tx=0x7efbe400be00 comp rx=0 tx=0).stop 2024-05-27T09:40:16.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.788+0000 7efbf9a00700 1 -- 172.21.15.119:0/1258626100 shutdown_connections 2024-05-27T09:40:16.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.788+0000 7efbf9a00700 1 --2- 172.21.15.119:0/1258626100 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efbd8078a40 0x7efbd807aef0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:16.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.788+0000 7efbf9a00700 1 --2- 172.21.15.119:0/1258626100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf40ff390 0x7efbf4197f30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:16.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.788+0000 7efbf9a00700 1 --2- 172.21.15.119:0/1258626100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf40ffd30 0x7efbf4198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:16.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.788+0000 7efbf9a00700 1 -- 172.21.15.119:0/1258626100 >> 172.21.15.119:0/1258626100 conn(0x7efbf4074f50 msgr2=0x7efbf4068450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:16.789 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.788+0000 7efbf9a00700 1 -- 172.21.15.119:0/1258626100 shutdown_connections 2024-05-27T09:40:16.790 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:16.788+0000 7efbf9a00700 1 -- 172.21.15.119:0/1258626100 wait complete. 2024-05-27T09:40:17.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:17 smithi119 ceph-mon[88269]: pgmap v210: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:17.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:17 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/1258626100' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:40:17.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:17 smithi138 ceph-mon[70004]: pgmap v210: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:17.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:17 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/1258626100' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:40:18.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:18 smithi138 ceph-mon[70004]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:40:18.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:18 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:18.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:18 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:18.420 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:18 smithi119 ceph-mon[88269]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:40:18.420 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:18 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:18.420 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:18 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:19.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:19 smithi138 ceph-mon[70004]: pgmap v211: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:19.474 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:19 smithi119 ceph-mon[88269]: pgmap v211: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:21.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:21 smithi138 ceph-mon[70004]: pgmap v212: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:21.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:21 smithi119 ceph-mon[88269]: pgmap v212: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:23.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:23 smithi119 ceph-mon[88269]: pgmap v213: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:23.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:23 smithi138 ceph-mon[70004]: pgmap v213: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:24.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:24 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:24.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:24 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:24.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:24 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:40:24.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:24 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:40:24.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:24 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:24.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:24 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:40:24.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:24 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:40:24.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:24 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:40:24.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:24 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:40:24.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:24 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:40:24.419 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:24 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:24.419 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:24 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:24.419 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:24 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:40:24.419 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:24 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:40:24.419 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:24 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:24.419 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:24 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:40:24.419 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:24 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:40:24.419 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:24 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:40:24.420 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:24 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:40:24.420 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:24 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:40:25.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:25 smithi138 ceph-mon[70004]: pgmap v214: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:25.419 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:25 smithi119 ceph-mon[88269]: pgmap v214: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:27.131 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:26 smithi119 ceph-mon[88269]: pgmap v215: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:27.131 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:26 smithi119 ceph-mon[88269]: Upgrade: Updating mds.cephfs.smithi119.ppmzzk 2024-05-27T09:40:27.131 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:26 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:27.131 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:26 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi119.ppmzzk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:40:27.131 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:26 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:40:27.131 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:26 smithi119 ceph-mon[88269]: Deploying daemon mds.cephfs.smithi119.ppmzzk on smithi119 2024-05-27T09:40:27.189 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:26 smithi138 ceph-mon[70004]: pgmap v215: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:27.189 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:26 smithi138 ceph-mon[70004]: Upgrade: Updating mds.cephfs.smithi119.ppmzzk 2024-05-27T09:40:27.189 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:26 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:27.189 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:26 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi119.ppmzzk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:40:27.189 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:26 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:40:27.189 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:26 smithi138 ceph-mon[70004]: Deploying daemon mds.cephfs.smithi119.ppmzzk on smithi119 2024-05-27T09:40:28.097 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:27 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:40:28.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:27 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:40:29.238 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:28 smithi119 ceph-mon[88269]: pgmap v216: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:29.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:28 smithi138 ceph-mon[70004]: pgmap v216: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:31.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:30 smithi119 ceph-mon[88269]: pgmap v217: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:31.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:30 smithi119 ceph-mon[88269]: osdmap e83: 6 total, 6 up, 6 in 2024-05-27T09:40:31.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:30 smithi119 ceph-mon[88269]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-27T09:40:31.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:30 smithi138 ceph-mon[70004]: pgmap v217: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:31.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:30 smithi138 ceph-mon[70004]: osdmap e83: 6 total, 6 up, 6 in 2024-05-27T09:40:31.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:30 smithi138 ceph-mon[70004]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-27T09:40:33.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:32 smithi119 ceph-mon[88269]: pgmap v219: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:33.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:32 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:33.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:32 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:33.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:32 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:40:33.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:32 smithi138 ceph-mon[70004]: pgmap v219: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:33.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:32 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:33.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:32 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:33.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:32 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:40:34.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:33 smithi119 ceph-mon[88269]: mds.? [v2:172.21.15.119:6828/2430623742,v1:172.21.15.119:6829/2430623742] up:boot 2024-05-27T09:40:34.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:33 smithi119 ceph-mon[88269]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-27T09:40:34.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:33 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.ppmzzk"}]: dispatch 2024-05-27T09:40:34.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:33 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:34.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:33 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:34.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:33 smithi138 ceph-mon[70004]: mds.? [v2:172.21.15.119:6828/2430623742,v1:172.21.15.119:6829/2430623742] up:boot 2024-05-27T09:40:34.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:33 smithi138 ceph-mon[70004]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-27T09:40:34.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:33 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi119.ppmzzk"}]: dispatch 2024-05-27T09:40:34.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:33 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:34.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:33 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:35.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:34 smithi119 ceph-mon[88269]: pgmap v220: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:35.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:34 smithi138 ceph-mon[70004]: pgmap v220: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:37.157 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:36 smithi119 ceph-mon[88269]: pgmap v221: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:37.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:36 smithi138 ceph-mon[70004]: pgmap v221: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:39.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:38 smithi119 ceph-mon[88269]: pgmap v222: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:39.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:38 smithi138 ceph-mon[70004]: pgmap v222: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:41.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:40 smithi138 ceph-mon[70004]: pgmap v223: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:41.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:40 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:41.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:40 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:41.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:40 smithi119 ceph-mon[88269]: pgmap v223: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:41.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:40 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:41.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:40 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:43.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:42 smithi138 ceph-mon[70004]: pgmap v224: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:43.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:42 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:43.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:42 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:43.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:42 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:40:43.168 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:42 smithi119 ceph-mon[88269]: pgmap v224: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:43.168 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:42 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:43.168 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:42 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:43.168 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:42 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:40:44.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:44 smithi119 ceph-mon[88269]: pgmap v225: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:45.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:44 smithi138 ceph-mon[70004]: pgmap v225: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:46.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.964+0000 7fcfa9400700 1 -- 172.21.15.119:0/2453465604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfa4101970 msgr2=0x7fcfa410a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:46.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.964+0000 7fcfa9400700 1 --2- 172.21.15.119:0/2453465604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfa4101970 0x7fcfa410a040 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009910 tx=0x7fcf90009c20 comp rx=0 tx=0).stop 2024-05-27T09:40:46.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.964+0000 7fcfa9400700 1 -- 172.21.15.119:0/2453465604 shutdown_connections 2024-05-27T09:40:46.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.964+0000 7fcfa9400700 1 --2- 172.21.15.119:0/2453465604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfa4101970 0x7fcfa410a040 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:46.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.964+0000 7fcfa9400700 1 --2- 172.21.15.119:0/2453465604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfa4101060 0x7fcfa4101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:46.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.964+0000 7fcfa9400700 1 -- 172.21.15.119:0/2453465604 >> 172.21.15.119:0/2453465604 conn(0x7fcfa40fa9d0 msgr2=0x7fcfa40fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:46.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.964+0000 7fcfa9400700 1 -- 172.21.15.119:0/2453465604 shutdown_connections 2024-05-27T09:40:46.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.964+0000 7fcfa9400700 1 -- 172.21.15.119:0/2453465604 wait complete. 2024-05-27T09:40:46.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.965+0000 7fcfa9400700 1 Processor -- start 2024-05-27T09:40:46.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.965+0000 7fcfa9400700 1 -- start start 2024-05-27T09:40:46.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.966+0000 7fcfa9400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfa4101060 0x7fcfa419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:46.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.966+0000 7fcfa9400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfa4101970 0x7fcfa419be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:46.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.966+0000 7fcfa9400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa419c4c0 con 0x7fcfa4101060 2024-05-27T09:40:46.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.966+0000 7fcfa9400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa4195970 con 0x7fcfa4101970 2024-05-27T09:40:46.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.966+0000 7fcfa3e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfa4101060 0x7fcfa419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:46.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.966+0000 7fcfa3e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfa4101060 0x7fcfa419b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60182/0 (socket says 172.21.15.119:60182) 2024-05-27T09:40:46.967 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.966+0000 7fcfa3e00700 1 -- 172.21.15.119:0/3492618645 learned_addr learned my addr 172.21.15.119:0/3492618645 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:40:46.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.966+0000 7fcfa3400700 1 --2- 172.21.15.119:0/3492618645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfa4101970 0x7fcfa419be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:46.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.967+0000 7fcfa3e00700 1 -- 172.21.15.119:0/3492618645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfa4101970 msgr2=0x7fcfa419be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:46.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.967+0000 7fcfa3e00700 1 --2- 172.21.15.119:0/3492618645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfa4101970 0x7fcfa419be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:46.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.967+0000 7fcfa3e00700 1 -- 172.21.15.119:0/3492618645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf900095f0 con 0x7fcfa4101060 2024-05-27T09:40:46.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.967+0000 7fcfa3400700 1 --2- 172.21.15.119:0/3492618645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfa4101970 0x7fcfa419be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:40:46.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.967+0000 7fcfa3e00700 1 --2- 172.21.15.119:0/3492618645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfa4101060 0x7fcfa419b8f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400c8a0 tx=0x7fcf9400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:46.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.967+0000 7fcfa0c00700 1 -- 172.21.15.119:0/3492618645 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf9400cea0 con 0x7fcfa4101060 2024-05-27T09:40:46.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.967+0000 7fcfa0c00700 1 -- 172.21.15.119:0/3492618645 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf94004830 con 0x7fcfa4101060 2024-05-27T09:40:46.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.967+0000 7fcfa0c00700 1 -- 172.21.15.119:0/3492618645 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf940055b0 con 0x7fcfa4101060 2024-05-27T09:40:46.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.967+0000 7fcfa9400700 1 -- 172.21.15.119:0/3492618645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa4195c50 con 0x7fcfa4101060 2024-05-27T09:40:46.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.967+0000 7fcfa9400700 1 -- 172.21.15.119:0/3492618645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa41961a0 con 0x7fcfa4101060 2024-05-27T09:40:46.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.969+0000 7fcfa9400700 1 -- 172.21.15.119:0/3492618645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa41076e0 con 0x7fcfa4101060 2024-05-27T09:40:46.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.970+0000 7fcfa0c00700 1 -- 172.21.15.119:0/3492618645 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fcf940049a0 con 0x7fcfa4101060 2024-05-27T09:40:46.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.971+0000 7fcfa0c00700 1 --2- 172.21.15.119:0/3492618645 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fcf88078a40 0x7fcf8807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:46.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.971+0000 7fcfa0c00700 1 -- 172.21.15.119:0/3492618645 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcf9409a950 con 0x7fcfa4101060 2024-05-27T09:40:46.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.972+0000 7fcfa3400700 1 --2- 172.21.15.119:0/3492618645 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fcf88078a40 0x7fcf8807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:46.973 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.973+0000 7fcfa3400700 1 --2- 172.21.15.119:0/3492618645 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fcf88078a40 0x7fcf8807aef0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcf90006010 tx=0x7fcf9000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:46.977 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:46.977+0000 7fcfa0c00700 1 -- 172.21.15.119:0/3492618645 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fcf940625a0 con 0x7fcfa4101060 2024-05-27T09:40:47.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:46 smithi138 ceph-mon[70004]: pgmap v226: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:47.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.239+0000 7fcfa9400700 1 -- 172.21.15.119:0/3492618645 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfa4196570 con 0x7fcf88078a40 2024-05-27T09:40:47.240 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:46 smithi119 ceph-mon[88269]: pgmap v226: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:47.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.243+0000 7fcfa0c00700 1 -- 172.21.15.119:0/3492618645 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcfa4196570 con 0x7fcf88078a40 2024-05-27T09:40:47.246 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.246+0000 7fcfa9400700 1 -- 172.21.15.119:0/3492618645 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fcf88078a40 msgr2=0x7fcf8807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:47.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.246+0000 7fcfa9400700 1 --2- 172.21.15.119:0/3492618645 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fcf88078a40 0x7fcf8807aef0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcf90006010 tx=0x7fcf9000b540 comp rx=0 tx=0).stop 2024-05-27T09:40:47.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.246+0000 7fcfa9400700 1 -- 172.21.15.119:0/3492618645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfa4101060 msgr2=0x7fcfa419b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:47.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.246+0000 7fcfa9400700 1 --2- 172.21.15.119:0/3492618645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfa4101060 0x7fcfa419b8f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400c8a0 tx=0x7fcf9400cc60 comp rx=0 tx=0).stop 2024-05-27T09:40:47.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.246+0000 7fcfa9400700 1 -- 172.21.15.119:0/3492618645 shutdown_connections 2024-05-27T09:40:47.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.246+0000 7fcfa9400700 1 --2- 172.21.15.119:0/3492618645 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fcf88078a40 0x7fcf8807aef0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:47.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.246+0000 7fcfa9400700 1 --2- 172.21.15.119:0/3492618645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfa4101060 0x7fcfa419b8f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:47.247 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.246+0000 7fcfa9400700 1 --2- 172.21.15.119:0/3492618645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfa4101970 0x7fcfa419be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:47.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.246+0000 7fcfa9400700 1 -- 172.21.15.119:0/3492618645 >> 172.21.15.119:0/3492618645 conn(0x7fcfa40fa9d0 msgr2=0x7fcfa4104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:47.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.246+0000 7fcfa9400700 1 -- 172.21.15.119:0/3492618645 shutdown_connections 2024-05-27T09:40:47.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.246+0000 7fcfa9400700 1 -- 172.21.15.119:0/3492618645 wait complete. 2024-05-27T09:40:47.263 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:40:47.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.425+0000 7f1165600700 1 -- 172.21.15.119:0/137579392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11601003b0 msgr2=0x7f1160100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:47.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.425+0000 7f1165600700 1 --2- 172.21.15.119:0/137579392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11601003b0 0x7f1160100780 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1150009910 tx=0x7f1150009c20 comp rx=0 tx=0).stop 2024-05-27T09:40:47.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.426+0000 7f1165600700 1 -- 172.21.15.119:0/137579392 shutdown_connections 2024-05-27T09:40:47.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.426+0000 7f1165600700 1 --2- 172.21.15.119:0/137579392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1160100d50 0x7f1160104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:47.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.426+0000 7f1165600700 1 --2- 172.21.15.119:0/137579392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11601003b0 0x7f1160100780 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:47.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.426+0000 7f1165600700 1 -- 172.21.15.119:0/137579392 >> 172.21.15.119:0/137579392 conn(0x7f11600fbc30 msgr2=0x7f11600fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:47.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.426+0000 7f1165600700 1 -- 172.21.15.119:0/137579392 shutdown_connections 2024-05-27T09:40:47.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.426+0000 7f1165600700 1 -- 172.21.15.119:0/137579392 wait complete. 2024-05-27T09:40:47.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.427+0000 7f1165600700 1 Processor -- start 2024-05-27T09:40:47.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.427+0000 7f1165600700 1 -- start start 2024-05-27T09:40:47.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.427+0000 7f1165600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11601003b0 0x7f1160195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:47.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.427+0000 7f1165600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1160100d50 0x7f1160196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:47.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.427+0000 7f1165600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1160196a40 con 0x7f1160100d50 2024-05-27T09:40:47.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.427+0000 7f1165600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116019a7d0 con 0x7f11601003b0 2024-05-27T09:40:47.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.428+0000 7f115e000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1160100d50 0x7f1160196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:47.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.428+0000 7f115e000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1160100d50 0x7f1160196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60204/0 (socket says 172.21.15.119:60204) 2024-05-27T09:40:47.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.428+0000 7f115e000700 1 -- 172.21.15.119:0/2236669006 learned_addr learned my addr 172.21.15.119:0/2236669006 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:40:47.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.428+0000 7f115ea00700 1 --2- 172.21.15.119:0/2236669006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11601003b0 0x7f1160195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:47.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.428+0000 7f115e000700 1 -- 172.21.15.119:0/2236669006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11601003b0 msgr2=0x7f1160195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:47.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.428+0000 7f115e000700 1 --2- 172.21.15.119:0/2236669006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11601003b0 0x7f1160195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:47.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.428+0000 7f115e000700 1 -- 172.21.15.119:0/2236669006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11500095f0 con 0x7f1160100d50 2024-05-27T09:40:47.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.429+0000 7f115e000700 1 --2- 172.21.15.119:0/2236669006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1160100d50 0x7f1160196360 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f114c00b480 tx=0x7f114c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:47.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.429+0000 7f1157400700 1 -- 172.21.15.119:0/2236669006 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f114c0118c0 con 0x7f1160100d50 2024-05-27T09:40:47.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.429+0000 7f1157400700 1 -- 172.21.15.119:0/2236669006 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f114c010380 con 0x7f1160100d50 2024-05-27T09:40:47.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.429+0000 7f1157400700 1 -- 172.21.15.119:0/2236669006 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f114c00f4f0 con 0x7f1160100d50 2024-05-27T09:40:47.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.429+0000 7f1165600700 1 -- 172.21.15.119:0/2236669006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f116019aab0 con 0x7f1160100d50 2024-05-27T09:40:47.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.429+0000 7f1165600700 1 -- 172.21.15.119:0/2236669006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116019b000 con 0x7f1160100d50 2024-05-27T09:40:47.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.430+0000 7f1165600700 1 -- 172.21.15.119:0/2236669006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11601086c0 con 0x7f1160100d50 2024-05-27T09:40:47.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.432+0000 7f1157400700 1 -- 172.21.15.119:0/2236669006 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f114c0104f0 con 0x7f1160100d50 2024-05-27T09:40:47.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.433+0000 7f1157400700 1 --2- 172.21.15.119:0/2236669006 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1144078a40 0x7f114407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:47.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.433+0000 7f1157400700 1 -- 172.21.15.119:0/2236669006 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f114c09a990 con 0x7f1160100d50 2024-05-27T09:40:47.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.433+0000 7f115ea00700 1 --2- 172.21.15.119:0/2236669006 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1144078a40 0x7f114407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:47.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.434+0000 7f115ea00700 1 --2- 172.21.15.119:0/2236669006 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1144078a40 0x7f114407aef0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1150006010 tx=0x7f115001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:47.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.439+0000 7f1157400700 1 -- 172.21.15.119:0/2236669006 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f114c062540 con 0x7f1160100d50 2024-05-27T09:40:47.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.708+0000 7f1165600700 1 -- 172.21.15.119:0/2236669006 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f116019b410 con 0x7f1144078a40 2024-05-27T09:40:47.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.711+0000 7f1157400700 1 -- 172.21.15.119:0/2236669006 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f116019b410 con 0x7f1144078a40 2024-05-27T09:40:47.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.714+0000 7f1165600700 1 -- 172.21.15.119:0/2236669006 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1144078a40 msgr2=0x7f114407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:47.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.714+0000 7f1165600700 1 --2- 172.21.15.119:0/2236669006 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1144078a40 0x7f114407aef0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1150006010 tx=0x7f115001a040 comp rx=0 tx=0).stop 2024-05-27T09:40:47.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.714+0000 7f1165600700 1 -- 172.21.15.119:0/2236669006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1160100d50 msgr2=0x7f1160196360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:47.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.714+0000 7f1165600700 1 --2- 172.21.15.119:0/2236669006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1160100d50 0x7f1160196360 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f114c00b480 tx=0x7f114c00b840 comp rx=0 tx=0).stop 2024-05-27T09:40:47.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.714+0000 7f1165600700 1 -- 172.21.15.119:0/2236669006 shutdown_connections 2024-05-27T09:40:47.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.714+0000 7f1165600700 1 --2- 172.21.15.119:0/2236669006 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1144078a40 0x7f114407aef0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:47.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.714+0000 7f1165600700 1 --2- 172.21.15.119:0/2236669006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11601003b0 0x7f1160195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:47.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.714+0000 7f1165600700 1 --2- 172.21.15.119:0/2236669006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1160100d50 0x7f1160196360 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:47.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.714+0000 7f1165600700 1 -- 172.21.15.119:0/2236669006 >> 172.21.15.119:0/2236669006 conn(0x7f11600fbc30 msgr2=0x7f11600fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:47.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.714+0000 7f1165600700 1 -- 172.21.15.119:0/2236669006 shutdown_connections 2024-05-27T09:40:47.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.715+0000 7f1165600700 1 -- 172.21.15.119:0/2236669006 wait complete. 2024-05-27T09:40:47.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.876+0000 7efda1e00700 1 -- 172.21.15.119:0/1682426302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd9c101190 msgr2=0x7efd9c101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:47.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.876+0000 7efda1e00700 1 --2- 172.21.15.119:0/1682426302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd9c101190 0x7efd9c101560 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efd90009910 tx=0x7efd90009c20 comp rx=0 tx=0).stop 2024-05-27T09:40:47.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.876+0000 7efda1e00700 1 -- 172.21.15.119:0/1682426302 shutdown_connections 2024-05-27T09:40:47.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.876+0000 7efda1e00700 1 --2- 172.21.15.119:0/1682426302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd9c101aa0 0x7efd9c109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:47.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.876+0000 7efda1e00700 1 --2- 172.21.15.119:0/1682426302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd9c101190 0x7efd9c101560 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:47.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.876+0000 7efda1e00700 1 -- 172.21.15.119:0/1682426302 >> 172.21.15.119:0/1682426302 conn(0x7efd9c0fa9d0 msgr2=0x7efd9c0fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:47.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.877+0000 7efda1e00700 1 -- 172.21.15.119:0/1682426302 shutdown_connections 2024-05-27T09:40:47.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.877+0000 7efda1e00700 1 -- 172.21.15.119:0/1682426302 wait complete. 2024-05-27T09:40:47.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.878+0000 7efda1e00700 1 Processor -- start 2024-05-27T09:40:47.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.878+0000 7efda1e00700 1 -- start start 2024-05-27T09:40:47.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.878+0000 7efda1e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd9c101190 0x7efd9c197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:47.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.878+0000 7efda1e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd9c101aa0 0x7efd9c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:47.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.878+0000 7efda1e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c198990 con 0x7efd9c101190 2024-05-27T09:40:47.879 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.878+0000 7efda1e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c19c720 con 0x7efd9c101aa0 2024-05-27T09:40:47.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.879+0000 7efda0a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd9c101190 0x7efd9c197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:47.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.879+0000 7efda0a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd9c101190 0x7efd9c197d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60208/0 (socket says 172.21.15.119:60208) 2024-05-27T09:40:47.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.879+0000 7efda0a00700 1 -- 172.21.15.119:0/1413105329 learned_addr learned my addr 172.21.15.119:0/1413105329 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:40:47.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.879+0000 7efd9be00700 1 --2- 172.21.15.119:0/1413105329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd9c101aa0 0x7efd9c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:47.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.879+0000 7efda0a00700 1 -- 172.21.15.119:0/1413105329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd9c101aa0 msgr2=0x7efd9c1982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:47.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.879+0000 7efda0a00700 1 --2- 172.21.15.119:0/1413105329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd9c101aa0 0x7efd9c1982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:47.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.879+0000 7efda0a00700 1 -- 172.21.15.119:0/1413105329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd900095f0 con 0x7efd9c101190 2024-05-27T09:40:47.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.879+0000 7efda0a00700 1 --2- 172.21.15.119:0/1413105329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd9c101190 0x7efd9c197d70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efd9000b5c0 tx=0x7efd9000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:47.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.880+0000 7efd99600700 1 -- 172.21.15.119:0/1413105329 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd9001d070 con 0x7efd9c101190 2024-05-27T09:40:47.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.880+0000 7efda1e00700 1 -- 172.21.15.119:0/1413105329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9c19c9a0 con 0x7efd9c101190 2024-05-27T09:40:47.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.880+0000 7efd99600700 1 -- 172.21.15.119:0/1413105329 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd90005e00 con 0x7efd9c101190 2024-05-27T09:40:47.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.880+0000 7efd99600700 1 -- 172.21.15.119:0/1413105329 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd90017400 con 0x7efd9c101190 2024-05-27T09:40:47.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.880+0000 7efda1e00700 1 -- 172.21.15.119:0/1413105329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9c19ce90 con 0x7efd9c101190 2024-05-27T09:40:47.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.881+0000 7efda1e00700 1 -- 172.21.15.119:0/1413105329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd9c109a70 con 0x7efd9c101190 2024-05-27T09:40:47.882 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.882+0000 7efd99600700 1 -- 172.21.15.119:0/1413105329 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7efd9000f460 con 0x7efd9c101190 2024-05-27T09:40:47.883 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.883+0000 7efd99600700 1 --2- 172.21.15.119:0/1413105329 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efd84078a40 0x7efd8407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:47.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.883+0000 7efd99600700 1 -- 172.21.15.119:0/1413105329 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efd9009c260 con 0x7efd9c101190 2024-05-27T09:40:47.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.883+0000 7efd9be00700 1 --2- 172.21.15.119:0/1413105329 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efd84078a40 0x7efd8407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:47.884 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.884+0000 7efd9be00700 1 --2- 172.21.15.119:0/1413105329 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efd84078a40 0x7efd8407aef0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efd88005950 tx=0x7efd880058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:47.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:47.888+0000 7efd99600700 1 -- 172.21.15.119:0/1413105329 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7efd90063e10 con 0x7efd9c101190 2024-05-27T09:40:48.142 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.141+0000 7efda1e00700 1 -- 172.21.15.119:0/1413105329 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efd9c19d230 con 0x7efd84078a40 2024-05-27T09:40:48.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.155+0000 7efd99600700 1 -- 172.21.15.119:0/1413105329 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7efd9c19d230 con 0x7efd84078a40 2024-05-27T09:40:48.156 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:40:48.156 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (8m) 8s ago 21m 15.2M - 0.25.0 c8568f914cd2 6b90564dcef7 2024-05-27T09:40:48.156 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (22m) 8s ago 22m 9298k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:40:48.156 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (20m) 59s ago 20m 8954k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:40:48.156 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (4m) 8s ago 22m 6669k - 19.0.0-3868-ga6507388 850a06a84d28 011d811ce7fc 2024-05-27T09:40:48.156 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (4m) 59s ago 20m 6665k - 19.0.0-3868-ga6507388 850a06a84d28 b546b3ed1afc 2024-05-27T09:40:48.156 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (7m) 8s ago 21m 74.2M - 9.4.12 2bacad6d85d8 b3abdab08db4 2024-05-27T09:40:48.156 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (40s) 8s ago 14m 79.5M - 19.0.0-3868-ga6507388 850a06a84d28 5a02732ff870 2024-05-27T09:40:48.156 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (16s) 8s ago 14m 78.4M - 19.0.0-3868-ga6507388 850a06a84d28 3faccd4c18e8 2024-05-27T09:40:48.156 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (14m) 59s ago 14m 18.5M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:40:48.156 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (14m) 59s ago 14m 128M - 18.2.0 dc2bc1663786 e3d8daf83b9c 2024-05-27T09:40:48.156 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (10m) 8s ago 23m 593M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:40:48.156 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (9m) 59s ago 20m 534M - 19.0.0-3868-ga6507388 850a06a84d28 2fc3a80696fa 2024-05-27T09:40:48.157 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (6m) 8s ago 23m 123M 2048M 19.0.0-3868-ga6507388 850a06a84d28 3e5c7a9ad372 2024-05-27T09:40:48.157 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (5m) 59s ago 20m 108M 2048M 19.0.0-3868-ga6507388 850a06a84d28 e4d471274cf1 2024-05-27T09:40:48.157 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (9m) 8s ago 22m 7814k - 1.5.0 0da6a335fe13 709c37815cff 2024-05-27T09:40:48.157 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (9m) 59s ago 20m 8292k - 1.5.0 0da6a335fe13 999047a1a5bf 2024-05-27T09:40:48.157 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (4m) 8s ago 19m 157M 1216M 19.0.0-3868-ga6507388 850a06a84d28 5a0e162d870e 2024-05-27T09:40:48.157 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (3m) 8s ago 18m 171M 1216M 19.0.0-3868-ga6507388 850a06a84d28 effcc7c53dd3 2024-05-27T09:40:48.157 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (3m) 8s ago 18m 149M 1216M 19.0.0-3868-ga6507388 850a06a84d28 f0bd34ca416f 2024-05-27T09:40:48.157 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (2m) 59s ago 17m 174M 2240M 19.0.0-3868-ga6507388 850a06a84d28 5e8d3e398c21 2024-05-27T09:40:48.157 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (110s) 59s ago 17m 162M 2240M 19.0.0-3868-ga6507388 850a06a84d28 250e443364de 2024-05-27T09:40:48.157 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (65s) 59s ago 16m 78.5M 2240M 19.0.0-3868-ga6507388 850a06a84d28 90d98c5b2f16 2024-05-27T09:40:48.157 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (8m) 8s ago 21m 72.5M - 2.43.0 a07b618ecd1d cba50454ed21 2024-05-27T09:40:48.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.158+0000 7efda1e00700 1 -- 172.21.15.119:0/1413105329 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efd84078a40 msgr2=0x7efd8407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:48.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.158+0000 7efda1e00700 1 --2- 172.21.15.119:0/1413105329 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efd84078a40 0x7efd8407aef0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efd88005950 tx=0x7efd880058e0 comp rx=0 tx=0).stop 2024-05-27T09:40:48.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.158+0000 7efda1e00700 1 -- 172.21.15.119:0/1413105329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd9c101190 msgr2=0x7efd9c197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:48.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.158+0000 7efda1e00700 1 --2- 172.21.15.119:0/1413105329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd9c101190 0x7efd9c197d70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efd9000b5c0 tx=0x7efd9000be20 comp rx=0 tx=0).stop 2024-05-27T09:40:48.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.158+0000 7efda1e00700 1 -- 172.21.15.119:0/1413105329 shutdown_connections 2024-05-27T09:40:48.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.158+0000 7efda1e00700 1 --2- 172.21.15.119:0/1413105329 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efd84078a40 0x7efd8407aef0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:48.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.158+0000 7efda1e00700 1 --2- 172.21.15.119:0/1413105329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd9c101190 0x7efd9c197d70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:48.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.158+0000 7efda1e00700 1 --2- 172.21.15.119:0/1413105329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd9c101aa0 0x7efd9c1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:48.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.158+0000 7efda1e00700 1 -- 172.21.15.119:0/1413105329 >> 172.21.15.119:0/1413105329 conn(0x7efd9c0fa9d0 msgr2=0x7efd9c106bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:48.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.158+0000 7efda1e00700 1 -- 172.21.15.119:0/1413105329 shutdown_connections 2024-05-27T09:40:48.159 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.158+0000 7efda1e00700 1 -- 172.21.15.119:0/1413105329 wait complete. 2024-05-27T09:40:48.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.305+0000 7f9e77400700 1 -- 172.21.15.119:0/4038592473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e7010c010 msgr2=0x7f9e70106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:48.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.305+0000 7f9e77400700 1 --2- 172.21.15.119:0/4038592473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e7010c010 0x7f9e70106b80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c009910 tx=0x7f9e5c009c20 comp rx=0 tx=0).stop 2024-05-27T09:40:48.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.305+0000 7f9e77400700 1 -- 172.21.15.119:0/4038592473 shutdown_connections 2024-05-27T09:40:48.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.305+0000 7f9e77400700 1 --2- 172.21.15.119:0/4038592473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e7010c010 0x7f9e70106b80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:48.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.305+0000 7f9e77400700 1 --2- 172.21.15.119:0/4038592473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e7010b700 0x7f9e7010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:48.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.305+0000 7f9e77400700 1 -- 172.21.15.119:0/4038592473 >> 172.21.15.119:0/4038592473 conn(0x7f9e70076000 msgr2=0x7f9e70078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:48.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.305+0000 7f9e77400700 1 -- 172.21.15.119:0/4038592473 shutdown_connections 2024-05-27T09:40:48.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.306+0000 7f9e77400700 1 -- 172.21.15.119:0/4038592473 wait complete. 2024-05-27T09:40:48.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.306+0000 7f9e77400700 1 Processor -- start 2024-05-27T09:40:48.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.307+0000 7f9e77400700 1 -- start start 2024-05-27T09:40:48.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.307+0000 7f9e77400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e7010b700 0x7f9e7019c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:48.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.307+0000 7f9e77400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e7010c010 0x7f9e7019c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:48.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.307+0000 7f9e77400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7019cfd0 con 0x7f9e7010c010 2024-05-27T09:40:48.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.307+0000 7f9e77400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e701a0d60 con 0x7f9e7010b700 2024-05-27T09:40:48.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.307+0000 7f9e6fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e7010c010 0x7f9e7019c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:48.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.307+0000 7f9e6fe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e7010c010 0x7f9e7019c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60230/0 (socket says 172.21.15.119:60230) 2024-05-27T09:40:48.308 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.307+0000 7f9e6fe00700 1 -- 172.21.15.119:0/1044458260 learned_addr learned my addr 172.21.15.119:0/1044458260 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:40:48.309 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.308+0000 7f9e74a00700 1 --2- 172.21.15.119:0/1044458260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e7010b700 0x7f9e7019c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:48.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.308+0000 7f9e6fe00700 1 -- 172.21.15.119:0/1044458260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e7010b700 msgr2=0x7f9e7019c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:48.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.308+0000 7f9e6fe00700 1 --2- 172.21.15.119:0/1044458260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e7010b700 0x7f9e7019c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:48.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.308+0000 7f9e6fe00700 1 -- 172.21.15.119:0/1044458260 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5c0095f0 con 0x7f9e7010c010 2024-05-27T09:40:48.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.308+0000 7f9e6fe00700 1 --2- 172.21.15.119:0/1044458260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e7010c010 0x7f9e7019c8f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00bda0 tx=0x7f9e5c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:48.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.308+0000 7f9e6d600700 1 -- 172.21.15.119:0/1044458260 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e5c01d070 con 0x7f9e7010c010 2024-05-27T09:40:48.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.308+0000 7f9e6d600700 1 -- 172.21.15.119:0/1044458260 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e5c004fb0 con 0x7f9e7010c010 2024-05-27T09:40:48.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.309+0000 7f9e77400700 1 -- 172.21.15.119:0/1044458260 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e701a0fe0 con 0x7f9e7010c010 2024-05-27T09:40:48.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.309+0000 7f9e6d600700 1 -- 172.21.15.119:0/1044458260 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e5c00f460 con 0x7f9e7010c010 2024-05-27T09:40:48.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.309+0000 7f9e77400700 1 -- 172.21.15.119:0/1044458260 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e701a1550 con 0x7f9e7010c010 2024-05-27T09:40:48.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.310+0000 7f9e77400700 1 -- 172.21.15.119:0/1044458260 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e7010ecf0 con 0x7f9e7010c010 2024-05-27T09:40:48.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.311+0000 7f9e6d600700 1 -- 172.21.15.119:0/1044458260 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f9e5c022800 con 0x7f9e7010c010 2024-05-27T09:40:48.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.312+0000 7f9e6d600700 1 --2- 172.21.15.119:0/1044458260 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9e58078a40 0x7f9e5807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:48.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.312+0000 7f9e6d600700 1 -- 172.21.15.119:0/1044458260 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9e5c09c2f0 con 0x7f9e7010c010 2024-05-27T09:40:48.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.312+0000 7f9e74a00700 1 --2- 172.21.15.119:0/1044458260 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9e58078a40 0x7f9e5807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:48.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.313+0000 7f9e74a00700 1 --2- 172.21.15.119:0/1044458260 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9e58078a40 0x7f9e5807aef0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9e64006fd0 tx=0x7f9e64008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:48.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.317+0000 7f9e6d600700 1 -- 172.21.15.119:0/1044458260 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f9e5c063f50 con 0x7f9e7010c010 2024-05-27T09:40:48.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:48 smithi119 ceph-mon[88269]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:40:48.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:48 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:48.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:48 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:48.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:48 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:40:48.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:48 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:40:48.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:48 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:48.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:48 smithi119 ceph-mon[88269]: pgmap v227: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:48.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:48 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:40:48.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:48 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:40:48.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:48 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:40:48.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:48 smithi119 ceph-mon[88269]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:40:48.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:48 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:40:48.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:48 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:40:48.622 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:48 smithi138 ceph-mon[70004]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:40:48.622 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:48 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:48.622 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:48 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:48.622 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:48 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:40:48.622 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:48 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:40:48.622 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:48 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:48.622 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:48 smithi138 ceph-mon[70004]: pgmap v227: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:48.622 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:48 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:40:48.623 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:48 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:40:48.623 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:48 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:40:48.623 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:48 smithi138 ceph-mon[70004]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:40:48.623 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:48 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:40:48.623 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:48 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:40:48.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.632+0000 7f9e77400700 1 -- 172.21.15.119:0/1044458260 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9e7004ea50 con 0x7f9e7010c010 2024-05-27T09:40:48.634 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.633+0000 7f9e6d600700 1 -- 172.21.15.119:0/1044458260 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f9e5c0638f0 con 0x7f9e7010c010 2024-05-27T09:40:48.634 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:40:48.634 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:40:48.634 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:40:48.634 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:40:48.634 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:40:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:40:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:40:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:40:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 6 2024-05-27T09:40:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:40:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:40:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2, 2024-05-27T09:40:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:40:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:40:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:40:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2, 2024-05-27T09:40:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 12 2024-05-27T09:40:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:40:48.635 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:40:48.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.636+0000 7f9e77400700 1 -- 172.21.15.119:0/1044458260 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9e58078a40 msgr2=0x7f9e5807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:48.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.636+0000 7f9e77400700 1 --2- 172.21.15.119:0/1044458260 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9e58078a40 0x7f9e5807aef0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9e64006fd0 tx=0x7f9e64008040 comp rx=0 tx=0).stop 2024-05-27T09:40:48.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.636+0000 7f9e77400700 1 -- 172.21.15.119:0/1044458260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e7010c010 msgr2=0x7f9e7019c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:48.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.636+0000 7f9e77400700 1 --2- 172.21.15.119:0/1044458260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e7010c010 0x7f9e7019c8f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00bda0 tx=0x7f9e5c00be80 comp rx=0 tx=0).stop 2024-05-27T09:40:48.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.636+0000 7f9e77400700 1 -- 172.21.15.119:0/1044458260 shutdown_connections 2024-05-27T09:40:48.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.636+0000 7f9e77400700 1 --2- 172.21.15.119:0/1044458260 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f9e58078a40 0x7f9e5807aef0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:48.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.636+0000 7f9e77400700 1 --2- 172.21.15.119:0/1044458260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e7010b700 0x7f9e7019c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:48.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.636+0000 7f9e77400700 1 --2- 172.21.15.119:0/1044458260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e7010c010 0x7f9e7019c8f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:48.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.636+0000 7f9e77400700 1 -- 172.21.15.119:0/1044458260 >> 172.21.15.119:0/1044458260 conn(0x7f9e70076000 msgr2=0x7f9e70076a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:48.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.637+0000 7f9e77400700 1 -- 172.21.15.119:0/1044458260 shutdown_connections 2024-05-27T09:40:48.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.637+0000 7f9e77400700 1 -- 172.21.15.119:0/1044458260 wait complete. 2024-05-27T09:40:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.792+0000 7fcb20a00700 1 -- 172.21.15.119:0/784515318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb1c103040 msgr2=0x7fcb1c107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.792+0000 7fcb20a00700 1 --2- 172.21.15.119:0/784515318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb1c103040 0x7fcb1c107010 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009910 tx=0x7fcb08009c20 comp rx=0 tx=0).stop 2024-05-27T09:40:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.792+0000 7fcb20a00700 1 -- 172.21.15.119:0/784515318 shutdown_connections 2024-05-27T09:40:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.792+0000 7fcb20a00700 1 --2- 172.21.15.119:0/784515318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb1c103040 0x7fcb1c107010 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.792+0000 7fcb20a00700 1 --2- 172.21.15.119:0/784515318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb1c1026a0 0x7fcb1c102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.792+0000 7fcb20a00700 1 -- 172.21.15.119:0/784515318 >> 172.21.15.119:0/784515318 conn(0x7fcb1c0fdf00 msgr2=0x7fcb1c100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.792+0000 7fcb20a00700 1 -- 172.21.15.119:0/784515318 shutdown_connections 2024-05-27T09:40:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.793+0000 7fcb20a00700 1 -- 172.21.15.119:0/784515318 wait complete. 2024-05-27T09:40:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.794+0000 7fcb20a00700 1 Processor -- start 2024-05-27T09:40:48.795 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.794+0000 7fcb20a00700 1 -- start start 2024-05-27T09:40:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.794+0000 7fcb20a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb1c1026a0 0x7fcb1c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.794+0000 7fcb20a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb1c103040 0x7fcb1c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.794+0000 7fcb20a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c198b90 con 0x7fcb1c103040 2024-05-27T09:40:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.794+0000 7fcb20a00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c198cd0 con 0x7fcb1c1026a0 2024-05-27T09:40:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.795+0000 7fcb19400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb1c103040 0x7fcb1c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.795+0000 7fcb19400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb1c103040 0x7fcb1c198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54646/0 (socket says 172.21.15.119:54646) 2024-05-27T09:40:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.795+0000 7fcb19400700 1 -- 172.21.15.119:0/3463579651 learned_addr learned my addr 172.21.15.119:0/3463579651 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:40:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.795+0000 7fcb19e00700 1 --2- 172.21.15.119:0/3463579651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb1c1026a0 0x7fcb1c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.795+0000 7fcb19400700 1 -- 172.21.15.119:0/3463579651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb1c1026a0 msgr2=0x7fcb1c198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.795+0000 7fcb19400700 1 --2- 172.21.15.119:0/3463579651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb1c1026a0 0x7fcb1c198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.795+0000 7fcb19400700 1 -- 172.21.15.119:0/3463579651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb080095f0 con 0x7fcb1c103040 2024-05-27T09:40:48.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.796+0000 7fcb19400700 1 --2- 172.21.15.119:0/3463579651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb1c103040 0x7fcb1c198540 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800bda0 tx=0x7fcb0800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:48.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.796+0000 7fcb12a00700 1 -- 172.21.15.119:0/3463579651 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb0801d070 con 0x7fcb1c103040 2024-05-27T09:40:48.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.796+0000 7fcb12a00700 1 -- 172.21.15.119:0/3463579651 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb0800f460 con 0x7fcb1c103040 2024-05-27T09:40:48.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.796+0000 7fcb20a00700 1 -- 172.21.15.119:0/3463579651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb1c19cac0 con 0x7fcb1c103040 2024-05-27T09:40:48.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.796+0000 7fcb12a00700 1 -- 172.21.15.119:0/3463579651 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb08017630 con 0x7fcb1c103040 2024-05-27T09:40:48.797 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.796+0000 7fcb20a00700 1 -- 172.21.15.119:0/3463579651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb1c19cfb0 con 0x7fcb1c103040 2024-05-27T09:40:48.798 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.797+0000 7fcb20a00700 1 -- 172.21.15.119:0/3463579651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb1c04ea50 con 0x7fcb1c103040 2024-05-27T09:40:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.798+0000 7fcb12a00700 1 -- 172.21.15.119:0/3463579651 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fcb0800fab0 con 0x7fcb1c103040 2024-05-27T09:40:48.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.799+0000 7fcb12a00700 1 --2- 172.21.15.119:0/3463579651 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fcb00078a40 0x7fcb0007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:48.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.799+0000 7fcb12a00700 1 -- 172.21.15.119:0/3463579651 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcb0809c230 con 0x7fcb1c103040 2024-05-27T09:40:48.800 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.800+0000 7fcb19e00700 1 --2- 172.21.15.119:0/3463579651 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fcb00078a40 0x7fcb0007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:48.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.800+0000 7fcb19e00700 1 --2- 172.21.15.119:0/3463579651 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fcb00078a40 0x7fcb0007aef0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00ac50 tx=0x7fcb0c00a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:48.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:48.805+0000 7fcb12a00700 1 -- 172.21.15.119:0/3463579651 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fcb08063de0 con 0x7fcb1c103040 2024-05-27T09:40:49.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.086+0000 7fcb20a00700 1 -- 172.21.15.119:0/3463579651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcb1c199360 con 0x7fcb1c103040 2024-05-27T09:40:49.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.087+0000 7fcb12a00700 1 -- 172.21.15.119:0/3463579651 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1878 (secure 0 0 0) 0x7fcb08063780 con 0x7fcb1c103040 2024-05-27T09:40:49.088 INFO:teuthology.orchestra.run.smithi119.stdout:e28 2024-05-27T09:40:49.088 INFO:teuthology.orchestra.run.smithi119.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T09:40:49.088 INFO:teuthology.orchestra.run.smithi119.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:40:49.089 INFO:teuthology.orchestra.run.smithi119.stdout:legacy client fscid: 1 2024-05-27T09:40:49.089 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:40:49.089 INFO:teuthology.orchestra.run.smithi119.stdout:Filesystem 'cephfs' (1) 2024-05-27T09:40:49.089 INFO:teuthology.orchestra.run.smithi119.stdout:fs_name cephfs 2024-05-27T09:40:49.089 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 23 2024-05-27T09:40:49.089 INFO:teuthology.orchestra.run.smithi119.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-27T09:40:49.089 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-05-27T09:25:53.387388+0000 2024-05-27T09:40:49.089 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2024-05-27T09:39:57.913154+0000 2024-05-27T09:40:49.089 INFO:teuthology.orchestra.run.smithi119.stdout:tableserver 0 2024-05-27T09:40:49.089 INFO:teuthology.orchestra.run.smithi119.stdout:root 0 2024-05-27T09:40:49.089 INFO:teuthology.orchestra.run.smithi119.stdout:session_timeout 60 2024-05-27T09:40:49.089 INFO:teuthology.orchestra.run.smithi119.stdout:session_autoclose 300 2024-05-27T09:40:49.089 INFO:teuthology.orchestra.run.smithi119.stdout:max_file_size 1099511627776 2024-05-27T09:40:49.089 INFO:teuthology.orchestra.run.smithi119.stdout:max_xattr_size 65536 2024-05-27T09:40:49.090 INFO:teuthology.orchestra.run.smithi119.stdout:required_client_features {} 2024-05-27T09:40:49.090 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure 0 2024-05-27T09:40:49.090 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure_osd_epoch 81 2024-05-27T09:40:49.090 INFO:teuthology.orchestra.run.smithi119.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:40:49.090 INFO:teuthology.orchestra.run.smithi119.stdout:max_mds 2 2024-05-27T09:40:49.090 INFO:teuthology.orchestra.run.smithi119.stdout:in 0,1 2024-05-27T09:40:49.090 INFO:teuthology.orchestra.run.smithi119.stdout:up {} 2024-05-27T09:40:49.090 INFO:teuthology.orchestra.run.smithi119.stdout:failed 0,1 2024-05-27T09:40:49.090 INFO:teuthology.orchestra.run.smithi119.stdout:damaged 2024-05-27T09:40:49.090 INFO:teuthology.orchestra.run.smithi119.stdout:stopped 2024-05-27T09:40:49.090 INFO:teuthology.orchestra.run.smithi119.stdout:data_pools [3] 2024-05-27T09:40:49.090 INFO:teuthology.orchestra.run.smithi119.stdout:metadata_pool 2 2024-05-27T09:40:49.091 INFO:teuthology.orchestra.run.smithi119.stdout:inline_data enabled 2024-05-27T09:40:49.091 INFO:teuthology.orchestra.run.smithi119.stdout:balancer 2024-05-27T09:40:49.091 INFO:teuthology.orchestra.run.smithi119.stdout:bal_rank_mask -1 2024-05-27T09:40:49.091 INFO:teuthology.orchestra.run.smithi119.stdout:standby_count_wanted 1 2024-05-27T09:40:49.091 INFO:teuthology.orchestra.run.smithi119.stdout:qdb_cluster leader: 0 members: 2024-05-27T09:40:49.091 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:40:49.091 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:40:49.091 INFO:teuthology.orchestra.run.smithi119.stdout:Standby daemons: 2024-05-27T09:40:49.091 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:40:49.091 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.alszpk{-1:24253} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/2345607053,v1:172.21.15.138:6827/2345607053] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:40:49.091 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.nghcey{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.119:6826/3357235708,v1:172.21.15.119:6827/3357235708] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T09:40:49.091 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.ppmzzk{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.119:6828/2430623742,v1:172.21.15.119:6829/2430623742] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T09:40:49.091 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.ndfkhd{-1:44261} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/2055700210,v1:172.21.15.138:6825/2055700210] compat {c=[1],r=[1],i=[7ff]}] 2024-05-27T09:40:49.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.090+0000 7fcb20a00700 1 -- 172.21.15.119:0/3463579651 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fcb00078a40 msgr2=0x7fcb0007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:49.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.090+0000 7fcb20a00700 1 --2- 172.21.15.119:0/3463579651 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fcb00078a40 0x7fcb0007aef0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00ac50 tx=0x7fcb0c00a380 comp rx=0 tx=0).stop 2024-05-27T09:40:49.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.090+0000 7fcb20a00700 1 -- 172.21.15.119:0/3463579651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb1c103040 msgr2=0x7fcb1c198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:49.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.090+0000 7fcb20a00700 1 --2- 172.21.15.119:0/3463579651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb1c103040 0x7fcb1c198540 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800bda0 tx=0x7fcb0800bdd0 comp rx=0 tx=0).stop 2024-05-27T09:40:49.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.090+0000 7fcb20a00700 1 -- 172.21.15.119:0/3463579651 shutdown_connections 2024-05-27T09:40:49.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.090+0000 7fcb20a00700 1 --2- 172.21.15.119:0/3463579651 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fcb00078a40 0x7fcb0007aef0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:49.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.090+0000 7fcb20a00700 1 --2- 172.21.15.119:0/3463579651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb1c1026a0 0x7fcb1c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:49.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.090+0000 7fcb20a00700 1 --2- 172.21.15.119:0/3463579651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb1c103040 0x7fcb1c198540 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:49.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.090+0000 7fcb20a00700 1 -- 172.21.15.119:0/3463579651 >> 172.21.15.119:0/3463579651 conn(0x7fcb1c0fdf00 msgr2=0x7fcb1c1002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:49.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.091+0000 7fcb20a00700 1 -- 172.21.15.119:0/3463579651 shutdown_connections 2024-05-27T09:40:49.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.091+0000 7fcb20a00700 1 -- 172.21.15.119:0/3463579651 wait complete. 2024-05-27T09:40:49.093 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 28 2024-05-27T09:40:49.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.247+0000 7f060e400700 1 -- 172.21.15.119:0/2162885501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0608100d70 msgr2=0x7f0608104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:49.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.247+0000 7f060e400700 1 --2- 172.21.15.119:0/2162885501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0608100d70 0x7f0608104d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009910 tx=0x7f05f4009c20 comp rx=0 tx=0).stop 2024-05-27T09:40:49.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.248+0000 7f060e400700 1 -- 172.21.15.119:0/2162885501 shutdown_connections 2024-05-27T09:40:49.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.248+0000 7f060e400700 1 --2- 172.21.15.119:0/2162885501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0608100d70 0x7f0608104d40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:49.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.248+0000 7f060e400700 1 --2- 172.21.15.119:0/2162885501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06081003d0 0x7f06081007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:49.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.248+0000 7f060e400700 1 -- 172.21.15.119:0/2162885501 >> 172.21.15.119:0/2162885501 conn(0x7f06080fbc90 msgr2=0x7f06080fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:49.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.248+0000 7f060e400700 1 -- 172.21.15.119:0/2162885501 shutdown_connections 2024-05-27T09:40:49.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.248+0000 7f060e400700 1 -- 172.21.15.119:0/2162885501 wait complete. 2024-05-27T09:40:49.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.249+0000 7f060e400700 1 Processor -- start 2024-05-27T09:40:49.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.249+0000 7f060e400700 1 -- start start 2024-05-27T09:40:49.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.249+0000 7f060e400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06081003d0 0x7f0608198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:49.250 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.249+0000 7f060e400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0608100d70 0x7f06081985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:49.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.249+0000 7f060e400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0608198ca0 con 0x7f0608100d70 2024-05-27T09:40:49.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.249+0000 7f060e400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060819ca30 con 0x7f06081003d0 2024-05-27T09:40:49.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.249+0000 7f0606a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0608100d70 0x7f06081985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:49.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.249+0000 7f0606a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0608100d70 0x7f06081985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54662/0 (socket says 172.21.15.119:54662) 2024-05-27T09:40:49.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.249+0000 7f0606a00700 1 -- 172.21.15.119:0/3408603938 learned_addr learned my addr 172.21.15.119:0/3408603938 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:40:49.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.250+0000 7f0607400700 1 --2- 172.21.15.119:0/3408603938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06081003d0 0x7f0608198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:49.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.250+0000 7f0606a00700 1 -- 172.21.15.119:0/3408603938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06081003d0 msgr2=0x7f0608198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:49.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.250+0000 7f0606a00700 1 --2- 172.21.15.119:0/3408603938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06081003d0 0x7f0608198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:49.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.250+0000 7f0606a00700 1 -- 172.21.15.119:0/3408603938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f40095f0 con 0x7f0608100d70 2024-05-27T09:40:49.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.250+0000 7f0606a00700 1 --2- 172.21.15.119:0/3408603938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0608100d70 0x7f06081985c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f05f400bda0 tx=0x7f05f400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:49.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.250+0000 7f05ffe00700 1 -- 172.21.15.119:0/3408603938 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05f401d070 con 0x7f0608100d70 2024-05-27T09:40:49.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.250+0000 7f05ffe00700 1 -- 172.21.15.119:0/3408603938 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05f4004fb0 con 0x7f0608100d70 2024-05-27T09:40:49.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.250+0000 7f05ffe00700 1 -- 172.21.15.119:0/3408603938 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05f400f460 con 0x7f0608100d70 2024-05-27T09:40:49.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.251+0000 7f060e400700 1 -- 172.21.15.119:0/3408603938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060819ccb0 con 0x7f0608100d70 2024-05-27T09:40:49.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.251+0000 7f060e400700 1 -- 172.21.15.119:0/3408603938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060819d1a0 con 0x7f0608100d70 2024-05-27T09:40:49.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.252+0000 7f060e400700 1 -- 172.21.15.119:0/3408603938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06081044c0 con 0x7f0608100d70 2024-05-27T09:40:49.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.253+0000 7f05ffe00700 1 -- 172.21.15.119:0/3408603938 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f05f4022800 con 0x7f0608100d70 2024-05-27T09:40:49.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.254+0000 7f05ffe00700 1 --2- 172.21.15.119:0/3408603938 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f05ec078a40 0x7f05ec07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:49.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.255+0000 7f05ffe00700 1 -- 172.21.15.119:0/3408603938 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f05f409c360 con 0x7f0608100d70 2024-05-27T09:40:49.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.255+0000 7f0607400700 1 --2- 172.21.15.119:0/3408603938 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f05ec078a40 0x7f05ec07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:49.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.255+0000 7f0607400700 1 --2- 172.21.15.119:0/3408603938 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f05ec078a40 0x7f05ec07aef0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f05f8006fd0 tx=0x7f05f8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:49.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.259+0000 7f05ffe00700 1 -- 172.21.15.119:0/3408603938 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f05f4063f40 con 0x7f0608100d70 2024-05-27T09:40:49.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.515+0000 7f060e400700 1 -- 172.21.15.119:0/3408603938 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f060802cd70 con 0x7f05ec078a40 2024-05-27T09:40:49.519 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.518+0000 7f05ffe00700 1 -- 172.21.15.119:0/3408603938 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f060802cd70 con 0x7f05ec078a40 2024-05-27T09:40:49.519 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:40:49.519 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:40:49.519 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:40:49.519 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-27T09:40:49.519 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [ 2024-05-27T09:40:49.519 INFO:teuthology.orchestra.run.smithi119.stdout: "crash", 2024-05-27T09:40:49.519 INFO:teuthology.orchestra.run.smithi119.stdout: "mon", 2024-05-27T09:40:49.519 INFO:teuthology.orchestra.run.smithi119.stdout: "osd", 2024-05-27T09:40:49.520 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr" 2024-05-27T09:40:49.520 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-05-27T09:40:49.520 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "14/23 daemons upgraded", 2024-05-27T09:40:49.520 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "Currently upgrading mds daemons", 2024-05-27T09:40:49.520 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:40:49.520 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:40:49.521 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.521+0000 7f060e400700 1 -- 172.21.15.119:0/3408603938 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f05ec078a40 msgr2=0x7f05ec07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:49.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.521+0000 7f060e400700 1 --2- 172.21.15.119:0/3408603938 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f05ec078a40 0x7f05ec07aef0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f05f8006fd0 tx=0x7f05f8008040 comp rx=0 tx=0).stop 2024-05-27T09:40:49.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.521+0000 7f060e400700 1 -- 172.21.15.119:0/3408603938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0608100d70 msgr2=0x7f06081985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:49.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.521+0000 7f060e400700 1 --2- 172.21.15.119:0/3408603938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0608100d70 0x7f06081985c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f05f400bda0 tx=0x7f05f400be80 comp rx=0 tx=0).stop 2024-05-27T09:40:49.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.521+0000 7f060e400700 1 -- 172.21.15.119:0/3408603938 shutdown_connections 2024-05-27T09:40:49.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.521+0000 7f060e400700 1 --2- 172.21.15.119:0/3408603938 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f05ec078a40 0x7f05ec07aef0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:49.522 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.521+0000 7f060e400700 1 --2- 172.21.15.119:0/3408603938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06081003d0 0x7f0608198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:49.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.521+0000 7f060e400700 1 --2- 172.21.15.119:0/3408603938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0608100d70 0x7f06081985c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:49.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.521+0000 7f060e400700 1 -- 172.21.15.119:0/3408603938 >> 172.21.15.119:0/3408603938 conn(0x7f06080fbc90 msgr2=0x7f06080fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:49.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.522+0000 7f060e400700 1 -- 172.21.15.119:0/3408603938 shutdown_connections 2024-05-27T09:40:49.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.522+0000 7f060e400700 1 -- 172.21.15.119:0/3408603938 wait complete. 2024-05-27T09:40:49.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:49 smithi138 ceph-mon[70004]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:40:49.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:49 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/1044458260' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:40:49.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:49 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/3463579651' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:40:49.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.678+0000 7f3017c00700 1 -- 172.21.15.119:0/533043693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3010102fd0 msgr2=0x7f3010106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:49.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.678+0000 7f3017c00700 1 --2- 172.21.15.119:0/533043693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3010102fd0 0x7f3010106fa0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009910 tx=0x7f2ffc009c20 comp rx=0 tx=0).stop 2024-05-27T09:40:49.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.678+0000 7f3017c00700 1 -- 172.21.15.119:0/533043693 shutdown_connections 2024-05-27T09:40:49.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.678+0000 7f3017c00700 1 --2- 172.21.15.119:0/533043693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3010102fd0 0x7f3010106fa0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:49.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.678+0000 7f3017c00700 1 --2- 172.21.15.119:0/533043693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3010102630 0x7f3010102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:49.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.678+0000 7f3017c00700 1 -- 172.21.15.119:0/533043693 >> 172.21.15.119:0/533043693 conn(0x7f30100fdef0 msgr2=0x7f3010100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:49.680 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:49 smithi119 ceph-mon[88269]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:40:49.680 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:49 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/1044458260' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:40:49.680 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:49 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/3463579651' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:40:49.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.678+0000 7f3017c00700 1 -- 172.21.15.119:0/533043693 shutdown_connections 2024-05-27T09:40:49.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.678+0000 7f3017c00700 1 -- 172.21.15.119:0/533043693 wait complete. 2024-05-27T09:40:49.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.679+0000 7f3017c00700 1 Processor -- start 2024-05-27T09:40:49.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.679+0000 7f3017c00700 1 -- start start 2024-05-27T09:40:49.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.680+0000 7f3017c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3010102630 0x7f3010198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:49.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.680+0000 7f3017c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3010102fd0 0x7f3010198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:49.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.680+0000 7f3017c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3010198d20 con 0x7f3010102630 2024-05-27T09:40:49.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.680+0000 7f3017c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301019cab0 con 0x7f3010102fd0 2024-05-27T09:40:49.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.680+0000 7f3015200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3010102630 0x7f3010198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:49.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.680+0000 7f3015200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3010102630 0x7f3010198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54690/0 (socket says 172.21.15.119:54690) 2024-05-27T09:40:49.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.680+0000 7f3015200700 1 -- 172.21.15.119:0/3516912621 learned_addr learned my addr 172.21.15.119:0/3516912621 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:40:49.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.681+0000 7f300fe00700 1 --2- 172.21.15.119:0/3516912621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3010102fd0 0x7f3010198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.681+0000 7f3015200700 1 -- 172.21.15.119:0/3516912621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3010102fd0 msgr2=0x7f3010198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.681+0000 7f3015200700 1 --2- 172.21.15.119:0/3516912621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3010102fd0 0x7f3010198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.681+0000 7f3015200700 1 -- 172.21.15.119:0/3516912621 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ffc0095f0 con 0x7f3010102630 2024-05-27T09:40:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.681+0000 7f3015200700 1 --2- 172.21.15.119:0/3516912621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3010102630 0x7f3010198100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f300400b4b0 tx=0x7f300400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.681+0000 7f300d600700 1 -- 172.21.15.119:0/3516912621 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3004011800 con 0x7f3010102630 2024-05-27T09:40:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.682+0000 7f300d600700 1 -- 172.21.15.119:0/3516912621 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3004011e20 con 0x7f3010102630 2024-05-27T09:40:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.682+0000 7f3017c00700 1 -- 172.21.15.119:0/3516912621 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f301019cd90 con 0x7f3010102630 2024-05-27T09:40:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.682+0000 7f3017c00700 1 -- 172.21.15.119:0/3516912621 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f301019d2e0 con 0x7f3010102630 2024-05-27T09:40:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.682+0000 7f300d600700 1 -- 172.21.15.119:0/3516912621 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f300400f2e0 con 0x7f3010102630 2024-05-27T09:40:49.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.683+0000 7f3017c00700 1 -- 172.21.15.119:0/3516912621 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f301010a8d0 con 0x7f3010102630 2024-05-27T09:40:49.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.684+0000 7f300d600700 1 -- 172.21.15.119:0/3516912621 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f3004011960 con 0x7f3010102630 2024-05-27T09:40:49.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.685+0000 7f300d600700 1 --2- 172.21.15.119:0/3516912621 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2ff8078a40 0x7f2ff807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:40:49.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.685+0000 7f300d600700 1 -- 172.21.15.119:0/3516912621 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f300409a5d0 con 0x7f3010102630 2024-05-27T09:40:49.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.686+0000 7f300fe00700 1 --2- 172.21.15.119:0/3516912621 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2ff8078a40 0x7f2ff807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:40:49.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.686+0000 7f300fe00700 1 --2- 172.21.15.119:0/3516912621 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2ff8078a40 0x7f2ff807aef0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3010199720 tx=0x7f2ffc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:40:49.691 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:49.690+0000 7f300d600700 1 -- 172.21.15.119:0/3516912621 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f30040621a0 con 0x7f3010102630 2024-05-27T09:40:50.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:50.024+0000 7f3017c00700 1 -- 172.21.15.119:0/3516912621 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f301004ea50 con 0x7f3010102630 2024-05-27T09:40:50.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:50.025+0000 7f300d600700 1 -- 172.21.15.119:0/3516912621 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f3004061b40 con 0x7f3010102630 2024-05-27T09:40:50.027 INFO:teuthology.orchestra.run.smithi119.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-27T09:40:50.027 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-27T09:40:50.027 INFO:teuthology.orchestra.run.smithi119.stdout: fs cephfs is degraded 2024-05-27T09:40:50.027 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:40:50.027 INFO:teuthology.orchestra.run.smithi119.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:40:50.027 INFO:teuthology.orchestra.run.smithi119.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-27T09:40:50.027 INFO:teuthology.orchestra.run.smithi119.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-27T09:40:50.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:50.028+0000 7f3017c00700 1 -- 172.21.15.119:0/3516912621 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2ff8078a40 msgr2=0x7f2ff807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:50.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:50.028+0000 7f3017c00700 1 --2- 172.21.15.119:0/3516912621 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2ff8078a40 0x7f2ff807aef0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3010199720 tx=0x7f2ffc01a040 comp rx=0 tx=0).stop 2024-05-27T09:40:50.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:50.028+0000 7f3017c00700 1 -- 172.21.15.119:0/3516912621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3010102630 msgr2=0x7f3010198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:40:50.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:50.028+0000 7f3017c00700 1 --2- 172.21.15.119:0/3516912621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3010102630 0x7f3010198100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f300400b4b0 tx=0x7f300400b870 comp rx=0 tx=0).stop 2024-05-27T09:40:50.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:50.029+0000 7f3017c00700 1 -- 172.21.15.119:0/3516912621 shutdown_connections 2024-05-27T09:40:50.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:50.029+0000 7f3017c00700 1 --2- 172.21.15.119:0/3516912621 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2ff8078a40 0x7f2ff807aef0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:50.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:50.029+0000 7f3017c00700 1 --2- 172.21.15.119:0/3516912621 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3010102630 0x7f3010198100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:50.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:50.029+0000 7f3017c00700 1 --2- 172.21.15.119:0/3516912621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3010102fd0 0x7f3010198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:40:50.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:50.029+0000 7f3017c00700 1 -- 172.21.15.119:0/3516912621 >> 172.21.15.119:0/3516912621 conn(0x7f30100fdef0 msgr2=0x7f30100ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:40:50.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:50.029+0000 7f3017c00700 1 -- 172.21.15.119:0/3516912621 shutdown_connections 2024-05-27T09:40:50.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:40:50.029+0000 7f3017c00700 1 -- 172.21.15.119:0/3516912621 wait complete. 2024-05-27T09:40:50.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:50 smithi119 ceph-mon[88269]: pgmap v228: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:50.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:50 smithi119 ceph-mon[88269]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:40:50.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:50 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/3516912621' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:40:50.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:50 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:50.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:50 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.ndfkhd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:40:50.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:50 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:40:50.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:50 smithi138 ceph-mon[70004]: pgmap v228: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:50.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:50 smithi138 ceph-mon[70004]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:40:50.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:50 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/3516912621' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:40:50.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:50 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:40:50.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:50 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.ndfkhd", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:40:50.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:50 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:40:51.582 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:51 smithi138 ceph-mon[70004]: Upgrade: Updating mds.cephfs.smithi138.ndfkhd 2024-05-27T09:40:51.582 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:51 smithi138 ceph-mon[70004]: Deploying daemon mds.cephfs.smithi138.ndfkhd on smithi138 2024-05-27T09:40:51.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:51 smithi119 ceph-mon[88269]: Upgrade: Updating mds.cephfs.smithi138.ndfkhd 2024-05-27T09:40:51.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:51 smithi119 ceph-mon[88269]: Deploying daemon mds.cephfs.smithi138.ndfkhd on smithi138 2024-05-27T09:40:52.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:52 smithi119 ceph-mon[88269]: pgmap v229: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:52.873 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:52 smithi138 ceph-mon[70004]: pgmap v229: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:54.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:54 smithi119 ceph-mon[88269]: pgmap v230: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:54.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:54 smithi119 ceph-mon[88269]: osdmap e84: 6 total, 6 up, 6 in 2024-05-27T09:40:54.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:54 smithi119 ceph-mon[88269]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-27T09:40:54.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:54 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.ndfkhd"}]: dispatch 2024-05-27T09:40:54.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:54 smithi138 ceph-mon[70004]: pgmap v230: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:54.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:54 smithi138 ceph-mon[70004]: osdmap e84: 6 total, 6 up, 6 in 2024-05-27T09:40:54.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:54 smithi138 ceph-mon[70004]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-27T09:40:54.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:54 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.ndfkhd"}]: dispatch 2024-05-27T09:40:56.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:56 smithi119 ceph-mon[88269]: pgmap v232: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:56.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:56 smithi138 ceph-mon[70004]: pgmap v232: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:57.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:40:57.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:40:58.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:40:58 smithi119 ceph-mon[88269]: pgmap v233: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:40:58.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:40:58 smithi138 ceph-mon[70004]: pgmap v233: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:01.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:00 smithi119 ceph-mon[88269]: pgmap v234: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:01.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:00 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:01.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:00 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:01.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:00 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:41:01.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:00 smithi119 ceph-mon[88269]: mds.? [v2:172.21.15.138:6824/3387966485,v1:172.21.15.138:6825/3387966485] up:boot 2024-05-27T09:41:01.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:00 smithi119 ceph-mon[88269]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-27T09:41:01.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:00 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.ndfkhd"}]: dispatch 2024-05-27T09:41:01.258 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:00 smithi138 ceph-mon[70004]: pgmap v234: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:01.258 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:00 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:01.258 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:00 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:01.258 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:00 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:41:01.258 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:00 smithi138 ceph-mon[70004]: mds.? [v2:172.21.15.138:6824/3387966485,v1:172.21.15.138:6825/3387966485] up:boot 2024-05-27T09:41:01.258 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:00 smithi138 ceph-mon[70004]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-27T09:41:01.258 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:00 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.ndfkhd"}]: dispatch 2024-05-27T09:41:03.175 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:02 smithi138 ceph-mon[70004]: pgmap v235: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:03.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:02 smithi119 ceph-mon[88269]: pgmap v235: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:04.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:04 smithi119 ceph-mon[88269]: pgmap v236: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:04.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:04 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:04.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:04 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:05.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:04 smithi138 ceph-mon[70004]: pgmap v236: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:05.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:04 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:05.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:04 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:07.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:06 smithi119 ceph-mon[88269]: pgmap v237: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:07.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:06 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:07.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:06 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:07.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:06 smithi138 ceph-mon[70004]: pgmap v237: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:07.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:06 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:07.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:06 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:09.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:08 smithi119 ceph-mon[88269]: pgmap v238: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:09.322 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:08 smithi138 ceph-mon[70004]: pgmap v238: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:11.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:10 smithi119 ceph-mon[88269]: pgmap v239: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:11.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:10 smithi138 ceph-mon[70004]: pgmap v239: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:12 smithi119 ceph-mon[88269]: pgmap v240: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:41:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:41:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:41:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:13.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:12 smithi138 ceph-mon[70004]: pgmap v240: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:13.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:41:13.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:13.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:13.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:41:13.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:41:13.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:14.126 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:13 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:41:14.126 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:13 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:14.126 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:13 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:14.126 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:13 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:14.126 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:13 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:14.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:13 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:41:14.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:13 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:14.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:13 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:14.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:13 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:14.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:13 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:15.221 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:14 smithi119 ceph-mon[88269]: pgmap v241: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:15.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:14 smithi138 ceph-mon[70004]: pgmap v241: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:16.718 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:16 smithi138 ceph-mon[70004]: pgmap v242: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:16.718 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:16 smithi138 ceph-mon[70004]: Upgrade: Updating mds.cephfs.smithi138.alszpk 2024-05-27T09:41:16.718 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:16 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:16.718 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:16 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.alszpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:41:16.718 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:16 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:41:16.718 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:16 smithi138 ceph-mon[70004]: Deploying daemon mds.cephfs.smithi138.alszpk on smithi138 2024-05-27T09:41:16.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:16 smithi119 ceph-mon[88269]: pgmap v242: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:16.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:16 smithi119 ceph-mon[88269]: Upgrade: Updating mds.cephfs.smithi138.alszpk 2024-05-27T09:41:16.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:16 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:16.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:16 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.alszpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T09:41:16.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:16 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:41:16.999 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:16 smithi119 ceph-mon[88269]: Deploying daemon mds.cephfs.smithi138.alszpk on smithi138 2024-05-27T09:41:18.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:18 smithi138 ceph-mon[70004]: pgmap v243: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:18.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:18 smithi119 ceph-mon[88269]: pgmap v243: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:20.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.202+0000 7f834ae00700 1 -- 172.21.15.119:0/1867245367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8344100d50 msgr2=0x7f8344104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:20.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.202+0000 7f834ae00700 1 --2- 172.21.15.119:0/1867245367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8344100d50 0x7f8344104d20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8330009910 tx=0x7f8330009c20 comp rx=0 tx=0).stop 2024-05-27T09:41:20.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.202+0000 7f834ae00700 1 -- 172.21.15.119:0/1867245367 shutdown_connections 2024-05-27T09:41:20.204 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.202+0000 7f834ae00700 1 --2- 172.21.15.119:0/1867245367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8344100d50 0x7f8344104d20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:20.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.202+0000 7f834ae00700 1 --2- 172.21.15.119:0/1867245367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83441003b0 0x7f8344100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:20.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.202+0000 7f834ae00700 1 -- 172.21.15.119:0/1867245367 >> 172.21.15.119:0/1867245367 conn(0x7f83440fbc30 msgr2=0x7f83440fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:20.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.202+0000 7f834ae00700 1 -- 172.21.15.119:0/1867245367 shutdown_connections 2024-05-27T09:41:20.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.202+0000 7f834ae00700 1 -- 172.21.15.119:0/1867245367 wait complete. 2024-05-27T09:41:20.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.203+0000 7f834ae00700 1 Processor -- start 2024-05-27T09:41:20.205 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.204+0000 7f834ae00700 1 -- start start 2024-05-27T09:41:20.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.204+0000 7f834ae00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83441003b0 0x7f8344195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:20.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.204+0000 7f834ae00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8344100d50 0x7f8344196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:20.206 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.204+0000 7f834ae00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83441969e0 con 0x7f83441003b0 2024-05-27T09:41:20.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.204+0000 7f834ae00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834419a770 con 0x7f8344100d50 2024-05-27T09:41:20.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.204+0000 7f8343e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83441003b0 0x7f8344195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:20.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.204+0000 7f8343e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83441003b0 0x7f8344195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43806/0 (socket says 172.21.15.119:43806) 2024-05-27T09:41:20.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.204+0000 7f8343e00700 1 -- 172.21.15.119:0/1273811805 learned_addr learned my addr 172.21.15.119:0/1273811805 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:41:20.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.205+0000 7f8343400700 1 --2- 172.21.15.119:0/1273811805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8344100d50 0x7f8344196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:20.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.205+0000 7f8343e00700 1 -- 172.21.15.119:0/1273811805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8344100d50 msgr2=0x7f8344196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:20.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.205+0000 7f8343e00700 1 --2- 172.21.15.119:0/1273811805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8344100d50 0x7f8344196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:20.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.205+0000 7f8343e00700 1 -- 172.21.15.119:0/1273811805 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83300095f0 con 0x7f83441003b0 2024-05-27T09:41:20.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.205+0000 7f8343e00700 1 --2- 172.21.15.119:0/1273811805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83441003b0 0x7f8344195dc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f833400d8d0 tx=0x7f833400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:20.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.205+0000 7f8340c00700 1 -- 172.21.15.119:0/1273811805 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8334009940 con 0x7f83441003b0 2024-05-27T09:41:20.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.205+0000 7f8340c00700 1 -- 172.21.15.119:0/1273811805 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8334010460 con 0x7f83441003b0 2024-05-27T09:41:20.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.205+0000 7f8340c00700 1 -- 172.21.15.119:0/1273811805 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f833400f5f0 con 0x7f83441003b0 2024-05-27T09:41:20.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.205+0000 7f834ae00700 1 -- 172.21.15.119:0/1273811805 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f834419aa50 con 0x7f83441003b0 2024-05-27T09:41:20.207 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.206+0000 7f834ae00700 1 -- 172.21.15.119:0/1273811805 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f834419afa0 con 0x7f83441003b0 2024-05-27T09:41:20.208 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.207+0000 7f834ae00700 1 -- 172.21.15.119:0/1273811805 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8344108720 con 0x7f83441003b0 2024-05-27T09:41:20.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.208+0000 7f8340c00700 1 -- 172.21.15.119:0/1273811805 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f83340105d0 con 0x7f83441003b0 2024-05-27T09:41:20.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.209+0000 7f8340c00700 1 --2- 172.21.15.119:0/1273811805 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8328078a40 0x7f832807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:20.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.209+0000 7f8340c00700 1 -- 172.21.15.119:0/1273811805 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f833409ae80 con 0x7f83441003b0 2024-05-27T09:41:20.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.210+0000 7f8343400700 1 --2- 172.21.15.119:0/1273811805 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8328078a40 0x7f832807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:20.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.210+0000 7f8343400700 1 --2- 172.21.15.119:0/1273811805 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8328078a40 0x7f832807aef0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8330006010 tx=0x7f833000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:20.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.215+0000 7f8340c00700 1 -- 172.21.15.119:0/1273811805 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f83340629b0 con 0x7f83441003b0 2024-05-27T09:41:20.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.470+0000 7f834ae00700 1 -- 172.21.15.119:0/1273811805 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f834419b230 con 0x7f8328078a40 2024-05-27T09:41:20.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.473+0000 7f8340c00700 1 -- 172.21.15.119:0/1273811805 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f834419b230 con 0x7f8328078a40 2024-05-27T09:41:20.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.476+0000 7f834ae00700 1 -- 172.21.15.119:0/1273811805 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8328078a40 msgr2=0x7f832807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:20.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.476+0000 7f834ae00700 1 --2- 172.21.15.119:0/1273811805 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8328078a40 0x7f832807aef0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8330006010 tx=0x7f833000b540 comp rx=0 tx=0).stop 2024-05-27T09:41:20.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.476+0000 7f834ae00700 1 -- 172.21.15.119:0/1273811805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83441003b0 msgr2=0x7f8344195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:20.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.476+0000 7f834ae00700 1 --2- 172.21.15.119:0/1273811805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83441003b0 0x7f8344195dc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f833400d8d0 tx=0x7f833400dc90 comp rx=0 tx=0).stop 2024-05-27T09:41:20.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.476+0000 7f834ae00700 1 -- 172.21.15.119:0/1273811805 shutdown_connections 2024-05-27T09:41:20.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.476+0000 7f834ae00700 1 --2- 172.21.15.119:0/1273811805 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8328078a40 0x7f832807aef0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:20.477 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.476+0000 7f834ae00700 1 --2- 172.21.15.119:0/1273811805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83441003b0 0x7f8344195dc0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:20.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.476+0000 7f834ae00700 1 --2- 172.21.15.119:0/1273811805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8344100d50 0x7f8344196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:20.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.476+0000 7f834ae00700 1 -- 172.21.15.119:0/1273811805 >> 172.21.15.119:0/1273811805 conn(0x7f83440fbc30 msgr2=0x7f83440fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:20.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.476+0000 7f834ae00700 1 -- 172.21.15.119:0/1273811805 shutdown_connections 2024-05-27T09:41:20.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.476+0000 7f834ae00700 1 -- 172.21.15.119:0/1273811805 wait complete. 2024-05-27T09:41:20.494 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:41:20.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.637+0000 7f1bcb200700 1 -- 172.21.15.119:0/613551256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bc40730f0 msgr2=0x7f1bc40734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:20.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.637+0000 7f1bcb200700 1 --2- 172.21.15.119:0/613551256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bc40730f0 0x7f1bc40734c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1bb400b0a0 tx=0x7f1bb400b3b0 comp rx=0 tx=0).stop 2024-05-27T09:41:20.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.637+0000 7f1bcb200700 1 -- 172.21.15.119:0/613551256 shutdown_connections 2024-05-27T09:41:20.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.637+0000 7f1bcb200700 1 --2- 172.21.15.119:0/613551256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bc4073a00 0x7f1bc410c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:20.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.637+0000 7f1bcb200700 1 --2- 172.21.15.119:0/613551256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bc40730f0 0x7f1bc40734c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:20.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.637+0000 7f1bcb200700 1 -- 172.21.15.119:0/613551256 >> 172.21.15.119:0/613551256 conn(0x7f1bc40fbc90 msgr2=0x7f1bc40fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:20.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.637+0000 7f1bcb200700 1 -- 172.21.15.119:0/613551256 shutdown_connections 2024-05-27T09:41:20.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.637+0000 7f1bcb200700 1 -- 172.21.15.119:0/613551256 wait complete. 2024-05-27T09:41:20.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.638+0000 7f1bcb200700 1 Processor -- start 2024-05-27T09:41:20.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.638+0000 7f1bcb200700 1 -- start start 2024-05-27T09:41:20.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.639+0000 7f1bcb200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bc40730f0 0x7f1bc4198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:20.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.639+0000 7f1bcb200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bc4073a00 0x7f1bc4198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:20.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.639+0000 7f1bcb200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc419c9c0 con 0x7f1bc40730f0 2024-05-27T09:41:20.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.639+0000 7f1bcb200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc419cb30 con 0x7f1bc4073a00 2024-05-27T09:41:20.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.639+0000 7f1bc3e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bc40730f0 0x7f1bc4198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:20.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.639+0000 7f1bc3e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bc40730f0 0x7f1bc4198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43814/0 (socket says 172.21.15.119:43814) 2024-05-27T09:41:20.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.639+0000 7f1bc3e00700 1 -- 172.21.15.119:0/4164862818 learned_addr learned my addr 172.21.15.119:0/4164862818 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:41:20.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.639+0000 7f1bbb400700 1 --2- 172.21.15.119:0/4164862818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bc4073a00 0x7f1bc4198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:20.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.639+0000 7f1bc3e00700 1 -- 172.21.15.119:0/4164862818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bc4073a00 msgr2=0x7f1bc4198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:20.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.639+0000 7f1bc3e00700 1 --2- 172.21.15.119:0/4164862818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bc4073a00 0x7f1bc4198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:20.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.639+0000 7f1bc3e00700 1 -- 172.21.15.119:0/4164862818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb4009d00 con 0x7f1bc40730f0 2024-05-27T09:41:20.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.639+0000 7f1bbb400700 1 --2- 172.21.15.119:0/4164862818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bc4073a00 0x7f1bc4198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:41:20.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.640+0000 7f1bc3e00700 1 --2- 172.21.15.119:0/4164862818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bc40730f0 0x7f1bc4198010 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1bb400b070 tx=0x7f1bb4003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:20.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.640+0000 7f1bc1600700 1 -- 172.21.15.119:0/4164862818 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bb400e040 con 0x7f1bc40730f0 2024-05-27T09:41:20.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.640+0000 7f1bc1600700 1 -- 172.21.15.119:0/4164862818 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1bb4007dd0 con 0x7f1bc40730f0 2024-05-27T09:41:20.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.640+0000 7f1bcb200700 1 -- 172.21.15.119:0/4164862818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc419cdb0 con 0x7f1bc40730f0 2024-05-27T09:41:20.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.640+0000 7f1bc1600700 1 -- 172.21.15.119:0/4164862818 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bb401d8b0 con 0x7f1bc40730f0 2024-05-27T09:41:20.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.640+0000 7f1bcb200700 1 -- 172.21.15.119:0/4164862818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc419d300 con 0x7f1bc40730f0 2024-05-27T09:41:20.642 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.641+0000 7f1bcb200700 1 -- 172.21.15.119:0/4164862818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc4109b70 con 0x7f1bc40730f0 2024-05-27T09:41:20.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.643+0000 7f1bc1600700 1 -- 172.21.15.119:0/4164862818 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f1bb4004790 con 0x7f1bc40730f0 2024-05-27T09:41:20.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.644+0000 7f1bc1600700 1 --2- 172.21.15.119:0/4164862818 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1ba8078a40 0x7f1ba807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:20.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.644+0000 7f1bc1600700 1 -- 172.21.15.119:0/4164862818 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1bb409c680 con 0x7f1bc40730f0 2024-05-27T09:41:20.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.644+0000 7f1bbb400700 1 --2- 172.21.15.119:0/4164862818 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1ba8078a40 0x7f1ba807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:20.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.645+0000 7f1bbb400700 1 --2- 172.21.15.119:0/4164862818 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1ba8078a40 0x7f1ba807aef0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1bac007900 tx=0x7f1bac008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:20.650 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.649+0000 7f1bc1600700 1 -- 172.21.15.119:0/4164862818 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f1bb40641b0 con 0x7f1bc40730f0 2024-05-27T09:41:20.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:20 smithi119 ceph-mon[88269]: pgmap v244: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:20.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:20 smithi119 ceph-mon[88269]: osdmap e85: 6 total, 6 up, 6 in 2024-05-27T09:41:20.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:20 smithi119 ceph-mon[88269]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-27T09:41:20.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:20 smithi138 ceph-mon[70004]: pgmap v244: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:20.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:20 smithi138 ceph-mon[70004]: osdmap e85: 6 total, 6 up, 6 in 2024-05-27T09:41:20.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:20 smithi138 ceph-mon[70004]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-05-27T09:41:20.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.898+0000 7f1bcb200700 1 -- 172.21.15.119:0/4164862818 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bc419d520 con 0x7f1ba8078a40 2024-05-27T09:41:20.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.900+0000 7f1bc1600700 1 -- 172.21.15.119:0/4164862818 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1bc419d520 con 0x7f1ba8078a40 2024-05-27T09:41:20.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.904+0000 7f1bcb200700 1 -- 172.21.15.119:0/4164862818 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1ba8078a40 msgr2=0x7f1ba807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:20.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.904+0000 7f1bcb200700 1 --2- 172.21.15.119:0/4164862818 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1ba8078a40 0x7f1ba807aef0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1bac007900 tx=0x7f1bac008040 comp rx=0 tx=0).stop 2024-05-27T09:41:20.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.904+0000 7f1bcb200700 1 -- 172.21.15.119:0/4164862818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bc40730f0 msgr2=0x7f1bc4198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:20.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.904+0000 7f1bcb200700 1 --2- 172.21.15.119:0/4164862818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bc40730f0 0x7f1bc4198010 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1bb400b070 tx=0x7f1bb4003ce0 comp rx=0 tx=0).stop 2024-05-27T09:41:20.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.904+0000 7f1bcb200700 1 -- 172.21.15.119:0/4164862818 shutdown_connections 2024-05-27T09:41:20.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.904+0000 7f1bcb200700 1 --2- 172.21.15.119:0/4164862818 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1ba8078a40 0x7f1ba807aef0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:20.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.904+0000 7f1bcb200700 1 --2- 172.21.15.119:0/4164862818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1bc40730f0 0x7f1bc4198010 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:20.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.904+0000 7f1bcb200700 1 --2- 172.21.15.119:0/4164862818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bc4073a00 0x7f1bc4198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:20.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.904+0000 7f1bcb200700 1 -- 172.21.15.119:0/4164862818 >> 172.21.15.119:0/4164862818 conn(0x7f1bc40fbc90 msgr2=0x7f1bc4106cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:20.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.904+0000 7f1bcb200700 1 -- 172.21.15.119:0/4164862818 shutdown_connections 2024-05-27T09:41:20.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:20.904+0000 7f1bcb200700 1 -- 172.21.15.119:0/4164862818 wait complete. 2024-05-27T09:41:21.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.051+0000 7fa571200700 1 -- 172.21.15.119:0/1448306415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa56c102450 msgr2=0x7fa56c102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:21.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.051+0000 7fa571200700 1 --2- 172.21.15.119:0/1448306415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa56c102450 0x7fa56c102820 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009910 tx=0x7fa55c009c20 comp rx=0 tx=0).stop 2024-05-27T09:41:21.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.052+0000 7fa571200700 1 -- 172.21.15.119:0/1448306415 shutdown_connections 2024-05-27T09:41:21.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.052+0000 7fa571200700 1 --2- 172.21.15.119:0/1448306415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa56c102df0 0x7fa56c106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:21.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.052+0000 7fa571200700 1 --2- 172.21.15.119:0/1448306415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa56c102450 0x7fa56c102820 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:21.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.052+0000 7fa571200700 1 -- 172.21.15.119:0/1448306415 >> 172.21.15.119:0/1448306415 conn(0x7fa56c0fdd10 msgr2=0x7fa56c100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:21.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.052+0000 7fa571200700 1 -- 172.21.15.119:0/1448306415 shutdown_connections 2024-05-27T09:41:21.053 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.052+0000 7fa571200700 1 -- 172.21.15.119:0/1448306415 wait complete. 2024-05-27T09:41:21.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.053+0000 7fa571200700 1 Processor -- start 2024-05-27T09:41:21.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.053+0000 7fa571200700 1 -- start start 2024-05-27T09:41:21.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.053+0000 7fa571200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa56c102450 0x7fa56c197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:21.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.053+0000 7fa571200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa56c102df0 0x7fa56c1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:21.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.053+0000 7fa571200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c1989c0 con 0x7fa56c102df0 2024-05-27T09:41:21.054 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.053+0000 7fa571200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c19c750 con 0x7fa56c102450 2024-05-27T09:41:21.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.054+0000 7fa56aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa56c102df0 0x7fa56c1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:21.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.054+0000 7fa56aa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa56c102df0 0x7fa56c1982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43826/0 (socket says 172.21.15.119:43826) 2024-05-27T09:41:21.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.054+0000 7fa56aa00700 1 -- 172.21.15.119:0/1642253748 learned_addr learned my addr 172.21.15.119:0/1642253748 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:41:21.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.054+0000 7fa56b400700 1 --2- 172.21.15.119:0/1642253748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa56c102450 0x7fa56c197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:21.055 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.054+0000 7fa56aa00700 1 -- 172.21.15.119:0/1642253748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa56c102450 msgr2=0x7fa56c197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:21.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.054+0000 7fa56aa00700 1 --2- 172.21.15.119:0/1642253748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa56c102450 0x7fa56c197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:21.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.054+0000 7fa56aa00700 1 -- 172.21.15.119:0/1642253748 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c0095f0 con 0x7fa56c102df0 2024-05-27T09:41:21.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.055+0000 7fa56aa00700 1 --2- 172.21.15.119:0/1642253748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa56c102df0 0x7fa56c1982e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa55800c8f0 tx=0x7fa55800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:21.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.055+0000 7fa563e00700 1 -- 172.21.15.119:0/1642253748 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5580078a0 con 0x7fa56c102df0 2024-05-27T09:41:21.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.055+0000 7fa563e00700 1 -- 172.21.15.119:0/1642253748 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa55800f450 con 0x7fa56c102df0 2024-05-27T09:41:21.056 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.055+0000 7fa563e00700 1 -- 172.21.15.119:0/1642253748 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa558018610 con 0x7fa56c102df0 2024-05-27T09:41:21.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.055+0000 7fa571200700 1 -- 172.21.15.119:0/1642253748 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa56c19ca30 con 0x7fa56c102df0 2024-05-27T09:41:21.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.055+0000 7fa571200700 1 -- 172.21.15.119:0/1642253748 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56c19cf80 con 0x7fa56c102df0 2024-05-27T09:41:21.057 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.057+0000 7fa571200700 1 -- 172.21.15.119:0/1642253748 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56c10a760 con 0x7fa56c102df0 2024-05-27T09:41:21.058 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.058+0000 7fa563e00700 1 -- 172.21.15.119:0/1642253748 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fa55800f5c0 con 0x7fa56c102df0 2024-05-27T09:41:21.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.059+0000 7fa563e00700 1 --2- 172.21.15.119:0/1642253748 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fa550078a40 0x7fa55007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:21.059 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.059+0000 7fa563e00700 1 -- 172.21.15.119:0/1642253748 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa55809acd0 con 0x7fa56c102df0 2024-05-27T09:41:21.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.059+0000 7fa56b400700 1 --2- 172.21.15.119:0/1642253748 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fa550078a40 0x7fa55007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:21.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.060+0000 7fa56b400700 1 --2- 172.21.15.119:0/1642253748 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fa550078a40 0x7fa55007aef0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006010 tx=0x7fa55c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:21.064 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.064+0000 7fa563e00700 1 -- 172.21.15.119:0/1642253748 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fa558062880 con 0x7fa56c102df0 2024-05-27T09:41:21.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.310+0000 7fa571200700 1 -- 172.21.15.119:0/1642253748 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa56c19d320 con 0x7fa550078a40 2024-05-27T09:41:21.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.341+0000 7fa563e00700 1 -- 172.21.15.119:0/1642253748 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa56c19d320 con 0x7fa550078a40 2024-05-27T09:41:21.343 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:41:21.343 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (8m) 41s ago 22m 15.2M - 0.25.0 c8568f914cd2 6b90564dcef7 2024-05-27T09:41:21.343 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (22m) 41s ago 22m 9298k - 18.2.0 dc2bc1663786 302c427b51e8 2024-05-27T09:41:21.343 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (21m) 16s ago 21m 9281k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:41:21.343 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (5m) 41s ago 22m 6669k - 19.0.0-3868-ga6507388 850a06a84d28 011d811ce7fc 2024-05-27T09:41:21.343 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (5m) 16s ago 21m 6665k - 19.0.0-3868-ga6507388 850a06a84d28 b546b3ed1afc 2024-05-27T09:41:21.343 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (8m) 41s ago 22m 74.2M - 9.4.12 2bacad6d85d8 b3abdab08db4 2024-05-27T09:41:21.343 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (73s) 41s ago 15m 79.5M - 19.0.0-3868-ga6507388 850a06a84d28 5a02732ff870 2024-05-27T09:41:21.343 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (49s) 41s ago 15m 78.4M - 19.0.0-3868-ga6507388 850a06a84d28 3faccd4c18e8 2024-05-27T09:41:21.343 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (15m) 16s ago 15m 19.0M - 18.2.0 dc2bc1663786 108878452ab4 2024-05-27T09:41:21.343 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (22s) 16s ago 15m 78.4M - 19.0.0-3868-ga6507388 850a06a84d28 866712d7b973 2024-05-27T09:41:21.343 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (11m) 41s ago 24m 593M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:41:21.343 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (10m) 16s ago 21m 535M - 19.0.0-3868-ga6507388 850a06a84d28 2fc3a80696fa 2024-05-27T09:41:21.344 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (7m) 41s ago 24m 123M 2048M 19.0.0-3868-ga6507388 850a06a84d28 3e5c7a9ad372 2024-05-27T09:41:21.344 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (6m) 16s ago 21m 111M 2048M 19.0.0-3868-ga6507388 850a06a84d28 e4d471274cf1 2024-05-27T09:41:21.344 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (9m) 41s ago 22m 7814k - 1.5.0 0da6a335fe13 709c37815cff 2024-05-27T09:41:21.344 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (9m) 16s ago 21m 8162k - 1.5.0 0da6a335fe13 999047a1a5bf 2024-05-27T09:41:21.344 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (4m) 41s ago 20m 157M 1216M 19.0.0-3868-ga6507388 850a06a84d28 5a0e162d870e 2024-05-27T09:41:21.344 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (4m) 41s ago 19m 171M 1216M 19.0.0-3868-ga6507388 850a06a84d28 effcc7c53dd3 2024-05-27T09:41:21.344 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (3m) 41s ago 18m 149M 1216M 19.0.0-3868-ga6507388 850a06a84d28 f0bd34ca416f 2024-05-27T09:41:21.344 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (3m) 16s ago 18m 174M 2240M 19.0.0-3868-ga6507388 850a06a84d28 5e8d3e398c21 2024-05-27T09:41:21.344 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (2m) 16s ago 17m 163M 2240M 19.0.0-3868-ga6507388 850a06a84d28 250e443364de 2024-05-27T09:41:21.344 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (98s) 16s ago 17m 153M 2240M 19.0.0-3868-ga6507388 850a06a84d28 90d98c5b2f16 2024-05-27T09:41:21.344 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (9m) 41s ago 21m 72.5M - 2.43.0 a07b618ecd1d cba50454ed21 2024-05-27T09:41:21.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.345+0000 7fa571200700 1 -- 172.21.15.119:0/1642253748 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fa550078a40 msgr2=0x7fa55007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:21.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.345+0000 7fa571200700 1 --2- 172.21.15.119:0/1642253748 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fa550078a40 0x7fa55007aef0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006010 tx=0x7fa55c01a040 comp rx=0 tx=0).stop 2024-05-27T09:41:21.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.345+0000 7fa571200700 1 -- 172.21.15.119:0/1642253748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa56c102df0 msgr2=0x7fa56c1982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:21.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.345+0000 7fa571200700 1 --2- 172.21.15.119:0/1642253748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa56c102df0 0x7fa56c1982e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa55800c8f0 tx=0x7fa55800cc00 comp rx=0 tx=0).stop 2024-05-27T09:41:21.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.345+0000 7fa571200700 1 -- 172.21.15.119:0/1642253748 shutdown_connections 2024-05-27T09:41:21.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.345+0000 7fa571200700 1 --2- 172.21.15.119:0/1642253748 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fa550078a40 0x7fa55007aef0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:21.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.345+0000 7fa571200700 1 --2- 172.21.15.119:0/1642253748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa56c102450 0x7fa56c197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:21.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.345+0000 7fa571200700 1 --2- 172.21.15.119:0/1642253748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa56c102df0 0x7fa56c1982e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:21.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.345+0000 7fa571200700 1 -- 172.21.15.119:0/1642253748 >> 172.21.15.119:0/1642253748 conn(0x7fa56c0fdd10 msgr2=0x7fa56c1000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:21.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.345+0000 7fa571200700 1 -- 172.21.15.119:0/1642253748 shutdown_connections 2024-05-27T09:41:21.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.345+0000 7fa571200700 1 -- 172.21.15.119:0/1642253748 wait complete. 2024-05-27T09:41:21.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.492+0000 7f728c800700 1 -- 172.21.15.119:0/2020835879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7284100d70 msgr2=0x7f7284104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:21.493 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.492+0000 7f728c800700 1 --2- 172.21.15.119:0/2020835879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7284100d70 0x7f7284104d40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7274009910 tx=0x7f7274009c20 comp rx=0 tx=0).stop 2024-05-27T09:41:21.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.492+0000 7f728c800700 1 -- 172.21.15.119:0/2020835879 shutdown_connections 2024-05-27T09:41:21.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.492+0000 7f728c800700 1 --2- 172.21.15.119:0/2020835879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7284100d70 0x7f7284104d40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:21.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.492+0000 7f728c800700 1 --2- 172.21.15.119:0/2020835879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72841003d0 0x7f72841007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:21.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.492+0000 7f728c800700 1 -- 172.21.15.119:0/2020835879 >> 172.21.15.119:0/2020835879 conn(0x7f72840fbc90 msgr2=0x7f72840fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:21.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.493+0000 7f728c800700 1 -- 172.21.15.119:0/2020835879 shutdown_connections 2024-05-27T09:41:21.494 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.493+0000 7f728c800700 1 -- 172.21.15.119:0/2020835879 wait complete. 2024-05-27T09:41:21.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.494+0000 7f728c800700 1 Processor -- start 2024-05-27T09:41:21.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.494+0000 7f728c800700 1 -- start start 2024-05-27T09:41:21.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.494+0000 7f728c800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72841003d0 0x7f7284197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:21.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.494+0000 7f728c800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7284100d70 0x7f7284198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:21.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.494+0000 7f728c800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7284198b40 con 0x7f72841003d0 2024-05-27T09:41:21.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.494+0000 7f728c800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728419c8d0 con 0x7f7284100d70 2024-05-27T09:41:21.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.494+0000 7f7289e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72841003d0 0x7f7284197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:21.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.494+0000 7f7289e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72841003d0 0x7f7284197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43840/0 (socket says 172.21.15.119:43840) 2024-05-27T09:41:21.496 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.494+0000 7f7289e00700 1 -- 172.21.15.119:0/2026940115 learned_addr learned my addr 172.21.15.119:0/2026940115 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:41:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.495+0000 7f7289400700 1 --2- 172.21.15.119:0/2026940115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7284100d70 0x7f7284198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.495+0000 7f7289e00700 1 -- 172.21.15.119:0/2026940115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7284100d70 msgr2=0x7f7284198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.495+0000 7f7289e00700 1 --2- 172.21.15.119:0/2026940115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7284100d70 0x7f7284198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.495+0000 7f7289e00700 1 -- 172.21.15.119:0/2026940115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72740095f0 con 0x7f72841003d0 2024-05-27T09:41:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.495+0000 7f7289400700 1 --2- 172.21.15.119:0/2026940115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7284100d70 0x7f7284198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:41:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.495+0000 7f7289e00700 1 --2- 172.21.15.119:0/2026940115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72841003d0 0x7f7284197f20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f727800c8a0 tx=0x7f727800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.495+0000 7f727ea00700 1 -- 172.21.15.119:0/2026940115 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f727800cea0 con 0x7f72841003d0 2024-05-27T09:41:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.495+0000 7f728c800700 1 -- 172.21.15.119:0/2026940115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f728419cbb0 con 0x7f72841003d0 2024-05-27T09:41:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.495+0000 7f727ea00700 1 -- 172.21.15.119:0/2026940115 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7278004830 con 0x7f72841003d0 2024-05-27T09:41:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.495+0000 7f727ea00700 1 -- 172.21.15.119:0/2026940115 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72780055b0 con 0x7f72841003d0 2024-05-27T09:41:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.496+0000 7f728c800700 1 -- 172.21.15.119:0/2026940115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728419d100 con 0x7f72841003d0 2024-05-27T09:41:21.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.497+0000 7f728c800700 1 -- 172.21.15.119:0/2026940115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72841044c0 con 0x7f72841003d0 2024-05-27T09:41:21.498 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.498+0000 7f727ea00700 1 -- 172.21.15.119:0/2026940115 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f72780049a0 con 0x7f72841003d0 2024-05-27T09:41:21.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.499+0000 7f727ea00700 1 --2- 172.21.15.119:0/2026940115 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f726c078a40 0x7f726c07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:21.499 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.499+0000 7f727ea00700 1 -- 172.21.15.119:0/2026940115 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f727809aac0 con 0x7f72841003d0 2024-05-27T09:41:21.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.499+0000 7f7289400700 1 --2- 172.21.15.119:0/2026940115 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f726c078a40 0x7f726c07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:21.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.500+0000 7f7289400700 1 --2- 172.21.15.119:0/2026940115 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f726c078a40 0x7f726c07aef0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7274006010 tx=0x7f727400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:21.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.504+0000 7f727ea00700 1 -- 172.21.15.119:0/2026940115 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f72780625f0 con 0x7f72841003d0 2024-05-27T09:41:21.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.837+0000 7f728c800700 1 -- 172.21.15.119:0/2026940115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f728419d480 con 0x7f72841003d0 2024-05-27T09:41:21.838 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:21 smithi119 ceph-mon[88269]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:41:21.838 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:21 smithi119 ceph-mon[88269]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:41:21.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.838+0000 7f727ea00700 1 -- 172.21.15.119:0/2026940115 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f7278004c50 con 0x7f72841003d0 2024-05-27T09:41:21.839 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:41:21.839 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:41:21.839 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:41:21.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:41:21.839 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:41:21.839 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:41:21.839 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:41:21.839 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:41:21.840 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 6 2024-05-27T09:41:21.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:41:21.840 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:41:21.840 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 3 2024-05-27T09:41:21.840 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:41:21.840 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:41:21.840 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 13 2024-05-27T09:41:21.840 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:41:21.840 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:41:21.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.841+0000 7f728c800700 1 -- 172.21.15.119:0/2026940115 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f726c078a40 msgr2=0x7f726c07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:21.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.841+0000 7f728c800700 1 --2- 172.21.15.119:0/2026940115 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f726c078a40 0x7f726c07aef0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7274006010 tx=0x7f727400b540 comp rx=0 tx=0).stop 2024-05-27T09:41:21.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.841+0000 7f728c800700 1 -- 172.21.15.119:0/2026940115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72841003d0 msgr2=0x7f7284197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:21.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.841+0000 7f728c800700 1 --2- 172.21.15.119:0/2026940115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72841003d0 0x7f7284197f20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f727800c8a0 tx=0x7f727800cc60 comp rx=0 tx=0).stop 2024-05-27T09:41:21.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.841+0000 7f728c800700 1 -- 172.21.15.119:0/2026940115 shutdown_connections 2024-05-27T09:41:21.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.841+0000 7f728c800700 1 --2- 172.21.15.119:0/2026940115 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f726c078a40 0x7f726c07aef0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:21.842 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.841+0000 7f728c800700 1 --2- 172.21.15.119:0/2026940115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72841003d0 0x7f7284197f20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:21.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.841+0000 7f728c800700 1 --2- 172.21.15.119:0/2026940115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7284100d70 0x7f7284198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:21.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.841+0000 7f728c800700 1 -- 172.21.15.119:0/2026940115 >> 172.21.15.119:0/2026940115 conn(0x7f72840fbc90 msgr2=0x7f72840fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:21.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.842+0000 7f728c800700 1 -- 172.21.15.119:0/2026940115 shutdown_connections 2024-05-27T09:41:21.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.842+0000 7f728c800700 1 -- 172.21.15.119:0/2026940115 wait complete. 2024-05-27T09:41:21.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:21 smithi138 ceph-mon[70004]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:41:21.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:21 smithi138 ceph-mon[70004]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:41:21.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.994+0000 7fa4aa400700 1 -- 172.21.15.119:0/842835727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4a410b700 msgr2=0x7fa4a410bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:21.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.994+0000 7fa4aa400700 1 --2- 172.21.15.119:0/842835727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4a410b700 0x7fa4a410bad0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa494009910 tx=0x7fa494009c20 comp rx=0 tx=0).stop 2024-05-27T09:41:21.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.994+0000 7fa4aa400700 1 -- 172.21.15.119:0/842835727 shutdown_connections 2024-05-27T09:41:21.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.994+0000 7fa4aa400700 1 --2- 172.21.15.119:0/842835727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4a410c010 0x7fa4a4106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:21.995 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.994+0000 7fa4aa400700 1 --2- 172.21.15.119:0/842835727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4a410b700 0x7fa4a410bad0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:21.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.994+0000 7fa4aa400700 1 -- 172.21.15.119:0/842835727 >> 172.21.15.119:0/842835727 conn(0x7fa4a4076000 msgr2=0x7fa4a4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:21.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.994+0000 7fa4aa400700 1 -- 172.21.15.119:0/842835727 shutdown_connections 2024-05-27T09:41:21.996 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.994+0000 7fa4aa400700 1 -- 172.21.15.119:0/842835727 wait complete. 2024-05-27T09:41:21.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.995+0000 7fa4aa400700 1 Processor -- start 2024-05-27T09:41:21.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.995+0000 7fa4aa400700 1 -- start start 2024-05-27T09:41:21.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.996+0000 7fa4aa400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4a410b700 0x7fa4a419c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:21.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.996+0000 7fa4aa400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4a410c010 0x7fa4a419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:21.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.996+0000 7fa4aa400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a419ced0 con 0x7fa4a410c010 2024-05-27T09:41:21.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.996+0000 7fa4aa400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41a0c60 con 0x7fa4a410b700 2024-05-27T09:41:21.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.996+0000 7fa4a2a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4a410c010 0x7fa4a419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:21.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.996+0000 7fa4a2a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4a410c010 0x7fa4a419c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43860/0 (socket says 172.21.15.119:43860) 2024-05-27T09:41:21.997 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.996+0000 7fa4a2a00700 1 -- 172.21.15.119:0/3796928933 learned_addr learned my addr 172.21.15.119:0/3796928933 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:41:21.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.996+0000 7fa4a3400700 1 --2- 172.21.15.119:0/3796928933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4a410b700 0x7fa4a419c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:21.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.996+0000 7fa4a2a00700 1 -- 172.21.15.119:0/3796928933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4a410b700 msgr2=0x7fa4a419c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:21.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.996+0000 7fa4a2a00700 1 --2- 172.21.15.119:0/3796928933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4a410b700 0x7fa4a419c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:21.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.996+0000 7fa4a2a00700 1 -- 172.21.15.119:0/3796928933 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4940095f0 con 0x7fa4a410c010 2024-05-27T09:41:21.998 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.996+0000 7fa4a3400700 1 --2- 172.21.15.119:0/3796928933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4a410b700 0x7fa4a419c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:41:21.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.997+0000 7fa4a2a00700 1 --2- 172.21.15.119:0/3796928933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4a410c010 0x7fa4a419c7f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa49000d900 tx=0x7fa49000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:21.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.997+0000 7fa49be00700 1 -- 172.21.15.119:0/3796928933 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa490009900 con 0x7fa4a410c010 2024-05-27T09:41:21.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.997+0000 7fa49be00700 1 -- 172.21.15.119:0/3796928933 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa490010460 con 0x7fa4a410c010 2024-05-27T09:41:21.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.997+0000 7fa4aa400700 1 -- 172.21.15.119:0/3796928933 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a41a0f40 con 0x7fa4a410c010 2024-05-27T09:41:21.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.997+0000 7fa49be00700 1 -- 172.21.15.119:0/3796928933 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa49000f5f0 con 0x7fa4a410c010 2024-05-27T09:41:21.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.997+0000 7fa4aa400700 1 -- 172.21.15.119:0/3796928933 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a41a1510 con 0x7fa4a410c010 2024-05-27T09:41:21.999 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:21.998+0000 7fa4aa400700 1 -- 172.21.15.119:0/3796928933 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a404ea50 con 0x7fa4a410c010 2024-05-27T09:41:22.000 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.000+0000 7fa49be00700 1 -- 172.21.15.119:0/3796928933 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fa4900105d0 con 0x7fa4a410c010 2024-05-27T09:41:22.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.001+0000 7fa49be00700 1 --2- 172.21.15.119:0/3796928933 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fa488078a40 0x7fa48807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:22.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.001+0000 7fa49be00700 1 -- 172.21.15.119:0/3796928933 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa49009abc0 con 0x7fa4a410c010 2024-05-27T09:41:22.001 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.001+0000 7fa4a3400700 1 --2- 172.21.15.119:0/3796928933 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fa488078a40 0x7fa48807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:22.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.002+0000 7fa4a3400700 1 --2- 172.21.15.119:0/3796928933 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fa488078a40 0x7fa48807aef0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa494006010 tx=0x7fa49400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:22.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.006+0000 7fa49be00700 1 -- 172.21.15.119:0/3796928933 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fa490062770 con 0x7fa4a410c010 2024-05-27T09:41:22.288 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.288+0000 7fa4aa400700 1 -- 172.21.15.119:0/3796928933 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa4a419d610 con 0x7fa4a410c010 2024-05-27T09:41:22.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.289+0000 7fa49be00700 1 -- 172.21.15.119:0/3796928933 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1701 (secure 0 0 0) 0x7fa490062110 con 0x7fa4a410c010 2024-05-27T09:41:22.290 INFO:teuthology.orchestra.run.smithi119.stdout:e31 2024-05-27T09:41:22.290 INFO:teuthology.orchestra.run.smithi119.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T09:41:22.290 INFO:teuthology.orchestra.run.smithi119.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:41:22.290 INFO:teuthology.orchestra.run.smithi119.stdout:legacy client fscid: 1 2024-05-27T09:41:22.290 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:41:22.290 INFO:teuthology.orchestra.run.smithi119.stdout:Filesystem 'cephfs' (1) 2024-05-27T09:41:22.290 INFO:teuthology.orchestra.run.smithi119.stdout:fs_name cephfs 2024-05-27T09:41:22.290 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 23 2024-05-27T09:41:22.290 INFO:teuthology.orchestra.run.smithi119.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-05-27T09:41:22.290 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-05-27T09:25:53.387388+0000 2024-05-27T09:41:22.291 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2024-05-27T09:39:57.913154+0000 2024-05-27T09:41:22.291 INFO:teuthology.orchestra.run.smithi119.stdout:tableserver 0 2024-05-27T09:41:22.291 INFO:teuthology.orchestra.run.smithi119.stdout:root 0 2024-05-27T09:41:22.291 INFO:teuthology.orchestra.run.smithi119.stdout:session_timeout 60 2024-05-27T09:41:22.291 INFO:teuthology.orchestra.run.smithi119.stdout:session_autoclose 300 2024-05-27T09:41:22.291 INFO:teuthology.orchestra.run.smithi119.stdout:max_file_size 1099511627776 2024-05-27T09:41:22.291 INFO:teuthology.orchestra.run.smithi119.stdout:max_xattr_size 65536 2024-05-27T09:41:22.291 INFO:teuthology.orchestra.run.smithi119.stdout:required_client_features {} 2024-05-27T09:41:22.291 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure 0 2024-05-27T09:41:22.291 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure_osd_epoch 81 2024-05-27T09:41:22.291 INFO:teuthology.orchestra.run.smithi119.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:41:22.291 INFO:teuthology.orchestra.run.smithi119.stdout:max_mds 2 2024-05-27T09:41:22.291 INFO:teuthology.orchestra.run.smithi119.stdout:in 0,1 2024-05-27T09:41:22.291 INFO:teuthology.orchestra.run.smithi119.stdout:up {} 2024-05-27T09:41:22.291 INFO:teuthology.orchestra.run.smithi119.stdout:failed 0,1 2024-05-27T09:41:22.292 INFO:teuthology.orchestra.run.smithi119.stdout:damaged 2024-05-27T09:41:22.292 INFO:teuthology.orchestra.run.smithi119.stdout:stopped 2024-05-27T09:41:22.292 INFO:teuthology.orchestra.run.smithi119.stdout:data_pools [3] 2024-05-27T09:41:22.292 INFO:teuthology.orchestra.run.smithi119.stdout:metadata_pool 2 2024-05-27T09:41:22.292 INFO:teuthology.orchestra.run.smithi119.stdout:inline_data enabled 2024-05-27T09:41:22.292 INFO:teuthology.orchestra.run.smithi119.stdout:balancer 2024-05-27T09:41:22.292 INFO:teuthology.orchestra.run.smithi119.stdout:bal_rank_mask -1 2024-05-27T09:41:22.292 INFO:teuthology.orchestra.run.smithi119.stdout:standby_count_wanted 1 2024-05-27T09:41:22.292 INFO:teuthology.orchestra.run.smithi119.stdout:qdb_cluster leader: 0 members: 2024-05-27T09:41:22.292 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:41:22.292 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:41:22.292 INFO:teuthology.orchestra.run.smithi119.stdout:Standby daemons: 2024-05-27T09:41:22.293 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:41:22.293 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.nghcey{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.119:6826/3357235708,v1:172.21.15.119:6827/3357235708] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T09:41:22.293 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.ppmzzk{-1:34468} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.119:6828/2430623742,v1:172.21.15.119:6829/2430623742] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T09:41:22.293 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.ndfkhd{-1:44289} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/3387966485,v1:172.21.15.138:6825/3387966485] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T09:41:22.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.292+0000 7fa4aa400700 1 -- 172.21.15.119:0/3796928933 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fa488078a40 msgr2=0x7fa48807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:22.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.292+0000 7fa4aa400700 1 --2- 172.21.15.119:0/3796928933 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fa488078a40 0x7fa48807aef0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa494006010 tx=0x7fa49400b540 comp rx=0 tx=0).stop 2024-05-27T09:41:22.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.292+0000 7fa4aa400700 1 -- 172.21.15.119:0/3796928933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4a410c010 msgr2=0x7fa4a419c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:22.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.292+0000 7fa4aa400700 1 --2- 172.21.15.119:0/3796928933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4a410c010 0x7fa4a419c7f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa49000d900 tx=0x7fa49000dcc0 comp rx=0 tx=0).stop 2024-05-27T09:41:22.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.292+0000 7fa4aa400700 1 -- 172.21.15.119:0/3796928933 shutdown_connections 2024-05-27T09:41:22.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.292+0000 7fa4aa400700 1 --2- 172.21.15.119:0/3796928933 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fa488078a40 0x7fa48807aef0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:22.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.292+0000 7fa4aa400700 1 --2- 172.21.15.119:0/3796928933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4a410b700 0x7fa4a419c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:22.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.292+0000 7fa4aa400700 1 --2- 172.21.15.119:0/3796928933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4a410c010 0x7fa4a419c7f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:22.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.292+0000 7fa4aa400700 1 -- 172.21.15.119:0/3796928933 >> 172.21.15.119:0/3796928933 conn(0x7fa4a4076000 msgr2=0x7fa4a4076880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:22.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.292+0000 7fa4aa400700 1 -- 172.21.15.119:0/3796928933 shutdown_connections 2024-05-27T09:41:22.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.292+0000 7fa4aa400700 1 -- 172.21.15.119:0/3796928933 wait complete. 2024-05-27T09:41:22.295 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 31 2024-05-27T09:41:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.436+0000 7f7564c00700 1 -- 172.21.15.119:0/87025474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7560106b80 msgr2=0x7f7560106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.436+0000 7f7564c00700 1 --2- 172.21.15.119:0/87025474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7560106b80 0x7f7560106ff0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f754c009910 tx=0x7f754c009c20 comp rx=0 tx=0).stop 2024-05-27T09:41:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.436+0000 7f7564c00700 1 -- 172.21.15.119:0/87025474 shutdown_connections 2024-05-27T09:41:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.436+0000 7f7564c00700 1 --2- 172.21.15.119:0/87025474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7560106b80 0x7f7560106ff0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.436+0000 7f7564c00700 1 --2- 172.21.15.119:0/87025474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f756010b770 0x7f756010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.436+0000 7f7564c00700 1 -- 172.21.15.119:0/87025474 >> 172.21.15.119:0/87025474 conn(0x7f7560076030 msgr2=0x7f7560078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.436+0000 7f7564c00700 1 -- 172.21.15.119:0/87025474 shutdown_connections 2024-05-27T09:41:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.436+0000 7f7564c00700 1 -- 172.21.15.119:0/87025474 wait complete. 2024-05-27T09:41:22.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.438+0000 7f7564c00700 1 Processor -- start 2024-05-27T09:41:22.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.438+0000 7f7564c00700 1 -- start start 2024-05-27T09:41:22.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.438+0000 7f7564c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7560106b80 0x7f756019c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:22.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.438+0000 7f7564c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f756010b770 0x7f756019c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:22.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.438+0000 7f7564c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756019cf90 con 0x7f756010b770 2024-05-27T09:41:22.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.438+0000 7f7564c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75601a0d20 con 0x7f7560106b80 2024-05-27T09:41:22.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.439+0000 7f755d400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f756010b770 0x7f756019c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:22.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.439+0000 7f755d400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f756010b770 0x7f756019c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43884/0 (socket says 172.21.15.119:43884) 2024-05-27T09:41:22.440 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.439+0000 7f755d400700 1 -- 172.21.15.119:0/3886375988 learned_addr learned my addr 172.21.15.119:0/3886375988 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:41:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.439+0000 7f755de00700 1 --2- 172.21.15.119:0/3886375988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7560106b80 0x7f756019c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.439+0000 7f755d400700 1 -- 172.21.15.119:0/3886375988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7560106b80 msgr2=0x7f756019c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.439+0000 7f755d400700 1 --2- 172.21.15.119:0/3886375988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7560106b80 0x7f756019c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.439+0000 7f755d400700 1 -- 172.21.15.119:0/3886375988 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c0095f0 con 0x7f756010b770 2024-05-27T09:41:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.440+0000 7f755d400700 1 --2- 172.21.15.119:0/3886375988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f756010b770 0x7f756019c8b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f754c006010 tx=0x7f754c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.440+0000 7f7556a00700 1 -- 172.21.15.119:0/3886375988 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f754c01d070 con 0x7f756010b770 2024-05-27T09:41:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.440+0000 7f7556a00700 1 -- 172.21.15.119:0/3886375988 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f754c004fb0 con 0x7f756010b770 2024-05-27T09:41:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.440+0000 7f7564c00700 1 -- 172.21.15.119:0/3886375988 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75601a0fa0 con 0x7f756010b770 2024-05-27T09:41:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.440+0000 7f7556a00700 1 -- 172.21.15.119:0/3886375988 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f754c022ea0 con 0x7f756010b770 2024-05-27T09:41:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.440+0000 7f7564c00700 1 -- 172.21.15.119:0/3886375988 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75601a1510 con 0x7f756010b770 2024-05-27T09:41:22.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.441+0000 7f7564c00700 1 -- 172.21.15.119:0/3886375988 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f756010ed40 con 0x7f756010b770 2024-05-27T09:41:22.443 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.443+0000 7f7556a00700 1 -- 172.21.15.119:0/3886375988 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f754c022470 con 0x7f756010b770 2024-05-27T09:41:22.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.444+0000 7f7556a00700 1 --2- 172.21.15.119:0/3886375988 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7544078a40 0x7f754407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:22.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.444+0000 7f7556a00700 1 -- 172.21.15.119:0/3886375988 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f754c09c7a0 con 0x7f756010b770 2024-05-27T09:41:22.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.444+0000 7f755de00700 1 --2- 172.21.15.119:0/3886375988 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7544078a40 0x7f754407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:22.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.445+0000 7f755de00700 1 --2- 172.21.15.119:0/3886375988 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7544078a40 0x7f754407aef0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7550006fd0 tx=0x7f7550008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:22.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.448+0000 7f7556a00700 1 -- 172.21.15.119:0/3886375988 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f754c0645a0 con 0x7f756010b770 2024-05-27T09:41:22.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.687+0000 7f7564c00700 1 -- 172.21.15.119:0/3886375988 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f75601a18c0 con 0x7f7544078a40 2024-05-27T09:41:22.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.690+0000 7f7556a00700 1 -- 172.21.15.119:0/3886375988 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f75601a18c0 con 0x7f7544078a40 2024-05-27T09:41:22.690 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:41:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:41:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:41:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-27T09:41:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [ 2024-05-27T09:41:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: "crash", 2024-05-27T09:41:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: "mon", 2024-05-27T09:41:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: "osd", 2024-05-27T09:41:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr" 2024-05-27T09:41:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-05-27T09:41:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "15/23 daemons upgraded", 2024-05-27T09:41:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "Currently upgrading mds daemons", 2024-05-27T09:41:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:41:22.691 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:41:22.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.692+0000 7f7564c00700 1 -- 172.21.15.119:0/3886375988 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7544078a40 msgr2=0x7f754407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:22.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.693+0000 7f7564c00700 1 --2- 172.21.15.119:0/3886375988 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7544078a40 0x7f754407aef0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7550006fd0 tx=0x7f7550008040 comp rx=0 tx=0).stop 2024-05-27T09:41:22.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.693+0000 7f7564c00700 1 -- 172.21.15.119:0/3886375988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f756010b770 msgr2=0x7f756019c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:22.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.693+0000 7f7564c00700 1 --2- 172.21.15.119:0/3886375988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f756010b770 0x7f756019c8b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f754c006010 tx=0x7f754c00be20 comp rx=0 tx=0).stop 2024-05-27T09:41:22.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.693+0000 7f7564c00700 1 -- 172.21.15.119:0/3886375988 shutdown_connections 2024-05-27T09:41:22.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.693+0000 7f7564c00700 1 --2- 172.21.15.119:0/3886375988 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7544078a40 0x7f754407aef0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:22.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.693+0000 7f7564c00700 1 --2- 172.21.15.119:0/3886375988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7560106b80 0x7f756019c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:22.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.693+0000 7f7564c00700 1 --2- 172.21.15.119:0/3886375988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f756010b770 0x7f756019c8b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:22.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.693+0000 7f7564c00700 1 -- 172.21.15.119:0/3886375988 >> 172.21.15.119:0/3886375988 conn(0x7f7560076030 msgr2=0x7f7560076b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:22.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.693+0000 7f7564c00700 1 -- 172.21.15.119:0/3886375988 shutdown_connections 2024-05-27T09:41:22.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.693+0000 7f7564c00700 1 -- 172.21.15.119:0/3886375988 wait complete. 2024-05-27T09:41:22.836 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.836+0000 7f8556e00700 1 -- 172.21.15.119:0/1176603999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8550102d50 msgr2=0x7f8550106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:22.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.836+0000 7f8556e00700 1 --2- 172.21.15.119:0/1176603999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8550102d50 0x7f8550106d20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f853c009910 tx=0x7f853c009c20 comp rx=0 tx=0).stop 2024-05-27T09:41:22.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.836+0000 7f8556e00700 1 -- 172.21.15.119:0/1176603999 shutdown_connections 2024-05-27T09:41:22.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.836+0000 7f8556e00700 1 --2- 172.21.15.119:0/1176603999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8550102d50 0x7f8550106d20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:22.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.836+0000 7f8556e00700 1 --2- 172.21.15.119:0/1176603999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85501023b0 0x7f8550102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:22.837 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.836+0000 7f8556e00700 1 -- 172.21.15.119:0/1176603999 >> 172.21.15.119:0/1176603999 conn(0x7f85500fdcb0 msgr2=0x7f85501000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:22.837 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:22 smithi119 ceph-mon[88269]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:41:22.837 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:22 smithi119 ceph-mon[88269]: pgmap v246: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:22.837 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:22 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/2026940115' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:22.838 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:22 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/3796928933' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:41:22.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.836+0000 7f8556e00700 1 -- 172.21.15.119:0/1176603999 shutdown_connections 2024-05-27T09:41:22.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.836+0000 7f8556e00700 1 -- 172.21.15.119:0/1176603999 wait complete. 2024-05-27T09:41:22.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.837+0000 7f8556e00700 1 Processor -- start 2024-05-27T09:41:22.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.838+0000 7f8556e00700 1 -- start start 2024-05-27T09:41:22.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.838+0000 7f8556e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85501023b0 0x7f8550197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:22.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.838+0000 7f8556e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8550102d50 0x7f85501982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:22.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.838+0000 7f8556e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8550198980 con 0x7f85501023b0 2024-05-27T09:41:22.839 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.838+0000 7f8556e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855019c710 con 0x7f8550102d50 2024-05-27T09:41:22.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.838+0000 7f8555a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85501023b0 0x7f8550197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:22.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.839+0000 7f8555a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85501023b0 0x7f8550197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43892/0 (socket says 172.21.15.119:43892) 2024-05-27T09:41:22.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.839+0000 7f8555a00700 1 -- 172.21.15.119:0/438744512 learned_addr learned my addr 172.21.15.119:0/438744512 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:41:22.840 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.839+0000 7f8555000700 1 --2- 172.21.15.119:0/438744512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8550102d50 0x7f85501982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:22.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.839+0000 7f8555a00700 1 -- 172.21.15.119:0/438744512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8550102d50 msgr2=0x7f85501982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:22.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.839+0000 7f8555a00700 1 --2- 172.21.15.119:0/438744512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8550102d50 0x7f85501982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:22.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.839+0000 7f8555a00700 1 -- 172.21.15.119:0/438744512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853c0095f0 con 0x7f85501023b0 2024-05-27T09:41:22.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.839+0000 7f8555a00700 1 --2- 172.21.15.119:0/438744512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85501023b0 0x7f8550197d60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f854400c8a0 tx=0x7f854400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:22.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.840+0000 7f854e000700 1 -- 172.21.15.119:0/438744512 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f854400cea0 con 0x7f85501023b0 2024-05-27T09:41:22.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.840+0000 7f854e000700 1 -- 172.21.15.119:0/438744512 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8544004830 con 0x7f85501023b0 2024-05-27T09:41:22.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.840+0000 7f8556e00700 1 -- 172.21.15.119:0/438744512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855019c9f0 con 0x7f85501023b0 2024-05-27T09:41:22.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.840+0000 7f854e000700 1 -- 172.21.15.119:0/438744512 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85440055b0 con 0x7f85501023b0 2024-05-27T09:41:22.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.840+0000 7f8556e00700 1 -- 172.21.15.119:0/438744512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855019cf40 con 0x7f85501023b0 2024-05-27T09:41:22.841 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.841+0000 7f8556e00700 1 -- 172.21.15.119:0/438744512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f855004f2a0 con 0x7f85501023b0 2024-05-27T09:41:22.843 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.843+0000 7f854e000700 1 -- 172.21.15.119:0/438744512 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f85440049a0 con 0x7f85501023b0 2024-05-27T09:41:22.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.844+0000 7f854e000700 1 --2- 172.21.15.119:0/438744512 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8538078a40 0x7f853807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:22.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.844+0000 7f854e000700 1 -- 172.21.15.119:0/438744512 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f854409a9f0 con 0x7f85501023b0 2024-05-27T09:41:22.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.844+0000 7f8555000700 1 --2- 172.21.15.119:0/438744512 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8538078a40 0x7f853807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:22.845 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.845+0000 7f8555000700 1 --2- 172.21.15.119:0/438744512 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8538078a40 0x7f853807aef0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8550199380 tx=0x7f853c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:22.850 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:22.849+0000 7f854e000700 1 -- 172.21.15.119:0/438744512 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f854409ce60 con 0x7f85501023b0 2024-05-27T09:41:22.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:22 smithi138 ceph-mon[70004]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:41:22.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:22 smithi138 ceph-mon[70004]: pgmap v246: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:22.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:22 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/2026940115' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:22.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:22 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/3796928933' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:41:23.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:23.157+0000 7f8556e00700 1 -- 172.21.15.119:0/438744512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f85501990c0 con 0x7f85501023b0 2024-05-27T09:41:23.158 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:23.158+0000 7f854e000700 1 -- 172.21.15.119:0/438744512 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f85440625c0 con 0x7f85501023b0 2024-05-27T09:41:23.158 INFO:teuthology.orchestra.run.smithi119.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-05-27T09:41:23.158 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-27T09:41:23.159 INFO:teuthology.orchestra.run.smithi119.stdout: fs cephfs is degraded 2024-05-27T09:41:23.159 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:41:23.159 INFO:teuthology.orchestra.run.smithi119.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:41:23.159 INFO:teuthology.orchestra.run.smithi119.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-05-27T09:41:23.159 INFO:teuthology.orchestra.run.smithi119.stdout: fs cephfs is offline because no MDS is active for it. 2024-05-27T09:41:23.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:23.160+0000 7f8556e00700 1 -- 172.21.15.119:0/438744512 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8538078a40 msgr2=0x7f853807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:23.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:23.160+0000 7f8556e00700 1 --2- 172.21.15.119:0/438744512 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8538078a40 0x7f853807aef0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8550199380 tx=0x7f853c01a040 comp rx=0 tx=0).stop 2024-05-27T09:41:23.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:23.161+0000 7f8556e00700 1 -- 172.21.15.119:0/438744512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85501023b0 msgr2=0x7f8550197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:23.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:23.161+0000 7f8556e00700 1 --2- 172.21.15.119:0/438744512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85501023b0 0x7f8550197d60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f854400c8a0 tx=0x7f854400cc60 comp rx=0 tx=0).stop 2024-05-27T09:41:23.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:23.161+0000 7f8556e00700 1 -- 172.21.15.119:0/438744512 shutdown_connections 2024-05-27T09:41:23.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:23.161+0000 7f8556e00700 1 --2- 172.21.15.119:0/438744512 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8538078a40 0x7f853807aef0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:23.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:23.161+0000 7f8556e00700 1 --2- 172.21.15.119:0/438744512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85501023b0 0x7f8550197d60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:23.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:23.161+0000 7f8556e00700 1 --2- 172.21.15.119:0/438744512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8550102d50 0x7f85501982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:23.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:23.161+0000 7f8556e00700 1 -- 172.21.15.119:0/438744512 >> 172.21.15.119:0/438744512 conn(0x7f85500fdcb0 msgr2=0x7f85500ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:23.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:23.161+0000 7f8556e00700 1 -- 172.21.15.119:0/438744512 shutdown_connections 2024-05-27T09:41:23.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:23.161+0000 7f8556e00700 1 -- 172.21.15.119:0/438744512 wait complete. 2024-05-27T09:41:23.863 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:23 smithi138 ceph-mon[70004]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:41:23.863 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:23 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/438744512' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:41:23.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:23 smithi119 ceph-mon[88269]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:41:23.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:23 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/438744512' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:41:25.113 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:24 smithi138 ceph-mon[70004]: pgmap v247: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:25.113 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:24 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:25.113 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:24 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:25.114 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:24 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:41:25.114 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:24 smithi138 ceph-mon[70004]: mds.? [v2:172.21.15.138:6826/3198147602,v1:172.21.15.138:6827/3198147602] up:boot 2024-05-27T09:41:25.114 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:24 smithi138 ceph-mon[70004]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-27T09:41:25.114 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:24 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.alszpk"}]: dispatch 2024-05-27T09:41:25.221 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:24 smithi119 ceph-mon[88269]: pgmap v247: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:25.221 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:24 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:25.221 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:24 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:25.221 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:24 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:41:25.221 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:24 smithi119 ceph-mon[88269]: mds.? [v2:172.21.15.138:6826/3198147602,v1:172.21.15.138:6827/3198147602] up:boot 2024-05-27T09:41:25.221 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:24 smithi119 ceph-mon[88269]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-05-27T09:41:25.221 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:24 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.alszpk"}]: dispatch 2024-05-27T09:41:27.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:26 smithi138 ceph-mon[70004]: pgmap v248: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:27.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:26 smithi119 ceph-mon[88269]: pgmap v248: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:28.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:27 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:41:28.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:27 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:41:29.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:28 smithi138 ceph-mon[70004]: pgmap v249: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:29.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:28 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:29.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:28 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:29.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:28 smithi119 ceph-mon[88269]: pgmap v249: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:29.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:29.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:28 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:31.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:30 smithi138 ceph-mon[70004]: pgmap v250: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:31.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:30 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:31.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:30 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:31.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:30 smithi119 ceph-mon[88269]: pgmap v250: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:31.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:30 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:31.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:30 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:33.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:32 smithi138 ceph-mon[70004]: pgmap v251: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:33.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:32 smithi119 ceph-mon[88269]: pgmap v251: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:35.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:34 smithi138 ceph-mon[70004]: pgmap v252: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:35.220 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:34 smithi119 ceph-mon[88269]: pgmap v252: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:37.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:36 smithi119 ceph-mon[88269]: pgmap v253: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:37.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:36 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:37.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:36 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:37.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:36 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:41:37.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:36 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:41:37.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:36 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:37.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:36 smithi138 ceph-mon[70004]: pgmap v253: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:37.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:36 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:37.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:36 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:37.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:36 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:41:37.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:36 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:41:37.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:36 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:38.216 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:41:38.216 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:38.216 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:38.216 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:38.216 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:38.216 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:38.216 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: Upgrade: Setting container_image for all mds 2024-05-27T09:41:38.216 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:38.216 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi119.nghcey"}]: dispatch 2024-05-27T09:41:38.216 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi119.nghcey"}]': finished 2024-05-27T09:41:38.217 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi119.ppmzzk"}]: dispatch 2024-05-27T09:41:38.217 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi119.ppmzzk"}]': finished 2024-05-27T09:41:38.217 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.alszpk"}]: dispatch 2024-05-27T09:41:38.217 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.alszpk"}]': finished 2024-05-27T09:41:38.217 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.ndfkhd"}]: dispatch 2024-05-27T09:41:38.217 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.ndfkhd"}]': finished 2024-05-27T09:41:38.217 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: Upgrade: Setting filesystem cephfs Joinable 2024-05-27T09:41:38.217 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-05-27T09:41:38.217 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: Standby daemon mds.cephfs.smithi119.nghcey assigned to filesystem cephfs as rank 0 2024-05-27T09:41:38.217 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: Standby daemon mds.cephfs.smithi119.ppmzzk assigned to filesystem cephfs as rank 1 2024-05-27T09:41:38.217 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-27T09:41:38.217 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-05-27T09:41:38.217 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: fsmap cephfs:2/2 {0=cephfs.smithi119.nghcey=up:replay,1=cephfs.smithi119.ppmzzk=up:replay} 2 up:standby 2024-05-27T09:41:38.218 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:38.218 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:38.218 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:38.218 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:38.218 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:37 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:38.218 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:41:38.218 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:38.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:38.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:38.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:38.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:38.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: Upgrade: Setting container_image for all mds 2024-05-27T09:41:38.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:38.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi119.nghcey"}]: dispatch 2024-05-27T09:41:38.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi119.nghcey"}]': finished 2024-05-27T09:41:38.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi119.ppmzzk"}]: dispatch 2024-05-27T09:41:38.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi119.ppmzzk"}]': finished 2024-05-27T09:41:38.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.alszpk"}]: dispatch 2024-05-27T09:41:38.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.alszpk"}]': finished 2024-05-27T09:41:38.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.ndfkhd"}]: dispatch 2024-05-27T09:41:38.219 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.ndfkhd"}]': finished 2024-05-27T09:41:38.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: Upgrade: Setting filesystem cephfs Joinable 2024-05-27T09:41:38.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-05-27T09:41:38.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: Standby daemon mds.cephfs.smithi119.nghcey assigned to filesystem cephfs as rank 0 2024-05-27T09:41:38.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: Standby daemon mds.cephfs.smithi119.ppmzzk assigned to filesystem cephfs as rank 1 2024-05-27T09:41:38.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-27T09:41:38.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-05-27T09:41:38.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: fsmap cephfs:2/2 {0=cephfs.smithi119.nghcey=up:replay,1=cephfs.smithi119.ppmzzk=up:replay} 2 up:standby 2024-05-27T09:41:38.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:38.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:38.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:38.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:38.220 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:37 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:39.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:38 smithi119 ceph-mon[88269]: Upgrade: Setting container_image for all rgw 2024-05-27T09:41:39.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:38 smithi119 ceph-mon[88269]: Upgrade: Setting container_image for all rbd-mirror 2024-05-27T09:41:39.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:38 smithi119 ceph-mon[88269]: pgmap v254: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:39.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:38 smithi138 ceph-mon[70004]: Upgrade: Setting container_image for all rgw 2024-05-27T09:41:39.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:38 smithi138 ceph-mon[70004]: Upgrade: Setting container_image for all rbd-mirror 2024-05-27T09:41:39.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:38 smithi138 ceph-mon[70004]: pgmap v254: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:41:41.184 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:40 smithi119 ceph-mon[88269]: pgmap v255: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s rd, 1 op/s 2024-05-27T09:41:41.184 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:40 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:41.184 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:40 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi119", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-27T09:41:41.184 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:40 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:41:41.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:40 smithi138 ceph-mon[70004]: pgmap v255: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s rd, 1 op/s 2024-05-27T09:41:41.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:40 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:41.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:40 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi119", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-27T09:41:41.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:40 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:41:42.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:41 smithi119 ceph-mon[88269]: Upgrade: Updating ceph-exporter.smithi119 (1/2) 2024-05-27T09:41:42.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:41 smithi119 ceph-mon[88269]: Deploying daemon ceph-exporter.smithi119 on smithi119 2024-05-27T09:41:42.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:41 smithi138 ceph-mon[70004]: Upgrade: Updating ceph-exporter.smithi119 (1/2) 2024-05-27T09:41:42.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:41 smithi138 ceph-mon[70004]: Deploying daemon ceph-exporter.smithi119 on smithi119 2024-05-27T09:41:43.236 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:42 smithi119 ceph-mon[88269]: pgmap v256: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 7.3 MiB/s rd, 3 op/s 2024-05-27T09:41:43.236 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:42 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:43.236 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:42 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:41:43.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:42 smithi138 ceph-mon[70004]: pgmap v256: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 7.3 MiB/s rd, 3 op/s 2024-05-27T09:41:43.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:42 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:43.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:42 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:41:45.198 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:44 smithi138 ceph-mon[70004]: pgmap v257: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 6 op/s 2024-05-27T09:41:45.220 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:44 smithi119 ceph-mon[88269]: pgmap v257: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 6 op/s 2024-05-27T09:41:47.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:46 smithi119 ceph-mon[88269]: pgmap v258: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-05-27T09:41:47.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:46 smithi119 ceph-mon[88269]: mds.? [v2:172.21.15.119:6828/2430623742,v1:172.21.15.119:6829/2430623742] up:resolve 2024-05-27T09:41:47.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:46 smithi119 ceph-mon[88269]: fsmap cephfs:2/2 {0=cephfs.smithi119.nghcey=up:replay,1=cephfs.smithi119.ppmzzk=up:resolve} 2 up:standby 2024-05-27T09:41:47.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:46 smithi138 ceph-mon[70004]: pgmap v258: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-05-27T09:41:47.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:46 smithi138 ceph-mon[70004]: mds.? [v2:172.21.15.119:6828/2430623742,v1:172.21.15.119:6829/2430623742] up:resolve 2024-05-27T09:41:47.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:46 smithi138 ceph-mon[70004]: fsmap cephfs:2/2 {0=cephfs.smithi119.nghcey=up:replay,1=cephfs.smithi119.ppmzzk=up:resolve} 2 up:standby 2024-05-27T09:41:48.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:47 smithi119 ceph-mon[88269]: mds.? [v2:172.21.15.119:6826/3357235708,v1:172.21.15.119:6827/3357235708] up:resolve 2024-05-27T09:41:48.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:47 smithi119 ceph-mon[88269]: fsmap cephfs:2/2 {0=cephfs.smithi119.nghcey=up:resolve,1=cephfs.smithi119.ppmzzk=up:resolve} 2 up:standby 2024-05-27T09:41:48.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:47 smithi138 ceph-mon[70004]: mds.? [v2:172.21.15.119:6826/3357235708,v1:172.21.15.119:6827/3357235708] up:resolve 2024-05-27T09:41:48.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:47 smithi138 ceph-mon[70004]: fsmap cephfs:2/2 {0=cephfs.smithi119.nghcey=up:resolve,1=cephfs.smithi119.ppmzzk=up:resolve} 2 up:standby 2024-05-27T09:41:49.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:48 smithi119 ceph-mon[88269]: pgmap v259: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 8 op/s 2024-05-27T09:41:49.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:48 smithi119 ceph-mon[88269]: mds.? [v2:172.21.15.119:6826/3357235708,v1:172.21.15.119:6827/3357235708] up:reconnect 2024-05-27T09:41:49.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:48 smithi119 ceph-mon[88269]: mds.? [v2:172.21.15.119:6828/2430623742,v1:172.21.15.119:6829/2430623742] up:reconnect 2024-05-27T09:41:49.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:48 smithi119 ceph-mon[88269]: fsmap cephfs:2/2 {0=cephfs.smithi119.nghcey=up:reconnect,1=cephfs.smithi119.ppmzzk=up:reconnect} 2 up:standby 2024-05-27T09:41:49.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:48 smithi119 ceph-mon[88269]: reconnect by client.24275 v1:192.168.0.1:0/2208627419 after 0.0179996 2024-05-27T09:41:49.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:48 smithi119 ceph-mon[88269]: reconnect by client.24275 v1:192.168.0.1:0/2208627419 after 0.0199995 2024-05-27T09:41:49.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:48 smithi119 ceph-mon[88269]: reconnect by client.24273 v1:172.21.15.119:0/3745732299 after 0.0189995 2024-05-27T09:41:49.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:48 smithi119 ceph-mon[88269]: reconnect by client.24273 v1:172.21.15.119:0/3745732299 after 0.0199995 2024-05-27T09:41:49.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:48 smithi138 ceph-mon[70004]: pgmap v259: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 8 op/s 2024-05-27T09:41:49.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:48 smithi138 ceph-mon[70004]: mds.? [v2:172.21.15.119:6826/3357235708,v1:172.21.15.119:6827/3357235708] up:reconnect 2024-05-27T09:41:49.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:48 smithi138 ceph-mon[70004]: mds.? [v2:172.21.15.119:6828/2430623742,v1:172.21.15.119:6829/2430623742] up:reconnect 2024-05-27T09:41:49.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:48 smithi138 ceph-mon[70004]: fsmap cephfs:2/2 {0=cephfs.smithi119.nghcey=up:reconnect,1=cephfs.smithi119.ppmzzk=up:reconnect} 2 up:standby 2024-05-27T09:41:49.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:48 smithi138 ceph-mon[70004]: reconnect by client.24275 v1:192.168.0.1:0/2208627419 after 0.0179996 2024-05-27T09:41:49.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:48 smithi138 ceph-mon[70004]: reconnect by client.24275 v1:192.168.0.1:0/2208627419 after 0.0199995 2024-05-27T09:41:49.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:48 smithi138 ceph-mon[70004]: reconnect by client.24273 v1:172.21.15.119:0/3745732299 after 0.0189995 2024-05-27T09:41:49.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:48 smithi138 ceph-mon[70004]: reconnect by client.24273 v1:172.21.15.119:0/3745732299 after 0.0199995 2024-05-27T09:41:50.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:49 smithi119 ceph-mon[88269]: mds.? [v2:172.21.15.119:6826/3357235708,v1:172.21.15.119:6827/3357235708] up:rejoin 2024-05-27T09:41:50.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:49 smithi119 ceph-mon[88269]: mds.? [v2:172.21.15.119:6828/2430623742,v1:172.21.15.119:6829/2430623742] up:rejoin 2024-05-27T09:41:50.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:49 smithi119 ceph-mon[88269]: fsmap cephfs:2/2 {0=cephfs.smithi119.nghcey=up:rejoin,1=cephfs.smithi119.ppmzzk=up:rejoin} 2 up:standby 2024-05-27T09:41:50.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:49 smithi119 ceph-mon[88269]: daemon mds.cephfs.smithi119.nghcey is now active in filesystem cephfs as rank 0 2024-05-27T09:41:50.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:49 smithi119 ceph-mon[88269]: daemon mds.cephfs.smithi119.ppmzzk is now active in filesystem cephfs as rank 1 2024-05-27T09:41:50.364 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:49 smithi138 ceph-mon[70004]: mds.? [v2:172.21.15.119:6826/3357235708,v1:172.21.15.119:6827/3357235708] up:rejoin 2024-05-27T09:41:50.364 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:49 smithi138 ceph-mon[70004]: mds.? [v2:172.21.15.119:6828/2430623742,v1:172.21.15.119:6829/2430623742] up:rejoin 2024-05-27T09:41:50.364 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:49 smithi138 ceph-mon[70004]: fsmap cephfs:2/2 {0=cephfs.smithi119.nghcey=up:rejoin,1=cephfs.smithi119.ppmzzk=up:rejoin} 2 up:standby 2024-05-27T09:41:50.365 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:49 smithi138 ceph-mon[70004]: daemon mds.cephfs.smithi119.nghcey is now active in filesystem cephfs as rank 0 2024-05-27T09:41:50.365 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:49 smithi138 ceph-mon[70004]: daemon mds.cephfs.smithi119.ppmzzk is now active in filesystem cephfs as rank 1 2024-05-27T09:41:51.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:50 smithi119 ceph-mon[88269]: pgmap v260: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 8 op/s 2024-05-27T09:41:51.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:50 smithi119 ceph-mon[88269]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-27T09:41:51.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:50 smithi119 ceph-mon[88269]: mds.? [v2:172.21.15.119:6826/3357235708,v1:172.21.15.119:6827/3357235708] up:active 2024-05-27T09:41:51.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:50 smithi119 ceph-mon[88269]: mds.? [v2:172.21.15.119:6828/2430623742,v1:172.21.15.119:6829/2430623742] up:active 2024-05-27T09:41:51.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:50 smithi119 ceph-mon[88269]: fsmap cephfs:2 {0=cephfs.smithi119.nghcey=up:active,1=cephfs.smithi119.ppmzzk=up:active} 2 up:standby 2024-05-27T09:41:51.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:50 smithi138 ceph-mon[70004]: pgmap v260: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 8 op/s 2024-05-27T09:41:51.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:50 smithi138 ceph-mon[70004]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-27T09:41:51.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:50 smithi138 ceph-mon[70004]: mds.? [v2:172.21.15.119:6826/3357235708,v1:172.21.15.119:6827/3357235708] up:active 2024-05-27T09:41:51.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:50 smithi138 ceph-mon[70004]: mds.? [v2:172.21.15.119:6828/2430623742,v1:172.21.15.119:6829/2430623742] up:active 2024-05-27T09:41:51.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:50 smithi138 ceph-mon[70004]: fsmap cephfs:2 {0=cephfs.smithi119.nghcey=up:active,1=cephfs.smithi119.ppmzzk=up:active} 2 up:standby 2024-05-27T09:41:53.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:52 smithi119 ceph-mon[88269]: pgmap v261: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 1023 B/s wr, 9 op/s 2024-05-27T09:41:53.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:52 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:53.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:52 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:53.298 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:52 smithi138 ceph-mon[70004]: pgmap v261: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 1023 B/s wr, 9 op/s 2024-05-27T09:41:53.299 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:52 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:53.299 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:52 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:53.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.346+0000 7f2dada00700 1 -- 172.21.15.119:0/3918066372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2da8073090 msgr2=0x7f2da8073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:53.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.346+0000 7f2dada00700 1 --2- 172.21.15.119:0/3918066372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2da8073090 0x7f2da8073460 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009940 tx=0x7f2d98009c50 comp rx=0 tx=0).stop 2024-05-27T09:41:53.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.346+0000 7f2dada00700 1 -- 172.21.15.119:0/3918066372 shutdown_connections 2024-05-27T09:41:53.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.346+0000 7f2dada00700 1 --2- 172.21.15.119:0/3918066372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da80739a0 0x7f2da810c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:53.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.346+0000 7f2dada00700 1 --2- 172.21.15.119:0/3918066372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2da8073090 0x7f2da8073460 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:53.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.346+0000 7f2dada00700 1 -- 172.21.15.119:0/3918066372 >> 172.21.15.119:0/3918066372 conn(0x7f2da80fbb10 msgr2=0x7f2da80fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:53.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.346+0000 7f2dada00700 1 -- 172.21.15.119:0/3918066372 shutdown_connections 2024-05-27T09:41:53.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.346+0000 7f2dada00700 1 -- 172.21.15.119:0/3918066372 wait complete. 2024-05-27T09:41:53.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.347+0000 7f2dada00700 1 Processor -- start 2024-05-27T09:41:53.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.347+0000 7f2dada00700 1 -- start start 2024-05-27T09:41:53.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.347+0000 7f2dada00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2da8073090 0x7f2da8197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:53.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.347+0000 7f2dada00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da80739a0 0x7f2da8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:53.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.347+0000 7f2dada00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da8198b30 con 0x7f2da8073090 2024-05-27T09:41:53.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.347+0000 7f2dada00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da819c8c0 con 0x7f2da80739a0 2024-05-27T09:41:53.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.348+0000 7f2da7e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2da8073090 0x7f2da8197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:53.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.348+0000 7f2da7e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2da8073090 0x7f2da8197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35422/0 (socket says 172.21.15.119:35422) 2024-05-27T09:41:53.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.348+0000 7f2da7e00700 1 -- 172.21.15.119:0/2806545869 learned_addr learned my addr 172.21.15.119:0/2806545869 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:41:53.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.348+0000 7f2d9f400700 1 --2- 172.21.15.119:0/2806545869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da80739a0 0x7f2da8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:53.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.348+0000 7f2da7e00700 1 -- 172.21.15.119:0/2806545869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da80739a0 msgr2=0x7f2da8198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:53.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.348+0000 7f2da7e00700 1 --2- 172.21.15.119:0/2806545869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da80739a0 0x7f2da8198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:53.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.348+0000 7f2da7e00700 1 -- 172.21.15.119:0/2806545869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d980095f0 con 0x7f2da8073090 2024-05-27T09:41:53.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.349+0000 7f2da7e00700 1 --2- 172.21.15.119:0/2806545869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2da8073090 0x7f2da8197f10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2d98005620 tx=0x7f2d98005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:53.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.349+0000 7f2da5600700 1 -- 172.21.15.119:0/2806545869 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d9801d070 con 0x7f2da8073090 2024-05-27T09:41:53.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.349+0000 7f2da5600700 1 -- 172.21.15.119:0/2806545869 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d9800f460 con 0x7f2da8073090 2024-05-27T09:41:53.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.349+0000 7f2da5600700 1 -- 172.21.15.119:0/2806545869 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d98021640 con 0x7f2da8073090 2024-05-27T09:41:53.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.349+0000 7f2dada00700 1 -- 172.21.15.119:0/2806545869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da819cb40 con 0x7f2da8073090 2024-05-27T09:41:53.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.349+0000 7f2dada00700 1 -- 172.21.15.119:0/2806545869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da819d030 con 0x7f2da8073090 2024-05-27T09:41:53.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.350+0000 7f2dada00700 1 -- 172.21.15.119:0/2806545869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da8109a30 con 0x7f2da8073090 2024-05-27T09:41:53.353 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.352+0000 7f2da5600700 1 -- 172.21.15.119:0/2806545869 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f2d9800fa40 con 0x7f2da8073090 2024-05-27T09:41:53.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.353+0000 7f2da5600700 1 --2- 172.21.15.119:0/2806545869 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2d8c078a40 0x7f2d8c07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:53.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.353+0000 7f2da5600700 1 -- 172.21.15.119:0/2806545869 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2d9809c4a0 con 0x7f2da8073090 2024-05-27T09:41:53.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.353+0000 7f2d9f400700 1 --2- 172.21.15.119:0/2806545869 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2d8c078a40 0x7f2d8c07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:53.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.354+0000 7f2d9f400700 1 --2- 172.21.15.119:0/2806545869 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2d8c078a40 0x7f2d8c07aef0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2da8199530 tx=0x7f2d900094f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:53.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.357+0000 7f2da5600700 1 -- 172.21.15.119:0/2806545869 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f2d980640d0 con 0x7f2da8073090 2024-05-27T09:41:53.614 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.613+0000 7f2dada00700 1 -- 172.21.15.119:0/2806545869 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2da819d310 con 0x7f2d8c078a40 2024-05-27T09:41:53.618 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.617+0000 7f2da5600700 1 -- 172.21.15.119:0/2806545869 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f2da819d310 con 0x7f2d8c078a40 2024-05-27T09:41:53.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.620+0000 7f2dada00700 1 -- 172.21.15.119:0/2806545869 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2d8c078a40 msgr2=0x7f2d8c07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:53.622 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.620+0000 7f2dada00700 1 --2- 172.21.15.119:0/2806545869 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2d8c078a40 0x7f2d8c07aef0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2da8199530 tx=0x7f2d900094f0 comp rx=0 tx=0).stop 2024-05-27T09:41:53.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.620+0000 7f2dada00700 1 -- 172.21.15.119:0/2806545869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2da8073090 msgr2=0x7f2da8197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:53.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.620+0000 7f2dada00700 1 --2- 172.21.15.119:0/2806545869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2da8073090 0x7f2da8197f10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2d98005620 tx=0x7f2d98005700 comp rx=0 tx=0).stop 2024-05-27T09:41:53.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.620+0000 7f2dada00700 1 -- 172.21.15.119:0/2806545869 shutdown_connections 2024-05-27T09:41:53.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.620+0000 7f2dada00700 1 --2- 172.21.15.119:0/2806545869 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2d8c078a40 0x7f2d8c07aef0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:53.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.620+0000 7f2dada00700 1 --2- 172.21.15.119:0/2806545869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2da8073090 0x7f2da8197f10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:53.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.620+0000 7f2dada00700 1 --2- 172.21.15.119:0/2806545869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da80739a0 0x7f2da8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:53.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.620+0000 7f2dada00700 1 -- 172.21.15.119:0/2806545869 >> 172.21.15.119:0/2806545869 conn(0x7f2da80fbb10 msgr2=0x7f2da8106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:53.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.621+0000 7f2dada00700 1 -- 172.21.15.119:0/2806545869 shutdown_connections 2024-05-27T09:41:53.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.621+0000 7f2dada00700 1 -- 172.21.15.119:0/2806545869 wait complete. 2024-05-27T09:41:53.638 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:41:53.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.772+0000 7f2986e00700 1 -- 172.21.15.119:0/1977127029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f298010b610 msgr2=0x7f298010b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:53.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.772+0000 7f2986e00700 1 --2- 172.21.15.119:0/1977127029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f298010b610 0x7f298010b9e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f297400b0a0 tx=0x7f297400b3b0 comp rx=0 tx=0).stop 2024-05-27T09:41:53.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.772+0000 7f2986e00700 1 -- 172.21.15.119:0/1977127029 shutdown_connections 2024-05-27T09:41:53.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.772+0000 7f2986e00700 1 --2- 172.21.15.119:0/1977127029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2980106b80 0x7f2980106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:53.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.772+0000 7f2986e00700 1 --2- 172.21.15.119:0/1977127029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f298010b610 0x7f298010b9e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:53.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.772+0000 7f2986e00700 1 -- 172.21.15.119:0/1977127029 >> 172.21.15.119:0/1977127029 conn(0x7f2980075df0 msgr2=0x7f2980078200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:53.774 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.772+0000 7f2986e00700 1 -- 172.21.15.119:0/1977127029 shutdown_connections 2024-05-27T09:41:53.775 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.772+0000 7f2986e00700 1 -- 172.21.15.119:0/1977127029 wait complete. 2024-05-27T09:41:53.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.773+0000 7f2986e00700 1 Processor -- start 2024-05-27T09:41:53.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.773+0000 7f2986e00700 1 -- start start 2024-05-27T09:41:53.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.774+0000 7f2986e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2980106b80 0x7f298019c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:53.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.774+0000 7f2986e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f298010b610 0x7f298019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:53.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.774+0000 7f2986e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298019cd50 con 0x7f2980106b80 2024-05-27T09:41:53.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.774+0000 7f2986e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29801a0ae0 con 0x7f298010b610 2024-05-27T09:41:53.776 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.774+0000 7f2985a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2980106b80 0x7f298019c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:53.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.774+0000 7f2985a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2980106b80 0x7f298019c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35440/0 (socket says 172.21.15.119:35440) 2024-05-27T09:41:53.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.774+0000 7f2985a00700 1 -- 172.21.15.119:0/407118440 learned_addr learned my addr 172.21.15.119:0/407118440 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:41:53.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.774+0000 7f2985000700 1 --2- 172.21.15.119:0/407118440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f298010b610 0x7f298019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:53.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.774+0000 7f2985a00700 1 -- 172.21.15.119:0/407118440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f298010b610 msgr2=0x7f298019c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:53.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.774+0000 7f2985a00700 1 --2- 172.21.15.119:0/407118440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f298010b610 0x7f298019c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:53.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.774+0000 7f2985a00700 1 -- 172.21.15.119:0/407118440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2974009d00 con 0x7f2980106b80 2024-05-27T09:41:53.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.774+0000 7f2985000700 1 --2- 172.21.15.119:0/407118440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f298010b610 0x7f298019c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:41:53.777 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.775+0000 7f2985a00700 1 --2- 172.21.15.119:0/407118440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2980106b80 0x7f298019c130 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f297400b070 tx=0x7f297400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:53.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.775+0000 7f297e000700 1 -- 172.21.15.119:0/407118440 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f297400e040 con 0x7f2980106b80 2024-05-27T09:41:53.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.775+0000 7f297e000700 1 -- 172.21.15.119:0/407118440 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2974003de0 con 0x7f2980106b80 2024-05-27T09:41:53.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.775+0000 7f2986e00700 1 -- 172.21.15.119:0/407118440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29801a0d60 con 0x7f2980106b80 2024-05-27T09:41:53.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.775+0000 7f297e000700 1 -- 172.21.15.119:0/407118440 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f297401d8e0 con 0x7f2980106b80 2024-05-27T09:41:53.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.775+0000 7f2986e00700 1 -- 172.21.15.119:0/407118440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29801a12d0 con 0x7f2980106b80 2024-05-27T09:41:53.778 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.776+0000 7f2986e00700 1 -- 172.21.15.119:0/407118440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f298004f2a0 con 0x7f2980106b80 2024-05-27T09:41:53.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.777+0000 7f297e000700 1 -- 172.21.15.119:0/407118440 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f2974004040 con 0x7f2980106b80 2024-05-27T09:41:53.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.778+0000 7f297e000700 1 --2- 172.21.15.119:0/407118440 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2968078a40 0x7f296807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:53.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.778+0000 7f297e000700 1 -- 172.21.15.119:0/407118440 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f297409c640 con 0x7f2980106b80 2024-05-27T09:41:53.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.779+0000 7f2985000700 1 --2- 172.21.15.119:0/407118440 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2968078a40 0x7f296807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:53.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.779+0000 7f2985000700 1 --2- 172.21.15.119:0/407118440 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2968078a40 0x7f296807aef0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f296c005fd0 tx=0x7f296c005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:53.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:53.783+0000 7f297e000700 1 -- 172.21.15.119:0/407118440 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f2974064170 con 0x7f2980106b80 2024-05-27T09:41:54.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.043+0000 7f2986e00700 1 -- 172.21.15.119:0/407118440 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f29801a1850 con 0x7f2968078a40 2024-05-27T09:41:54.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.046+0000 7f297e000700 1 -- 172.21.15.119:0/407118440 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f29801a1850 con 0x7f2968078a40 2024-05-27T09:41:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.049+0000 7f2986e00700 1 -- 172.21.15.119:0/407118440 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2968078a40 msgr2=0x7f296807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.049+0000 7f2986e00700 1 --2- 172.21.15.119:0/407118440 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2968078a40 0x7f296807aef0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f296c005fd0 tx=0x7f296c005ee0 comp rx=0 tx=0).stop 2024-05-27T09:41:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.049+0000 7f2986e00700 1 -- 172.21.15.119:0/407118440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2980106b80 msgr2=0x7f298019c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.049+0000 7f2986e00700 1 --2- 172.21.15.119:0/407118440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2980106b80 0x7f298019c130 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f297400b070 tx=0x7f297400cdd0 comp rx=0 tx=0).stop 2024-05-27T09:41:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.049+0000 7f2986e00700 1 -- 172.21.15.119:0/407118440 shutdown_connections 2024-05-27T09:41:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.049+0000 7f2986e00700 1 --2- 172.21.15.119:0/407118440 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2968078a40 0x7f296807aef0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.049+0000 7f2986e00700 1 --2- 172.21.15.119:0/407118440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2980106b80 0x7f298019c130 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.049+0000 7f2986e00700 1 --2- 172.21.15.119:0/407118440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f298010b610 0x7f298019c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.049+0000 7f2986e00700 1 -- 172.21.15.119:0/407118440 >> 172.21.15.119:0/407118440 conn(0x7f2980075df0 msgr2=0x7f29800776a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:54.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.049+0000 7f2986e00700 1 -- 172.21.15.119:0/407118440 shutdown_connections 2024-05-27T09:41:54.052 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.049+0000 7f2986e00700 1 -- 172.21.15.119:0/407118440 wait complete. 2024-05-27T09:41:54.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.207+0000 7f5b9b200700 1 -- 172.21.15.119:0/3122158883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b9c101150 msgr2=0x7f5b9c101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:54.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.207+0000 7f5b9b200700 1 --2- 172.21.15.119:0/3122158883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b9c101150 0x7f5b9c101520 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009910 tx=0x7f5b8c009c20 comp rx=0 tx=0).stop 2024-05-27T09:41:54.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.208+0000 7f5b9b200700 1 -- 172.21.15.119:0/3122158883 shutdown_connections 2024-05-27T09:41:54.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.208+0000 7f5b9b200700 1 --2- 172.21.15.119:0/3122158883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b9c101a60 0x7f5b9c109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:54.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.208+0000 7f5b9b200700 1 --2- 172.21.15.119:0/3122158883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b9c101150 0x7f5b9c101520 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:54.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.208+0000 7f5b9b200700 1 -- 172.21.15.119:0/3122158883 >> 172.21.15.119:0/3122158883 conn(0x7f5b9c0fa990 msgr2=0x7f5b9c0fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:54.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.208+0000 7f5b9b200700 1 -- 172.21.15.119:0/3122158883 shutdown_connections 2024-05-27T09:41:54.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.208+0000 7f5b9b200700 1 -- 172.21.15.119:0/3122158883 wait complete. 2024-05-27T09:41:54.210 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.209+0000 7f5b9b200700 1 Processor -- start 2024-05-27T09:41:54.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.209+0000 7f5b9b200700 1 -- start start 2024-05-27T09:41:54.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.209+0000 7f5b9b200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b9c101150 0x7f5b9c197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:54.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.209+0000 7f5b9b200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b9c101a60 0x7f5b9c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:54.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.209+0000 7f5b9b200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c198a70 con 0x7f5b9c101150 2024-05-27T09:41:54.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.209+0000 7f5b9b200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c19c800 con 0x7f5b9c101a60 2024-05-27T09:41:54.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.210+0000 7f5b99e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b9c101150 0x7f5b9c197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:54.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.210+0000 7f5b99e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b9c101150 0x7f5b9c197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35458/0 (socket says 172.21.15.119:35458) 2024-05-27T09:41:54.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.210+0000 7f5b99e00700 1 -- 172.21.15.119:0/857318888 learned_addr learned my addr 172.21.15.119:0/857318888 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:41:54.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.210+0000 7f5b93e00700 1 --2- 172.21.15.119:0/857318888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b9c101a60 0x7f5b9c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:54.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.210+0000 7f5b99e00700 1 -- 172.21.15.119:0/857318888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b9c101a60 msgr2=0x7f5b9c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:54.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.210+0000 7f5b99e00700 1 --2- 172.21.15.119:0/857318888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b9c101a60 0x7f5b9c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:54.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.210+0000 7f5b99e00700 1 -- 172.21.15.119:0/857318888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8c0095f0 con 0x7f5b9c101150 2024-05-27T09:41:54.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.210+0000 7f5b93e00700 1 --2- 172.21.15.119:0/857318888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b9c101a60 0x7f5b9c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:41:54.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.211+0000 7f5b99e00700 1 --2- 172.21.15.119:0/857318888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b9c101150 0x7f5b9c197e50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c006010 tx=0x7f5b8c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:54.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.211+0000 7f5b92a00700 1 -- 172.21.15.119:0/857318888 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b8c01d070 con 0x7f5b9c101150 2024-05-27T09:41:54.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.211+0000 7f5b92a00700 1 -- 172.21.15.119:0/857318888 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b8c004fb0 con 0x7f5b9c101150 2024-05-27T09:41:54.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.211+0000 7f5b9b200700 1 -- 172.21.15.119:0/857318888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b9c19ca80 con 0x7f5b9c101150 2024-05-27T09:41:54.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.211+0000 7f5b92a00700 1 -- 172.21.15.119:0/857318888 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b8c00f690 con 0x7f5b9c101150 2024-05-27T09:41:54.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.211+0000 7f5b9b200700 1 -- 172.21.15.119:0/857318888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b9c19cf70 con 0x7f5b9c101150 2024-05-27T09:41:54.214 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.213+0000 7f5b9b200700 1 -- 172.21.15.119:0/857318888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b9c109a70 con 0x7f5b9c101150 2024-05-27T09:41:54.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.214+0000 7f5b92a00700 1 -- 172.21.15.119:0/857318888 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f5b8c0228a0 con 0x7f5b9c101150 2024-05-27T09:41:54.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.214+0000 7f5b92a00700 1 --2- 172.21.15.119:0/857318888 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5b80078a40 0x7f5b8007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:54.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.215+0000 7f5b92a00700 1 -- 172.21.15.119:0/857318888 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5b8c09c690 con 0x7f5b9c101150 2024-05-27T09:41:54.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.215+0000 7f5b93e00700 1 --2- 172.21.15.119:0/857318888 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5b80078a40 0x7f5b8007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:54.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.216+0000 7f5b93e00700 1 --2- 172.21.15.119:0/857318888 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5b80078a40 0x7f5b8007aef0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c199470 tx=0x7f5b84008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:54.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.220+0000 7f5b92a00700 1 -- 172.21.15.119:0/857318888 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f5b8c0641c0 con 0x7f5b9c101150 2024-05-27T09:41:54.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.468+0000 7f5b9b200700 1 -- 172.21.15.119:0/857318888 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b9c02ccf0 con 0x7f5b80078a40 2024-05-27T09:41:54.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.481+0000 7f5b92a00700 1 -- 172.21.15.119:0/857318888 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f5b9c02ccf0 con 0x7f5b80078a40 2024-05-27T09:41:54.486 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:41:54.487 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (9m) 74s ago 23m 15.2M - 0.25.0 c8568f914cd2 6b90564dcef7 2024-05-27T09:41:54.487 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 starting - - - - 2024-05-27T09:41:54.487 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (22m) 25s ago 22m 9307k - 18.2.0 dc2bc1663786 1e97ed803334 2024-05-27T09:41:54.487 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (5m) 74s ago 23m 6669k - 19.0.0-3868-ga6507388 850a06a84d28 011d811ce7fc 2024-05-27T09:41:54.487 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (5m) 25s ago 21m 6665k - 19.0.0-3868-ga6507388 850a06a84d28 b546b3ed1afc 2024-05-27T09:41:54.487 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (8m) 74s ago 22m 74.2M - 9.4.12 2bacad6d85d8 b3abdab08db4 2024-05-27T09:41:54.487 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (106s) 74s ago 15m 79.5M - 19.0.0-3868-ga6507388 850a06a84d28 5a02732ff870 2024-05-27T09:41:54.487 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (82s) 74s ago 15m 78.4M - 19.0.0-3868-ga6507388 850a06a84d28 3faccd4c18e8 2024-05-27T09:41:54.487 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (31s) 25s ago 15m 78.4M - 19.0.0-3868-ga6507388 850a06a84d28 8da881d730a9 2024-05-27T09:41:54.487 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (55s) 25s ago 15m 79.4M - 19.0.0-3868-ga6507388 850a06a84d28 866712d7b973 2024-05-27T09:41:54.487 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (11m) 74s ago 24m 593M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:41:54.487 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (10m) 25s ago 21m 535M - 19.0.0-3868-ga6507388 850a06a84d28 2fc3a80696fa 2024-05-27T09:41:54.487 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (7m) 74s ago 24m 123M 2048M 19.0.0-3868-ga6507388 850a06a84d28 3e5c7a9ad372 2024-05-27T09:41:54.488 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (7m) 25s ago 21m 112M 2048M 19.0.0-3868-ga6507388 850a06a84d28 e4d471274cf1 2024-05-27T09:41:54.488 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (10m) 74s ago 23m 7814k - 1.5.0 0da6a335fe13 709c37815cff 2024-05-27T09:41:54.488 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (10m) 25s ago 21m 7876k - 1.5.0 0da6a335fe13 999047a1a5bf 2024-05-27T09:41:54.488 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (5m) 74s ago 20m 157M 1216M 19.0.0-3868-ga6507388 850a06a84d28 5a0e162d870e 2024-05-27T09:41:54.488 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (4m) 74s ago 20m 171M 1216M 19.0.0-3868-ga6507388 850a06a84d28 effcc7c53dd3 2024-05-27T09:41:54.488 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (4m) 74s ago 19m 149M 1216M 19.0.0-3868-ga6507388 850a06a84d28 f0bd34ca416f 2024-05-27T09:41:54.488 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (3m) 25s ago 18m 174M 2240M 19.0.0-3868-ga6507388 850a06a84d28 5e8d3e398c21 2024-05-27T09:41:54.488 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (2m) 25s ago 18m 163M 2240M 19.0.0-3868-ga6507388 850a06a84d28 250e443364de 2024-05-27T09:41:54.488 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (2m) 25s ago 17m 153M 2240M 19.0.0-3868-ga6507388 850a06a84d28 90d98c5b2f16 2024-05-27T09:41:54.488 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (9m) 74s ago 22m 72.5M - 2.43.0 a07b618ecd1d cba50454ed21 2024-05-27T09:41:54.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.485+0000 7f5b9b200700 1 -- 172.21.15.119:0/857318888 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5b80078a40 msgr2=0x7f5b8007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:54.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.485+0000 7f5b9b200700 1 --2- 172.21.15.119:0/857318888 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5b80078a40 0x7f5b8007aef0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c199470 tx=0x7f5b84008040 comp rx=0 tx=0).stop 2024-05-27T09:41:54.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.485+0000 7f5b9b200700 1 -- 172.21.15.119:0/857318888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b9c101150 msgr2=0x7f5b9c197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:54.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.485+0000 7f5b9b200700 1 --2- 172.21.15.119:0/857318888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b9c101150 0x7f5b9c197e50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c006010 tx=0x7f5b8c00be20 comp rx=0 tx=0).stop 2024-05-27T09:41:54.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.485+0000 7f5b9b200700 1 -- 172.21.15.119:0/857318888 shutdown_connections 2024-05-27T09:41:54.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.485+0000 7f5b9b200700 1 --2- 172.21.15.119:0/857318888 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f5b80078a40 0x7f5b8007aef0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:54.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.485+0000 7f5b9b200700 1 --2- 172.21.15.119:0/857318888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b9c101150 0x7f5b9c197e50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:54.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.485+0000 7f5b9b200700 1 --2- 172.21.15.119:0/857318888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b9c101a60 0x7f5b9c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:54.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.485+0000 7f5b9b200700 1 -- 172.21.15.119:0/857318888 >> 172.21.15.119:0/857318888 conn(0x7f5b9c0fa990 msgr2=0x7f5b9c0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:54.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.485+0000 7f5b9b200700 1 -- 172.21.15.119:0/857318888 shutdown_connections 2024-05-27T09:41:54.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.486+0000 7f5b9b200700 1 -- 172.21.15.119:0/857318888 wait complete. 2024-05-27T09:41:54.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.635+0000 7fc131e00700 1 -- 172.21.15.119:0/3513307249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc12c102fd0 msgr2=0x7fc12c106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:54.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.635+0000 7fc131e00700 1 --2- 172.21.15.119:0/3513307249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc12c102fd0 0x7fc12c106fa0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc118009910 tx=0x7fc118009c20 comp rx=0 tx=0).stop 2024-05-27T09:41:54.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.635+0000 7fc131e00700 1 -- 172.21.15.119:0/3513307249 shutdown_connections 2024-05-27T09:41:54.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.635+0000 7fc131e00700 1 --2- 172.21.15.119:0/3513307249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc12c102fd0 0x7fc12c106fa0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:54.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.635+0000 7fc131e00700 1 --2- 172.21.15.119:0/3513307249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc12c102630 0x7fc12c102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:54.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.635+0000 7fc131e00700 1 -- 172.21.15.119:0/3513307249 >> 172.21.15.119:0/3513307249 conn(0x7fc12c0fdef0 msgr2=0x7fc12c100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:54.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.635+0000 7fc131e00700 1 -- 172.21.15.119:0/3513307249 shutdown_connections 2024-05-27T09:41:54.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.635+0000 7fc131e00700 1 -- 172.21.15.119:0/3513307249 wait complete. 2024-05-27T09:41:54.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.636+0000 7fc131e00700 1 Processor -- start 2024-05-27T09:41:54.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.637+0000 7fc131e00700 1 -- start start 2024-05-27T09:41:54.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.637+0000 7fc131e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc12c102630 0x7fc12c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:54.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.637+0000 7fc131e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc12c102fd0 0x7fc12c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:54.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.637+0000 7fc131e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c198c00 con 0x7fc12c102fd0 2024-05-27T09:41:54.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.637+0000 7fc131e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c19c990 con 0x7fc12c102630 2024-05-27T09:41:54.639 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.637+0000 7fc12a000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc12c102fd0 0x7fc12c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:54.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.637+0000 7fc12a000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc12c102fd0 0x7fc12c198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35482/0 (socket says 172.21.15.119:35482) 2024-05-27T09:41:54.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.637+0000 7fc12a000700 1 -- 172.21.15.119:0/793363165 learned_addr learned my addr 172.21.15.119:0/793363165 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:41:54.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.638+0000 7fc12aa00700 1 --2- 172.21.15.119:0/793363165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc12c102630 0x7fc12c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:54.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.638+0000 7fc12a000700 1 -- 172.21.15.119:0/793363165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc12c102630 msgr2=0x7fc12c197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:54.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.638+0000 7fc12a000700 1 --2- 172.21.15.119:0/793363165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc12c102630 0x7fc12c197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:54.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.638+0000 7fc12a000700 1 -- 172.21.15.119:0/793363165 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1180095f0 con 0x7fc12c102fd0 2024-05-27T09:41:54.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.638+0000 7fc12a000700 1 --2- 172.21.15.119:0/793363165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc12c102fd0 0x7fc12c198520 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc11800bd70 tx=0x7fc11800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:54.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.638+0000 7fc130a00700 1 -- 172.21.15.119:0/793363165 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc11801d070 con 0x7fc12c102fd0 2024-05-27T09:41:54.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.638+0000 7fc130a00700 1 -- 172.21.15.119:0/793363165 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc118005dc0 con 0x7fc12c102fd0 2024-05-27T09:41:54.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.638+0000 7fc131e00700 1 -- 172.21.15.119:0/793363165 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc12c19cc10 con 0x7fc12c102fd0 2024-05-27T09:41:54.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.638+0000 7fc130a00700 1 -- 172.21.15.119:0/793363165 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc118017400 con 0x7fc12c102fd0 2024-05-27T09:41:54.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.639+0000 7fc131e00700 1 -- 172.21.15.119:0/793363165 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12c19d100 con 0x7fc12c102fd0 2024-05-27T09:41:54.641 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.640+0000 7fc131e00700 1 -- 172.21.15.119:0/793363165 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc12c04ea50 con 0x7fc12c102fd0 2024-05-27T09:41:54.643 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.641+0000 7fc130a00700 1 -- 172.21.15.119:0/793363165 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fc11800f7f0 con 0x7fc12c102fd0 2024-05-27T09:41:54.644 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.643+0000 7fc130a00700 1 --2- 172.21.15.119:0/793363165 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc110078a40 0x7fc11007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:54.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.643+0000 7fc130a00700 1 -- 172.21.15.119:0/793363165 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc11809c2f0 con 0x7fc12c102fd0 2024-05-27T09:41:54.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.643+0000 7fc12aa00700 1 --2- 172.21.15.119:0/793363165 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc110078a40 0x7fc11007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:54.645 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.644+0000 7fc12aa00700 1 --2- 172.21.15.119:0/793363165 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc110078a40 0x7fc11007aef0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc11c005950 tx=0x7fc11c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:54.649 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.647+0000 7fc130a00700 1 -- 172.21.15.119:0/793363165 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fc118063e20 con 0x7fc12c102fd0 2024-05-27T09:41:54.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.985+0000 7fc131e00700 1 -- 172.21.15.119:0/793363165 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc12c199340 con 0x7fc12c102fd0 2024-05-27T09:41:54.988 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.986+0000 7fc130a00700 1 -- 172.21.15.119:0/793363165 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fc1180637c0 con 0x7fc12c102fd0 2024-05-27T09:41:54.988 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:41:54.988 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:41:54.988 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:41:54.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:41:54.988 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:41:54.988 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:41:54.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:41:54.988 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:41:54.988 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 6 2024-05-27T09:41:54.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:41:54.989 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:41:54.989 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 4 2024-05-27T09:41:54.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:41:54.989 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:41:54.989 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 14 2024-05-27T09:41:54.989 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:41:54.989 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:41:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.989+0000 7fc131e00700 1 -- 172.21.15.119:0/793363165 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc110078a40 msgr2=0x7fc11007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.989+0000 7fc131e00700 1 --2- 172.21.15.119:0/793363165 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc110078a40 0x7fc11007aef0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc11c005950 tx=0x7fc11c0058e0 comp rx=0 tx=0).stop 2024-05-27T09:41:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.989+0000 7fc131e00700 1 -- 172.21.15.119:0/793363165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc12c102fd0 msgr2=0x7fc12c198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.989+0000 7fc131e00700 1 --2- 172.21.15.119:0/793363165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc12c102fd0 0x7fc12c198520 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc11800bd70 tx=0x7fc11800be50 comp rx=0 tx=0).stop 2024-05-27T09:41:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.989+0000 7fc131e00700 1 -- 172.21.15.119:0/793363165 shutdown_connections 2024-05-27T09:41:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.989+0000 7fc131e00700 1 --2- 172.21.15.119:0/793363165 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc110078a40 0x7fc11007aef0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.989+0000 7fc131e00700 1 --2- 172.21.15.119:0/793363165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc12c102630 0x7fc12c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.989+0000 7fc131e00700 1 --2- 172.21.15.119:0/793363165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc12c102fd0 0x7fc12c198520 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.989+0000 7fc131e00700 1 -- 172.21.15.119:0/793363165 >> 172.21.15.119:0/793363165 conn(0x7fc12c0fdef0 msgr2=0x7fc12c0ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.989+0000 7fc131e00700 1 -- 172.21.15.119:0/793363165 shutdown_connections 2024-05-27T09:41:54.992 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:54.989+0000 7fc131e00700 1 -- 172.21.15.119:0/793363165 wait complete. 2024-05-27T09:41:55.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.147+0000 7f458a200700 1 -- 172.21.15.119:0/1370166567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45840683d0 msgr2=0x7f45840687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:55.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.147+0000 7f458a200700 1 --2- 172.21.15.119:0/1370166567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45840683d0 0x7f45840687a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4578009910 tx=0x7f4578009c20 comp rx=0 tx=0).stop 2024-05-27T09:41:55.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.148+0000 7f458a200700 1 -- 172.21.15.119:0/1370166567 shutdown_connections 2024-05-27T09:41:55.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.148+0000 7f458a200700 1 --2- 172.21.15.119:0/1370166567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4584068ce0 0x7f458410c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:55.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.148+0000 7f458a200700 1 --2- 172.21.15.119:0/1370166567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45840683d0 0x7f45840687a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:55.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.148+0000 7f458a200700 1 -- 172.21.15.119:0/1370166567 >> 172.21.15.119:0/1370166567 conn(0x7f4584074d00 msgr2=0x7f4584075100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:55.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.148+0000 7f458a200700 1 -- 172.21.15.119:0/1370166567 shutdown_connections 2024-05-27T09:41:55.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.148+0000 7f458a200700 1 -- 172.21.15.119:0/1370166567 wait complete. 2024-05-27T09:41:55.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.149+0000 7f458a200700 1 Processor -- start 2024-05-27T09:41:55.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.149+0000 7f458a200700 1 -- start start 2024-05-27T09:41:55.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.149+0000 7f458a200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45840683d0 0x7f4584197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:55.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.149+0000 7f458a200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4584068ce0 0x7f4584198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:55.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.149+0000 7f458a200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45841988c0 con 0x7f45840683d0 2024-05-27T09:41:55.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.149+0000 7f458a200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4584198a00 con 0x7f4584068ce0 2024-05-27T09:41:55.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.149+0000 7f4588e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45840683d0 0x7f4584197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:55.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.149+0000 7f4588e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45840683d0 0x7f4584197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35512/0 (socket says 172.21.15.119:35512) 2024-05-27T09:41:55.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.149+0000 7f4588e00700 1 -- 172.21.15.119:0/1590046437 learned_addr learned my addr 172.21.15.119:0/1590046437 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:41:55.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.150+0000 7f4583e00700 1 --2- 172.21.15.119:0/1590046437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4584068ce0 0x7f4584198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:55.152 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.150+0000 7f4588e00700 1 -- 172.21.15.119:0/1590046437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4584068ce0 msgr2=0x7f4584198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:55.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.150+0000 7f4588e00700 1 --2- 172.21.15.119:0/1590046437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4584068ce0 0x7f4584198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:55.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.150+0000 7f4588e00700 1 -- 172.21.15.119:0/1590046437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45780095f0 con 0x7f45840683d0 2024-05-27T09:41:55.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.150+0000 7f4588e00700 1 --2- 172.21.15.119:0/1590046437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45840683d0 0x7f4584197d30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4578006010 tx=0x7f457800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:55.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.150+0000 7f4581600700 1 -- 172.21.15.119:0/1590046437 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f457801d070 con 0x7f45840683d0 2024-05-27T09:41:55.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.151+0000 7f4581600700 1 -- 172.21.15.119:0/1590046437 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4578004fb0 con 0x7f45840683d0 2024-05-27T09:41:55.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.151+0000 7f458a200700 1 -- 172.21.15.119:0/1590046437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458419c850 con 0x7f45840683d0 2024-05-27T09:41:55.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.151+0000 7f458a200700 1 -- 172.21.15.119:0/1590046437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458419cda0 con 0x7f45840683d0 2024-05-27T09:41:55.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.151+0000 7f4581600700 1 -- 172.21.15.119:0/1590046437 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4578022ea0 con 0x7f45840683d0 2024-05-27T09:41:55.153 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.152+0000 7f458a200700 1 -- 172.21.15.119:0/1590046437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4584109d50 con 0x7f45840683d0 2024-05-27T09:41:55.154 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.153+0000 7f4581600700 1 -- 172.21.15.119:0/1590046437 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f4578022470 con 0x7f45840683d0 2024-05-27T09:41:55.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.154+0000 7f4581600700 1 --2- 172.21.15.119:0/1590046437 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f456c078a40 0x7f456c07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:55.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.154+0000 7f4581600700 1 -- 172.21.15.119:0/1590046437 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f457809c340 con 0x7f45840683d0 2024-05-27T09:41:55.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.154+0000 7f4583e00700 1 --2- 172.21.15.119:0/1590046437 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f456c078a40 0x7f456c07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:55.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.155+0000 7f4583e00700 1 --2- 172.21.15.119:0/1590046437 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f456c078a40 0x7f456c07aef0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4570005ea0 tx=0x7f4570005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:55.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.159+0000 7f4581600700 1 -- 172.21.15.119:0/1590046437 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f4578063f20 con 0x7f45840683d0 2024-05-27T09:41:55.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:55 smithi119 ceph-mon[88269]: pgmap v262: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-27T09:41:55.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:55 smithi119 ceph-mon[88269]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:41:55.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:55 smithi119 ceph-mon[88269]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:41:55.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:55 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:55.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:55 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi138", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-27T09:41:55.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:55 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:41:55.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:55 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/793363165' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:55.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:55 smithi138 ceph-mon[70004]: pgmap v262: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-27T09:41:55.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:55 smithi138 ceph-mon[70004]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:41:55.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:55 smithi138 ceph-mon[70004]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:41:55.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:55 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:55.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:55 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi138", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-05-27T09:41:55.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:55 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:41:55.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:55 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/793363165' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:41:55.428 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.426+0000 7f458a200700 1 -- 172.21.15.119:0/1590046437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f458404f2a0 con 0x7f45840683d0 2024-05-27T09:41:55.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.427+0000 7f4581600700 1 -- 172.21.15.119:0/1590046437 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 38 v38) v1 ==== 76+0+1958 (secure 0 0 0) 0x7f4578022720 con 0x7f45840683d0 2024-05-27T09:41:55.430 INFO:teuthology.orchestra.run.smithi119.stdout:e38 2024-05-27T09:41:55.430 INFO:teuthology.orchestra.run.smithi119.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T09:41:55.430 INFO:teuthology.orchestra.run.smithi119.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:41:55.430 INFO:teuthology.orchestra.run.smithi119.stdout:legacy client fscid: 1 2024-05-27T09:41:55.430 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:41:55.430 INFO:teuthology.orchestra.run.smithi119.stdout:Filesystem 'cephfs' (1) 2024-05-27T09:41:55.431 INFO:teuthology.orchestra.run.smithi119.stdout:fs_name cephfs 2024-05-27T09:41:55.431 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 38 2024-05-27T09:41:55.431 INFO:teuthology.orchestra.run.smithi119.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-27T09:41:55.431 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-05-27T09:25:53.387388+0000 2024-05-27T09:41:55.431 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2024-05-27T09:41:49.972954+0000 2024-05-27T09:41:55.431 INFO:teuthology.orchestra.run.smithi119.stdout:tableserver 0 2024-05-27T09:41:55.431 INFO:teuthology.orchestra.run.smithi119.stdout:root 0 2024-05-27T09:41:55.431 INFO:teuthology.orchestra.run.smithi119.stdout:session_timeout 60 2024-05-27T09:41:55.431 INFO:teuthology.orchestra.run.smithi119.stdout:session_autoclose 300 2024-05-27T09:41:55.431 INFO:teuthology.orchestra.run.smithi119.stdout:max_file_size 1099511627776 2024-05-27T09:41:55.431 INFO:teuthology.orchestra.run.smithi119.stdout:max_xattr_size 65536 2024-05-27T09:41:55.431 INFO:teuthology.orchestra.run.smithi119.stdout:required_client_features {} 2024-05-27T09:41:55.431 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure 0 2024-05-27T09:41:55.431 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure_osd_epoch 81 2024-05-27T09:41:55.431 INFO:teuthology.orchestra.run.smithi119.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T09:41:55.432 INFO:teuthology.orchestra.run.smithi119.stdout:max_mds 2 2024-05-27T09:41:55.432 INFO:teuthology.orchestra.run.smithi119.stdout:in 0,1 2024-05-27T09:41:55.432 INFO:teuthology.orchestra.run.smithi119.stdout:up {0=34436,1=34468} 2024-05-27T09:41:55.432 INFO:teuthology.orchestra.run.smithi119.stdout:failed 2024-05-27T09:41:55.432 INFO:teuthology.orchestra.run.smithi119.stdout:damaged 2024-05-27T09:41:55.432 INFO:teuthology.orchestra.run.smithi119.stdout:stopped 2024-05-27T09:41:55.432 INFO:teuthology.orchestra.run.smithi119.stdout:data_pools [3] 2024-05-27T09:41:55.432 INFO:teuthology.orchestra.run.smithi119.stdout:metadata_pool 2 2024-05-27T09:41:55.432 INFO:teuthology.orchestra.run.smithi119.stdout:inline_data enabled 2024-05-27T09:41:55.432 INFO:teuthology.orchestra.run.smithi119.stdout:balancer 2024-05-27T09:41:55.432 INFO:teuthology.orchestra.run.smithi119.stdout:bal_rank_mask -1 2024-05-27T09:41:55.432 INFO:teuthology.orchestra.run.smithi119.stdout:standby_count_wanted 1 2024-05-27T09:41:55.432 INFO:teuthology.orchestra.run.smithi119.stdout:qdb_cluster leader: 34436 members: 34436,34468 2024-05-27T09:41:55.432 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.nghcey{0:34436} state up:active seq 29 join_fscid=1 addr [v2:172.21.15.119:6826/3357235708,v1:172.21.15.119:6827/3357235708] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T09:41:55.432 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.ppmzzk{1:34468} state up:active seq 23 join_fscid=1 addr [v2:172.21.15.119:6828/2430623742,v1:172.21.15.119:6829/2430623742] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T09:41:55.433 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:41:55.433 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:41:55.433 INFO:teuthology.orchestra.run.smithi119.stdout:Standby daemons: 2024-05-27T09:41:55.433 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:41:55.433 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.ndfkhd{-1:44289} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/3387966485,v1:172.21.15.138:6825/3387966485] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T09:41:55.433 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.alszpk{-1:44303} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/3198147602,v1:172.21.15.138:6827/3198147602] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T09:41:55.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.430+0000 7f458a200700 1 -- 172.21.15.119:0/1590046437 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f456c078a40 msgr2=0x7f456c07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:55.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.430+0000 7f458a200700 1 --2- 172.21.15.119:0/1590046437 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f456c078a40 0x7f456c07aef0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4570005ea0 tx=0x7f4570005e30 comp rx=0 tx=0).stop 2024-05-27T09:41:55.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.430+0000 7f458a200700 1 -- 172.21.15.119:0/1590046437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45840683d0 msgr2=0x7f4584197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:55.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.430+0000 7f458a200700 1 --2- 172.21.15.119:0/1590046437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45840683d0 0x7f4584197d30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4578006010 tx=0x7f457800be20 comp rx=0 tx=0).stop 2024-05-27T09:41:55.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.430+0000 7f458a200700 1 -- 172.21.15.119:0/1590046437 shutdown_connections 2024-05-27T09:41:55.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.430+0000 7f458a200700 1 --2- 172.21.15.119:0/1590046437 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f456c078a40 0x7f456c07aef0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:55.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.430+0000 7f458a200700 1 --2- 172.21.15.119:0/1590046437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45840683d0 0x7f4584197d30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:55.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.430+0000 7f458a200700 1 --2- 172.21.15.119:0/1590046437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4584068ce0 0x7f4584198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:55.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.430+0000 7f458a200700 1 -- 172.21.15.119:0/1590046437 >> 172.21.15.119:0/1590046437 conn(0x7f4584074d00 msgr2=0x7f45840fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:55.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.431+0000 7f458a200700 1 -- 172.21.15.119:0/1590046437 shutdown_connections 2024-05-27T09:41:55.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.431+0000 7f458a200700 1 -- 172.21.15.119:0/1590046437 wait complete. 2024-05-27T09:41:55.434 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 38 2024-05-27T09:41:55.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.593+0000 7fb4a4c00700 1 -- 172.21.15.119:0/1676987692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4a0102630 msgr2=0x7fb4a0102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:55.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.593+0000 7fb4a4c00700 1 --2- 172.21.15.119:0/1676987692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4a0102630 0x7fb4a0102a00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb490009910 tx=0x7fb490009c20 comp rx=0 tx=0).stop 2024-05-27T09:41:55.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.594+0000 7fb4a4c00700 1 -- 172.21.15.119:0/1676987692 shutdown_connections 2024-05-27T09:41:55.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.594+0000 7fb4a4c00700 1 --2- 172.21.15.119:0/1676987692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a0102fd0 0x7fb4a0106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:55.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.594+0000 7fb4a4c00700 1 --2- 172.21.15.119:0/1676987692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4a0102630 0x7fb4a0102a00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:55.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.594+0000 7fb4a4c00700 1 -- 172.21.15.119:0/1676987692 >> 172.21.15.119:0/1676987692 conn(0x7fb4a00fdef0 msgr2=0x7fb4a0100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:55.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.594+0000 7fb4a4c00700 1 -- 172.21.15.119:0/1676987692 shutdown_connections 2024-05-27T09:41:55.596 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.594+0000 7fb4a4c00700 1 -- 172.21.15.119:0/1676987692 wait complete. 2024-05-27T09:41:55.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.595+0000 7fb4a4c00700 1 Processor -- start 2024-05-27T09:41:55.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.595+0000 7fb4a4c00700 1 -- start start 2024-05-27T09:41:55.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.595+0000 7fb4a4c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4a0102630 0x7fb4a0198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:55.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.595+0000 7fb4a4c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a0102fd0 0x7fb4a0198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:55.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.595+0000 7fb4a4c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a0198bb0 con 0x7fb4a0102630 2024-05-27T09:41:55.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.595+0000 7fb4a4c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a0198cf0 con 0x7fb4a0102fd0 2024-05-27T09:41:55.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.596+0000 7fb49de00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4a0102630 0x7fb4a0198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:55.597 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.596+0000 7fb49de00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4a0102630 0x7fb4a0198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35534/0 (socket says 172.21.15.119:35534) 2024-05-27T09:41:55.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.596+0000 7fb49de00700 1 -- 172.21.15.119:0/1235146330 learned_addr learned my addr 172.21.15.119:0/1235146330 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:41:55.598 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.596+0000 7fb49d400700 1 --2- 172.21.15.119:0/1235146330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a0102fd0 0x7fb4a0198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:55.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.596+0000 7fb49de00700 1 -- 172.21.15.119:0/1235146330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a0102fd0 msgr2=0x7fb4a0198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:55.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.596+0000 7fb49de00700 1 --2- 172.21.15.119:0/1235146330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a0102fd0 0x7fb4a0198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:55.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.596+0000 7fb49de00700 1 -- 172.21.15.119:0/1235146330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4900095f0 con 0x7fb4a0102630 2024-05-27T09:41:55.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.597+0000 7fb49de00700 1 --2- 172.21.15.119:0/1235146330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4a0102630 0x7fb4a0198020 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb490006010 tx=0x7fb49000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:55.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.597+0000 7fb496a00700 1 -- 172.21.15.119:0/1235146330 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb49001d070 con 0x7fb4a0102630 2024-05-27T09:41:55.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.597+0000 7fb496a00700 1 -- 172.21.15.119:0/1235146330 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb490004fb0 con 0x7fb4a0102630 2024-05-27T09:41:55.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.597+0000 7fb4a4c00700 1 -- 172.21.15.119:0/1235146330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a019cae0 con 0x7fb4a0102630 2024-05-27T09:41:55.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.597+0000 7fb496a00700 1 -- 172.21.15.119:0/1235146330 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb49000f690 con 0x7fb4a0102630 2024-05-27T09:41:55.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.597+0000 7fb4a4c00700 1 -- 172.21.15.119:0/1235146330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a019cfd0 con 0x7fb4a0102630 2024-05-27T09:41:55.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.598+0000 7fb4a4c00700 1 -- 172.21.15.119:0/1235146330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a010a940 con 0x7fb4a0102630 2024-05-27T09:41:55.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.599+0000 7fb496a00700 1 -- 172.21.15.119:0/1235146330 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fb4900228a0 con 0x7fb4a0102630 2024-05-27T09:41:55.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.600+0000 7fb496a00700 1 --2- 172.21.15.119:0/1235146330 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb484078a40 0x7fb48407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:55.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.601+0000 7fb496a00700 1 -- 172.21.15.119:0/1235146330 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb49009c5e0 con 0x7fb4a0102630 2024-05-27T09:41:55.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.601+0000 7fb49d400700 1 --2- 172.21.15.119:0/1235146330 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb484078a40 0x7fb48407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:55.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.601+0000 7fb49d400700 1 --2- 172.21.15.119:0/1235146330 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb484078a40 0x7fb48407aef0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb48c005ea0 tx=0x7fb48c005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:55.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.606+0000 7fb496a00700 1 -- 172.21.15.119:0/1235146330 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fb490064190 con 0x7fb4a0102630 2024-05-27T09:41:55.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.868+0000 7fb4a4c00700 1 -- 172.21.15.119:0/1235146330 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4a002cd20 con 0x7fb484078a40 2024-05-27T09:41:55.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.870+0000 7fb496a00700 1 -- 172.21.15.119:0/1235146330 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7fb4a002cd20 con 0x7fb484078a40 2024-05-27T09:41:55.872 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:41:55.872 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6", 2024-05-27T09:41:55.872 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": true, 2024-05-27T09:41:55.873 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-27T09:41:55.873 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [ 2024-05-27T09:41:55.873 INFO:teuthology.orchestra.run.smithi119.stdout: "crash", 2024-05-27T09:41:55.873 INFO:teuthology.orchestra.run.smithi119.stdout: "osd", 2024-05-27T09:41:55.873 INFO:teuthology.orchestra.run.smithi119.stdout: "mds", 2024-05-27T09:41:55.873 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr", 2024-05-27T09:41:55.873 INFO:teuthology.orchestra.run.smithi119.stdout: "mon" 2024-05-27T09:41:55.873 INFO:teuthology.orchestra.run.smithi119.stdout: ], 2024-05-27T09:41:55.873 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": "16/23 daemons upgraded", 2024-05-27T09:41:55.873 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-05-27T09:41:55.873 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:41:55.873 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:41:55.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.873+0000 7fb4a4c00700 1 -- 172.21.15.119:0/1235146330 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb484078a40 msgr2=0x7fb48407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:55.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.873+0000 7fb4a4c00700 1 --2- 172.21.15.119:0/1235146330 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb484078a40 0x7fb48407aef0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb48c005ea0 tx=0x7fb48c005e30 comp rx=0 tx=0).stop 2024-05-27T09:41:55.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.874+0000 7fb4a4c00700 1 -- 172.21.15.119:0/1235146330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4a0102630 msgr2=0x7fb4a0198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:55.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.874+0000 7fb4a4c00700 1 --2- 172.21.15.119:0/1235146330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4a0102630 0x7fb4a0198020 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb490006010 tx=0x7fb49000be20 comp rx=0 tx=0).stop 2024-05-27T09:41:55.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.874+0000 7fb4a4c00700 1 -- 172.21.15.119:0/1235146330 shutdown_connections 2024-05-27T09:41:55.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.874+0000 7fb4a4c00700 1 --2- 172.21.15.119:0/1235146330 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb484078a40 0x7fb48407aef0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:55.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.874+0000 7fb4a4c00700 1 --2- 172.21.15.119:0/1235146330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4a0102630 0x7fb4a0198020 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:55.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.874+0000 7fb4a4c00700 1 --2- 172.21.15.119:0/1235146330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a0102fd0 0x7fb4a0198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:55.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.874+0000 7fb4a4c00700 1 -- 172.21.15.119:0/1235146330 >> 172.21.15.119:0/1235146330 conn(0x7fb4a00fdef0 msgr2=0x7fb4a01002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:55.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.874+0000 7fb4a4c00700 1 -- 172.21.15.119:0/1235146330 shutdown_connections 2024-05-27T09:41:55.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:55.874+0000 7fb4a4c00700 1 -- 172.21.15.119:0/1235146330 wait complete. 2024-05-27T09:41:56.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.035+0000 7f2d65800700 1 -- 172.21.15.119:0/465598279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d60102fd0 msgr2=0x7f2d60106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:56.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.035+0000 7f2d65800700 1 --2- 172.21.15.119:0/465598279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d60102fd0 0x7f2d60106fa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c009870 tx=0x7f2d4c009b80 comp rx=0 tx=0).stop 2024-05-27T09:41:56.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.035+0000 7f2d65800700 1 -- 172.21.15.119:0/465598279 shutdown_connections 2024-05-27T09:41:56.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.035+0000 7f2d65800700 1 --2- 172.21.15.119:0/465598279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d60102fd0 0x7f2d60106fa0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:56.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.035+0000 7f2d65800700 1 --2- 172.21.15.119:0/465598279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d60102630 0x7f2d60102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:56.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.035+0000 7f2d65800700 1 -- 172.21.15.119:0/465598279 >> 172.21.15.119:0/465598279 conn(0x7f2d600fdef0 msgr2=0x7f2d60100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:56.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.035+0000 7f2d65800700 1 -- 172.21.15.119:0/465598279 shutdown_connections 2024-05-27T09:41:56.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.035+0000 7f2d65800700 1 -- 172.21.15.119:0/465598279 wait complete. 2024-05-27T09:41:56.038 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.036+0000 7f2d65800700 1 Processor -- start 2024-05-27T09:41:56.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.036+0000 7f2d65800700 1 -- start start 2024-05-27T09:41:56.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.037+0000 7f2d65800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d60102630 0x7f2d60197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:56.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.037+0000 7f2d65800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d60102fd0 0x7f2d60198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:56.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.037+0000 7f2d65800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d60198c00 con 0x7f2d60102fd0 2024-05-27T09:41:56.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.037+0000 7f2d65800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6019c990 con 0x7f2d60102630 2024-05-27T09:41:56.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.037+0000 7f2d5e000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d60102fd0 0x7f2d60198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:56.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.037+0000 7f2d5ea00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d60102630 0x7f2d60197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:56.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.037+0000 7f2d5e000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d60102fd0 0x7f2d60198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35544/0 (socket says 172.21.15.119:35544) 2024-05-27T09:41:56.039 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.037+0000 7f2d5e000700 1 -- 172.21.15.119:0/2230749037 learned_addr learned my addr 172.21.15.119:0/2230749037 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:41:56.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.037+0000 7f2d5e000700 1 -- 172.21.15.119:0/2230749037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d60102630 msgr2=0x7f2d60197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:56.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.037+0000 7f2d5e000700 1 --2- 172.21.15.119:0/2230749037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d60102630 0x7f2d60197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:56.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.037+0000 7f2d5e000700 1 -- 172.21.15.119:0/2230749037 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d500095f0 con 0x7f2d60102fd0 2024-05-27T09:41:56.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.038+0000 7f2d5e000700 1 --2- 172.21.15.119:0/2230749037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d60102fd0 0x7f2d60198520 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c004700 tx=0x7f2d4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:56.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.038+0000 7f2d57400700 1 -- 172.21.15.119:0/2230749037 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d4c01c070 con 0x7f2d60102fd0 2024-05-27T09:41:56.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.038+0000 7f2d57400700 1 -- 172.21.15.119:0/2230749037 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d4c00fb70 con 0x7f2d60102fd0 2024-05-27T09:41:56.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.038+0000 7f2d65800700 1 -- 172.21.15.119:0/2230749037 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4c009520 con 0x7f2d60102fd0 2024-05-27T09:41:56.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.038+0000 7f2d57400700 1 -- 172.21.15.119:0/2230749037 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d4c022c60 con 0x7f2d60102fd0 2024-05-27T09:41:56.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.038+0000 7f2d65800700 1 -- 172.21.15.119:0/2230749037 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6019cf70 con 0x7f2d60102fd0 2024-05-27T09:41:56.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.039+0000 7f2d65800700 1 -- 172.21.15.119:0/2230749037 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6004ea50 con 0x7f2d60102fd0 2024-05-27T09:41:56.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.041+0000 7f2d57400700 1 -- 172.21.15.119:0/2230749037 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f2d4c020070 con 0x7f2d60102fd0 2024-05-27T09:41:56.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.041+0000 7f2d57400700 1 --2- 172.21.15.119:0/2230749037 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2d44078a40 0x7f2d4407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:41:56.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.042+0000 7f2d57400700 1 -- 172.21.15.119:0/2230749037 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2d4c09be10 con 0x7f2d60102fd0 2024-05-27T09:41:56.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.042+0000 7f2d5ea00700 1 --2- 172.21.15.119:0/2230749037 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2d44078a40 0x7f2d4407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:41:56.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.042+0000 7f2d5ea00700 1 --2- 172.21.15.119:0/2230749037 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2d44078a40 0x7f2d4407aef0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2d50005fd0 tx=0x7f2d50009310 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:41:56.048 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.047+0000 7f2d57400700 1 -- 172.21.15.119:0/2230749037 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f2d4c0639f0 con 0x7f2d60102fd0 2024-05-27T09:41:56.058 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:56 smithi138 ceph-mon[70004]: from='client.34544 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:41:56.058 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:56 smithi138 ceph-mon[70004]: Upgrade: Updating ceph-exporter.smithi138 (2/2) 2024-05-27T09:41:56.059 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:56 smithi138 ceph-mon[70004]: Deploying daemon ceph-exporter.smithi138 on smithi138 2024-05-27T09:41:56.059 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:56 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/1590046437' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:41:56.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:56 smithi119 ceph-mon[88269]: from='client.34544 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:41:56.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:56 smithi119 ceph-mon[88269]: Upgrade: Updating ceph-exporter.smithi138 (2/2) 2024-05-27T09:41:56.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:56 smithi119 ceph-mon[88269]: Deploying daemon ceph-exporter.smithi138 on smithi138 2024-05-27T09:41:56.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:56 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/1590046437' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:41:56.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.357+0000 7f2d65800700 1 -- 172.21.15.119:0/2230749037 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2d6019cb50 con 0x7f2d60102fd0 2024-05-27T09:41:56.360 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.358+0000 7f2d57400700 1 -- 172.21.15.119:0/2230749037 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f2d4c063390 con 0x7f2d60102fd0 2024-05-27T09:41:56.360 INFO:teuthology.orchestra.run.smithi119.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-27T09:41:56.361 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:41:56.361 INFO:teuthology.orchestra.run.smithi119.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:41:56.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.361+0000 7f2d65800700 1 -- 172.21.15.119:0/2230749037 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2d44078a40 msgr2=0x7f2d4407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:56.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.361+0000 7f2d65800700 1 --2- 172.21.15.119:0/2230749037 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2d44078a40 0x7f2d4407aef0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2d50005fd0 tx=0x7f2d50009310 comp rx=0 tx=0).stop 2024-05-27T09:41:56.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.361+0000 7f2d65800700 1 -- 172.21.15.119:0/2230749037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d60102fd0 msgr2=0x7f2d60198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:41:56.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.361+0000 7f2d65800700 1 --2- 172.21.15.119:0/2230749037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d60102fd0 0x7f2d60198520 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c004700 tx=0x7f2d4c004730 comp rx=0 tx=0).stop 2024-05-27T09:41:56.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.362+0000 7f2d65800700 1 -- 172.21.15.119:0/2230749037 shutdown_connections 2024-05-27T09:41:56.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.362+0000 7f2d65800700 1 --2- 172.21.15.119:0/2230749037 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2d44078a40 0x7f2d4407aef0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:56.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.362+0000 7f2d65800700 1 --2- 172.21.15.119:0/2230749037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d60102630 0x7f2d60197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:56.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.362+0000 7f2d65800700 1 --2- 172.21.15.119:0/2230749037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d60102fd0 0x7f2d60198520 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:41:56.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.362+0000 7f2d65800700 1 -- 172.21.15.119:0/2230749037 >> 172.21.15.119:0/2230749037 conn(0x7f2d600fdef0 msgr2=0x7f2d600ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:41:56.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.362+0000 7f2d65800700 1 -- 172.21.15.119:0/2230749037 shutdown_connections 2024-05-27T09:41:56.364 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:41:56.362+0000 7f2d65800700 1 -- 172.21.15.119:0/2230749037 wait complete. 2024-05-27T09:41:57.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:57 smithi138 ceph-mon[70004]: pgmap v263: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 9.4 MiB/s rd, 6.7 KiB/s wr, 12 op/s 2024-05-27T09:41:57.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:57 smithi138 ceph-mon[70004]: from='client.34556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:41:57.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:57 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/2230749037' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:41:57.376 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:57 smithi119 ceph-mon[88269]: pgmap v263: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 9.4 MiB/s rd, 6.7 KiB/s wr, 12 op/s 2024-05-27T09:41:57.376 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:57 smithi119 ceph-mon[88269]: from='client.34556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:41:57.376 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:57 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/2230749037' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:41:58.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:58 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:58.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:58 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:41:58.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:41:58 smithi138 ceph-mon[70004]: pgmap v264: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 4.4 MiB/s rd, 14 KiB/s wr, 13 op/s 2024-05-27T09:41:58.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:58 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:41:58.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:58 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:41:58.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:41:58 smithi119 ceph-mon[88269]: pgmap v264: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 4.4 MiB/s rd, 14 KiB/s wr, 13 op/s 2024-05-27T09:42:00.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:00 smithi138 ceph-mon[70004]: pgmap v265: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-05-27T09:42:00.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:00 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:00.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:00 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:00.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:00 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:42:00.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:00 smithi119 ceph-mon[88269]: pgmap v265: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-05-27T09:42:00.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:00 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:00.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:00 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:00.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:00 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:42:02.792 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:02 smithi138 ceph-mon[70004]: pgmap v266: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 7.5 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-05-27T09:42:02.907 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:02 smithi119 ceph-mon[88269]: pgmap v266: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 7.5 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-05-27T09:42:04.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:04 smithi138 ceph-mon[70004]: pgmap v267: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 4.5 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-05-27T09:42:04.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:04 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:04.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:04 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:04.909 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:04 smithi119 ceph-mon[88269]: pgmap v267: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 4.5 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-05-27T09:42:04.909 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:04 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:04.909 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:04 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:06.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:06 smithi138 ceph-mon[70004]: pgmap v268: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-05-27T09:42:06.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:06 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:06.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:06 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:06.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:06 smithi119 ceph-mon[88269]: pgmap v268: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-05-27T09:42:06.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:06 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:06.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:06 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:07.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:07 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:07.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:07 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:08.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:07 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:08.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:07 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:09.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:08 smithi138 ceph-mon[70004]: pgmap v269: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 7.8 KiB/s wr, 1 op/s 2024-05-27T09:42:09.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:08 smithi119 ceph-mon[88269]: pgmap v269: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 7.8 KiB/s wr, 1 op/s 2024-05-27T09:42:10.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:09 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:10.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:09 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:10.207 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:09 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:10.207 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:09 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:10.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:10 smithi119 ceph-mon[88269]: pgmap v270: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:11.115 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:10 smithi138 ceph-mon[70004]: pgmap v270: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:12.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:12 smithi119 ceph-mon[88269]: pgmap v271: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:12.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:12.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:12.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:42:13.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:12 smithi138 ceph-mon[70004]: pgmap v271: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:13.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:13.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:13.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:42:15.221 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:14 smithi119 ceph-mon[88269]: pgmap v272: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:15.221 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:15.221 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:15.221 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:42:15.221 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:42:15.221 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:15.221 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:42:15.222 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:15.222 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:15.222 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:15.222 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:15.222 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:15.222 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:14 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-05-27T09:42:15.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:14 smithi138 ceph-mon[70004]: pgmap v272: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:15.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:15.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:15.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:42:15.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:42:15.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:15.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:42:15.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:15.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:15.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:15.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:15.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:15.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:14 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-05-27T09:42:16.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: Upgrade: Setting filesystem cephfs Joinable 2024-05-27T09:42:16.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-05-27T09:42:16.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: fsmap cephfs:2 {0=cephfs.smithi119.nghcey=up:active,1=cephfs.smithi119.ppmzzk=up:active} 2 up:standby 2024-05-27T09:42:16.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:16.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi119"}]: dispatch 2024-05-27T09:42:16.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi119"}]': finished 2024-05-27T09:42:16.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi138"}]: dispatch 2024-05-27T09:42:16.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi138"}]': finished 2024-05-27T09:42:16.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:16.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:16.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:16.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.250 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-27T09:42:16.251 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-27T09:42:16.251 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:42:16.251 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-27T09:42:16.251 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-27T09:42:16.251 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-05-27T09:42:16.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-27T09:42:16.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-05-27T09:42:16.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-27T09:42:16.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-05-27T09:42:16.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-27T09:42:16.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-27T09:42:16.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-27T09:42:16.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-27T09:42:16.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:42:16.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-27T09:42:16.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-05-27T09:42:16.252 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-27T09:42:16.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-27T09:42:16.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-27T09:42:16.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-27T09:42:16.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-27T09:42:16.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-27T09:42:16.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:42:16.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:42:16.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:42:16.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:42:16.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:42:16.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:42:16.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-27T09:42:16.253 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-27T09:42:16.254 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:42:16.254 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:42:16.254 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:42:16.254 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:15 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:16.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: Upgrade: Setting filesystem cephfs Joinable 2024-05-27T09:42:16.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-05-27T09:42:16.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: fsmap cephfs:2 {0=cephfs.smithi119.nghcey=up:active,1=cephfs.smithi119.ppmzzk=up:active} 2 up:standby 2024-05-27T09:42:16.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:16.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi119"}]: dispatch 2024-05-27T09:42:16.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi119"}]': finished 2024-05-27T09:42:16.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi138"}]: dispatch 2024-05-27T09:42:16.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi138"}]': finished 2024-05-27T09:42:16.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:16.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:16.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:16.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:16.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-27T09:42:16.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-27T09:42:16.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:42:16.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-27T09:42:16.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-27T09:42:16.390 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-05-27T09:42:16.390 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-27T09:42:16.390 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-05-27T09:42:16.390 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-27T09:42:16.390 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-05-27T09:42:16.390 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-27T09:42:16.390 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-27T09:42:16.390 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-27T09:42:16.390 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-27T09:42:16.390 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:42:16.390 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-27T09:42:16.390 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-05-27T09:42:16.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-27T09:42:16.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-27T09:42:16.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-27T09:42:16.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-27T09:42:16.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-27T09:42:16.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-27T09:42:16.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:42:16.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:42:16.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:42:16.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:42:16.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:42:16.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-27T09:42:16.391 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-27T09:42:16.392 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-27T09:42:16.392 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:42:16.392 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:42:16.392 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:42:16.392 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:15 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:17.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:16 smithi119 ceph-mon[88269]: pgmap v273: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:17.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:16 smithi119 ceph-mon[88269]: Upgrade: Setting container_image for all ceph-exporter 2024-05-27T09:42:17.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:16 smithi119 ceph-mon[88269]: Upgrade: Setting container_image for all iscsi 2024-05-27T09:42:17.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:16 smithi119 ceph-mon[88269]: Upgrade: Setting container_image for all nfs 2024-05-27T09:42:17.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:16 smithi119 ceph-mon[88269]: Upgrade: Setting container_image for all nvmeof 2024-05-27T09:42:17.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:16 smithi119 ceph-mon[88269]: Upgrade: Finalizing container_image settings 2024-05-27T09:42:17.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:16 smithi119 ceph-mon[88269]: Upgrade: Complete! 2024-05-27T09:42:17.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:16 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:42:17.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:16 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:42:17.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:16 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:42:17.249 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:16 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:17.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:16 smithi138 ceph-mon[70004]: pgmap v273: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:17.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:16 smithi138 ceph-mon[70004]: Upgrade: Setting container_image for all ceph-exporter 2024-05-27T09:42:17.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:16 smithi138 ceph-mon[70004]: Upgrade: Setting container_image for all iscsi 2024-05-27T09:42:17.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:16 smithi138 ceph-mon[70004]: Upgrade: Setting container_image for all nfs 2024-05-27T09:42:17.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:16 smithi138 ceph-mon[70004]: Upgrade: Setting container_image for all nvmeof 2024-05-27T09:42:17.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:16 smithi138 ceph-mon[70004]: Upgrade: Finalizing container_image settings 2024-05-27T09:42:17.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:16 smithi138 ceph-mon[70004]: Upgrade: Complete! 2024-05-27T09:42:17.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:16 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:42:17.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:16 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:42:17.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:16 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:42:17.387 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:16 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:18.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:18 smithi119 ceph-mon[88269]: pgmap v274: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:18.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:18 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:18.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:18 smithi138 ceph-mon[70004]: pgmap v274: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:18.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:18 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:42:20.863 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:20 smithi138 ceph-mon[70004]: pgmap v275: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:20.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:20 smithi119 ceph-mon[88269]: pgmap v275: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:22.836 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:22 smithi119 ceph-mon[88269]: pgmap v276: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:22.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:22 smithi138 ceph-mon[70004]: pgmap v276: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:24.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:24 smithi138 ceph-mon[70004]: pgmap v277: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:24.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:24 smithi119 ceph-mon[88269]: pgmap v277: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:26.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.534+0000 7f838e000700 1 -- 172.21.15.119:0/3223210846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83881025f0 msgr2=0x7f83881029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:42:26.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.534+0000 7f838e000700 1 --2- 172.21.15.119:0/3223210846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83881025f0 0x7f83881029c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8378009910 tx=0x7f8378009c20 comp rx=0 tx=0).stop 2024-05-27T09:42:26.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.534+0000 7f838e000700 1 -- 172.21.15.119:0/3223210846 shutdown_connections 2024-05-27T09:42:26.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.535+0000 7f838e000700 1 --2- 172.21.15.119:0/3223210846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8388102f90 0x7f8388106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:26.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.535+0000 7f838e000700 1 --2- 172.21.15.119:0/3223210846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83881025f0 0x7f83881029c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:26.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.535+0000 7f838e000700 1 -- 172.21.15.119:0/3223210846 >> 172.21.15.119:0/3223210846 conn(0x7f83880fde90 msgr2=0x7f83881002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:42:26.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.535+0000 7f838e000700 1 -- 172.21.15.119:0/3223210846 shutdown_connections 2024-05-27T09:42:26.536 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.535+0000 7f838e000700 1 -- 172.21.15.119:0/3223210846 wait complete. 2024-05-27T09:42:26.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.536+0000 7f838e000700 1 Processor -- start 2024-05-27T09:42:26.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.536+0000 7f838e000700 1 -- start start 2024-05-27T09:42:26.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.536+0000 7f838e000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83881025f0 0x7f8388197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:42:26.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.536+0000 7f838e000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8388102f90 0x7f83881984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:42:26.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.536+0000 7f838e000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8388198bc0 con 0x7f83881025f0 2024-05-27T09:42:26.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.536+0000 7f838e000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838819c950 con 0x7f8388102f90 2024-05-27T09:42:26.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.536+0000 7f8387400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83881025f0 0x7f8388197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:42:26.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.537+0000 7f8387400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83881025f0 0x7f8388197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40134/0 (socket says 172.21.15.119:40134) 2024-05-27T09:42:26.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.537+0000 7f8387400700 1 -- 172.21.15.119:0/2755532870 learned_addr learned my addr 172.21.15.119:0/2755532870 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:42:26.538 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.537+0000 7f8387400700 1 -- 172.21.15.119:0/2755532870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8388102f90 msgr2=0x7f83881984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:42:26.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.537+0000 7f8386a00700 1 --2- 172.21.15.119:0/2755532870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8388102f90 0x7f83881984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:42:26.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.537+0000 7f8387400700 1 --2- 172.21.15.119:0/2755532870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8388102f90 0x7f83881984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:26.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.537+0000 7f8387400700 1 -- 172.21.15.119:0/2755532870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83780095f0 con 0x7f83881025f0 2024-05-27T09:42:26.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.537+0000 7f8386a00700 1 --2- 172.21.15.119:0/2755532870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8388102f90 0x7f83881984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:42:26.539 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.537+0000 7f8387400700 1 --2- 172.21.15.119:0/2755532870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83881025f0 0x7f8388197fa0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8378006010 tx=0x7f837800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:42:26.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.538+0000 7f837fe00700 1 -- 172.21.15.119:0/2755532870 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f837801d070 con 0x7f83881025f0 2024-05-27T09:42:26.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.538+0000 7f837fe00700 1 -- 172.21.15.119:0/2755532870 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8378004fb0 con 0x7f83881025f0 2024-05-27T09:42:26.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.538+0000 7f838e000700 1 -- 172.21.15.119:0/2755532870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f838819cbd0 con 0x7f83881025f0 2024-05-27T09:42:26.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.538+0000 7f837fe00700 1 -- 172.21.15.119:0/2755532870 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8378022ea0 con 0x7f83881025f0 2024-05-27T09:42:26.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.538+0000 7f838e000700 1 -- 172.21.15.119:0/2755532870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f838819d0c0 con 0x7f83881025f0 2024-05-27T09:42:26.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.539+0000 7f838e000700 1 -- 172.21.15.119:0/2755532870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838804ea50 con 0x7f83881025f0 2024-05-27T09:42:26.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.540+0000 7f837fe00700 1 -- 172.21.15.119:0/2755532870 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f8378022470 con 0x7f83881025f0 2024-05-27T09:42:26.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.541+0000 7f837fe00700 1 --2- 172.21.15.119:0/2755532870 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f836c078a40 0x7f836c07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:42:26.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.541+0000 7f837fe00700 1 -- 172.21.15.119:0/2755532870 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f837809c3f0 con 0x7f83881025f0 2024-05-27T09:42:26.542 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.542+0000 7f8386a00700 1 --2- 172.21.15.119:0/2755532870 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f836c078a40 0x7f836c07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:42:26.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.542+0000 7f8386a00700 1 --2- 172.21.15.119:0/2755532870 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f836c078a40 0x7f836c07aef0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8374005fd0 tx=0x7f8374005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:42:26.548 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.547+0000 7f837fe00700 1 -- 172.21.15.119:0/2755532870 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f8378063fd0 con 0x7f83881025f0 2024-05-27T09:42:26.801 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.800+0000 7f838e000700 1 -- 172.21.15.119:0/2755532870 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f838819d5c0 con 0x7f836c078a40 2024-05-27T09:42:26.802 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:26 smithi119 ceph-mon[88269]: pgmap v278: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:26.804 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.803+0000 7f837fe00700 1 -- 172.21.15.119:0/2755532870 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f838819d5c0 con 0x7f836c078a40 2024-05-27T09:42:26.807 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.806+0000 7f838e000700 1 -- 172.21.15.119:0/2755532870 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f836c078a40 msgr2=0x7f836c07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:42:26.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.806+0000 7f838e000700 1 --2- 172.21.15.119:0/2755532870 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f836c078a40 0x7f836c07aef0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8374005fd0 tx=0x7f8374005dc0 comp rx=0 tx=0).stop 2024-05-27T09:42:26.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.807+0000 7f838e000700 1 -- 172.21.15.119:0/2755532870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83881025f0 msgr2=0x7f8388197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:42:26.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.807+0000 7f838e000700 1 --2- 172.21.15.119:0/2755532870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83881025f0 0x7f8388197fa0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8378006010 tx=0x7f837800be20 comp rx=0 tx=0).stop 2024-05-27T09:42:26.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.807+0000 7f838e000700 1 -- 172.21.15.119:0/2755532870 shutdown_connections 2024-05-27T09:42:26.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.807+0000 7f838e000700 1 --2- 172.21.15.119:0/2755532870 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f836c078a40 0x7f836c07aef0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:26.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.807+0000 7f838e000700 1 --2- 172.21.15.119:0/2755532870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83881025f0 0x7f8388197fa0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:26.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.807+0000 7f838e000700 1 --2- 172.21.15.119:0/2755532870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8388102f90 0x7f83881984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:26.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.807+0000 7f838e000700 1 -- 172.21.15.119:0/2755532870 >> 172.21.15.119:0/2755532870 conn(0x7f83880fde90 msgr2=0x7f83880ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:42:26.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.807+0000 7f838e000700 1 -- 172.21.15.119:0/2755532870 shutdown_connections 2024-05-27T09:42:26.808 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:26.807+0000 7f838e000700 1 -- 172.21.15.119:0/2755532870 wait complete. 2024-05-27T09:42:26.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:26 smithi138 ceph-mon[70004]: pgmap v278: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:27.715 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'ceph orch ps' 2024-05-27T09:42:27.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:27 smithi138 ceph-mon[70004]: from='client.34564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:42:27.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:27 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:42:27.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:27 smithi119 ceph-mon[88269]: from='client.34564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:42:27.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:27 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:42:28.162 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:42:28.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:28 smithi138 ceph-mon[70004]: pgmap v279: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:28.936 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:28 smithi119 ceph-mon[88269]: pgmap v279: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:30.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.476+0000 7f7740a00700 1 -- 172.21.15.119:0/3190329411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f773c102fd0 msgr2=0x7f773c106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:42:30.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.476+0000 7f7740a00700 1 --2- 172.21.15.119:0/3190329411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f773c102fd0 0x7f773c106fa0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7728009910 tx=0x7f7728009c20 comp rx=0 tx=0).stop 2024-05-27T09:42:30.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.476+0000 7f7740a00700 1 -- 172.21.15.119:0/3190329411 shutdown_connections 2024-05-27T09:42:30.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.476+0000 7f7740a00700 1 --2- 172.21.15.119:0/3190329411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f773c102fd0 0x7f773c106fa0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:30.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.476+0000 7f7740a00700 1 --2- 172.21.15.119:0/3190329411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f773c102630 0x7f773c102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:30.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.476+0000 7f7740a00700 1 -- 172.21.15.119:0/3190329411 >> 172.21.15.119:0/3190329411 conn(0x7f773c0fdef0 msgr2=0x7f773c100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:42:30.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.476+0000 7f7740a00700 1 -- 172.21.15.119:0/3190329411 shutdown_connections 2024-05-27T09:42:30.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.476+0000 7f7740a00700 1 -- 172.21.15.119:0/3190329411 wait complete. 2024-05-27T09:42:30.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.478+0000 7f7740a00700 1 Processor -- start 2024-05-27T09:42:30.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.478+0000 7f7740a00700 1 -- start start 2024-05-27T09:42:30.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.478+0000 7f7740a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f773c102630 0x7f773c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:42:30.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.478+0000 7f7740a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f773c102fd0 0x7f773c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:42:30.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.478+0000 7f7740a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c198c00 con 0x7f773c102fd0 2024-05-27T09:42:30.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.478+0000 7f7740a00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c19c990 con 0x7f773c102630 2024-05-27T09:42:30.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.479+0000 7f7739400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f773c102fd0 0x7f773c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:42:30.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.479+0000 7f7739400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f773c102fd0 0x7f773c198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41826/0 (socket says 172.21.15.119:41826) 2024-05-27T09:42:30.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.479+0000 7f7739400700 1 -- 172.21.15.119:0/589530802 learned_addr learned my addr 172.21.15.119:0/589530802 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:42:30.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.479+0000 7f7739e00700 1 --2- 172.21.15.119:0/589530802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f773c102630 0x7f773c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:42:30.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.479+0000 7f7739400700 1 -- 172.21.15.119:0/589530802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f773c102630 msgr2=0x7f773c197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:42:30.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.479+0000 7f7739400700 1 --2- 172.21.15.119:0/589530802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f773c102630 0x7f773c197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:30.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.479+0000 7f7739400700 1 -- 172.21.15.119:0/589530802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77280095f0 con 0x7f773c102fd0 2024-05-27T09:42:30.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.480+0000 7f7739400700 1 --2- 172.21.15.119:0/589530802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f773c102fd0 0x7f773c198520 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f772800bda0 tx=0x7f772800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:42:30.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.480+0000 7f7732a00700 1 -- 172.21.15.119:0/589530802 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f772801d070 con 0x7f773c102fd0 2024-05-27T09:42:30.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.480+0000 7f7740a00700 1 -- 172.21.15.119:0/589530802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f773c19cc10 con 0x7f773c102fd0 2024-05-27T09:42:30.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.480+0000 7f7732a00700 1 -- 172.21.15.119:0/589530802 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7728004fb0 con 0x7f773c102fd0 2024-05-27T09:42:30.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.480+0000 7f7732a00700 1 -- 172.21.15.119:0/589530802 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f772800f460 con 0x7f773c102fd0 2024-05-27T09:42:30.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.480+0000 7f7740a00700 1 -- 172.21.15.119:0/589530802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773c19d100 con 0x7f773c102fd0 2024-05-27T09:42:30.482 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.481+0000 7f7740a00700 1 -- 172.21.15.119:0/589530802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f773c04ea50 con 0x7f773c102fd0 2024-05-27T09:42:30.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.483+0000 7f7732a00700 1 -- 172.21.15.119:0/589530802 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f7728022800 con 0x7f773c102fd0 2024-05-27T09:42:30.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.484+0000 7f7732a00700 1 --2- 172.21.15.119:0/589530802 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7720078a40 0x7f772007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:42:30.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.484+0000 7f7732a00700 1 -- 172.21.15.119:0/589530802 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f772809c2c0 con 0x7f773c102fd0 2024-05-27T09:42:30.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.484+0000 7f7739e00700 1 --2- 172.21.15.119:0/589530802 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7720078a40 0x7f772007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:42:30.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.485+0000 7f7739e00700 1 --2- 172.21.15.119:0/589530802 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7720078a40 0x7f772007aef0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f772c005fd0 tx=0x7f772c00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:42:30.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.489+0000 7f7732a00700 1 -- 172.21.15.119:0/589530802 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f7728063f40 con 0x7f773c102fd0 2024-05-27T09:42:30.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.737+0000 7f7740a00700 1 -- 172.21.15.119:0/589530802 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f773c19d660 con 0x7f7720078a40 2024-05-27T09:42:30.738 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:30 smithi119 ceph-mon[88269]: pgmap v280: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:30.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.755+0000 7f7732a00700 1 -- 172.21.15.119:0/589530802 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f773c19d660 con 0x7f7720078a40 2024-05-27T09:42:30.756 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:42:30.756 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (10m) 23s ago 23m 15.2M - 0.25.0 c8568f914cd2 6b90564dcef7 2024-05-27T09:42:30.756 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (39s) 23s ago 23m 74.6M - 19.0.0-3868-ga6507388 850a06a84d28 8fd6d12ce584 2024-05-27T09:42:30.756 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (31s) 26s ago 22m 73.9M - 19.0.0-3868-ga6507388 850a06a84d28 decac6811dac 2024-05-27T09:42:30.757 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (6m) 23s ago 23m 6669k - 19.0.0-3868-ga6507388 850a06a84d28 011d811ce7fc 2024-05-27T09:42:30.757 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (6m) 26s ago 22m 6665k - 19.0.0-3868-ga6507388 850a06a84d28 b546b3ed1afc 2024-05-27T09:42:30.757 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (9m) 23s ago 23m 74.6M - 9.4.12 2bacad6d85d8 b3abdab08db4 2024-05-27T09:42:30.757 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (2m) 23s ago 16m 132M - 19.0.0-3868-ga6507388 850a06a84d28 5a02732ff870 2024-05-27T09:42:30.757 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (118s) 23s ago 16m 137M - 19.0.0-3868-ga6507388 850a06a84d28 3faccd4c18e8 2024-05-27T09:42:30.757 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (67s) 26s ago 16m 79.5M - 19.0.0-3868-ga6507388 850a06a84d28 8da881d730a9 2024-05-27T09:42:30.757 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (91s) 26s ago 16m 79.7M - 19.0.0-3868-ga6507388 850a06a84d28 866712d7b973 2024-05-27T09:42:30.757 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (12m) 23s ago 25m 594M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:42:30.757 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (11m) 26s ago 22m 535M - 19.0.0-3868-ga6507388 850a06a84d28 2fc3a80696fa 2024-05-27T09:42:30.757 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (8m) 23s ago 25m 127M 2048M 19.0.0-3868-ga6507388 850a06a84d28 3e5c7a9ad372 2024-05-27T09:42:30.757 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (7m) 26s ago 22m 113M 2048M 19.0.0-3868-ga6507388 850a06a84d28 e4d471274cf1 2024-05-27T09:42:30.757 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (10m) 23s ago 23m 7977k - 1.5.0 0da6a335fe13 709c37815cff 2024-05-27T09:42:30.757 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (10m) 26s ago 22m 7940k - 1.5.0 0da6a335fe13 999047a1a5bf 2024-05-27T09:42:30.758 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (5m) 23s ago 21m 165M 1216M 19.0.0-3868-ga6507388 850a06a84d28 5a0e162d870e 2024-05-27T09:42:30.758 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (5m) 23s ago 20m 181M 1216M 19.0.0-3868-ga6507388 850a06a84d28 effcc7c53dd3 2024-05-27T09:42:30.758 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (4m) 23s ago 20m 156M 1216M 19.0.0-3868-ga6507388 850a06a84d28 f0bd34ca416f 2024-05-27T09:42:30.758 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (4m) 26s ago 19m 186M 2240M 19.0.0-3868-ga6507388 850a06a84d28 5e8d3e398c21 2024-05-27T09:42:30.758 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (3m) 26s ago 18m 170M 2240M 19.0.0-3868-ga6507388 850a06a84d28 250e443364de 2024-05-27T09:42:30.758 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (2m) 26s ago 18m 162M 2240M 19.0.0-3868-ga6507388 850a06a84d28 90d98c5b2f16 2024-05-27T09:42:30.758 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (10m) 23s ago 23m 77.1M - 2.43.0 a07b618ecd1d cba50454ed21 2024-05-27T09:42:30.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.758+0000 7f7740a00700 1 -- 172.21.15.119:0/589530802 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7720078a40 msgr2=0x7f772007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:42:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.758+0000 7f7740a00700 1 --2- 172.21.15.119:0/589530802 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7720078a40 0x7f772007aef0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f772c005fd0 tx=0x7f772c00a560 comp rx=0 tx=0).stop 2024-05-27T09:42:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.759+0000 7f7740a00700 1 -- 172.21.15.119:0/589530802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f773c102fd0 msgr2=0x7f773c198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:42:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.759+0000 7f7740a00700 1 --2- 172.21.15.119:0/589530802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f773c102fd0 0x7f773c198520 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f772800bda0 tx=0x7f772800be80 comp rx=0 tx=0).stop 2024-05-27T09:42:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.759+0000 7f7740a00700 1 -- 172.21.15.119:0/589530802 shutdown_connections 2024-05-27T09:42:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.759+0000 7f7740a00700 1 --2- 172.21.15.119:0/589530802 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7720078a40 0x7f772007aef0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.759+0000 7f7740a00700 1 --2- 172.21.15.119:0/589530802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f773c102630 0x7f773c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.759+0000 7f7740a00700 1 --2- 172.21.15.119:0/589530802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f773c102fd0 0x7f773c198520 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.759+0000 7f7740a00700 1 -- 172.21.15.119:0/589530802 >> 172.21.15.119:0/589530802 conn(0x7f773c0fdef0 msgr2=0x7f773c0ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:42:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.759+0000 7f7740a00700 1 -- 172.21.15.119:0/589530802 shutdown_connections 2024-05-27T09:42:30.760 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:30.759+0000 7f7740a00700 1 -- 172.21.15.119:0/589530802 wait complete. 2024-05-27T09:42:30.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:30 smithi138 ceph-mon[70004]: pgmap v280: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:31.678 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'ceph orch upgrade status' 2024-05-27T09:42:31.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:31 smithi138 ceph-mon[70004]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:42:31.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:31 smithi119 ceph-mon[88269]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:42:32.082 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:42:32.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:32 smithi138 ceph-mon[70004]: pgmap v281: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:32.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:32 smithi119 ceph-mon[88269]: pgmap v281: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:34.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.334+0000 7fdc43e00700 1 -- 172.21.15.119:0/1340699449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc3c068fd0 msgr2=0x7fdc3c104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:42:34.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.334+0000 7fdc43e00700 1 --2- 172.21.15.119:0/1340699449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc3c068fd0 0x7fdc3c104df0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdc28009910 tx=0x7fdc28009c20 comp rx=0 tx=0).stop 2024-05-27T09:42:34.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.335+0000 7fdc43e00700 1 -- 172.21.15.119:0/1340699449 shutdown_connections 2024-05-27T09:42:34.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.335+0000 7fdc43e00700 1 --2- 172.21.15.119:0/1340699449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc3c068fd0 0x7fdc3c104df0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:34.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.335+0000 7fdc43e00700 1 --2- 172.21.15.119:0/1340699449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc3c068630 0x7fdc3c068a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:34.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.335+0000 7fdc43e00700 1 -- 172.21.15.119:0/1340699449 >> 172.21.15.119:0/1340699449 conn(0x7fdc3c074f50 msgr2=0x7fdc3c075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:42:34.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.335+0000 7fdc43e00700 1 -- 172.21.15.119:0/1340699449 shutdown_connections 2024-05-27T09:42:34.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.335+0000 7fdc43e00700 1 -- 172.21.15.119:0/1340699449 wait complete. 2024-05-27T09:42:34.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.336+0000 7fdc43e00700 1 Processor -- start 2024-05-27T09:42:34.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.337+0000 7fdc43e00700 1 -- start start 2024-05-27T09:42:34.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.337+0000 7fdc43e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc3c068630 0x7fdc3c0ff200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:42:34.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.337+0000 7fdc43e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc3c068fd0 0x7fdc3c0ff740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:42:34.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.337+0000 7fdc43e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c0ffc80 con 0x7fdc3c068fd0 2024-05-27T09:42:34.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.337+0000 7fdc43e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c0ffdc0 con 0x7fdc3c068630 2024-05-27T09:42:34.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.337+0000 7fdc40a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc3c068fd0 0x7fdc3c0ff740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:42:34.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.338+0000 7fdc40a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc3c068fd0 0x7fdc3c0ff740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41848/0 (socket says 172.21.15.119:41848) 2024-05-27T09:42:34.340 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.338+0000 7fdc40a00700 1 -- 172.21.15.119:0/1228994793 learned_addr learned my addr 172.21.15.119:0/1228994793 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:42:34.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.338+0000 7fdc41400700 1 --2- 172.21.15.119:0/1228994793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc3c068630 0x7fdc3c0ff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:42:34.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.338+0000 7fdc40a00700 1 -- 172.21.15.119:0/1228994793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc3c068630 msgr2=0x7fdc3c0ff200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:42:34.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.338+0000 7fdc40a00700 1 --2- 172.21.15.119:0/1228994793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc3c068630 0x7fdc3c0ff200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:34.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.338+0000 7fdc40a00700 1 -- 172.21.15.119:0/1228994793 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc280095f0 con 0x7fdc3c068fd0 2024-05-27T09:42:34.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.338+0000 7fdc41400700 1 --2- 172.21.15.119:0/1228994793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc3c068630 0x7fdc3c0ff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:42:34.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.338+0000 7fdc40a00700 1 --2- 172.21.15.119:0/1228994793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc3c068fd0 0x7fdc3c0ff740 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800bda0 tx=0x7fdc2800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:42:34.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.339+0000 7fdc3a000700 1 -- 172.21.15.119:0/1228994793 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc2801d070 con 0x7fdc3c068fd0 2024-05-27T09:42:34.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.339+0000 7fdc3a000700 1 -- 172.21.15.119:0/1228994793 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc2800f460 con 0x7fdc3c068fd0 2024-05-27T09:42:34.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.339+0000 7fdc3a000700 1 -- 172.21.15.119:0/1228994793 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc28017610 con 0x7fdc3c068fd0 2024-05-27T09:42:34.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.339+0000 7fdc43e00700 1 -- 172.21.15.119:0/1228994793 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3c100040 con 0x7fdc3c068fd0 2024-05-27T09:42:34.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.339+0000 7fdc43e00700 1 -- 172.21.15.119:0/1228994793 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3c1a2320 con 0x7fdc3c068fd0 2024-05-27T09:42:34.342 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.340+0000 7fdc43e00700 1 -- 172.21.15.119:0/1228994793 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc3c1086d0 con 0x7fdc3c068fd0 2024-05-27T09:42:34.344 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.342+0000 7fdc3a000700 1 -- 172.21.15.119:0/1228994793 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fdc2800fa40 con 0x7fdc3c068fd0 2024-05-27T09:42:34.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.343+0000 7fdc3a000700 1 --2- 172.21.15.119:0/1228994793 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdc24078a40 0x7fdc2407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:42:34.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.343+0000 7fdc3a000700 1 -- 172.21.15.119:0/1228994793 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdc2809c9d0 con 0x7fdc3c068fd0 2024-05-27T09:42:34.345 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.344+0000 7fdc41400700 1 --2- 172.21.15.119:0/1228994793 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdc24078a40 0x7fdc2407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:42:34.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.345+0000 7fdc41400700 1 --2- 172.21.15.119:0/1228994793 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdc24078a40 0x7fdc2407aef0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdc30005950 tx=0x7fdc300058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:42:34.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.352+0000 7fdc3a000700 1 -- 172.21.15.119:0/1228994793 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fdc280645c0 con 0x7fdc3c068fd0 2024-05-27T09:42:34.625 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.623+0000 7fdc43e00700 1 -- 172.21.15.119:0/1228994793 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc3c061190 con 0x7fdc24078a40 2024-05-27T09:42:34.627 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.625+0000 7fdc3a000700 1 -- 172.21.15.119:0/1228994793 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fdc3c061190 con 0x7fdc24078a40 2024-05-27T09:42:34.627 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:42:34.627 INFO:teuthology.orchestra.run.smithi119.stdout: "target_image": null, 2024-05-27T09:42:34.627 INFO:teuthology.orchestra.run.smithi119.stdout: "in_progress": false, 2024-05-27T09:42:34.627 INFO:teuthology.orchestra.run.smithi119.stdout: "which": "", 2024-05-27T09:42:34.627 INFO:teuthology.orchestra.run.smithi119.stdout: "services_complete": [], 2024-05-27T09:42:34.627 INFO:teuthology.orchestra.run.smithi119.stdout: "progress": null, 2024-05-27T09:42:34.627 INFO:teuthology.orchestra.run.smithi119.stdout: "message": "", 2024-05-27T09:42:34.627 INFO:teuthology.orchestra.run.smithi119.stdout: "is_paused": false 2024-05-27T09:42:34.627 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:42:34.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.628+0000 7fdc43e00700 1 -- 172.21.15.119:0/1228994793 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdc24078a40 msgr2=0x7fdc2407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:42:34.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.628+0000 7fdc43e00700 1 --2- 172.21.15.119:0/1228994793 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdc24078a40 0x7fdc2407aef0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdc30005950 tx=0x7fdc300058e0 comp rx=0 tx=0).stop 2024-05-27T09:42:34.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.628+0000 7fdc43e00700 1 -- 172.21.15.119:0/1228994793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc3c068fd0 msgr2=0x7fdc3c0ff740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:42:34.630 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.628+0000 7fdc43e00700 1 --2- 172.21.15.119:0/1228994793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc3c068fd0 0x7fdc3c0ff740 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800bda0 tx=0x7fdc2800be80 comp rx=0 tx=0).stop 2024-05-27T09:42:34.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.629+0000 7fdc43e00700 1 -- 172.21.15.119:0/1228994793 shutdown_connections 2024-05-27T09:42:34.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.629+0000 7fdc43e00700 1 --2- 172.21.15.119:0/1228994793 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fdc24078a40 0x7fdc2407aef0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:34.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.629+0000 7fdc43e00700 1 --2- 172.21.15.119:0/1228994793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc3c068630 0x7fdc3c0ff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:34.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.629+0000 7fdc43e00700 1 --2- 172.21.15.119:0/1228994793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc3c068fd0 0x7fdc3c0ff740 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:34.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.629+0000 7fdc43e00700 1 -- 172.21.15.119:0/1228994793 >> 172.21.15.119:0/1228994793 conn(0x7fdc3c074f50 msgr2=0x7fdc3c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:42:34.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.629+0000 7fdc43e00700 1 -- 172.21.15.119:0/1228994793 shutdown_connections 2024-05-27T09:42:34.631 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:34.629+0000 7fdc43e00700 1 -- 172.21.15.119:0/1228994793 wait complete. 2024-05-27T09:42:34.693 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:34 smithi119 ceph-mon[88269]: pgmap v282: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:35.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:34 smithi138 ceph-mon[70004]: pgmap v282: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:35.472 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'ceph health detail' 2024-05-27T09:42:35.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:35 smithi119 ceph-mon[88269]: from='client.34572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:42:35.871 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:42:36.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:35 smithi138 ceph-mon[70004]: from='client.34572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:42:36.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:36 smithi138 ceph-mon[70004]: pgmap v283: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:36.912 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:36 smithi119 ceph-mon[88269]: pgmap v283: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:38.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.253+0000 7f691c200700 1 -- 172.21.15.119:0/2695569482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6914100dd0 msgr2=0x7f6914104da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:42:38.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.253+0000 7f691c200700 1 --2- 172.21.15.119:0/2695569482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6914100dd0 0x7f6914104da0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6900009910 tx=0x7f6900009c20 comp rx=0 tx=0).stop 2024-05-27T09:42:38.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.254+0000 7f691c200700 1 -- 172.21.15.119:0/2695569482 shutdown_connections 2024-05-27T09:42:38.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.254+0000 7f691c200700 1 --2- 172.21.15.119:0/2695569482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6914100dd0 0x7f6914104da0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:38.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.254+0000 7f691c200700 1 --2- 172.21.15.119:0/2695569482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6914100430 0x7f6914100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:38.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.254+0000 7f691c200700 1 -- 172.21.15.119:0/2695569482 >> 172.21.15.119:0/2695569482 conn(0x7f69140fbcf0 msgr2=0x7f69140fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:42:38.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.254+0000 7f691c200700 1 -- 172.21.15.119:0/2695569482 shutdown_connections 2024-05-27T09:42:38.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.254+0000 7f691c200700 1 -- 172.21.15.119:0/2695569482 wait complete. 2024-05-27T09:42:38.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.255+0000 7f691c200700 1 Processor -- start 2024-05-27T09:42:38.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.255+0000 7f691c200700 1 -- start start 2024-05-27T09:42:38.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.255+0000 7f691c200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6914100430 0x7f6914197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:42:38.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.256+0000 7f691c200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6914100dd0 0x7f6914198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:42:38.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.256+0000 7f691c200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6914198b70 con 0x7f6914100430 2024-05-27T09:42:38.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.256+0000 7f691c200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691419c900 con 0x7f6914100dd0 2024-05-27T09:42:38.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.256+0000 7f6919800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6914100430 0x7f6914197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:42:38.258 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.256+0000 7f6919800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6914100430 0x7f6914197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41866/0 (socket says 172.21.15.119:41866) 2024-05-27T09:42:38.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.256+0000 7f6919800700 1 -- 172.21.15.119:0/3832152697 learned_addr learned my addr 172.21.15.119:0/3832152697 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:42:38.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.256+0000 7f6918e00700 1 --2- 172.21.15.119:0/3832152697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6914100dd0 0x7f6914198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:42:38.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.256+0000 7f6919800700 1 -- 172.21.15.119:0/3832152697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6914100dd0 msgr2=0x7f6914198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:42:38.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.256+0000 7f6919800700 1 --2- 172.21.15.119:0/3832152697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6914100dd0 0x7f6914198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:38.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.256+0000 7f6919800700 1 -- 172.21.15.119:0/3832152697 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69000095f0 con 0x7f6914100430 2024-05-27T09:42:38.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.257+0000 7f6919800700 1 --2- 172.21.15.119:0/3832152697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6914100430 0x7f6914197f50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f690400d8d0 tx=0x7f690400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:42:38.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.257+0000 7f6912000700 1 -- 172.21.15.119:0/3832152697 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6904009940 con 0x7f6914100430 2024-05-27T09:42:38.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.257+0000 7f691c200700 1 -- 172.21.15.119:0/3832152697 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f691419cbe0 con 0x7f6914100430 2024-05-27T09:42:38.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.257+0000 7f6912000700 1 -- 172.21.15.119:0/3832152697 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6904010460 con 0x7f6914100430 2024-05-27T09:42:38.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.257+0000 7f6912000700 1 -- 172.21.15.119:0/3832152697 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f690400f5f0 con 0x7f6914100430 2024-05-27T09:42:38.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.257+0000 7f691c200700 1 -- 172.21.15.119:0/3832152697 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f691419d130 con 0x7f6914100430 2024-05-27T09:42:38.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.258+0000 7f691c200700 1 -- 172.21.15.119:0/3832152697 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6914104520 con 0x7f6914100430 2024-05-27T09:42:38.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.260+0000 7f6912000700 1 -- 172.21.15.119:0/3832152697 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f69040105d0 con 0x7f6914100430 2024-05-27T09:42:38.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.260+0000 7f6912000700 1 --2- 172.21.15.119:0/3832152697 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f68f8078a40 0x7f68f807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:42:38.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.261+0000 7f6912000700 1 -- 172.21.15.119:0/3832152697 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f690409ae80 con 0x7f6914100430 2024-05-27T09:42:38.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.261+0000 7f6918e00700 1 --2- 172.21.15.119:0/3832152697 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f68f8078a40 0x7f68f807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:42:38.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.262+0000 7f6918e00700 1 --2- 172.21.15.119:0/3832152697 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f68f8078a40 0x7f68f807aef0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6914199570 tx=0x7f690001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:42:38.267 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.266+0000 7f6912000700 1 -- 172.21.15.119:0/3832152697 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f69040629b0 con 0x7f6914100430 2024-05-27T09:42:38.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.589+0000 7f691c200700 1 -- 172.21.15.119:0/3832152697 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f69141992b0 con 0x7f6914100430 2024-05-27T09:42:38.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.590+0000 7f6912000700 1 -- 172.21.15.119:0/3832152697 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f69141992b0 con 0x7f6914100430 2024-05-27T09:42:38.591 INFO:teuthology.orchestra.run.smithi119.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-27T09:42:38.592 INFO:teuthology.orchestra.run.smithi119.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-27T09:42:38.592 INFO:teuthology.orchestra.run.smithi119.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-27T09:42:38.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.592+0000 7f691c200700 1 -- 172.21.15.119:0/3832152697 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f68f8078a40 msgr2=0x7f68f807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:42:38.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.592+0000 7f691c200700 1 --2- 172.21.15.119:0/3832152697 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f68f8078a40 0x7f68f807aef0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6914199570 tx=0x7f690001a040 comp rx=0 tx=0).stop 2024-05-27T09:42:38.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.592+0000 7f691c200700 1 -- 172.21.15.119:0/3832152697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6914100430 msgr2=0x7f6914197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:42:38.594 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.592+0000 7f691c200700 1 --2- 172.21.15.119:0/3832152697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6914100430 0x7f6914197f50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f690400d8d0 tx=0x7f690400dc90 comp rx=0 tx=0).stop 2024-05-27T09:42:38.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.593+0000 7f691c200700 1 -- 172.21.15.119:0/3832152697 shutdown_connections 2024-05-27T09:42:38.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.593+0000 7f691c200700 1 --2- 172.21.15.119:0/3832152697 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f68f8078a40 0x7f68f807aef0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:38.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.593+0000 7f691c200700 1 --2- 172.21.15.119:0/3832152697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6914100430 0x7f6914197f50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:38.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.593+0000 7f691c200700 1 --2- 172.21.15.119:0/3832152697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6914100dd0 0x7f6914198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:38.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.593+0000 7f691c200700 1 -- 172.21.15.119:0/3832152697 >> 172.21.15.119:0/3832152697 conn(0x7f69140fbcf0 msgr2=0x7f69140fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:42:38.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.593+0000 7f691c200700 1 -- 172.21.15.119:0/3832152697 shutdown_connections 2024-05-27T09:42:38.595 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:38.593+0000 7f691c200700 1 -- 172.21.15.119:0/3832152697 wait complete. 2024-05-27T09:42:38.979 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:38 smithi119 ceph-mon[88269]: pgmap v284: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:38.979 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:38 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/3832152697' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:42:39.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:38 smithi138 ceph-mon[70004]: pgmap v284: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:39.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:38 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/3832152697' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-27T09:42:39.563 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'ceph versions' 2024-05-27T09:42:39.963 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:42:40.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:40 smithi119 ceph-mon[88269]: pgmap v285: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:41.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:40 smithi138 ceph-mon[70004]: pgmap v285: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:42.189 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.188+0000 7f7883600700 1 -- 172.21.15.119:0/951094037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f787c102630 msgr2=0x7f787c102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:42:42.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.188+0000 7f7883600700 1 --2- 172.21.15.119:0/951094037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f787c102630 0x7f787c102a00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7870009910 tx=0x7f7870009c20 comp rx=0 tx=0).stop 2024-05-27T09:42:42.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.188+0000 7f7883600700 1 -- 172.21.15.119:0/951094037 shutdown_connections 2024-05-27T09:42:42.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.188+0000 7f7883600700 1 --2- 172.21.15.119:0/951094037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f787c102fd0 0x7f787c106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:42.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.188+0000 7f7883600700 1 --2- 172.21.15.119:0/951094037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f787c102630 0x7f787c102a00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:42.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.188+0000 7f7883600700 1 -- 172.21.15.119:0/951094037 >> 172.21.15.119:0/951094037 conn(0x7f787c0fdef0 msgr2=0x7f787c100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:42:42.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.189+0000 7f7883600700 1 -- 172.21.15.119:0/951094037 shutdown_connections 2024-05-27T09:42:42.190 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.189+0000 7f7883600700 1 -- 172.21.15.119:0/951094037 wait complete. 2024-05-27T09:42:42.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.190+0000 7f7883600700 1 Processor -- start 2024-05-27T09:42:42.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.190+0000 7f7883600700 1 -- start start 2024-05-27T09:42:42.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.190+0000 7f7883600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f787c102630 0x7f787c197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:42:42.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.190+0000 7f7883600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f787c102fd0 0x7f787c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:42:42.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.190+0000 7f7883600700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c198ba0 con 0x7f787c102fd0 2024-05-27T09:42:42.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.190+0000 7f7883600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c19c930 con 0x7f787c102630 2024-05-27T09:42:42.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.191+0000 7f787be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f787c102fd0 0x7f787c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:42:42.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.191+0000 7f787be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f787c102fd0 0x7f787c1984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59628/0 (socket says 172.21.15.119:59628) 2024-05-27T09:42:42.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.191+0000 7f787be00700 1 -- 172.21.15.119:0/1522794898 learned_addr learned my addr 172.21.15.119:0/1522794898 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:42:42.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.191+0000 7f7880c00700 1 --2- 172.21.15.119:0/1522794898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f787c102630 0x7f787c197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:42:42.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.191+0000 7f787be00700 1 -- 172.21.15.119:0/1522794898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f787c102630 msgr2=0x7f787c197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:42:42.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.191+0000 7f787be00700 1 --2- 172.21.15.119:0/1522794898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f787c102630 0x7f787c197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:42.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.191+0000 7f787be00700 1 -- 172.21.15.119:0/1522794898 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78700095f0 con 0x7f787c102fd0 2024-05-27T09:42:42.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.192+0000 7f787be00700 1 --2- 172.21.15.119:0/1522794898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f787c102fd0 0x7f787c1984c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f786800c8f0 tx=0x7f786800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:42:42.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.192+0000 7f7879600700 1 -- 172.21.15.119:0/1522794898 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7868007a50 con 0x7f787c102fd0 2024-05-27T09:42:42.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.192+0000 7f7883600700 1 -- 172.21.15.119:0/1522794898 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787c19cc10 con 0x7f787c102fd0 2024-05-27T09:42:42.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.192+0000 7f7879600700 1 -- 172.21.15.119:0/1522794898 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7868007bb0 con 0x7f787c102fd0 2024-05-27T09:42:42.193 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.192+0000 7f7879600700 1 -- 172.21.15.119:0/1522794898 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7868018630 con 0x7f787c102fd0 2024-05-27T09:42:42.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.192+0000 7f7883600700 1 -- 172.21.15.119:0/1522794898 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787c19d160 con 0x7f787c102fd0 2024-05-27T09:42:42.194 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.193+0000 7f7883600700 1 -- 172.21.15.119:0/1522794898 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787c10a940 con 0x7f787c102fd0 2024-05-27T09:42:42.195 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.194+0000 7f7879600700 1 -- 172.21.15.119:0/1522794898 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f7868007d20 con 0x7f787c102fd0 2024-05-27T09:42:42.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.195+0000 7f7879600700 1 --2- 172.21.15.119:0/1522794898 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7864078a40 0x7f786407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:42:42.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.196+0000 7f7879600700 1 -- 172.21.15.119:0/1522794898 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f786809ad40 con 0x7f787c102fd0 2024-05-27T09:42:42.196 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.196+0000 7f7880c00700 1 --2- 172.21.15.119:0/1522794898 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7864078a40 0x7f786407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:42:42.197 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.197+0000 7f7880c00700 1 --2- 172.21.15.119:0/1522794898 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7864078a40 0x7f786407aef0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f787000b5c0 tx=0x7f787001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:42:42.201 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.201+0000 7f7879600700 1 -- 172.21.15.119:0/1522794898 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f78680628f0 con 0x7f787c102fd0 2024-05-27T09:42:42.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.539+0000 7f7883600700 1 -- 172.21.15.119:0/1522794898 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f787c04ea50 con 0x7f787c102fd0 2024-05-27T09:42:42.541 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.540+0000 7f7879600700 1 -- 172.21.15.119:0/1522794898 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f7868062290 con 0x7f787c102fd0 2024-05-27T09:42:42.542 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:42:42.542 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:42:42.542 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:42:42.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:42:42.542 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:42:42.542 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:42:42.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:42:42.542 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:42:42.542 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 6 2024-05-27T09:42:42.542 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:42:42.542 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:42:42.543 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 4 2024-05-27T09:42:42.543 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:42:42.543 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:42:42.543 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 14 2024-05-27T09:42:42.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:42:42.543 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:42:42.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.543+0000 7f7883600700 1 -- 172.21.15.119:0/1522794898 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7864078a40 msgr2=0x7f786407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:42:42.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.543+0000 7f7883600700 1 --2- 172.21.15.119:0/1522794898 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7864078a40 0x7f786407aef0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f787000b5c0 tx=0x7f787001a040 comp rx=0 tx=0).stop 2024-05-27T09:42:42.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.543+0000 7f7883600700 1 -- 172.21.15.119:0/1522794898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f787c102fd0 msgr2=0x7f787c1984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:42:42.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.543+0000 7f7883600700 1 --2- 172.21.15.119:0/1522794898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f787c102fd0 0x7f787c1984c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f786800c8f0 tx=0x7f786800ccb0 comp rx=0 tx=0).stop 2024-05-27T09:42:42.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.544+0000 7f7883600700 1 -- 172.21.15.119:0/1522794898 shutdown_connections 2024-05-27T09:42:42.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.544+0000 7f7883600700 1 --2- 172.21.15.119:0/1522794898 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7864078a40 0x7f786407aef0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:42.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.544+0000 7f7883600700 1 --2- 172.21.15.119:0/1522794898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f787c102630 0x7f787c197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:42.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.544+0000 7f7883600700 1 --2- 172.21.15.119:0/1522794898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f787c102fd0 0x7f787c1984c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:42:42.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.544+0000 7f7883600700 1 -- 172.21.15.119:0/1522794898 >> 172.21.15.119:0/1522794898 conn(0x7f787c0fdef0 msgr2=0x7f787c100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:42:42.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.544+0000 7f7883600700 1 -- 172.21.15.119:0/1522794898 shutdown_connections 2024-05-27T09:42:42.545 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:42:42.544+0000 7f7883600700 1 -- 172.21.15.119:0/1522794898 wait complete. 2024-05-27T09:42:42.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:42 smithi119 ceph-mon[88269]: pgmap v286: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:42.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:42 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:42:42.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:42 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/1522794898' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:43.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:42 smithi138 ceph-mon[70004]: pgmap v286: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:43.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:42 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:42:43.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:42 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/1522794898' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:42:43.546 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-05-27T09:42:43.967 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:42:44.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:44 smithi119 ceph-mon[88269]: pgmap v287: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:45.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:44 smithi138 ceph-mon[70004]: pgmap v287: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:46.092 INFO:teuthology.orchestra.run.smithi119.stdout:wait for servicemap items w/ changing names to refresh 2024-05-27T09:42:46.779 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'sleep 60' 2024-05-27T09:42:46.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:46 smithi119 ceph-mon[88269]: pgmap v288: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:47.057 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:46 smithi138 ceph-mon[70004]: pgmap v288: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:47.174 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:42:48.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:48 smithi119 ceph-mon[88269]: pgmap v289: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:49.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:48 smithi138 ceph-mon[70004]: pgmap v289: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:50.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:50 smithi119 ceph-mon[88269]: pgmap v290: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:51.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:50 smithi138 ceph-mon[70004]: pgmap v290: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:52.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:52 smithi119 ceph-mon[88269]: pgmap v291: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:53.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:52 smithi138 ceph-mon[70004]: pgmap v291: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:54.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:54 smithi119 ceph-mon[88269]: pgmap v292: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:55.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:54 smithi138 ceph-mon[70004]: pgmap v292: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:56.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:56 smithi119 ceph-mon[88269]: pgmap v293: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:57.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:56 smithi138 ceph-mon[70004]: pgmap v293: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:57.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:42:57.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:42:58.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:42:58 smithi119 ceph-mon[88269]: pgmap v294: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:42:59.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:42:58 smithi138 ceph-mon[70004]: pgmap v294: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:00.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:00 smithi119 ceph-mon[88269]: pgmap v295: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:01.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:00 smithi138 ceph-mon[70004]: pgmap v295: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:02.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:02 smithi119 ceph-mon[88269]: pgmap v296: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:03.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:02 smithi138 ceph-mon[70004]: pgmap v296: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:05.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:04 smithi138 ceph-mon[70004]: pgmap v297: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:05.220 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:04 smithi119 ceph-mon[88269]: pgmap v297: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:07.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:06 smithi138 ceph-mon[70004]: pgmap v298: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:07.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:06 smithi119 ceph-mon[88269]: pgmap v298: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:09.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:08 smithi138 ceph-mon[70004]: pgmap v299: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:09.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:08 smithi119 ceph-mon[88269]: pgmap v299: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:11.112 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:10 smithi119 ceph-mon[88269]: pgmap v300: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:11.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:10 smithi138 ceph-mon[70004]: pgmap v300: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:13.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:12 smithi138 ceph-mon[70004]: pgmap v301: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:13.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:43:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:12 smithi119 ceph-mon[88269]: pgmap v301: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:43:15.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:14 smithi138 ceph-mon[70004]: pgmap v302: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:15.220 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:14 smithi119 ceph-mon[88269]: pgmap v302: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:17.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:16 smithi138 ceph-mon[70004]: pgmap v303: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:17.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:16 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:43:17.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:16 smithi119 ceph-mon[88269]: pgmap v303: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:17.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:16 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:43:19.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:18 smithi138 ceph-mon[70004]: pgmap v304: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:19.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:18 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:43:19.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:18 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:43:19.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:18 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:43:19.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:18 smithi119 ceph-mon[88269]: pgmap v304: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:19.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:18 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:43:19.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:18 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:43:19.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:18 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:43:21.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:20 smithi138 ceph-mon[70004]: pgmap v305: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:21.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:20 smithi119 ceph-mon[88269]: pgmap v305: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:23.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:22 smithi138 ceph-mon[70004]: pgmap v306: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:23.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:22 smithi119 ceph-mon[88269]: pgmap v306: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:25.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:24 smithi138 ceph-mon[70004]: pgmap v307: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:25.220 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:24 smithi119 ceph-mon[88269]: pgmap v307: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:27.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:26 smithi138 ceph-mon[70004]: pgmap v308: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:27.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:26 smithi119 ceph-mon[88269]: pgmap v308: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:28.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:27 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:43:28.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:27 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:43:29.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:28 smithi138 ceph-mon[70004]: pgmap v309: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:29.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:28 smithi119 ceph-mon[88269]: pgmap v309: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:31.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:30 smithi138 ceph-mon[70004]: pgmap v310: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:31.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:30 smithi119 ceph-mon[88269]: pgmap v310: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:33.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:32 smithi138 ceph-mon[70004]: pgmap v311: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:33.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:32 smithi119 ceph-mon[88269]: pgmap v311: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:35.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:34 smithi138 ceph-mon[70004]: pgmap v312: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:35.219 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:34 smithi119 ceph-mon[88269]: pgmap v312: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:37.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:36 smithi138 ceph-mon[70004]: pgmap v313: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:37.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:36 smithi119 ceph-mon[88269]: pgmap v313: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:39.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:38 smithi138 ceph-mon[70004]: pgmap v314: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:39.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:38 smithi119 ceph-mon[88269]: pgmap v314: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:41.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:40 smithi138 ceph-mon[70004]: pgmap v315: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:41.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:40 smithi119 ceph-mon[88269]: pgmap v315: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:43.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:42 smithi138 ceph-mon[70004]: pgmap v316: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:43.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:42 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:43:43.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:42 smithi119 ceph-mon[88269]: pgmap v316: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:43.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:42 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:43:45.219 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:44 smithi119 ceph-mon[88269]: pgmap v317: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:45.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:44 smithi138 ceph-mon[70004]: pgmap v317: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:47.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:46 smithi119 ceph-mon[88269]: pgmap v318: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:47.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:46 smithi138 ceph-mon[70004]: pgmap v318: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:49.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:48 smithi119 ceph-mon[88269]: pgmap v319: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:49.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:48 smithi138 ceph-mon[70004]: pgmap v319: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:50.271 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'ceph orch ps' 2024-05-27T09:43:50.684 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:43:51.092 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:50 smithi119 ceph-mon[88269]: pgmap v320: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:51.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:50 smithi138 ceph-mon[70004]: pgmap v320: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:52.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.868+0000 7f37aac00700 1 -- 172.21.15.119:0/3978199767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37a410b700 msgr2=0x7f37a410bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:43:52.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.868+0000 7f37aac00700 1 --2- 172.21.15.119:0/3978199767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37a410b700 0x7f37a410bad0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3794009910 tx=0x7f3794009c20 comp rx=0 tx=0).stop 2024-05-27T09:43:52.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.868+0000 7f37aac00700 1 -- 172.21.15.119:0/3978199767 shutdown_connections 2024-05-27T09:43:52.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.868+0000 7f37aac00700 1 --2- 172.21.15.119:0/3978199767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37a410c010 0x7f37a4106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:43:52.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.868+0000 7f37aac00700 1 --2- 172.21.15.119:0/3978199767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37a410b700 0x7f37a410bad0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:43:52.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.868+0000 7f37aac00700 1 -- 172.21.15.119:0/3978199767 >> 172.21.15.119:0/3978199767 conn(0x7f37a4076000 msgr2=0x7f37a4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:43:52.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.868+0000 7f37aac00700 1 -- 172.21.15.119:0/3978199767 shutdown_connections 2024-05-27T09:43:52.870 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.868+0000 7f37aac00700 1 -- 172.21.15.119:0/3978199767 wait complete. 2024-05-27T09:43:52.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.869+0000 7f37aac00700 1 Processor -- start 2024-05-27T09:43:52.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.870+0000 7f37aac00700 1 -- start start 2024-05-27T09:43:52.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.870+0000 7f37aac00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37a410b700 0x7f37a419c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:43:52.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.870+0000 7f37aac00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37a410c010 0x7f37a419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:43:52.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.870+0000 7f37aac00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a419ced0 con 0x7f37a410c010 2024-05-27T09:43:52.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.870+0000 7f37aac00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a41a0c60 con 0x7f37a410b700 2024-05-27T09:43:52.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.870+0000 7f37a3400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37a410c010 0x7f37a419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:43:52.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.870+0000 7f37a3400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37a410c010 0x7f37a419c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34368/0 (socket says 172.21.15.119:34368) 2024-05-27T09:43:52.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.871+0000 7f37a3400700 1 -- 172.21.15.119:0/2981958366 learned_addr learned my addr 172.21.15.119:0/2981958366 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:43:52.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.871+0000 7f37a3e00700 1 --2- 172.21.15.119:0/2981958366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37a410b700 0x7f37a419c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:43:52.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.871+0000 7f37a3400700 1 -- 172.21.15.119:0/2981958366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37a410b700 msgr2=0x7f37a419c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:43:52.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.871+0000 7f37a3400700 1 --2- 172.21.15.119:0/2981958366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37a410b700 0x7f37a419c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:43:52.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.871+0000 7f37a3400700 1 -- 172.21.15.119:0/2981958366 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37940095f0 con 0x7f37a410c010 2024-05-27T09:43:52.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.871+0000 7f37a3400700 1 --2- 172.21.15.119:0/2981958366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37a410c010 0x7f37a419c7f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f379000d900 tx=0x7f379000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:43:52.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.872+0000 7f37a0c00700 1 -- 172.21.15.119:0/2981958366 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3790009900 con 0x7f37a410c010 2024-05-27T09:43:52.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.872+0000 7f37a0c00700 1 -- 172.21.15.119:0/2981958366 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3790010460 con 0x7f37a410c010 2024-05-27T09:43:52.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.872+0000 7f37aac00700 1 -- 172.21.15.119:0/2981958366 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a41a0f40 con 0x7f37a410c010 2024-05-27T09:43:52.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.872+0000 7f37a0c00700 1 -- 172.21.15.119:0/2981958366 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f379000f5f0 con 0x7f37a410c010 2024-05-27T09:43:52.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.872+0000 7f37aac00700 1 -- 172.21.15.119:0/2981958366 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a41a1510 con 0x7f37a410c010 2024-05-27T09:43:52.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.873+0000 7f37aac00700 1 -- 172.21.15.119:0/2981958366 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a404ea50 con 0x7f37a410c010 2024-05-27T09:43:52.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.875+0000 7f37a0c00700 1 -- 172.21.15.119:0/2981958366 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f37900105d0 con 0x7f37a410c010 2024-05-27T09:43:52.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.876+0000 7f37a0c00700 1 --2- 172.21.15.119:0/2981958366 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3788078a40 0x7f378807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:43:52.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.876+0000 7f37a0c00700 1 -- 172.21.15.119:0/2981958366 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f379009abd0 con 0x7f37a410c010 2024-05-27T09:43:52.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.876+0000 7f37a3e00700 1 --2- 172.21.15.119:0/2981958366 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3788078a40 0x7f378807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:43:52.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.877+0000 7f37a3e00700 1 --2- 172.21.15.119:0/2981958366 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3788078a40 0x7f378807aef0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f379400b5c0 tx=0x7f3794005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:43:52.880 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:52.880+0000 7f37a0c00700 1 -- 172.21.15.119:0/2981958366 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f3790062800 con 0x7f37a410c010 2024-05-27T09:43:52.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:52 smithi119 ceph-mon[88269]: pgmap v321: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:53.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:53.126+0000 7f37aac00700 1 -- 172.21.15.119:0/2981958366 --> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f37a41a1a70 con 0x7f3788078a40 2024-05-27T09:43:53.157 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:53.157+0000 7f37a0c00700 1 -- 172.21.15.119:0/2981958366 <== mgr.34104 v2:172.21.15.119:6800/1449054506 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f37a41a1a70 con 0x7f3788078a40 2024-05-27T09:43:53.158 INFO:teuthology.orchestra.run.smithi119.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T09:43:53.159 INFO:teuthology.orchestra.run.smithi119.stdout:alertmanager.smithi119 smithi119 *:9093,9094 running (11m) 106s ago 24m 15.2M - 0.25.0 c8568f914cd2 6b90564dcef7 2024-05-27T09:43:53.159 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi119 smithi119 running (2m) 106s ago 25m 74.6M - 19.0.0-3868-ga6507388 850a06a84d28 8fd6d12ce584 2024-05-27T09:43:53.159 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-exporter.smithi138 smithi138 running (113s) 108s ago 23m 73.9M - 19.0.0-3868-ga6507388 850a06a84d28 decac6811dac 2024-05-27T09:43:53.159 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi119 smithi119 running (7m) 106s ago 25m 6669k - 19.0.0-3868-ga6507388 850a06a84d28 011d811ce7fc 2024-05-27T09:43:53.159 INFO:teuthology.orchestra.run.smithi119.stdout:crash.smithi138 smithi138 running (7m) 108s ago 23m 6665k - 19.0.0-3868-ga6507388 850a06a84d28 b546b3ed1afc 2024-05-27T09:43:53.159 INFO:teuthology.orchestra.run.smithi119.stdout:grafana.smithi119 smithi119 *:3000 running (10m) 106s ago 24m 74.6M - 9.4.12 2bacad6d85d8 b3abdab08db4 2024-05-27T09:43:53.159 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.nghcey smithi119 running (3m) 106s ago 17m 132M - 19.0.0-3868-ga6507388 850a06a84d28 5a02732ff870 2024-05-27T09:43:53.159 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi119.ppmzzk smithi119 running (3m) 106s ago 17m 137M - 19.0.0-3868-ga6507388 850a06a84d28 3faccd4c18e8 2024-05-27T09:43:53.159 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.alszpk smithi138 running (2m) 108s ago 17m 79.5M - 19.0.0-3868-ga6507388 850a06a84d28 8da881d730a9 2024-05-27T09:43:53.159 INFO:teuthology.orchestra.run.smithi119.stdout:mds.cephfs.smithi138.ndfkhd smithi138 running (2m) 108s ago 17m 79.7M - 19.0.0-3868-ga6507388 850a06a84d28 866712d7b973 2024-05-27T09:43:53.159 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi119.ncwmog smithi119 *:8443,9283,8765 running (13m) 106s ago 26m 594M - 19.0.0-3868-ga6507388 850a06a84d28 356eb6ed443a 2024-05-27T09:43:53.159 INFO:teuthology.orchestra.run.smithi119.stdout:mgr.smithi138.dxsuqf smithi138 *:8443,9283,8765 running (12m) 108s ago 23m 535M - 19.0.0-3868-ga6507388 850a06a84d28 2fc3a80696fa 2024-05-27T09:43:53.160 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi119 smithi119 running (9m) 106s ago 26m 127M 2048M 19.0.0-3868-ga6507388 850a06a84d28 3e5c7a9ad372 2024-05-27T09:43:53.160 INFO:teuthology.orchestra.run.smithi119.stdout:mon.smithi138 smithi138 running (8m) 108s ago 23m 113M 2048M 19.0.0-3868-ga6507388 850a06a84d28 e4d471274cf1 2024-05-27T09:43:53.160 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi119 smithi119 *:9100 running (12m) 106s ago 25m 7977k - 1.5.0 0da6a335fe13 709c37815cff 2024-05-27T09:43:53.160 INFO:teuthology.orchestra.run.smithi119.stdout:node-exporter.smithi138 smithi138 *:9100 running (12m) 108s ago 23m 7940k - 1.5.0 0da6a335fe13 999047a1a5bf 2024-05-27T09:43:53.160 INFO:teuthology.orchestra.run.smithi119.stdout:osd.0 smithi119 running (7m) 106s ago 22m 165M 1216M 19.0.0-3868-ga6507388 850a06a84d28 5a0e162d870e 2024-05-27T09:43:53.160 INFO:teuthology.orchestra.run.smithi119.stdout:osd.1 smithi119 running (6m) 106s ago 22m 181M 1216M 19.0.0-3868-ga6507388 850a06a84d28 effcc7c53dd3 2024-05-27T09:43:53.160 INFO:teuthology.orchestra.run.smithi119.stdout:osd.2 smithi119 running (6m) 106s ago 21m 156M 1216M 19.0.0-3868-ga6507388 850a06a84d28 f0bd34ca416f 2024-05-27T09:43:53.160 INFO:teuthology.orchestra.run.smithi119.stdout:osd.3 smithi138 running (5m) 108s ago 20m 186M 2240M 19.0.0-3868-ga6507388 850a06a84d28 5e8d3e398c21 2024-05-27T09:43:53.160 INFO:teuthology.orchestra.run.smithi119.stdout:osd.4 smithi138 running (4m) 108s ago 20m 170M 2240M 19.0.0-3868-ga6507388 850a06a84d28 250e443364de 2024-05-27T09:43:53.160 INFO:teuthology.orchestra.run.smithi119.stdout:osd.5 smithi138 running (4m) 108s ago 19m 162M 2240M 19.0.0-3868-ga6507388 850a06a84d28 90d98c5b2f16 2024-05-27T09:43:53.160 INFO:teuthology.orchestra.run.smithi119.stdout:prometheus.smithi119 smithi119 *:9095 running (11m) 106s ago 24m 77.1M - 2.43.0 a07b618ecd1d cba50454ed21 2024-05-27T09:43:53.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:53.160+0000 7f37aac00700 1 -- 172.21.15.119:0/2981958366 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3788078a40 msgr2=0x7f378807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:43:53.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:53.160+0000 7f37aac00700 1 --2- 172.21.15.119:0/2981958366 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3788078a40 0x7f378807aef0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f379400b5c0 tx=0x7f3794005960 comp rx=0 tx=0).stop 2024-05-27T09:43:53.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:53.160+0000 7f37aac00700 1 -- 172.21.15.119:0/2981958366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37a410c010 msgr2=0x7f37a419c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:43:53.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:53.160+0000 7f37aac00700 1 --2- 172.21.15.119:0/2981958366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37a410c010 0x7f37a419c7f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f379000d900 tx=0x7f379000dcc0 comp rx=0 tx=0).stop 2024-05-27T09:43:53.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:53.160+0000 7f37aac00700 1 -- 172.21.15.119:0/2981958366 shutdown_connections 2024-05-27T09:43:53.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:53.160+0000 7f37aac00700 1 --2- 172.21.15.119:0/2981958366 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3788078a40 0x7f378807aef0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:43:53.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:53.160+0000 7f37aac00700 1 --2- 172.21.15.119:0/2981958366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37a410b700 0x7f37a419c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:43:53.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:53.160+0000 7f37aac00700 1 --2- 172.21.15.119:0/2981958366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37a410c010 0x7f37a419c7f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:43:53.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:53.160+0000 7f37aac00700 1 -- 172.21.15.119:0/2981958366 >> 172.21.15.119:0/2981958366 conn(0x7f37a4076000 msgr2=0x7f37a4076880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:43:53.161 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:53.160+0000 7f37aac00700 1 -- 172.21.15.119:0/2981958366 shutdown_connections 2024-05-27T09:43:53.162 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:53.160+0000 7f37aac00700 1 -- 172.21.15.119:0/2981958366 wait complete. 2024-05-27T09:43:53.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:52 smithi138 ceph-mon[70004]: pgmap v321: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:53.979 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'ceph versions' 2024-05-27T09:43:54.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:53 smithi119 ceph-mon[88269]: from='client.34584 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:43:54.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:53 smithi138 ceph-mon[70004]: from='client.34584 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T09:43:54.387 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:43:55.221 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:54 smithi119 ceph-mon[88269]: pgmap v322: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:55.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:54 smithi138 ceph-mon[70004]: pgmap v322: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:56.599 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.597+0000 7f3daac00700 1 -- 172.21.15.119:0/3824975061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3da40683d0 msgr2=0x7f3da40687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:43:56.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.597+0000 7f3daac00700 1 --2- 172.21.15.119:0/3824975061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3da40683d0 0x7f3da40687a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009910 tx=0x7f3d98009c20 comp rx=0 tx=0).stop 2024-05-27T09:43:56.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.597+0000 7f3daac00700 1 -- 172.21.15.119:0/3824975061 shutdown_connections 2024-05-27T09:43:56.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.597+0000 7f3daac00700 1 --2- 172.21.15.119:0/3824975061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3da4068ce0 0x7f3da410c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:43:56.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.597+0000 7f3daac00700 1 --2- 172.21.15.119:0/3824975061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3da40683d0 0x7f3da40687a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:43:56.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.597+0000 7f3daac00700 1 -- 172.21.15.119:0/3824975061 >> 172.21.15.119:0/3824975061 conn(0x7f3da4074d00 msgr2=0x7f3da4075100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:43:56.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.598+0000 7f3daac00700 1 -- 172.21.15.119:0/3824975061 shutdown_connections 2024-05-27T09:43:56.600 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.598+0000 7f3daac00700 1 -- 172.21.15.119:0/3824975061 wait complete. 2024-05-27T09:43:56.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.599+0000 7f3daac00700 1 Processor -- start 2024-05-27T09:43:56.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.599+0000 7f3daac00700 1 -- start start 2024-05-27T09:43:56.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.599+0000 7f3daac00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3da40683d0 0x7f3da4197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:43:56.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.599+0000 7f3daac00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3da4068ce0 0x7f3da4198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:43:56.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.599+0000 7f3daac00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da4198a10 con 0x7f3da4068ce0 2024-05-27T09:43:56.601 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.599+0000 7f3daac00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da419c7a0 con 0x7f3da40683d0 2024-05-27T09:43:56.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.599+0000 7f3da8e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3da4068ce0 0x7f3da4198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:43:56.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.600+0000 7f3da8e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3da4068ce0 0x7f3da4198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34388/0 (socket says 172.21.15.119:34388) 2024-05-27T09:43:56.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.600+0000 7f3da8e00700 1 -- 172.21.15.119:0/2765117147 learned_addr learned my addr 172.21.15.119:0/2765117147 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:43:56.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.600+0000 7f3da9800700 1 --2- 172.21.15.119:0/2765117147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3da40683d0 0x7f3da4197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:43:56.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.600+0000 7f3da8e00700 1 -- 172.21.15.119:0/2765117147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3da40683d0 msgr2=0x7f3da4197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:43:56.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.600+0000 7f3da8e00700 1 --2- 172.21.15.119:0/2765117147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3da40683d0 0x7f3da4197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:43:56.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.600+0000 7f3da8e00700 1 -- 172.21.15.119:0/2765117147 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d980095f0 con 0x7f3da4068ce0 2024-05-27T09:43:56.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.600+0000 7f3da8e00700 1 --2- 172.21.15.119:0/2765117147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3da4068ce0 0x7f3da4198330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000b4b0 tx=0x7f3d9000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:43:56.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.601+0000 7f3da2000700 1 -- 172.21.15.119:0/2765117147 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d90011800 con 0x7f3da4068ce0 2024-05-27T09:43:56.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.601+0000 7f3da2000700 1 -- 172.21.15.119:0/2765117147 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d90011e20 con 0x7f3da4068ce0 2024-05-27T09:43:56.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.601+0000 7f3daac00700 1 -- 172.21.15.119:0/2765117147 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da419ca80 con 0x7f3da4068ce0 2024-05-27T09:43:56.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.601+0000 7f3daac00700 1 -- 172.21.15.119:0/2765117147 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da419cfd0 con 0x7f3da4068ce0 2024-05-27T09:43:56.603 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.601+0000 7f3da2000700 1 -- 172.21.15.119:0/2765117147 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d9001a400 con 0x7f3da4068ce0 2024-05-27T09:43:56.604 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.602+0000 7f3daac00700 1 -- 172.21.15.119:0/2765117147 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da4109d50 con 0x7f3da4068ce0 2024-05-27T09:43:56.605 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.604+0000 7f3da2000700 1 -- 172.21.15.119:0/2765117147 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f3d90010380 con 0x7f3da4068ce0 2024-05-27T09:43:56.606 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.605+0000 7f3da2000700 1 --2- 172.21.15.119:0/2765117147 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3d8c078a40 0x7f3d8c07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:43:56.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.605+0000 7f3da2000700 1 -- 172.21.15.119:0/2765117147 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3d9009a9b0 con 0x7f3da4068ce0 2024-05-27T09:43:56.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.605+0000 7f3da9800700 1 --2- 172.21.15.119:0/2765117147 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3d8c078a40 0x7f3d8c07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:43:56.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.606+0000 7f3da9800700 1 --2- 172.21.15.119:0/2765117147 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3d8c078a40 0x7f3d8c07aef0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3d98006010 tx=0x7f3d9801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:43:56.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.610+0000 7f3da2000700 1 -- 172.21.15.119:0/2765117147 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f3d900624e0 con 0x7f3da4068ce0 2024-05-27T09:43:56.951 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.949+0000 7f3daac00700 1 -- 172.21.15.119:0/2765117147 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3da404f2a0 con 0x7f3da4068ce0 2024-05-27T09:43:56.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.951+0000 7f3da2000700 1 -- 172.21.15.119:0/2765117147 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f3d90061e80 con 0x7f3da4068ce0 2024-05-27T09:43:56.952 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-05-27T09:43:56.953 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": { 2024-05-27T09:43:56.953 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:43:56.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:43:56.953 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": { 2024-05-27T09:43:56.953 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 2 2024-05-27T09:43:56.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:43:56.953 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": { 2024-05-27T09:43:56.953 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 6 2024-05-27T09:43:56.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:43:56.953 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-05-27T09:43:56.953 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 4 2024-05-27T09:43:56.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-05-27T09:43:56.953 INFO:teuthology.orchestra.run.smithi119.stdout: "overall": { 2024-05-27T09:43:56.954 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)": 14 2024-05-27T09:43:56.954 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-05-27T09:43:56.954 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-05-27T09:43:56.955 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.954+0000 7f3daac00700 1 -- 172.21.15.119:0/2765117147 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3d8c078a40 msgr2=0x7f3d8c07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:43:56.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.954+0000 7f3daac00700 1 --2- 172.21.15.119:0/2765117147 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3d8c078a40 0x7f3d8c07aef0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3d98006010 tx=0x7f3d9801a040 comp rx=0 tx=0).stop 2024-05-27T09:43:56.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.954+0000 7f3daac00700 1 -- 172.21.15.119:0/2765117147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3da4068ce0 msgr2=0x7f3da4198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:43:56.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.954+0000 7f3daac00700 1 --2- 172.21.15.119:0/2765117147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3da4068ce0 0x7f3da4198330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000b4b0 tx=0x7f3d9000b870 comp rx=0 tx=0).stop 2024-05-27T09:43:56.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.954+0000 7f3daac00700 1 -- 172.21.15.119:0/2765117147 shutdown_connections 2024-05-27T09:43:56.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.954+0000 7f3daac00700 1 --2- 172.21.15.119:0/2765117147 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3d8c078a40 0x7f3d8c07aef0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:43:56.956 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.954+0000 7f3daac00700 1 --2- 172.21.15.119:0/2765117147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3da40683d0 0x7f3da4197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:43:56.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.954+0000 7f3daac00700 1 --2- 172.21.15.119:0/2765117147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3da4068ce0 0x7f3da4198330 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:43:56.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.954+0000 7f3daac00700 1 -- 172.21.15.119:0/2765117147 >> 172.21.15.119:0/2765117147 conn(0x7f3da4074d00 msgr2=0x7f3da40fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:43:56.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.954+0000 7f3daac00700 1 -- 172.21.15.119:0/2765117147 shutdown_connections 2024-05-27T09:43:56.957 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:43:56.954+0000 7f3daac00700 1 -- 172.21.15.119:0/2765117147 wait complete. 2024-05-27T09:43:57.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:56 smithi119 ceph-mon[88269]: pgmap v323: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:57.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:56 smithi138 ceph-mon[70004]: pgmap v323: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:57.953 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-05-27T09:43:58.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:57 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/2765117147' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:43:58.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:43:58.287 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:57 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/2765117147' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:43:58.287 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:43:58.372 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:43:59.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:43:58 smithi119 ceph-mon[88269]: pgmap v324: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:43:59.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:43:58 smithi138 ceph-mon[70004]: pgmap v324: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:00.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.467+0000 7f1154400700 1 -- 172.21.15.119:0/3118785482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f114c073a00 msgr2=0x7f114c10c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:00.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.467+0000 7f1154400700 1 --2- 172.21.15.119:0/3118785482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f114c073a00 0x7f114c10c450 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1138009910 tx=0x7f1138009c20 comp rx=0 tx=0).stop 2024-05-27T09:44:00.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.467+0000 7f1154400700 1 -- 172.21.15.119:0/3118785482 shutdown_connections 2024-05-27T09:44:00.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.467+0000 7f1154400700 1 --2- 172.21.15.119:0/3118785482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f114c073a00 0x7f114c10c450 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:00.469 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.467+0000 7f1154400700 1 --2- 172.21.15.119:0/3118785482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f114c0730f0 0x7f114c0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:00.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.467+0000 7f1154400700 1 -- 172.21.15.119:0/3118785482 >> 172.21.15.119:0/3118785482 conn(0x7f114c0fbc30 msgr2=0x7f114c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:00.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.467+0000 7f1154400700 1 -- 172.21.15.119:0/3118785482 shutdown_connections 2024-05-27T09:44:00.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.467+0000 7f1154400700 1 -- 172.21.15.119:0/3118785482 wait complete. 2024-05-27T09:44:00.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.468+0000 7f1154400700 1 Processor -- start 2024-05-27T09:44:00.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.468+0000 7f1154400700 1 -- start start 2024-05-27T09:44:00.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.468+0000 7f1154400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f114c0730f0 0x7f114c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:00.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.469+0000 7f1154400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f114c073a00 0x7f114c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:00.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.469+0000 7f1154400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c19c7c0 con 0x7f114c0730f0 2024-05-27T09:44:00.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.469+0000 7f1154400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c19c930 con 0x7f114c073a00 2024-05-27T09:44:00.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.469+0000 7f1151a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f114c0730f0 0x7f114c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:00.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.469+0000 7f1151a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f114c0730f0 0x7f114c197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34014/0 (socket says 172.21.15.119:34014) 2024-05-27T09:44:00.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.469+0000 7f1151a00700 1 -- 172.21.15.119:0/278565552 learned_addr learned my addr 172.21.15.119:0/278565552 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:44:00.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.469+0000 7f1151000700 1 --2- 172.21.15.119:0/278565552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f114c073a00 0x7f114c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:00.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.469+0000 7f1151a00700 1 -- 172.21.15.119:0/278565552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f114c073a00 msgr2=0x7f114c198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:00.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.469+0000 7f1151a00700 1 --2- 172.21.15.119:0/278565552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f114c073a00 0x7f114c198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:00.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.469+0000 7f1151a00700 1 -- 172.21.15.119:0/278565552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11380095f0 con 0x7f114c0730f0 2024-05-27T09:44:00.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.470+0000 7f1151a00700 1 --2- 172.21.15.119:0/278565552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f114c0730f0 0x7f114c197e10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f114000c8a0 tx=0x7f114000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:00.472 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.470+0000 7f114a000700 1 -- 172.21.15.119:0/278565552 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f114000cea0 con 0x7f114c0730f0 2024-05-27T09:44:00.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.470+0000 7f114a000700 1 -- 172.21.15.119:0/278565552 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1140004830 con 0x7f114c0730f0 2024-05-27T09:44:00.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.470+0000 7f1154400700 1 -- 172.21.15.119:0/278565552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f114c19cc10 con 0x7f114c0730f0 2024-05-27T09:44:00.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.470+0000 7f114a000700 1 -- 172.21.15.119:0/278565552 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11400055b0 con 0x7f114c0730f0 2024-05-27T09:44:00.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.470+0000 7f1154400700 1 -- 172.21.15.119:0/278565552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114c19d160 con 0x7f114c0730f0 2024-05-27T09:44:00.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.471+0000 7f1154400700 1 -- 172.21.15.119:0/278565552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f114c109b50 con 0x7f114c0730f0 2024-05-27T09:44:00.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.473+0000 7f114a000700 1 -- 172.21.15.119:0/278565552 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f11400049a0 con 0x7f114c0730f0 2024-05-27T09:44:00.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.474+0000 7f114a000700 1 --2- 172.21.15.119:0/278565552 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1134078a40 0x7f113407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:00.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.474+0000 7f114a000700 1 -- 172.21.15.119:0/278565552 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f114009aac0 con 0x7f114c0730f0 2024-05-27T09:44:00.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.474+0000 7f1151000700 1 --2- 172.21.15.119:0/278565552 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1134078a40 0x7f113407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:00.476 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.475+0000 7f1151000700 1 --2- 172.21.15.119:0/278565552 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1134078a40 0x7f113407aef0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1138000c00 tx=0x7f1138005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:00.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.480+0000 7f114a000700 1 -- 172.21.15.119:0/278565552 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f11400625f0 con 0x7f114c0730f0 2024-05-27T09:44:00.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.819+0000 7f1154400700 1 -- 172.21.15.119:0/278565552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f114c19d3e0 con 0x7f114c0730f0 2024-05-27T09:44:00.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.820+0000 7f114a000700 1 -- 172.21.15.119:0/278565552 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f1140004c50 con 0x7f114c0730f0 2024-05-27T09:44:00.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.823+0000 7f1154400700 1 -- 172.21.15.119:0/278565552 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1134078a40 msgr2=0x7f113407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:00.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.823+0000 7f1154400700 1 --2- 172.21.15.119:0/278565552 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1134078a40 0x7f113407aef0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1138000c00 tx=0x7f1138005fd0 comp rx=0 tx=0).stop 2024-05-27T09:44:00.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.823+0000 7f1154400700 1 -- 172.21.15.119:0/278565552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f114c0730f0 msgr2=0x7f114c197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:00.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.823+0000 7f1154400700 1 --2- 172.21.15.119:0/278565552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f114c0730f0 0x7f114c197e10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f114000c8a0 tx=0x7f114000cc60 comp rx=0 tx=0).stop 2024-05-27T09:44:00.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.823+0000 7f1154400700 1 -- 172.21.15.119:0/278565552 shutdown_connections 2024-05-27T09:44:00.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.823+0000 7f1154400700 1 --2- 172.21.15.119:0/278565552 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1134078a40 0x7f113407aef0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:00.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.823+0000 7f1154400700 1 --2- 172.21.15.119:0/278565552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f114c0730f0 0x7f114c197e10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:00.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.823+0000 7f1154400700 1 --2- 172.21.15.119:0/278565552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f114c073a00 0x7f114c198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:00.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.823+0000 7f1154400700 1 -- 172.21.15.119:0/278565552 >> 172.21.15.119:0/278565552 conn(0x7f114c0fbc30 msgr2=0x7f114c106c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:00.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.824+0000 7f1154400700 1 -- 172.21.15.119:0/278565552 shutdown_connections 2024-05-27T09:44:00.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:00.824+0000 7f1154400700 1 -- 172.21.15.119:0/278565552 wait complete. 2024-05-27T09:44:00.842 INFO:teuthology.orchestra.run.smithi119.stdout:true 2024-05-27T09:44:01.220 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:00 smithi119 ceph-mon[88269]: pgmap v325: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:01.220 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:00 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/278565552' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:44:01.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:00 smithi138 ceph-mon[70004]: pgmap v325: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:01.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:00 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/278565552' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:44:01.612 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -e sha1=a6507388dd5057528934822c0163b0c347ef1d5d -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-05-27T09:44:02.025 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:44:03.168 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:02 smithi119 ceph-mon[88269]: pgmap v326: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:03.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:02 smithi138 ceph-mon[70004]: pgmap v326: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:04.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.344+0000 7f8d85c00700 1 -- 172.21.15.119:0/748905082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d80102fd0 msgr2=0x7f8d80106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:04.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.344+0000 7f8d85c00700 1 --2- 172.21.15.119:0/748905082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d80102fd0 0x7f8d80106fa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009910 tx=0x7f8d6c009c20 comp rx=0 tx=0).stop 2024-05-27T09:44:04.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.344+0000 7f8d85c00700 1 -- 172.21.15.119:0/748905082 shutdown_connections 2024-05-27T09:44:04.346 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.344+0000 7f8d85c00700 1 --2- 172.21.15.119:0/748905082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d80102fd0 0x7f8d80106fa0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:04.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.344+0000 7f8d85c00700 1 --2- 172.21.15.119:0/748905082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d80102630 0x7f8d80102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:04.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.344+0000 7f8d85c00700 1 -- 172.21.15.119:0/748905082 >> 172.21.15.119:0/748905082 conn(0x7f8d800fdef0 msgr2=0x7f8d80100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:04.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.344+0000 7f8d85c00700 1 -- 172.21.15.119:0/748905082 shutdown_connections 2024-05-27T09:44:04.347 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.345+0000 7f8d85c00700 1 -- 172.21.15.119:0/748905082 wait complete. 2024-05-27T09:44:04.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.346+0000 7f8d85c00700 1 Processor -- start 2024-05-27T09:44:04.348 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.346+0000 7f8d85c00700 1 -- start start 2024-05-27T09:44:04.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.347+0000 7f8d85c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d80102630 0x7f8d80197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:04.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.347+0000 7f8d85c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d80102fd0 0x7f8d80198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:04.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.347+0000 7f8d85c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d80198c00 con 0x7f8d80102630 2024-05-27T09:44:04.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.347+0000 7f8d85c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8019c990 con 0x7f8d80102fd0 2024-05-27T09:44:04.349 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.347+0000 7f8d7ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d80102630 0x7f8d80197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:04.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.347+0000 7f8d7ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d80102630 0x7f8d80197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34034/0 (socket says 172.21.15.119:34034) 2024-05-27T09:44:04.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.347+0000 7f8d7ea00700 1 -- 172.21.15.119:0/573157242 learned_addr learned my addr 172.21.15.119:0/573157242 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:44:04.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.348+0000 7f8d7e000700 1 --2- 172.21.15.119:0/573157242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d80102fd0 0x7f8d80198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:04.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.348+0000 7f8d7ea00700 1 -- 172.21.15.119:0/573157242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d80102fd0 msgr2=0x7f8d80198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:04.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.348+0000 7f8d7ea00700 1 --2- 172.21.15.119:0/573157242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d80102fd0 0x7f8d80198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:04.350 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.348+0000 7f8d7ea00700 1 -- 172.21.15.119:0/573157242 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6c0095f0 con 0x7f8d80102630 2024-05-27T09:44:04.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.348+0000 7f8d7ea00700 1 --2- 172.21.15.119:0/573157242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d80102630 0x7f8d80197fe0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000d8d0 tx=0x7f8d7000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:04.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.348+0000 7f8d77400700 1 -- 172.21.15.119:0/573157242 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d70009940 con 0x7f8d80102630 2024-05-27T09:44:04.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.348+0000 7f8d77400700 1 -- 172.21.15.119:0/573157242 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d70010460 con 0x7f8d80102630 2024-05-27T09:44:04.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.348+0000 7f8d77400700 1 -- 172.21.15.119:0/573157242 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d7000f5f0 con 0x7f8d80102630 2024-05-27T09:44:04.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.348+0000 7f8d85c00700 1 -- 172.21.15.119:0/573157242 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8019cc70 con 0x7f8d80102630 2024-05-27T09:44:04.351 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.349+0000 7f8d85c00700 1 -- 172.21.15.119:0/573157242 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8019d170 con 0x7f8d80102630 2024-05-27T09:44:04.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.350+0000 7f8d85c00700 1 -- 172.21.15.119:0/573157242 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d8004ea50 con 0x7f8d80102630 2024-05-27T09:44:04.352 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.351+0000 7f8d77400700 1 -- 172.21.15.119:0/573157242 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f8d700105d0 con 0x7f8d80102630 2024-05-27T09:44:04.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.353+0000 7f8d77400700 1 --2- 172.21.15.119:0/573157242 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8d64078a40 0x7f8d6407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:04.354 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.353+0000 7f8d77400700 1 -- 172.21.15.119:0/573157242 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8d7009ae80 con 0x7f8d80102630 2024-05-27T09:44:04.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.353+0000 7f8d7e000700 1 --2- 172.21.15.119:0/573157242 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8d64078a40 0x7f8d6407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:04.355 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.354+0000 7f8d7e000700 1 --2- 172.21.15.119:0/573157242 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8d64078a40 0x7f8d6407aef0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8d80199600 tx=0x7f8d6c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:04.359 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.358+0000 7f8d77400700 1 -- 172.21.15.119:0/573157242 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f8d700629b0 con 0x7f8d80102630 2024-05-27T09:44:04.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.704+0000 7f8d85c00700 1 -- 172.21.15.119:0/573157242 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8d8019d610 con 0x7f8d80102630 2024-05-27T09:44:04.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.706+0000 7f8d77400700 1 -- 172.21.15.119:0/573157242 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f8d70062350 con 0x7f8d80102630 2024-05-27T09:44:04.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.709+0000 7f8d85c00700 1 -- 172.21.15.119:0/573157242 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8d64078a40 msgr2=0x7f8d6407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:04.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.709+0000 7f8d85c00700 1 --2- 172.21.15.119:0/573157242 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8d64078a40 0x7f8d6407aef0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8d80199600 tx=0x7f8d6c01a040 comp rx=0 tx=0).stop 2024-05-27T09:44:04.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.709+0000 7f8d85c00700 1 -- 172.21.15.119:0/573157242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d80102630 msgr2=0x7f8d80197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:04.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.709+0000 7f8d85c00700 1 --2- 172.21.15.119:0/573157242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d80102630 0x7f8d80197fe0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000d8d0 tx=0x7f8d7000dc90 comp rx=0 tx=0).stop 2024-05-27T09:44:04.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.709+0000 7f8d85c00700 1 -- 172.21.15.119:0/573157242 shutdown_connections 2024-05-27T09:44:04.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.709+0000 7f8d85c00700 1 --2- 172.21.15.119:0/573157242 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8d64078a40 0x7f8d6407aef0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:04.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.710+0000 7f8d85c00700 1 --2- 172.21.15.119:0/573157242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d80102630 0x7f8d80197fe0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:04.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.710+0000 7f8d85c00700 1 --2- 172.21.15.119:0/573157242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d80102fd0 0x7f8d80198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:04.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.710+0000 7f8d85c00700 1 -- 172.21.15.119:0/573157242 >> 172.21.15.119:0/573157242 conn(0x7f8d800fdef0 msgr2=0x7f8d800ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:04.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.710+0000 7f8d85c00700 1 -- 172.21.15.119:0/573157242 shutdown_connections 2024-05-27T09:44:04.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:04.710+0000 7f8d85c00700 1 -- 172.21.15.119:0/573157242 wait complete. 2024-05-27T09:44:04.726 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph version 19.0.0-3868-ga6507388 (a6507388dd5057528934822c0163b0c347ef1d5d) squid (dev)" 2024-05-27T09:44:05.220 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:04 smithi119 ceph-mon[88269]: pgmap v327: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:05.221 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:04 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/573157242' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:44:05.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:04 smithi138 ceph-mon[70004]: pgmap v327: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:05.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:04 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/573157242' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-27T09:44:05.427 DEBUG:teuthology.parallel:result is None 2024-05-27T09:44:05.427 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-27T09:44:05.440 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi119.front.sepia.ceph.com 2024-05-27T09:44:05.440 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-27T09:44:05.853 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:44:07.175 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:06 smithi119 ceph-mon[88269]: pgmap v328: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:07.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:06 smithi138 ceph-mon[70004]: pgmap v328: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:07.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.977+0000 7f667c000700 1 -- 172.21.15.119:0/3565401965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6674102fc0 msgr2=0x7f6674106f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:07.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.977+0000 7f667c000700 1 --2- 172.21.15.119:0/3565401965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6674102fc0 0x7f6674106f90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6660009910 tx=0x7f6660009c20 comp rx=0 tx=0).stop 2024-05-27T09:44:07.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.977+0000 7f667c000700 1 -- 172.21.15.119:0/3565401965 shutdown_connections 2024-05-27T09:44:07.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.977+0000 7f667c000700 1 --2- 172.21.15.119:0/3565401965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6674102fc0 0x7f6674106f90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:07.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.977+0000 7f667c000700 1 --2- 172.21.15.119:0/3565401965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6674102620 0x7f66741029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:07.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.977+0000 7f667c000700 1 -- 172.21.15.119:0/3565401965 >> 172.21.15.119:0/3565401965 conn(0x7f66740fdec0 msgr2=0x7f66741002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:07.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.977+0000 7f667c000700 1 -- 172.21.15.119:0/3565401965 shutdown_connections 2024-05-27T09:44:07.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.977+0000 7f667c000700 1 -- 172.21.15.119:0/3565401965 wait complete. 2024-05-27T09:44:07.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.978+0000 7f667c000700 1 Processor -- start 2024-05-27T09:44:07.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.979+0000 7f667c000700 1 -- start start 2024-05-27T09:44:07.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.979+0000 7f667c000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6674102620 0x7f6674198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:07.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.979+0000 7f667c000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6674102fc0 0x7f6674198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:07.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.979+0000 7f667c000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6674198d20 con 0x7f6674102620 2024-05-27T09:44:07.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.979+0000 7f667c000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667419cab0 con 0x7f6674102fc0 2024-05-27T09:44:07.980 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.979+0000 7f6679600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6674102620 0x7f6674198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.979+0000 7f6679600700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6674102620 0x7f6674198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34042/0 (socket says 172.21.15.119:34042) 2024-05-27T09:44:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.979+0000 7f6679600700 1 -- 172.21.15.119:0/2591950572 learned_addr learned my addr 172.21.15.119:0/2591950572 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:44:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.980+0000 7f6678c00700 1 --2- 172.21.15.119:0/2591950572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6674102fc0 0x7f6674198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.980+0000 7f6679600700 1 -- 172.21.15.119:0/2591950572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6674102fc0 msgr2=0x7f6674198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.980+0000 7f6679600700 1 --2- 172.21.15.119:0/2591950572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6674102fc0 0x7f6674198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.980+0000 7f6679600700 1 -- 172.21.15.119:0/2591950572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66600095f0 con 0x7f6674102620 2024-05-27T09:44:07.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.980+0000 7f6679600700 1 --2- 172.21.15.119:0/2591950572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6674102620 0x7f6674198100 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f666800b4b0 tx=0x7f666800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:07.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.980+0000 7f6672000700 1 -- 172.21.15.119:0/2591950572 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6668011800 con 0x7f6674102620 2024-05-27T09:44:07.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.980+0000 7f6672000700 1 -- 172.21.15.119:0/2591950572 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6668011e20 con 0x7f6674102620 2024-05-27T09:44:07.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.981+0000 7f667c000700 1 -- 172.21.15.119:0/2591950572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f667419cd90 con 0x7f6674102620 2024-05-27T09:44:07.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.981+0000 7f6672000700 1 -- 172.21.15.119:0/2591950572 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f666800f2e0 con 0x7f6674102620 2024-05-27T09:44:07.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.981+0000 7f667c000700 1 -- 172.21.15.119:0/2591950572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f667419d2e0 con 0x7f6674102620 2024-05-27T09:44:07.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.982+0000 7f667c000700 1 -- 172.21.15.119:0/2591950572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f667410a8c0 con 0x7f6674102620 2024-05-27T09:44:07.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.983+0000 7f6672000700 1 -- 172.21.15.119:0/2591950572 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f6668010380 con 0x7f6674102620 2024-05-27T09:44:07.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.984+0000 7f6672000700 1 --2- 172.21.15.119:0/2591950572 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f665c078a40 0x7f665c07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:07.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.984+0000 7f6672000700 1 -- 172.21.15.119:0/2591950572 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f666809a980 con 0x7f6674102620 2024-05-27T09:44:07.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.984+0000 7f6678c00700 1 --2- 172.21.15.119:0/2591950572 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f665c078a40 0x7f665c07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:07.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.985+0000 7f6678c00700 1 --2- 172.21.15.119:0/2591950572 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f665c078a40 0x7f665c07aef0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6674199720 tx=0x7f666001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:07.990 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:07.990+0000 7f6672000700 1 -- 172.21.15.119:0/2591950572 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f66680624b0 con 0x7f6674102620 2024-05-27T09:44:08.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:08.274+0000 7f667c000700 1 -- 172.21.15.119:0/2591950572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f667419d5c0 con 0x7f6674102620 2024-05-27T09:44:08.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:08.274+0000 7f6672000700 1 -- 172.21.15.119:0/2591950572 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 39 v39) v1 ==== 76+0+1958 (secure 0 0 0) 0x7f6668061e50 con 0x7f6674102620 2024-05-27T09:44:08.276 INFO:teuthology.orchestra.run.smithi119.stdout:e39 2024-05-27T09:44:08.276 INFO:teuthology.orchestra.run.smithi119.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T09:44:08.276 INFO:teuthology.orchestra.run.smithi119.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-27T09:44:08.276 INFO:teuthology.orchestra.run.smithi119.stdout:legacy client fscid: 1 2024-05-27T09:44:08.276 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:44:08.276 INFO:teuthology.orchestra.run.smithi119.stdout:Filesystem 'cephfs' (1) 2024-05-27T09:44:08.276 INFO:teuthology.orchestra.run.smithi119.stdout:fs_name cephfs 2024-05-27T09:44:08.276 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 39 2024-05-27T09:44:08.276 INFO:teuthology.orchestra.run.smithi119.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-27T09:44:08.276 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-05-27T09:25:53.387388+0000 2024-05-27T09:44:08.276 INFO:teuthology.orchestra.run.smithi119.stdout:modified 2024-05-27T09:42:14.897334+0000 2024-05-27T09:44:08.277 INFO:teuthology.orchestra.run.smithi119.stdout:tableserver 0 2024-05-27T09:44:08.277 INFO:teuthology.orchestra.run.smithi119.stdout:root 0 2024-05-27T09:44:08.277 INFO:teuthology.orchestra.run.smithi119.stdout:session_timeout 60 2024-05-27T09:44:08.277 INFO:teuthology.orchestra.run.smithi119.stdout:session_autoclose 300 2024-05-27T09:44:08.277 INFO:teuthology.orchestra.run.smithi119.stdout:max_file_size 1099511627776 2024-05-27T09:44:08.277 INFO:teuthology.orchestra.run.smithi119.stdout:max_xattr_size 65536 2024-05-27T09:44:08.277 INFO:teuthology.orchestra.run.smithi119.stdout:required_client_features {} 2024-05-27T09:44:08.277 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure 0 2024-05-27T09:44:08.277 INFO:teuthology.orchestra.run.smithi119.stdout:last_failure_osd_epoch 81 2024-05-27T09:44:08.277 INFO:teuthology.orchestra.run.smithi119.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T09:44:08.277 INFO:teuthology.orchestra.run.smithi119.stdout:max_mds 2 2024-05-27T09:44:08.277 INFO:teuthology.orchestra.run.smithi119.stdout:in 0,1 2024-05-27T09:44:08.277 INFO:teuthology.orchestra.run.smithi119.stdout:up {0=34436,1=34468} 2024-05-27T09:44:08.278 INFO:teuthology.orchestra.run.smithi119.stdout:failed 2024-05-27T09:44:08.278 INFO:teuthology.orchestra.run.smithi119.stdout:damaged 2024-05-27T09:44:08.278 INFO:teuthology.orchestra.run.smithi119.stdout:stopped 2024-05-27T09:44:08.278 INFO:teuthology.orchestra.run.smithi119.stdout:data_pools [3] 2024-05-27T09:44:08.278 INFO:teuthology.orchestra.run.smithi119.stdout:metadata_pool 2 2024-05-27T09:44:08.278 INFO:teuthology.orchestra.run.smithi119.stdout:inline_data enabled 2024-05-27T09:44:08.278 INFO:teuthology.orchestra.run.smithi119.stdout:balancer 2024-05-27T09:44:08.278 INFO:teuthology.orchestra.run.smithi119.stdout:bal_rank_mask -1 2024-05-27T09:44:08.278 INFO:teuthology.orchestra.run.smithi119.stdout:standby_count_wanted 1 2024-05-27T09:44:08.278 INFO:teuthology.orchestra.run.smithi119.stdout:qdb_cluster leader: 34436 members: 34468,34436 2024-05-27T09:44:08.278 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.nghcey{0:34436} state up:active seq 29 join_fscid=1 addr [v2:172.21.15.119:6826/3357235708,v1:172.21.15.119:6827/3357235708] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T09:44:08.278 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi119.ppmzzk{1:34468} state up:active seq 23 join_fscid=1 addr [v2:172.21.15.119:6828/2430623742,v1:172.21.15.119:6829/2430623742] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T09:44:08.278 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:44:08.279 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:44:08.279 INFO:teuthology.orchestra.run.smithi119.stdout:Standby daemons: 2024-05-27T09:44:08.279 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:44:08.279 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.ndfkhd{-1:44289} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/3387966485,v1:172.21.15.138:6825/3387966485] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T09:44:08.279 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.cephfs.smithi138.alszpk{-1:44303} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/3198147602,v1:172.21.15.138:6827/3198147602] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T09:44:08.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:08.278+0000 7f667c000700 1 -- 172.21.15.119:0/2591950572 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f665c078a40 msgr2=0x7f665c07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:08.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:08.278+0000 7f667c000700 1 --2- 172.21.15.119:0/2591950572 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f665c078a40 0x7f665c07aef0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6674199720 tx=0x7f666001a040 comp rx=0 tx=0).stop 2024-05-27T09:44:08.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:08.278+0000 7f667c000700 1 -- 172.21.15.119:0/2591950572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6674102620 msgr2=0x7f6674198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:08.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:08.278+0000 7f667c000700 1 --2- 172.21.15.119:0/2591950572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6674102620 0x7f6674198100 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f666800b4b0 tx=0x7f666800b870 comp rx=0 tx=0).stop 2024-05-27T09:44:08.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:08.278+0000 7f667c000700 1 -- 172.21.15.119:0/2591950572 shutdown_connections 2024-05-27T09:44:08.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:08.278+0000 7f667c000700 1 --2- 172.21.15.119:0/2591950572 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f665c078a40 0x7f665c07aef0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:08.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:08.278+0000 7f667c000700 1 --2- 172.21.15.119:0/2591950572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6674102620 0x7f6674198100 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:08.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:08.278+0000 7f667c000700 1 --2- 172.21.15.119:0/2591950572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6674102fc0 0x7f6674198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:08.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:08.278+0000 7f667c000700 1 -- 172.21.15.119:0/2591950572 >> 172.21.15.119:0/2591950572 conn(0x7f66740fdec0 msgr2=0x7f66740ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:08.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:08.278+0000 7f667c000700 1 -- 172.21.15.119:0/2591950572 shutdown_connections 2024-05-27T09:44:08.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:08.278+0000 7f667c000700 1 -- 172.21.15.119:0/2591950572 wait complete. 2024-05-27T09:44:08.281 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 39 2024-05-27T09:44:09.018 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-05-27T09:44:09.029 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T09:44:09.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:08 smithi119 ceph-mon[88269]: pgmap v329: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:09.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:08 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/2591950572' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:44:09.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:08 smithi138 ceph-mon[70004]: pgmap v329: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:09.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:08 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/2591950572' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T09:44:09.458 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:44:10.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:10 smithi119 ceph-mon[88269]: pgmap v330: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:11.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:10 smithi138 ceph-mon[70004]: pgmap v330: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:11.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.554+0000 7fb297200700 1 -- 172.21.15.119:0/3203925912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb290102de0 msgr2=0x7fb290106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:11.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.554+0000 7fb297200700 1 --2- 172.21.15.119:0/3203925912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb290102de0 0x7fb290106db0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb280009910 tx=0x7fb280009c20 comp rx=0 tx=0).stop 2024-05-27T09:44:11.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.555+0000 7fb297200700 1 -- 172.21.15.119:0/3203925912 shutdown_connections 2024-05-27T09:44:11.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.555+0000 7fb297200700 1 --2- 172.21.15.119:0/3203925912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb290102de0 0x7fb290106db0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:11.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.555+0000 7fb297200700 1 --2- 172.21.15.119:0/3203925912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb290102440 0x7fb290102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:11.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.555+0000 7fb297200700 1 -- 172.21.15.119:0/3203925912 >> 172.21.15.119:0/3203925912 conn(0x7fb2900fdce0 msgr2=0x7fb2901000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:11.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.555+0000 7fb297200700 1 -- 172.21.15.119:0/3203925912 shutdown_connections 2024-05-27T09:44:11.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.555+0000 7fb297200700 1 -- 172.21.15.119:0/3203925912 wait complete. 2024-05-27T09:44:11.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.557+0000 7fb297200700 1 Processor -- start 2024-05-27T09:44:11.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.557+0000 7fb297200700 1 -- start start 2024-05-27T09:44:11.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.557+0000 7fb297200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb290102440 0x7fb290197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:11.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.557+0000 7fb297200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb290102de0 0x7fb2901982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:11.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.557+0000 7fb297200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb290198900 con 0x7fb290102de0 2024-05-27T09:44:11.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.557+0000 7fb297200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb290198a40 con 0x7fb290102440 2024-05-27T09:44:11.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.558+0000 7fb295400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb290102de0 0x7fb2901982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:11.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.558+0000 7fb295400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb290102de0 0x7fb2901982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44516/0 (socket says 172.21.15.119:44516) 2024-05-27T09:44:11.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.558+0000 7fb295400700 1 -- 172.21.15.119:0/2058388435 learned_addr learned my addr 172.21.15.119:0/2058388435 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:44:11.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.558+0000 7fb295e00700 1 --2- 172.21.15.119:0/2058388435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb290102440 0x7fb290197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:11.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.558+0000 7fb295400700 1 -- 172.21.15.119:0/2058388435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb290102440 msgr2=0x7fb290197d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:11.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.558+0000 7fb295400700 1 --2- 172.21.15.119:0/2058388435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb290102440 0x7fb290197d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:11.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.558+0000 7fb295400700 1 -- 172.21.15.119:0/2058388435 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2800095f0 con 0x7fb290102de0 2024-05-27T09:44:11.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.558+0000 7fb295400700 1 --2- 172.21.15.119:0/2058388435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb290102de0 0x7fb2901982b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb2800092e0 tx=0x7fb28000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:11.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.559+0000 7fb28aa00700 1 -- 172.21.15.119:0/2058388435 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb28001d070 con 0x7fb290102de0 2024-05-27T09:44:11.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.559+0000 7fb28aa00700 1 -- 172.21.15.119:0/2058388435 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb28000f460 con 0x7fb290102de0 2024-05-27T09:44:11.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.559+0000 7fb297200700 1 -- 172.21.15.119:0/2058388435 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29019c830 con 0x7fb290102de0 2024-05-27T09:44:11.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.559+0000 7fb28aa00700 1 -- 172.21.15.119:0/2058388435 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2800175f0 con 0x7fb290102de0 2024-05-27T09:44:11.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.559+0000 7fb297200700 1 -- 172.21.15.119:0/2058388435 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29019cd20 con 0x7fb290102de0 2024-05-27T09:44:11.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.560+0000 7fb297200700 1 -- 172.21.15.119:0/2058388435 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb29004f2a0 con 0x7fb290102de0 2024-05-27T09:44:11.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.561+0000 7fb28aa00700 1 -- 172.21.15.119:0/2058388435 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fb28000f5d0 con 0x7fb290102de0 2024-05-27T09:44:11.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.562+0000 7fb28aa00700 1 --2- 172.21.15.119:0/2058388435 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb278078a40 0x7fb27807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:11.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.562+0000 7fb28aa00700 1 -- 172.21.15.119:0/2058388435 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb28009c390 con 0x7fb290102de0 2024-05-27T09:44:11.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.563+0000 7fb295e00700 1 --2- 172.21.15.119:0/2058388435 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb278078a40 0x7fb27807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:11.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.563+0000 7fb295e00700 1 --2- 172.21.15.119:0/2058388435 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb278078a40 0x7fb27807aef0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb284005ea0 tx=0x7fb284005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:11.568 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.568+0000 7fb28aa00700 1 -- 172.21.15.119:0/2058388435 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fb280063ec0 con 0x7fb290102de0 2024-05-27T09:44:11.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.857+0000 7fb297200700 1 -- 172.21.15.119:0/2058388435 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2901990d0 con 0x7fb290102de0 2024-05-27T09:44:11.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.859+0000 7fb28aa00700 1 -- 172.21.15.119:0/2058388435 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+5242 (secure 0 0 0) 0x7fb280063860 con 0x7fb290102de0 2024-05-27T09:44:11.860 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:44:11.860 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44289,"name":"cephfs.smithi138.ndfkhd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/3387966485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3387966485},{"type":"v1","addr":"172.21.15.138:6825","nonce":3387966485}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44303,"name":"cephfs.smithi138.alszpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/3198147602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":3198147602},{"type":"v1","addr":"172.21.15.138:6827","nonce":3198147602}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:25:53.387388+0000","modified":"2024-05-27T09:42:14.897334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34436,"mds_1":34468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34436":{"gid":34436,"name":"cephfs.smithi119.nghcey","rank":0,"incarnation":33,"state":"up:active","state_seq":29,"addr":"172.21.15.119:6827/3357235708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3357235708},{"type":"v1","addr":"172.21.15.119:6827","nonce":3357235708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34468":{"gid":34468,"name":"cephfs.smithi119.ppmzzk","rank":1,"incarnation":33,"state":"up:active","state_seq":23,"addr":"172.21.15.119:6829/2430623742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2430623742},{"type":"v1","addr":"172.21.15.119:6829","nonce":2430623742}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34436,"qdb_cluster":[34468,34436]},"id":1}]} 2024-05-27T09:44:11.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.862+0000 7fb297200700 1 -- 172.21.15.119:0/2058388435 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb278078a40 msgr2=0x7fb27807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:11.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.862+0000 7fb297200700 1 --2- 172.21.15.119:0/2058388435 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb278078a40 0x7fb27807aef0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb284005ea0 tx=0x7fb284005e30 comp rx=0 tx=0).stop 2024-05-27T09:44:11.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.862+0000 7fb297200700 1 -- 172.21.15.119:0/2058388435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb290102de0 msgr2=0x7fb2901982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:11.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.862+0000 7fb297200700 1 --2- 172.21.15.119:0/2058388435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb290102de0 0x7fb2901982b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb2800092e0 tx=0x7fb28000be20 comp rx=0 tx=0).stop 2024-05-27T09:44:11.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.862+0000 7fb297200700 1 -- 172.21.15.119:0/2058388435 shutdown_connections 2024-05-27T09:44:11.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.862+0000 7fb297200700 1 --2- 172.21.15.119:0/2058388435 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fb278078a40 0x7fb27807aef0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:11.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.862+0000 7fb297200700 1 --2- 172.21.15.119:0/2058388435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb290102440 0x7fb290197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:11.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.862+0000 7fb297200700 1 --2- 172.21.15.119:0/2058388435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb290102de0 0x7fb2901982b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:11.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.862+0000 7fb297200700 1 -- 172.21.15.119:0/2058388435 >> 172.21.15.119:0/2058388435 conn(0x7fb2900fdce0 msgr2=0x7fb2900ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:11.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.862+0000 7fb297200700 1 -- 172.21.15.119:0/2058388435 shutdown_connections 2024-05-27T09:44:11.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:11.862+0000 7fb297200700 1 -- 172.21.15.119:0/2058388435 wait complete. 2024-05-27T09:44:11.864 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 39 2024-05-27T09:44:12.234 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:11 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/2058388435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T09:44:12.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:11 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/2058388435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T09:44:12.678 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 18, 'max_mds': 2, 'flags': 18} 2024-05-27T09:44:12.679 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 19 2024-05-27T09:44:13.077 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:44:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:12 smithi119 ceph-mon[88269]: pgmap v331: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:13.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:44:13.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:12 smithi138 ceph-mon[70004]: pgmap v331: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:13.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:44:15.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.125+0000 7f1f78e00700 1 -- 172.21.15.119:0/3174554751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f74100200 msgr2=0x7f1f741005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:15.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.125+0000 7f1f78e00700 1 --2- 172.21.15.119:0/3174554751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f74100200 0x7f1f741005d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009910 tx=0x7f1f64009c20 comp rx=0 tx=0).stop 2024-05-27T09:44:15.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.125+0000 7f1f78e00700 1 -- 172.21.15.119:0/3174554751 shutdown_connections 2024-05-27T09:44:15.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.125+0000 7f1f78e00700 1 --2- 172.21.15.119:0/3174554751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f74100ba0 0x7f1f74104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:15.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.125+0000 7f1f78e00700 1 --2- 172.21.15.119:0/3174554751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f74100200 0x7f1f741005d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:15.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.125+0000 7f1f78e00700 1 -- 172.21.15.119:0/3174554751 >> 172.21.15.119:0/3174554751 conn(0x7f1f740fba80 msgr2=0x7f1f740fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:15.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.125+0000 7f1f78e00700 1 -- 172.21.15.119:0/3174554751 shutdown_connections 2024-05-27T09:44:15.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.125+0000 7f1f78e00700 1 -- 172.21.15.119:0/3174554751 wait complete. 2024-05-27T09:44:15.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.126+0000 7f1f78e00700 1 Processor -- start 2024-05-27T09:44:15.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.126+0000 7f1f78e00700 1 -- start start 2024-05-27T09:44:15.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.127+0000 7f1f78e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f74100200 0x7f1f74197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:15.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.127+0000 7f1f78e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f74100ba0 0x7f1f74198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:15.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.127+0000 7f1f78e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f74198a70 con 0x7f1f74100200 2024-05-27T09:44:15.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.127+0000 7f1f78e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7419c800 con 0x7f1f74100ba0 2024-05-27T09:44:15.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.127+0000 7f1f73400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f74100200 0x7f1f74197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:15.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.127+0000 7f1f73400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f74100200 0x7f1f74197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44534/0 (socket says 172.21.15.119:44534) 2024-05-27T09:44:15.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.127+0000 7f1f73400700 1 -- 172.21.15.119:0/1353775560 learned_addr learned my addr 172.21.15.119:0/1353775560 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:44:15.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.128+0000 7f1f73400700 1 -- 172.21.15.119:0/1353775560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f74100ba0 msgr2=0x7f1f74198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:15.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.128+0000 7f1f6be00700 1 --2- 172.21.15.119:0/1353775560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f74100ba0 0x7f1f74198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:15.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.128+0000 7f1f73400700 1 --2- 172.21.15.119:0/1353775560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f74100ba0 0x7f1f74198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:15.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.128+0000 7f1f73400700 1 -- 172.21.15.119:0/1353775560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f640095f0 con 0x7f1f74100200 2024-05-27T09:44:15.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.128+0000 7f1f6be00700 1 --2- 172.21.15.119:0/1353775560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f74100ba0 0x7f1f74198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:44:15.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.128+0000 7f1f73400700 1 --2- 172.21.15.119:0/1353775560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f74100200 0x7f1f74197e50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1f64006010 tx=0x7f1f6400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:15.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.128+0000 7f1f70c00700 1 -- 172.21.15.119:0/1353775560 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f6401d070 con 0x7f1f74100200 2024-05-27T09:44:15.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.128+0000 7f1f70c00700 1 -- 172.21.15.119:0/1353775560 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f64004fb0 con 0x7f1f74100200 2024-05-27T09:44:15.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.128+0000 7f1f78e00700 1 -- 172.21.15.119:0/1353775560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7419ca80 con 0x7f1f74100200 2024-05-27T09:44:15.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.128+0000 7f1f70c00700 1 -- 172.21.15.119:0/1353775560 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f64022ea0 con 0x7f1f74100200 2024-05-27T09:44:15.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.129+0000 7f1f78e00700 1 -- 172.21.15.119:0/1353775560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7419cf70 con 0x7f1f74100200 2024-05-27T09:44:15.130 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.130+0000 7f1f78e00700 1 -- 172.21.15.119:0/1353775560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f741042f0 con 0x7f1f74100200 2024-05-27T09:44:15.131 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.131+0000 7f1f70c00700 1 -- 172.21.15.119:0/1353775560 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f1f64022470 con 0x7f1f74100200 2024-05-27T09:44:15.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.132+0000 7f1f70c00700 1 --2- 172.21.15.119:0/1353775560 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1f58078a40 0x7f1f5807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:15.132 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.132+0000 7f1f70c00700 1 -- 172.21.15.119:0/1353775560 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1f6409c3f0 con 0x7f1f74100200 2024-05-27T09:44:15.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.132+0000 7f1f6be00700 1 --2- 172.21.15.119:0/1353775560 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1f58078a40 0x7f1f5807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:15.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.133+0000 7f1f6be00700 1 --2- 172.21.15.119:0/1353775560 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1f58078a40 0x7f1f5807aef0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1f74199470 tx=0x7f1f5c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:15.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.138+0000 7f1f70c00700 1 -- 172.21.15.119:0/1353775560 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f1f64063fd0 con 0x7f1f74100200 2024-05-27T09:44:15.220 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:15 smithi119 ceph-mon[88269]: pgmap v332: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:15.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:15 smithi138 ceph-mon[70004]: pgmap v332: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:15.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.424+0000 7f1f78e00700 1 -- 172.21.15.119:0/1353775560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f1f7402cc30 con 0x7f1f74100200 2024-05-27T09:44:15.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.426+0000 7f1f70c00700 1 -- 172.21.15.119:0/1353775560 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v39) v1 ==== 107+0+4879 (secure 0 0 0) 0x7f1f64022720 con 0x7f1f74100200 2024-05-27T09:44:15.427 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:44:15.427 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14482,"name":"cephfs.smithi119.nghcey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6827/413736191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":413736191},{"type":"v1","addr":"172.21.15.119:6827","nonce":413736191}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24253,"name":"cephfs.smithi138.alszpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.138:6827/2345607053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2345607053},{"type":"v1","addr":"172.21.15.138:6827","nonce":2345607053}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:25:53.387388+0000","modified":"2024-05-27T09:28:41.397122+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14472,"mds_1":24241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi119.ppmzzk","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6829/2100782307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2100782307},{"type":"v1","addr":"172.21.15.119:6829","nonce":2100782307}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24241":{"gid":24241,"name":"cephfs.smithi138.ndfkhd","rank":1,"incarnation":9,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6825/897058594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":897058594},{"type":"v1","addr":"172.21.15.138:6825","nonce":897058594}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-27T09:44:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.429+0000 7f1f78e00700 1 -- 172.21.15.119:0/1353775560 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1f58078a40 msgr2=0x7f1f5807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.429+0000 7f1f78e00700 1 --2- 172.21.15.119:0/1353775560 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1f58078a40 0x7f1f5807aef0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1f74199470 tx=0x7f1f5c008040 comp rx=0 tx=0).stop 2024-05-27T09:44:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.429+0000 7f1f78e00700 1 -- 172.21.15.119:0/1353775560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f74100200 msgr2=0x7f1f74197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.429+0000 7f1f78e00700 1 --2- 172.21.15.119:0/1353775560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f74100200 0x7f1f74197e50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1f64006010 tx=0x7f1f6400be20 comp rx=0 tx=0).stop 2024-05-27T09:44:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.429+0000 7f1f78e00700 1 -- 172.21.15.119:0/1353775560 shutdown_connections 2024-05-27T09:44:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.429+0000 7f1f78e00700 1 --2- 172.21.15.119:0/1353775560 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1f58078a40 0x7f1f5807aef0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.429+0000 7f1f78e00700 1 --2- 172.21.15.119:0/1353775560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f74100200 0x7f1f74197e50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.429+0000 7f1f78e00700 1 --2- 172.21.15.119:0/1353775560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f74100ba0 0x7f1f74198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.429+0000 7f1f78e00700 1 -- 172.21.15.119:0/1353775560 >> 172.21.15.119:0/1353775560 conn(0x7f1f740fba80 msgr2=0x7f1f740fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:15.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.429+0000 7f1f78e00700 1 -- 172.21.15.119:0/1353775560 shutdown_connections 2024-05-27T09:44:15.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:15.429+0000 7f1f78e00700 1 -- 172.21.15.119:0/1353775560 wait complete. 2024-05-27T09:44:15.431 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 19 2024-05-27T09:44:16.162 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:16 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/1353775560' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-05-27T09:44:16.210 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 20 2024-05-27T09:44:16.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:16 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/1353775560' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-05-27T09:44:16.626 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:44:17.119 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:17 smithi119 ceph-mon[88269]: pgmap v333: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:17.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:17 smithi138 ceph-mon[70004]: pgmap v333: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:18.480 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:18 smithi119 ceph-mon[88269]: pgmap v334: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:18.481 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:18 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:44:18.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:18 smithi138 ceph-mon[70004]: pgmap v334: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:18.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:18 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:44:18.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.697+0000 7fc927e00700 1 -- 172.21.15.119:0/634978050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc928073a00 msgr2=0x7fc92810c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:18.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.697+0000 7fc927e00700 1 --2- 172.21.15.119:0/634978050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc928073a00 0x7fc92810c2a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc914009910 tx=0x7fc914009c20 comp rx=0 tx=0).stop 2024-05-27T09:44:18.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.697+0000 7fc927e00700 1 -- 172.21.15.119:0/634978050 shutdown_connections 2024-05-27T09:44:18.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.697+0000 7fc927e00700 1 --2- 172.21.15.119:0/634978050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc928073a00 0x7fc92810c2a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:18.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.698+0000 7fc927e00700 1 --2- 172.21.15.119:0/634978050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9280730f0 0x7fc9280734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:18.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.698+0000 7fc927e00700 1 -- 172.21.15.119:0/634978050 >> 172.21.15.119:0/634978050 conn(0x7fc9280fba80 msgr2=0x7fc9280fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:18.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.698+0000 7fc927e00700 1 -- 172.21.15.119:0/634978050 shutdown_connections 2024-05-27T09:44:18.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.698+0000 7fc927e00700 1 -- 172.21.15.119:0/634978050 wait complete. 2024-05-27T09:44:18.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.699+0000 7fc927e00700 1 Processor -- start 2024-05-27T09:44:18.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.699+0000 7fc927e00700 1 -- start start 2024-05-27T09:44:18.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.699+0000 7fc927e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9280730f0 0x7fc92819da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:18.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.699+0000 7fc927e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc928073a00 0x7fc92819e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:18.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.699+0000 7fc927e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc928197ab0 con 0x7fc9280730f0 2024-05-27T09:44:18.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.699+0000 7fc927e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc928197c20 con 0x7fc928073a00 2024-05-27T09:44:18.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.700+0000 7fc926a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9280730f0 0x7fc92819da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:18.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.700+0000 7fc926a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9280730f0 0x7fc92819da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44550/0 (socket says 172.21.15.119:44550) 2024-05-27T09:44:18.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.700+0000 7fc926a00700 1 -- 172.21.15.119:0/3271459379 learned_addr learned my addr 172.21.15.119:0/3271459379 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:44:18.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.700+0000 7fc926000700 1 --2- 172.21.15.119:0/3271459379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc928073a00 0x7fc92819e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:18.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.700+0000 7fc926a00700 1 -- 172.21.15.119:0/3271459379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc928073a00 msgr2=0x7fc92819e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:18.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.700+0000 7fc926a00700 1 --2- 172.21.15.119:0/3271459379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc928073a00 0x7fc92819e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:18.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.700+0000 7fc926a00700 1 -- 172.21.15.119:0/3271459379 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9140095f0 con 0x7fc9280730f0 2024-05-27T09:44:18.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.701+0000 7fc926a00700 1 --2- 172.21.15.119:0/3271459379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9280730f0 0x7fc92819da30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc91800d8d0 tx=0x7fc91800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:18.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.701+0000 7fc91f400700 1 -- 172.21.15.119:0/3271459379 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc918009940 con 0x7fc9280730f0 2024-05-27T09:44:18.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.701+0000 7fc927e00700 1 -- 172.21.15.119:0/3271459379 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc928197f00 con 0x7fc9280730f0 2024-05-27T09:44:18.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.701+0000 7fc91f400700 1 -- 172.21.15.119:0/3271459379 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc918010460 con 0x7fc9280730f0 2024-05-27T09:44:18.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.701+0000 7fc91f400700 1 -- 172.21.15.119:0/3271459379 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc91800f5f0 con 0x7fc9280730f0 2024-05-27T09:44:18.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.701+0000 7fc927e00700 1 -- 172.21.15.119:0/3271459379 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc928198450 con 0x7fc9280730f0 2024-05-27T09:44:18.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.702+0000 7fc927e00700 1 -- 172.21.15.119:0/3271459379 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9281099a0 con 0x7fc9280730f0 2024-05-27T09:44:18.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.703+0000 7fc91f400700 1 -- 172.21.15.119:0/3271459379 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fc9180105d0 con 0x7fc9280730f0 2024-05-27T09:44:18.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.705+0000 7fc91f400700 1 --2- 172.21.15.119:0/3271459379 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc90c078a40 0x7fc90c07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:18.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.705+0000 7fc91f400700 1 -- 172.21.15.119:0/3271459379 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc91809ae80 con 0x7fc9280730f0 2024-05-27T09:44:18.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.705+0000 7fc926000700 1 --2- 172.21.15.119:0/3271459379 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc90c078a40 0x7fc90c07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:18.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.706+0000 7fc926000700 1 --2- 172.21.15.119:0/3271459379 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc90c078a40 0x7fc90c07aef0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc914006010 tx=0x7fc91401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:18.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:18.710+0000 7fc91f400700 1 -- 172.21.15.119:0/3271459379 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fc9180629b0 con 0x7fc9280730f0 2024-05-27T09:44:19.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:19.003+0000 7fc927e00700 1 -- 172.21.15.119:0/3271459379 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7fc92804f2a0 con 0x7fc9280730f0 2024-05-27T09:44:19.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:19.004+0000 7fc91f400700 1 -- 172.21.15.119:0/3271459379 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v39) v1 ==== 107+0+4880 (secure 0 0 0) 0x7fc92804f2a0 con 0x7fc9280730f0 2024-05-27T09:44:19.005 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:44:19.005 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14482,"name":"cephfs.smithi119.nghcey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6827/413736191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":413736191},{"type":"v1","addr":"172.21.15.119:6827","nonce":413736191}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24253,"name":"cephfs.smithi138.alszpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.138:6827/2345607053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2345607053},{"type":"v1","addr":"172.21.15.138:6827","nonce":2345607053}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:25:53.387388+0000","modified":"2024-05-27T09:30:07.116223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14472,"mds_1":24241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi119.ppmzzk","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6829/2100782307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2100782307},{"type":"v1","addr":"172.21.15.119:6829","nonce":2100782307}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24241":{"gid":24241,"name":"cephfs.smithi138.ndfkhd","rank":1,"incarnation":9,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6825/897058594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":897058594},{"type":"v1","addr":"172.21.15.138:6825","nonce":897058594}]},"join_fscid":1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-27T09:44:19.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:19.008+0000 7fc927e00700 1 -- 172.21.15.119:0/3271459379 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc90c078a40 msgr2=0x7fc90c07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:19.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:19.008+0000 7fc927e00700 1 --2- 172.21.15.119:0/3271459379 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc90c078a40 0x7fc90c07aef0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc914006010 tx=0x7fc91401a040 comp rx=0 tx=0).stop 2024-05-27T09:44:19.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:19.008+0000 7fc927e00700 1 -- 172.21.15.119:0/3271459379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9280730f0 msgr2=0x7fc92819da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:19.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:19.008+0000 7fc927e00700 1 --2- 172.21.15.119:0/3271459379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9280730f0 0x7fc92819da30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc91800d8d0 tx=0x7fc91800dc90 comp rx=0 tx=0).stop 2024-05-27T09:44:19.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:19.008+0000 7fc927e00700 1 -- 172.21.15.119:0/3271459379 shutdown_connections 2024-05-27T09:44:19.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:19.008+0000 7fc927e00700 1 --2- 172.21.15.119:0/3271459379 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fc90c078a40 0x7fc90c07aef0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:19.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:19.008+0000 7fc927e00700 1 --2- 172.21.15.119:0/3271459379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9280730f0 0x7fc92819da30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:19.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:19.008+0000 7fc927e00700 1 --2- 172.21.15.119:0/3271459379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc928073a00 0x7fc92819e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:19.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:19.008+0000 7fc927e00700 1 -- 172.21.15.119:0/3271459379 >> 172.21.15.119:0/3271459379 conn(0x7fc9280fba80 msgr2=0x7fc928106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:19.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:19.008+0000 7fc927e00700 1 -- 172.21.15.119:0/3271459379 shutdown_connections 2024-05-27T09:44:19.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:19.009+0000 7fc927e00700 1 -- 172.21.15.119:0/3271459379 wait complete. 2024-05-27T09:44:19.010 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 20 2024-05-27T09:44:19.600 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:19 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/3271459379' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-05-27T09:44:19.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:19 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/3271459379' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-05-27T09:44:19.649 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 21 2024-05-27T09:44:20.041 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:44:20.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:20 smithi119 ceph-mon[88269]: pgmap v335: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:20.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:20 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:44:20.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:20 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:44:20.499 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:20 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:44:20.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:20 smithi138 ceph-mon[70004]: pgmap v335: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:20.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:20 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:44:20.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:20 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:44:20.636 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:20 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:44:22.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.173+0000 7f6877200700 1 -- 172.21.15.119:0/4160583272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6878100bf0 msgr2=0x7f6878104bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:22.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.173+0000 7f6877200700 1 --2- 172.21.15.119:0/4160583272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6878100bf0 0x7f6878104bc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6864009910 tx=0x7f6864009c20 comp rx=0 tx=0).stop 2024-05-27T09:44:22.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.173+0000 7f6877200700 1 -- 172.21.15.119:0/4160583272 shutdown_connections 2024-05-27T09:44:22.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.173+0000 7f6877200700 1 --2- 172.21.15.119:0/4160583272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6878100bf0 0x7f6878104bc0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:22.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.173+0000 7f6877200700 1 --2- 172.21.15.119:0/4160583272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6878100250 0x7f6878100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:22.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.173+0000 7f6877200700 1 -- 172.21.15.119:0/4160583272 >> 172.21.15.119:0/4160583272 conn(0x7f68780fbaf0 msgr2=0x7f68780fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:22.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.174+0000 7f6877200700 1 -- 172.21.15.119:0/4160583272 shutdown_connections 2024-05-27T09:44:22.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.174+0000 7f6877200700 1 -- 172.21.15.119:0/4160583272 wait complete. 2024-05-27T09:44:22.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.175+0000 7f6877200700 1 Processor -- start 2024-05-27T09:44:22.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.175+0000 7f6877200700 1 -- start start 2024-05-27T09:44:22.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.176+0000 7f6877200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6878100250 0x7f6878073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:22.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.176+0000 7f6877200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6878100bf0 0x7f6878073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:22.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.176+0000 7f6877200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6878073ad0 con 0x7f6878100bf0 2024-05-27T09:44:22.176 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.176+0000 7f6877200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6878073c10 con 0x7f6878100250 2024-05-27T09:44:22.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.176+0000 7f6875400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6878100bf0 0x7f6878073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:22.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.176+0000 7f6875400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6878100bf0 0x7f6878073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53660/0 (socket says 172.21.15.119:53660) 2024-05-27T09:44:22.177 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.176+0000 7f6875400700 1 -- 172.21.15.119:0/692451885 learned_addr learned my addr 172.21.15.119:0/692451885 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:44:22.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.176+0000 7f6875e00700 1 --2- 172.21.15.119:0/692451885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6878100250 0x7f6878073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:22.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.177+0000 7f6875400700 1 -- 172.21.15.119:0/692451885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6878100250 msgr2=0x7f6878073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:22.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.177+0000 7f6875400700 1 --2- 172.21.15.119:0/692451885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6878100250 0x7f6878073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:22.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.177+0000 7f6875400700 1 -- 172.21.15.119:0/692451885 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68640095f0 con 0x7f6878100bf0 2024-05-27T09:44:22.178 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.177+0000 7f6875400700 1 --2- 172.21.15.119:0/692451885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6878100bf0 0x7f6878073590 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f686400bda0 tx=0x7f686400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:22.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.177+0000 7f686ea00700 1 -- 172.21.15.119:0/692451885 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f686401d070 con 0x7f6878100bf0 2024-05-27T09:44:22.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.177+0000 7f686ea00700 1 -- 172.21.15.119:0/692451885 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6864004fb0 con 0x7f6878100bf0 2024-05-27T09:44:22.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.177+0000 7f6877200700 1 -- 172.21.15.119:0/692451885 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6878073e90 con 0x7f6878100bf0 2024-05-27T09:44:22.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.177+0000 7f686ea00700 1 -- 172.21.15.119:0/692451885 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f686400f480 con 0x7f6878100bf0 2024-05-27T09:44:22.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.177+0000 7f6877200700 1 -- 172.21.15.119:0/692451885 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68781a20c0 con 0x7f6878100bf0 2024-05-27T09:44:22.179 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.179+0000 7f6877200700 1 -- 172.21.15.119:0/692451885 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68781085c0 con 0x7f6878100bf0 2024-05-27T09:44:22.180 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.180+0000 7f686ea00700 1 -- 172.21.15.119:0/692451885 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f6864022870 con 0x7f6878100bf0 2024-05-27T09:44:22.181 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.181+0000 7f686ea00700 1 --2- 172.21.15.119:0/692451885 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f685c078a40 0x7f685c07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:22.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.181+0000 7f686ea00700 1 -- 172.21.15.119:0/692451885 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f686409c340 con 0x7f6878100bf0 2024-05-27T09:44:22.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.181+0000 7f6875e00700 1 --2- 172.21.15.119:0/692451885 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f685c078a40 0x7f685c07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:22.182 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.182+0000 7f6875e00700 1 --2- 172.21.15.119:0/692451885 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f685c078a40 0x7f685c07aef0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6868005fd0 tx=0x7f686800a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:22.186 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.186+0000 7f686ea00700 1 -- 172.21.15.119:0/692451885 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f6864063f20 con 0x7f6878100bf0 2024-05-27T09:44:22.470 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.469+0000 7f6877200700 1 -- 172.21.15.119:0/692451885 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f687804f2a0 con 0x7f6878100bf0 2024-05-27T09:44:22.471 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.470+0000 7f686ea00700 1 -- 172.21.15.119:0/692451885 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v39) v1 ==== 107+0+4879 (secure 0 0 0) 0x7f68640638c0 con 0x7f6878100bf0 2024-05-27T09:44:22.471 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:44:22.471 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14482,"name":"cephfs.smithi119.nghcey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6827/413736191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":413736191},{"type":"v1","addr":"172.21.15.119:6827","nonce":413736191}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24253,"name":"cephfs.smithi138.alszpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.138:6827/2345607053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2345607053},{"type":"v1","addr":"172.21.15.138:6827","nonce":2345607053}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:25:53.387388+0000","modified":"2024-05-27T09:33:47.228896+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14472,"mds_1":24241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi119.ppmzzk","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6829/2100782307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2100782307},{"type":"v1","addr":"172.21.15.119:6829","nonce":2100782307}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24241":{"gid":24241,"name":"cephfs.smithi138.ndfkhd","rank":1,"incarnation":9,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6825/897058594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":897058594},{"type":"v1","addr":"172.21.15.138:6825","nonce":897058594}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-27T09:44:22.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.473+0000 7f6877200700 1 -- 172.21.15.119:0/692451885 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f685c078a40 msgr2=0x7f685c07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:22.473 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.473+0000 7f6877200700 1 --2- 172.21.15.119:0/692451885 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f685c078a40 0x7f685c07aef0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6868005fd0 tx=0x7f686800a560 comp rx=0 tx=0).stop 2024-05-27T09:44:22.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.473+0000 7f6877200700 1 -- 172.21.15.119:0/692451885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6878100bf0 msgr2=0x7f6878073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:22.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.473+0000 7f6877200700 1 --2- 172.21.15.119:0/692451885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6878100bf0 0x7f6878073590 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f686400bda0 tx=0x7f686400be80 comp rx=0 tx=0).stop 2024-05-27T09:44:22.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.473+0000 7f6877200700 1 -- 172.21.15.119:0/692451885 shutdown_connections 2024-05-27T09:44:22.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.473+0000 7f6877200700 1 --2- 172.21.15.119:0/692451885 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f685c078a40 0x7f685c07aef0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:22.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.473+0000 7f6877200700 1 --2- 172.21.15.119:0/692451885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6878100250 0x7f6878073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:22.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.473+0000 7f6877200700 1 --2- 172.21.15.119:0/692451885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6878100bf0 0x7f6878073590 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:22.474 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.473+0000 7f6877200700 1 -- 172.21.15.119:0/692451885 >> 172.21.15.119:0/692451885 conn(0x7f68780fbaf0 msgr2=0x7f68780fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:22.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.474+0000 7f6877200700 1 -- 172.21.15.119:0/692451885 shutdown_connections 2024-05-27T09:44:22.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:22.474+0000 7f6877200700 1 -- 172.21.15.119:0/692451885 wait complete. 2024-05-27T09:44:22.476 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 21 2024-05-27T09:44:22.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:22 smithi119 ceph-mon[88269]: pgmap v336: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:22.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:22 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/692451885' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-05-27T09:44:22.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:22 smithi138 ceph-mon[70004]: pgmap v336: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:22.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:22 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/692451885' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-05-27T09:44:23.351 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 22 2024-05-27T09:44:23.741 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:44:24.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:24 smithi138 ceph-mon[70004]: pgmap v337: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:24.920 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:24 smithi119 ceph-mon[88269]: pgmap v337: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:25.958 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.957+0000 7f7365c00700 1 -- 172.21.15.119:0/848745601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73601003d0 msgr2=0x7f73601007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:25.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.957+0000 7f7365c00700 1 --2- 172.21.15.119:0/848745601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73601003d0 0x7f73601007a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7350009910 tx=0x7f7350009c20 comp rx=0 tx=0).stop 2024-05-27T09:44:25.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.957+0000 7f7365c00700 1 -- 172.21.15.119:0/848745601 shutdown_connections 2024-05-27T09:44:25.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.957+0000 7f7365c00700 1 --2- 172.21.15.119:0/848745601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7360100d70 0x7f7360104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:25.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.957+0000 7f7365c00700 1 --2- 172.21.15.119:0/848745601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73601003d0 0x7f73601007a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:25.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.957+0000 7f7365c00700 1 -- 172.21.15.119:0/848745601 >> 172.21.15.119:0/848745601 conn(0x7f73600fbc90 msgr2=0x7f73600fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:25.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.958+0000 7f7365c00700 1 -- 172.21.15.119:0/848745601 shutdown_connections 2024-05-27T09:44:25.959 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.958+0000 7f7365c00700 1 -- 172.21.15.119:0/848745601 wait complete. 2024-05-27T09:44:25.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.959+0000 7f7365c00700 1 Processor -- start 2024-05-27T09:44:25.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.959+0000 7f7365c00700 1 -- start start 2024-05-27T09:44:25.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.959+0000 7f7365c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73601003d0 0x7f7360197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:25.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.959+0000 7f7365c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7360100d70 0x7f7360198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:25.960 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.959+0000 7f7365c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7360198b40 con 0x7f73601003d0 2024-05-27T09:44:25.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.959+0000 7f7365c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736019c8d0 con 0x7f7360100d70 2024-05-27T09:44:25.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.960+0000 7f735ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73601003d0 0x7f7360197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:25.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.960+0000 7f735ea00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73601003d0 0x7f7360197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53678/0 (socket says 172.21.15.119:53678) 2024-05-27T09:44:25.961 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.960+0000 7f735ea00700 1 -- 172.21.15.119:0/1083994384 learned_addr learned my addr 172.21.15.119:0/1083994384 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:44:25.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.960+0000 7f735e000700 1 --2- 172.21.15.119:0/1083994384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7360100d70 0x7f7360198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:25.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.960+0000 7f735ea00700 1 -- 172.21.15.119:0/1083994384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7360100d70 msgr2=0x7f7360198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:25.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.960+0000 7f735ea00700 1 --2- 172.21.15.119:0/1083994384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7360100d70 0x7f7360198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:25.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.960+0000 7f735ea00700 1 -- 172.21.15.119:0/1083994384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73500095f0 con 0x7f73601003d0 2024-05-27T09:44:25.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.960+0000 7f735e000700 1 --2- 172.21.15.119:0/1083994384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7360100d70 0x7f7360198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:44:25.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.960+0000 7f735ea00700 1 --2- 172.21.15.119:0/1083994384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73601003d0 0x7f7360197f20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7350005f50 tx=0x7f735000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:25.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.961+0000 7f7357400700 1 -- 172.21.15.119:0/1083994384 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f735001d070 con 0x7f73601003d0 2024-05-27T09:44:25.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.961+0000 7f7357400700 1 -- 172.21.15.119:0/1083994384 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f735000b610 con 0x7f73601003d0 2024-05-27T09:44:25.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.961+0000 7f7365c00700 1 -- 172.21.15.119:0/1083994384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f736019cb50 con 0x7f73601003d0 2024-05-27T09:44:25.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.961+0000 7f7357400700 1 -- 172.21.15.119:0/1083994384 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7350022470 con 0x7f73601003d0 2024-05-27T09:44:25.962 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.961+0000 7f7365c00700 1 -- 172.21.15.119:0/1083994384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f736019d040 con 0x7f73601003d0 2024-05-27T09:44:25.963 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.962+0000 7f7365c00700 1 -- 172.21.15.119:0/1083994384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73601044c0 con 0x7f73601003d0 2024-05-27T09:44:25.964 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.963+0000 7f7357400700 1 -- 172.21.15.119:0/1083994384 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f735000fca0 con 0x7f73601003d0 2024-05-27T09:44:25.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.964+0000 7f7357400700 1 --2- 172.21.15.119:0/1083994384 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7344078a40 0x7f734407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:25.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.964+0000 7f7357400700 1 -- 172.21.15.119:0/1083994384 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f735009c1c0 con 0x7f73601003d0 2024-05-27T09:44:25.965 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.965+0000 7f735e000700 1 --2- 172.21.15.119:0/1083994384 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7344078a40 0x7f734407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:25.966 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.965+0000 7f735e000700 1 --2- 172.21.15.119:0/1083994384 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7344078a40 0x7f734407aef0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7360199540 tx=0x7f734c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:25.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:25.970+0000 7f7357400700 1 -- 172.21.15.119:0/1083994384 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f7350063d70 con 0x7f73601003d0 2024-05-27T09:44:26.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:26.256+0000 7f7365c00700 1 -- 172.21.15.119:0/1083994384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f736004ea50 con 0x7f73601003d0 2024-05-27T09:44:26.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:26.258+0000 7f7357400700 1 -- 172.21.15.119:0/1083994384 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v39) v1 ==== 107+0+4878 (secure 0 0 0) 0x7f7350063710 con 0x7f73601003d0 2024-05-27T09:44:26.259 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:44:26.259 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14482,"name":"cephfs.smithi119.nghcey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6827/413736191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":413736191},{"type":"v1","addr":"172.21.15.119:6827","nonce":413736191}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24253,"name":"cephfs.smithi138.alszpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.138:6827/2345607053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2345607053},{"type":"v1","addr":"172.21.15.138:6827","nonce":2345607053}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:25:53.387388+0000","modified":"2024-05-27T09:34:01.250304+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14472,"mds_1":24241},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14472":{"gid":14472,"name":"cephfs.smithi119.ppmzzk","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.119:6829/2100782307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2100782307},{"type":"v1","addr":"172.21.15.119:6829","nonce":2100782307}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24241":{"gid":24241,"name":"cephfs.smithi138.ndfkhd","rank":1,"incarnation":9,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6825/897058594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":897058594},{"type":"v1","addr":"172.21.15.138:6825","nonce":897058594}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-27T09:44:26.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:26.261+0000 7f7365c00700 1 -- 172.21.15.119:0/1083994384 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7344078a40 msgr2=0x7f734407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:26.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:26.261+0000 7f7365c00700 1 --2- 172.21.15.119:0/1083994384 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7344078a40 0x7f734407aef0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7360199540 tx=0x7f734c009260 comp rx=0 tx=0).stop 2024-05-27T09:44:26.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:26.261+0000 7f7365c00700 1 -- 172.21.15.119:0/1083994384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73601003d0 msgr2=0x7f7360197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:26.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:26.261+0000 7f7365c00700 1 --2- 172.21.15.119:0/1083994384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73601003d0 0x7f7360197f20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7350005f50 tx=0x7f735000f740 comp rx=0 tx=0).stop 2024-05-27T09:44:26.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:26.261+0000 7f7365c00700 1 -- 172.21.15.119:0/1083994384 shutdown_connections 2024-05-27T09:44:26.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:26.261+0000 7f7365c00700 1 --2- 172.21.15.119:0/1083994384 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f7344078a40 0x7f734407aef0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:26.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:26.261+0000 7f7365c00700 1 --2- 172.21.15.119:0/1083994384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73601003d0 0x7f7360197f20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:26.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:26.261+0000 7f7365c00700 1 --2- 172.21.15.119:0/1083994384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7360100d70 0x7f7360198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:26.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:26.261+0000 7f7365c00700 1 -- 172.21.15.119:0/1083994384 >> 172.21.15.119:0/1083994384 conn(0x7f73600fbc90 msgr2=0x7f73600fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:26.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:26.261+0000 7f7365c00700 1 -- 172.21.15.119:0/1083994384 shutdown_connections 2024-05-27T09:44:26.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:26.262+0000 7f7365c00700 1 -- 172.21.15.119:0/1083994384 wait complete. 2024-05-27T09:44:26.263 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 22 2024-05-27T09:44:26.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:26 smithi138 ceph-mon[70004]: pgmap v338: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:26.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:26 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/1083994384' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-05-27T09:44:26.980 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:26 smithi119 ceph-mon[88269]: pgmap v338: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:26.980 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:26 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/1083994384' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-05-27T09:44:27.039 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 23 2024-05-27T09:44:27.467 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:44:27.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:27 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:44:27.976 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:27 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:44:28.678 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:28 smithi119 ceph-mon[88269]: pgmap v339: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:28.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:28 smithi138 ceph-mon[70004]: pgmap v339: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:29.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.701+0000 7f525de00700 1 -- 172.21.15.119:0/1406617348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525810c010 msgr2=0x7f5258106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:29.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.701+0000 7f525de00700 1 --2- 172.21.15.119:0/1406617348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525810c010 0x7f5258106b80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5244009910 tx=0x7f5244009c20 comp rx=0 tx=0).stop 2024-05-27T09:44:29.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.701+0000 7f525de00700 1 -- 172.21.15.119:0/1406617348 shutdown_connections 2024-05-27T09:44:29.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.701+0000 7f525de00700 1 --2- 172.21.15.119:0/1406617348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525810c010 0x7f5258106b80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:29.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.701+0000 7f525de00700 1 --2- 172.21.15.119:0/1406617348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f525810b700 0x7f525810bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:29.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.701+0000 7f525de00700 1 -- 172.21.15.119:0/1406617348 >> 172.21.15.119:0/1406617348 conn(0x7f5258076000 msgr2=0x7f5258078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:29.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.701+0000 7f525de00700 1 -- 172.21.15.119:0/1406617348 shutdown_connections 2024-05-27T09:44:29.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.701+0000 7f525de00700 1 -- 172.21.15.119:0/1406617348 wait complete. 2024-05-27T09:44:29.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.702+0000 7f525de00700 1 Processor -- start 2024-05-27T09:44:29.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.703+0000 7f525de00700 1 -- start start 2024-05-27T09:44:29.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.703+0000 7f525de00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f525810b700 0x7f525819c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:29.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.703+0000 7f525de00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525810c010 0x7f525819c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:29.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.703+0000 7f525de00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525819cf00 con 0x7f525810c010 2024-05-27T09:44:29.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.703+0000 7f525de00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52581a0c90 con 0x7f525810b700 2024-05-27T09:44:29.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.703+0000 7f5256000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525810c010 0x7f525819c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:29.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.704+0000 7f5256000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525810c010 0x7f525819c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56434/0 (socket says 172.21.15.119:56434) 2024-05-27T09:44:29.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.704+0000 7f5256000700 1 -- 172.21.15.119:0/1326702489 learned_addr learned my addr 172.21.15.119:0/1326702489 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:44:29.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.704+0000 7f5256a00700 1 --2- 172.21.15.119:0/1326702489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f525810b700 0x7f525819c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:29.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.704+0000 7f5256000700 1 -- 172.21.15.119:0/1326702489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f525810b700 msgr2=0x7f525819c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:29.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.704+0000 7f5256000700 1 --2- 172.21.15.119:0/1326702489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f525810b700 0x7f525819c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:29.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.704+0000 7f5256000700 1 -- 172.21.15.119:0/1326702489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52440095f0 con 0x7f525810c010 2024-05-27T09:44:29.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.704+0000 7f5256000700 1 --2- 172.21.15.119:0/1326702489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525810c010 0x7f525819c820 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f524400bda0 tx=0x7f524400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:29.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.705+0000 7f525ca00700 1 -- 172.21.15.119:0/1326702489 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f524401d070 con 0x7f525810c010 2024-05-27T09:44:29.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.705+0000 7f525ca00700 1 -- 172.21.15.119:0/1326702489 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f524400f460 con 0x7f525810c010 2024-05-27T09:44:29.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.705+0000 7f525de00700 1 -- 172.21.15.119:0/1326702489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52581a0f10 con 0x7f525810c010 2024-05-27T09:44:29.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.705+0000 7f525ca00700 1 -- 172.21.15.119:0/1326702489 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5244017630 con 0x7f525810c010 2024-05-27T09:44:29.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.705+0000 7f525de00700 1 -- 172.21.15.119:0/1326702489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52581a1480 con 0x7f525810c010 2024-05-27T09:44:29.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.706+0000 7f525de00700 1 -- 172.21.15.119:0/1326702489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525804ea50 con 0x7f525810c010 2024-05-27T09:44:29.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.707+0000 7f525ca00700 1 -- 172.21.15.119:0/1326702489 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f524400fa40 con 0x7f525810c010 2024-05-27T09:44:29.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.709+0000 7f525ca00700 1 --2- 172.21.15.119:0/1326702489 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f523c078a40 0x7f523c07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:29.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.709+0000 7f525ca00700 1 -- 172.21.15.119:0/1326702489 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f524409c1a0 con 0x7f525810c010 2024-05-27T09:44:29.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.709+0000 7f5256a00700 1 --2- 172.21.15.119:0/1326702489 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f523c078a40 0x7f523c07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:29.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.710+0000 7f5256a00700 1 --2- 172.21.15.119:0/1326702489 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f523c078a40 0x7f523c07aef0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5248005950 tx=0x7f52480058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:29.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:29.715+0000 7f525ca00700 1 -- 172.21.15.119:0/1326702489 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f5244063d50 con 0x7f525810c010 2024-05-27T09:44:30.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:30.002+0000 7f525de00700 1 -- 172.21.15.119:0/1326702489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f52581a18b0 con 0x7f525810c010 2024-05-27T09:44:30.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:30.004+0000 7f525ca00700 1 -- 172.21.15.119:0/1326702489 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v39) v1 ==== 107+0+3296 (secure 0 0 0) 0x7f52440636f0 con 0x7f525810c010 2024-05-27T09:44:30.005 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:44:30.005 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14482,"name":"cephfs.smithi119.nghcey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6827/413736191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":413736191},{"type":"v1","addr":"172.21.15.119:6827","nonce":413736191}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24253,"name":"cephfs.smithi138.alszpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.138:6827/2345607053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2345607053},{"type":"v1","addr":"172.21.15.138:6827","nonce":2345607053}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:25:53.387388+0000","modified":"2024-05-27T09:39:57.913154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-27T09:44:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:30.007+0000 7f525de00700 1 -- 172.21.15.119:0/1326702489 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f523c078a40 msgr2=0x7f523c07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:30.007+0000 7f525de00700 1 --2- 172.21.15.119:0/1326702489 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f523c078a40 0x7f523c07aef0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5248005950 tx=0x7f52480058e0 comp rx=0 tx=0).stop 2024-05-27T09:44:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:30.007+0000 7f525de00700 1 -- 172.21.15.119:0/1326702489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525810c010 msgr2=0x7f525819c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:30.007+0000 7f525de00700 1 --2- 172.21.15.119:0/1326702489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525810c010 0x7f525819c820 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f524400bda0 tx=0x7f524400be80 comp rx=0 tx=0).stop 2024-05-27T09:44:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:30.007+0000 7f525de00700 1 -- 172.21.15.119:0/1326702489 shutdown_connections 2024-05-27T09:44:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:30.007+0000 7f525de00700 1 --2- 172.21.15.119:0/1326702489 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f523c078a40 0x7f523c07aef0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:30.007+0000 7f525de00700 1 --2- 172.21.15.119:0/1326702489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f525810b700 0x7f525819c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:30.007+0000 7f525de00700 1 --2- 172.21.15.119:0/1326702489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525810c010 0x7f525819c820 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:30.007+0000 7f525de00700 1 -- 172.21.15.119:0/1326702489 >> 172.21.15.119:0/1326702489 conn(0x7f5258076000 msgr2=0x7f5258076940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:30.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:30.007+0000 7f525de00700 1 -- 172.21.15.119:0/1326702489 shutdown_connections 2024-05-27T09:44:30.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:30.007+0000 7f525de00700 1 -- 172.21.15.119:0/1326702489 wait complete. 2024-05-27T09:44:30.009 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 23 2024-05-27T09:44:30.720 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:30 smithi119 ceph-mon[88269]: pgmap v340: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:30.720 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:30 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/1326702489' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-05-27T09:44:30.776 DEBUG:tasks.fs:max_mds not reduced in epoch 23 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-27T09:44:30.777 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 24 2024-05-27T09:44:30.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:30 smithi138 ceph-mon[70004]: pgmap v340: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:30.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:30 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/1326702489' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-05-27T09:44:31.184 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:44:32.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:32 smithi119 ceph-mon[88269]: pgmap v341: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:32.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:32 smithi138 ceph-mon[70004]: pgmap v341: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:33.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.313+0000 7f6522c00700 1 -- 172.21.15.119:0/2561064512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c102f90 msgr2=0x7f651c106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:33.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.313+0000 7f6522c00700 1 --2- 172.21.15.119:0/2561064512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c102f90 0x7f651c106f60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6508009910 tx=0x7f6508009c20 comp rx=0 tx=0).stop 2024-05-27T09:44:33.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.313+0000 7f6522c00700 1 -- 172.21.15.119:0/2561064512 shutdown_connections 2024-05-27T09:44:33.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.313+0000 7f6522c00700 1 --2- 172.21.15.119:0/2561064512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c102f90 0x7f651c106f60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:33.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.313+0000 7f6522c00700 1 --2- 172.21.15.119:0/2561064512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651c1025f0 0x7f651c1029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:33.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.313+0000 7f6522c00700 1 -- 172.21.15.119:0/2561064512 >> 172.21.15.119:0/2561064512 conn(0x7f651c0fde90 msgr2=0x7f651c1002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:33.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.313+0000 7f6522c00700 1 -- 172.21.15.119:0/2561064512 shutdown_connections 2024-05-27T09:44:33.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.313+0000 7f6522c00700 1 -- 172.21.15.119:0/2561064512 wait complete. 2024-05-27T09:44:33.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.314+0000 7f6522c00700 1 Processor -- start 2024-05-27T09:44:33.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.314+0000 7f6522c00700 1 -- start start 2024-05-27T09:44:33.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.315+0000 7f6522c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c1025f0 0x7f651c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:33.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.315+0000 7f6522c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651c102f90 0x7f651c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:33.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.315+0000 7f6522c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c198c20 con 0x7f651c1025f0 2024-05-27T09:44:33.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.315+0000 7f6522c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c19c9b0 con 0x7f651c102f90 2024-05-27T09:44:33.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.315+0000 7f651be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c1025f0 0x7f651c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:33.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.315+0000 7f651be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c1025f0 0x7f651c198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56442/0 (socket says 172.21.15.119:56442) 2024-05-27T09:44:33.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.315+0000 7f651be00700 1 -- 172.21.15.119:0/3045015222 learned_addr learned my addr 172.21.15.119:0/3045015222 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:44:33.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.315+0000 7f651b400700 1 --2- 172.21.15.119:0/3045015222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651c102f90 0x7f651c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:33.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.316+0000 7f651be00700 1 -- 172.21.15.119:0/3045015222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651c102f90 msgr2=0x7f651c198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:33.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.316+0000 7f651be00700 1 --2- 172.21.15.119:0/3045015222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651c102f90 0x7f651c198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:33.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.316+0000 7f651be00700 1 -- 172.21.15.119:0/3045015222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65080095f0 con 0x7f651c1025f0 2024-05-27T09:44:33.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.316+0000 7f651b400700 1 --2- 172.21.15.119:0/3045015222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651c102f90 0x7f651c198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:44:33.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.316+0000 7f651be00700 1 --2- 172.21.15.119:0/3045015222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c1025f0 0x7f651c198000 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f650c00d8d0 tx=0x7f650c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:33.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.316+0000 7f6518c00700 1 -- 172.21.15.119:0/3045015222 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f650c009940 con 0x7f651c1025f0 2024-05-27T09:44:33.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.316+0000 7f6518c00700 1 -- 172.21.15.119:0/3045015222 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f650c010460 con 0x7f651c1025f0 2024-05-27T09:44:33.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.316+0000 7f6522c00700 1 -- 172.21.15.119:0/3045015222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651c19cc90 con 0x7f651c1025f0 2024-05-27T09:44:33.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.316+0000 7f6518c00700 1 -- 172.21.15.119:0/3045015222 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f650c00f5f0 con 0x7f651c1025f0 2024-05-27T09:44:33.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.316+0000 7f6522c00700 1 -- 172.21.15.119:0/3045015222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651c19d1e0 con 0x7f651c1025f0 2024-05-27T09:44:33.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.318+0000 7f6522c00700 1 -- 172.21.15.119:0/3045015222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f651c10a960 con 0x7f651c1025f0 2024-05-27T09:44:33.319 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.319+0000 7f6518c00700 1 -- 172.21.15.119:0/3045015222 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f650c0105d0 con 0x7f651c1025f0 2024-05-27T09:44:33.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.320+0000 7f6518c00700 1 --2- 172.21.15.119:0/3045015222 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f6500078a40 0x7f650007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:33.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.320+0000 7f6518c00700 1 -- 172.21.15.119:0/3045015222 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f650c09ae80 con 0x7f651c1025f0 2024-05-27T09:44:33.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.320+0000 7f651b400700 1 --2- 172.21.15.119:0/3045015222 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f6500078a40 0x7f650007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:33.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.321+0000 7f651b400700 1 --2- 172.21.15.119:0/3045015222 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f6500078a40 0x7f650007aef0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f651c199620 tx=0x7f650801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:33.326 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.326+0000 7f6518c00700 1 -- 172.21.15.119:0/3045015222 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f650c0629b0 con 0x7f651c1025f0 2024-05-27T09:44:33.623 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.622+0000 7f6522c00700 1 -- 172.21.15.119:0/3045015222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f651c04ea50 con 0x7f651c1025f0 2024-05-27T09:44:33.624 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.623+0000 7f6518c00700 1 -- 172.21.15.119:0/3045015222 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v39) v1 ==== 107+0+4860 (secure 0 0 0) 0x7f650c062350 con 0x7f651c1025f0 2024-05-27T09:44:33.624 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:44:33.624 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14482,"name":"cephfs.smithi119.nghcey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6827/413736191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":413736191},{"type":"v1","addr":"172.21.15.119:6827","nonce":413736191}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24253,"name":"cephfs.smithi138.alszpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.138:6827/2345607053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2345607053},{"type":"v1","addr":"172.21.15.138:6827","nonce":2345607053}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":34428,"name":"cephfs.smithi119.ppmzzk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6829/2854413001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2854413001},{"type":"v1","addr":"172.21.15.119:6829","nonce":2854413001}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44261,"name":"cephfs.smithi138.ndfkhd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/2055700210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2055700210},{"type":"v1","addr":"172.21.15.138:6825","nonce":2055700210}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:25:53.387388+0000","modified":"2024-05-27T09:39:57.913154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-27T09:44:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.627+0000 7f6522c00700 1 -- 172.21.15.119:0/3045015222 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f6500078a40 msgr2=0x7f650007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.627+0000 7f6522c00700 1 --2- 172.21.15.119:0/3045015222 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f6500078a40 0x7f650007aef0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f651c199620 tx=0x7f650801a040 comp rx=0 tx=0).stop 2024-05-27T09:44:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.627+0000 7f6522c00700 1 -- 172.21.15.119:0/3045015222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c1025f0 msgr2=0x7f651c198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.627+0000 7f6522c00700 1 --2- 172.21.15.119:0/3045015222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c1025f0 0x7f651c198000 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f650c00d8d0 tx=0x7f650c00dc90 comp rx=0 tx=0).stop 2024-05-27T09:44:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.627+0000 7f6522c00700 1 -- 172.21.15.119:0/3045015222 shutdown_connections 2024-05-27T09:44:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.627+0000 7f6522c00700 1 --2- 172.21.15.119:0/3045015222 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f6500078a40 0x7f650007aef0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.627+0000 7f6522c00700 1 --2- 172.21.15.119:0/3045015222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c1025f0 0x7f651c198000 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.627+0000 7f6522c00700 1 --2- 172.21.15.119:0/3045015222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651c102f90 0x7f651c198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.627+0000 7f6522c00700 1 -- 172.21.15.119:0/3045015222 >> 172.21.15.119:0/3045015222 conn(0x7f651c0fde90 msgr2=0x7f651c0ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:33.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.627+0000 7f6522c00700 1 -- 172.21.15.119:0/3045015222 shutdown_connections 2024-05-27T09:44:33.629 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:33.627+0000 7f6522c00700 1 -- 172.21.15.119:0/3045015222 wait complete. 2024-05-27T09:44:33.629 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 24 2024-05-27T09:44:34.357 DEBUG:tasks.fs:max_mds not reduced in epoch 24 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-27T09:44:34.357 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 25 2024-05-27T09:44:34.762 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:44:34.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:34 smithi138 ceph-mon[70004]: pgmap v342: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:34.886 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:34 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/3045015222' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-05-27T09:44:34.913 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:34 smithi119 ceph-mon[88269]: pgmap v342: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:34.913 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:34 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/3045015222' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-05-27T09:44:36.838 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:36 smithi119 ceph-mon[88269]: pgmap v343: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:36.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:36 smithi138 ceph-mon[70004]: pgmap v343: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:37.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.001+0000 7fcc74200700 1 -- 172.21.15.119:0/2494632288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc6c10b700 msgr2=0x7fcc6c10bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:37.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.001+0000 7fcc74200700 1 --2- 172.21.15.119:0/2494632288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc6c10b700 0x7fcc6c10bad0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcc60009910 tx=0x7fcc60009c20 comp rx=0 tx=0).stop 2024-05-27T09:44:37.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.001+0000 7fcc74200700 1 -- 172.21.15.119:0/2494632288 shutdown_connections 2024-05-27T09:44:37.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.001+0000 7fcc74200700 1 --2- 172.21.15.119:0/2494632288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc6c10c010 0x7fcc6c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:37.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.001+0000 7fcc74200700 1 --2- 172.21.15.119:0/2494632288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc6c10b700 0x7fcc6c10bad0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:37.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.001+0000 7fcc74200700 1 -- 172.21.15.119:0/2494632288 >> 172.21.15.119:0/2494632288 conn(0x7fcc6c076000 msgr2=0x7fcc6c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:37.002 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.001+0000 7fcc74200700 1 -- 172.21.15.119:0/2494632288 shutdown_connections 2024-05-27T09:44:37.003 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.001+0000 7fcc74200700 1 -- 172.21.15.119:0/2494632288 wait complete. 2024-05-27T09:44:37.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.002+0000 7fcc74200700 1 Processor -- start 2024-05-27T09:44:37.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.002+0000 7fcc74200700 1 -- start start 2024-05-27T09:44:37.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.002+0000 7fcc74200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc6c10b700 0x7fcc6c19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:37.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.002+0000 7fcc74200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc6c10c010 0x7fcc6c19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:37.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.002+0000 7fcc74200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c19ced0 con 0x7fcc6c10c010 2024-05-27T09:44:37.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.002+0000 7fcc74200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c1a0c60 con 0x7fcc6c10b700 2024-05-27T09:44:37.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.003+0000 7fcc70e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc6c10c010 0x7fcc6c19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:37.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.003+0000 7fcc70e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc6c10c010 0x7fcc6c19c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56470/0 (socket says 172.21.15.119:56470) 2024-05-27T09:44:37.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.003+0000 7fcc70e00700 1 -- 172.21.15.119:0/1888832524 learned_addr learned my addr 172.21.15.119:0/1888832524 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:44:37.004 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.003+0000 7fcc71800700 1 --2- 172.21.15.119:0/1888832524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc6c10b700 0x7fcc6c19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:37.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.003+0000 7fcc70e00700 1 -- 172.21.15.119:0/1888832524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc6c10b700 msgr2=0x7fcc6c19c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:37.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.003+0000 7fcc70e00700 1 --2- 172.21.15.119:0/1888832524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc6c10b700 0x7fcc6c19c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:37.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.003+0000 7fcc70e00700 1 -- 172.21.15.119:0/1888832524 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc600095f0 con 0x7fcc6c10c010 2024-05-27T09:44:37.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.004+0000 7fcc70e00700 1 --2- 172.21.15.119:0/1888832524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc6c10c010 0x7fcc6c19c7f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800e8f0 tx=0x7fcc5800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:37.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.004+0000 7fcc6a000700 1 -- 172.21.15.119:0/1888832524 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc58009790 con 0x7fcc6c10c010 2024-05-27T09:44:37.005 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.004+0000 7fcc6a000700 1 -- 172.21.15.119:0/1888832524 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc58004cf0 con 0x7fcc6c10c010 2024-05-27T09:44:37.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.004+0000 7fcc74200700 1 -- 172.21.15.119:0/1888832524 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6c1a0f40 con 0x7fcc6c10c010 2024-05-27T09:44:37.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.004+0000 7fcc6a000700 1 -- 172.21.15.119:0/1888832524 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc58005690 con 0x7fcc6c10c010 2024-05-27T09:44:37.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.004+0000 7fcc74200700 1 -- 172.21.15.119:0/1888832524 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6c1a1510 con 0x7fcc6c10c010 2024-05-27T09:44:37.006 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.005+0000 7fcc74200700 1 -- 172.21.15.119:0/1888832524 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6c04ea50 con 0x7fcc6c10c010 2024-05-27T09:44:37.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.006+0000 7fcc6a000700 1 -- 172.21.15.119:0/1888832524 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fcc58004750 con 0x7fcc6c10c010 2024-05-27T09:44:37.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.007+0000 7fcc6a000700 1 --2- 172.21.15.119:0/1888832524 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fcc54078a40 0x7fcc5407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:37.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.008+0000 7fcc6a000700 1 -- 172.21.15.119:0/1888832524 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcc58014070 con 0x7fcc6c10c010 2024-05-27T09:44:37.008 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.008+0000 7fcc71800700 1 --2- 172.21.15.119:0/1888832524 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fcc54078a40 0x7fcc5407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:37.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.009+0000 7fcc71800700 1 --2- 172.21.15.119:0/1888832524 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fcc54078a40 0x7fcc5407aef0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000b760 tx=0x7fcc60009f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:37.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.013+0000 7fcc6a000700 1 -- 172.21.15.119:0/1888832524 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fcc58062d20 con 0x7fcc6c10c010 2024-05-27T09:44:37.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.293+0000 7fcc74200700 1 -- 172.21.15.119:0/1888832524 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7fcc6c1a1910 con 0x7fcc6c10c010 2024-05-27T09:44:37.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.294+0000 7fcc6a000700 1 -- 172.21.15.119:0/1888832524 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v39) v1 ==== 107+0+4081 (secure 0 0 0) 0x7fcc580626c0 con 0x7fcc6c10c010 2024-05-27T09:44:37.295 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:44:37.295 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24253,"name":"cephfs.smithi138.alszpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.138:6827/2345607053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2345607053},{"type":"v1","addr":"172.21.15.138:6827","nonce":2345607053}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":34428,"name":"cephfs.smithi119.ppmzzk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6829/2854413001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2854413001},{"type":"v1","addr":"172.21.15.119:6829","nonce":2854413001}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44261,"name":"cephfs.smithi138.ndfkhd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/2055700210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2055700210},{"type":"v1","addr":"172.21.15.138:6825","nonce":2055700210}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:25:53.387388+0000","modified":"2024-05-27T09:39:57.913154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-27T09:44:37.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.297+0000 7fcc74200700 1 -- 172.21.15.119:0/1888832524 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fcc54078a40 msgr2=0x7fcc5407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:37.298 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.297+0000 7fcc74200700 1 --2- 172.21.15.119:0/1888832524 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fcc54078a40 0x7fcc5407aef0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000b760 tx=0x7fcc60009f90 comp rx=0 tx=0).stop 2024-05-27T09:44:37.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.297+0000 7fcc74200700 1 -- 172.21.15.119:0/1888832524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc6c10c010 msgr2=0x7fcc6c19c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:37.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.297+0000 7fcc74200700 1 --2- 172.21.15.119:0/1888832524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc6c10c010 0x7fcc6c19c7f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800e8f0 tx=0x7fcc5800ecb0 comp rx=0 tx=0).stop 2024-05-27T09:44:37.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.297+0000 7fcc74200700 1 -- 172.21.15.119:0/1888832524 shutdown_connections 2024-05-27T09:44:37.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.297+0000 7fcc74200700 1 --2- 172.21.15.119:0/1888832524 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fcc54078a40 0x7fcc5407aef0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:37.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.297+0000 7fcc74200700 1 --2- 172.21.15.119:0/1888832524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc6c10b700 0x7fcc6c19c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:37.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.297+0000 7fcc74200700 1 --2- 172.21.15.119:0/1888832524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc6c10c010 0x7fcc6c19c7f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:37.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.297+0000 7fcc74200700 1 -- 172.21.15.119:0/1888832524 >> 172.21.15.119:0/1888832524 conn(0x7fcc6c076000 msgr2=0x7fcc6c076880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:37.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.298+0000 7fcc74200700 1 -- 172.21.15.119:0/1888832524 shutdown_connections 2024-05-27T09:44:37.299 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:37.298+0000 7fcc74200700 1 -- 172.21.15.119:0/1888832524 wait complete. 2024-05-27T09:44:37.300 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 25 2024-05-27T09:44:37.885 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:37 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/1888832524' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-05-27T09:44:37.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:37 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/1888832524' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-05-27T09:44:38.130 DEBUG:tasks.fs:max_mds not reduced in epoch 25 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-27T09:44:38.131 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 26 2024-05-27T09:44:38.537 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:44:38.895 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:38 smithi119 ceph-mon[88269]: pgmap v344: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:39.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:38 smithi138 ceph-mon[70004]: pgmap v344: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:40.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.736+0000 7fe0f2800700 1 -- 172.21.15.119:0/3820221985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0ec1003b0 msgr2=0x7fe0ec100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:40.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.736+0000 7fe0f2800700 1 --2- 172.21.15.119:0/3820221985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0ec1003b0 0x7fe0ec100780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00b0a0 tx=0x7fe0dc00b3b0 comp rx=0 tx=0).stop 2024-05-27T09:44:40.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.736+0000 7fe0f2800700 1 -- 172.21.15.119:0/3820221985 shutdown_connections 2024-05-27T09:44:40.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.736+0000 7fe0f2800700 1 --2- 172.21.15.119:0/3820221985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0ec100d50 0x7fe0ec104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:40.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.736+0000 7fe0f2800700 1 --2- 172.21.15.119:0/3820221985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0ec1003b0 0x7fe0ec100780 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:40.811 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.736+0000 7fe0f2800700 1 -- 172.21.15.119:0/3820221985 >> 172.21.15.119:0/3820221985 conn(0x7fe0ec0fbc30 msgr2=0x7fe0ec0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:40.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.736+0000 7fe0f2800700 1 -- 172.21.15.119:0/3820221985 shutdown_connections 2024-05-27T09:44:40.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.736+0000 7fe0f2800700 1 -- 172.21.15.119:0/3820221985 wait complete. 2024-05-27T09:44:40.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.738+0000 7fe0f2800700 1 Processor -- start 2024-05-27T09:44:40.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.738+0000 7fe0f2800700 1 -- start start 2024-05-27T09:44:40.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.738+0000 7fe0f2800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0ec1003b0 0x7fe0ec195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:40.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.738+0000 7fe0f2800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0ec100d50 0x7fe0ec1963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:40.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.738+0000 7fe0f2800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec196aa0 con 0x7fe0ec100d50 2024-05-27T09:44:40.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.738+0000 7fe0f2800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec19a830 con 0x7fe0ec1003b0 2024-05-27T09:44:40.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.738+0000 7fe0e3e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0ec100d50 0x7fe0ec1963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:40.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.739+0000 7fe0e3e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0ec100d50 0x7fe0ec1963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58694/0 (socket says 172.21.15.119:58694) 2024-05-27T09:44:40.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.739+0000 7fe0e3e00700 1 -- 172.21.15.119:0/1739575798 learned_addr learned my addr 172.21.15.119:0/1739575798 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:44:40.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.739+0000 7fe0eb400700 1 --2- 172.21.15.119:0/1739575798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0ec1003b0 0x7fe0ec195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:40.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.739+0000 7fe0e3e00700 1 -- 172.21.15.119:0/1739575798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0ec1003b0 msgr2=0x7fe0ec195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:40.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.739+0000 7fe0e3e00700 1 --2- 172.21.15.119:0/1739575798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0ec1003b0 0x7fe0ec195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:40.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.739+0000 7fe0e3e00700 1 -- 172.21.15.119:0/1739575798 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc009d00 con 0x7fe0ec100d50 2024-05-27T09:44:40.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.739+0000 7fe0e3e00700 1 --2- 172.21.15.119:0/1739575798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0ec100d50 0x7fe0ec1963c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400b480 tx=0x7fe0d400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:40.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.740+0000 7fe0e8c00700 1 -- 172.21.15.119:0/1739575798 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0d40118c0 con 0x7fe0ec100d50 2024-05-27T09:44:40.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.740+0000 7fe0e8c00700 1 -- 172.21.15.119:0/1739575798 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe0d4010380 con 0x7fe0ec100d50 2024-05-27T09:44:40.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.740+0000 7fe0f2800700 1 -- 172.21.15.119:0/1739575798 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0ec19ab10 con 0x7fe0ec100d50 2024-05-27T09:44:40.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.740+0000 7fe0e8c00700 1 -- 172.21.15.119:0/1739575798 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0d400f4f0 con 0x7fe0ec100d50 2024-05-27T09:44:40.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.740+0000 7fe0f2800700 1 -- 172.21.15.119:0/1739575798 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ec19b060 con 0x7fe0ec100d50 2024-05-27T09:44:40.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.741+0000 7fe0f2800700 1 -- 172.21.15.119:0/1739575798 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0ec1086c0 con 0x7fe0ec100d50 2024-05-27T09:44:40.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.742+0000 7fe0e8c00700 1 -- 172.21.15.119:0/1739575798 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7fe0d40104f0 con 0x7fe0ec100d50 2024-05-27T09:44:40.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.744+0000 7fe0e8c00700 1 --2- 172.21.15.119:0/1739575798 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe0d0078a40 0x7fe0d007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:40.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.744+0000 7fe0e8c00700 1 -- 172.21.15.119:0/1739575798 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe0d409aa30 con 0x7fe0ec100d50 2024-05-27T09:44:40.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.744+0000 7fe0eb400700 1 --2- 172.21.15.119:0/1739575798 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe0d0078a40 0x7fe0d007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:40.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.745+0000 7fe0eb400700 1 --2- 172.21.15.119:0/1739575798 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe0d0078a40 0x7fe0d007aef0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc009480 tx=0x7fe0dc00b7f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:40.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:40.749+0000 7fe0e8c00700 1 -- 172.21.15.119:0/1739575798 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7fe0d4062560 con 0x7fe0ec100d50 2024-05-27T09:44:40.814 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:40 smithi119 ceph-mon[88269]: pgmap v345: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:41.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:41.034+0000 7fe0f2800700 1 -- 172.21.15.119:0/1739575798 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7fe0ec19b2e0 con 0x7fe0ec100d50 2024-05-27T09:44:41.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:41.036+0000 7fe0e8c00700 1 -- 172.21.15.119:0/1739575798 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v39) v1 ==== 107+0+4931 (secure 0 0 0) 0x7fe0d4061f00 con 0x7fe0ec100d50 2024-05-27T09:44:41.038 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:44:41.038 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24253,"name":"cephfs.smithi138.alszpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.138:6827/2345607053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2345607053},{"type":"v1","addr":"172.21.15.138:6827","nonce":2345607053}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":34428,"name":"cephfs.smithi119.ppmzzk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6829/2854413001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2854413001},{"type":"v1","addr":"172.21.15.119:6829","nonce":2854413001}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":34436,"name":"cephfs.smithi119.nghcey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6827/3357235708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3357235708},{"type":"v1","addr":"172.21.15.119:6827","nonce":3357235708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44261,"name":"cephfs.smithi138.ndfkhd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/2055700210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2055700210},{"type":"v1","addr":"172.21.15.138:6825","nonce":2055700210}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:25:53.387388+0000","modified":"2024-05-27T09:39:57.913154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-27T09:44:41.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:41.039+0000 7fe0f2800700 1 -- 172.21.15.119:0/1739575798 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe0d0078a40 msgr2=0x7fe0d007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:41.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:41.039+0000 7fe0f2800700 1 --2- 172.21.15.119:0/1739575798 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe0d0078a40 0x7fe0d007aef0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc009480 tx=0x7fe0dc00b7f0 comp rx=0 tx=0).stop 2024-05-27T09:44:41.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:41.039+0000 7fe0f2800700 1 -- 172.21.15.119:0/1739575798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0ec100d50 msgr2=0x7fe0ec1963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:41.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:41.039+0000 7fe0f2800700 1 --2- 172.21.15.119:0/1739575798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0ec100d50 0x7fe0ec1963c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400b480 tx=0x7fe0d400b840 comp rx=0 tx=0).stop 2024-05-27T09:44:41.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:41.039+0000 7fe0f2800700 1 -- 172.21.15.119:0/1739575798 shutdown_connections 2024-05-27T09:44:41.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:41.039+0000 7fe0f2800700 1 --2- 172.21.15.119:0/1739575798 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7fe0d0078a40 0x7fe0d007aef0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:41.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:41.039+0000 7fe0f2800700 1 --2- 172.21.15.119:0/1739575798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0ec1003b0 0x7fe0ec195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:41.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:41.039+0000 7fe0f2800700 1 --2- 172.21.15.119:0/1739575798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0ec100d50 0x7fe0ec1963c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:41.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:41.039+0000 7fe0f2800700 1 -- 172.21.15.119:0/1739575798 >> 172.21.15.119:0/1739575798 conn(0x7fe0ec0fbc30 msgr2=0x7fe0ec0fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:41.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:41.039+0000 7fe0f2800700 1 -- 172.21.15.119:0/1739575798 shutdown_connections 2024-05-27T09:44:41.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:41.039+0000 7fe0f2800700 1 -- 172.21.15.119:0/1739575798 wait complete. 2024-05-27T09:44:41.042 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 26 2024-05-27T09:44:41.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:40 smithi138 ceph-mon[70004]: pgmap v345: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:41.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:41 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/1739575798' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-05-27T09:44:41.867 DEBUG:tasks.fs:max_mds not reduced in epoch 26 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-27T09:44:41.867 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 27 2024-05-27T09:44:42.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:41 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/1739575798' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-05-27T09:44:42.261 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:44:42.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:42 smithi119 ceph-mon[88269]: pgmap v346: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:42.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:42 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:44:43.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:42 smithi138 ceph-mon[70004]: pgmap v346: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:43.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:42 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:44:44.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.481+0000 7f43ad800700 1 -- 172.21.15.119:0/1257600109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43a8105f90 msgr2=0x7f43a8106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:44.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.481+0000 7f43ad800700 1 --2- 172.21.15.119:0/1257600109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43a8105f90 0x7f43a8106360 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4398009910 tx=0x7f4398009c20 comp rx=0 tx=0).stop 2024-05-27T09:44:44.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.481+0000 7f43ad800700 1 -- 172.21.15.119:0/1257600109 shutdown_connections 2024-05-27T09:44:44.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.481+0000 7f43ad800700 1 --2- 172.21.15.119:0/1257600109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43a80683d0 0x7f43a8068840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:44.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.481+0000 7f43ad800700 1 --2- 172.21.15.119:0/1257600109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43a8105f90 0x7f43a8106360 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:44.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.481+0000 7f43ad800700 1 -- 172.21.15.119:0/1257600109 >> 172.21.15.119:0/1257600109 conn(0x7f43a8074d00 msgr2=0x7f43a8075100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:44.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.481+0000 7f43ad800700 1 -- 172.21.15.119:0/1257600109 shutdown_connections 2024-05-27T09:44:44.484 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.481+0000 7f43ad800700 1 -- 172.21.15.119:0/1257600109 wait complete. 2024-05-27T09:44:44.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.482+0000 7f43ad800700 1 Processor -- start 2024-05-27T09:44:44.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.483+0000 7f43ad800700 1 -- start start 2024-05-27T09:44:44.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.483+0000 7f43ad800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43a80683d0 0x7f43a819b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:44.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.483+0000 7f43ad800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43a8105f90 0x7f43a819be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:44.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.483+0000 7f43ad800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a819c490 con 0x7f43a8105f90 2024-05-27T09:44:44.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.483+0000 7f43ad800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a81959d0 con 0x7f43a80683d0 2024-05-27T09:44:44.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.484+0000 7f43a7400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43a8105f90 0x7f43a819be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:44.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.484+0000 7f43a7400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43a8105f90 0x7f43a819be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58712/0 (socket says 172.21.15.119:58712) 2024-05-27T09:44:44.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.484+0000 7f43a7400700 1 -- 172.21.15.119:0/2666735235 learned_addr learned my addr 172.21.15.119:0/2666735235 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:44:44.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.484+0000 7f43a7e00700 1 --2- 172.21.15.119:0/2666735235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43a80683d0 0x7f43a819b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:44.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.484+0000 7f43a7400700 1 -- 172.21.15.119:0/2666735235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43a80683d0 msgr2=0x7f43a819b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:44.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.484+0000 7f43a7400700 1 --2- 172.21.15.119:0/2666735235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43a80683d0 0x7f43a819b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:44.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.484+0000 7f43a7400700 1 -- 172.21.15.119:0/2666735235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43980095f0 con 0x7f43a8105f90 2024-05-27T09:44:44.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.485+0000 7f43a7400700 1 --2- 172.21.15.119:0/2666735235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43a8105f90 0x7f43a819be90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f439400c930 tx=0x7f439400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:44.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.485+0000 7f43a4c00700 1 -- 172.21.15.119:0/2666735235 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4394007ad0 con 0x7f43a8105f90 2024-05-27T09:44:44.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.485+0000 7f43a4c00700 1 -- 172.21.15.119:0/2666735235 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4394007c30 con 0x7f43a8105f90 2024-05-27T09:44:44.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.485+0000 7f43ad800700 1 -- 172.21.15.119:0/2666735235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a8195cb0 con 0x7f43a8105f90 2024-05-27T09:44:44.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.485+0000 7f43a4c00700 1 -- 172.21.15.119:0/2666735235 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4394018610 con 0x7f43a8105f90 2024-05-27T09:44:44.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.485+0000 7f43ad800700 1 -- 172.21.15.119:0/2666735235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a8196200 con 0x7f43a8105f90 2024-05-27T09:44:44.488 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.486+0000 7f43ad800700 1 -- 172.21.15.119:0/2666735235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43a8108470 con 0x7f43a8105f90 2024-05-27T09:44:44.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.488+0000 7f43a4c00700 1 -- 172.21.15.119:0/2666735235 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f439400f450 con 0x7f43a8105f90 2024-05-27T09:44:44.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.489+0000 7f43a4c00700 1 --2- 172.21.15.119:0/2666735235 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f438c078a40 0x7f438c07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:44.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.489+0000 7f43a4c00700 1 -- 172.21.15.119:0/2666735235 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f439409afe0 con 0x7f43a8105f90 2024-05-27T09:44:44.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.489+0000 7f43a7e00700 1 --2- 172.21.15.119:0/2666735235 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f438c078a40 0x7f438c07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:44.491 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.490+0000 7f43a7e00700 1 --2- 172.21.15.119:0/2666735235 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f438c078a40 0x7f438c07aef0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f439800b5c0 tx=0x7f439801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:44.495 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.494+0000 7f43a4c00700 1 -- 172.21.15.119:0/2666735235 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f4394062b70 con 0x7f43a8105f90 2024-05-27T09:44:44.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:44 smithi119 ceph-mon[88269]: pgmap v347: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:44.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.780+0000 7f43ad800700 1 -- 172.21.15.119:0/2666735235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f43a804f2a0 con 0x7f43a8105f90 2024-05-27T09:44:44.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.781+0000 7f43a4c00700 1 -- 172.21.15.119:0/2666735235 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v39) v1 ==== 107+0+4149 (secure 0 0 0) 0x7f4394062510 con 0x7f43a8105f90 2024-05-27T09:44:44.783 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:44:44.783 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24253,"name":"cephfs.smithi138.alszpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.138:6827/2345607053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2345607053},{"type":"v1","addr":"172.21.15.138:6827","nonce":2345607053}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":34436,"name":"cephfs.smithi119.nghcey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6827/3357235708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3357235708},{"type":"v1","addr":"172.21.15.119:6827","nonce":3357235708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44261,"name":"cephfs.smithi138.ndfkhd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/2055700210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2055700210},{"type":"v1","addr":"172.21.15.138:6825","nonce":2055700210}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:25:53.387388+0000","modified":"2024-05-27T09:39:57.913154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-27T09:44:44.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.784+0000 7f43ad800700 1 -- 172.21.15.119:0/2666735235 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f438c078a40 msgr2=0x7f438c07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:44.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.784+0000 7f43ad800700 1 --2- 172.21.15.119:0/2666735235 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f438c078a40 0x7f438c07aef0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f439800b5c0 tx=0x7f439801a040 comp rx=0 tx=0).stop 2024-05-27T09:44:44.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.784+0000 7f43ad800700 1 -- 172.21.15.119:0/2666735235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43a8105f90 msgr2=0x7f43a819be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.784+0000 7f43ad800700 1 --2- 172.21.15.119:0/2666735235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43a8105f90 0x7f43a819be90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f439400c930 tx=0x7f439400ccf0 comp rx=0 tx=0).stop 2024-05-27T09:44:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.784+0000 7f43ad800700 1 -- 172.21.15.119:0/2666735235 shutdown_connections 2024-05-27T09:44:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.784+0000 7f43ad800700 1 --2- 172.21.15.119:0/2666735235 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f438c078a40 0x7f438c07aef0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.784+0000 7f43ad800700 1 --2- 172.21.15.119:0/2666735235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43a80683d0 0x7f43a819b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.784+0000 7f43ad800700 1 --2- 172.21.15.119:0/2666735235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43a8105f90 0x7f43a819be90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.784+0000 7f43ad800700 1 -- 172.21.15.119:0/2666735235 >> 172.21.15.119:0/2666735235 conn(0x7f43a8074d00 msgr2=0x7f43a80fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.785+0000 7f43ad800700 1 -- 172.21.15.119:0/2666735235 shutdown_connections 2024-05-27T09:44:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:44.785+0000 7f43ad800700 1 -- 172.21.15.119:0/2666735235 wait complete. 2024-05-27T09:44:44.788 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 27 2024-05-27T09:44:45.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:44 smithi138 ceph-mon[70004]: pgmap v347: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:45.602 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-27T09:44:45.602 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 28 2024-05-27T09:44:45.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:45 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/2666735235' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-05-27T09:44:46.023 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:44:46.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:45 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/2666735235' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-05-27T09:44:46.748 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:46 smithi119 ceph-mon[88269]: pgmap v348: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:47.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:46 smithi138 ceph-mon[70004]: pgmap v348: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:48.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.327+0000 7efc6fa00700 1 -- 172.21.15.119:0/1123292566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc68102f90 msgr2=0x7efc68106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:48.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.327+0000 7efc6fa00700 1 --2- 172.21.15.119:0/1123292566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc68102f90 0x7efc68106f60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efc54009910 tx=0x7efc54009c20 comp rx=0 tx=0).stop 2024-05-27T09:44:48.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.327+0000 7efc6fa00700 1 -- 172.21.15.119:0/1123292566 shutdown_connections 2024-05-27T09:44:48.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.327+0000 7efc6fa00700 1 --2- 172.21.15.119:0/1123292566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc68102f90 0x7efc68106f60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:48.329 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.327+0000 7efc6fa00700 1 --2- 172.21.15.119:0/1123292566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc681025f0 0x7efc681029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:48.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.327+0000 7efc6fa00700 1 -- 172.21.15.119:0/1123292566 >> 172.21.15.119:0/1123292566 conn(0x7efc680fde90 msgr2=0x7efc681002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:48.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.328+0000 7efc6fa00700 1 -- 172.21.15.119:0/1123292566 shutdown_connections 2024-05-27T09:44:48.330 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.328+0000 7efc6fa00700 1 -- 172.21.15.119:0/1123292566 wait complete. 2024-05-27T09:44:48.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.329+0000 7efc6fa00700 1 Processor -- start 2024-05-27T09:44:48.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.329+0000 7efc6fa00700 1 -- start start 2024-05-27T09:44:48.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.330+0000 7efc6fa00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc681025f0 0x7efc68198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:48.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.330+0000 7efc6fa00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc68102f90 0x7efc68198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:48.331 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.330+0000 7efc6fa00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc68198c70 con 0x7efc68102f90 2024-05-27T09:44:48.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.330+0000 7efc6fa00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6819c990 con 0x7efc681025f0 2024-05-27T09:44:48.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.330+0000 7efc67e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc68102f90 0x7efc68198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:48.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.330+0000 7efc67e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc68102f90 0x7efc68198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58716/0 (socket says 172.21.15.119:58716) 2024-05-27T09:44:48.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.330+0000 7efc67e00700 1 -- 172.21.15.119:0/1065388780 learned_addr learned my addr 172.21.15.119:0/1065388780 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:44:48.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.330+0000 7efc6d000700 1 --2- 172.21.15.119:0/1065388780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc681025f0 0x7efc68198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:48.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.331+0000 7efc67e00700 1 -- 172.21.15.119:0/1065388780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc681025f0 msgr2=0x7efc68198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:48.332 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.331+0000 7efc67e00700 1 --2- 172.21.15.119:0/1065388780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc681025f0 0x7efc68198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:48.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.331+0000 7efc67e00700 1 -- 172.21.15.119:0/1065388780 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc540095f0 con 0x7efc68102f90 2024-05-27T09:44:48.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.331+0000 7efc67e00700 1 --2- 172.21.15.119:0/1065388780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc68102f90 0x7efc68198590 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efc54005f50 tx=0x7efc5400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:48.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.331+0000 7efc65600700 1 -- 172.21.15.119:0/1065388780 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc5401d070 con 0x7efc68102f90 2024-05-27T09:44:48.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.331+0000 7efc65600700 1 -- 172.21.15.119:0/1065388780 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc5400f460 con 0x7efc68102f90 2024-05-27T09:44:48.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.331+0000 7efc6fa00700 1 -- 172.21.15.119:0/1065388780 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc6819cc10 con 0x7efc68102f90 2024-05-27T09:44:48.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.331+0000 7efc65600700 1 -- 172.21.15.119:0/1065388780 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc54017630 con 0x7efc68102f90 2024-05-27T09:44:48.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.331+0000 7efc6fa00700 1 -- 172.21.15.119:0/1065388780 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc6819d100 con 0x7efc68102f90 2024-05-27T09:44:48.334 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.333+0000 7efc6fa00700 1 -- 172.21.15.119:0/1065388780 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc6810a8b0 con 0x7efc68102f90 2024-05-27T09:44:48.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.334+0000 7efc65600700 1 -- 172.21.15.119:0/1065388780 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7efc5400f5d0 con 0x7efc68102f90 2024-05-27T09:44:48.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.335+0000 7efc65600700 1 --2- 172.21.15.119:0/1065388780 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efc50078a40 0x7efc5007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:48.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.335+0000 7efc65600700 1 -- 172.21.15.119:0/1065388780 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efc5409c410 con 0x7efc68102f90 2024-05-27T09:44:48.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.335+0000 7efc6d000700 1 --2- 172.21.15.119:0/1065388780 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efc50078a40 0x7efc5007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:48.336 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.336+0000 7efc6d000700 1 --2- 172.21.15.119:0/1065388780 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efc50078a40 0x7efc5007aef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efc5c006fd0 tx=0x7efc5c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:48.341 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.341+0000 7efc65600700 1 -- 172.21.15.119:0/1065388780 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7efc54063f40 con 0x7efc68102f90 2024-05-27T09:44:48.632 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.632+0000 7efc6fa00700 1 -- 172.21.15.119:0/1065388780 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7efc6802cc30 con 0x7efc68102f90 2024-05-27T09:44:48.633 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.633+0000 7efc65600700 1 -- 172.21.15.119:0/1065388780 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v39) v1 ==== 107+0+4999 (secure 0 0 0) 0x7efc540638e0 con 0x7efc68102f90 2024-05-27T09:44:48.634 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:44:48.634 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24253,"name":"cephfs.smithi138.alszpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.138:6827/2345607053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2345607053},{"type":"v1","addr":"172.21.15.138:6827","nonce":2345607053}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":34436,"name":"cephfs.smithi119.nghcey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6827/3357235708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3357235708},{"type":"v1","addr":"172.21.15.119:6827","nonce":3357235708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":34468,"name":"cephfs.smithi119.ppmzzk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6829/2430623742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2430623742},{"type":"v1","addr":"172.21.15.119:6829","nonce":2430623742}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44261,"name":"cephfs.smithi138.ndfkhd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/2055700210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2055700210},{"type":"v1","addr":"172.21.15.138:6825","nonce":2055700210}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:25:53.387388+0000","modified":"2024-05-27T09:39:57.913154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-27T09:44:48.636 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.636+0000 7efc6fa00700 1 -- 172.21.15.119:0/1065388780 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efc50078a40 msgr2=0x7efc5007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:48.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.636+0000 7efc6fa00700 1 --2- 172.21.15.119:0/1065388780 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efc50078a40 0x7efc5007aef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efc5c006fd0 tx=0x7efc5c008040 comp rx=0 tx=0).stop 2024-05-27T09:44:48.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.636+0000 7efc6fa00700 1 -- 172.21.15.119:0/1065388780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc68102f90 msgr2=0x7efc68198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:48.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.636+0000 7efc6fa00700 1 --2- 172.21.15.119:0/1065388780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc68102f90 0x7efc68198590 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efc54005f50 tx=0x7efc5400be20 comp rx=0 tx=0).stop 2024-05-27T09:44:48.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.636+0000 7efc6fa00700 1 -- 172.21.15.119:0/1065388780 shutdown_connections 2024-05-27T09:44:48.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.636+0000 7efc6fa00700 1 --2- 172.21.15.119:0/1065388780 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7efc50078a40 0x7efc5007aef0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:48.637 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.636+0000 7efc6fa00700 1 --2- 172.21.15.119:0/1065388780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc681025f0 0x7efc68198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:48.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.636+0000 7efc6fa00700 1 --2- 172.21.15.119:0/1065388780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc68102f90 0x7efc68198590 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:48.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.636+0000 7efc6fa00700 1 -- 172.21.15.119:0/1065388780 >> 172.21.15.119:0/1065388780 conn(0x7efc680fde90 msgr2=0x7efc680ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:48.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.636+0000 7efc6fa00700 1 -- 172.21.15.119:0/1065388780 shutdown_connections 2024-05-27T09:44:48.638 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:48.637+0000 7efc6fa00700 1 -- 172.21.15.119:0/1065388780 wait complete. 2024-05-27T09:44:48.638 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 28 2024-05-27T09:44:48.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:48 smithi119 ceph-mon[88269]: pgmap v349: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:48.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:48 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/1065388780' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-05-27T09:44:49.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:48 smithi138 ceph-mon[70004]: pgmap v349: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:49.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:48 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/1065388780' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-05-27T09:44:49.430 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-27T09:44:49.430 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 29 2024-05-27T09:44:49.836 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:44:50.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:50 smithi119 ceph-mon[88269]: pgmap v350: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:51.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:50 smithi138 ceph-mon[70004]: pgmap v350: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:52.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.012+0000 7f3a64e00700 1 -- 172.21.15.119:0/4075756244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a601003b0 msgr2=0x7f3a60100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:52.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.012+0000 7f3a64e00700 1 --2- 172.21.15.119:0/4075756244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a601003b0 0x7f3a60100780 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000b0a0 tx=0x7f3a5000b3b0 comp rx=0 tx=0).stop 2024-05-27T09:44:52.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.012+0000 7f3a64e00700 1 -- 172.21.15.119:0/4075756244 shutdown_connections 2024-05-27T09:44:52.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.012+0000 7f3a64e00700 1 --2- 172.21.15.119:0/4075756244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a60100d50 0x7f3a60104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:52.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.012+0000 7f3a64e00700 1 --2- 172.21.15.119:0/4075756244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a601003b0 0x7f3a60100780 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:52.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.012+0000 7f3a64e00700 1 -- 172.21.15.119:0/4075756244 >> 172.21.15.119:0/4075756244 conn(0x7f3a600fbc30 msgr2=0x7f3a600fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:52.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.012+0000 7f3a64e00700 1 -- 172.21.15.119:0/4075756244 shutdown_connections 2024-05-27T09:44:52.013 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.012+0000 7f3a64e00700 1 -- 172.21.15.119:0/4075756244 wait complete. 2024-05-27T09:44:52.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.013+0000 7f3a64e00700 1 Processor -- start 2024-05-27T09:44:52.014 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.013+0000 7f3a64e00700 1 -- start start 2024-05-27T09:44:52.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.014+0000 7f3a64e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a601003b0 0x7f3a60073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:52.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.014+0000 7f3a64e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a60100d50 0x7f3a60073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:52.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.014+0000 7f3a64e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a60073ad0 con 0x7f3a60100d50 2024-05-27T09:44:52.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.014+0000 7f3a64e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a60073c10 con 0x7f3a601003b0 2024-05-27T09:44:52.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.014+0000 7f3a55400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a60100d50 0x7f3a60073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:52.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.014+0000 7f3a55400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a60100d50 0x7f3a60073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49060/0 (socket says 172.21.15.119:49060) 2024-05-27T09:44:52.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.014+0000 7f3a55400700 1 -- 172.21.15.119:0/3584628026 learned_addr learned my addr 172.21.15.119:0/3584628026 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:44:52.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.015+0000 7f3a5de00700 1 --2- 172.21.15.119:0/3584628026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a601003b0 0x7f3a60073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:52.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.015+0000 7f3a55400700 1 -- 172.21.15.119:0/3584628026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a601003b0 msgr2=0x7f3a60073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:52.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.015+0000 7f3a55400700 1 --2- 172.21.15.119:0/3584628026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a601003b0 0x7f3a60073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:52.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.015+0000 7f3a55400700 1 -- 172.21.15.119:0/3584628026 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a50009d00 con 0x7f3a60100d50 2024-05-27T09:44:52.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.015+0000 7f3a55400700 1 --2- 172.21.15.119:0/3584628026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a60100d50 0x7f3a60073590 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800d900 tx=0x7f3a4800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:52.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.015+0000 7f3a57400700 1 -- 172.21.15.119:0/3584628026 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a48009900 con 0x7f3a60100d50 2024-05-27T09:44:52.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.015+0000 7f3a57400700 1 -- 172.21.15.119:0/3584628026 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a48010460 con 0x7f3a60100d50 2024-05-27T09:44:52.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.015+0000 7f3a64e00700 1 -- 172.21.15.119:0/3584628026 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a601a1f80 con 0x7f3a60100d50 2024-05-27T09:44:52.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.015+0000 7f3a57400700 1 -- 172.21.15.119:0/3584628026 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a4800f5f0 con 0x7f3a60100d50 2024-05-27T09:44:52.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.016+0000 7f3a64e00700 1 -- 172.21.15.119:0/3584628026 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a601a2390 con 0x7f3a60100d50 2024-05-27T09:44:52.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.017+0000 7f3a64e00700 1 -- 172.21.15.119:0/3584628026 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a601086c0 con 0x7f3a60100d50 2024-05-27T09:44:52.019 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.019+0000 7f3a57400700 1 -- 172.21.15.119:0/3584628026 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f3a480105d0 con 0x7f3a60100d50 2024-05-27T09:44:52.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.020+0000 7f3a57400700 1 --2- 172.21.15.119:0/3584628026 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3a44078a40 0x7f3a4407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:52.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.020+0000 7f3a57400700 1 -- 172.21.15.119:0/3584628026 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3a4809ac30 con 0x7f3a60100d50 2024-05-27T09:44:52.020 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.020+0000 7f3a5de00700 1 --2- 172.21.15.119:0/3584628026 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3a44078a40 0x7f3a4407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:52.021 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.021+0000 7f3a5de00700 1 --2- 172.21.15.119:0/3584628026 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3a44078a40 0x7f3a4407aef0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3a50009480 tx=0x7f3a5000b7f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:52.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.025+0000 7f3a57400700 1 -- 172.21.15.119:0/3584628026 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f3a480a1080 con 0x7f3a60100d50 2024-05-27T09:44:52.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.312+0000 7f3a64e00700 1 -- 172.21.15.119:0/3584628026 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f3a6004ea50 con 0x7f3a60100d50 2024-05-27T09:44:52.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.313+0000 7f3a57400700 1 -- 172.21.15.119:0/3584628026 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v39) v1 ==== 107+0+4217 (secure 0 0 0) 0x7f3a480627a0 con 0x7f3a60100d50 2024-05-27T09:44:52.314 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:44:52.314 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24253,"name":"cephfs.smithi138.alszpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.138:6827/2345607053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2345607053},{"type":"v1","addr":"172.21.15.138:6827","nonce":2345607053}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":34436,"name":"cephfs.smithi119.nghcey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6827/3357235708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3357235708},{"type":"v1","addr":"172.21.15.119:6827","nonce":3357235708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":34468,"name":"cephfs.smithi119.ppmzzk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6829/2430623742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2430623742},{"type":"v1","addr":"172.21.15.119:6829","nonce":2430623742}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:25:53.387388+0000","modified":"2024-05-27T09:39:57.913154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-27T09:44:52.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.316+0000 7f3a64e00700 1 -- 172.21.15.119:0/3584628026 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3a44078a40 msgr2=0x7f3a4407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:52.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.316+0000 7f3a64e00700 1 --2- 172.21.15.119:0/3584628026 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3a44078a40 0x7f3a4407aef0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3a50009480 tx=0x7f3a5000b7f0 comp rx=0 tx=0).stop 2024-05-27T09:44:52.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.316+0000 7f3a64e00700 1 -- 172.21.15.119:0/3584628026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a60100d50 msgr2=0x7f3a60073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:52.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.316+0000 7f3a64e00700 1 --2- 172.21.15.119:0/3584628026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a60100d50 0x7f3a60073590 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800d900 tx=0x7f3a4800dcc0 comp rx=0 tx=0).stop 2024-05-27T09:44:52.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.316+0000 7f3a64e00700 1 -- 172.21.15.119:0/3584628026 shutdown_connections 2024-05-27T09:44:52.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.316+0000 7f3a64e00700 1 --2- 172.21.15.119:0/3584628026 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f3a44078a40 0x7f3a4407aef0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:52.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.316+0000 7f3a64e00700 1 --2- 172.21.15.119:0/3584628026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a601003b0 0x7f3a60073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:52.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.316+0000 7f3a64e00700 1 --2- 172.21.15.119:0/3584628026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a60100d50 0x7f3a60073590 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:52.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.316+0000 7f3a64e00700 1 -- 172.21.15.119:0/3584628026 >> 172.21.15.119:0/3584628026 conn(0x7f3a600fbc30 msgr2=0x7f3a600fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:52.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.316+0000 7f3a64e00700 1 -- 172.21.15.119:0/3584628026 shutdown_connections 2024-05-27T09:44:52.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:52.316+0000 7f3a64e00700 1 -- 172.21.15.119:0/3584628026 wait complete. 2024-05-27T09:44:52.318 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 29 2024-05-27T09:44:52.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:52 smithi119 ceph-mon[88269]: pgmap v351: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:52.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:52 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/3584628026' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-05-27T09:44:53.099 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-27T09:44:53.099 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 30 2024-05-27T09:44:53.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:52 smithi138 ceph-mon[70004]: pgmap v351: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:53.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:52 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/3584628026' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-05-27T09:44:53.500 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:44:54.901 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:54 smithi119 ceph-mon[88269]: pgmap v352: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:55.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:54 smithi138 ceph-mon[70004]: pgmap v352: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:55.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.673+0000 7f86bac00700 1 -- 172.21.15.119:0/2383230244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86b4100200 msgr2=0x7f86b41005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:55.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.673+0000 7f86bac00700 1 --2- 172.21.15.119:0/2383230244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86b4100200 0x7f86b41005d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009910 tx=0x7f86a8009c20 comp rx=0 tx=0).stop 2024-05-27T09:44:55.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.674+0000 7f86bac00700 1 -- 172.21.15.119:0/2383230244 shutdown_connections 2024-05-27T09:44:55.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.674+0000 7f86bac00700 1 --2- 172.21.15.119:0/2383230244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86b4100ba0 0x7f86b4104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:55.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.674+0000 7f86bac00700 1 --2- 172.21.15.119:0/2383230244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86b4100200 0x7f86b41005d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:55.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.674+0000 7f86bac00700 1 -- 172.21.15.119:0/2383230244 >> 172.21.15.119:0/2383230244 conn(0x7f86b40fba80 msgr2=0x7f86b40fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:55.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.674+0000 7f86bac00700 1 -- 172.21.15.119:0/2383230244 shutdown_connections 2024-05-27T09:44:55.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.674+0000 7f86bac00700 1 -- 172.21.15.119:0/2383230244 wait complete. 2024-05-27T09:44:55.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.675+0000 7f86bac00700 1 Processor -- start 2024-05-27T09:44:55.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.675+0000 7f86bac00700 1 -- start start 2024-05-27T09:44:55.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.675+0000 7f86bac00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86b4100200 0x7f86b4197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:55.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.675+0000 7f86bac00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86b4100ba0 0x7f86b4198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:55.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.675+0000 7f86bac00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b4198980 con 0x7f86b4100200 2024-05-27T09:44:55.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.676+0000 7f86bac00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b4198ac0 con 0x7f86b4100ba0 2024-05-27T09:44:55.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.676+0000 7f86b9800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86b4100200 0x7f86b4197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:55.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.676+0000 7f86b9800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86b4100200 0x7f86b4197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49090/0 (socket says 172.21.15.119:49090) 2024-05-27T09:44:55.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.676+0000 7f86b9800700 1 -- 172.21.15.119:0/2347909161 learned_addr learned my addr 172.21.15.119:0/2347909161 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:44:55.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.676+0000 7f86b8e00700 1 --2- 172.21.15.119:0/2347909161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86b4100ba0 0x7f86b4198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:55.678 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.676+0000 7f86b9800700 1 -- 172.21.15.119:0/2347909161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86b4100ba0 msgr2=0x7f86b4198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:55.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.676+0000 7f86b9800700 1 --2- 172.21.15.119:0/2347909161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86b4100ba0 0x7f86b4198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:55.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.676+0000 7f86b9800700 1 -- 172.21.15.119:0/2347909161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a80095f0 con 0x7f86b4100200 2024-05-27T09:44:55.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.676+0000 7f86b8e00700 1 --2- 172.21.15.119:0/2347909161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86b4100ba0 0x7f86b4198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T09:44:55.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.677+0000 7f86b9800700 1 --2- 172.21.15.119:0/2347909161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86b4100200 0x7f86b4197df0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f86a800b5c0 tx=0x7f86a800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:55.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.677+0000 7f86b2000700 1 -- 172.21.15.119:0/2347909161 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86a801d070 con 0x7f86b4100200 2024-05-27T09:44:55.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.677+0000 7f86bac00700 1 -- 172.21.15.119:0/2347909161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b419c8b0 con 0x7f86b4100200 2024-05-27T09:44:55.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.677+0000 7f86b2000700 1 -- 172.21.15.119:0/2347909161 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f86a800f460 con 0x7f86b4100200 2024-05-27T09:44:55.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.677+0000 7f86bac00700 1 -- 172.21.15.119:0/2347909161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b419cda0 con 0x7f86b4100200 2024-05-27T09:44:55.679 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.677+0000 7f86b2000700 1 -- 172.21.15.119:0/2347909161 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86a802b610 con 0x7f86b4100200 2024-05-27T09:44:55.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.678+0000 7f86bac00700 1 -- 172.21.15.119:0/2347909161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b41042f0 con 0x7f86b4100200 2024-05-27T09:44:55.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.680+0000 7f86b2000700 1 -- 172.21.15.119:0/2347909161 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f86a800f5d0 con 0x7f86b4100200 2024-05-27T09:44:55.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.681+0000 7f86b2000700 1 --2- 172.21.15.119:0/2347909161 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f869c078a40 0x7f869c07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:55.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.681+0000 7f86b2000700 1 -- 172.21.15.119:0/2347909161 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f86a809c450 con 0x7f86b4100200 2024-05-27T09:44:55.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.681+0000 7f86b8e00700 1 --2- 172.21.15.119:0/2347909161 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f869c078a40 0x7f869c07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:55.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.682+0000 7f86b8e00700 1 --2- 172.21.15.119:0/2347909161 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f869c078a40 0x7f869c07aef0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f86a0005fd0 tx=0x7f86a0005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:55.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.687+0000 7f86b2000700 1 -- 172.21.15.119:0/2347909161 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f86a8063f80 con 0x7f86b4100200 2024-05-27T09:44:55.969 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.968+0000 7f86bac00700 1 -- 172.21.15.119:0/2347909161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f86b404f2a0 con 0x7f86b4100200 2024-05-27T09:44:55.970 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.969+0000 7f86b2000700 1 -- 172.21.15.119:0/2347909161 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v39) v1 ==== 107+0+5067 (secure 0 0 0) 0x7f86a8063920 con 0x7f86b4100200 2024-05-27T09:44:55.970 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:44:55.970 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24253,"name":"cephfs.smithi138.alszpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.138:6827/2345607053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2345607053},{"type":"v1","addr":"172.21.15.138:6827","nonce":2345607053}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":34436,"name":"cephfs.smithi119.nghcey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6827/3357235708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3357235708},{"type":"v1","addr":"172.21.15.119:6827","nonce":3357235708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":34468,"name":"cephfs.smithi119.ppmzzk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6829/2430623742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2430623742},{"type":"v1","addr":"172.21.15.119:6829","nonce":2430623742}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44289,"name":"cephfs.smithi138.ndfkhd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/3387966485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3387966485},{"type":"v1","addr":"172.21.15.138:6825","nonce":3387966485}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:25:53.387388+0000","modified":"2024-05-27T09:39:57.913154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-27T09:44:55.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.973+0000 7f86bac00700 1 -- 172.21.15.119:0/2347909161 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f869c078a40 msgr2=0x7f869c07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:55.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.973+0000 7f86bac00700 1 --2- 172.21.15.119:0/2347909161 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f869c078a40 0x7f869c07aef0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f86a0005fd0 tx=0x7f86a0005e20 comp rx=0 tx=0).stop 2024-05-27T09:44:55.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.973+0000 7f86bac00700 1 -- 172.21.15.119:0/2347909161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86b4100200 msgr2=0x7f86b4197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:55.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.973+0000 7f86bac00700 1 --2- 172.21.15.119:0/2347909161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86b4100200 0x7f86b4197df0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f86a800b5c0 tx=0x7f86a800be00 comp rx=0 tx=0).stop 2024-05-27T09:44:55.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.973+0000 7f86bac00700 1 -- 172.21.15.119:0/2347909161 shutdown_connections 2024-05-27T09:44:55.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.973+0000 7f86bac00700 1 --2- 172.21.15.119:0/2347909161 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f869c078a40 0x7f869c07aef0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:55.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.973+0000 7f86bac00700 1 --2- 172.21.15.119:0/2347909161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86b4100200 0x7f86b4197df0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:55.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.973+0000 7f86bac00700 1 --2- 172.21.15.119:0/2347909161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86b4100ba0 0x7f86b4198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:55.974 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.973+0000 7f86bac00700 1 -- 172.21.15.119:0/2347909161 >> 172.21.15.119:0/2347909161 conn(0x7f86b40fba80 msgr2=0x7f86b40fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:55.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.974+0000 7f86bac00700 1 -- 172.21.15.119:0/2347909161 shutdown_connections 2024-05-27T09:44:55.975 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:55.974+0000 7f86bac00700 1 -- 172.21.15.119:0/2347909161 wait complete. 2024-05-27T09:44:55.975 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 30 2024-05-27T09:44:56.792 DEBUG:tasks.fs:max_mds not reduced in epoch 30 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-27T09:44:56.793 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 31 2024-05-27T09:44:56.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:56 smithi119 ceph-mon[88269]: pgmap v353: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:56.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:56 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/2347909161' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-05-27T09:44:57.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:56 smithi138 ceph-mon[70004]: pgmap v353: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:57.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:56 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/2347909161' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-05-27T09:44:57.208 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:44:57.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:57 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:44:58.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:57 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:44:58.871 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:58 smithi119 ceph-mon[88269]: pgmap v354: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:59.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:58 smithi138 ceph-mon[70004]: pgmap v354: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:44:59.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.400+0000 7f2ef8c00700 1 -- 172.21.15.119:0/4148558906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ef4073a00 msgr2=0x7f2ef4110c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:59.401 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.400+0000 7f2ef8c00700 1 --2- 172.21.15.119:0/4148558906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ef4073a00 0x7f2ef4110c20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009910 tx=0x7f2ee0009c20 comp rx=0 tx=0).stop 2024-05-27T09:44:59.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.400+0000 7f2ef8c00700 1 -- 172.21.15.119:0/4148558906 shutdown_connections 2024-05-27T09:44:59.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.400+0000 7f2ef8c00700 1 --2- 172.21.15.119:0/4148558906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ef4073a00 0x7f2ef4110c20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:59.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.400+0000 7f2ef8c00700 1 --2- 172.21.15.119:0/4148558906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ef40730f0 0x7f2ef40734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:59.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.400+0000 7f2ef8c00700 1 -- 172.21.15.119:0/4148558906 >> 172.21.15.119:0/4148558906 conn(0x7f2ef40fbc30 msgr2=0x7f2ef40fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:59.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.400+0000 7f2ef8c00700 1 -- 172.21.15.119:0/4148558906 shutdown_connections 2024-05-27T09:44:59.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.401+0000 7f2ef8c00700 1 -- 172.21.15.119:0/4148558906 wait complete. 2024-05-27T09:44:59.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.402+0000 7f2ef8c00700 1 Processor -- start 2024-05-27T09:44:59.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.402+0000 7f2ef8c00700 1 -- start start 2024-05-27T09:44:59.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.402+0000 7f2ef8c00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ef40730f0 0x7f2ef419c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:59.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.402+0000 7f2ef8c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ef4073a00 0x7f2ef419c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:59.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.402+0000 7f2ef8c00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef419cf20 con 0x7f2ef40730f0 2024-05-27T09:44:59.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.402+0000 7f2ef8c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef41a0cb0 con 0x7f2ef4073a00 2024-05-27T09:44:59.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.403+0000 7f2ef1e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ef40730f0 0x7f2ef419c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:59.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.403+0000 7f2ef1e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ef40730f0 0x7f2ef419c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60950/0 (socket says 172.21.15.119:60950) 2024-05-27T09:44:59.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.403+0000 7f2ef1e00700 1 -- 172.21.15.119:0/3135760054 learned_addr learned my addr 172.21.15.119:0/3135760054 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:44:59.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.403+0000 7f2ef1400700 1 --2- 172.21.15.119:0/3135760054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ef4073a00 0x7f2ef419c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:59.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.403+0000 7f2ef1e00700 1 -- 172.21.15.119:0/3135760054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ef4073a00 msgr2=0x7f2ef419c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:59.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.403+0000 7f2ef1e00700 1 --2- 172.21.15.119:0/3135760054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ef4073a00 0x7f2ef419c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:59.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.403+0000 7f2ef1e00700 1 -- 172.21.15.119:0/3135760054 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee00095f0 con 0x7f2ef40730f0 2024-05-27T09:44:59.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.403+0000 7f2ef1e00700 1 --2- 172.21.15.119:0/3135760054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ef40730f0 0x7f2ef419c300 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400c8a0 tx=0x7f2ee400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:59.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.404+0000 7f2eeaa00700 1 -- 172.21.15.119:0/3135760054 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ee400cea0 con 0x7f2ef40730f0 2024-05-27T09:44:59.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.404+0000 7f2ef8c00700 1 -- 172.21.15.119:0/3135760054 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef41a0f90 con 0x7f2ef40730f0 2024-05-27T09:44:59.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.404+0000 7f2eeaa00700 1 -- 172.21.15.119:0/3135760054 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ee4004830 con 0x7f2ef40730f0 2024-05-27T09:44:59.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.404+0000 7f2eeaa00700 1 -- 172.21.15.119:0/3135760054 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ee40055b0 con 0x7f2ef40730f0 2024-05-27T09:44:59.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.404+0000 7f2ef8c00700 1 -- 172.21.15.119:0/3135760054 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef41a14e0 con 0x7f2ef40730f0 2024-05-27T09:44:59.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.405+0000 7f2ef8c00700 1 -- 172.21.15.119:0/3135760054 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef410e3a0 con 0x7f2ef40730f0 2024-05-27T09:44:59.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.406+0000 7f2eeaa00700 1 -- 172.21.15.119:0/3135760054 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f2ee40049a0 con 0x7f2ef40730f0 2024-05-27T09:44:59.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.407+0000 7f2eeaa00700 1 --2- 172.21.15.119:0/3135760054 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2ed8078a40 0x7f2ed807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:44:59.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.408+0000 7f2eeaa00700 1 -- 172.21.15.119:0/3135760054 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2ee409aac0 con 0x7f2ef40730f0 2024-05-27T09:44:59.408 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.408+0000 7f2ef1400700 1 --2- 172.21.15.119:0/3135760054 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2ed8078a40 0x7f2ed807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:44:59.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.408+0000 7f2ef1400700 1 --2- 172.21.15.119:0/3135760054 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2ed8078a40 0x7f2ed807aef0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0005f50 tx=0x7f2ee001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:44:59.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.413+0000 7f2eeaa00700 1 -- 172.21.15.119:0/3135760054 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f2ee40625f0 con 0x7f2ef40730f0 2024-05-27T09:44:59.699 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.698+0000 7f2ef8c00700 1 -- 172.21.15.119:0/3135760054 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f2ef404ea50 con 0x7f2ef40730f0 2024-05-27T09:44:59.700 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.699+0000 7f2eeaa00700 1 -- 172.21.15.119:0/3135760054 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v39) v1 ==== 107+0+4285 (secure 0 0 0) 0x7f2ef404ea50 con 0x7f2ef40730f0 2024-05-27T09:44:59.700 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:44:59.700 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi119.nghcey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6827/3357235708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3357235708},{"type":"v1","addr":"172.21.15.119:6827","nonce":3357235708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":34468,"name":"cephfs.smithi119.ppmzzk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6829/2430623742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2430623742},{"type":"v1","addr":"172.21.15.119:6829","nonce":2430623742}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44289,"name":"cephfs.smithi138.ndfkhd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/3387966485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3387966485},{"type":"v1","addr":"172.21.15.138:6825","nonce":3387966485}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:25:53.387388+0000","modified":"2024-05-27T09:39:57.913154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-27T09:44:59.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.703+0000 7f2ef8c00700 1 -- 172.21.15.119:0/3135760054 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2ed8078a40 msgr2=0x7f2ed807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:59.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.703+0000 7f2ef8c00700 1 --2- 172.21.15.119:0/3135760054 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2ed8078a40 0x7f2ed807aef0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0005f50 tx=0x7f2ee001a040 comp rx=0 tx=0).stop 2024-05-27T09:44:59.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.703+0000 7f2ef8c00700 1 -- 172.21.15.119:0/3135760054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ef40730f0 msgr2=0x7f2ef419c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:44:59.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.703+0000 7f2ef8c00700 1 --2- 172.21.15.119:0/3135760054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ef40730f0 0x7f2ef419c300 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400c8a0 tx=0x7f2ee400cc60 comp rx=0 tx=0).stop 2024-05-27T09:44:59.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.703+0000 7f2ef8c00700 1 -- 172.21.15.119:0/3135760054 shutdown_connections 2024-05-27T09:44:59.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.703+0000 7f2ef8c00700 1 --2- 172.21.15.119:0/3135760054 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f2ed8078a40 0x7f2ed807aef0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:59.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.703+0000 7f2ef8c00700 1 --2- 172.21.15.119:0/3135760054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ef40730f0 0x7f2ef419c300 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:59.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.703+0000 7f2ef8c00700 1 --2- 172.21.15.119:0/3135760054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ef4073a00 0x7f2ef419c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:44:59.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.703+0000 7f2ef8c00700 1 -- 172.21.15.119:0/3135760054 >> 172.21.15.119:0/3135760054 conn(0x7f2ef40fbc30 msgr2=0x7f2ef4102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:44:59.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.704+0000 7f2ef8c00700 1 -- 172.21.15.119:0/3135760054 shutdown_connections 2024-05-27T09:44:59.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:44:59.704+0000 7f2ef8c00700 1 -- 172.21.15.119:0/3135760054 wait complete. 2024-05-27T09:44:59.705 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 31 2024-05-27T09:44:59.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:44:59 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/3135760054' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-05-27T09:45:00.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:44:59 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/3135760054' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-05-27T09:45:00.540 DEBUG:tasks.fs:max_mds not reduced in epoch 31 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-27T09:45:00.541 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 32 2024-05-27T09:45:00.747 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:00 smithi119 ceph-mon[88269]: pgmap v355: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:00.955 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:45:01.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:00 smithi138 ceph-mon[70004]: pgmap v355: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:02.940 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:02 smithi119 ceph-mon[88269]: pgmap v356: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:03.112 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:02 smithi138 ceph-mon[70004]: pgmap v356: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:03.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.280+0000 7f1cb5e00700 1 -- 172.21.15.119:0/3515238582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cb0073a00 msgr2=0x7f1cb0110b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:03.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.280+0000 7f1cb5e00700 1 --2- 172.21.15.119:0/3515238582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cb0073a00 0x7f1cb0110b00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009910 tx=0x7f1c9c009c20 comp rx=0 tx=0).stop 2024-05-27T09:45:03.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.281+0000 7f1cb5e00700 1 -- 172.21.15.119:0/3515238582 shutdown_connections 2024-05-27T09:45:03.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.281+0000 7f1cb5e00700 1 --2- 172.21.15.119:0/3515238582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cb0073a00 0x7f1cb0110b00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:03.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.281+0000 7f1cb5e00700 1 --2- 172.21.15.119:0/3515238582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cb00730f0 0x7f1cb00734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:03.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.281+0000 7f1cb5e00700 1 -- 172.21.15.119:0/3515238582 >> 172.21.15.119:0/3515238582 conn(0x7f1cb00fbb90 msgr2=0x7f1cb00fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:45:03.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.281+0000 7f1cb5e00700 1 -- 172.21.15.119:0/3515238582 shutdown_connections 2024-05-27T09:45:03.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.281+0000 7f1cb5e00700 1 -- 172.21.15.119:0/3515238582 wait complete. 2024-05-27T09:45:03.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.282+0000 7f1cb5e00700 1 Processor -- start 2024-05-27T09:45:03.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.282+0000 7f1cb5e00700 1 -- start start 2024-05-27T09:45:03.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.282+0000 7f1cb5e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cb00730f0 0x7f1cb019c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:03.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.282+0000 7f1cb5e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cb0073a00 0x7f1cb019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:03.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.282+0000 7f1cb5e00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb019cde0 con 0x7f1cb00730f0 2024-05-27T09:45:03.283 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.282+0000 7f1cb5e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb01a0b70 con 0x7f1cb0073a00 2024-05-27T09:45:03.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.283+0000 7f1cb4a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cb00730f0 0x7f1cb019c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:03.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.283+0000 7f1cb4a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cb00730f0 0x7f1cb019c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60968/0 (socket says 172.21.15.119:60968) 2024-05-27T09:45:03.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.283+0000 7f1cb4a00700 1 -- 172.21.15.119:0/1682777275 learned_addr learned my addr 172.21.15.119:0/1682777275 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:45:03.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.283+0000 7f1cafe00700 1 --2- 172.21.15.119:0/1682777275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cb0073a00 0x7f1cb019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:03.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.283+0000 7f1cb4a00700 1 -- 172.21.15.119:0/1682777275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cb0073a00 msgr2=0x7f1cb019c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:03.284 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.283+0000 7f1cb4a00700 1 --2- 172.21.15.119:0/1682777275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cb0073a00 0x7f1cb019c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:03.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.283+0000 7f1cb4a00700 1 -- 172.21.15.119:0/1682777275 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9c0095f0 con 0x7f1cb00730f0 2024-05-27T09:45:03.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.284+0000 7f1cb4a00700 1 --2- 172.21.15.119:0/1682777275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cb00730f0 0x7f1cb019c1c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400c8a0 tx=0x7f1ca400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:45:03.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.284+0000 7f1cad600700 1 -- 172.21.15.119:0/1682777275 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ca400cea0 con 0x7f1cb00730f0 2024-05-27T09:45:03.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.284+0000 7f1cad600700 1 -- 172.21.15.119:0/1682777275 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ca4004830 con 0x7f1cb00730f0 2024-05-27T09:45:03.285 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.284+0000 7f1cb5e00700 1 -- 172.21.15.119:0/1682777275 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb01a0e50 con 0x7f1cb00730f0 2024-05-27T09:45:03.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.284+0000 7f1cad600700 1 -- 172.21.15.119:0/1682777275 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ca40055b0 con 0x7f1cb00730f0 2024-05-27T09:45:03.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.284+0000 7f1cb5e00700 1 -- 172.21.15.119:0/1682777275 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb01a13a0 con 0x7f1cb00730f0 2024-05-27T09:45:03.286 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.286+0000 7f1cb5e00700 1 -- 172.21.15.119:0/1682777275 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb010e280 con 0x7f1cb00730f0 2024-05-27T09:45:03.287 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.287+0000 7f1cad600700 1 -- 172.21.15.119:0/1682777275 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f1ca40049a0 con 0x7f1cb00730f0 2024-05-27T09:45:03.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.288+0000 7f1cad600700 1 --2- 172.21.15.119:0/1682777275 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1c98078a40 0x7f1c9807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:03.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.288+0000 7f1cad600700 1 -- 172.21.15.119:0/1682777275 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1ca409aac0 con 0x7f1cb00730f0 2024-05-27T09:45:03.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.288+0000 7f1cafe00700 1 --2- 172.21.15.119:0/1682777275 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1c98078a40 0x7f1c9807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:03.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.289+0000 7f1cafe00700 1 --2- 172.21.15.119:0/1682777275 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1c98078a40 0x7f1c9807aef0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c005100 tx=0x7f1c9c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:45:03.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.294+0000 7f1cad600700 1 -- 172.21.15.119:0/1682777275 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f1ca40625f0 con 0x7f1cb00730f0 2024-05-27T09:45:03.579 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.579+0000 7f1cb5e00700 1 -- 172.21.15.119:0/1682777275 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f1cb004f2a0 con 0x7f1cb00730f0 2024-05-27T09:45:03.580 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.580+0000 7f1cad600700 1 -- 172.21.15.119:0/1682777275 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v39) v1 ==== 107+0+5135 (secure 0 0 0) 0x7f1ca4004c50 con 0x7f1cb00730f0 2024-05-27T09:45:03.581 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:45:03.581 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi119.nghcey","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6827/3357235708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3357235708},{"type":"v1","addr":"172.21.15.119:6827","nonce":3357235708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":34468,"name":"cephfs.smithi119.ppmzzk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6829/2430623742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2430623742},{"type":"v1","addr":"172.21.15.119:6829","nonce":2430623742}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44289,"name":"cephfs.smithi138.ndfkhd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/3387966485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3387966485},{"type":"v1","addr":"172.21.15.138:6825","nonce":3387966485}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44303,"name":"cephfs.smithi138.alszpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/3198147602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":3198147602},{"type":"v1","addr":"172.21.15.138:6827","nonce":3198147602}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:25:53.387388+0000","modified":"2024-05-27T09:39:57.913154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-27T09:45:03.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.583+0000 7f1cb5e00700 1 -- 172.21.15.119:0/1682777275 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1c98078a40 msgr2=0x7f1c9807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:03.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.583+0000 7f1cb5e00700 1 --2- 172.21.15.119:0/1682777275 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1c98078a40 0x7f1c9807aef0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c005100 tx=0x7f1c9c01a040 comp rx=0 tx=0).stop 2024-05-27T09:45:03.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.583+0000 7f1cb5e00700 1 -- 172.21.15.119:0/1682777275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cb00730f0 msgr2=0x7f1cb019c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:03.584 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.583+0000 7f1cb5e00700 1 --2- 172.21.15.119:0/1682777275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cb00730f0 0x7f1cb019c1c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400c8a0 tx=0x7f1ca400cc60 comp rx=0 tx=0).stop 2024-05-27T09:45:03.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.584+0000 7f1cb5e00700 1 -- 172.21.15.119:0/1682777275 shutdown_connections 2024-05-27T09:45:03.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.584+0000 7f1cb5e00700 1 --2- 172.21.15.119:0/1682777275 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f1c98078a40 0x7f1c9807aef0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:03.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.584+0000 7f1cb5e00700 1 --2- 172.21.15.119:0/1682777275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cb00730f0 0x7f1cb019c1c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:03.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.584+0000 7f1cb5e00700 1 --2- 172.21.15.119:0/1682777275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cb0073a00 0x7f1cb019c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:03.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.584+0000 7f1cb5e00700 1 -- 172.21.15.119:0/1682777275 >> 172.21.15.119:0/1682777275 conn(0x7f1cb00fbb90 msgr2=0x7f1cb0102650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:45:03.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.584+0000 7f1cb5e00700 1 -- 172.21.15.119:0/1682777275 shutdown_connections 2024-05-27T09:45:03.585 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:03.584+0000 7f1cb5e00700 1 -- 172.21.15.119:0/1682777275 wait complete. 2024-05-27T09:45:03.586 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 32 2024-05-27T09:45:03.962 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:03 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/1682777275' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-05-27T09:45:04.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:03 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/1682777275' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-05-27T09:45:04.357 DEBUG:tasks.fs:max_mds not reduced in epoch 32 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-05-27T09:45:04.357 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 33 2024-05-27T09:45:04.762 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:45:05.083 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:04 smithi119 ceph-mon[88269]: pgmap v357: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:05.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:04 smithi138 ceph-mon[70004]: pgmap v357: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:06.993 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:06 smithi119 ceph-mon[88269]: pgmap v358: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:07.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.039+0000 7f24d5200700 1 -- 172.21.15.119:0/3604684093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24d0106b80 msgr2=0x7f24d0106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:07.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.039+0000 7f24d5200700 1 --2- 172.21.15.119:0/3604684093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24d0106b80 0x7f24d0106ff0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f24bc009910 tx=0x7f24bc009c20 comp rx=0 tx=0).stop 2024-05-27T09:45:07.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.039+0000 7f24d5200700 1 -- 172.21.15.119:0/3604684093 shutdown_connections 2024-05-27T09:45:07.040 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.039+0000 7f24d5200700 1 --2- 172.21.15.119:0/3604684093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24d0106b80 0x7f24d0106ff0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:07.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.039+0000 7f24d5200700 1 --2- 172.21.15.119:0/3604684093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24d010b610 0x7f24d010b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:07.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.039+0000 7f24d5200700 1 -- 172.21.15.119:0/3604684093 >> 172.21.15.119:0/3604684093 conn(0x7f24d0075df0 msgr2=0x7f24d0078200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:45:07.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.039+0000 7f24d5200700 1 -- 172.21.15.119:0/3604684093 shutdown_connections 2024-05-27T09:45:07.041 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.039+0000 7f24d5200700 1 -- 172.21.15.119:0/3604684093 wait complete. 2024-05-27T09:45:07.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.040+0000 7f24d5200700 1 Processor -- start 2024-05-27T09:45:07.042 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.040+0000 7f24d5200700 1 -- start start 2024-05-27T09:45:07.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.040+0000 7f24d5200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24d0106b80 0x7f24d019c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:07.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.040+0000 7f24d5200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24d010b610 0x7f24d019c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:07.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.040+0000 7f24d5200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d019ce80 con 0x7f24d0106b80 2024-05-27T09:45:07.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.040+0000 7f24d5200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d01a0c10 con 0x7f24d010b610 2024-05-27T09:45:07.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.041+0000 7f24cf400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24d0106b80 0x7f24d019c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:07.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.041+0000 7f24cf400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24d0106b80 0x7f24d019c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60986/0 (socket says 172.21.15.119:60986) 2024-05-27T09:45:07.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.041+0000 7f24cf400700 1 -- 172.21.15.119:0/2605625371 learned_addr learned my addr 172.21.15.119:0/2605625371 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:45:07.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.041+0000 7f24cf400700 1 -- 172.21.15.119:0/2605625371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24d010b610 msgr2=0x7f24d019c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:07.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.041+0000 7f24cf400700 1 --2- 172.21.15.119:0/2605625371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24d010b610 0x7f24d019c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:07.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.041+0000 7f24cf400700 1 -- 172.21.15.119:0/2605625371 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24bc0095f0 con 0x7f24d0106b80 2024-05-27T09:45:07.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.041+0000 7f24cf400700 1 --2- 172.21.15.119:0/2605625371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24d0106b80 0x7f24d019c260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f24c000b4b0 tx=0x7f24c000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:45:07.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.042+0000 7f24c7e00700 1 -- 172.21.15.119:0/2605625371 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24c0011800 con 0x7f24d0106b80 2024-05-27T09:45:07.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.042+0000 7f24c7e00700 1 -- 172.21.15.119:0/2605625371 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f24c0011e20 con 0x7f24d0106b80 2024-05-27T09:45:07.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.042+0000 7f24c7e00700 1 -- 172.21.15.119:0/2605625371 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24c000f2e0 con 0x7f24d0106b80 2024-05-27T09:45:07.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.042+0000 7f24d5200700 1 -- 172.21.15.119:0/2605625371 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d01a0ef0 con 0x7f24d0106b80 2024-05-27T09:45:07.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.042+0000 7f24d5200700 1 -- 172.21.15.119:0/2605625371 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d01a14c0 con 0x7f24d0106b80 2024-05-27T09:45:07.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.043+0000 7f24d5200700 1 -- 172.21.15.119:0/2605625371 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d010ea00 con 0x7f24d0106b80 2024-05-27T09:45:07.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.044+0000 7f24c7e00700 1 -- 172.21.15.119:0/2605625371 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f24c0010380 con 0x7f24d0106b80 2024-05-27T09:45:07.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.045+0000 7f24c7e00700 1 --2- 172.21.15.119:0/2605625371 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f24b4078a40 0x7f24b407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:07.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.045+0000 7f24c7e00700 1 -- 172.21.15.119:0/2605625371 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f24c009a8f0 con 0x7f24d0106b80 2024-05-27T09:45:07.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.045+0000 7f24cea00700 1 --2- 172.21.15.119:0/2605625371 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f24b4078a40 0x7f24b407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:07.047 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.046+0000 7f24cea00700 1 --2- 172.21.15.119:0/2605625371 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f24b4078a40 0x7f24b407aef0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f24bc00b5c0 tx=0x7f24bc005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:45:07.051 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.050+0000 7f24c7e00700 1 -- 172.21.15.119:0/2605625371 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f24c00624a0 con 0x7f24d0106b80 2024-05-27T09:45:07.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:06 smithi138 ceph-mon[70004]: pgmap v358: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:07.333 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.332+0000 7f24d5200700 1 -- 172.21.15.119:0/2605625371 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f24d01a1830 con 0x7f24d0106b80 2024-05-27T09:45:07.335 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.334+0000 7f24c7e00700 1 -- 172.21.15.119:0/2605625371 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v39) v1 ==== 107+0+5225 (secure 0 0 0) 0x7f24c0061e40 con 0x7f24d0106b80 2024-05-27T09:45:07.335 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:45:07.335 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44289,"name":"cephfs.smithi138.ndfkhd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/3387966485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3387966485},{"type":"v1","addr":"172.21.15.138:6825","nonce":3387966485}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44303,"name":"cephfs.smithi138.alszpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/3198147602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":3198147602},{"type":"v1","addr":"172.21.15.138:6827","nonce":3198147602}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:25:53.387388+0000","modified":"2024-05-27T09:41:37.257563+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34436,"mds_1":34468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34436":{"gid":34436,"name":"cephfs.smithi119.nghcey","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.119:6827/3357235708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3357235708},{"type":"v1","addr":"172.21.15.119:6827","nonce":3357235708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34468":{"gid":34468,"name":"cephfs.smithi119.ppmzzk","rank":1,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.119:6829/2430623742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2430623742},{"type":"v1","addr":"172.21.15.119:6829","nonce":2430623742}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-27T09:45:07.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.337+0000 7f24d5200700 1 -- 172.21.15.119:0/2605625371 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f24b4078a40 msgr2=0x7f24b407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:07.337 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.337+0000 7f24d5200700 1 --2- 172.21.15.119:0/2605625371 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f24b4078a40 0x7f24b407aef0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f24bc00b5c0 tx=0x7f24bc005fd0 comp rx=0 tx=0).stop 2024-05-27T09:45:07.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.337+0000 7f24d5200700 1 -- 172.21.15.119:0/2605625371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24d0106b80 msgr2=0x7f24d019c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:07.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.337+0000 7f24d5200700 1 --2- 172.21.15.119:0/2605625371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24d0106b80 0x7f24d019c260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f24c000b4b0 tx=0x7f24c000b870 comp rx=0 tx=0).stop 2024-05-27T09:45:07.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.337+0000 7f24d5200700 1 -- 172.21.15.119:0/2605625371 shutdown_connections 2024-05-27T09:45:07.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.337+0000 7f24d5200700 1 --2- 172.21.15.119:0/2605625371 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f24b4078a40 0x7f24b407aef0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:07.338 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.337+0000 7f24d5200700 1 --2- 172.21.15.119:0/2605625371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24d0106b80 0x7f24d019c260 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:07.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.337+0000 7f24d5200700 1 --2- 172.21.15.119:0/2605625371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24d010b610 0x7f24d019c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:07.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.337+0000 7f24d5200700 1 -- 172.21.15.119:0/2605625371 >> 172.21.15.119:0/2605625371 conn(0x7f24d0075df0 msgr2=0x7f24d00777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:45:07.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.337+0000 7f24d5200700 1 -- 172.21.15.119:0/2605625371 shutdown_connections 2024-05-27T09:45:07.339 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:07.338+0000 7f24d5200700 1 -- 172.21.15.119:0/2605625371 wait complete. 2024-05-27T09:45:07.339 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 33 2024-05-27T09:45:08.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:07 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/2605625371' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-05-27T09:45:08.139 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:07 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/2605625371' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-05-27T09:45:08.188 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 34 2024-05-27T09:45:08.588 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:45:08.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:08 smithi119 ceph-mon[88269]: pgmap v359: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:09.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:08 smithi138 ceph-mon[70004]: pgmap v359: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:10.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.778+0000 7f8fbf200700 1 -- 172.21.15.119:0/1265645263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc0101aa0 msgr2=0x7f8fc0109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:10.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.778+0000 7f8fbf200700 1 --2- 172.21.15.119:0/1265645263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc0101aa0 0x7f8fc0109fa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8fac009910 tx=0x7f8fac009c20 comp rx=0 tx=0).stop 2024-05-27T09:45:10.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.778+0000 7f8fbf200700 1 -- 172.21.15.119:0/1265645263 shutdown_connections 2024-05-27T09:45:10.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.778+0000 7f8fbf200700 1 --2- 172.21.15.119:0/1265645263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc0101aa0 0x7f8fc0109fa0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:10.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.778+0000 7f8fbf200700 1 --2- 172.21.15.119:0/1265645263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fc0101190 0x7f8fc0101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:10.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.778+0000 7f8fbf200700 1 -- 172.21.15.119:0/1265645263 >> 172.21.15.119:0/1265645263 conn(0x7f8fc00fa9d0 msgr2=0x7f8fc00fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:45:10.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.779+0000 7f8fbf200700 1 -- 172.21.15.119:0/1265645263 shutdown_connections 2024-05-27T09:45:10.780 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.779+0000 7f8fbf200700 1 -- 172.21.15.119:0/1265645263 wait complete. 2024-05-27T09:45:10.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.780+0000 7f8fbf200700 1 Processor -- start 2024-05-27T09:45:10.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.780+0000 7f8fbf200700 1 -- start start 2024-05-27T09:45:10.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.780+0000 7f8fbf200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc0101190 0x7f8fc019b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:10.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.780+0000 7f8fbf200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fc0101aa0 0x7f8fc019be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:10.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.780+0000 7f8fbf200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc019c4c0 con 0x7f8fc0101190 2024-05-27T09:45:10.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.780+0000 7f8fbf200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc0195970 con 0x7f8fc0101aa0 2024-05-27T09:45:10.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.781+0000 7f8fbde00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc0101190 0x7f8fc019b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:10.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.781+0000 7f8fbde00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc0101190 0x7f8fc019b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36580/0 (socket says 172.21.15.119:36580) 2024-05-27T09:45:10.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.781+0000 7f8fbde00700 1 -- 172.21.15.119:0/429712842 learned_addr learned my addr 172.21.15.119:0/429712842 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:45:10.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.781+0000 7f8fbd400700 1 --2- 172.21.15.119:0/429712842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fc0101aa0 0x7f8fc019be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:10.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.781+0000 7f8fbde00700 1 -- 172.21.15.119:0/429712842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fc0101aa0 msgr2=0x7f8fc019be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:10.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.781+0000 7f8fbde00700 1 --2- 172.21.15.119:0/429712842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fc0101aa0 0x7f8fc019be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:10.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.781+0000 7f8fbde00700 1 -- 172.21.15.119:0/429712842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac0095f0 con 0x7f8fc0101190 2024-05-27T09:45:10.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.781+0000 7f8fbd400700 1 --2- 172.21.15.119:0/429712842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fc0101aa0 0x7f8fc019be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T09:45:10.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.782+0000 7f8fbde00700 1 --2- 172.21.15.119:0/429712842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc0101190 0x7f8fc019b8f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000d8d0 tx=0x7f8fb000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:45:10.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.782+0000 7f8fb6a00700 1 -- 172.21.15.119:0/429712842 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fb0009940 con 0x7f8fc0101190 2024-05-27T09:45:10.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.782+0000 7f8fbf200700 1 -- 172.21.15.119:0/429712842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc0195c50 con 0x7f8fc0101190 2024-05-27T09:45:10.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.782+0000 7f8fb6a00700 1 -- 172.21.15.119:0/429712842 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8fb0010460 con 0x7f8fc0101190 2024-05-27T09:45:10.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.782+0000 7f8fb6a00700 1 -- 172.21.15.119:0/429712842 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fb000f5f0 con 0x7f8fc0101190 2024-05-27T09:45:10.783 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.782+0000 7f8fbf200700 1 -- 172.21.15.119:0/429712842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc01961a0 con 0x7f8fc0101190 2024-05-27T09:45:10.784 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.783+0000 7f8fbf200700 1 -- 172.21.15.119:0/429712842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc0107610 con 0x7f8fc0101190 2024-05-27T09:45:10.785 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.784+0000 7f8fb6a00700 1 -- 172.21.15.119:0/429712842 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f8fb00105d0 con 0x7f8fc0101190 2024-05-27T09:45:10.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.785+0000 7f8fb6a00700 1 --2- 172.21.15.119:0/429712842 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8fa4078a40 0x7f8fa407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:10.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.785+0000 7f8fb6a00700 1 -- 172.21.15.119:0/429712842 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8fb009ae80 con 0x7f8fc0101190 2024-05-27T09:45:10.786 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.786+0000 7f8fbd400700 1 --2- 172.21.15.119:0/429712842 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8fa4078a40 0x7f8fa407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:10.787 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.786+0000 7f8fbd400700 1 --2- 172.21.15.119:0/429712842 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8fa4078a40 0x7f8fa407aef0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00b5c0 tx=0x7f8fac01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:45:10.791 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:10.791+0000 7f8fb6a00700 1 -- 172.21.15.119:0/429712842 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f8fb00629b0 con 0x7f8fc0101190 2024-05-27T09:45:10.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:10 smithi119 ceph-mon[88269]: pgmap v360: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:11.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:11.078+0000 7f8fbf200700 1 -- 172.21.15.119:0/429712842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f8fc01963c0 con 0x7f8fc0101190 2024-05-27T09:45:11.079 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:11.079+0000 7f8fb6a00700 1 -- 172.21.15.119:0/429712842 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v39) v1 ==== 107+0+5227 (secure 0 0 0) 0x7f8fb0062350 con 0x7f8fc0101190 2024-05-27T09:45:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:45:11.080 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44289,"name":"cephfs.smithi138.ndfkhd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/3387966485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3387966485},{"type":"v1","addr":"172.21.15.138:6825","nonce":3387966485}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44303,"name":"cephfs.smithi138.alszpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/3198147602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":3198147602},{"type":"v1","addr":"172.21.15.138:6827","nonce":3198147602}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:25:53.387388+0000","modified":"2024-05-27T09:41:45.602819+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34436,"mds_1":34468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34436":{"gid":34436,"name":"cephfs.smithi119.nghcey","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.119:6827/3357235708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3357235708},{"type":"v1","addr":"172.21.15.119:6827","nonce":3357235708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34468":{"gid":34468,"name":"cephfs.smithi119.ppmzzk","rank":1,"incarnation":33,"state":"up:resolve","state_seq":20,"addr":"172.21.15.119:6829/2430623742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2430623742},{"type":"v1","addr":"172.21.15.119:6829","nonce":2430623742}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-27T09:45:11.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:11.082+0000 7f8fbf200700 1 -- 172.21.15.119:0/429712842 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8fa4078a40 msgr2=0x7f8fa407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:11.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:11.082+0000 7f8fbf200700 1 --2- 172.21.15.119:0/429712842 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8fa4078a40 0x7f8fa407aef0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00b5c0 tx=0x7f8fac01a040 comp rx=0 tx=0).stop 2024-05-27T09:45:11.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:11.082+0000 7f8fbf200700 1 -- 172.21.15.119:0/429712842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc0101190 msgr2=0x7f8fc019b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:11.083 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:11.082+0000 7f8fbf200700 1 --2- 172.21.15.119:0/429712842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc0101190 0x7f8fc019b8f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000d8d0 tx=0x7f8fb000dc90 comp rx=0 tx=0).stop 2024-05-27T09:45:11.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:11.082+0000 7f8fbf200700 1 -- 172.21.15.119:0/429712842 shutdown_connections 2024-05-27T09:45:11.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:11.083+0000 7f8fbf200700 1 --2- 172.21.15.119:0/429712842 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f8fa4078a40 0x7f8fa407aef0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:11.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:11.083+0000 7f8fbf200700 1 --2- 172.21.15.119:0/429712842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc0101190 0x7f8fc019b8f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:11.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:11.083+0000 7f8fbf200700 1 --2- 172.21.15.119:0/429712842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fc0101aa0 0x7f8fc019be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:11.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:11.083+0000 7f8fbf200700 1 -- 172.21.15.119:0/429712842 >> 172.21.15.119:0/429712842 conn(0x7f8fc00fa9d0 msgr2=0x7f8fc0104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:45:11.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:11.083+0000 7f8fbf200700 1 -- 172.21.15.119:0/429712842 shutdown_connections 2024-05-27T09:45:11.084 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:11.083+0000 7f8fbf200700 1 -- 172.21.15.119:0/429712842 wait complete. 2024-05-27T09:45:11.085 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 34 2024-05-27T09:45:11.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:10 smithi138 ceph-mon[70004]: pgmap v360: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:11.847 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 35 2024-05-27T09:45:12.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:11 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/429712842' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-05-27T09:45:12.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:11 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/429712842' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-05-27T09:45:12.253 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:45:12.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:12 smithi119 ceph-mon[88269]: pgmap v361: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:12.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:12 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:45:13.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:12 smithi138 ceph-mon[70004]: pgmap v361: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:13.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:12 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:45:14.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.508+0000 7f955be00700 1 -- 172.21.15.119:0/2172585139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95541026a0 msgr2=0x7f9554102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:14.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.508+0000 7f955be00700 1 --2- 172.21.15.119:0/2172585139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95541026a0 0x7f9554102a70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9548009910 tx=0x7f9548009c20 comp rx=0 tx=0).stop 2024-05-27T09:45:14.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.508+0000 7f955be00700 1 -- 172.21.15.119:0/2172585139 shutdown_connections 2024-05-27T09:45:14.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.508+0000 7f955be00700 1 --2- 172.21.15.119:0/2172585139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9554103040 0x7f9554107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:14.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.508+0000 7f955be00700 1 --2- 172.21.15.119:0/2172585139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95541026a0 0x7f9554102a70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:14.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.508+0000 7f955be00700 1 -- 172.21.15.119:0/2172585139 >> 172.21.15.119:0/2172585139 conn(0x7f95540fdf00 msgr2=0x7f9554100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:45:14.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.508+0000 7f955be00700 1 -- 172.21.15.119:0/2172585139 shutdown_connections 2024-05-27T09:45:14.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.508+0000 7f955be00700 1 -- 172.21.15.119:0/2172585139 wait complete. 2024-05-27T09:45:14.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.510+0000 7f955be00700 1 Processor -- start 2024-05-27T09:45:14.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.510+0000 7f955be00700 1 -- start start 2024-05-27T09:45:14.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.511+0000 7f955be00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95541026a0 0x7f9554198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:14.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.511+0000 7f955be00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9554103040 0x7f95541985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:14.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.511+0000 7f955be00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9554198ca0 con 0x7f9554103040 2024-05-27T09:45:14.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.511+0000 7f955be00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955419ca30 con 0x7f95541026a0 2024-05-27T09:45:14.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.511+0000 7f9558a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9554103040 0x7f95541985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:14.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.511+0000 7f9558a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9554103040 0x7f95541985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36594/0 (socket says 172.21.15.119:36594) 2024-05-27T09:45:14.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.511+0000 7f9558a00700 1 -- 172.21.15.119:0/942799882 learned_addr learned my addr 172.21.15.119:0/942799882 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:45:14.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.511+0000 7f9559400700 1 --2- 172.21.15.119:0/942799882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95541026a0 0x7f9554198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:14.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.512+0000 7f9558a00700 1 -- 172.21.15.119:0/942799882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95541026a0 msgr2=0x7f9554198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:14.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.512+0000 7f9558a00700 1 --2- 172.21.15.119:0/942799882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95541026a0 0x7f9554198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:14.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.512+0000 7f9558a00700 1 -- 172.21.15.119:0/942799882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95480095f0 con 0x7f9554103040 2024-05-27T09:45:14.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.512+0000 7f9558a00700 1 --2- 172.21.15.119:0/942799882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9554103040 0x7f95541985c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f954000b480 tx=0x7f954000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:45:14.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.512+0000 7f9552000700 1 -- 172.21.15.119:0/942799882 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9540011800 con 0x7f9554103040 2024-05-27T09:45:14.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.512+0000 7f9552000700 1 -- 172.21.15.119:0/942799882 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9540011e20 con 0x7f9554103040 2024-05-27T09:45:14.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.512+0000 7f9552000700 1 -- 172.21.15.119:0/942799882 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9540010ea0 con 0x7f9554103040 2024-05-27T09:45:14.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.512+0000 7f955be00700 1 -- 172.21.15.119:0/942799882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f955419cd10 con 0x7f9554103040 2024-05-27T09:45:14.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.513+0000 7f955be00700 1 -- 172.21.15.119:0/942799882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955419d210 con 0x7f9554103040 2024-05-27T09:45:14.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.514+0000 7f955be00700 1 -- 172.21.15.119:0/942799882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f955410a9b0 con 0x7f9554103040 2024-05-27T09:45:14.516 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.515+0000 7f9552000700 1 -- 172.21.15.119:0/942799882 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f9540010380 con 0x7f9554103040 2024-05-27T09:45:14.517 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.517+0000 7f9552000700 1 --2- 172.21.15.119:0/942799882 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f953c078a40 0x7f953c07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:14.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.517+0000 7f9552000700 1 -- 172.21.15.119:0/942799882 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f954009a4c0 con 0x7f9554103040 2024-05-27T09:45:14.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.517+0000 7f9559400700 1 --2- 172.21.15.119:0/942799882 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f953c078a40 0x7f953c07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:14.518 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.518+0000 7f9559400700 1 --2- 172.21.15.119:0/942799882 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f953c078a40 0x7f953c07aef0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9548006010 tx=0x7f954801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:45:14.523 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.523+0000 7f9552000700 1 -- 172.21.15.119:0/942799882 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f9540062070 con 0x7f9554103040 2024-05-27T09:45:14.810 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.809+0000 7f955be00700 1 -- 172.21.15.119:0/942799882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f955404ea50 con 0x7f9554103040 2024-05-27T09:45:14.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.812+0000 7f9552000700 1 -- 172.21.15.119:0/942799882 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v39) v1 ==== 107+0+5229 (secure 0 0 0) 0x7f9540061a10 con 0x7f9554103040 2024-05-27T09:45:14.813 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:45:14.813 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44289,"name":"cephfs.smithi138.ndfkhd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/3387966485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3387966485},{"type":"v1","addr":"172.21.15.138:6825","nonce":3387966485}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44303,"name":"cephfs.smithi138.alszpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/3198147602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":3198147602},{"type":"v1","addr":"172.21.15.138:6827","nonce":3198147602}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:25:53.387388+0000","modified":"2024-05-27T09:41:46.738883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34436,"mds_1":34468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34436":{"gid":34436,"name":"cephfs.smithi119.nghcey","rank":0,"incarnation":33,"state":"up:resolve","state_seq":26,"addr":"172.21.15.119:6827/3357235708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3357235708},{"type":"v1","addr":"172.21.15.119:6827","nonce":3357235708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34468":{"gid":34468,"name":"cephfs.smithi119.ppmzzk","rank":1,"incarnation":33,"state":"up:resolve","state_seq":20,"addr":"172.21.15.119:6829/2430623742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2430623742},{"type":"v1","addr":"172.21.15.119:6829","nonce":2430623742}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-27T09:45:14.815 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.815+0000 7f955be00700 1 -- 172.21.15.119:0/942799882 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f953c078a40 msgr2=0x7f953c07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:14.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.815+0000 7f955be00700 1 --2- 172.21.15.119:0/942799882 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f953c078a40 0x7f953c07aef0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9548006010 tx=0x7f954801a040 comp rx=0 tx=0).stop 2024-05-27T09:45:14.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.815+0000 7f955be00700 1 -- 172.21.15.119:0/942799882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9554103040 msgr2=0x7f95541985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:14.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.815+0000 7f955be00700 1 --2- 172.21.15.119:0/942799882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9554103040 0x7f95541985c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f954000b480 tx=0x7f954000b790 comp rx=0 tx=0).stop 2024-05-27T09:45:14.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.815+0000 7f955be00700 1 -- 172.21.15.119:0/942799882 shutdown_connections 2024-05-27T09:45:14.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.815+0000 7f955be00700 1 --2- 172.21.15.119:0/942799882 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f953c078a40 0x7f953c07aef0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:14.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.815+0000 7f955be00700 1 --2- 172.21.15.119:0/942799882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95541026a0 0x7f9554198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:14.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.815+0000 7f955be00700 1 --2- 172.21.15.119:0/942799882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9554103040 0x7f95541985c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:14.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.815+0000 7f955be00700 1 -- 172.21.15.119:0/942799882 >> 172.21.15.119:0/942799882 conn(0x7f95540fdf00 msgr2=0x7f95540ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:45:14.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.815+0000 7f955be00700 1 -- 172.21.15.119:0/942799882 shutdown_connections 2024-05-27T09:45:14.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:14.816+0000 7f955be00700 1 -- 172.21.15.119:0/942799882 wait complete. 2024-05-27T09:45:14.818 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 35 2024-05-27T09:45:15.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:14 smithi138 ceph-mon[70004]: pgmap v362: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:15.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:14 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/942799882' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-05-27T09:45:15.173 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:14 smithi119 ceph-mon[88269]: pgmap v362: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:15.174 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:14 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/942799882' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-05-27T09:45:15.605 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 36 2024-05-27T09:45:16.020 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:45:17.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:16 smithi138 ceph-mon[70004]: pgmap v363: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:17.157 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:16 smithi119 ceph-mon[88269]: pgmap v363: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:18.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.230+0000 7f32c3400700 1 -- 172.21.15.119:0/3480244739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32bc102630 msgr2=0x7f32bc102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:18.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.230+0000 7f32c3400700 1 --2- 172.21.15.119:0/3480244739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32bc102630 0x7f32bc102a00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009910 tx=0x7f32b0009c20 comp rx=0 tx=0).stop 2024-05-27T09:45:18.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.230+0000 7f32c3400700 1 -- 172.21.15.119:0/3480244739 shutdown_connections 2024-05-27T09:45:18.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.230+0000 7f32c3400700 1 --2- 172.21.15.119:0/3480244739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32bc102fd0 0x7f32bc106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:18.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.230+0000 7f32c3400700 1 --2- 172.21.15.119:0/3480244739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32bc102630 0x7f32bc102a00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:18.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.230+0000 7f32c3400700 1 -- 172.21.15.119:0/3480244739 >> 172.21.15.119:0/3480244739 conn(0x7f32bc0fdef0 msgr2=0x7f32bc100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:45:18.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.230+0000 7f32c3400700 1 -- 172.21.15.119:0/3480244739 shutdown_connections 2024-05-27T09:45:18.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.230+0000 7f32c3400700 1 -- 172.21.15.119:0/3480244739 wait complete. 2024-05-27T09:45:18.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.231+0000 7f32c3400700 1 Processor -- start 2024-05-27T09:45:18.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.231+0000 7f32c3400700 1 -- start start 2024-05-27T09:45:18.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.231+0000 7f32c3400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32bc102630 0x7f32bc198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:18.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.231+0000 7f32c3400700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32bc102fd0 0x7f32bc198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:18.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.231+0000 7f32c3400700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc198c40 con 0x7f32bc102fd0 2024-05-27T09:45:18.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.231+0000 7f32c3400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc19c9d0 con 0x7f32bc102630 2024-05-27T09:45:18.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.232+0000 7f32bbe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32bc102fd0 0x7f32bc198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:18.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.232+0000 7f32bbe00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32bc102fd0 0x7f32bc198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36608/0 (socket says 172.21.15.119:36608) 2024-05-27T09:45:18.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.232+0000 7f32bbe00700 1 -- 172.21.15.119:0/3838827886 learned_addr learned my addr 172.21.15.119:0/3838827886 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:45:18.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.232+0000 7f32c0a00700 1 --2- 172.21.15.119:0/3838827886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32bc102630 0x7f32bc198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:18.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.232+0000 7f32bbe00700 1 -- 172.21.15.119:0/3838827886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32bc102630 msgr2=0x7f32bc198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:18.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.232+0000 7f32bbe00700 1 --2- 172.21.15.119:0/3838827886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32bc102630 0x7f32bc198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:18.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.232+0000 7f32bbe00700 1 -- 172.21.15.119:0/3838827886 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b00095f0 con 0x7f32bc102fd0 2024-05-27T09:45:18.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.233+0000 7f32bbe00700 1 --2- 172.21.15.119:0/3838827886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32bc102fd0 0x7f32bc198560 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f32a800d900 tx=0x7f32a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:45:18.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.233+0000 7f32b9600700 1 -- 172.21.15.119:0/3838827886 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32a8009900 con 0x7f32bc102fd0 2024-05-27T09:45:18.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.233+0000 7f32c3400700 1 -- 172.21.15.119:0/3838827886 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32bc19ccb0 con 0x7f32bc102fd0 2024-05-27T09:45:18.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.233+0000 7f32b9600700 1 -- 172.21.15.119:0/3838827886 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32a8010460 con 0x7f32bc102fd0 2024-05-27T09:45:18.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.233+0000 7f32b9600700 1 -- 172.21.15.119:0/3838827886 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32a800f5f0 con 0x7f32bc102fd0 2024-05-27T09:45:18.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.233+0000 7f32c3400700 1 -- 172.21.15.119:0/3838827886 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32bc19d200 con 0x7f32bc102fd0 2024-05-27T09:45:18.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.234+0000 7f32c3400700 1 -- 172.21.15.119:0/3838827886 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32bc10a940 con 0x7f32bc102fd0 2024-05-27T09:45:18.236 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.236+0000 7f32b9600700 1 -- 172.21.15.119:0/3838827886 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f32a80105d0 con 0x7f32bc102fd0 2024-05-27T09:45:18.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.237+0000 7f32b9600700 1 --2- 172.21.15.119:0/3838827886 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f32a4078a40 0x7f32a407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:18.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.237+0000 7f32b9600700 1 -- 172.21.15.119:0/3838827886 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f32a809ac70 con 0x7f32bc102fd0 2024-05-27T09:45:18.238 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.237+0000 7f32c0a00700 1 --2- 172.21.15.119:0/3838827886 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f32a4078a40 0x7f32a407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:18.239 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.238+0000 7f32c0a00700 1 --2- 172.21.15.119:0/3838827886 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f32a4078a40 0x7f32a407aef0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f32b0006010 tx=0x7f32b001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:45:18.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.243+0000 7f32b9600700 1 -- 172.21.15.119:0/3838827886 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f32a80a1080 con 0x7f32bc102fd0 2024-05-27T09:45:18.529 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.528+0000 7f32c3400700 1 -- 172.21.15.119:0/3838827886 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f32bc04ea50 con 0x7f32bc102fd0 2024-05-27T09:45:18.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.530+0000 7f32b9600700 1 -- 172.21.15.119:0/3838827886 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v39) v1 ==== 107+0+5233 (secure 0 0 0) 0x7f32a80627a0 con 0x7f32bc102fd0 2024-05-27T09:45:18.531 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:45:18.531 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44289,"name":"cephfs.smithi138.ndfkhd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/3387966485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3387966485},{"type":"v1","addr":"172.21.15.138:6825","nonce":3387966485}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44303,"name":"cephfs.smithi138.alszpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/3198147602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":3198147602},{"type":"v1","addr":"172.21.15.138:6827","nonce":3198147602}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:25:53.387388+0000","modified":"2024-05-27T09:41:47.034476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34436,"mds_1":34468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34436":{"gid":34436,"name":"cephfs.smithi119.nghcey","rank":0,"incarnation":33,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.119:6827/3357235708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3357235708},{"type":"v1","addr":"172.21.15.119:6827","nonce":3357235708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34468":{"gid":34468,"name":"cephfs.smithi119.ppmzzk","rank":1,"incarnation":33,"state":"up:reconnect","state_seq":21,"addr":"172.21.15.119:6829/2430623742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2430623742},{"type":"v1","addr":"172.21.15.119:6829","nonce":2430623742}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-27T09:45:18.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.533+0000 7f32c3400700 1 -- 172.21.15.119:0/3838827886 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f32a4078a40 msgr2=0x7f32a407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:18.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.533+0000 7f32c3400700 1 --2- 172.21.15.119:0/3838827886 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f32a4078a40 0x7f32a407aef0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f32b0006010 tx=0x7f32b001a040 comp rx=0 tx=0).stop 2024-05-27T09:45:18.534 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.533+0000 7f32c3400700 1 -- 172.21.15.119:0/3838827886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32bc102fd0 msgr2=0x7f32bc198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:18.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.533+0000 7f32c3400700 1 --2- 172.21.15.119:0/3838827886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32bc102fd0 0x7f32bc198560 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f32a800d900 tx=0x7f32a800dcc0 comp rx=0 tx=0).stop 2024-05-27T09:45:18.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.533+0000 7f32c3400700 1 -- 172.21.15.119:0/3838827886 shutdown_connections 2024-05-27T09:45:18.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.533+0000 7f32c3400700 1 --2- 172.21.15.119:0/3838827886 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f32a4078a40 0x7f32a407aef0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:18.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.533+0000 7f32c3400700 1 --2- 172.21.15.119:0/3838827886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32bc102630 0x7f32bc198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:18.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.533+0000 7f32c3400700 1 --2- 172.21.15.119:0/3838827886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32bc102fd0 0x7f32bc198560 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:18.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.533+0000 7f32c3400700 1 -- 172.21.15.119:0/3838827886 >> 172.21.15.119:0/3838827886 conn(0x7f32bc0fdef0 msgr2=0x7f32bc1002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:45:18.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.533+0000 7f32c3400700 1 -- 172.21.15.119:0/3838827886 shutdown_connections 2024-05-27T09:45:18.535 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:18.534+0000 7f32c3400700 1 -- 172.21.15.119:0/3838827886 wait complete. 2024-05-27T09:45:18.536 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 36 2024-05-27T09:45:19.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:18 smithi138 ceph-mon[70004]: pgmap v364: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:19.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:18 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/3838827886' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-05-27T09:45:19.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:18 smithi119 ceph-mon[88269]: pgmap v364: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:19.248 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:18 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/3838827886' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-05-27T09:45:19.359 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 37 2024-05-27T09:45:19.758 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:45:20.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:20 smithi119 ceph-mon[88269]: pgmap v365: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:20.997 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:20 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:45:21.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:20 smithi138 ceph-mon[70004]: pgmap v365: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:21.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:20 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T09:45:21.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.883+0000 7f54eb200700 1 -- 172.21.15.119:0/3122025134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54ec10b610 msgr2=0x7f54ec10b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:21.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.883+0000 7f54eb200700 1 --2- 172.21.15.119:0/3122025134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54ec10b610 0x7f54ec10b9e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009910 tx=0x7f54dc009c20 comp rx=0 tx=0).stop 2024-05-27T09:45:21.885 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.883+0000 7f54eb200700 1 -- 172.21.15.119:0/3122025134 shutdown_connections 2024-05-27T09:45:21.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.883+0000 7f54eb200700 1 --2- 172.21.15.119:0/3122025134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54ec106b80 0x7f54ec106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:21.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.883+0000 7f54eb200700 1 --2- 172.21.15.119:0/3122025134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54ec10b610 0x7f54ec10b9e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:21.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.883+0000 7f54eb200700 1 -- 172.21.15.119:0/3122025134 >> 172.21.15.119:0/3122025134 conn(0x7f54ec075df0 msgr2=0x7f54ec078200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:45:21.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.883+0000 7f54eb200700 1 -- 172.21.15.119:0/3122025134 shutdown_connections 2024-05-27T09:45:21.886 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.883+0000 7f54eb200700 1 -- 172.21.15.119:0/3122025134 wait complete. 2024-05-27T09:45:21.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.884+0000 7f54eb200700 1 Processor -- start 2024-05-27T09:45:21.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.885+0000 7f54eb200700 1 -- start start 2024-05-27T09:45:21.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.885+0000 7f54eb200700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54ec106b80 0x7f54ec19c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:21.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.885+0000 7f54eb200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54ec10b610 0x7f54ec19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:21.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.885+0000 7f54eb200700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec19cd10 con 0x7f54ec106b80 2024-05-27T09:45:21.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.885+0000 7f54eb200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec1a0aa0 con 0x7f54ec10b610 2024-05-27T09:45:21.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.885+0000 7f54e9e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54ec106b80 0x7f54ec19c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:21.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.885+0000 7f54e9e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54ec106b80 0x7f54ec19c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52216/0 (socket says 172.21.15.119:52216) 2024-05-27T09:45:21.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.886+0000 7f54e9e00700 1 -- 172.21.15.119:0/4220671331 learned_addr learned my addr 172.21.15.119:0/4220671331 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:45:21.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.886+0000 7f54e9400700 1 --2- 172.21.15.119:0/4220671331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54ec10b610 0x7f54ec19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:21.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.886+0000 7f54e9e00700 1 -- 172.21.15.119:0/4220671331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54ec10b610 msgr2=0x7f54ec19c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:21.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.886+0000 7f54e9e00700 1 --2- 172.21.15.119:0/4220671331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54ec10b610 0x7f54ec19c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:21.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.886+0000 7f54e9e00700 1 -- 172.21.15.119:0/4220671331 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54dc0095f0 con 0x7f54ec106b80 2024-05-27T09:45:21.888 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.886+0000 7f54e9e00700 1 --2- 172.21.15.119:0/4220671331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54ec106b80 0x7f54ec19c0f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00f690 tx=0x7f54dc00f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:45:21.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.887+0000 7f54e2a00700 1 -- 172.21.15.119:0/4220671331 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54dc01d070 con 0x7f54ec106b80 2024-05-27T09:45:21.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.887+0000 7f54e2a00700 1 -- 172.21.15.119:0/4220671331 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f54dc0050d0 con 0x7f54ec106b80 2024-05-27T09:45:21.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.887+0000 7f54eb200700 1 -- 172.21.15.119:0/4220671331 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ec1a0d20 con 0x7f54ec106b80 2024-05-27T09:45:21.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.887+0000 7f54e2a00700 1 -- 172.21.15.119:0/4220671331 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54dc022470 con 0x7f54ec106b80 2024-05-27T09:45:21.889 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.887+0000 7f54eb200700 1 -- 172.21.15.119:0/4220671331 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ec1a1290 con 0x7f54ec106b80 2024-05-27T09:45:21.890 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.888+0000 7f54eb200700 1 -- 172.21.15.119:0/4220671331 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ec04f2a0 con 0x7f54ec106b80 2024-05-27T09:45:21.891 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.890+0000 7f54e2a00700 1 -- 172.21.15.119:0/4220671331 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f54dc003680 con 0x7f54ec106b80 2024-05-27T09:45:21.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.890+0000 7f54e2a00700 1 --2- 172.21.15.119:0/4220671331 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f54d0078a40 0x7f54d007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:21.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.891+0000 7f54e2a00700 1 -- 172.21.15.119:0/4220671331 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f54dc09c280 con 0x7f54ec106b80 2024-05-27T09:45:21.892 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.891+0000 7f54e9400700 1 --2- 172.21.15.119:0/4220671331 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f54d0078a40 0x7f54d007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:21.893 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.892+0000 7f54e9400700 1 --2- 172.21.15.119:0/4220671331 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f54d0078a40 0x7f54d007aef0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f54ec19d710 tx=0x7f54d8009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:45:21.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:21.896+0000 7f54e2a00700 1 -- 172.21.15.119:0/4220671331 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f54dc063ee0 con 0x7f54ec106b80 2024-05-27T09:45:21.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:21 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:45:21.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:21 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:45:21.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:21 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:45:22.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:21 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T09:45:22.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:21 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T09:45:22.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:21 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' 2024-05-27T09:45:22.187 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:22.185+0000 7f54eb200700 1 -- 172.21.15.119:0/4220671331 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7f54ec19d450 con 0x7f54ec106b80 2024-05-27T09:45:22.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:22.186+0000 7f54e2a00700 1 -- 172.21.15.119:0/4220671331 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v39) v1 ==== 107+0+5227 (secure 0 0 0) 0x7f54dc063880 con 0x7f54ec106b80 2024-05-27T09:45:22.188 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:45:22.188 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44289,"name":"cephfs.smithi138.ndfkhd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/3387966485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3387966485},{"type":"v1","addr":"172.21.15.138:6825","nonce":3387966485}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44303,"name":"cephfs.smithi138.alszpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/3198147602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":3198147602},{"type":"v1","addr":"172.21.15.138:6827","nonce":3198147602}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:25:53.387388+0000","modified":"2024-05-27T09:41:47.979147+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34436,"mds_1":34468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34436":{"gid":34436,"name":"cephfs.smithi119.nghcey","rank":0,"incarnation":33,"state":"up:rejoin","state_seq":28,"addr":"172.21.15.119:6827/3357235708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3357235708},{"type":"v1","addr":"172.21.15.119:6827","nonce":3357235708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34468":{"gid":34468,"name":"cephfs.smithi119.ppmzzk","rank":1,"incarnation":33,"state":"up:rejoin","state_seq":22,"addr":"172.21.15.119:6829/2430623742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2430623742},{"type":"v1","addr":"172.21.15.119:6829","nonce":2430623742}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-27T09:45:22.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:22.189+0000 7f54eb200700 1 -- 172.21.15.119:0/4220671331 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f54d0078a40 msgr2=0x7f54d007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:22.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:22.189+0000 7f54eb200700 1 --2- 172.21.15.119:0/4220671331 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f54d0078a40 0x7f54d007aef0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f54ec19d710 tx=0x7f54d8009260 comp rx=0 tx=0).stop 2024-05-27T09:45:22.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:22.189+0000 7f54eb200700 1 -- 172.21.15.119:0/4220671331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54ec106b80 msgr2=0x7f54ec19c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:22.191 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:22.189+0000 7f54eb200700 1 --2- 172.21.15.119:0/4220671331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54ec106b80 0x7f54ec19c0f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00f690 tx=0x7f54dc00f770 comp rx=0 tx=0).stop 2024-05-27T09:45:22.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:22.189+0000 7f54eb200700 1 -- 172.21.15.119:0/4220671331 shutdown_connections 2024-05-27T09:45:22.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:22.189+0000 7f54eb200700 1 --2- 172.21.15.119:0/4220671331 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f54d0078a40 0x7f54d007aef0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:22.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:22.190+0000 7f54eb200700 1 --2- 172.21.15.119:0/4220671331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54ec106b80 0x7f54ec19c0f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:22.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:22.190+0000 7f54eb200700 1 --2- 172.21.15.119:0/4220671331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54ec10b610 0x7f54ec19c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:22.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:22.190+0000 7f54eb200700 1 -- 172.21.15.119:0/4220671331 >> 172.21.15.119:0/4220671331 conn(0x7f54ec075df0 msgr2=0x7f54ec077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:45:22.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:22.190+0000 7f54eb200700 1 -- 172.21.15.119:0/4220671331 shutdown_connections 2024-05-27T09:45:22.192 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:22.190+0000 7f54eb200700 1 -- 172.21.15.119:0/4220671331 wait complete. 2024-05-27T09:45:22.193 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 37 2024-05-27T09:45:22.944 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:22 smithi119 ceph-mon[88269]: pgmap v366: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:22.944 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:22 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/4220671331' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-05-27T09:45:23.059 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 38 2024-05-27T09:45:23.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:22 smithi138 ceph-mon[70004]: pgmap v366: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:23.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:22 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/4220671331' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-05-27T09:45:23.476 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:45:25.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:24 smithi138 ceph-mon[70004]: pgmap v367: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:25.152 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:24 smithi119 ceph-mon[88269]: pgmap v367: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:25.825 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.823+0000 7f0e20a00700 1 -- 172.21.15.119:0/3781097364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c102630 msgr2=0x7f0e1c102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:25.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.823+0000 7f0e20a00700 1 --2- 172.21.15.119:0/3781097364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c102630 0x7f0e1c102a00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009910 tx=0x7f0e0c009c20 comp rx=0 tx=0).stop 2024-05-27T09:45:25.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.823+0000 7f0e20a00700 1 -- 172.21.15.119:0/3781097364 shutdown_connections 2024-05-27T09:45:25.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.823+0000 7f0e20a00700 1 --2- 172.21.15.119:0/3781097364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e1c102fd0 0x7f0e1c106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:25.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.823+0000 7f0e20a00700 1 --2- 172.21.15.119:0/3781097364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c102630 0x7f0e1c102a00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:25.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.823+0000 7f0e20a00700 1 -- 172.21.15.119:0/3781097364 >> 172.21.15.119:0/3781097364 conn(0x7f0e1c0fdef0 msgr2=0x7f0e1c100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:45:25.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.823+0000 7f0e20a00700 1 -- 172.21.15.119:0/3781097364 shutdown_connections 2024-05-27T09:45:25.826 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.823+0000 7f0e20a00700 1 -- 172.21.15.119:0/3781097364 wait complete. 2024-05-27T09:45:25.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.824+0000 7f0e20a00700 1 Processor -- start 2024-05-27T09:45:25.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.825+0000 7f0e20a00700 1 -- start start 2024-05-27T09:45:25.828 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.825+0000 7f0e20a00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c102630 0x7f0e1c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:25.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.825+0000 7f0e20a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e1c102fd0 0x7f0e1c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:25.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.825+0000 7f0e20a00700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c198bb0 con 0x7f0e1c102630 2024-05-27T09:45:25.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.825+0000 7f0e20a00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c198cf0 con 0x7f0e1c102fd0 2024-05-27T09:45:25.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.825+0000 7f0e19e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c102630 0x7f0e1c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:25.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.825+0000 7f0e19e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c102630 0x7f0e1c198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52244/0 (socket says 172.21.15.119:52244) 2024-05-27T09:45:25.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.825+0000 7f0e19e00700 1 -- 172.21.15.119:0/1999668607 learned_addr learned my addr 172.21.15.119:0/1999668607 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:45:25.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.826+0000 7f0e19e00700 1 -- 172.21.15.119:0/1999668607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e1c102fd0 msgr2=0x7f0e1c198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:25.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.826+0000 7f0e19400700 1 --2- 172.21.15.119:0/1999668607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e1c102fd0 0x7f0e1c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:25.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.826+0000 7f0e19e00700 1 --2- 172.21.15.119:0/1999668607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e1c102fd0 0x7f0e1c198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:25.829 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.826+0000 7f0e19e00700 1 -- 172.21.15.119:0/1999668607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c0095f0 con 0x7f0e1c102630 2024-05-27T09:45:25.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.826+0000 7f0e19e00700 1 --2- 172.21.15.119:0/1999668607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c102630 0x7f0e1c198020 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c006010 tx=0x7f0e0c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:45:25.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.826+0000 7f0e12a00700 1 -- 172.21.15.119:0/1999668607 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e0c01d070 con 0x7f0e1c102630 2024-05-27T09:45:25.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.827+0000 7f0e12a00700 1 -- 172.21.15.119:0/1999668607 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e0c004fb0 con 0x7f0e1c102630 2024-05-27T09:45:25.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.827+0000 7f0e20a00700 1 -- 172.21.15.119:0/1999668607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1c19cae0 con 0x7f0e1c102630 2024-05-27T09:45:25.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.827+0000 7f0e12a00700 1 -- 172.21.15.119:0/1999668607 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e0c00f690 con 0x7f0e1c102630 2024-05-27T09:45:25.830 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.827+0000 7f0e20a00700 1 -- 172.21.15.119:0/1999668607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1c19cfd0 con 0x7f0e1c102630 2024-05-27T09:45:25.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.828+0000 7f0e20a00700 1 -- 172.21.15.119:0/1999668607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e1c10a940 con 0x7f0e1c102630 2024-05-27T09:45:25.831 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.829+0000 7f0e12a00700 1 -- 172.21.15.119:0/1999668607 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f0e0c0228a0 con 0x7f0e1c102630 2024-05-27T09:45:25.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.831+0000 7f0e12a00700 1 --2- 172.21.15.119:0/1999668607 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0e00078a40 0x7f0e0007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:25.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.831+0000 7f0e12a00700 1 -- 172.21.15.119:0/1999668607 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0e0c09c5e0 con 0x7f0e1c102630 2024-05-27T09:45:25.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.831+0000 7f0e19400700 1 --2- 172.21.15.119:0/1999668607 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0e00078a40 0x7f0e0007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:25.833 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.832+0000 7f0e19400700 1 --2- 172.21.15.119:0/1999668607 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0e00078a40 0x7f0e0007aef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c199640 tx=0x7f0e08005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:45:25.838 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:25.837+0000 7f0e12a00700 1 -- 172.21.15.119:0/1999668607 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f0e0c064190 con 0x7f0e1c102630 2024-05-27T09:45:26.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:26.116+0000 7f0e20a00700 1 -- 172.21.15.119:0/1999668607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7f0e1c04ea50 con 0x7f0e1c102630 2024-05-27T09:45:26.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:26.118+0000 7f0e12a00700 1 -- 172.21.15.119:0/1999668607 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v39) v1 ==== 107+0+5242 (secure 0 0 0) 0x7f0e0c063b30 con 0x7f0e1c102630 2024-05-27T09:45:26.120 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-05-27T09:45:26.120 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44289,"name":"cephfs.smithi138.ndfkhd","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/3387966485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3387966485},{"type":"v1","addr":"172.21.15.138:6825","nonce":3387966485}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44303,"name":"cephfs.smithi138.alszpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/3198147602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":3198147602},{"type":"v1","addr":"172.21.15.138:6827","nonce":3198147602}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T09:25:53.387388+0000","modified":"2024-05-27T09:41:49.972954+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34436,"mds_1":34468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34436":{"gid":34436,"name":"cephfs.smithi119.nghcey","rank":0,"incarnation":33,"state":"up:active","state_seq":29,"addr":"172.21.15.119:6827/3357235708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":3357235708},{"type":"v1","addr":"172.21.15.119:6827","nonce":3357235708}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34468":{"gid":34468,"name":"cephfs.smithi119.ppmzzk","rank":1,"incarnation":33,"state":"up:active","state_seq":23,"addr":"172.21.15.119:6829/2430623742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":2430623742},{"type":"v1","addr":"172.21.15.119:6829","nonce":2430623742}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34436,"qdb_cluster":[34436,34468]},"id":1}]} 2024-05-27T09:45:26.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:26.121+0000 7f0e20a00700 1 -- 172.21.15.119:0/1999668607 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0e00078a40 msgr2=0x7f0e0007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:26.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:26.121+0000 7f0e20a00700 1 --2- 172.21.15.119:0/1999668607 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0e00078a40 0x7f0e0007aef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c199640 tx=0x7f0e08005d50 comp rx=0 tx=0).stop 2024-05-27T09:45:26.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:26.121+0000 7f0e20a00700 1 -- 172.21.15.119:0/1999668607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c102630 msgr2=0x7f0e1c198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:26.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:26.121+0000 7f0e20a00700 1 --2- 172.21.15.119:0/1999668607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c102630 0x7f0e1c198020 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c006010 tx=0x7f0e0c00be20 comp rx=0 tx=0).stop 2024-05-27T09:45:26.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:26.122+0000 7f0e20a00700 1 -- 172.21.15.119:0/1999668607 shutdown_connections 2024-05-27T09:45:26.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:26.122+0000 7f0e20a00700 1 --2- 172.21.15.119:0/1999668607 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f0e00078a40 0x7f0e0007aef0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:26.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:26.122+0000 7f0e20a00700 1 --2- 172.21.15.119:0/1999668607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e1c102630 0x7f0e1c198020 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:26.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:26.122+0000 7f0e20a00700 1 --2- 172.21.15.119:0/1999668607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e1c102fd0 0x7f0e1c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:26.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:26.122+0000 7f0e20a00700 1 -- 172.21.15.119:0/1999668607 >> 172.21.15.119:0/1999668607 conn(0x7f0e1c0fdef0 msgr2=0x7f0e1c1002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:45:26.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:26.122+0000 7f0e20a00700 1 -- 172.21.15.119:0/1999668607 shutdown_connections 2024-05-27T09:45:26.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:26.122+0000 7f0e20a00700 1 -- 172.21.15.119:0/1999668607 wait complete. 2024-05-27T09:45:26.125 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 38 2024-05-27T09:45:27.015 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-27T09:45:27.026 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-27T09:45:27.026 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-27T09:45:27.026 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T09:45:27.085 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-05-27T09:45:27.085 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T09:45:27.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:26 smithi138 ceph-mon[70004]: pgmap v368: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:27.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:26 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/1999668607' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-05-27T09:45:27.138 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-05-27T09:45:27.208 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:26 smithi119 ceph-mon[88269]: pgmap v368: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:27.209 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:26 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/1999668607' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-05-27T09:45:27.550 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:45:27.886 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:27 smithi119 ceph-mon[88269]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:45:28.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:27 smithi138 ceph-mon[70004]: from='mgr.34104 172.21.15.119:0/1909074675' entity='mgr.smithi119.ncwmog' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-27T09:45:28.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:28 smithi119 ceph-mon[88269]: pgmap v369: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:29.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:28 smithi138 ceph-mon[70004]: pgmap v369: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:29.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.905+0000 7ff9e1800700 1 -- 172.21.15.119:0/880252716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9dc1003b0 msgr2=0x7ff9dc100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:29.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.905+0000 7ff9e1800700 1 --2- 172.21.15.119:0/880252716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9dc1003b0 0x7ff9dc100780 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009910 tx=0x7ff9cc009c20 comp rx=0 tx=0).stop 2024-05-27T09:45:29.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.905+0000 7ff9e1800700 1 -- 172.21.15.119:0/880252716 shutdown_connections 2024-05-27T09:45:29.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.905+0000 7ff9e1800700 1 --2- 172.21.15.119:0/880252716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9dc100d50 0x7ff9dc104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:29.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.905+0000 7ff9e1800700 1 --2- 172.21.15.119:0/880252716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9dc1003b0 0x7ff9dc100780 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:29.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.905+0000 7ff9e1800700 1 -- 172.21.15.119:0/880252716 >> 172.21.15.119:0/880252716 conn(0x7ff9dc0fbc30 msgr2=0x7ff9dc0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:45:29.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.905+0000 7ff9e1800700 1 -- 172.21.15.119:0/880252716 shutdown_connections 2024-05-27T09:45:29.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.905+0000 7ff9e1800700 1 -- 172.21.15.119:0/880252716 wait complete. 2024-05-27T09:45:29.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.906+0000 7ff9e1800700 1 Processor -- start 2024-05-27T09:45:29.907 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.906+0000 7ff9e1800700 1 -- start start 2024-05-27T09:45:29.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.906+0000 7ff9e1800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9dc1003b0 0x7ff9dc195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:29.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.906+0000 7ff9e1800700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9dc100d50 0x7ff9dc196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:29.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.906+0000 7ff9e1800700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc196930 con 0x7ff9dc100d50 2024-05-27T09:45:29.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.906+0000 7ff9e1800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc19a6c0 con 0x7ff9dc1003b0 2024-05-27T09:45:29.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.907+0000 7ff9d3e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9dc100d50 0x7ff9dc196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:29.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.907+0000 7ff9d3e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9dc100d50 0x7ff9dc196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58732/0 (socket says 172.21.15.119:58732) 2024-05-27T09:45:29.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.907+0000 7ff9d3e00700 1 -- 172.21.15.119:0/3907168506 learned_addr learned my addr 172.21.15.119:0/3907168506 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:45:29.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.907+0000 7ff9daa00700 1 --2- 172.21.15.119:0/3907168506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9dc1003b0 0x7ff9dc195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:29.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.907+0000 7ff9d3e00700 1 -- 172.21.15.119:0/3907168506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9dc1003b0 msgr2=0x7ff9dc195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:29.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.907+0000 7ff9d3e00700 1 --2- 172.21.15.119:0/3907168506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9dc1003b0 0x7ff9dc195d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:29.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.907+0000 7ff9d3e00700 1 -- 172.21.15.119:0/3907168506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9cc0095f0 con 0x7ff9dc100d50 2024-05-27T09:45:29.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.908+0000 7ff9d3e00700 1 --2- 172.21.15.119:0/3907168506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9dc100d50 0x7ff9dc196250 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400d900 tx=0x7ff9c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:45:29.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.908+0000 7ff9d3400700 1 -- 172.21.15.119:0/3907168506 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9c4009900 con 0x7ff9dc100d50 2024-05-27T09:45:29.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.908+0000 7ff9d3400700 1 -- 172.21.15.119:0/3907168506 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff9c4010460 con 0x7ff9dc100d50 2024-05-27T09:45:29.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.908+0000 7ff9e1800700 1 -- 172.21.15.119:0/3907168506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9dc19a9a0 con 0x7ff9dc100d50 2024-05-27T09:45:29.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.908+0000 7ff9d3400700 1 -- 172.21.15.119:0/3907168506 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9c400f5f0 con 0x7ff9dc100d50 2024-05-27T09:45:29.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.908+0000 7ff9e1800700 1 -- 172.21.15.119:0/3907168506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9dc19aef0 con 0x7ff9dc100d50 2024-05-27T09:45:29.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.909+0000 7ff9e1800700 1 -- 172.21.15.119:0/3907168506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9dc1086c0 con 0x7ff9dc100d50 2024-05-27T09:45:29.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.910+0000 7ff9d3400700 1 -- 172.21.15.119:0/3907168506 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7ff9c40105d0 con 0x7ff9dc100d50 2024-05-27T09:45:29.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.911+0000 7ff9d3400700 1 --2- 172.21.15.119:0/3907168506 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff9c0078a40 0x7ff9c007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:29.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.912+0000 7ff9d3400700 1 -- 172.21.15.119:0/3907168506 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff9c409abb0 con 0x7ff9dc100d50 2024-05-27T09:45:29.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.912+0000 7ff9daa00700 1 --2- 172.21.15.119:0/3907168506 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff9c0078a40 0x7ff9c007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:29.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.913+0000 7ff9daa00700 1 --2- 172.21.15.119:0/3907168506 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff9c0078a40 0x7ff9c007aef0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc000c00 tx=0x7ff9cc005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:45:29.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:29.917+0000 7ff9d3400700 1 -- 172.21.15.119:0/3907168506 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7ff9c4062760 con 0x7ff9dc100d50 2024-05-27T09:45:30.169 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:30.168+0000 7ff9e1800700 1 -- 172.21.15.119:0/3907168506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff9dc19b160 con 0x7ff9dc100d50 2024-05-27T09:45:30.170 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:30.169+0000 7ff9d3400700 1 -- 172.21.15.119:0/3907168506 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v85) v1 ==== 81+0+2458 (secure 0 0 0) 0x7ff9c4062100 con 0x7ff9dc100d50 2024-05-27T09:45:30.171 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6829/2100782307 2024-05-28T09:39:57.913110+0000 2024-05-27T09:45:30.171 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6828/2100782307 2024-05-28T09:39:57.913110+0000 2024-05-27T09:45:30.171 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.138:6824/897058594 2024-05-28T09:39:57.913143+0000 2024-05-27T09:45:30.171 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6801/1743331737 2024-05-28T09:17:30.956826+0000 2024-05-27T09:45:30.171 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/2275270800 2024-05-28T09:29:10.391394+0000 2024-05-27T09:45:30.171 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6801/1378367028 2024-05-28T09:34:27.228390+0000 2024-05-27T09:45:30.171 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6800/1743331737 2024-05-28T09:17:30.956826+0000 2024-05-27T09:45:30.171 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6801/2191862652 2024-05-28T09:29:10.391394+0000 2024-05-27T09:45:30.171 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/2367404262 2024-05-28T09:17:30.956826+0000 2024-05-27T09:45:30.172 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/4074206116 2024-05-28T09:18:11.825050+0000 2024-05-27T09:45:30.172 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6800/2191862652 2024-05-28T09:29:10.391394+0000 2024-05-27T09:45:30.172 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/2281796205 2024-05-28T09:19:36.161573+0000 2024-05-27T09:45:30.172 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.138:0/322898796 2024-05-28T09:30:27.742131+0000 2024-05-27T09:45:30.172 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.138:6829/1384904888 2024-05-28T09:30:27.742131+0000 2024-05-27T09:45:30.172 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.138:0/3899635028 2024-05-28T09:30:27.742131+0000 2024-05-27T09:45:30.172 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6800/1553098156 2024-05-28T09:18:11.825050+0000 2024-05-27T09:45:30.172 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.138:0/4267168311 2024-05-28T09:30:27.742131+0000 2024-05-27T09:45:30.172 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6800/1378367028 2024-05-28T09:34:27.228390+0000 2024-05-27T09:45:30.172 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.138:0/2426873830 2024-05-28T09:30:27.742131+0000 2024-05-27T09:45:30.172 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/2176658945 2024-05-28T09:17:30.956826+0000 2024-05-27T09:45:30.172 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/2970229585 2024-05-28T09:18:11.825050+0000 2024-05-27T09:45:30.172 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6827/303813653 2024-05-28T09:26:09.620558+0000 2024-05-27T09:45:30.172 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.138:6825/897058594 2024-05-28T09:39:57.913143+0000 2024-05-27T09:45:30.172 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/3152330534 2024-05-28T09:34:27.228390+0000 2024-05-27T09:45:30.173 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6801/2118907438 2024-05-28T09:19:36.161573+0000 2024-05-27T09:45:30.173 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6826/303813653 2024-05-28T09:26:09.620558+0000 2024-05-27T09:45:30.173 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6800/2118907438 2024-05-28T09:19:36.161573+0000 2024-05-27T09:45:30.173 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/2308226979 2024-05-28T09:19:36.161573+0000 2024-05-27T09:45:30.173 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/1310642907 2024-05-28T09:19:36.161573+0000 2024-05-27T09:45:30.173 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6801/1553098156 2024-05-28T09:18:11.825050+0000 2024-05-27T09:45:30.173 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/3311320241 2024-05-28T09:17:30.956826+0000 2024-05-27T09:45:30.173 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/993242246 2024-05-28T09:34:27.228390+0000 2024-05-27T09:45:30.173 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.138:6828/1384904888 2024-05-28T09:30:27.742131+0000 2024-05-27T09:45:30.173 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.138:0/813155227 2024-05-28T09:30:27.742131+0000 2024-05-27T09:45:30.173 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/1141668291 2024-05-28T09:29:10.391394+0000 2024-05-27T09:45:30.173 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/987818704 2024-05-28T09:18:11.825050+0000 2024-05-27T09:45:30.174 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/893995738 2024-05-28T09:29:10.391394+0000 2024-05-27T09:45:30.174 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/553584352 2024-05-28T09:29:10.391394+0000 2024-05-27T09:45:30.174 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.138:0/694666219 2024-05-28T09:30:27.742131+0000 2024-05-27T09:45:30.174 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/854928408 2024-05-28T09:34:27.228390+0000 2024-05-27T09:45:30.174 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/5713929 2024-05-28T09:34:27.228390+0000 2024-05-27T09:45:30.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:30.173+0000 7ff9e1800700 1 -- 172.21.15.119:0/3907168506 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff9c0078a40 msgr2=0x7ff9c007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:30.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:30.173+0000 7ff9e1800700 1 --2- 172.21.15.119:0/3907168506 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff9c0078a40 0x7ff9c007aef0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc000c00 tx=0x7ff9cc005fd0 comp rx=0 tx=0).stop 2024-05-27T09:45:30.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:30.173+0000 7ff9e1800700 1 -- 172.21.15.119:0/3907168506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9dc100d50 msgr2=0x7ff9dc196250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:30.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:30.173+0000 7ff9e1800700 1 --2- 172.21.15.119:0/3907168506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9dc100d50 0x7ff9dc196250 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400d900 tx=0x7ff9c400dcc0 comp rx=0 tx=0).stop 2024-05-27T09:45:30.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:30.173+0000 7ff9e1800700 1 -- 172.21.15.119:0/3907168506 shutdown_connections 2024-05-27T09:45:30.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:30.173+0000 7ff9e1800700 1 --2- 172.21.15.119:0/3907168506 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7ff9c0078a40 0x7ff9c007aef0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:30.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:30.173+0000 7ff9e1800700 1 --2- 172.21.15.119:0/3907168506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9dc1003b0 0x7ff9dc195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:30.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:30.173+0000 7ff9e1800700 1 --2- 172.21.15.119:0/3907168506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9dc100d50 0x7ff9dc196250 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:30.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:30.173+0000 7ff9e1800700 1 -- 172.21.15.119:0/3907168506 >> 172.21.15.119:0/3907168506 conn(0x7ff9dc0fbc30 msgr2=0x7ff9dc0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:45:30.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:30.173+0000 7ff9e1800700 1 -- 172.21.15.119:0/3907168506 shutdown_connections 2024-05-27T09:45:30.175 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:30.173+0000 7ff9e1800700 1 -- 172.21.15.119:0/3907168506 wait complete. 2024-05-27T09:45:30.176 INFO:teuthology.orchestra.run.smithi119.stderr:listed 41 entries 2024-05-27T09:45:31.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:30 smithi138 ceph-mon[70004]: pgmap v370: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:31.136 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:30 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/3907168506' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T09:45:31.199 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:30 smithi119 ceph-mon[88269]: pgmap v370: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:31.199 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:30 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/3907168506' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T09:45:31.251 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-27T09:45:31.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:45:31.251 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-27T09:45:31.340 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2024-05-27T09:45:31.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:45:31.340 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-27T09:45:31.411 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-27T09:45:31.411 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T09:45:31.469 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-27T09:45:31.469 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T09:45:31.520 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-05-27T09:45:31.924 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/mon.smithi119/config 2024-05-27T09:45:33.105 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:32 smithi119 ceph-mon[88269]: pgmap v371: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:33.135 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:32 smithi138 ceph-mon[70004]: pgmap v371: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-05-27T09:45:34.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.120+0000 7f80f1000700 1 -- 172.21.15.119:0/296238903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80ec10b700 msgr2=0x7f80ec10bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:34.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.120+0000 7f80f1000700 1 --2- 172.21.15.119:0/296238903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80ec10b700 0x7f80ec10bad0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009910 tx=0x7f80dc009c20 comp rx=0 tx=0).stop 2024-05-27T09:45:34.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.120+0000 7f80f1000700 1 -- 172.21.15.119:0/296238903 shutdown_connections 2024-05-27T09:45:34.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.120+0000 7f80f1000700 1 --2- 172.21.15.119:0/296238903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80ec10c010 0x7f80ec106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:34.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.120+0000 7f80f1000700 1 --2- 172.21.15.119:0/296238903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80ec10b700 0x7f80ec10bad0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:34.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.120+0000 7f80f1000700 1 -- 172.21.15.119:0/296238903 >> 172.21.15.119:0/296238903 conn(0x7f80ec076000 msgr2=0x7f80ec078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:45:34.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.121+0000 7f80f1000700 1 -- 172.21.15.119:0/296238903 shutdown_connections 2024-05-27T09:45:34.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.121+0000 7f80f1000700 1 -- 172.21.15.119:0/296238903 wait complete. 2024-05-27T09:45:34.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.122+0000 7f80f1000700 1 Processor -- start 2024-05-27T09:45:34.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.122+0000 7f80f1000700 1 -- start start 2024-05-27T09:45:34.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.122+0000 7f80f1000700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80ec10b700 0x7f80ec0815f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:34.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.122+0000 7f80f1000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80ec10c010 0x7f80ec07c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:34.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.122+0000 7f80f1000700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec07cb30 con 0x7f80ec10b700 2024-05-27T09:45:34.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.122+0000 7f80f1000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec07cca0 con 0x7f80ec10c010 2024-05-27T09:45:34.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.123+0000 7f80e9e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80ec10b700 0x7f80ec0815f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:34.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.123+0000 7f80e9e00700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80ec10b700 0x7f80ec0815f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58748/0 (socket says 172.21.15.119:58748) 2024-05-27T09:45:34.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.123+0000 7f80e9e00700 1 -- 172.21.15.119:0/1033373927 learned_addr learned my addr 172.21.15.119:0/1033373927 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-05-27T09:45:34.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.123+0000 7f80e3e00700 1 --2- 172.21.15.119:0/1033373927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80ec10c010 0x7f80ec07c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:34.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.123+0000 7f80e9e00700 1 -- 172.21.15.119:0/1033373927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80ec10c010 msgr2=0x7f80ec07c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:34.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.123+0000 7f80e9e00700 1 --2- 172.21.15.119:0/1033373927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80ec10c010 0x7f80ec07c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:34.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.123+0000 7f80e9e00700 1 -- 172.21.15.119:0/1033373927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80dc0095f0 con 0x7f80ec10b700 2024-05-27T09:45:34.125 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.124+0000 7f80e9e00700 1 --2- 172.21.15.119:0/1033373927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80ec10b700 0x7f80ec0815f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f80dc00b5c0 tx=0x7f80dc00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:45:34.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.124+0000 7f80e2a00700 1 -- 172.21.15.119:0/1033373927 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80dc01d070 con 0x7f80ec10b700 2024-05-27T09:45:34.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.124+0000 7f80e2a00700 1 -- 172.21.15.119:0/1033373927 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80dc00f460 con 0x7f80ec10b700 2024-05-27T09:45:34.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.124+0000 7f80f1000700 1 -- 172.21.15.119:0/1033373927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ec07cf20 con 0x7f80ec10b700 2024-05-27T09:45:34.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.124+0000 7f80e2a00700 1 -- 172.21.15.119:0/1033373927 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80dc017610 con 0x7f80ec10b700 2024-05-27T09:45:34.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.124+0000 7f80f1000700 1 -- 172.21.15.119:0/1033373927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ec1af0d0 con 0x7f80ec10b700 2024-05-27T09:45:34.126 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.125+0000 7f80f1000700 1 -- 172.21.15.119:0/1033373927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80ec04ea50 con 0x7f80ec10b700 2024-05-27T09:45:34.127 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.126+0000 7f80e2a00700 1 -- 172.21.15.119:0/1033373927 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 100082+0+0 (secure 0 0 0) 0x7f80dc00f5d0 con 0x7f80ec10b700 2024-05-27T09:45:34.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.127+0000 7f80e2a00700 1 --2- 172.21.15.119:0/1033373927 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f80d0078a40 0x7f80d007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T09:45:34.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.127+0000 7f80e2a00700 1 -- 172.21.15.119:0/1033373927 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f80dc09c370 con 0x7f80ec10b700 2024-05-27T09:45:34.128 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.128+0000 7f80e3e00700 1 --2- 172.21.15.119:0/1033373927 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f80d0078a40 0x7f80d007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T09:45:34.129 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.128+0000 7f80e3e00700 1 --2- 172.21.15.119:0/1033373927 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f80d0078a40 0x7f80d007aef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f80d4005fd0 tx=0x7f80d400a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T09:45:34.133 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.133+0000 7f80e2a00700 1 -- 172.21.15.119:0/1033373927 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+191963 (secure 0 0 0) 0x7f80dc063f20 con 0x7f80ec10b700 2024-05-27T09:45:34.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.388+0000 7f80f1000700 1 -- 172.21.15.119:0/1033373927 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f80ec1af260 con 0x7f80ec10b700 2024-05-27T09:45:34.390 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.389+0000 7f80e2a00700 1 -- 172.21.15.119:0/1033373927 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v85) v1 ==== 81+0+2458 (secure 0 0 0) 0x7f80dc0638c0 con 0x7f80ec10b700 2024-05-27T09:45:34.391 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6829/2100782307 2024-05-28T09:39:57.913110+0000 2024-05-27T09:45:34.391 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6828/2100782307 2024-05-28T09:39:57.913110+0000 2024-05-27T09:45:34.391 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.138:6824/897058594 2024-05-28T09:39:57.913143+0000 2024-05-27T09:45:34.391 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6801/1743331737 2024-05-28T09:17:30.956826+0000 2024-05-27T09:45:34.391 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/2275270800 2024-05-28T09:29:10.391394+0000 2024-05-27T09:45:34.391 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6801/1378367028 2024-05-28T09:34:27.228390+0000 2024-05-27T09:45:34.391 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6800/1743331737 2024-05-28T09:17:30.956826+0000 2024-05-27T09:45:34.391 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6801/2191862652 2024-05-28T09:29:10.391394+0000 2024-05-27T09:45:34.391 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/2367404262 2024-05-28T09:17:30.956826+0000 2024-05-27T09:45:34.392 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/4074206116 2024-05-28T09:18:11.825050+0000 2024-05-27T09:45:34.392 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6800/2191862652 2024-05-28T09:29:10.391394+0000 2024-05-27T09:45:34.392 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/2281796205 2024-05-28T09:19:36.161573+0000 2024-05-27T09:45:34.392 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.138:0/322898796 2024-05-28T09:30:27.742131+0000 2024-05-27T09:45:34.392 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.138:6829/1384904888 2024-05-28T09:30:27.742131+0000 2024-05-27T09:45:34.392 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.138:0/3899635028 2024-05-28T09:30:27.742131+0000 2024-05-27T09:45:34.392 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6800/1553098156 2024-05-28T09:18:11.825050+0000 2024-05-27T09:45:34.392 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.138:0/4267168311 2024-05-28T09:30:27.742131+0000 2024-05-27T09:45:34.392 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6800/1378367028 2024-05-28T09:34:27.228390+0000 2024-05-27T09:45:34.392 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.138:0/2426873830 2024-05-28T09:30:27.742131+0000 2024-05-27T09:45:34.392 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/2176658945 2024-05-28T09:17:30.956826+0000 2024-05-27T09:45:34.392 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/2970229585 2024-05-28T09:18:11.825050+0000 2024-05-27T09:45:34.392 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6827/303813653 2024-05-28T09:26:09.620558+0000 2024-05-27T09:45:34.392 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.138:6825/897058594 2024-05-28T09:39:57.913143+0000 2024-05-27T09:45:34.393 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/3152330534 2024-05-28T09:34:27.228390+0000 2024-05-27T09:45:34.393 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6801/2118907438 2024-05-28T09:19:36.161573+0000 2024-05-27T09:45:34.393 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6826/303813653 2024-05-28T09:26:09.620558+0000 2024-05-27T09:45:34.393 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6800/2118907438 2024-05-28T09:19:36.161573+0000 2024-05-27T09:45:34.393 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/2308226979 2024-05-28T09:19:36.161573+0000 2024-05-27T09:45:34.393 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/1310642907 2024-05-28T09:19:36.161573+0000 2024-05-27T09:45:34.393 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6801/1553098156 2024-05-28T09:18:11.825050+0000 2024-05-27T09:45:34.393 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/3311320241 2024-05-28T09:17:30.956826+0000 2024-05-27T09:45:34.393 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/993242246 2024-05-28T09:34:27.228390+0000 2024-05-27T09:45:34.393 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.138:6828/1384904888 2024-05-28T09:30:27.742131+0000 2024-05-27T09:45:34.393 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.138:0/813155227 2024-05-28T09:30:27.742131+0000 2024-05-27T09:45:34.393 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/1141668291 2024-05-28T09:29:10.391394+0000 2024-05-27T09:45:34.393 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/987818704 2024-05-28T09:18:11.825050+0000 2024-05-27T09:45:34.393 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/893995738 2024-05-28T09:29:10.391394+0000 2024-05-27T09:45:34.394 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/553584352 2024-05-28T09:29:10.391394+0000 2024-05-27T09:45:34.394 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.138:0/694666219 2024-05-28T09:30:27.742131+0000 2024-05-27T09:45:34.394 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/854928408 2024-05-28T09:34:27.228390+0000 2024-05-27T09:45:34.394 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/5713929 2024-05-28T09:34:27.228390+0000 2024-05-27T09:45:34.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.393+0000 7f80f1000700 1 -- 172.21.15.119:0/1033373927 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f80d0078a40 msgr2=0x7f80d007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:34.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.393+0000 7f80f1000700 1 --2- 172.21.15.119:0/1033373927 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f80d0078a40 0x7f80d007aef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f80d4005fd0 tx=0x7f80d400a560 comp rx=0 tx=0).stop 2024-05-27T09:45:34.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.393+0000 7f80f1000700 1 -- 172.21.15.119:0/1033373927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80ec10b700 msgr2=0x7f80ec0815f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T09:45:34.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.393+0000 7f80f1000700 1 --2- 172.21.15.119:0/1033373927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80ec10b700 0x7f80ec0815f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f80dc00b5c0 tx=0x7f80dc00be00 comp rx=0 tx=0).stop 2024-05-27T09:45:34.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.393+0000 7f80f1000700 1 -- 172.21.15.119:0/1033373927 shutdown_connections 2024-05-27T09:45:34.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.393+0000 7f80f1000700 1 --2- 172.21.15.119:0/1033373927 >> [v2:172.21.15.119:6800/1449054506,v1:172.21.15.119:6801/1449054506] conn(0x7f80d0078a40 0x7f80d007aef0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:34.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.393+0000 7f80f1000700 1 --2- 172.21.15.119:0/1033373927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80ec10b700 0x7f80ec0815f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:34.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.393+0000 7f80f1000700 1 --2- 172.21.15.119:0/1033373927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80ec10c010 0x7f80ec07c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T09:45:34.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.393+0000 7f80f1000700 1 -- 172.21.15.119:0/1033373927 >> 172.21.15.119:0/1033373927 conn(0x7f80ec076000 msgr2=0x7f80ec076880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T09:45:34.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.394+0000 7f80f1000700 1 -- 172.21.15.119:0/1033373927 shutdown_connections 2024-05-27T09:45:34.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-05-27T09:45:34.394+0000 7f80f1000700 1 -- 172.21.15.119:0/1033373927 wait complete. 2024-05-27T09:45:34.396 INFO:teuthology.orchestra.run.smithi119.stderr:listed 41 entries 2024-05-27T09:45:35.089 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:34 smithi119 ceph-mon[88269]: pgmap v372: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-05-27T09:45:35.089 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:34 smithi119 ceph-mon[88269]: from='client.? 172.21.15.119:0/1033373927' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T09:45:35.140 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-27T09:45:35.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:45:35.140 DEBUG:teuthology.orchestra.run.smithi138:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-27T09:45:35.222 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:34 smithi138 ceph-mon[70004]: pgmap v372: 65 pgs: 65 active+clean; 332 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-05-27T09:45:35.223 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:34 smithi138 ceph-mon[70004]: from='client.? 172.21.15.119:0/1033373927' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T09:45:35.242 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-27T09:45:35.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:45:35.242 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-27T09:45:35.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:45:35.318 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2024-05-27T09:45:35.378 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-27T09:45:35.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:45:35.378 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T09:45:35.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:45:35.465 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2024-05-27T09:45:35.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:45:35.577 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-05-27T09:45:35.636 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-27T09:45:35.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:45:35.637 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-27T09:45:35.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T09:45:35.724 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-05-27T09:45:35.844 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-27T09:45:35.858 INFO:tasks.cephadm:Teardown begin 2024-05-27T09:45:35.858 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T09:45:35.951 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T09:45:36.037 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-27T09:45:36.037 DEBUG:teuthology.orchestra.run.smithi119:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-27T09:45:36.092 DEBUG:teuthology.orchestra.run.smithi138:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-27T09:45:36.148 INFO:tasks.cephadm:Stopping all daemons... 2024-05-27T09:45:36.149 INFO:tasks.cephadm.mon.smithi119:Stopping mon.smithi119... 2024-05-27T09:45:36.149 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@mon.smithi119 2024-05-27T09:45:36.498 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:36 smithi119 systemd[1]: Stopping Ceph mon.smithi119 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:45:36.955 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:36 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119[88265]: 2024-05-27T09:45:36.676+0000 7f901f400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi119 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T09:45:36.955 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:36 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119[88265]: 2024-05-27T09:45:36.676+0000 7f901f400700 -1 mon.smithi119@0(leader) e3 *** Got Signal Terminated *** 2024-05-27T09:45:37.247 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:36 smithi119 podman[136820]: 2024-05-27 09:45:36.954133498 +0000 UTC m=+0.419515338 container died 3e5c7a9ad372ff49da57e9334c4528c3069c1e1b46c4d00d1e8cbac016bf7e52 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-27T09:45:37.640 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:37 smithi119 podman[136820]: 2024-05-27 09:45:37.376294664 +0000 UTC m=+0.841676514 container cleanup 3e5c7a9ad372ff49da57e9334c4528c3069c1e1b46c4d00d1e8cbac016bf7e52 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-27T09:45:37.640 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:37 smithi119 bash[136820]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119 2024-05-27T09:45:37.998 INFO:journalctl@ceph.mon.smithi119.smithi119.stdout:May 27 09:45:37 smithi119 podman[136850]: 2024-05-27 09:45:37.661152756 +0000 UTC m=+0.691135302 container remove 3e5c7a9ad372ff49da57e9334c4528c3069c1e1b46c4d00d1e8cbac016bf7e52 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi119, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-27T09:45:38.178 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@mon.smithi119.service' 2024-05-27T09:45:38.337 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T09:45:38.337 INFO:tasks.cephadm.mon.smithi119:Stopped mon.smithi119 2024-05-27T09:45:38.337 INFO:tasks.cephadm.mon.smithi138:Stopping mon.smithi138... 2024-05-27T09:45:38.337 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@mon.smithi138 2024-05-27T09:45:38.764 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:38 smithi138 systemd[1]: Stopping Ceph mon.smithi138 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:45:39.017 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:38 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi138[70000]: 2024-05-27T09:45:38.943+0000 7f32d7600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi138 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T09:45:39.017 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:38 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi138[70000]: 2024-05-27T09:45:38.943+0000 7f32d7600700 -1 mon.smithi138@1(peon) e3 *** Got Signal Terminated *** 2024-05-27T09:45:39.386 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:39 smithi138 podman[106559]: 2024-05-27 09:45:39.017030428 +0000 UTC m=+0.280783061 container died e4d471274cf172a54ea627d0e92c17ec3d31a6434d446bcc9f835f7ac4d853f3 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi138, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T09:45:39.996 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:39 smithi138 podman[106559]: 2024-05-27 09:45:39.718722366 +0000 UTC m=+0.982475009 container cleanup e4d471274cf172a54ea627d0e92c17ec3d31a6434d446bcc9f835f7ac4d853f3 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi138, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, ceph=True) 2024-05-27T09:45:39.996 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:39 smithi138 bash[106559]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi138 2024-05-27T09:45:40.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:May 27 09:45:39 smithi138 podman[106572]: 2024-05-27 09:45:39.995326841 +0000 UTC m=+0.958955025 container remove e4d471274cf172a54ea627d0e92c17ec3d31a6434d446bcc9f835f7ac4d853f3 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-mon-smithi138, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T09:45:40.470 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@mon.smithi138.service' 2024-05-27T09:45:40.587 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T09:45:40.587 INFO:tasks.cephadm.mon.smithi138:Stopped mon.smithi138 2024-05-27T09:45:40.587 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-27T09:45:40.587 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.0 2024-05-27T09:45:40.997 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:45:40 smithi119 systemd[1]: Stopping Ceph osd.0 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:45:41.498 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:45:41 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0[100032]: 2024-05-27T09:45:41.179+0000 7f35c3600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T09:45:41.498 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:45:41 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0[100032]: 2024-05-27T09:45:41.179+0000 7f35c3600700 -1 osd.0 85 *** Got signal Terminated *** 2024-05-27T09:45:41.498 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:45:41 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0[100032]: 2024-05-27T09:45:41.179+0000 7f35c3600700 -1 osd.0 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T09:45:46.998 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:45:46 smithi119 podman[136963]: 2024-05-27 09:45:46.699224948 +0000 UTC m=+5.659441089 container died 5a0e162d870ecb78aa3fd00f593a8789de937ea70e8a6c2e3939448da312b6dd (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-27T09:45:47.493 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:45:47 smithi119 podman[136963]: 2024-05-27 09:45:47.281050649 +0000 UTC m=+6.241266799 container cleanup 5a0e162d870ecb78aa3fd00f593a8789de937ea70e8a6c2e3939448da312b6dd (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-27T09:45:47.493 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:45:47 smithi119 bash[136963]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0 2024-05-27T09:45:47.747 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:45:47 smithi119 podman[136982]: 2024-05-27 09:45:47.491089504 +0000 UTC m=+0.774395534 container remove 5a0e162d870ecb78aa3fd00f593a8789de937ea70e8a6c2e3939448da312b6dd (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T09:45:48.514 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:45:48 smithi119 podman[137069]: 2024-05-27 09:45:48.166329169 +0000 UTC m=+0.032996170 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:45:48.514 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:45:48 smithi119 podman[137069]: 2024-05-27 09:45:48.275638053 +0000 UTC m=+0.142305110 container create b01a6d2c1ed6e7a76729ae9a198874e91ab82b46d506604c5bdfa7b7c52cad43 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2) 2024-05-27T09:45:48.514 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:45:48 smithi119 podman[137069]: 2024-05-27 09:45:48.513109405 +0000 UTC m=+0.379776486 container init b01a6d2c1ed6e7a76729ae9a198874e91ab82b46d506604c5bdfa7b7c52cad43 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-deactivate, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T09:45:48.976 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:45:48 smithi119 podman[137069]: 2024-05-27 09:45:48.525041086 +0000 UTC m=+0.391708114 container start b01a6d2c1ed6e7a76729ae9a198874e91ab82b46d506604c5bdfa7b7c52cad43 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-27T09:45:48.977 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:45:48 smithi119 podman[137069]: 2024-05-27 09:45:48.653317396 +0000 UTC m=+0.519984476 container attach b01a6d2c1ed6e7a76729ae9a198874e91ab82b46d506604c5bdfa7b7c52cad43 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-05-27T09:45:49.247 INFO:journalctl@ceph.osd.0.smithi119.stdout:May 27 09:45:48 smithi119 podman[137069]: 2024-05-27 09:45:48.97668 +0000 UTC m=+0.843347044 container died b01a6d2c1ed6e7a76729ae9a198874e91ab82b46d506604c5bdfa7b7c52cad43 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-27T09:45:49.713 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.0.service' 2024-05-27T09:45:49.827 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T09:45:49.827 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-27T09:45:49.827 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-27T09:45:49.827 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.1 2024-05-27T09:45:50.247 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:45:49 smithi119 systemd[1]: Stopping Ceph osd.1 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:45:50.748 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:45:50 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1[105380]: 2024-05-27T09:45:50.466+0000 7f7145c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T09:45:50.748 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:45:50 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1[105380]: 2024-05-27T09:45:50.466+0000 7f7145c00700 -1 osd.1 85 *** Got signal Terminated *** 2024-05-27T09:45:50.748 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:45:50 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1[105380]: 2024-05-27T09:45:50.466+0000 7f7145c00700 -1 osd.1 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T09:45:56.220 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:45:55 smithi119 podman[137173]: 2024-05-27 09:45:55.935535772 +0000 UTC m=+5.631747800 container died effcc7c53dd3bd55f6e6fa6153bdb88e85e4ccfc7202275c4b63a1bfe29bb26e (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T09:45:56.497 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:45:56 smithi119 podman[137173]: 2024-05-27 09:45:56.309540383 +0000 UTC m=+6.005752437 container cleanup effcc7c53dd3bd55f6e6fa6153bdb88e85e4ccfc7202275c4b63a1bfe29bb26e (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-27T09:45:56.498 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:45:56 smithi119 bash[137173]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1 2024-05-27T09:45:56.998 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:45:56 smithi119 podman[137190]: 2024-05-27 09:45:56.535828976 +0000 UTC m=+0.582053212 container remove effcc7c53dd3bd55f6e6fa6153bdb88e85e4ccfc7202275c4b63a1bfe29bb26e (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T09:45:57.748 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:45:57 smithi119 podman[137281]: 2024-05-27 09:45:57.359909942 +0000 UTC m=+0.033604917 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:45:57.748 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:45:57 smithi119 podman[137281]: 2024-05-27 09:45:57.470120357 +0000 UTC m=+0.143815368 container create cc32b4aa89e4f8ce77185189be41cabb98c51b5e6228055a6cb0564545c101be (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-27T09:45:58.225 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:45:57 smithi119 podman[137281]: 2024-05-27 09:45:57.757661026 +0000 UTC m=+0.431356044 container init cc32b4aa89e4f8ce77185189be41cabb98c51b5e6228055a6cb0564545c101be (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-27T09:45:58.226 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:45:57 smithi119 podman[137281]: 2024-05-27 09:45:57.770607804 +0000 UTC m=+0.444302845 container start cc32b4aa89e4f8ce77185189be41cabb98c51b5e6228055a6cb0564545c101be (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T09:45:58.226 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:45:57 smithi119 podman[137281]: 2024-05-27 09:45:57.864376813 +0000 UTC m=+0.538071808 container attach cc32b4aa89e4f8ce77185189be41cabb98c51b5e6228055a6cb0564545c101be (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-05-27T09:45:58.226 INFO:journalctl@ceph.osd.1.smithi119.stdout:May 27 09:45:58 smithi119 podman[137281]: 2024-05-27 09:45:58.224938616 +0000 UTC m=+0.898633720 container died cc32b4aa89e4f8ce77185189be41cabb98c51b5e6228055a6cb0564545c101be (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-27T09:45:58.931 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.1.service' 2024-05-27T09:45:59.022 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T09:45:59.022 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-27T09:45:59.023 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-27T09:45:59.023 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.2 2024-05-27T09:45:59.247 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:45:59 smithi119 systemd[1]: Stopping Ceph osd.2 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:45:59.865 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:45:59 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2[111198]: 2024-05-27T09:45:59.519+0000 7f9899200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T09:45:59.865 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:45:59 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2[111198]: 2024-05-27T09:45:59.519+0000 7f9899200700 -1 osd.2 85 *** Got signal Terminated *** 2024-05-27T09:45:59.865 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:45:59 smithi119 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2[111198]: 2024-05-27T09:45:59.519+0000 7f9899200700 -1 osd.2 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T09:46:05.321 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:46:05 smithi119 podman[137395]: 2024-05-27 09:46:05.060903019 +0000 UTC m=+5.675789315 container died f0bd34ca416f96bbd725da3952c260f184d177f64497284dd5559c4c869af748 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-27T09:46:05.658 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:46:05 smithi119 podman[137395]: 2024-05-27 09:46:05.429482405 +0000 UTC m=+6.044368724 container cleanup f0bd34ca416f96bbd725da3952c260f184d177f64497284dd5559c4c869af748 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T09:46:05.658 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:46:05 smithi119 bash[137395]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2 2024-05-27T09:46:05.997 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:46:05 smithi119 podman[137412]: 2024-05-27 09:46:05.657286024 +0000 UTC m=+0.580757744 container remove f0bd34ca416f96bbd725da3952c260f184d177f64497284dd5559c4c869af748 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, ceph=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-27T09:46:06.936 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:46:06 smithi119 podman[137503]: 2024-05-27 09:46:06.492605599 +0000 UTC m=+0.035357486 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:46:06.936 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:46:06 smithi119 podman[137503]: 2024-05-27 09:46:06.64834782 +0000 UTC m=+0.191099749 container create 6251b9618e0743f62f7841067a82990a0990a841461af9ed2e2e99b7b607a314 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-27T09:46:06.936 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:46:06 smithi119 podman[137503]: 2024-05-27 09:46:06.935547588 +0000 UTC m=+0.478299537 container init 6251b9618e0743f62f7841067a82990a0990a841461af9ed2e2e99b7b607a314 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-05-27T09:46:07.248 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:46:06 smithi119 podman[137503]: 2024-05-27 09:46:06.947662471 +0000 UTC m=+0.490414363 container start 6251b9618e0743f62f7841067a82990a0990a841461af9ed2e2e99b7b607a314 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T09:46:07.248 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:46:07 smithi119 podman[137503]: 2024-05-27 09:46:07.009026793 +0000 UTC m=+0.551778719 container attach 6251b9618e0743f62f7841067a82990a0990a841461af9ed2e2e99b7b607a314 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-27T09:46:07.748 INFO:journalctl@ceph.osd.2.smithi119.stdout:May 27 09:46:07 smithi119 podman[137503]: 2024-05-27 09:46:07.397121404 +0000 UTC m=+0.939873376 container died 6251b9618e0743f62f7841067a82990a0990a841461af9ed2e2e99b7b607a314 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-27T09:46:08.080 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.2.service' 2024-05-27T09:46:08.135 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:07 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:07.639+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:43.831515+0000 front 2024-05-27T09:45:43.831710+0000 (oldest deadline 2024-05-27T09:46:06.731790+0000) 2024-05-27T09:46:08.195 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T09:46:08.195 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-27T09:46:08.195 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-27T09:46:08.195 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.3 2024-05-27T09:46:08.631 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:46:08 smithi138 systemd[1]: Stopping Ceph osd.3 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:46:08.885 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:08 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:08.629+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:43.831515+0000 front 2024-05-27T09:45:43.831710+0000 (oldest deadline 2024-05-27T09:46:06.731790+0000) 2024-05-27T09:46:08.886 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:46:08 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3[79199]: 2024-05-27T09:46:08.786+0000 7fd3e3400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T09:46:08.886 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:46:08 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3[79199]: 2024-05-27T09:46:08.786+0000 7fd3e3400700 -1 osd.3 85 *** Got signal Terminated *** 2024-05-27T09:46:08.886 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:46:08 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3[79199]: 2024-05-27T09:46:08.786+0000 7fd3e3400700 -1 osd.3 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T09:46:09.385 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:46:08 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3[79199]: 2024-05-27T09:46:08.898+0000 7fd3d9c00700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.526878+0000 front 2024-05-27T09:45:44.526925+0000 (oldest deadline 2024-05-27T09:46:08.626248+0000) 2024-05-27T09:46:09.885 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:09 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:09.589+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:43.831515+0000 front 2024-05-27T09:45:43.831710+0000 (oldest deadline 2024-05-27T09:46:06.731790+0000) 2024-05-27T09:46:10.386 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:46:09 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3[79199]: 2024-05-27T09:46:09.931+0000 7fd3d9c00700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.526878+0000 front 2024-05-27T09:45:44.526925+0000 (oldest deadline 2024-05-27T09:46:08.626248+0000) 2024-05-27T09:46:10.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:09 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:09.916+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:10.885 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:10 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:10.592+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:43.831515+0000 front 2024-05-27T09:45:43.831710+0000 (oldest deadline 2024-05-27T09:46:06.731790+0000) 2024-05-27T09:46:11.385 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:46:10 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3[79199]: 2024-05-27T09:46:10.958+0000 7fd3d9c00700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.526878+0000 front 2024-05-27T09:45:44.526925+0000 (oldest deadline 2024-05-27T09:46:08.626248+0000) 2024-05-27T09:46:11.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:10 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:10.886+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:11.886 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:11 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:11.574+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:43.831515+0000 front 2024-05-27T09:45:43.831710+0000 (oldest deadline 2024-05-27T09:46:06.731790+0000) 2024-05-27T09:46:12.385 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:11 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:11.928+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:12.386 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:46:11 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3[79199]: 2024-05-27T09:46:11.911+0000 7fd3d9c00700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.526878+0000 front 2024-05-27T09:45:44.526925+0000 (oldest deadline 2024-05-27T09:46:08.626248+0000) 2024-05-27T09:46:12.885 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:12 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:12.589+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:43.831515+0000 front 2024-05-27T09:45:43.831710+0000 (oldest deadline 2024-05-27T09:46:06.731790+0000) 2024-05-27T09:46:13.385 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:12 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:12.974+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:13.386 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:46:12 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3[79199]: 2024-05-27T09:46:12.950+0000 7fd3d9c00700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.526878+0000 front 2024-05-27T09:45:44.526925+0000 (oldest deadline 2024-05-27T09:46:08.626248+0000) 2024-05-27T09:46:13.886 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:13 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:13.578+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:43.831515+0000 front 2024-05-27T09:45:43.831710+0000 (oldest deadline 2024-05-27T09:46:06.731790+0000) 2024-05-27T09:46:14.323 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:13 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:13.954+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:14.635 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:46:14 smithi138 podman[106693]: 2024-05-27 09:46:14.323555444 +0000 UTC m=+5.682867935 container died 5e8d3e398c210896afc240ac7734da3ee6cdd4a86aaea5301790d35ac3fc5f5b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-27T09:46:14.636 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:14 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:14.539+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:43.831515+0000 front 2024-05-27T09:45:43.831710+0000 (oldest deadline 2024-05-27T09:46:06.731790+0000) 2024-05-27T09:46:14.971 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:46:14 smithi138 podman[106693]: 2024-05-27 09:46:14.887886795 +0000 UTC m=+6.247199357 container cleanup 5e8d3e398c210896afc240ac7734da3ee6cdd4a86aaea5301790d35ac3fc5f5b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_BRANCH=HEAD) 2024-05-27T09:46:14.971 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:46:14 smithi138 bash[106693]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3 2024-05-27T09:46:15.385 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:14 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:14.970+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:15.386 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:46:15 smithi138 podman[106713]: 2024-05-27 09:46:15.181320916 +0000 UTC m=+0.850589277 container remove 5e8d3e398c210896afc240ac7734da3ee6cdd4a86aaea5301790d35ac3fc5f5b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-27T09:46:15.885 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:15 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:15.533+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:43.831515+0000 front 2024-05-27T09:45:43.831710+0000 (oldest deadline 2024-05-27T09:46:06.731790+0000) 2024-05-27T09:46:16.320 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:15 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:15.981+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:16.321 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:46:16 smithi138 podman[106801]: 2024-05-27 09:46:15.922860546 +0000 UTC m=+0.033128164 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:46:16.321 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:46:16 smithi138 podman[106801]: 2024-05-27 09:46:16.023769681 +0000 UTC m=+0.134037317 container create 2f1e729742611746849015ee017aff096ed96f6c7df1abe37b0866889e51cf9b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-27T09:46:16.321 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:46:16 smithi138 podman[106801]: 2024-05-27 09:46:16.319385166 +0000 UTC m=+0.429652810 container init 2f1e729742611746849015ee017aff096ed96f6c7df1abe37b0866889e51cf9b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T09:46:16.635 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:46:16 smithi138 podman[106801]: 2024-05-27 09:46:16.332846479 +0000 UTC m=+0.443114163 container start 2f1e729742611746849015ee017aff096ed96f6c7df1abe37b0866889e51cf9b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738) 2024-05-27T09:46:16.636 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:46:16 smithi138 podman[106801]: 2024-05-27 09:46:16.42032576 +0000 UTC m=+0.530593467 container attach 2f1e729742611746849015ee017aff096ed96f6c7df1abe37b0866889e51cf9b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-27T09:46:16.636 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:16 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:16.565+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:43.831515+0000 front 2024-05-27T09:45:43.831710+0000 (oldest deadline 2024-05-27T09:46:06.731790+0000) 2024-05-27T09:46:16.636 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:16 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:16.565+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:54.333967+0000 front 2024-05-27T09:45:54.334100+0000 (oldest deadline 2024-05-27T09:46:16.033623+0000) 2024-05-27T09:46:17.130 INFO:journalctl@ceph.osd.3.smithi138.stdout:May 27 09:46:16 smithi138 podman[106801]: 2024-05-27 09:46:16.785712033 +0000 UTC m=+0.895979650 container died 2f1e729742611746849015ee017aff096ed96f6c7df1abe37b0866889e51cf9b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T09:46:17.131 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:17 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:17.018+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:17.131 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:17 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:17.018+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:53.674669+0000 front 2024-05-27T09:45:53.674517+0000 (oldest deadline 2024-05-27T09:46:16.574400+0000) 2024-05-27T09:46:17.652 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.3.service' 2024-05-27T09:46:17.746 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T09:46:17.746 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-27T09:46:17.746 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-27T09:46:17.747 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.4 2024-05-27T09:46:17.856 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:17 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:17.590+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:43.831515+0000 front 2024-05-27T09:45:43.831710+0000 (oldest deadline 2024-05-27T09:46:06.731790+0000) 2024-05-27T09:46:17.856 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:17 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:17.590+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:54.333967+0000 front 2024-05-27T09:45:54.334100+0000 (oldest deadline 2024-05-27T09:46:16.033623+0000) 2024-05-27T09:46:18.135 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:17 smithi138 systemd[1]: Stopping Ceph osd.4 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:46:18.136 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:18 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:18.055+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:18.136 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:18 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:18.055+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:53.674669+0000 front 2024-05-27T09:45:53.674517+0000 (oldest deadline 2024-05-27T09:46:16.574400+0000) 2024-05-27T09:46:18.597 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:18 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:18.343+0000 7f4838800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T09:46:18.597 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:18 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:18.343+0000 7f4838800700 -1 osd.4 85 *** Got signal Terminated *** 2024-05-27T09:46:18.597 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:18 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:18.343+0000 7f4838800700 -1 osd.4 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T09:46:18.885 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:18 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:18.595+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:43.831515+0000 front 2024-05-27T09:45:43.831710+0000 (oldest deadline 2024-05-27T09:46:06.731790+0000) 2024-05-27T09:46:18.885 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:18 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:18.595+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:54.333967+0000 front 2024-05-27T09:45:54.334100+0000 (oldest deadline 2024-05-27T09:46:16.033623+0000) 2024-05-27T09:46:19.385 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:19 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:19.052+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:19.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:19 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:19.052+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:53.674669+0000 front 2024-05-27T09:45:53.674517+0000 (oldest deadline 2024-05-27T09:46:16.574400+0000) 2024-05-27T09:46:19.886 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:19 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:19.579+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:43.831515+0000 front 2024-05-27T09:45:43.831710+0000 (oldest deadline 2024-05-27T09:46:06.731790+0000) 2024-05-27T09:46:19.886 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:19 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:19.579+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:54.333967+0000 front 2024-05-27T09:45:54.334100+0000 (oldest deadline 2024-05-27T09:46:16.033623+0000) 2024-05-27T09:46:20.385 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:20 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:20.032+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:20.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:20 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:20.032+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:53.674669+0000 front 2024-05-27T09:45:53.674517+0000 (oldest deadline 2024-05-27T09:46:16.574400+0000) 2024-05-27T09:46:20.885 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:20 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:20.597+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:43.831515+0000 front 2024-05-27T09:45:43.831710+0000 (oldest deadline 2024-05-27T09:46:06.731790+0000) 2024-05-27T09:46:20.885 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:20 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:20.597+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:54.333967+0000 front 2024-05-27T09:45:54.334100+0000 (oldest deadline 2024-05-27T09:46:16.033623+0000) 2024-05-27T09:46:21.385 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:21 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:21.077+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:21.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:21 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:21.077+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:53.674669+0000 front 2024-05-27T09:45:53.674517+0000 (oldest deadline 2024-05-27T09:46:16.574400+0000) 2024-05-27T09:46:21.885 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:21 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:21.578+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:43.831515+0000 front 2024-05-27T09:45:43.831710+0000 (oldest deadline 2024-05-27T09:46:06.731790+0000) 2024-05-27T09:46:21.886 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:21 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:21.578+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:54.333967+0000 front 2024-05-27T09:45:54.334100+0000 (oldest deadline 2024-05-27T09:46:16.033623+0000) 2024-05-27T09:46:22.385 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:22 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:22.085+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:22.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:22 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:22.085+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:53.674669+0000 front 2024-05-27T09:45:53.674517+0000 (oldest deadline 2024-05-27T09:46:16.574400+0000) 2024-05-27T09:46:22.885 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:22 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:22.617+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:43.831515+0000 front 2024-05-27T09:45:43.831710+0000 (oldest deadline 2024-05-27T09:46:06.731790+0000) 2024-05-27T09:46:22.886 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:22 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4[84489]: 2024-05-27T09:46:22.617+0000 7f4834000700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:54.333967+0000 front 2024-05-27T09:45:54.334100+0000 (oldest deadline 2024-05-27T09:46:16.033623+0000) 2024-05-27T09:46:23.385 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:23 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:23.132+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:23.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:23 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:23.132+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:53.674669+0000 front 2024-05-27T09:45:53.674517+0000 (oldest deadline 2024-05-27T09:46:16.574400+0000) 2024-05-27T09:46:24.059 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:23 smithi138 podman[106917]: 2024-05-27 09:46:23.753858235 +0000 UTC m=+5.547981122 container died 250e443364de2a3a7d490de113c3f9787dc35f40621cab2bea8c6c554fdf174b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0) 2024-05-27T09:46:24.312 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:24 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:24.099+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:24.312 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:24 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:24.099+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:53.674669+0000 front 2024-05-27T09:45:53.674517+0000 (oldest deadline 2024-05-27T09:46:16.574400+0000) 2024-05-27T09:46:24.614 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:24 smithi138 podman[106917]: 2024-05-27 09:46:24.311958666 +0000 UTC m=+6.106081593 container cleanup 250e443364de2a3a7d490de113c3f9787dc35f40621cab2bea8c6c554fdf174b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-27T09:46:24.614 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:24 smithi138 bash[106917]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4 2024-05-27T09:46:24.885 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:24 smithi138 podman[106937]: 2024-05-27 09:46:24.613772694 +0000 UTC m=+0.841294135 container remove 250e443364de2a3a7d490de113c3f9787dc35f40621cab2bea8c6c554fdf174b (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T09:46:25.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:25 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:25.065+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:25.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:25 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:25.065+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:53.674669+0000 front 2024-05-27T09:45:53.674517+0000 (oldest deadline 2024-05-27T09:46:16.574400+0000) 2024-05-27T09:46:25.744 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:25 smithi138 podman[107024]: 2024-05-27 09:46:25.3150226 +0000 UTC m=+0.035180803 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:46:25.744 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:25 smithi138 podman[107024]: 2024-05-27 09:46:25.431343297 +0000 UTC m=+0.151501538 container create bdca98adcb0f67f038000e4f34c69df755e40878a2bd81c55bfa2bcefa3893e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-27T09:46:25.745 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:25 smithi138 podman[107024]: 2024-05-27 09:46:25.743804511 +0000 UTC m=+0.463962769 container init bdca98adcb0f67f038000e4f34c69df755e40878a2bd81c55bfa2bcefa3893e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T09:46:26.048 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:25 smithi138 podman[107024]: 2024-05-27 09:46:25.755092619 +0000 UTC m=+0.475250831 container start bdca98adcb0f67f038000e4f34c69df755e40878a2bd81c55bfa2bcefa3893e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-05-27T09:46:26.048 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:25 smithi138 podman[107024]: 2024-05-27 09:46:25.870541366 +0000 UTC m=+0.590699654 container attach bdca98adcb0f67f038000e4f34c69df755e40878a2bd81c55bfa2bcefa3893e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521) 2024-05-27T09:46:26.385 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:26 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:26.045+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:26.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:26 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:26.045+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:53.674669+0000 front 2024-05-27T09:45:53.674517+0000 (oldest deadline 2024-05-27T09:46:16.574400+0000) 2024-05-27T09:46:26.386 INFO:journalctl@ceph.osd.4.smithi138.stdout:May 27 09:46:26 smithi138 podman[107024]: 2024-05-27 09:46:26.207194343 +0000 UTC m=+0.927352588 container died bdca98adcb0f67f038000e4f34c69df755e40878a2bd81c55bfa2bcefa3893e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T09:46:27.144 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.4.service' 2024-05-27T09:46:27.245 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T09:46:27.245 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-27T09:46:27.245 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-27T09:46:27.245 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.5 2024-05-27T09:46:27.355 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:27 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:27.073+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:27.355 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:27 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:27.073+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:53.674669+0000 front 2024-05-27T09:45:53.674517+0000 (oldest deadline 2024-05-27T09:46:16.574400+0000) 2024-05-27T09:46:27.635 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:27 smithi138 systemd[1]: Stopping Ceph osd.5 for b966b06e-1c09-11ef-bc9b-c7b262605968... 2024-05-27T09:46:28.136 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:27 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:27.863+0000 7f763e200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T09:46:28.136 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:27 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:27.863+0000 7f763e200700 -1 osd.5 85 *** Got signal Terminated *** 2024-05-27T09:46:28.136 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:27 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:27.863+0000 7f763e200700 -1 osd.5 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T09:46:28.136 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:28 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:28.075+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:28.136 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:28 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:28.075+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:53.674669+0000 front 2024-05-27T09:45:53.674517+0000 (oldest deadline 2024-05-27T09:46:16.574400+0000) 2024-05-27T09:46:28.136 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:28 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:28.075+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6822 osd.2 since back 2024-05-27T09:46:03.576874+0000 front 2024-05-27T09:46:03.576916+0000 (oldest deadline 2024-05-27T09:46:27.676435+0000) 2024-05-27T09:46:29.385 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:29 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:29.083+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:29.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:29 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:29.083+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:53.674669+0000 front 2024-05-27T09:45:53.674517+0000 (oldest deadline 2024-05-27T09:46:16.574400+0000) 2024-05-27T09:46:29.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:29 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:29.083+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6822 osd.2 since back 2024-05-27T09:46:03.576874+0000 front 2024-05-27T09:46:03.576916+0000 (oldest deadline 2024-05-27T09:46:27.676435+0000) 2024-05-27T09:46:30.326 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:30 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:30.061+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:30.327 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:30 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:30.061+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:53.674669+0000 front 2024-05-27T09:45:53.674517+0000 (oldest deadline 2024-05-27T09:46:16.574400+0000) 2024-05-27T09:46:30.327 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:30 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:30.061+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6822 osd.2 since back 2024-05-27T09:46:03.576874+0000 front 2024-05-27T09:46:03.576916+0000 (oldest deadline 2024-05-27T09:46:27.676435+0000) 2024-05-27T09:46:31.385 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:31 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:31.075+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:31.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:31 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:31.075+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:53.674669+0000 front 2024-05-27T09:45:53.674517+0000 (oldest deadline 2024-05-27T09:46:16.574400+0000) 2024-05-27T09:46:31.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:31 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:31.075+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6822 osd.2 since back 2024-05-27T09:46:03.576874+0000 front 2024-05-27T09:46:03.576916+0000 (oldest deadline 2024-05-27T09:46:27.676435+0000) 2024-05-27T09:46:32.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:32 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:32.055+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6806 osd.0 since back 2024-05-27T09:45:44.873024+0000 front 2024-05-27T09:45:44.872421+0000 (oldest deadline 2024-05-27T09:46:08.972700+0000) 2024-05-27T09:46:32.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:32 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:32.055+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6814 osd.1 since back 2024-05-27T09:45:53.674669+0000 front 2024-05-27T09:45:53.674517+0000 (oldest deadline 2024-05-27T09:46:16.574400+0000) 2024-05-27T09:46:32.386 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:32 smithi138 ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5[89641]: 2024-05-27T09:46:32.055+0000 7f7639a00700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.119:6822 osd.2 since back 2024-05-27T09:46:03.576874+0000 front 2024-05-27T09:46:03.576916+0000 (oldest deadline 2024-05-27T09:46:27.676435+0000) 2024-05-27T09:46:33.635 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:33 smithi138 podman[107127]: 2024-05-27 09:46:33.37390707 +0000 UTC m=+5.663767074 container died 90d98c5b2f163bd1ed8e56f7fadf9fa35495395416645ebb1516c3222cc2cfd7 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-27T09:46:33.917 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:33 smithi138 podman[107127]: 2024-05-27 09:46:33.883836622 +0000 UTC m=+6.173696665 container cleanup 90d98c5b2f163bd1ed8e56f7fadf9fa35495395416645ebb1516c3222cc2cfd7 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T09:46:33.917 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:33 smithi138 bash[107127]: ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5 2024-05-27T09:46:34.635 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:34 smithi138 podman[107156]: 2024-05-27 09:46:34.194049284 +0000 UTC m=+0.810506951 container remove 90d98c5b2f163bd1ed8e56f7fadf9fa35495395416645ebb1516c3222cc2cfd7 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T09:46:35.299 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:35 smithi138 podman[107242]: 2024-05-27 09:46:34.921813649 +0000 UTC m=+0.036893683 image pull 850a06a84d28fb71dec1431a907aeab5c3ccf2f0fcd791cd2cbd1ea45413ef76 quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6 2024-05-27T09:46:35.299 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:35 smithi138 podman[107242]: 2024-05-27 09:46:35.028078334 +0000 UTC m=+0.143158399 container create 2b98e6cac126138c776933e3c33dfc335c99d56a9e50db5009feb9e3a864b328 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-27T09:46:35.300 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:35 smithi138 podman[107242]: 2024-05-27 09:46:35.298830046 +0000 UTC m=+0.413910151 container init 2b98e6cac126138c776933e3c33dfc335c99d56a9e50db5009feb9e3a864b328 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-deactivate, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521) 2024-05-27T09:46:35.635 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:35 smithi138 podman[107242]: 2024-05-27 09:46:35.311576397 +0000 UTC m=+0.426656432 container start 2b98e6cac126138c776933e3c33dfc335c99d56a9e50db5009feb9e3a864b328 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-27T09:46:35.636 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:35 smithi138 podman[107242]: 2024-05-27 09:46:35.389590832 +0000 UTC m=+0.504670947 container attach 2b98e6cac126138c776933e3c33dfc335c99d56a9e50db5009feb9e3a864b328 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-27T09:46:36.066 INFO:journalctl@ceph.osd.5.smithi138.stdout:May 27 09:46:35 smithi138 podman[107242]: 2024-05-27 09:46:35.759838109 +0000 UTC m=+0.874918149 container died 2b98e6cac126138c776933e3c33dfc335c99d56a9e50db5009feb9e3a864b328 (image=quay.ceph.io/ceph-ci/ceph@sha256:40667fa40b383802316dd30b2fb5cc99041435f0e277cde147e3c3dc5c88e9a6, name=ceph-b966b06e-1c09-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.105740-debug-a650738, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-27T09:46:36.548 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b966b06e-1c09-11ef-bc9b-c7b262605968@osd.5.service' 2024-05-27T09:46:36.659 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T09:46:36.659 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-27T09:46:36.659 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 --force --keep-logs 2024-05-27T09:46:36.923 INFO:teuthology.orchestra.run.smithi119.stdout:Deleting cluster with fsid: b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:47:08.501 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 --force --keep-logs 2024-05-27T09:47:08.782 INFO:teuthology.orchestra.run.smithi138.stdout:Deleting cluster with fsid: b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:47:30.050 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T09:47:30.146 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T09:47:30.238 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-27T09:47:30.240 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727678/remote/smithi119/crash 2024-05-27T09:47:30.241 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/crash -- . 2024-05-27T09:47:30.318 INFO:teuthology.orchestra.run.smithi119.stderr:tar: /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-05-27T09:47:30.318 INFO:teuthology.orchestra.run.smithi119.stderr:tar: Error is not recoverable: exiting now 2024-05-27T09:47:30.322 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727678/remote/smithi138/crash 2024-05-27T09:47:30.323 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/crash -- . 2024-05-27T09:47:30.408 INFO:teuthology.orchestra.run.smithi138.stderr:tar: /var/lib/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-05-27T09:47:30.408 INFO:teuthology.orchestra.run.smithi138.stderr:tar: Error is not recoverable: exiting now 2024-05-27T09:47:30.412 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-27T09:47:30.412 DEBUG:teuthology.orchestra.run.smithi119:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-05-27T09:47:30.514 INFO:teuthology.orchestra.run.smithi119.stdout:2024-05-27T09:40:00.000322+0000 mon.smithi119 (mon.0) 563 : cluster [ERR] fs cephfs is degraded 2024-05-27T09:47:30.515 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-05-27T09:47:30.515 DEBUG:teuthology.orchestra.run.smithi119:> sudo egrep '\[SEC\]' /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-05-27T09:47:30.603 DEBUG:teuthology.orchestra.run.smithi119:> sudo egrep '\[ERR\]' /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-05-27T09:47:30.686 INFO:teuthology.orchestra.run.smithi119.stdout:2024-05-27T09:40:00.000322+0000 mon.smithi119 (mon.0) 563 : cluster [ERR] fs cephfs is degraded 2024-05-27T09:47:30.687 INFO:tasks.cephadm:Compressing logs... 2024-05-27T09:47:30.687 DEBUG:teuthology.orchestra.run.smithi119:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-27T09:47:30.691 DEBUG:teuthology.orchestra.run.smithi138:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-27T09:47:30.746 INFO:teuthology.orchestra.run.smithi119.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-27T09:47:30.746 INFO:teuthology.orchestra.run.smithi119.stderr:: No such file or directory 2024-05-27T09:47:30.747 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph.log 2024-05-27T09:47:30.748 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-05-27T09:47:30.749 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph.audit.log 2024-05-27T09:47:30.750 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-05-27T09:47:30.751 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-05-27T09:47:30.752 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-osd.1.log: 88.5% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph.log.gz 2024-05-27T09:47:30.752 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mgr.smithi119.ncwmog.log 2024-05-27T09:47:30.753 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-client.ceph-exporter.smithi119.log 2024-05-27T09:47:30.755 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mgr.smithi119.ncwmog.log: gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-05-27T09:47:30.756 INFO:teuthology.orchestra.run.smithi119.stderr: 91.4% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-05-27T09:47:30.756 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-client.ceph-exporter.smithi119.log: gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mon.smithi119.log 2024-05-27T09:47:30.759 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph.cephadm.log: /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mon.smithi119.log: 85.8% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-05-27T09:47:30.759 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi119.ppmzzk.log 2024-05-27T09:47:30.759 INFO:teuthology.orchestra.run.smithi119.stderr: 94.1% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-client.ceph-exporter.smithi119.log.gz 2024-05-27T09:47:30.763 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi119.ppmzzk.log: gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-volume.log 2024-05-27T09:47:30.764 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi119.nghcey.log 2024-05-27T09:47:30.775 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-volume.log: /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi119.nghcey.log: 91.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-27T09:47:30.776 INFO:teuthology.orchestra.run.smithi138.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-27T09:47:30.776 INFO:teuthology.orchestra.run.smithi138.stderr:: No such file or directory 2024-05-27T09:47:30.777 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mgr.smithi138.dxsuqf.log 2024-05-27T09:47:30.778 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph.log 2024-05-27T09:47:30.779 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph.audit.log 2024-05-27T09:47:30.780 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mgr.smithi138.dxsuqf.log: /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-05-27T09:47:30.780 INFO:teuthology.orchestra.run.smithi138.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-27T09:47:30.782 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-05-27T09:47:30.782 INFO:teuthology.orchestra.run.smithi119.stderr: 91.4% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-05-27T09:47:30.783 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-05-27T09:47:30.784 INFO:teuthology.orchestra.run.smithi138.stderr: 88.5% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph.log.gz 2024-05-27T09:47:30.785 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi138.ndfkhd.log 2024-05-27T09:47:30.785 INFO:teuthology.orchestra.run.smithi138.stderr: 91.5% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-05-27T09:47:30.786 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-client.ceph-exporter.smithi138.log 2024-05-27T09:47:30.787 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-osd.4.log: /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi138.ndfkhd.log: gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-05-27T09:47:30.789 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-client.ceph-exporter.smithi138.log: gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi138.alszpk.log 2024-05-27T09:47:30.791 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph.cephadm.log: 85.7% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-05-27T09:47:30.791 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi138.alszpk.log: 94.1% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-client.ceph-exporter.smithi138.log.gz 2024-05-27T09:47:30.793 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-volume.log 2024-05-27T09:47:30.794 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mon.smithi138.log 2024-05-27T09:47:30.800 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-volume.log: /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mon.smithi138.log: 94.0% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi138.alszpk.log.gz 2024-05-27T09:47:30.813 INFO:teuthology.orchestra.run.smithi138.stderr: 91.2% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-05-27T09:47:30.841 INFO:teuthology.orchestra.run.smithi138.stderr: 89.1% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mgr.smithi138.dxsuqf.log.gz 2024-05-27T09:47:31.153 INFO:teuthology.orchestra.run.smithi138.stderr: 92.4% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mon.smithi138.log.gz 2024-05-27T09:47:31.224 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mgr.smithi119.ncwmog.log.gz 2024-05-27T09:47:32.153 INFO:teuthology.orchestra.run.smithi119.stderr: 90.4% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mon.smithi119.log.gz 2024-05-27T09:47:34.878 INFO:teuthology.orchestra.run.smithi119.stderr: 93.5% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-05-27T09:47:35.174 INFO:teuthology.orchestra.run.smithi138.stderr: 93.5% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-05-27T09:47:35.675 INFO:teuthology.orchestra.run.smithi119.stderr: 93.5% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-05-27T09:47:36.434 INFO:teuthology.orchestra.run.smithi138.stderr: 93.5% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-05-27T09:47:36.486 INFO:teuthology.orchestra.run.smithi119.stderr: 95.3% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi119.nghcey.log.gz 2024-05-27T09:47:37.226 INFO:teuthology.orchestra.run.smithi138.stderr: 93.5% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-05-27T09:47:37.526 INFO:teuthology.orchestra.run.smithi119.stderr: 93.5% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-05-27T09:48:16.806 INFO:teuthology.orchestra.run.smithi138.stderr: 93.0% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi138.ndfkhd.log.gz 2024-05-27T09:48:16.809 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-05-27T09:48:16.809 INFO:teuthology.orchestra.run.smithi138.stderr:real 0m46.064s 2024-05-27T09:48:16.810 INFO:teuthology.orchestra.run.smithi138.stderr:user 0m58.252s 2024-05-27T09:48:16.810 INFO:teuthology.orchestra.run.smithi138.stderr:sys 0m4.639s 2024-05-27T09:48:18.940 INFO:teuthology.orchestra.run.smithi119.stderr: 93.3% -- replaced with /var/log/ceph/b966b06e-1c09-11ef-bc9b-c7b262605968/ceph-mds.cephfs.smithi119.ppmzzk.log.gz 2024-05-27T09:48:18.943 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-05-27T09:48:18.943 INFO:teuthology.orchestra.run.smithi119.stderr:real 0m48.221s 2024-05-27T09:48:18.943 INFO:teuthology.orchestra.run.smithi119.stderr:user 1m6.476s 2024-05-27T09:48:18.944 INFO:teuthology.orchestra.run.smithi119.stderr:sys 0m5.185s 2024-05-27T09:48:18.945 INFO:tasks.cephadm:Archiving logs... 2024-05-27T09:48:18.945 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727678/remote/smithi119/log 2024-05-27T09:48:18.946 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-27T09:48:28.403 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727678/remote/smithi138/log 2024-05-27T09:48:28.404 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-27T09:48:37.220 INFO:tasks.cephadm:Removing cluster... 2024-05-27T09:48:37.221 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 --force 2024-05-27T09:48:37.489 INFO:teuthology.orchestra.run.smithi119.stdout:Deleting cluster with fsid: b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:48:38.396 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b966b06e-1c09-11ef-bc9b-c7b262605968 --force 2024-05-27T09:48:38.673 INFO:teuthology.orchestra.run.smithi138.stdout:Deleting cluster with fsid: b966b06e-1c09-11ef-bc9b-c7b262605968 2024-05-27T09:48:39.560 INFO:tasks.cephadm:Removing cephadm ... 2024-05-27T09:48:39.560 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-27T09:48:39.618 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-27T09:48:39.677 INFO:tasks.cephadm:Teardown complete 2024-05-27T09:48:39.677 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-27T09:48:39.689 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-27T09:48:39.689 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-27T09:48:39.696 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-27T09:48:39.827 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-27T09:48:39.838 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-27T09:48:39.838 DEBUG:teuthology.orchestra.run.smithi119:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T09:48:39.846 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T09:48:39.893 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-05-27T09:48:39.901 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-05-27T09:48:39.903 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T09:48:39.903 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-05-27T09:48:39.904 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 42 +1117us[+1121us] +/- 73ms 2024-05-27T09:48:39.904 INFO:teuthology.orchestra.run.smithi119.stdout:^* hv02.front.sepia.ceph.com 3 6 377 40 -1418us[-1414us] +/- 62ms 2024-05-27T09:48:39.904 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +667us[ +671us] +/- 70ms 2024-05-27T09:48:39.904 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:48:39.911 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T09:48:39.911 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-05-27T09:48:39.911 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 42 +1092us[+1103us] +/- 73ms 2024-05-27T09:48:39.912 INFO:teuthology.orchestra.run.smithi138.stdout:^* hv02.front.sepia.ceph.com 3 6 377 41 -1455us[-1445us] +/- 62ms 2024-05-27T09:48:39.912 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 +652us[ +662us] +/- 70ms 2024-05-27T09:48:39.912 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T09:48:39.913 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-27T09:48:39.923 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-27T09:48:39.924 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-27T09:48:39.933 DEBUG:teuthology.orchestra.run.smithi119:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-27T09:48:40.146 DEBUG:teuthology.orchestra.run.smithi138:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-27T09:48:40.345 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T09:48:40.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:48:40.440 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T09:48:40.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:48:40.535 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-27T09:48:40.546 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-27T09:48:40.556 INFO:teuthology.task.internal:Duration was 2506.470815 seconds 2024-05-27T09:48:40.556 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-27T09:48:40.566 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-27T09:48:40.566 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-27T09:48:40.574 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-27T09:48:40.695 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T09:48:40.707 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T09:48:41.088 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-27T09:48:41.088 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi119.front.sepia.ceph.com 2024-05-27T09:48:41.088 DEBUG:teuthology.orchestra.run.smithi119:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-27T09:48:41.150 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-05-27T09:48:41.150 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-27T09:48:41.226 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-27T09:48:41.227 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-27T09:48:41.234 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-27T09:48:41.319 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-27T09:48:41.319 DEBUG:teuthology.orchestra.run.smithi119:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-27T09:48:41.324 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-27T09:48:42.147 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-27T09:48:42.158 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-27T09:48:42.158 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-27T09:48:42.237 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-27T09:48:42.328 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-27T09:48:42.338 DEBUG:teuthology.orchestra.run.smithi119:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-27T09:48:42.346 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-27T09:48:42.435 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = core 2024-05-27T09:48:42.440 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-05-27T09:48:42.480 DEBUG:teuthology.orchestra.run.smithi119:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-27T09:48:42.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:48:42.527 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-27T09:48:42.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T09:48:42.584 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-27T09:48:42.595 INFO:teuthology.task.internal:Transferring archived files... 2024-05-27T09:48:42.596 DEBUG:teuthology.misc:Transferring archived files from smithi119:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727678/remote/smithi119 2024-05-27T09:48:42.596 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-27T09:48:42.706 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-27_06:46:53-fs-wip-vshankar-testing-20240521.105740-debug-testing-default-smithi/7727678/remote/smithi138 2024-05-27T09:48:42.706 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-27T09:48:42.801 INFO:teuthology.task.internal:Removing archive directory... 2024-05-27T09:48:42.801 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-27T09:48:42.809 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-27T09:48:42.897 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-27T09:48:42.910 INFO:teuthology.task.internal:Not uploading archives. 2024-05-27T09:48:42.910 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-27T09:48:42.921 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-27T09:48:42.921 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-27T09:48:42.928 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-27T09:48:42.978 INFO:teuthology.orchestra.run.smithi119.stdout: 393949 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 27 09:48 /home/ubuntu/cephtest 2024-05-27T09:48:42.985 INFO:teuthology.orchestra.run.smithi138.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 27 09:48 /home/ubuntu/cephtest 2024-05-27T09:48:42.988 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-27T09:48:42.999 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-27T09:48:43.023 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2506.470814704895 failure_reason: '"2024-05-27T09:40:00.000322+0000 mon.smithi119 (mon.0) 563 : cluster [ERR] fs cephfs is degraded" in cluster log' flavor: default host.a-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 host.b-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 owner: scheduled_vshankar@teuthology success: false 2024-05-27T09:48:43.023 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-27T09:48:43.109 INFO:teuthology.run:FAIL